2023-11-08T08:02:03.181 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-08T08:02:03.182 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} --archive /home/teuthworker/archive/neesingh-2023-11-08_07:51:45-fs-wip-neesingh-testing-54557-distro-default-smithi/7451825 --owner scheduled_neesingh@teuthology --name neesingh-2023-11-08_07:51:45-fs-wip-neesingh-testing-54557-distro-default-smithi --verbose -- /home/teuthworker/archive/neesingh-2023-11-08_07:51:45-fs-wip-neesingh-testing-54557-distro-default-smithi/7451825/orig.config.yaml 2023-11-08T08:02:03.229 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-08T08:02:03.316 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2023-11-08_07:51:45-fs-wip-neesingh-testing-54557-distro-default-smithi/7451825 branch: wip-neesingh-testing-54557 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7451825' kernel: client: branch: testing kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2023-11-08_07:51:45-fs-wip-neesingh-testing-54557-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-neesingh-testing-54557 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: b820560f30d310ab2570761b9a294523225d6998 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: b820560f30d310ab2570761b9a294523225d6998 kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-neesingh-testing-54557 sha1: b820560f30d310ab2570761b9a294523225d6998 owner: scheduled_neesingh@teuthology priority: 10 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5135 sha1: b820560f30d310ab2570761b9a294523225d6998 sleep_before_teardown: 0 suite: fs suite_branch: wip-neesingh-testing-54557 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_b820560f30d310ab2570761b9a294523225d6998/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: b820560f30d310ab2570761b9a294523225d6998 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi169.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 6899cd26fceddb2fec83dc1a1349394b28c8998e timestamp: 2023-11-08_07:51:45 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3590714 2023-11-08T08:02:03.317 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_b820560f30d310ab2570761b9a294523225d6998/qa; will attempt to use it 2023-11-08T08:02:03.318 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_b820560f30d310ab2570761b9a294523225d6998/qa/tasks 2023-11-08T08:02:03.318 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-08T08:02:03.320 INFO:teuthology.task.internal:Checking packages... 2023-11-08T08:02:03.346 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'b820560f30d310ab2570761b9a294523225d6998' 2023-11-08T08:02:03.346 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-08T08:02:03.347 INFO:teuthology.packaging:ref: None 2023-11-08T08:02:03.347 INFO:teuthology.packaging:tag: None 2023-11-08T08:02:03.347 INFO:teuthology.packaging:branch: wip-neesingh-testing-54557 2023-11-08T08:02:03.347 INFO:teuthology.packaging:sha1: b820560f30d310ab2570761b9a294523225d6998 2023-11-08T08:02:03.348 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-neesingh-testing-54557 2023-11-08T08:02:03.559 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-2698-gb820560f-1focal 2023-11-08T08:02:03.560 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-08T08:02:03.571 INFO:teuthology.task.internal:no buildpackages task found 2023-11-08T08:02:03.571 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-08T08:02:03.590 INFO:teuthology.task.internal:Saving configuration 2023-11-08T08:02:03.608 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-08T08:02:03.619 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-08T08:02:03.642 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-08_07:51:45-fs-wip-neesingh-testing-54557-distro-default-smithi/7451825', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-08 07:55:42.226709', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-11-08T08:02:03.663 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-08_07:51:45-fs-wip-neesingh-testing-54557-distro-default-smithi/7451825', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-08 07:55:42.228277', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-11-08T08:02:03.664 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-08T08:02:03.674 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-08T08:02:03.675 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-08T08:02:03.675 INFO:teuthology.run_tasks:Running task console_log... 2023-11-08T08:02:03.755 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd210afbc10>, signals=[15]) 2023-11-08T08:02:03.756 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-08T08:02:03.767 INFO:teuthology.task.internal:Opening connections... 2023-11-08T08:02:03.767 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi040.front.sepia.ceph.com 2023-11-08T08:02:03.769 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T08:02:03.819 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2023-11-08T08:02:03.820 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T08:02:03.867 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-08T08:02:03.876 DEBUG:teuthology.orchestra.run.smithi040:> uname -m 2023-11-08T08:02:03.900 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2023-11-08T08:02:03.901 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2023-11-08T08:02:03.955 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="Ubuntu" 2023-11-08T08:02:03.956 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-11-08T08:02:03.956 INFO:teuthology.orchestra.run.smithi040.stdout:ID=ubuntu 2023-11-08T08:02:03.956 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE=debian 2023-11-08T08:02:03.956 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-11-08T08:02:03.957 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="20.04" 2023-11-08T08:02:03.957 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-08T08:02:03.957 INFO:teuthology.orchestra.run.smithi040.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-08T08:02:03.957 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-08T08:02:03.957 INFO:teuthology.orchestra.run.smithi040.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-08T08:02:03.958 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_CODENAME=focal 2023-11-08T08:02:03.958 INFO:teuthology.orchestra.run.smithi040.stdout:UBUNTU_CODENAME=focal 2023-11-08T08:02:03.959 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2023-11-08T08:02:03.983 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2023-11-08T08:02:03.990 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2023-11-08T08:02:03.991 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2023-11-08T08:02:04.038 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="Ubuntu" 2023-11-08T08:02:04.039 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-11-08T08:02:04.039 INFO:teuthology.orchestra.run.smithi169.stdout:ID=ubuntu 2023-11-08T08:02:04.039 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE=debian 2023-11-08T08:02:04.039 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-11-08T08:02:04.040 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="20.04" 2023-11-08T08:02:04.040 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-08T08:02:04.040 INFO:teuthology.orchestra.run.smithi169.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-08T08:02:04.040 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-08T08:02:04.041 INFO:teuthology.orchestra.run.smithi169.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-08T08:02:04.041 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_CODENAME=focal 2023-11-08T08:02:04.041 INFO:teuthology.orchestra.run.smithi169.stdout:UBUNTU_CODENAME=focal 2023-11-08T08:02:04.041 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2023-11-08T08:02:04.066 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-08T08:02:04.079 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-08T08:02:04.087 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-08T08:02:04.087 DEBUG:teuthology.orchestra.run.smithi040:> test '!' -e /home/ubuntu/cephtest 2023-11-08T08:02:04.090 DEBUG:teuthology.orchestra.run.smithi169:> test '!' -e /home/ubuntu/cephtest 2023-11-08T08:02:04.096 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-08T08:02:04.103 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-08T08:02:04.104 DEBUG:teuthology.orchestra.run.smithi040:> test -z $(ls -A /var/lib/ceph) 2023-11-08T08:02:04.106 DEBUG:teuthology.orchestra.run.smithi169:> test -z $(ls -A /var/lib/ceph) 2023-11-08T08:02:04.148 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-08T08:02:04.208 INFO:teuthology.run_tasks:Running task kernel... 2023-11-08T08:02:04.225 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-11-08T08:02:04.226 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-11-08T08:02:04.226 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-11-08T08:02:04.227 DEBUG:teuthology.orchestra.run.smithi040:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-08T08:02:04.227 DEBUG:teuthology.orchestra.run.smithi169:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-08T08:02:04.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T08:02:04.234 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=testing 2023-11-08T08:02:04.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T08:02:04.238 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=testing 2023-11-08T08:02:04.448 DEBUG:teuthology.task.kernel:sha1 for client.1 is 3b796cfe2969fb2f2d017e8146dccc1ef582e837 2023-11-08T08:02:04.448 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "3b796cfe2969fb2f2d017e8146dccc1ef582e837"... 2023-11-08T08:02:04.449 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2023-11-08T08:02:04.456 INFO:teuthology.orchestra.run.smithi169.stdout:5.4.0-124-generic 2023-11-08T08:02:04.456 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-124-generic vs 3b796cfe2969fb2f2d017e8146dccc1ef582e837 2023-11-08T08:02:04.457 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-08T08:02:04.457 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2023-11-08T08:02:04.458 DEBUG:teuthology.orchestra.run.smithi169:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-11-08T08:02:04.526 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 0: cd: /lib/firmware/updates: No such file or directory 2023-11-08T08:02:04.541 DEBUG:teuthology.task.kernel:sha1 for client.0 is 3b796cfe2969fb2f2d017e8146dccc1ef582e837 2023-11-08T08:02:04.541 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "3b796cfe2969fb2f2d017e8146dccc1ef582e837"... 2023-11-08T08:02:04.541 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2023-11-08T08:02:04.548 INFO:teuthology.orchestra.run.smithi040.stdout:5.4.0-124-generic 2023-11-08T08:02:04.548 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-124-generic vs 3b796cfe2969fb2f2d017e8146dccc1ef582e837 2023-11-08T08:02:04.548 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-08T08:02:04.549 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2023-11-08T08:02:04.549 DEBUG:teuthology.orchestra.run.smithi040:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-11-08T08:02:04.606 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 0: cd: /lib/firmware/updates: No such file or directory 2023-11-08T08:02:04.626 INFO:teuthology.orchestra.run.smithi169.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-11-08T08:02:04.628 DEBUG:teuthology.orchestra.run.smithi169:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-11-08T08:02:04.666 DEBUG:teuthology.orchestra.run.smithi169:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-11-08T08:02:04.681 DEBUG:teuthology.orchestra.run.smithi169:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-11-08T08:02:04.721 INFO:teuthology.orchestra.run.smithi040.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-11-08T08:02:04.723 DEBUG:teuthology.orchestra.run.smithi040:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-11-08T08:02:04.756 DEBUG:teuthology.orchestra.run.smithi040:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-11-08T08:02:04.775 DEBUG:teuthology.orchestra.run.smithi040:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-11-08T08:02:50.507 INFO:teuthology.orchestra.run.smithi169.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-11-08T08:02:50.508 INFO:teuthology.orchestra.run.smithi169.stderr: * [new branch] main -> origin/main 2023-11-08T08:02:50.565 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20190312 -> 20190312 2023-11-08T08:02:50.565 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20190416 -> 20190416 2023-11-08T08:02:50.565 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20190514 -> 20190514 2023-11-08T08:02:50.565 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20190618 -> 20190618 2023-11-08T08:02:50.566 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20190717 -> 20190717 2023-11-08T08:02:50.566 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20190815 -> 20190815 2023-11-08T08:02:50.566 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20190923 -> 20190923 2023-11-08T08:02:50.566 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20191022 -> 20191022 2023-11-08T08:02:50.568 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20191215 -> 20191215 2023-11-08T08:02:50.568 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20200122 -> 20200122 2023-11-08T08:02:50.569 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20200316 -> 20200316 2023-11-08T08:02:50.569 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20200421 -> 20200421 2023-11-08T08:02:50.569 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20200519 -> 20200519 2023-11-08T08:02:50.569 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20200619 -> 20200619 2023-11-08T08:02:50.570 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20200721 -> 20200721 2023-11-08T08:02:50.570 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20200817 -> 20200817 2023-11-08T08:02:50.570 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20200918 -> 20200918 2023-11-08T08:02:50.570 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20201022 -> 20201022 2023-11-08T08:02:50.570 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20201118 -> 20201118 2023-11-08T08:02:50.571 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20201218 -> 20201218 2023-11-08T08:02:50.571 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20210208 -> 20210208 2023-11-08T08:02:50.571 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20210315 -> 20210315 2023-11-08T08:02:50.571 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20210511 -> 20210511 2023-11-08T08:02:50.571 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20210716 -> 20210716 2023-11-08T08:02:50.572 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20210818 -> 20210818 2023-11-08T08:02:50.574 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20210919 -> 20210919 2023-11-08T08:02:50.574 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20211027 -> 20211027 2023-11-08T08:02:50.574 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20211216 -> 20211216 2023-11-08T08:02:50.575 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20220209 -> 20220209 2023-11-08T08:02:50.575 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20220310 -> 20220310 2023-11-08T08:02:50.575 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20220411 -> 20220411 2023-11-08T08:02:50.575 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20220509 -> 20220509 2023-11-08T08:02:50.575 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20220610 -> 20220610 2023-11-08T08:02:50.575 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20220708 -> 20220708 2023-11-08T08:02:50.576 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20220815 -> 20220815 2023-11-08T08:02:50.576 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20220913 -> 20220913 2023-11-08T08:02:50.576 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20221012 -> 20221012 2023-11-08T08:02:50.576 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20221109 -> 20221109 2023-11-08T08:02:50.576 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20221214 -> 20221214 2023-11-08T08:02:50.577 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20230117 -> 20230117 2023-11-08T08:02:50.577 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20230210 -> 20230210 2023-11-08T08:02:50.577 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20230310 -> 20230310 2023-11-08T08:02:50.577 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20230404 -> 20230404 2023-11-08T08:02:50.577 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20230515 -> 20230515 2023-11-08T08:02:50.578 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20230625 -> 20230625 2023-11-08T08:02:50.578 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20230804 -> 20230804 2023-11-08T08:02:50.578 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20230919 -> 20230919 2023-11-08T08:02:50.578 INFO:teuthology.orchestra.run.smithi169.stderr: * [new tag] 20231030 -> 20231030 2023-11-08T08:02:50.908 INFO:teuthology.orchestra.run.smithi040.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-11-08T08:02:50.908 INFO:teuthology.orchestra.run.smithi040.stderr: * [new branch] main -> origin/main 2023-11-08T08:02:50.967 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20190312 -> 20190312 2023-11-08T08:02:50.967 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20190416 -> 20190416 2023-11-08T08:02:50.967 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20190514 -> 20190514 2023-11-08T08:02:50.968 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20190618 -> 20190618 2023-11-08T08:02:50.968 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20190717 -> 20190717 2023-11-08T08:02:50.968 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20190815 -> 20190815 2023-11-08T08:02:50.968 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20190923 -> 20190923 2023-11-08T08:02:50.968 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20191022 -> 20191022 2023-11-08T08:02:50.969 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20191215 -> 20191215 2023-11-08T08:02:50.971 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20200122 -> 20200122 2023-11-08T08:02:50.971 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20200316 -> 20200316 2023-11-08T08:02:50.971 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20200421 -> 20200421 2023-11-08T08:02:50.971 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20200519 -> 20200519 2023-11-08T08:02:50.972 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20200619 -> 20200619 2023-11-08T08:02:50.972 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20200721 -> 20200721 2023-11-08T08:02:50.972 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20200817 -> 20200817 2023-11-08T08:02:50.972 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20200918 -> 20200918 2023-11-08T08:02:50.972 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20201022 -> 20201022 2023-11-08T08:02:50.973 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20201118 -> 20201118 2023-11-08T08:02:50.973 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20201218 -> 20201218 2023-11-08T08:02:50.973 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20210208 -> 20210208 2023-11-08T08:02:50.973 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20210315 -> 20210315 2023-11-08T08:02:50.973 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20210511 -> 20210511 2023-11-08T08:02:50.973 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20210716 -> 20210716 2023-11-08T08:02:50.974 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20210818 -> 20210818 2023-11-08T08:02:50.976 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20210919 -> 20210919 2023-11-08T08:02:50.977 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20211027 -> 20211027 2023-11-08T08:02:50.977 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20211216 -> 20211216 2023-11-08T08:02:50.977 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20220209 -> 20220209 2023-11-08T08:02:50.977 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20220310 -> 20220310 2023-11-08T08:02:50.977 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20220411 -> 20220411 2023-11-08T08:02:50.978 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20220509 -> 20220509 2023-11-08T08:02:50.978 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20220610 -> 20220610 2023-11-08T08:02:50.978 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20220708 -> 20220708 2023-11-08T08:02:50.978 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20220815 -> 20220815 2023-11-08T08:02:50.978 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20220913 -> 20220913 2023-11-08T08:02:50.979 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20221012 -> 20221012 2023-11-08T08:02:50.979 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20221109 -> 20221109 2023-11-08T08:02:50.979 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20221214 -> 20221214 2023-11-08T08:02:50.979 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20230117 -> 20230117 2023-11-08T08:02:50.979 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20230210 -> 20230210 2023-11-08T08:02:50.980 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20230310 -> 20230310 2023-11-08T08:02:50.980 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20230404 -> 20230404 2023-11-08T08:02:50.980 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20230515 -> 20230515 2023-11-08T08:02:50.980 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20230625 -> 20230625 2023-11-08T08:02:50.980 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20230804 -> 20230804 2023-11-08T08:02:50.980 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20230919 -> 20230919 2023-11-08T08:02:50.981 INFO:teuthology.orchestra.run.smithi040.stderr: * [new tag] 20231030 -> 20231030 2023-11-08T08:02:54.237 INFO:teuthology.orchestra.run.smithi169.stderr:Updating files: 51% (1651/3177) Updating files: 52% (1653/3177) Updating files: 53% (1684/3177) Updating files: 54% (1716/3177) Updating files: 55% (1748/3177) Updating files: 56% (1780/3177) Updating files: 57% (1811/3177) Updating files: 58% (1843/3177) Updating files: 59% (1875/3177) Updating files: 60% (1907/3177) Updating files: 61% (1938/3177) Updating files: 62% (1970/3177) Updating files: 63% (2002/3177) Updating files: 64% (2034/3177) Updating files: 65% (2066/3177) Updating files: 66% (2097/3177) Updating files: 66% (2119/3177) Updating files: 67% (2129/3177) Updating files: 68% (2161/3177) Updating files: 69% (2193/3177) Updating files: 70% (2224/3177) Updating files: 71% (2256/3177) Updating files: 72% (2288/3177) Updating files: 73% (2320/3177) Updating files: 74% (2351/3177) Updating files: 75% (2383/3177) Updating files: 76% (2415/3177) Updating files: 77% (2447/3177) Updating files: 78% (2479/3177) Updating files: 79% (2510/3177) Updating files: 80% (2542/3177) Updating files: 81% (2574/3177) Updating files: 82% (2606/3177) Updating files: 83% (2637/3177) Updating files: 84% (2669/3177) Updating files: 85% (2701/3177) Updating files: 86% (2733/3177) Updating files: 87% (2764/3177) Updating files: 88% (2796/3177) Updating files: 89% (2828/3177) Updating files: 90% (2860/3177) Updating files: 91% (2892/3177) Updating files: 92% (2923/3177) Updating files: 93% (2955/3177) Updating files: 94% (2987/3177) Updating files: 95% (3019/3177) Updating files: 96% (3050/3177) Updating files: 97% (3082/3177) Updating files: 98% (3114/3177) Updating files: 99% (3146/3177) Updating files: 100% (3177/3177) Updating files: 100% (3177/3177), done. 2023-11-08T08:02:54.242 INFO:teuthology.orchestra.run.smithi169.stdout:HEAD is now at 2340796d Merge branch 'main' into 'main' 2023-11-08T08:02:54.257 INFO:teuthology.task.kernel:Downloading kernel 3b796cfe2969fb2f2d017e8146dccc1ef582e837 on client.1... 2023-11-08T08:02:54.258 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=3b796cfe2969fb2f2d017e8146dccc1ef582e837 2023-11-08T08:02:54.407 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/3b796cfe2969fb2f2d017e8146dccc1ef582e837/ubuntu/focal/flavors/default/pool/main/l/linux-6.6.0-rc5-g3b796cfe2969/ 2023-11-08T08:02:54.407 DEBUG:teuthology.orchestra.run.smithi169:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-rc5-g3b796cfe2969_6.6.0-rc5-g3b796cfe2969-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://1.chacra.ceph.com/r/kernel/testing/3b796cfe2969fb2f2d017e8146dccc1ef582e837/ubuntu/focal/flavors/default/pool/main/l/linux-6.6.0-rc5-g3b796cfe2969/ --input-file=- 2023-11-08T08:02:54.410 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi169.front.sepia.ceph.com to complete... 2023-11-08T08:02:54.824 INFO:teuthology.orchestra.run.smithi040.stderr:Updating files: 50% (1595/3177) Updating files: 51% (1621/3177) Updating files: 52% (1653/3177) Updating files: 53% (1684/3177) Updating files: 54% (1716/3177) Updating files: 55% (1748/3177) Updating files: 56% (1780/3177) Updating files: 57% (1811/3177) Updating files: 58% (1843/3177) Updating files: 59% (1875/3177) Updating files: 60% (1907/3177) Updating files: 61% (1938/3177) Updating files: 62% (1970/3177) Updating files: 63% (2002/3177) Updating files: 64% (2034/3177) Updating files: 64% (2059/3177) Updating files: 65% (2066/3177) Updating files: 66% (2097/3177) Updating files: 67% (2129/3177) Updating files: 68% (2161/3177) Updating files: 69% (2193/3177) Updating files: 70% (2224/3177) Updating files: 71% (2256/3177) Updating files: 72% (2288/3177) Updating files: 73% (2320/3177) Updating files: 74% (2351/3177) Updating files: 75% (2383/3177) Updating files: 76% (2415/3177) Updating files: 77% (2447/3177) Updating files: 78% (2479/3177) Updating files: 79% (2510/3177) Updating files: 80% (2542/3177) Updating files: 81% (2574/3177) Updating files: 81% (2592/3177) Updating files: 82% (2606/3177) Updating files: 83% (2637/3177) Updating files: 84% (2669/3177) Updating files: 85% (2701/3177) Updating files: 86% (2733/3177) Updating files: 87% (2764/3177) Updating files: 88% (2796/3177) Updating files: 89% (2828/3177) Updating files: 90% (2860/3177) Updating files: 91% (2892/3177) Updating files: 92% (2923/3177) Updating files: 93% (2955/3177) Updating files: 94% (2987/3177) Updating files: 95% (3019/3177) Updating files: 96% (3050/3177) Updating files: 97% (3082/3177) Updating files: 98% (3114/3177) Updating files: 99% (3146/3177) Updating files: 100% (3177/3177) Updating files: 100% (3177/3177), done. 2023-11-08T08:02:54.829 INFO:teuthology.orchestra.run.smithi040.stdout:HEAD is now at 2340796d Merge branch 'main' into 'main' 2023-11-08T08:02:54.845 INFO:teuthology.task.kernel:Downloading kernel 3b796cfe2969fb2f2d017e8146dccc1ef582e837 on client.0... 2023-11-08T08:02:54.845 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=3b796cfe2969fb2f2d017e8146dccc1ef582e837 2023-11-08T08:02:55.065 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/3b796cfe2969fb2f2d017e8146dccc1ef582e837/ubuntu/focal/flavors/default/pool/main/l/linux-6.6.0-rc5-g3b796cfe2969/ 2023-11-08T08:02:55.066 DEBUG:teuthology.orchestra.run.smithi040:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-rc5-g3b796cfe2969_6.6.0-rc5-g3b796cfe2969-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://1.chacra.ceph.com/r/kernel/testing/3b796cfe2969fb2f2d017e8146dccc1ef582e837/ubuntu/focal/flavors/default/pool/main/l/linux-6.6.0-rc5-g3b796cfe2969/ --input-file=- 2023-11-08T08:02:55.068 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi040.front.sepia.ceph.com to complete... 2023-11-08T08:02:57.533 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08 08:02:57 URL:https://1.chacra.ceph.com/r/kernel/testing/3b796cfe2969fb2f2d017e8146dccc1ef582e837/ubuntu/focal/flavors/default/pool/main/l/linux-6.6.0-rc5-g3b796cfe2969/linux-image-6.6.0-rc5-g3b796cfe2969_6.6.0-rc5-g3b796cfe2969-1_amd64.deb [38912044/38912044] -> "/tmp/linux-image.deb" [1] 2023-11-08T08:02:57.533 INFO:teuthology.orchestra.run.smithi169.stderr:FINISHED --2023-11-08 08:02:57-- 2023-11-08T08:02:57.533 INFO:teuthology.orchestra.run.smithi169.stderr:Total wall clock time: 3.1s 2023-11-08T08:02:57.534 INFO:teuthology.orchestra.run.smithi169.stderr:Downloaded: 1 files, 37M in 3.0s (12.6 MB/s) 2023-11-08T08:02:57.535 INFO:teuthology.task.kernel:Installing kernel 3b796cfe2969fb2f2d017e8146dccc1ef582e837 on client.1... 2023-11-08T08:02:57.535 DEBUG:teuthology.orchestra.run.smithi169:> sudo dpkg -i /tmp/linux-image.deb 2023-11-08T08:02:57.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08 08:02:58 URL:https://1.chacra.ceph.com/r/kernel/testing/3b796cfe2969fb2f2d017e8146dccc1ef582e837/ubuntu/focal/flavors/default/pool/main/l/linux-6.6.0-rc5-g3b796cfe2969/linux-image-6.6.0-rc5-g3b796cfe2969_6.6.0-rc5-g3b796cfe2969-1_amd64.deb [38912044/38912044] -> "/tmp/linux-image.deb" [1] 2023-11-08T08:02:57.701 INFO:teuthology.orchestra.run.smithi040.stderr:FINISHED --2023-11-08 08:02:58-- 2023-11-08T08:02:57.701 INFO:teuthology.orchestra.run.smithi040.stderr:Total wall clock time: 2.6s 2023-11-08T08:02:57.702 INFO:teuthology.orchestra.run.smithi040.stderr:Downloaded: 1 files, 37M in 2.4s (15.4 MB/s) 2023-11-08T08:02:57.703 INFO:teuthology.task.kernel:Installing kernel 3b796cfe2969fb2f2d017e8146dccc1ef582e837 on client.0... 2023-11-08T08:02:57.703 DEBUG:teuthology.orchestra.run.smithi040:> sudo dpkg -i /tmp/linux-image.deb 2023-11-08T08:02:57.789 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package linux-image-6.6.0-rc5-g3b796cfe2969. 2023-11-08T08:02:57.996 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-image-6.6.0-rc5-g3b796cfe2969. 2023-11-08T08:02:59.579 INFO:teuthology.orchestra.run.smithi169.stdout:(Reading database ... 104373 files and directories currently installed.) 2023-11-08T08:02:59.584 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-11-08T08:02:59.702 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... 104373 files and directories currently installed.) 2023-11-08T08:02:59.707 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-11-08T08:02:59.741 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking linux-image-6.6.0-rc5-g3b796cfe2969 (6.6.0-rc5-g3b796cfe2969-1) ... 2023-11-08T08:02:59.889 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-image-6.6.0-rc5-g3b796cfe2969 (6.6.0-rc5-g3b796cfe2969-1) ... 2023-11-08T08:03:02.791 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up linux-image-6.6.0-rc5-g3b796cfe2969 (6.6.0-rc5-g3b796cfe2969-1) ... 2023-11-08T08:03:02.832 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-image-6.6.0-rc5-g3b796cfe2969 (6.6.0-rc5-g3b796cfe2969-1) ... 2023-11-08T08:03:02.915 INFO:teuthology.orchestra.run.smithi169.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:02.960 INFO:teuthology.orchestra.run.smithi040.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:28.326 INFO:teuthology.orchestra.run.smithi169.stderr:Sourcing file `/etc/default/grub' 2023-11-08T08:03:28.338 INFO:teuthology.orchestra.run.smithi169.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-08T08:03:28.357 INFO:teuthology.orchestra.run.smithi169.stderr:Generating grub configuration file ... 2023-11-08T08:03:28.501 INFO:teuthology.orchestra.run.smithi040.stderr:Sourcing file `/etc/default/grub' 2023-11-08T08:03:28.518 INFO:teuthology.orchestra.run.smithi040.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-08T08:03:28.540 INFO:teuthology.orchestra.run.smithi040.stderr:Generating grub configuration file ... 2023-11-08T08:03:28.719 INFO:teuthology.orchestra.run.smithi169.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:28.758 INFO:teuthology.orchestra.run.smithi169.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:28.971 INFO:teuthology.orchestra.run.smithi040.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:29.000 INFO:teuthology.orchestra.run.smithi040.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:29.134 INFO:teuthology.orchestra.run.smithi169.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-11-08T08:03:29.147 INFO:teuthology.orchestra.run.smithi169.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-11-08T08:03:29.455 INFO:teuthology.orchestra.run.smithi040.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-11-08T08:03:29.468 INFO:teuthology.orchestra.run.smithi040.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-11-08T08:03:31.205 INFO:teuthology.orchestra.run.smithi169.stderr:done 2023-11-08T08:03:31.377 DEBUG:teuthology.orchestra.run.smithi169:> dpkg-deb -c /tmp/linux-image.deb 2023-11-08T08:03:31.387 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./ 2023-11-08T08:03:31.387 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./boot/ 2023-11-08T08:03:31.388 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 4415097 2023-11-07 03:19 ./boot/System.map-6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:31.404 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 169233 2023-11-07 03:19 ./boot/config-6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:31.405 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11352928 2023-11-07 03:19 ./boot/vmlinuz-6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:31.429 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/ 2023-11-08T08:03:31.430 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/kernel/ 2023-11-08T08:03:31.430 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/kernel/postinst.d/ 2023-11-08T08:03:31.430 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/kernel/postrm.d/ 2023-11-08T08:03:31.430 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/kernel/preinst.d/ 2023-11-08T08:03:31.430 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/kernel/prerm.d/ 2023-11-08T08:03:31.431 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/ 2023-11-08T08:03:31.431 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/ 2023-11-08T08:03:31.431 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/ 2023-11-08T08:03:31.431 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/ 2023-11-08T08:03:31.431 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/ 2023-11-08T08:03:31.432 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/ 2023-11-08T08:03:31.432 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/ 2023-11-08T08:03:31.432 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 490233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/aesni-intel.ko 2023-11-08T08:03:31.432 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-08T08:03:31.432 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-08T08:03:31.433 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-08T08:03:31.433 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 93113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-08T08:03:31.433 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 106729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-08T08:03:31.433 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 152241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-08T08:03:31.433 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-08T08:03:31.434 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-08T08:03:31.434 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 83889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-08T08:03:31.434 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-08T08:03:31.434 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-08T08:03:31.434 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-08T08:03:31.435 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-08T08:03:31.435 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-08T08:03:31.435 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-08T08:03:31.435 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 58585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-08T08:03:31.435 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-08T08:03:31.436 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-08T08:03:31.436 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-08T08:03:31.436 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/ 2023-11-08T08:03:31.436 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/cpu/ 2023-11-08T08:03:31.436 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/cpu/mce/ 2023-11-08T08:03:31.437 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-08T08:03:31.437 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/cpuid.ko 2023-11-08T08:03:31.437 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/msr.ko 2023-11-08T08:03:31.437 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kvm/ 2023-11-08T08:03:31.437 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 353761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-08T08:03:31.499 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 881713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-08T08:03:31.499 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 2276817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kvm/kvm.ko 2023-11-08T08:03:31.500 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/ 2023-11-08T08:03:31.500 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 53761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/af_alg.ko 2023-11-08T08:03:31.500 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/algif_hash.ko 2023-11-08T08:03:31.500 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/algif_skcipher.ko 2023-11-08T08:03:31.500 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/ansi_cprng.ko 2023-11-08T08:03:31.501 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/anubis.ko 2023-11-08T08:03:31.501 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/arc4.ko 2023-11-08T08:03:31.501 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/asymmetric_keys/ 2023-11-08T08:03:31.501 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-11-08T08:03:31.502 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/ 2023-11-08T08:03:31.502 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/async_memcpy.ko 2023-11-08T08:03:31.502 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/async_pq.ko 2023-11-08T08:03:31.502 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-08T08:03:31.502 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/async_tx.ko 2023-11-08T08:03:31.503 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/async_xor.ko 2023-11-08T08:03:31.503 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/raid6test.ko 2023-11-08T08:03:31.503 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/authenc.ko 2023-11-08T08:03:31.503 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/authencesn.ko 2023-11-08T08:03:31.503 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/blake2b_generic.ko 2023-11-08T08:03:31.504 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/blowfish_common.ko 2023-11-08T08:03:31.504 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/blowfish_generic.ko 2023-11-08T08:03:31.504 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/camellia_generic.ko 2023-11-08T08:03:31.504 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cast5_generic.ko 2023-11-08T08:03:31.504 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cast6_generic.ko 2023-11-08T08:03:31.505 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cast_common.ko 2023-11-08T08:03:31.505 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/ccm.ko 2023-11-08T08:03:31.505 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cmac.ko 2023-11-08T08:03:31.505 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/crc32_generic.ko 2023-11-08T08:03:31.505 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cryptd.ko 2023-11-08T08:03:31.506 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/crypto_engine.ko 2023-11-08T08:03:31.506 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/crypto_simd.ko 2023-11-08T08:03:31.506 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/crypto_user.ko 2023-11-08T08:03:31.506 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cts.ko 2023-11-08T08:03:31.507 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/des_generic.ko 2023-11-08T08:03:31.507 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/dh_generic.ko 2023-11-08T08:03:31.507 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/echainiv.ko 2023-11-08T08:03:31.507 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/essiv.ko 2023-11-08T08:03:31.507 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/fcrypt.ko 2023-11-08T08:03:31.508 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/khazad.ko 2023-11-08T08:03:31.508 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/lrw.ko 2023-11-08T08:03:31.508 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/lz4.ko 2023-11-08T08:03:31.508 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/lz4hc.ko 2023-11-08T08:03:31.508 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/md4.ko 2023-11-08T08:03:31.509 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/michael_mic.ko 2023-11-08T08:03:31.509 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/pcbc.ko 2023-11-08T08:03:31.509 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/pcrypt.ko 2023-11-08T08:03:31.509 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/rmd160.ko 2023-11-08T08:03:31.509 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/seed.ko 2023-11-08T08:03:31.510 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/serpent_generic.ko 2023-11-08T08:03:31.510 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 149193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/tcrypt.ko 2023-11-08T08:03:31.510 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/tea.ko 2023-11-08T08:03:31.510 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/twofish_common.ko 2023-11-08T08:03:31.510 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/twofish_generic.ko 2023-11-08T08:03:31.511 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/vmac.ko 2023-11-08T08:03:31.511 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/wp512.ko 2023-11-08T08:03:31.511 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/xcbc.ko 2023-11-08T08:03:31.511 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/xor.ko 2023-11-08T08:03:31.511 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/xts.ko 2023-11-08T08:03:31.512 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/xxhash_generic.ko 2023-11-08T08:03:31.512 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ 2023-11-08T08:03:31.512 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/ 2023-11-08T08:03:31.512 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/acpi_extlog.ko 2023-11-08T08:03:31.512 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-08T08:03:31.512 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/acpi_pad.ko 2023-11-08T08:03:31.513 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/apei/ 2023-11-08T08:03:31.513 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/apei/einj.ko 2023-11-08T08:03:31.513 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/ec_sys.ko 2023-11-08T08:03:31.513 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/platform_profile.ko 2023-11-08T08:03:31.513 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/sbs.ko 2023-11-08T08:03:31.514 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/sbshc.ko 2023-11-08T08:03:31.514 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 103025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/video.ko 2023-11-08T08:03:31.514 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/ 2023-11-08T08:03:31.514 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/acard-ahci.ko 2023-11-08T08:03:31.515 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 88865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/ahci.ko 2023-11-08T08:03:31.515 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/ahci_platform.ko 2023-11-08T08:03:31.515 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 88289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/libahci.ko 2023-11-08T08:03:31.515 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/libahci_platform.ko 2023-11-08T08:03:31.515 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_acpi.ko 2023-11-08T08:03:31.516 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_ali.ko 2023-11-08T08:03:31.516 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_amd.ko 2023-11-08T08:03:31.516 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_artop.ko 2023-11-08T08:03:31.516 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_atiixp.ko 2023-11-08T08:03:31.516 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_atp867x.ko 2023-11-08T08:03:31.517 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_cmd640.ko 2023-11-08T08:03:31.517 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_cmd64x.ko 2023-11-08T08:03:31.517 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_cypress.ko 2023-11-08T08:03:31.517 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_efar.ko 2023-11-08T08:03:31.517 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_hpt366.ko 2023-11-08T08:03:31.518 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_hpt37x.ko 2023-11-08T08:03:31.518 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-08T08:03:31.518 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-08T08:03:31.518 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_it8213.ko 2023-11-08T08:03:31.518 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_it821x.ko 2023-11-08T08:03:31.519 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_jmicron.ko 2023-11-08T08:03:31.519 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_legacy.ko 2023-11-08T08:03:31.519 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_marvell.ko 2023-11-08T08:03:31.519 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_mpiix.ko 2023-11-08T08:03:31.519 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_netcell.ko 2023-11-08T08:03:31.519 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_ninja32.ko 2023-11-08T08:03:31.520 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_ns87410.ko 2023-11-08T08:03:31.520 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_ns87415.ko 2023-11-08T08:03:31.520 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_oldpiix.ko 2023-11-08T08:03:31.520 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_opti.ko 2023-11-08T08:03:31.520 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_optidma.ko 2023-11-08T08:03:31.521 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_pcmcia.ko 2023-11-08T08:03:31.521 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-08T08:03:31.521 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-08T08:03:31.521 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_piccolo.ko 2023-11-08T08:03:31.521 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_radisys.ko 2023-11-08T08:03:31.522 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_rdc.ko 2023-11-08T08:03:31.522 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_rz1000.ko 2023-11-08T08:03:31.522 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_sch.ko 2023-11-08T08:03:31.522 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_serverworks.ko 2023-11-08T08:03:31.522 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_sil680.ko 2023-11-08T08:03:31.523 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_sl82c105.ko 2023-11-08T08:03:31.523 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_triflex.ko 2023-11-08T08:03:31.523 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_via.ko 2023-11-08T08:03:31.523 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pdc_adma.ko 2023-11-08T08:03:31.523 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_inic162x.ko 2023-11-08T08:03:31.524 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 83345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_mv.ko 2023-11-08T08:03:31.524 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 70393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_nv.ko 2023-11-08T08:03:31.524 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_promise.ko 2023-11-08T08:03:31.524 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_qstor.ko 2023-11-08T08:03:31.524 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_sil.ko 2023-11-08T08:03:31.525 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_sil24.ko 2023-11-08T08:03:31.525 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_sis.ko 2023-11-08T08:03:31.525 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_svw.ko 2023-11-08T08:03:31.525 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_sx4.ko 2023-11-08T08:03:31.525 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_uli.ko 2023-11-08T08:03:31.526 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_via.ko 2023-11-08T08:03:31.526 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_vsc.ko 2023-11-08T08:03:31.526 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/auxdisplay/ 2023-11-08T08:03:31.526 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/auxdisplay/ks0108.ko 2023-11-08T08:03:31.526 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/bcma/ 2023-11-08T08:03:31.527 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 124473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/bcma/bcma.ko 2023-11-08T08:03:31.527 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/ 2023-11-08T08:03:31.527 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/aoe/ 2023-11-08T08:03:31.527 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 113665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/aoe/aoe.ko 2023-11-08T08:03:31.527 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/drbd/ 2023-11-08T08:03:31.528 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 965585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/drbd/drbd.ko 2023-11-08T08:03:31.528 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 173225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/floppy.ko 2023-11-08T08:03:31.528 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/mtip32xx/ 2023-11-08T08:03:31.528 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 94297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-08T08:03:31.529 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 115793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/nbd.ko 2023-11-08T08:03:31.529 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/null_blk/ 2023-11-08T08:03:31.529 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 110449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/null_blk/null_blk.ko 2023-11-08T08:03:31.529 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 98193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/pktcdvd.ko 2023-11-08T08:03:31.529 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 263769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/rbd.ko 2023-11-08T08:03:31.530 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/xen-blkback/ 2023-11-08T08:03:31.530 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 98145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-08T08:03:31.530 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/zram/ 2023-11-08T08:03:31.530 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 60217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/zram/zram.ko 2023-11-08T08:03:31.530 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ 2023-11-08T08:03:31.531 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/applicom.ko 2023-11-08T08:03:31.531 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hangcheck-timer.ko 2023-11-08T08:03:31.531 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/ 2023-11-08T08:03:31.532 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-08T08:03:31.532 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-08T08:03:31.532 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-08T08:03:31.532 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/via-rng.ko 2023-11-08T08:03:31.533 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-08T08:03:31.533 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ 2023-11-08T08:03:31.533 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-08T08:03:31.533 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 148601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-08T08:03:31.534 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-08T08:03:31.534 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 139313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-08T08:03:31.534 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 70921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-08T08:03:31.535 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 53513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-08T08:03:31.535 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/lp.ko 2023-11-08T08:03:31.535 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/mwave/ 2023-11-08T08:03:31.535 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 84097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/mwave/mwave.ko 2023-11-08T08:03:31.535 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/nvram.ko 2023-11-08T08:03:31.536 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ppdev.ko 2023-11-08T08:03:31.536 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tlclk.ko 2023-11-08T08:03:31.536 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/ 2023-11-08T08:03:31.536 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-08T08:03:31.537 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-11-08T08:03:31.537 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-11-08T08:03:31.537 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-11-08T08:03:31.538 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-08T08:03:31.538 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-08T08:03:31.538 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-11-08T08:03:31.538 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/xillybus/ 2023-11-08T08:03:31.539 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/xillybus/xillybus_class.ko 2023-11-08T08:03:31.539 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/xillybus/xillybus_core.ko 2023-11-08T08:03:31.539 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-11-08T08:03:31.539 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/clk/ 2023-11-08T08:03:31.540 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/clk/clk-palmas.ko 2023-11-08T08:03:31.540 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/clk/clk-si5351.ko 2023-11-08T08:03:31.540 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/clk/clk-twl6040.ko 2023-11-08T08:03:31.540 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/clk/clk-wm831x.ko 2023-11-08T08:03:31.540 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/cpufreq/ 2023-11-08T08:03:31.541 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-08T08:03:31.541 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-08T08:03:31.541 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-08T08:03:31.541 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/ 2023-11-08T08:03:31.542 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/ccp/ 2023-11-08T08:03:31.542 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 70801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-08T08:03:31.542 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 243025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/ccp/ccp.ko 2023-11-08T08:03:31.542 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/ 2023-11-08T08:03:31.543 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/qat/ 2023-11-08T08:03:31.543 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/qat/qat_common/ 2023-11-08T08:03:31.543 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 438761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-08T08:03:31.543 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-11-08T08:03:31.543 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-08T08:03:31.544 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/padlock-aes.ko 2023-11-08T08:03:31.544 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/padlock-sha.ko 2023-11-08T08:03:31.544 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/virtio/ 2023-11-08T08:03:31.544 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 68857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-08T08:03:31.545 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dca/ 2023-11-08T08:03:31.545 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dca/dca.ko 2023-11-08T08:03:31.545 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dma/ 2023-11-08T08:03:31.545 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dma/dw/ 2023-11-08T08:03:31.546 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-08T08:03:31.546 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dma/ioat/ 2023-11-08T08:03:31.546 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 122569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-08T08:03:31.546 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/ 2023-11-08T08:03:31.547 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 79953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/amd64_edac.ko 2023-11-08T08:03:31.547 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/e752x_edac.ko 2023-11-08T08:03:31.547 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 63617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/edac_mce_amd.ko 2023-11-08T08:03:31.547 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i3000_edac.ko 2023-11-08T08:03:31.547 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i3200_edac.ko 2023-11-08T08:03:31.547 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i5100_edac.ko 2023-11-08T08:03:31.548 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i5400_edac.ko 2023-11-08T08:03:31.548 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i7300_edac.ko 2023-11-08T08:03:31.548 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 52041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i7core_edac.ko 2023-11-08T08:03:31.549 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i82975x_edac.ko 2023-11-08T08:03:31.549 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/ie31200_edac.ko 2023-11-08T08:03:31.549 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 56825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/sb_edac.ko 2023-11-08T08:03:31.549 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/x38_edac.ko 2023-11-08T08:03:31.550 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/ 2023-11-08T08:03:31.550 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-gpio.ko 2023-11-08T08:03:31.550 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-max14577.ko 2023-11-08T08:03:31.550 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-max77693.ko 2023-11-08T08:03:31.551 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-max8997.ko 2023-11-08T08:03:31.551 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-palmas.ko 2023-11-08T08:03:31.551 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-rt8973a.ko 2023-11-08T08:03:31.551 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-sm5502.ko 2023-11-08T08:03:31.552 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/firmware/ 2023-11-08T08:03:31.552 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/firmware/dmi-sysfs.ko 2023-11-08T08:03:31.552 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/firmware/efi/ 2023-11-08T08:03:31.552 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/firmware/efi/efi-pstore.ko 2023-11-08T08:03:31.553 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-08T08:03:31.553 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/ 2023-11-08T08:03:31.553 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-adp5520.ko 2023-11-08T08:03:31.554 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-amd8111.ko 2023-11-08T08:03:31.554 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-arizona.ko 2023-11-08T08:03:31.554 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-da9052.ko 2023-11-08T08:03:31.554 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-da9055.ko 2023-11-08T08:03:31.555 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-dln2.ko 2023-11-08T08:03:31.555 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 77849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-f7188x.ko 2023-11-08T08:03:31.555 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-generic.ko 2023-11-08T08:03:31.555 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-ich.ko 2023-11-08T08:03:31.556 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-11-08T08:03:31.556 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-kempld.ko 2023-11-08T08:03:31.556 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-lp3943.ko 2023-11-08T08:03:31.557 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-max7300.ko 2023-11-08T08:03:31.557 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-max730x.ko 2023-11-08T08:03:31.557 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-max732x.ko 2023-11-08T08:03:31.557 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-11-08T08:03:31.558 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-pca953x.ko 2023-11-08T08:03:31.558 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-pcf857x.ko 2023-11-08T08:03:31.558 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-rdc321x.ko 2023-11-08T08:03:31.559 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-sch.ko 2023-11-08T08:03:31.559 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-sch311x.ko 2023-11-08T08:03:31.559 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-tps65912.ko 2023-11-08T08:03:31.559 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-twl4030.ko 2023-11-08T08:03:31.560 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-twl6040.ko 2023-11-08T08:03:31.560 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-08T08:03:31.560 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-vx855.ko 2023-11-08T08:03:31.561 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-wm831x.ko 2023-11-08T08:03:31.561 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-wm8350.ko 2023-11-08T08:03:31.561 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-wm8994.ko 2023-11-08T08:03:31.561 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/ 2023-11-08T08:03:31.562 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-a4tech.ko 2023-11-08T08:03:31.562 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-apple.ko 2023-11-08T08:03:31.562 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-appleir.ko 2023-11-08T08:03:31.562 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-aureal.ko 2023-11-08T08:03:31.562 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-axff.ko 2023-11-08T08:03:31.563 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-belkin.ko 2023-11-08T08:03:31.563 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-cherry.ko 2023-11-08T08:03:31.563 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-chicony.ko 2023-11-08T08:03:31.563 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 60649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-cp2112.ko 2023-11-08T08:03:31.564 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-cypress.ko 2023-11-08T08:03:31.564 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-dr.ko 2023-11-08T08:03:31.564 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-elecom.ko 2023-11-08T08:03:31.564 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-elo.ko 2023-11-08T08:03:31.564 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-emsff.ko 2023-11-08T08:03:31.565 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-ezkey.ko 2023-11-08T08:03:31.565 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-gaff.ko 2023-11-08T08:03:31.565 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-generic.ko 2023-11-08T08:03:31.565 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-gt683r.ko 2023-11-08T08:03:31.565 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-gyration.ko 2023-11-08T08:03:31.566 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-08T08:03:31.566 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-08T08:03:31.566 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-holtekff.ko 2023-11-08T08:03:31.566 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-hyperv.ko 2023-11-08T08:03:31.567 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-icade.ko 2023-11-08T08:03:31.567 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-kensington.ko 2023-11-08T08:03:31.567 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-keytouch.ko 2023-11-08T08:03:31.569 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-kye.ko 2023-11-08T08:03:31.569 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-lcpower.ko 2023-11-08T08:03:31.569 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-led.ko 2023-11-08T08:03:31.570 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-lenovo.ko 2023-11-08T08:03:31.570 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-lg-g15.ko 2023-11-08T08:03:31.570 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 58185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-08T08:03:31.570 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 101369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-08T08:03:31.570 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 80993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-logitech.ko 2023-11-08T08:03:31.571 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-magicmouse.ko 2023-11-08T08:03:31.571 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-microsoft.ko 2023-11-08T08:03:31.571 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-monterey.ko 2023-11-08T08:03:31.571 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-multitouch.ko 2023-11-08T08:03:31.572 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-ntrig.ko 2023-11-08T08:03:31.572 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-ortek.ko 2023-11-08T08:03:31.572 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-penmount.ko 2023-11-08T08:03:31.572 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-petalynx.ko 2023-11-08T08:03:31.572 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 72937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-picolcd.ko 2023-11-08T08:03:31.573 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-pl.ko 2023-11-08T08:03:31.573 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-plantronics.ko 2023-11-08T08:03:31.573 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-primax.ko 2023-11-08T08:03:31.573 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-rmi.ko 2023-11-08T08:03:31.573 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-08T08:03:31.574 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-common.ko 2023-11-08T08:03:31.574 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-08T08:03:31.574 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-08T08:03:31.574 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-08T08:03:31.575 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-08T08:03:31.575 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-08T08:03:31.575 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-08T08:03:31.575 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-08T08:03:31.575 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-08T08:03:31.576 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-08T08:03:31.576 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat.ko 2023-11-08T08:03:31.576 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-saitek.ko 2023-11-08T08:03:31.576 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-samsung.ko 2023-11-08T08:03:31.576 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-08T08:03:31.577 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-sjoy.ko 2023-11-08T08:03:31.577 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-sony.ko 2023-11-08T08:03:31.577 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-speedlink.ko 2023-11-08T08:03:31.577 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-steelseries.ko 2023-11-08T08:03:31.578 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-sunplus.ko 2023-11-08T08:03:31.578 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-08T08:03:31.578 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-tivo.ko 2023-11-08T08:03:31.578 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-tmff.ko 2023-11-08T08:03:31.579 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-topseed.ko 2023-11-08T08:03:31.579 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-twinhan.ko 2023-11-08T08:03:31.579 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 71073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-uclogic.ko 2023-11-08T08:03:31.579 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-waltop.ko 2023-11-08T08:03:31.579 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 143473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-wiimote.ko 2023-11-08T08:03:31.580 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-xinmo.ko 2023-11-08T08:03:31.580 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-zpff.ko 2023-11-08T08:03:31.580 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-zydacron.ko 2023-11-08T08:03:31.580 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 267825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid.ko 2023-11-08T08:03:31.580 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/uhid.ko 2023-11-08T08:03:31.581 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/usbhid/ 2023-11-08T08:03:31.581 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 125721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/usbhid/usbhid.ko 2023-11-08T08:03:31.581 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/usbhid/usbkbd.ko 2023-11-08T08:03:31.581 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/usbhid/usbmouse.ko 2023-11-08T08:03:31.582 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 217873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/wacom.ko 2023-11-08T08:03:31.582 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hv/ 2023-11-08T08:03:31.582 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 64201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hv/hv_balloon.ko 2023-11-08T08:03:31.582 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 89625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hv/hv_utils.ko 2023-11-08T08:03:31.582 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 305465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hv/hv_vmbus.ko 2023-11-08T08:03:31.583 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ 2023-11-08T08:03:31.583 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 66521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/abituguru.ko 2023-11-08T08:03:31.583 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 61393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/abituguru3.ko 2023-11-08T08:03:31.583 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-08T08:03:31.584 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ad7414.ko 2023-11-08T08:03:31.584 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ad7418.ko 2023-11-08T08:03:31.584 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adc128d818.ko 2023-11-08T08:03:31.584 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adm1025.ko 2023-11-08T08:03:31.584 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 97977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adm1026.ko 2023-11-08T08:03:31.585 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adm1029.ko 2023-11-08T08:03:31.585 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 45553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adm1031.ko 2023-11-08T08:03:31.585 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adm9240.ko 2023-11-08T08:03:31.585 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ads7828.ko 2023-11-08T08:03:31.585 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7410.ko 2023-11-08T08:03:31.585 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7411.ko 2023-11-08T08:03:31.586 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 74433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7462.ko 2023-11-08T08:03:31.586 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7470.ko 2023-11-08T08:03:31.586 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 67929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7475.ko 2023-11-08T08:03:31.586 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7x10.ko 2023-11-08T08:03:31.586 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/amc6821.ko 2023-11-08T08:03:31.587 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/applesmc.ko 2023-11-08T08:03:31.587 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/asb100.ko 2023-11-08T08:03:31.587 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 58617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/asc7621.ko 2023-11-08T08:03:31.588 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-08T08:03:31.588 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/atxp1.ko 2023-11-08T08:03:31.588 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/coretemp.ko 2023-11-08T08:03:31.588 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/da9052-hwmon.ko 2023-11-08T08:03:31.588 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/da9055-hwmon.ko 2023-11-08T08:03:31.589 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-08T08:03:31.589 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 99601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/dme1737.ko 2023-11-08T08:03:31.589 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ds1621.ko 2023-11-08T08:03:31.589 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ds620.ko 2023-11-08T08:03:31.590 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/emc1403.ko 2023-11-08T08:03:31.590 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/emc2103.ko 2023-11-08T08:03:31.590 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/emc6w201.ko 2023-11-08T08:03:31.590 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 66985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/f71805f.ko 2023-11-08T08:03:31.591 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 94457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/f71882fg.ko 2023-11-08T08:03:31.591 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/f75375s.ko 2023-11-08T08:03:31.591 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/fam15h_power.ko 2023-11-08T08:03:31.591 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/fschmd.ko 2023-11-08T08:03:31.592 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/g760a.ko 2023-11-08T08:03:31.592 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/g762.ko 2023-11-08T08:03:31.592 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 46801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/gl518sm.ko 2023-11-08T08:03:31.593 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 45529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/gl520sm.ko 2023-11-08T08:03:31.593 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/hih6130.ko 2023-11-08T08:03:31.593 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-08T08:03:31.593 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/i5500_temp.ko 2023-11-08T08:03:31.593 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/i5k_amb.ko 2023-11-08T08:03:31.594 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ibmaem.ko 2023-11-08T08:03:31.594 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ibmpex.ko 2023-11-08T08:03:31.594 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ina209.ko 2023-11-08T08:03:31.594 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ina2xx.ko 2023-11-08T08:03:31.595 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 130449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/it87.ko 2023-11-08T08:03:31.595 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/jc42.ko 2023-11-08T08:03:31.595 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/k10temp.ko 2023-11-08T08:03:31.595 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/k8temp.ko 2023-11-08T08:03:31.596 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lineage-pem.ko 2023-11-08T08:03:31.596 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 48601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm63.ko 2023-11-08T08:03:31.596 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm73.ko 2023-11-08T08:03:31.597 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm75.ko 2023-11-08T08:03:31.597 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm77.ko 2023-11-08T08:03:31.597 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm78.ko 2023-11-08T08:03:31.597 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm80.ko 2023-11-08T08:03:31.598 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm83.ko 2023-11-08T08:03:31.598 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 72833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm85.ko 2023-11-08T08:03:31.598 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 46153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm87.ko 2023-11-08T08:03:31.598 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm90.ko 2023-11-08T08:03:31.598 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm92.ko 2023-11-08T08:03:31.599 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 108841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm93.ko 2023-11-08T08:03:31.599 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm95234.ko 2023-11-08T08:03:31.599 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm95241.ko 2023-11-08T08:03:31.599 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm95245.ko 2023-11-08T08:03:31.600 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc2945.ko 2023-11-08T08:03:31.600 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4151.ko 2023-11-08T08:03:31.600 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4215.ko 2023-11-08T08:03:31.600 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4222.ko 2023-11-08T08:03:31.601 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4245.ko 2023-11-08T08:03:31.601 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4260.ko 2023-11-08T08:03:31.601 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4261.ko 2023-11-08T08:03:31.601 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max16065.ko 2023-11-08T08:03:31.601 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max1619.ko 2023-11-08T08:03:31.602 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max1668.ko 2023-11-08T08:03:31.602 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max197.ko 2023-11-08T08:03:31.602 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max6639.ko 2023-11-08T08:03:31.602 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max6650.ko 2023-11-08T08:03:31.603 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max6697.ko 2023-11-08T08:03:31.603 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/mc13783-adc.ko 2023-11-08T08:03:31.603 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/mcp3021.ko 2023-11-08T08:03:31.603 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-11-08T08:03:31.604 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/nct6683.ko 2023-11-08T08:03:31.604 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 119817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/nct6775-core.ko 2023-11-08T08:03:31.604 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 60705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/nct6775.ko 2023-11-08T08:03:31.604 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 59737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/nct7802.ko 2023-11-08T08:03:31.605 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 82337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pc87360.ko 2023-11-08T08:03:31.605 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 54433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pc87427.ko 2023-11-08T08:03:31.605 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pcf8591.ko 2023-11-08T08:03:31.605 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/ 2023-11-08T08:03:31.605 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-08T08:03:31.606 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-08T08:03:31.606 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-08T08:03:31.606 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-08T08:03:31.606 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-08T08:03:31.607 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-08T08:03:31.607 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-08T08:03:31.607 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 82577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-08T08:03:31.607 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-08T08:03:31.607 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-08T08:03:31.608 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-08T08:03:31.608 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-08T08:03:31.608 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/powr1220.ko 2023-11-08T08:03:31.608 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sch5627.ko 2023-11-08T08:03:31.608 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sch5636.ko 2023-11-08T08:03:31.609 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-08T08:03:31.609 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sht15.ko 2023-11-08T08:03:31.609 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sht21.ko 2023-11-08T08:03:31.609 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/shtc1.ko 2023-11-08T08:03:31.610 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sis5595.ko 2023-11-08T08:03:31.610 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/smsc47b397.ko 2023-11-08T08:03:31.610 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/smsc47m1.ko 2023-11-08T08:03:31.610 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/smsc47m192.ko 2023-11-08T08:03:31.610 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/thmc50.ko 2023-11-08T08:03:31.611 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/tmp102.ko 2023-11-08T08:03:31.611 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/tmp103.ko 2023-11-08T08:03:31.611 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/tmp401.ko 2023-11-08T08:03:31.611 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/tmp421.ko 2023-11-08T08:03:31.611 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/via-cputemp.ko 2023-11-08T08:03:31.612 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/via686a.ko 2023-11-08T08:03:31.612 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 52457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/vt1211.ko 2023-11-08T08:03:31.612 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/vt8231.ko 2023-11-08T08:03:31.612 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 57513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83627ehf.ko 2023-11-08T08:03:31.613 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 78873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83627hf.ko 2023-11-08T08:03:31.613 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 71345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83781d.ko 2023-11-08T08:03:31.613 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 63289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83791d.ko 2023-11-08T08:03:31.613 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 71321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83792d.ko 2023-11-08T08:03:31.613 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 101473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83793.ko 2023-11-08T08:03:31.614 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 119505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83795.ko 2023-11-08T08:03:31.614 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83l785ts.ko 2023-11-08T08:03:31.614 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83l786ng.ko 2023-11-08T08:03:31.614 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-11-08T08:03:31.615 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-11-08T08:03:31.615 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/ 2023-11-08T08:03:31.615 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/algos/ 2023-11-08T08:03:31.615 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-08T08:03:31.615 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-08T08:03:31.616 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/ 2023-11-08T08:03:31.616 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-11-08T08:03:31.616 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-11-08T08:03:31.616 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-11-08T08:03:31.616 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-08T08:03:31.617 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-08T08:03:31.617 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-08T08:03:31.617 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-11-08T08:03:31.617 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 6929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-08T08:03:31.617 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-11-08T08:03:31.618 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-08T08:03:31.618 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-08T08:03:31.618 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-08T08:03:31.618 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-08T08:03:31.619 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-11-08T08:03:31.619 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-11-08T08:03:31.619 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 62817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-08T08:03:31.619 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-08T08:03:31.620 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-08T08:03:31.620 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-11-08T08:03:31.620 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-08T08:03:31.620 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-08T08:03:31.620 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-11-08T08:03:31.621 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-08T08:03:31.621 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-08T08:03:31.621 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-08T08:03:31.621 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-11-08T08:03:31.622 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-08T08:03:31.622 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-08T08:03:31.622 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-11-08T08:03:31.622 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-11-08T08:03:31.622 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-08T08:03:31.623 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-11-08T08:03:31.623 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-08T08:03:31.623 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-08T08:03:31.623 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-08T08:03:31.624 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-08T08:03:31.624 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-11-08T08:03:31.624 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/i2c-mux.ko 2023-11-08T08:03:31.624 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/i2c-smbus.ko 2023-11-08T08:03:31.625 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/i2c-stub.ko 2023-11-08T08:03:31.625 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/muxes/ 2023-11-08T08:03:31.625 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-11-08T08:03:31.625 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-11-08T08:03:31.625 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-11-08T08:03:31.626 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ 2023-11-08T08:03:31.626 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/ 2023-11-08T08:03:31.626 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 273609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-08T08:03:31.626 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 879969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/ib_core.ko 2023-11-08T08:03:31.627 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 69337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-08T08:03:31.627 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 324217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-08T08:03:31.627 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 97985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-08T08:03:31.627 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 282273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-08T08:03:31.627 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 66201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-08T08:03:31.628 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/ 2023-11-08T08:03:31.628 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/cxgb4/ 2023-11-08T08:03:31.628 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 493073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-08T08:03:31.628 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mlx4/ 2023-11-08T08:03:31.629 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 419737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-08T08:03:31.629 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mlx5/ 2023-11-08T08:03:31.629 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 739025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-08T08:03:31.629 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mthca/ 2023-11-08T08:03:31.629 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 296625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-08T08:03:31.630 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/ocrdma/ 2023-11-08T08:03:31.630 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 153697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-11-08T08:03:31.630 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/usnic/ 2023-11-08T08:03:31.630 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 168969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-11-08T08:03:31.630 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/ 2023-11-08T08:03:31.631 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/ipoib/ 2023-11-08T08:03:31.631 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 263081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-08T08:03:31.631 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/iser/ 2023-11-08T08:03:31.631 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 100241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-08T08:03:31.632 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/isert/ 2023-11-08T08:03:31.632 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 111425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-08T08:03:31.632 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/srp/ 2023-11-08T08:03:31.632 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 128753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-08T08:03:31.633 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/srpt/ 2023-11-08T08:03:31.633 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 129529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-08T08:03:31.633 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/ 2023-11-08T08:03:31.633 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/evbug.ko 2023-11-08T08:03:31.633 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/ff-memless.ko 2023-11-08T08:03:31.634 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/gameport/ 2023-11-08T08:03:31.634 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-08T08:03:31.634 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-08T08:03:31.634 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/gameport/gameport.ko 2023-11-08T08:03:31.635 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/joydev.ko 2023-11-08T08:03:31.635 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/ 2023-11-08T08:03:31.635 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-11-08T08:03:31.635 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-11-08T08:03:31.635 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-11-08T08:03:31.636 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-11-08T08:03:31.636 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-08T08:03:31.636 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-11-08T08:03:31.636 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/lkkbd.ko 2023-11-08T08:03:31.637 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/lm8323.ko 2023-11-08T08:03:31.637 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/lm8333.ko 2023-11-08T08:03:31.637 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-11-08T08:03:31.637 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-11-08T08:03:31.638 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-11-08T08:03:31.638 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-11-08T08:03:31.638 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/newtonkbd.ko 2023-11-08T08:03:31.638 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-11-08T08:03:31.638 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/qt1070.ko 2023-11-08T08:03:31.639 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/qt2160.ko 2023-11-08T08:03:31.639 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-11-08T08:03:31.639 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/stowaway.ko 2023-11-08T08:03:31.639 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/sunkbd.ko 2023-11-08T08:03:31.640 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-11-08T08:03:31.640 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-11-08T08:03:31.640 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-11-08T08:03:31.640 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/xtkbd.ko 2023-11-08T08:03:31.640 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/matrix-keymap.ko 2023-11-08T08:03:31.641 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/ 2023-11-08T08:03:31.641 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/appletouch.ko 2023-11-08T08:03:31.641 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/bcm5974.ko 2023-11-08T08:03:31.641 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 123681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/cyapatp.ko 2023-11-08T08:03:31.642 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 89449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-08T08:03:31.642 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/gpio_mouse.ko 2023-11-08T08:03:31.642 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 335977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/psmouse.ko 2023-11-08T08:03:31.642 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/sermouse.ko 2023-11-08T08:03:31.642 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-08T08:03:31.643 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-08T08:03:31.643 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-08T08:03:31.643 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/rmi4/ 2023-11-08T08:03:31.643 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 109873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-08T08:03:31.644 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/ 2023-11-08T08:03:31.644 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/altera_ps2.ko 2023-11-08T08:03:31.644 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/arc_ps2.ko 2023-11-08T08:03:31.644 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/ct82c710.ko 2023-11-08T08:03:31.645 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-08T08:03:31.645 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/parkbd.ko 2023-11-08T08:03:31.645 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/pcips2.ko 2023-11-08T08:03:31.645 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/ps2mult.ko 2023-11-08T08:03:31.645 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/serio_raw.ko 2023-11-08T08:03:31.646 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/serport.ko 2023-11-08T08:03:31.646 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/sparse-keymap.ko 2023-11-08T08:03:31.646 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/iommu/ 2023-11-08T08:03:31.646 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/iommu/amd/ 2023-11-08T08:03:31.647 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-08T08:03:31.647 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/ 2023-11-08T08:03:31.647 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-88pm860x.ko 2023-11-08T08:03:31.647 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-adp5520.ko 2023-11-08T08:03:31.648 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 59753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-bd2802.ko 2023-11-08T08:03:31.648 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-blinkm.ko 2023-11-08T08:03:31.648 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-da903x.ko 2023-11-08T08:03:31.648 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-da9052.ko 2023-11-08T08:03:31.648 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-gpio.ko 2023-11-08T08:03:31.649 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lm3530.ko 2023-11-08T08:03:31.649 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lm3533.ko 2023-11-08T08:03:31.649 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lm355x.ko 2023-11-08T08:03:31.649 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lm3642.ko 2023-11-08T08:03:31.649 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lp3944.ko 2023-11-08T08:03:31.650 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lp8788.ko 2023-11-08T08:03:31.650 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-max8997.ko 2023-11-08T08:03:31.650 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-mc13783.ko 2023-11-08T08:03:31.650 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-menf21bmc.ko 2023-11-08T08:03:31.651 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-pca9532.ko 2023-11-08T08:03:31.651 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-pca955x.ko 2023-11-08T08:03:31.651 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-pca963x.ko 2023-11-08T08:03:31.651 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-ss4200.ko 2023-11-08T08:03:31.651 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-tca6507.ko 2023-11-08T08:03:31.652 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-wm831x-status.ko 2023-11-08T08:03:31.652 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-wm8350.ko 2023-11-08T08:03:31.652 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ 2023-11-08T08:03:31.652 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-08T08:03:31.653 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-08T08:03:31.653 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-08T08:03:31.653 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-08T08:03:31.653 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-08T08:03:31.654 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-08T08:03:31.654 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-08T08:03:31.654 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-08T08:03:31.654 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/ 2023-11-08T08:03:31.655 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/bcache/ 2023-11-08T08:03:31.655 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 654969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/bcache/bcache.ko 2023-11-08T08:03:31.655 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-bio-prison.ko 2023-11-08T08:03:31.655 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 89601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-bufio.ko 2023-11-08T08:03:31.655 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-cache-smq.ko 2023-11-08T08:03:31.656 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 156009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-cache.ko 2023-11-08T08:03:31.656 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 106897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-crypt.ko 2023-11-08T08:03:31.656 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-delay.ko 2023-11-08T08:03:31.656 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 53857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-era.ko 2023-11-08T08:03:31.657 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-flakey.ko 2023-11-08T08:03:31.657 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-log-userspace.ko 2023-11-08T08:03:31.657 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-log.ko 2023-11-08T08:03:31.657 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-mirror.ko 2023-11-08T08:03:31.658 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 78033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-multipath.ko 2023-11-08T08:03:31.658 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-queue-length.ko 2023-11-08T08:03:31.658 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 70825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-raid.ko 2023-11-08T08:03:31.658 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-region-hash.ko 2023-11-08T08:03:31.658 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-round-robin.ko 2023-11-08T08:03:31.659 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-service-time.ko 2023-11-08T08:03:31.659 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 109593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-snapshot.ko 2023-11-08T08:03:31.659 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-switch.ko 2023-11-08T08:03:31.659 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 181865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-thin-pool.ko 2023-11-08T08:03:31.660 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-verity.ko 2023-11-08T08:03:31.660 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-zero.ko 2023-11-08T08:03:31.660 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/faulty.ko 2023-11-08T08:03:31.660 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/linear.ko 2023-11-08T08:03:31.660 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/multipath.ko 2023-11-08T08:03:31.661 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/persistent-data/ 2023-11-08T08:03:31.661 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 199937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-08T08:03:31.661 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/raid0.ko 2023-11-08T08:03:31.661 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 113073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/raid1.ko 2023-11-08T08:03:31.662 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 158681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/raid10.ko 2023-11-08T08:03:31.662 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 395721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/raid456.ko 2023-11-08T08:03:31.662 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/ 2023-11-08T08:03:31.662 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/ 2023-11-08T08:03:31.662 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 178137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptbase.ko 2023-11-08T08:03:31.663 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 63385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptctl.ko 2023-11-08T08:03:31.663 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptfc.ko 2023-11-08T08:03:31.663 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptlan.ko 2023-11-08T08:03:31.663 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 123321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptsas.ko 2023-11-08T08:03:31.664 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 79041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptscsih.ko 2023-11-08T08:03:31.664 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 53081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptspi.ko 2023-11-08T08:03:31.664 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/ 2023-11-08T08:03:31.664 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/88pm800.ko 2023-11-08T08:03:31.664 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/88pm805.ko 2023-11-08T08:03:31.665 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/88pm80x.ko 2023-11-08T08:03:31.665 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/arizona-i2c.ko 2023-11-08T08:03:31.665 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 123721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/arizona.ko 2023-11-08T08:03:31.665 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/bcm590xx.ko 2023-11-08T08:03:31.666 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/cros_ec_dev.ko 2023-11-08T08:03:31.666 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/dln2.ko 2023-11-08T08:03:31.666 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/janz-cmodio.ko 2023-11-08T08:03:31.666 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 53585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/kempld-core.ko 2023-11-08T08:03:31.666 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/lm3533-core.ko 2023-11-08T08:03:31.667 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-11-08T08:03:31.667 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/lp3943.ko 2023-11-08T08:03:31.667 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 54569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/lpc_ich.ko 2023-11-08T08:03:31.667 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/lpc_sch.ko 2023-11-08T08:03:31.668 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/max8907.ko 2023-11-08T08:03:31.668 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/mc13xxx-core.ko 2023-11-08T08:03:31.668 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-11-08T08:03:31.668 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/menf21bmc.ko 2023-11-08T08:03:31.668 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/pcf50633-adc.ko 2023-11-08T08:03:31.669 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/pcf50633-gpio.ko 2023-11-08T08:03:31.669 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/pcf50633.ko 2023-11-08T08:03:31.669 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-11-08T08:03:31.669 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/retu-mfd.ko 2023-11-08T08:03:31.670 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 80425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/si476x-core.ko 2023-11-08T08:03:31.670 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 48457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/sm501.ko 2023-11-08T08:03:31.670 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-11-08T08:03:31.670 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/tps65010.ko 2023-11-08T08:03:31.670 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/tps6507x.ko 2023-11-08T08:03:31.671 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/viperboard.ko 2023-11-08T08:03:31.671 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/vx855.ko 2023-11-08T08:03:31.671 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/wl1273-core.ko 2023-11-08T08:03:31.671 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/wm8994.ko 2023-11-08T08:03:31.671 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ 2023-11-08T08:03:31.672 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-11-08T08:03:31.672 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 48489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ad525x_dpot.ko 2023-11-08T08:03:31.672 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/altera-stapl/ 2023-11-08T08:03:31.672 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 46585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-08T08:03:31.672 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/apds9802als.ko 2023-11-08T08:03:31.673 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/apds990x.ko 2023-11-08T08:03:31.673 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 45593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/bh1770glc.ko 2023-11-08T08:03:31.673 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/c2port/ 2023-11-08T08:03:31.673 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-11-08T08:03:31.674 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/c2port/core.ko 2023-11-08T08:03:31.674 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/cb710/ 2023-11-08T08:03:31.674 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/cb710/cb710.ko 2023-11-08T08:03:31.674 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ds1682.ko 2023-11-08T08:03:31.674 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/dummy-irq.ko 2023-11-08T08:03:31.675 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/echo/ 2023-11-08T08:03:31.675 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/echo/echo.ko 2023-11-08T08:03:31.675 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/eeprom/ 2023-11-08T08:03:31.675 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/eeprom/at24.ko 2023-11-08T08:03:31.675 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-08T08:03:31.676 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-08T08:03:31.676 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/eeprom/max6875.ko 2023-11-08T08:03:31.676 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/enclosure.ko 2023-11-08T08:03:31.676 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/genwqe/ 2023-11-08T08:03:31.676 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 152201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-11-08T08:03:31.677 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/hmc6352.ko 2023-11-08T08:03:31.677 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/hpilo.ko 2023-11-08T08:03:31.677 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ibmasm/ 2023-11-08T08:03:31.677 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 72129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-11-08T08:03:31.677 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ics932s401.ko 2023-11-08T08:03:31.678 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/isl29003.ko 2023-11-08T08:03:31.678 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/isl29020.ko 2023-11-08T08:03:31.678 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/lis3lv02d/ 2023-11-08T08:03:31.678 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-08T08:03:31.679 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-08T08:03:31.679 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/mei/ 2023-11-08T08:03:31.679 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 125209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/mei/mei-me.ko 2023-11-08T08:03:31.679 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 56521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/mei/mei-txe.ko 2023-11-08T08:03:31.679 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 310113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/mei/mei.ko 2023-11-08T08:03:31.680 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/phantom.ko 2023-11-08T08:03:31.680 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ti-st/ 2023-11-08T08:03:31.680 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 74265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ti-st/st_drv.ko 2023-11-08T08:03:31.680 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/tifm_7xx1.ko 2023-11-08T08:03:31.680 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/tifm_core.ko 2023-11-08T08:03:31.681 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/tsl2550.ko 2023-11-08T08:03:31.681 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 48697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/vmw_balloon.ko 2023-11-08T08:03:31.681 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/vmw_vmci/ 2023-11-08T08:03:31.681 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 183609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-08T08:03:31.681 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ 2023-11-08T08:03:31.682 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/bonding/ 2023-11-08T08:03:31.682 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 490129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/bonding/bonding.ko 2023-11-08T08:03:31.682 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/ 2023-11-08T08:03:31.682 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/b53/ 2023-11-08T08:03:31.682 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 106033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-08T08:03:31.683 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-08T08:03:31.683 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-08T08:03:31.683 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/mv88e6xxx/ 2023-11-08T08:03:31.683 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 453737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-08T08:03:31.683 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dummy.ko 2023-11-08T08:03:31.684 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/eql.ko 2023-11-08T08:03:31.684 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/ 2023-11-08T08:03:31.684 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/3com/ 2023-11-08T08:03:31.684 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 45785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-08T08:03:31.684 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-08T08:03:31.685 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 130553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-08T08:03:31.685 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 68849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-08T08:03:31.685 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/8390/ 2023-11-08T08:03:31.685 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-08T08:03:31.685 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 59689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-08T08:03:31.686 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-08T08:03:31.686 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 99113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-08T08:03:31.686 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/adaptec/ 2023-11-08T08:03:31.686 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 54521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-08T08:03:31.687 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/agere/ 2023-11-08T08:03:31.687 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 69201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-08T08:03:31.687 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/alteon/ 2023-11-08T08:03:31.687 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 59585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-08T08:03:31.687 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/altera/ 2023-11-08T08:03:31.688 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-08T08:03:31.688 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/amd/ 2023-11-08T08:03:31.688 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 54609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-08T08:03:31.688 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-08T08:03:31.688 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 87297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-08T08:03:31.689 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/ 2023-11-08T08:03:31.689 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/alx/ 2023-11-08T08:03:31.689 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 97337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-08T08:03:31.689 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-11-08T08:03:31.689 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 104673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-08T08:03:31.690 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-11-08T08:03:31.690 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 90257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-08T08:03:31.690 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atlx/ 2023-11-08T08:03:31.690 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 81569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-08T08:03:31.690 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 69729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-08T08:03:31.691 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/ 2023-11-08T08:03:31.691 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 78401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-08T08:03:31.692 INFO:teuthology.orchestra.run.smithi040.stderr:done 2023-11-08T08:03:31.754 DEBUG:teuthology.orchestra.run.smithi040:> dpkg-deb -c /tmp/linux-image.deb 2023-11-08T08:03:31.769 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./ 2023-11-08T08:03:31.769 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./boot/ 2023-11-08T08:03:31.769 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 4415097 2023-11-07 03:19 ./boot/System.map-6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:31.789 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 169233 2023-11-07 03:19 ./boot/config-6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:31.790 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11352928 2023-11-07 03:19 ./boot/vmlinuz-6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:31.805 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 176761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-08T08:03:31.805 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-11-08T08:03:31.805 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1421497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-08T08:03:31.805 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 128569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-08T08:03:31.805 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/genet/ 2023-11-08T08:03:31.806 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 132089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-08T08:03:31.806 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 312081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-08T08:03:31.806 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/brocade/ 2023-11-08T08:03:31.806 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/brocade/bna/ 2023-11-08T08:03:31.807 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 323513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-08T08:03:31.807 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/ 2023-11-08T08:03:31.807 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-11-08T08:03:31.807 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 193497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-08T08:03:31.807 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-11-08T08:03:31.808 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 347417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-08T08:03:31.808 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-11-08T08:03:31.808 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1044433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-08T08:03:31.808 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-11-08T08:03:31.808 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 166193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-08T08:03:31.809 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-11-08T08:03:31.809 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-08T08:03:31.809 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/cisco/ 2023-11-08T08:03:31.809 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/cisco/enic/ 2023-11-08T08:03:31.810 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 204345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-08T08:03:31.810 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/ 2023-11-08T08:03:31.810 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/ 2023-11-08T08:03:31.810 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 69449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-08T08:03:31.810 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 61489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-08T08:03:31.811 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 141225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-08T08:03:31.811 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-08T08:03:31.811 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 62817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-08T08:03:31.811 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-08T08:03:31.812 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dlink/ 2023-11-08T08:03:31.812 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 60873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-08T08:03:31.812 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 62833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-08T08:03:31.812 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 45721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dnet.ko 2023-11-08T08:03:31.812 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/ec_bhf.ko 2023-11-08T08:03:31.813 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/emulex/ 2023-11-08T08:03:31.813 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/emulex/benet/ 2023-11-08T08:03:31.813 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 281137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-08T08:03:31.813 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/ethoc.ko 2023-11-08T08:03:31.813 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/fealnx.ko 2023-11-08T08:03:31.814 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/fujitsu/ 2023-11-08T08:03:31.814 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 45345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-11-08T08:03:31.814 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/ 2023-11-08T08:03:31.814 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 90465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-08T08:03:31.815 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/e1000/ 2023-11-08T08:03:31.815 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 295577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-08T08:03:31.815 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/e1000e/ 2023-11-08T08:03:31.815 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 584497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-08T08:03:31.815 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/fm10k/ 2023-11-08T08:03:31.816 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 246209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-08T08:03:31.816 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/i40e/ 2023-11-08T08:03:31.816 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 925697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-08T08:03:31.816 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/iavf/ 2023-11-08T08:03:31.816 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 302849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-08T08:03:31.817 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/igb/ 2023-11-08T08:03:31.817 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 514681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-08T08:03:31.817 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/igbvf/ 2023-11-08T08:03:31.817 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 93593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-08T08:03:31.817 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-11-08T08:03:31.818 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 734305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-08T08:03:31.818 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-11-08T08:03:31.818 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 151769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-08T08:03:31.818 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 86921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/jme.ko 2023-11-08T08:03:31.819 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/marvell/ 2023-11-08T08:03:31.819 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-08T08:03:31.819 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 102289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-08T08:03:31.819 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 122649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-08T08:03:31.819 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/ 2023-11-08T08:03:31.820 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-11-08T08:03:31.820 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 727601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-08T08:03:31.820 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 278937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-08T08:03:31.820 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-11-08T08:03:31.821 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-11-08T08:03:31.821 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1113993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-08T08:03:31.821 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/micrel/ 2023-11-08T08:03:31.821 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 53105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-11-08T08:03:31.821 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-11-08T08:03:31.822 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-11-08T08:03:31.822 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 95665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-08T08:03:31.822 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/myricom/ 2023-11-08T08:03:31.822 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-11-08T08:03:31.822 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 108649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-08T08:03:31.823 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/natsemi/ 2023-11-08T08:03:31.823 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 73441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-08T08:03:31.823 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-08T08:03:31.823 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/neterion/ 2023-11-08T08:03:31.823 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 150905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-08T08:03:31.824 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/nvidia/ 2023-11-08T08:03:31.824 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 136721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-08T08:03:31.824 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/packetengines/ 2023-11-08T08:03:31.824 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-08T08:03:31.824 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-08T08:03:31.825 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/ 2023-11-08T08:03:31.825 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-11-08T08:03:31.825 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 227897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-08T08:03:31.825 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 87081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-08T08:03:31.825 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-11-08T08:03:31.826 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 672049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-08T08:03:31.826 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/rdc/ 2023-11-08T08:03:31.826 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-08T08:03:31.826 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/realtek/ 2023-11-08T08:03:31.826 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 67385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-08T08:03:31.827 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 83985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-08T08:03:31.827 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-08T08:03:31.827 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 189801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-08T08:03:31.827 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/samsung/ 2023-11-08T08:03:31.827 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-11-08T08:03:31.828 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 104241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-11-08T08:03:31.828 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sfc/ 2023-11-08T08:03:31.828 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1053921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-08T08:03:31.828 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/silan/ 2023-11-08T08:03:31.829 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-08T08:03:31.829 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sis/ 2023-11-08T08:03:31.829 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 54817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-08T08:03:31.829 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 63337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-08T08:03:31.829 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/smsc/ 2023-11-08T08:03:31.830 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 61601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-08T08:03:31.830 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 67193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-08T08:03:31.830 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 73409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-08T08:03:31.830 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 68177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-08T08:03:31.831 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/ 2023-11-08T08:03:31.831 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-11-08T08:03:31.831 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-11-08T08:03:31.831 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-08T08:03:31.831 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-11-08T08:03:31.832 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 486961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-08T08:03:31.832 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sun/ 2023-11-08T08:03:31.832 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 103337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-08T08:03:31.832 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 166161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-08T08:03:31.833 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 66329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-08T08:03:31.833 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 78801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-08T08:03:31.833 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/tehuti/ 2023-11-08T08:03:31.833 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-08T08:03:31.833 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/ti/ 2023-11-08T08:03:31.834 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 86529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-08T08:03:31.834 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/via/ 2023-11-08T08:03:31.834 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 80273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-08T08:03:31.835 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 83633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-08T08:03:31.835 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/wiznet/ 2023-11-08T08:03:31.835 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 46241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-08T08:03:31.835 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-08T08:03:31.835 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/xircom/ 2023-11-08T08:03:31.836 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 60057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-08T08:03:31.836 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/fddi/ 2023-11-08T08:03:31.836 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/fddi/defxx.ko 2023-11-08T08:03:31.836 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/fddi/skfp/ 2023-11-08T08:03:31.837 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 216985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/fddi/skfp/skfp.ko 2023-11-08T08:03:31.837 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 77345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/geneve.ko 2023-11-08T08:03:31.837 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/hyperv/ 2023-11-08T08:03:31.837 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 235521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-08T08:03:31.837 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ieee802154/ 2023-11-08T08:03:31.838 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-08T08:03:31.838 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ifb.ko 2023-11-08T08:03:31.838 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ipvlan/ 2023-11-08T08:03:31.838 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 83769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-08T08:03:31.838 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 62209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/macvlan.ko 2023-11-08T08:03:31.839 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/macvtap.ko 2023-11-08T08:03:31.839 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio/ 2023-11-08T08:03:31.839 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-08T08:03:31.839 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-08T08:03:31.840 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio/mdio-gpio.ko 2023-11-08T08:03:31.840 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-08T08:03:31.840 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio.ko 2023-11-08T08:03:31.840 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mii.ko 2023-11-08T08:03:31.840 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/netconsole.ko 2023-11-08T08:03:31.841 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/nlmon.ko 2023-11-08T08:03:31.841 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/pcs/ 2023-11-08T08:03:31.841 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-08T08:03:31.841 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-08T08:03:31.841 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/ 2023-11-08T08:03:31.842 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/amd.ko 2023-11-08T08:03:31.842 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-08T08:03:31.842 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-08T08:03:31.842 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/bcm87xx.ko 2023-11-08T08:03:31.843 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/broadcom.ko 2023-11-08T08:03:31.843 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/cicada.ko 2023-11-08T08:03:31.843 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/davicom.ko 2023-11-08T08:03:31.843 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/et1011c.ko 2023-11-08T08:03:31.843 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/icplus.ko 2023-11-08T08:03:31.844 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/lxt.ko 2023-11-08T08:03:31.844 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 82049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/marvell.ko 2023-11-08T08:03:31.844 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 76177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/micrel.ko 2023-11-08T08:03:31.844 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/national.ko 2023-11-08T08:03:31.845 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 97857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/phylink.ko 2023-11-08T08:03:31.845 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/qsemi.ko 2023-11-08T08:03:31.845 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/realtek.ko 2023-11-08T08:03:31.845 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/smsc.ko 2023-11-08T08:03:31.846 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/ste10Xp.ko 2023-11-08T08:03:31.846 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/vitesse.ko 2023-11-08T08:03:31.846 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/plip/ 2023-11-08T08:03:31.846 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/plip/plip.ko 2023-11-08T08:03:31.847 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/ 2023-11-08T08:03:31.847 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-08T08:03:31.847 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/ppp_async.ko 2023-11-08T08:03:31.847 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-08T08:03:31.847 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-08T08:03:31.848 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-08T08:03:31.848 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/pppoe.ko 2023-11-08T08:03:31.848 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/pppox.ko 2023-11-08T08:03:31.848 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/pptp.ko 2023-11-08T08:03:31.849 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/sb1000.ko 2023-11-08T08:03:31.849 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/slip/ 2023-11-08T08:03:31.849 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/slip/slip.ko 2023-11-08T08:03:31.849 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/sungem_phy.ko 2023-11-08T08:03:31.850 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 64385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/tap.ko 2023-11-08T08:03:31.850 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/ 2023-11-08T08:03:31.850 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 106169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team.ko 2023-11-08T08:03:31.850 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-08T08:03:31.850 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-08T08:03:31.851 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-08T08:03:31.851 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team_mode_random.ko 2023-11-08T08:03:31.851 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-08T08:03:31.851 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 87017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/veth.ko 2023-11-08T08:03:31.852 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/vmxnet3/ 2023-11-08T08:03:31.852 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 144297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-08T08:03:31.852 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/vxlan/ 2023-11-08T08:03:31.852 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 297329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/vxlan/vxlan.ko 2023-11-08T08:03:31.852 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/xen-netback/ 2023-11-08T08:03:31.853 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 144649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-08T08:03:31.853 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ntb/ 2023-11-08T08:03:31.853 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ntb/ntb.ko 2023-11-08T08:03:31.853 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/nvme/ 2023-11-08T08:03:31.853 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/nvme/host/ 2023-11-08T08:03:31.854 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 264937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/nvme/host/nvme-core.ko 2023-11-08T08:03:31.854 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 102209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/nvme/host/nvme.ko 2023-11-08T08:03:31.854 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/parport/ 2023-11-08T08:03:31.854 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 133841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/parport/parport.ko 2023-11-08T08:03:31.855 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/parport/parport_cs.ko 2023-11-08T08:03:31.855 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 84465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/parport/parport_pc.ko 2023-11-08T08:03:31.855 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/parport/parport_serial.ko 2023-11-08T08:03:31.855 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/ 2023-11-08T08:03:31.855 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/hotplug/ 2023-11-08T08:03:31.856 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-08T08:03:31.856 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-11-08T08:03:31.856 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-11-08T08:03:31.856 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/pci-stub.ko 2023-11-08T08:03:31.857 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 46529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/xen-pcifront.ko 2023-11-08T08:03:31.857 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/ 2023-11-08T08:03:31.857 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/i82092.ko 2023-11-08T08:03:31.857 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 145809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/pcmcia.ko 2023-11-08T08:03:31.857 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 62137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/pcmcia_core.ko 2023-11-08T08:03:31.858 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-11-08T08:03:31.858 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/pd6729.ko 2023-11-08T08:03:31.858 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 112889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-08T08:03:31.858 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/phy/ 2023-11-08T08:03:31.858 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/phy/broadcom/ 2023-11-08T08:03:31.859 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-11-08T08:03:31.859 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/phy/samsung/ 2023-11-08T08:03:31.859 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-11-08T08:03:31.859 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pinctrl/ 2023-11-08T08:03:31.860 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pinctrl/intel/ 2023-11-08T08:03:31.860 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 70153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-08T08:03:31.860 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/ 2023-11-08T08:03:31.860 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/ 2023-11-08T08:03:31.860 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-11-08T08:03:31.861 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_ec.ko 2023-11-08T08:03:31.861 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-11-08T08:03:31.861 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-11-08T08:03:31.861 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-11-08T08:03:31.862 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-11-08T08:03:31.862 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-11-08T08:03:31.862 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/ 2023-11-08T08:03:31.862 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 67097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-08T08:03:31.862 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/acerhdf.ko 2023-11-08T08:03:31.863 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-08T08:03:31.863 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 59665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-08T08:03:31.863 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-08T08:03:31.863 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 127569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-08T08:03:31.864 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-08T08:03:31.864 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-08T08:03:31.864 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-08T08:03:31.864 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-08T08:03:31.864 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-08T08:03:31.865 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/hdaps.ko 2023-11-08T08:03:31.865 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/ibm_rtl.ko 2023-11-08T08:03:31.865 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/intel/ 2023-11-08T08:03:31.865 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-08T08:03:31.865 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-11-08T08:03:31.866 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/intel_ips.ko 2023-11-08T08:03:31.866 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-08T08:03:31.866 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-08T08:03:31.866 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-08T08:03:31.867 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-08T08:03:31.867 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-08T08:03:31.867 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-08T08:03:31.867 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-08T08:03:31.867 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-08T08:03:31.868 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-08T08:03:31.868 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 57569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/wmi.ko 2023-11-08T08:03:31.868 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/ 2023-11-08T08:03:31.868 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/ 2023-11-08T08:03:31.868 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/88pm860x_battery.ko 2023-11-08T08:03:31.869 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/88pm860x_charger.ko 2023-11-08T08:03:31.869 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/bq2415x_charger.ko 2023-11-08T08:03:31.869 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 54081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/bq24190_charger.ko 2023-11-08T08:03:31.869 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/bq24735-charger.ko 2023-11-08T08:03:31.870 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-11-08T08:03:31.870 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/da9030_battery.ko 2023-11-08T08:03:31.870 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/da9052-battery.ko 2023-11-08T08:03:31.870 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/ds2760_battery.ko 2023-11-08T08:03:31.870 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/ds2780_battery.ko 2023-11-08T08:03:31.871 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/ds2781_battery.ko 2023-11-08T08:03:31.871 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/ds2782_battery.ko 2023-11-08T08:03:31.871 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/gpio-charger.ko 2023-11-08T08:03:31.871 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/isp1704_charger.ko 2023-11-08T08:03:31.871 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/lp8727_charger.ko 2023-11-08T08:03:31.872 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/max14577_charger.ko 2023-11-08T08:03:31.872 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/max17040_battery.ko 2023-11-08T08:03:31.872 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/max17042_battery.ko 2023-11-08T08:03:31.872 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/max8903_charger.ko 2023-11-08T08:03:31.873 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/max8925_power.ko 2023-11-08T08:03:31.873 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/pcf50633-charger.ko 2023-11-08T08:03:31.873 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/sbs-battery.ko 2023-11-08T08:03:31.873 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/test_power.ko 2023-11-08T08:03:31.873 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/tps65090-charger.ko 2023-11-08T08:03:31.874 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/wm831x_backup.ko 2023-11-08T08:03:31.874 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/wm831x_power.ko 2023-11-08T08:03:31.874 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/wm8350_power.ko 2023-11-08T08:03:31.874 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/powercap/ 2023-11-08T08:03:31.874 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 57729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-08T08:03:31.875 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-08T08:03:31.875 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/ 2023-11-08T08:03:31.875 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/clients/ 2023-11-08T08:03:31.875 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-08T08:03:31.876 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-08T08:03:31.876 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/clients/pps_parport.ko 2023-11-08T08:03:31.876 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/pps_core.ko 2023-11-08T08:03:31.876 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ptp/ 2023-11-08T08:03:31.876 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 66385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ptp/ptp.ko 2023-11-08T08:03:31.877 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ptp/ptp_kvm.ko 2023-11-08T08:03:31.877 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/ 2023-11-08T08:03:31.877 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-88pm80x.ko 2023-11-08T08:03:31.877 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-88pm860x.ko 2023-11-08T08:03:31.878 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-08T08:03:31.878 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-da9052.ko 2023-11-08T08:03:31.878 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-da9055.ko 2023-11-08T08:03:31.878 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-da9063.ko 2023-11-08T08:03:31.878 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-08T08:03:31.879 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 68577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-08T08:03:31.879 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-08T08:03:31.879 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-08T08:03:31.879 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-08T08:03:31.880 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-08T08:03:31.880 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-08T08:03:31.880 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-08T08:03:31.880 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-08T08:03:31.880 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-em3027.ko 2023-11-08T08:03:31.881 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-08T08:03:31.881 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-08T08:03:31.881 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-08T08:03:31.881 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-lp8788.ko 2023-11-08T08:03:31.882 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-08T08:03:31.882 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-08T08:03:31.882 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-08T08:03:31.882 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-m48t86.ko 2023-11-08T08:03:31.882 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-max6900.ko 2023-11-08T08:03:31.883 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-max8907.ko 2023-11-08T08:03:31.883 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-max8925.ko 2023-11-08T08:03:31.883 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-max8997.ko 2023-11-08T08:03:31.883 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-max8998.ko 2023-11-08T08:03:31.883 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-11-08T08:03:31.884 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-08T08:03:31.884 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-palmas.ko 2023-11-08T08:03:31.884 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 46633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-08T08:03:31.884 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf50633.ko 2023-11-08T08:03:31.885 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-08T08:03:31.885 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-08T08:03:31.885 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-08T08:03:31.885 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-08T08:03:31.885 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rc5t583.ko 2023-11-08T08:03:31.886 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-08T08:03:31.886 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-08T08:03:31.886 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-08T08:03:31.886 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-08T08:03:31.886 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-08T08:03:31.887 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-s35390a.ko 2023-11-08T08:03:31.887 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-08T08:03:31.887 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-tps6586x.ko 2023-11-08T08:03:31.887 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-tps65910.ko 2023-11-08T08:03:31.888 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-wm831x.ko 2023-11-08T08:03:31.888 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-wm8350.ko 2023-11-08T08:03:31.888 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-x1205.ko 2023-11-08T08:03:31.888 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ 2023-11-08T08:03:31.888 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 77617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/3w-9xxx.ko 2023-11-08T08:03:31.889 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 57665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/3w-sas.ko 2023-11-08T08:03:31.889 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/3w-xxxx.ko 2023-11-08T08:03:31.889 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 135185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/BusLogic.ko 2023-11-08T08:03:31.889 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/a100u2w.ko 2023-11-08T08:03:31.889 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aacraid/ 2023-11-08T08:03:31.890 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 272305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-08T08:03:31.890 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 100617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/advansys.ko 2023-11-08T08:03:31.890 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aic7xxx/ 2023-11-08T08:03:31.890 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 356265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-08T08:03:31.891 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 295801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-08T08:03:31.891 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aic94xx/ 2023-11-08T08:03:31.891 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 189185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-11-08T08:03:31.891 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/am53c974.ko 2023-11-08T08:03:31.891 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/arcmsr/ 2023-11-08T08:03:31.892 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 106881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-08T08:03:31.892 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/atp870u.ko 2023-11-08T08:03:31.892 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/be2iscsi/ 2023-11-08T08:03:31.892 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 246985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-08T08:03:31.893 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bfa/ 2023-11-08T08:03:31.893 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1171945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bfa/bfa.ko 2023-11-08T08:03:31.893 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bnx2fc/ 2023-11-08T08:03:31.893 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 237273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-08T08:03:31.893 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bnx2i/ 2023-11-08T08:03:31.894 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 115369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-08T08:03:31.894 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ch.ko 2023-11-08T08:03:31.894 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/csiostor/ 2023-11-08T08:03:31.894 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 308921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-08T08:03:31.894 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/ 2023-11-08T08:03:31.895 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-11-08T08:03:31.895 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 76145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-08T08:03:31.895 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-11-08T08:03:31.895 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 101449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-08T08:03:31.895 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 146689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-08T08:03:31.896 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 75553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/dc395x.ko 2023-11-08T08:03:31.896 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/dmx3191d.ko 2023-11-08T08:03:31.896 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/esas2r/ 2023-11-08T08:03:31.896 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 212425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-08T08:03:31.896 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 60505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/esp_scsi.ko 2023-11-08T08:03:31.897 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fcoe/ 2023-11-08T08:03:31.897 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 78209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-08T08:03:31.897 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 158505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-08T08:03:31.897 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fdomain.ko 2023-11-08T08:03:31.898 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fnic/ 2023-11-08T08:03:31.898 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 263489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fnic/fnic.ko 2023-11-08T08:03:31.898 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 201817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/hpsa.ko 2023-11-08T08:03:31.898 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/hptiop.ko 2023-11-08T08:03:31.898 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/hv_storvsc.ko 2023-11-08T08:03:31.899 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/imm.ko 2023-11-08T08:03:31.899 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/initio.ko 2023-11-08T08:03:31.899 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 266537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ipr.ko 2023-11-08T08:03:31.899 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 101881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ips.ko 2023-11-08T08:03:31.899 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/isci/ 2023-11-08T08:03:31.900 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 317153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/isci/isci.ko 2023-11-08T08:03:31.900 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-08T08:03:31.900 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 52961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-08T08:03:31.900 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libfc/ 2023-11-08T08:03:31.900 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 314865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libfc/libfc.ko 2023-11-08T08:03:31.901 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 145585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libiscsi.ko 2023-11-08T08:03:31.901 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 53521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-08T08:03:31.901 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libsas/ 2023-11-08T08:03:31.901 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 216665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libsas/libsas.ko 2023-11-08T08:03:31.902 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/lpfc/ 2023-11-08T08:03:31.902 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 2197249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-08T08:03:31.902 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/megaraid/ 2023-11-08T08:03:31.902 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 73569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-08T08:03:31.902 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-08T08:03:31.903 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 310617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-08T08:03:31.903 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 84721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/megaraid.ko 2023-11-08T08:03:31.903 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/mpt3sas/ 2023-11-08T08:03:31.903 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 659777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-08T08:03:31.903 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/mvsas/ 2023-11-08T08:03:31.904 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 119561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-08T08:03:31.904 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 78129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/mvumi.ko 2023-11-08T08:03:31.904 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pcmcia/ 2023-11-08T08:03:31.904 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 81625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-11-08T08:03:31.904 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-11-08T08:03:31.905 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-11-08T08:03:31.905 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-11-08T08:03:31.905 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pm8001/ 2023-11-08T08:03:31.905 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 364105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-08T08:03:31.906 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 123617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pmcraid.ko 2023-11-08T08:03:31.906 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ppa.ko 2023-11-08T08:03:31.906 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla1280.ko 2023-11-08T08:03:31.906 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla2xxx/ 2023-11-08T08:03:31.907 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1876625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-08T08:03:31.907 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 85289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-08T08:03:31.907 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla4xxx/ 2023-11-08T08:03:31.907 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 559033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-08T08:03:31.907 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qlogicfas408.ko 2023-11-08T08:03:31.908 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/raid_class.ko 2023-11-08T08:03:31.908 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 238825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_debug.ko 2023-11-08T08:03:31.908 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 173937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-08T08:03:31.908 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 279313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-08T08:03:31.909 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 97729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-08T08:03:31.909 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 73881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-08T08:03:31.909 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-08T08:03:31.909 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ses.ko 2023-11-08T08:03:31.909 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 115889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/st.ko 2023-11-08T08:03:31.910 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 46825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/stex.ko 2023-11-08T08:03:31.910 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/sym53c8xx_2/ 2023-11-08T08:03:31.910 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 162121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-08T08:03:31.910 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/virtio_scsi.ko 2023-11-08T08:03:31.911 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-08T08:03:31.911 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/wd719x.ko 2023-11-08T08:03:31.911 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/xen-scsifront.ko 2023-11-08T08:03:31.911 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ssb/ 2023-11-08T08:03:31.911 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 129441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ssb/ssb.ko 2023-11-08T08:03:31.912 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/ 2023-11-08T08:03:31.912 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/iscsi/ 2023-11-08T08:03:31.912 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 736601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-08T08:03:31.912 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/loopback/ 2023-11-08T08:03:31.913 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-08T08:03:31.913 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/target_core_file.ko 2023-11-08T08:03:31.913 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/target_core_iblock.ko 2023-11-08T08:03:31.913 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 930321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/target_core_mod.ko 2023-11-08T08:03:31.913 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/target_core_pscsi.ko 2023-11-08T08:03:31.914 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 122521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/target_core_user.ko 2023-11-08T08:03:31.914 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/tcm_fc/ 2023-11-08T08:03:31.914 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 75593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-08T08:03:31.914 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/ 2023-11-08T08:03:31.915 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/ 2023-11-08T08:03:31.915 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/ 2023-11-08T08:03:31.915 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-08T08:03:31.915 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-08T08:03:31.915 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-08T08:03:31.916 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-08T08:03:31.916 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-08T08:03:31.916 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-08T08:03:31.916 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-08T08:03:31.917 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-08T08:03:31.917 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-08T08:03:31.917 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-08T08:03:31.917 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-08T08:03:31.917 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-08T08:03:31.918 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-11-08T08:03:31.918 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-08T08:03:31.918 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-08T08:03:31.918 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-08T08:03:31.919 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/ 2023-11-08T08:03:31.919 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/ipwireless/ 2023-11-08T08:03:31.919 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 78425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-08T08:03:31.919 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 57865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/moxa.ko 2023-11-08T08:03:31.920 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 50953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/mxser.ko 2023-11-08T08:03:31.920 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 108601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/n_gsm.ko 2023-11-08T08:03:31.920 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/n_hdlc.ko 2023-11-08T08:03:31.920 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 67929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/nozomi.ko 2023-11-08T08:03:31.920 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/ 2023-11-08T08:03:31.921 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/8250/ 2023-11-08T08:03:31.921 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-11-08T08:03:31.921 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 59497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-08T08:03:31.921 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-11-08T08:03:31.922 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/altera_uart.ko 2023-11-08T08:03:31.922 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/arc_uart.ko 2023-11-08T08:03:31.922 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 72425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-11-08T08:03:31.922 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/jsm/ 2023-11-08T08:03:31.922 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 103073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-08T08:03:31.923 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/rp2.ko 2023-11-08T08:03:31.923 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 52817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/sc16is7xx.ko 2023-11-08T08:03:31.923 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/ 2023-11-08T08:03:31.923 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/core/ 2023-11-08T08:03:31.924 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 531265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-08T08:03:31.924 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/host/ 2023-11-08T08:03:31.924 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-08T08:03:31.924 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-11-08T08:03:31.924 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/ 2023-11-08T08:03:31.925 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio.ko 2023-11-08T08:03:31.925 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_aec.ko 2023-11-08T08:03:31.925 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_cif.ko 2023-11-08T08:03:31.925 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_dmem_genirq.ko 2023-11-08T08:03:31.926 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_mf624.ko 2023-11-08T08:03:31.926 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_netx.ko 2023-11-08T08:03:31.926 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_pci_generic.ko 2023-11-08T08:03:31.926 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-11-08T08:03:31.927 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_sercos3.ko 2023-11-08T08:03:31.927 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/ 2023-11-08T08:03:31.927 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/c67x00/ 2023-11-08T08:03:31.927 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 73297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/c67x00/c67x00.ko 2023-11-08T08:03:31.927 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/chipidea/ 2023-11-08T08:03:31.928 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 144377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-11-08T08:03:31.928 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-11-08T08:03:31.928 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-11-08T08:03:31.928 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-11-08T08:03:31.928 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/class/ 2023-11-08T08:03:31.929 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 87089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/class/cdc-acm.ko 2023-11-08T08:03:31.929 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-08T08:03:31.929 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/class/usblp.ko 2023-11-08T08:03:31.929 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 89857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/class/usbtmc.ko 2023-11-08T08:03:31.929 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/common/ 2023-11-08T08:03:31.930 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/common/ulpi.ko 2023-11-08T08:03:31.930 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc2/ 2023-11-08T08:03:31.930 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-11-08T08:03:31.930 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc3/ 2023-11-08T08:03:31.931 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-11-08T08:03:31.931 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-11-08T08:03:31.931 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 444553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc3/dwc3.ko 2023-11-08T08:03:31.931 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/ 2023-11-08T08:03:31.931 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/ 2023-11-08T08:03:31.932 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 60177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/u_ether.ko 2023-11-08T08:03:31.932 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/u_serial.ko 2023-11-08T08:03:31.932 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-11-08T08:03:31.932 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-11-08T08:03:31.932 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-11-08T08:03:31.933 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-11-08T08:03:31.933 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 92489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-11-08T08:03:31.933 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-11-08T08:03:31.933 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 122081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-11-08T08:03:31.934 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 57617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-11-08T08:03:31.934 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-11-08T08:03:31.934 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-11-08T08:03:31.934 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 57729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-11-08T08:03:31.934 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 85609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-11-08T08:03:31.935 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-11-08T08:03:31.935 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 58113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-11-08T08:03:31.935 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 67337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-11-08T08:03:31.935 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/ 2023-11-08T08:03:31.935 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-11-08T08:03:31.936 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-11-08T08:03:31.936 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-11-08T08:03:31.936 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-11-08T08:03:31.936 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-11-08T08:03:31.936 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-11-08T08:03:31.937 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-11-08T08:03:31.937 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-11-08T08:03:31.937 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-11-08T08:03:31.937 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-11-08T08:03:31.938 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-11-08T08:03:31.938 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-11-08T08:03:31.938 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 62185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-11-08T08:03:31.938 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-11-08T08:03:31.938 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 161225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/libcomposite.ko 2023-11-08T08:03:31.939 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/ 2023-11-08T08:03:31.939 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-11-08T08:03:31.939 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/bdc/ 2023-11-08T08:03:31.939 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 157681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-11-08T08:03:31.940 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-11-08T08:03:31.940 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 52721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-11-08T08:03:31.940 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 69705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-11-08T08:03:31.940 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 56033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-11-08T08:03:31.940 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 76057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/net2272.ko 2023-11-08T08:03:31.941 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 75073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/net2280.ko 2023-11-08T08:03:31.941 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 73169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-11-08T08:03:31.941 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 65809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-11-08T08:03:31.941 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 66097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-11-08T08:03:31.941 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 81505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-11-08T08:03:31.942 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 156073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-11-08T08:03:31.942 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/ 2023-11-08T08:03:31.942 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/bcma-hcd.ko 2023-11-08T08:03:31.942 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 80113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/isp116x-hcd.ko 2023-11-08T08:03:31.943 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 87353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-11-08T08:03:31.943 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 78873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-11-08T08:03:31.943 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-08T08:03:31.943 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/sl811_cs.ko 2023-11-08T08:03:31.943 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/ssb-hcd.ko 2023-11-08T08:03:31.944 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-11-08T08:03:31.944 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/image/ 2023-11-08T08:03:31.944 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/image/mdc800.ko 2023-11-08T08:03:31.944 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/image/microtek.ko 2023-11-08T08:03:31.944 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/ 2023-11-08T08:03:31.945 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/adutux.ko 2023-11-08T08:03:31.945 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/appledisplay.ko 2023-11-08T08:03:31.945 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-11-08T08:03:31.945 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/cytherm.ko 2023-11-08T08:03:31.946 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/ehset.ko 2023-11-08T08:03:31.946 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/emi26.ko 2023-11-08T08:03:31.946 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/emi62.ko 2023-11-08T08:03:31.946 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/ezusb.ko 2023-11-08T08:03:31.946 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/idmouse.ko 2023-11-08T08:03:31.947 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/iowarrior.ko 2023-11-08T08:03:31.947 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-08T08:03:31.947 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/ldusb.ko 2023-11-08T08:03:31.947 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/legousbtower.ko 2023-11-08T08:03:31.947 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/lvstest.ko 2023-11-08T08:03:31.948 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/sisusbvga/ 2023-11-08T08:03:31.948 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 64841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-08T08:03:31.948 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-08T08:03:31.948 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/usb3503.ko 2023-11-08T08:03:31.948 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/usblcd.ko 2023-11-08T08:03:31.949 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-08T08:03:31.949 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 70873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/usbtest.ko 2023-11-08T08:03:31.949 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/uss720.ko 2023-11-08T08:03:31.949 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/yurex.ko 2023-11-08T08:03:31.949 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/mon/ 2023-11-08T08:03:31.950 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 72953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/mon/usbmon.ko 2023-11-08T08:03:31.950 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/musb/ 2023-11-08T08:03:31.950 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 301545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/musb/musb_hdrc.ko 2023-11-08T08:03:31.950 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/phy/ 2023-11-08T08:03:31.951 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/phy/phy-generic.ko 2023-11-08T08:03:31.951 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-11-08T08:03:31.951 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/phy/phy-isp1301.ko 2023-11-08T08:03:31.951 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/phy/phy-tahvo.ko 2023-11-08T08:03:31.951 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ 2023-11-08T08:03:31.952 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/aircable.ko 2023-11-08T08:03:31.952 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ark3116.ko 2023-11-08T08:03:31.952 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-08T08:03:31.952 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ch341.ko 2023-11-08T08:03:31.952 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 70257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/cp210x.ko 2023-11-08T08:03:31.953 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/cyberjack.ko 2023-11-08T08:03:31.953 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 58217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-08T08:03:31.953 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-08T08:03:31.953 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/empeg.ko 2023-11-08T08:03:31.953 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/f81232.ko 2023-11-08T08:03:31.954 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 175137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-08T08:03:31.954 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 53113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-08T08:03:31.954 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 137081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-08T08:03:31.954 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 124833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/io_ti.ko 2023-11-08T08:03:31.954 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 67769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ipaq.ko 2023-11-08T08:03:31.955 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ipw.ko 2023-11-08T08:03:31.955 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ir-usb.ko 2023-11-08T08:03:31.955 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 58233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-08T08:03:31.955 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 107521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/keyspan.ko 2023-11-08T08:03:31.956 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-08T08:03:31.956 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-08T08:03:31.956 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-08T08:03:31.956 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/mct_u232.ko 2023-11-08T08:03:31.956 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/metro-usb.ko 2023-11-08T08:03:31.957 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 66449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/mos7720.ko 2023-11-08T08:03:31.957 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 78217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/mos7840.ko 2023-11-08T08:03:31.957 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 56337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/mxuport.ko 2023-11-08T08:03:31.957 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/navman.ko 2023-11-08T08:03:31.957 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/omninet.ko 2023-11-08T08:03:31.958 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/opticon.ko 2023-11-08T08:03:31.958 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 190633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/option.ko 2023-11-08T08:03:31.958 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/oti6858.ko 2023-11-08T08:03:31.958 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 54353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/pl2303.ko 2023-11-08T08:03:31.958 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/qcaux.ko 2023-11-08T08:03:31.959 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/qcserial.ko 2023-11-08T08:03:31.959 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/quatech2.ko 2023-11-08T08:03:31.959 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/safe_serial.ko 2023-11-08T08:03:31.959 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 51625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/sierra.ko 2023-11-08T08:03:31.959 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-08T08:03:31.960 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ssu100.ko 2023-11-08T08:03:31.960 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/symbolserial.ko 2023-11-08T08:03:31.960 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 61489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-08T08:03:31.960 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-08T08:03:31.961 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/usb_debug.ko 2023-11-08T08:03:31.961 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-08T08:03:31.961 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 125609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/usbserial.ko 2023-11-08T08:03:31.961 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/visor.ko 2023-11-08T08:03:31.961 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/whiteheat.ko 2023-11-08T08:03:31.962 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/wishbone-serial.ko 2023-11-08T08:03:31.962 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-08T08:03:31.962 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ 2023-11-08T08:03:31.962 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 52913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/uas.ko 2023-11-08T08:03:31.963 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-08T08:03:31.963 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-08T08:03:31.963 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-08T08:03:31.963 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-08T08:03:31.963 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-08T08:03:31.964 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-08T08:03:31.964 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-08T08:03:31.964 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-karma.ko 2023-11-08T08:03:31.964 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-08T08:03:31.964 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-08T08:03:31.965 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-08T08:03:31.965 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-08T08:03:31.965 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-08T08:03:31.965 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 179969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/usb-storage.ko 2023-11-08T08:03:31.965 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/usbip/ 2023-11-08T08:03:31.966 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 67481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-08T08:03:31.966 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 72177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-08T08:03:31.966 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 107001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-08T08:03:31.966 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/ 2023-11-08T08:03:31.967 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/pci/ 2023-11-08T08:03:31.967 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 154297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-08T08:03:31.967 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-08T08:03:31.967 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 108737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/vfio.ko 2023-11-08T08:03:31.967 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 77609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-08T08:03:31.968 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vhost/ 2023-11-08T08:03:31.968 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 113873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vhost/vhost.ko 2023-11-08T08:03:31.968 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-08T08:03:31.968 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 58089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vhost/vhost_net.ko 2023-11-08T08:03:31.969 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 89249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vhost/vhost_scsi.ko 2023-11-08T08:03:31.969 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/ 2023-11-08T08:03:31.969 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/ 2023-11-08T08:03:31.969 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-11-08T08:03:31.969 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/aat2870_bl.ko 2023-11-08T08:03:31.970 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/adp5520_bl.ko 2023-11-08T08:03:31.970 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/adp8860_bl.ko 2023-11-08T08:03:31.970 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/adp8870_bl.ko 2023-11-08T08:03:31.970 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/apple_bl.ko 2023-11-08T08:03:31.970 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/as3711_bl.ko 2023-11-08T08:03:31.971 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/bd6107.ko 2023-11-08T08:03:31.971 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/da903x_bl.ko 2023-11-08T08:03:31.971 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/da9052_bl.ko 2023-11-08T08:03:31.971 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/gpio_backlight.ko 2023-11-08T08:03:31.972 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/kb3886_bl.ko 2023-11-08T08:03:31.972 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/lcd.ko 2023-11-08T08:03:31.972 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/lm3533_bl.ko 2023-11-08T08:03:31.972 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/lm3639_bl.ko 2023-11-08T08:03:31.972 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/lv5207lp.ko 2023-11-08T08:03:31.973 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/max8925_bl.ko 2023-11-08T08:03:31.973 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/pandora_bl.ko 2023-11-08T08:03:31.973 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-11-08T08:03:31.973 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-08T08:03:31.973 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/wm831x_bl.ko 2023-11-08T08:03:31.974 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/ 2023-11-08T08:03:31.974 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/masters/ 2023-11-08T08:03:31.974 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/masters/ds2482.ko 2023-11-08T08:03:31.974 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/masters/ds2490.ko 2023-11-08T08:03:31.975 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/masters/matrox_w1.ko 2023-11-08T08:03:31.975 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/masters/w1-gpio.ko 2023-11-08T08:03:31.975 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/ 2023-11-08T08:03:31.975 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-08T08:03:31.975 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-08T08:03:31.976 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-08T08:03:31.976 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-08T08:03:31.976 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-08T08:03:31.976 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-08T08:03:31.977 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-08T08:03:31.977 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-08T08:03:31.977 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-08T08:03:31.977 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-08T08:03:31.977 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 56425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-08T08:03:31.978 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 102673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/wire.ko 2023-11-08T08:03:31.978 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/ 2023-11-08T08:03:31.978 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/acquirewdt.ko 2023-11-08T08:03:31.978 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/advantechwdt.ko 2023-11-08T08:03:31.978 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-08T08:03:31.979 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-08T08:03:31.979 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/cpu5wdt.ko 2023-11-08T08:03:31.979 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/da9052_wdt.ko 2023-11-08T08:03:31.979 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/da9055_wdt.ko 2023-11-08T08:03:31.980 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/da9063_wdt.ko 2023-11-08T08:03:31.980 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/dw_wdt.ko 2023-11-08T08:03:31.980 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/eurotechwdt.ko 2023-11-08T08:03:31.980 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-08T08:03:31.980 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/hpwdt.ko 2023-11-08T08:03:31.981 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/i6300esb.ko 2023-11-08T08:03:31.981 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-08T08:03:31.981 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-08T08:03:31.981 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/ib700wdt.ko 2023-11-08T08:03:31.981 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/ibmasr.ko 2023-11-08T08:03:31.982 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-08T08:03:31.982 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-08T08:03:31.982 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/it87_wdt.ko 2023-11-08T08:03:31.982 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/kempld_wdt.ko 2023-11-08T08:03:31.983 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/machzwd.ko 2023-11-08T08:03:31.983 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/mena21_wdt.ko 2023-11-08T08:03:31.983 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-11-08T08:03:31.983 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/nv_tco.ko 2023-11-08T08:03:31.983 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-11-08T08:03:31.984 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/pc87413_wdt.ko 2023-11-08T08:03:31.984 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-08T08:03:31.984 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-08T08:03:31.984 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/retu_wdt.ko 2023-11-08T08:03:31.984 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-11-08T08:03:31.985 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-11-08T08:03:31.985 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-08T08:03:31.985 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sc1200wdt.ko 2023-11-08T08:03:31.985 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-08T08:03:31.986 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-11-08T08:03:31.986 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/softdog.ko 2023-11-08T08:03:31.986 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-08T08:03:31.986 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/twl4030_wdt.ko 2023-11-08T08:03:31.986 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/via_wdt.ko 2023-11-08T08:03:31.987 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-08T08:03:31.987 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-08T08:03:31.987 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-08T08:03:31.987 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/wafer5823wdt.ko 2023-11-08T08:03:31.987 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/wdt_pci.ko 2023-11-08T08:03:31.988 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/wm831x_wdt.ko 2023-11-08T08:03:31.988 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/wm8350_wdt.ko 2023-11-08T08:03:31.988 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/xen_wdt.ko 2023-11-08T08:03:31.988 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/ 2023-11-08T08:03:31.988 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-evtchn.ko 2023-11-08T08:03:31.989 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-gntalloc.ko 2023-11-08T08:03:31.989 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-gntdev.ko 2023-11-08T08:03:31.989 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-pciback/ 2023-11-08T08:03:31.989 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 176441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-08T08:03:31.990 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-privcmd.ko 2023-11-08T08:03:31.990 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 71249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-scsiback.ko 2023-11-08T08:03:31.990 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xenfs/ 2023-11-08T08:03:31.990 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-08T08:03:31.991 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ 2023-11-08T08:03:31.991 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/9p/ 2023-11-08T08:03:31.991 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 166425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/9p/9p.ko 2023-11-08T08:03:31.991 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/adfs/ 2023-11-08T08:03:31.991 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 61073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/adfs/adfs.ko 2023-11-08T08:03:31.992 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/affs/ 2023-11-08T08:03:31.992 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 163593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/affs/affs.ko 2023-11-08T08:03:31.992 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/afs/ 2023-11-08T08:03:31.992 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 932817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/afs/kafs.ko 2023-11-08T08:03:31.993 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/autofs/ 2023-11-08T08:03:31.993 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 101057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/autofs/autofs4.ko 2023-11-08T08:03:31.993 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/befs/ 2023-11-08T08:03:31.993 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 64545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/befs/befs.ko 2023-11-08T08:03:31.993 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/bfs/ 2023-11-08T08:03:31.994 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/bfs/bfs.ko 2023-11-08T08:03:31.994 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/binfmt_misc.ko 2023-11-08T08:03:31.994 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/btrfs/ 2023-11-08T08:03:31.994 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 3402585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/btrfs/btrfs.ko 2023-11-08T08:03:31.995 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/cachefiles/ 2023-11-08T08:03:31.995 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 306073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/cachefiles/cachefiles.ko 2023-11-08T08:03:31.995 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ceph/ 2023-11-08T08:03:31.995 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1323945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ceph/ceph.ko 2023-11-08T08:03:31.995 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/coda/ 2023-11-08T08:03:31.996 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 100801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/coda/coda.ko 2023-11-08T08:03:31.996 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/configfs/ 2023-11-08T08:03:31.996 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 108393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/configfs/configfs.ko 2023-11-08T08:03:31.996 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/cramfs/ 2023-11-08T08:03:31.996 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 29625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/cramfs/cramfs.ko 2023-11-08T08:03:31.997 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/dlm/ 2023-11-08T08:03:31.997 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 642577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/dlm/dlm.ko 2023-11-08T08:03:31.997 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/efs/ 2023-11-08T08:03:31.997 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/efs/efs.ko 2023-11-08T08:03:31.998 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/f2fs/ 2023-11-08T08:03:31.998 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1505025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/f2fs/f2fs.ko 2023-11-08T08:03:31.998 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fat/ 2023-11-08T08:03:31.998 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fat/msdos.ko 2023-11-08T08:03:31.998 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/freevxfs/ 2023-11-08T08:03:31.999 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/freevxfs/freevxfs.ko 2023-11-08T08:03:31.999 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fscache/ 2023-11-08T08:03:31.999 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 226481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fscache/fscache.ko 2023-11-08T08:03:31.999 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fuse/ 2023-11-08T08:03:32.000 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fuse/cuse.ko 2023-11-08T08:03:32.000 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/gfs2/ 2023-11-08T08:03:32.000 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1231529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/gfs2/gfs2.ko 2023-11-08T08:03:32.000 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hfs/ 2023-11-08T08:03:32.000 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 141209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hfs/hfs.ko 2023-11-08T08:03:32.001 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hfsplus/ 2023-11-08T08:03:32.001 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 222817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hfsplus/hfsplus.ko 2023-11-08T08:03:32.001 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hpfs/ 2023-11-08T08:03:32.001 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 192329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hpfs/hpfs.ko 2023-11-08T08:03:32.001 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/isofs/ 2023-11-08T08:03:32.002 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 74065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/isofs/isofs.ko 2023-11-08T08:03:32.002 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/jfs/ 2023-11-08T08:03:32.002 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 419497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/jfs/jfs.ko 2023-11-08T08:03:32.002 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/lockd/ 2023-11-08T08:03:32.003 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 280313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/lockd/lockd.ko 2023-11-08T08:03:32.003 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/minix/ 2023-11-08T08:03:32.003 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 89193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/minix/minix.ko 2023-11-08T08:03:32.003 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/netfs/ 2023-11-08T08:03:32.003 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 124649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/netfs/netfs.ko 2023-11-08T08:03:32.004 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/ 2023-11-08T08:03:32.004 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/blocklayout/ 2023-11-08T08:03:32.004 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 81449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-08T08:03:32.004 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/filelayout/ 2023-11-08T08:03:32.004 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 59449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-08T08:03:32.005 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/flexfilelayout/ 2023-11-08T08:03:32.005 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 117513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-08T08:03:32.005 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1055649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/nfs.ko 2023-11-08T08:03:32.005 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 66441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/nfsv2.ko 2023-11-08T08:03:32.005 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 120177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/nfsv3.ko 2023-11-08T08:03:32.006 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 2041689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/nfsv4.ko 2023-11-08T08:03:32.006 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs_common/ 2023-11-08T08:03:32.006 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs_common/grace.ko 2023-11-08T08:03:32.006 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs_common/nfs_acl.ko 2023-11-08T08:03:32.006 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfsd/ 2023-11-08T08:03:32.007 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1516809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfsd/nfsd.ko 2023-11-08T08:03:32.007 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nilfs2/ 2023-11-08T08:03:32.007 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 592833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nilfs2/nilfs2.ko 2023-11-08T08:03:32.007 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/ 2023-11-08T08:03:32.007 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-celtic.ko 2023-11-08T08:03:32.008 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-centeuro.ko 2023-11-08T08:03:32.008 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-croatian.ko 2023-11-08T08:03:32.008 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-cyrillic.ko 2023-11-08T08:03:32.008 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-gaelic.ko 2023-11-08T08:03:32.008 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-greek.ko 2023-11-08T08:03:32.009 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-iceland.ko 2023-11-08T08:03:32.009 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-inuit.ko 2023-11-08T08:03:32.009 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-roman.ko 2023-11-08T08:03:32.009 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-romanian.ko 2023-11-08T08:03:32.009 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-turkish.ko 2023-11-08T08:03:32.010 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_ascii.ko 2023-11-08T08:03:32.010 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp1250.ko 2023-11-08T08:03:32.010 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp1251.ko 2023-11-08T08:03:32.010 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp1255.ko 2023-11-08T08:03:32.010 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp737.ko 2023-11-08T08:03:32.011 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp775.ko 2023-11-08T08:03:32.011 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp850.ko 2023-11-08T08:03:32.011 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp852.ko 2023-11-08T08:03:32.011 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp855.ko 2023-11-08T08:03:32.011 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp857.ko 2023-11-08T08:03:32.012 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp860.ko 2023-11-08T08:03:32.012 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp861.ko 2023-11-08T08:03:32.012 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp862.ko 2023-11-08T08:03:32.012 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp863.ko 2023-11-08T08:03:32.012 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp864.ko 2023-11-08T08:03:32.013 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp865.ko 2023-11-08T08:03:32.013 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp866.ko 2023-11-08T08:03:32.013 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp869.ko 2023-11-08T08:03:32.013 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp874.ko 2023-11-08T08:03:32.014 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 94137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp932.ko 2023-11-08T08:03:32.014 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 142673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp936.ko 2023-11-08T08:03:32.014 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 165745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp949.ko 2023-11-08T08:03:32.014 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 118121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp950.ko 2023-11-08T08:03:32.014 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_euc-jp.ko 2023-11-08T08:03:32.015 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-1.ko 2023-11-08T08:03:32.015 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-13.ko 2023-11-08T08:03:32.015 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-14.ko 2023-11-08T08:03:32.015 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-15.ko 2023-11-08T08:03:32.015 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-2.ko 2023-11-08T08:03:32.016 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-3.ko 2023-11-08T08:03:32.016 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-4.ko 2023-11-08T08:03:32.016 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-5.ko 2023-11-08T08:03:32.016 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-6.ko 2023-11-08T08:03:32.016 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-7.ko 2023-11-08T08:03:32.017 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-9.ko 2023-11-08T08:03:32.017 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_koi8-r.ko 2023-11-08T08:03:32.017 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_koi8-ru.ko 2023-11-08T08:03:32.017 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_koi8-u.ko 2023-11-08T08:03:32.018 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-08T08:03:32.018 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_utf8.ko 2023-11-08T08:03:32.018 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ntfs/ 2023-11-08T08:03:32.018 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 219417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ntfs/ntfs.ko 2023-11-08T08:03:32.018 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/ 2023-11-08T08:03:32.019 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/cluster/ 2023-11-08T08:03:32.019 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 244409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-08T08:03:32.019 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/dlm/ 2023-11-08T08:03:32.019 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 440737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-08T08:03:32.020 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/dlmfs/ 2023-11-08T08:03:32.020 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 54049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-08T08:03:32.020 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 2592433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/ocfs2.ko 2023-11-08T08:03:32.020 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-08T08:03:32.020 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-08T08:03:32.021 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-08T08:03:32.021 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/omfs/ 2023-11-08T08:03:32.021 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 53433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/omfs/omfs.ko 2023-11-08T08:03:32.021 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/overlayfs/ 2023-11-08T08:03:32.021 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 352153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/overlayfs/overlay.ko 2023-11-08T08:03:32.022 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/pstore/ 2023-11-08T08:03:32.022 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 54809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/pstore/ramoops.ko 2023-11-08T08:03:32.022 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/qnx4/ 2023-11-08T08:03:32.022 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/qnx4/qnx4.ko 2023-11-08T08:03:32.023 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/qnx6/ 2023-11-08T08:03:32.023 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 46689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/qnx6/qnx6.ko 2023-11-08T08:03:32.023 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/quota/ 2023-11-08T08:03:32.023 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/quota/quota_tree.ko 2023-11-08T08:03:32.024 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/quota/quota_v1.ko 2023-11-08T08:03:32.024 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/quota/quota_v2.ko 2023-11-08T08:03:32.024 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/reiserfs/ 2023-11-08T08:03:32.024 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 532673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/reiserfs/reiserfs.ko 2023-11-08T08:03:32.024 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/romfs/ 2023-11-08T08:03:32.025 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/romfs/romfs.ko 2023-11-08T08:03:32.025 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/ 2023-11-08T08:03:32.025 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/client/ 2023-11-08T08:03:32.025 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 2668121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/client/cifs.ko 2023-11-08T08:03:32.025 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/common/ 2023-11-08T08:03:32.026 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/common/cifs_arc4.ko 2023-11-08T08:03:32.026 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/common/cifs_md4.ko 2023-11-08T08:03:32.026 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/squashfs/ 2023-11-08T08:03:32.026 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 134641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/squashfs/squashfs.ko 2023-11-08T08:03:32.027 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/sysv/ 2023-11-08T08:03:32.027 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 90721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/sysv/sysv.ko 2023-11-08T08:03:32.027 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/udf/ 2023-11-08T08:03:32.027 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 247737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/udf/udf.ko 2023-11-08T08:03:32.028 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ufs/ 2023-11-08T08:03:32.028 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 162497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ufs/ufs.ko 2023-11-08T08:03:32.028 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/xfs/ 2023-11-08T08:03:32.028 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 4010729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/xfs/xfs.ko 2023-11-08T08:03:32.028 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/ 2023-11-08T08:03:32.028 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/locking/ 2023-11-08T08:03:32.029 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 75489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/locking/locktorture.ko 2023-11-08T08:03:32.029 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/time/ 2023-11-08T08:03:32.029 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/time/test_udelay.ko 2023-11-08T08:03:32.029 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 60553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/torture.ko 2023-11-08T08:03:32.029 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/ 2023-11-08T08:03:32.030 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 6833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crc-itu-t.ko 2023-11-08T08:03:32.030 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 6529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crc7.ko 2023-11-08T08:03:32.030 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crc8.ko 2023-11-08T08:03:32.030 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crypto/ 2023-11-08T08:03:32.030 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crypto/libarc4.ko 2023-11-08T08:03:32.031 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crypto/libdes.ko 2023-11-08T08:03:32.031 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/interval_tree_test.ko 2023-11-08T08:03:32.031 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/libcrc32c.ko 2023-11-08T08:03:32.031 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/lru_cache.ko 2023-11-08T08:03:32.031 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/lz4/ 2023-11-08T08:03:32.032 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/lz4/lz4_compress.ko 2023-11-08T08:03:32.032 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/lz4/lz4hc_compress.ko 2023-11-08T08:03:32.032 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/math/ 2023-11-08T08:03:32.032 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 6617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/math/cordic.ko 2023-11-08T08:03:32.032 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/memory-notifier-error-inject.ko 2023-11-08T08:03:32.033 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/notifier-error-inject.ko 2023-11-08T08:03:32.033 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/percpu_test.ko 2023-11-08T08:03:32.033 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/pm-notifier-error-inject.ko 2023-11-08T08:03:32.033 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/raid6/ 2023-11-08T08:03:32.033 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 143433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/raid6/raid6_pq.ko 2023-11-08T08:03:32.034 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/rbtree_test.ko 2023-11-08T08:03:32.034 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/reed_solomon/ 2023-11-08T08:03:32.034 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-08T08:03:32.034 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test-kstrtox.ko 2023-11-08T08:03:32.035 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test-string_helpers.ko 2023-11-08T08:03:32.035 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 4760425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test_bpf.ko 2023-11-08T08:03:32.035 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test_firmware.ko 2023-11-08T08:03:32.035 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 6369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test_module.ko 2023-11-08T08:03:32.035 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test_user_copy.ko 2023-11-08T08:03:32.036 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/ts_bm.ko 2023-11-08T08:03:32.036 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/ts_fsm.ko 2023-11-08T08:03:32.036 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/ts_kmp.ko 2023-11-08T08:03:32.036 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/xz/ 2023-11-08T08:03:32.036 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/xz/xz_dec_test.ko 2023-11-08T08:03:32.037 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/mm/ 2023-11-08T08:03:32.037 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/mm/hwpoison-inject.ko 2023-11-08T08:03:32.037 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ 2023-11-08T08:03:32.037 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/ 2023-11-08T08:03:32.037 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/6lowpan.ko 2023-11-08T08:03:32.038 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_dest.ko 2023-11-08T08:03:32.038 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_fragment.ko 2023-11-08T08:03:32.038 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_hop.ko 2023-11-08T08:03:32.038 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 6985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-08T08:03:32.039 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_mobility.ko 2023-11-08T08:03:32.039 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_routing.ko 2023-11-08T08:03:32.039 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_udp.ko 2023-11-08T08:03:32.039 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/ 2023-11-08T08:03:32.039 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/garp.ko 2023-11-08T08:03:32.040 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/mrp.ko 2023-11-08T08:03:32.040 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/p8022.ko 2023-11-08T08:03:32.040 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/psnap.ko 2023-11-08T08:03:32.040 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/stp.ko 2023-11-08T08:03:32.040 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/8021q/ 2023-11-08T08:03:32.041 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 81737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/8021q/8021q.ko 2023-11-08T08:03:32.041 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/9p/ 2023-11-08T08:03:32.041 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 198337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/9p/9pnet.ko 2023-11-08T08:03:32.041 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/9p/9pnet_fd.ko 2023-11-08T08:03:32.041 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/9p/9pnet_rdma.ko 2023-11-08T08:03:32.041 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/9p/9pnet_virtio.ko 2023-11-08T08:03:32.042 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/batman-adv/ 2023-11-08T08:03:32.042 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 649297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/batman-adv/batman-adv.ko 2023-11-08T08:03:32.042 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/ 2023-11-08T08:03:32.042 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 63809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/br_netfilter.ko 2023-11-08T08:03:32.043 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 777689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/bridge.ko 2023-11-08T08:03:32.043 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ 2023-11-08T08:03:32.043 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-08T08:03:32.043 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-08T08:03:32.043 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-08T08:03:32.043 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-08T08:03:32.044 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-08T08:03:32.044 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-08T08:03:32.044 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-08T08:03:32.044 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-08T08:03:32.044 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-08T08:03:32.045 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-08T08:03:32.045 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-08T08:03:32.045 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-08T08:03:32.045 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-08T08:03:32.045 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-08T08:03:32.046 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-08T08:03:32.046 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-08T08:03:32.046 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-08T08:03:32.046 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-08T08:03:32.047 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-08T08:03:32.047 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-08T08:03:32.047 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 55929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebtables.ko 2023-11-08T08:03:32.047 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-08T08:03:32.047 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-08T08:03:32.048 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ceph/ 2023-11-08T08:03:32.048 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1046857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ceph/libceph.ko 2023-11-08T08:03:32.048 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/core/ 2023-11-08T08:03:32.048 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 125697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/core/pktgen.ko 2023-11-08T08:03:32.049 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dccp/ 2023-11-08T08:03:32.049 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 194209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dccp/dccp.ko 2023-11-08T08:03:32.049 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dccp/dccp_diag.ko 2023-11-08T08:03:32.049 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 57881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dccp/dccp_ipv4.ko 2023-11-08T08:03:32.050 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 60337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dccp/dccp_ipv6.ko 2023-11-08T08:03:32.050 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/ 2023-11-08T08:03:32.050 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 390473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/dsa_core.ko 2023-11-08T08:03:32.050 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/tag_brcm.ko 2023-11-08T08:03:32.050 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/tag_dsa.ko 2023-11-08T08:03:32.051 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/tag_none.ko 2023-11-08T08:03:32.051 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/tag_trailer.ko 2023-11-08T08:03:32.051 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/hsr/ 2023-11-08T08:03:32.051 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 99729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/hsr/hsr.ko 2023-11-08T08:03:32.052 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ieee802154/ 2023-11-08T08:03:32.052 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ieee802154/6lowpan/ 2023-11-08T08:03:32.052 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 45233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-08T08:03:32.052 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 254625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ieee802154/ieee802154.ko 2023-11-08T08:03:32.052 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-08T08:03:32.053 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ 2023-11-08T08:03:32.053 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ah4.ko 2023-11-08T08:03:32.053 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/esp4.ko 2023-11-08T08:03:32.053 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 48401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/fou.ko 2023-11-08T08:03:32.053 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/gre.ko 2023-11-08T08:03:32.054 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 48657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/inet_diag.ko 2023-11-08T08:03:32.054 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 60633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ip_gre.ko 2023-11-08T08:03:32.054 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 61225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ip_tunnel.ko 2023-11-08T08:03:32.054 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ip_vti.ko 2023-11-08T08:03:32.054 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ipcomp.ko 2023-11-08T08:03:32.055 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ipip.ko 2023-11-08T08:03:32.055 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ 2023-11-08T08:03:32.055 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-08T08:03:32.055 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-08T08:03:32.056 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-08T08:03:32.056 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ip_tables.ko 2023-11-08T08:03:32.056 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-08T08:03:32.056 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-11-08T08:03:32.056 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-08T08:03:32.057 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-08T08:03:32.057 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-08T08:03:32.057 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-11-08T08:03:32.057 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-08T08:03:32.058 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-08T08:03:32.058 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-08T08:03:32.058 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-08T08:03:32.058 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-08T08:03:32.058 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-08T08:03:32.059 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-08T08:03:32.059 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-08T08:03:32.059 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-08T08:03:32.059 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-11-08T08:03:32.059 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-08T08:03:32.060 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-08T08:03:32.060 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-08T08:03:32.060 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-08T08:03:32.060 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-08T08:03:32.060 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_bic.ko 2023-11-08T08:03:32.061 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_dctcp.ko 2023-11-08T08:03:32.061 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_diag.ko 2023-11-08T08:03:32.061 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_highspeed.ko 2023-11-08T08:03:32.061 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_htcp.ko 2023-11-08T08:03:32.061 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_hybla.ko 2023-11-08T08:03:32.062 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_illinois.ko 2023-11-08T08:03:32.062 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_lp.ko 2023-11-08T08:03:32.062 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_scalable.ko 2023-11-08T08:03:32.062 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_vegas.ko 2023-11-08T08:03:32.063 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_veno.ko 2023-11-08T08:03:32.063 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_westwood.ko 2023-11-08T08:03:32.063 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_yeah.ko 2023-11-08T08:03:32.063 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tunnel4.ko 2023-11-08T08:03:32.063 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/udp_diag.ko 2023-11-08T08:03:32.064 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 41665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/udp_tunnel.ko 2023-11-08T08:03:32.064 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-08T08:03:32.064 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ 2023-11-08T08:03:32.064 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ah6.ko 2023-11-08T08:03:32.065 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/esp6.ko 2023-11-08T08:03:32.065 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/fou6.ko 2023-11-08T08:03:32.065 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 80897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ip6_gre.ko 2023-11-08T08:03:32.065 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 93577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ip6_tunnel.ko 2023-11-08T08:03:32.065 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-08T08:03:32.066 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 53273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ip6_vti.ko 2023-11-08T08:03:32.066 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ipcomp6.ko 2023-11-08T08:03:32.066 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/mip6.ko 2023-11-08T08:03:32.066 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ 2023-11-08T08:03:32.067 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-08T08:03:32.067 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-11-08T08:03:32.067 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-08T08:03:32.067 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-08T08:03:32.068 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-08T08:03:32.068 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-08T08:03:32.068 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-08T08:03:32.068 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-08T08:03:32.069 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-08T08:03:32.069 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-08T08:03:32.069 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-08T08:03:32.069 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-08T08:03:32.069 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-08T08:03:32.070 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-08T08:03:32.070 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-08T08:03:32.070 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-08T08:03:32.070 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-08T08:03:32.071 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-08T08:03:32.071 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-08T08:03:32.071 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-08T08:03:32.071 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-08T08:03:32.071 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-08T08:03:32.072 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-08T08:03:32.072 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-08T08:03:32.072 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-08T08:03:32.072 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 74073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/sit.ko 2023-11-08T08:03:32.073 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/tunnel6.ko 2023-11-08T08:03:32.073 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-08T08:03:32.073 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/key/ 2023-11-08T08:03:32.073 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 82833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/key/af_key.ko 2023-11-08T08:03:32.073 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/ 2023-11-08T08:03:32.074 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 125281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_core.ko 2023-11-08T08:03:32.074 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-08T08:03:32.074 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 25953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_eth.ko 2023-11-08T08:03:32.074 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_ip.ko 2023-11-08T08:03:32.075 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_ip6.ko 2023-11-08T08:03:32.075 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_netlink.ko 2023-11-08T08:03:32.075 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 53241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_ppp.ko 2023-11-08T08:03:32.075 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/lapb/ 2023-11-08T08:03:32.075 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 47977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/lapb/lapb.ko 2023-11-08T08:03:32.076 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/llc/ 2023-11-08T08:03:32.076 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/llc/llc.ko 2023-11-08T08:03:32.076 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 267777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/llc/llc2.ko 2023-11-08T08:03:32.076 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/mac802154/ 2023-11-08T08:03:32.077 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 290417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/mac802154/mac802154.ko 2023-11-08T08:03:32.077 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/mpls/ 2023-11-08T08:03:32.077 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/mpls/mpls_gso.ko 2023-11-08T08:03:32.077 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ 2023-11-08T08:03:32.077 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ 2023-11-08T08:03:32.078 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 119585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set.ko 2023-11-08T08:03:32.078 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-08T08:03:32.078 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-08T08:03:32.078 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-08T08:03:32.079 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 100297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-08T08:03:32.079 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 101441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-08T08:03:32.079 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 102817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-08T08:03:32.079 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 103481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-08T08:03:32.079 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 112849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-08T08:03:32.080 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 60329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-08T08:03:32.080 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 109593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-08T08:03:32.080 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 113129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-08T08:03:32.080 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 112585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-08T08:03:32.080 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 111585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-08T08:03:32.081 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 114465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-08T08:03:32.081 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-08T08:03:32.081 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ 2023-11-08T08:03:32.081 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 411729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-08T08:03:32.081 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-08T08:03:32.082 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-08T08:03:32.082 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-08T08:03:32.082 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-08T08:03:32.082 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-08T08:03:32.083 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-08T08:03:32.083 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-08T08:03:32.083 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-08T08:03:32.083 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-08T08:03:32.083 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-08T08:03:32.084 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-08T08:03:32.084 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-08T08:03:32.084 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-08T08:03:32.084 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conncount.ko 2023-11-08T08:03:32.085 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 408257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack.ko 2023-11-08T08:03:32.085 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-08T08:03:32.085 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-08T08:03:32.085 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-08T08:03:32.085 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 118145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-08T08:03:32.086 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-08T08:03:32.086 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-08T08:03:32.086 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 107297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-08T08:03:32.086 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-08T08:03:32.086 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-08T08:03:32.087 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 66641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-08T08:03:32.087 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-08T08:03:32.087 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-08T08:03:32.087 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-08T08:03:32.088 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 89473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_flow_table.ko 2023-11-08T08:03:32.088 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-08T08:03:32.088 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 37129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_log_syslog.ko 2023-11-08T08:03:32.088 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 95417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat.ko 2023-11-08T08:03:32.088 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-08T08:03:32.089 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-08T08:03:32.089 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat_irc.ko 2023-11-08T08:03:32.089 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat_sip.ko 2023-11-08T08:03:32.089 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-08T08:03:32.089 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 44065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-08T08:03:32.090 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 615313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_tables.ko 2023-11-08T08:03:32.090 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 42561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink.ko 2023-11-08T08:03:32.090 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-08T08:03:32.090 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-11-08T08:03:32.091 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 39785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink_log.ko 2023-11-08T08:03:32.091 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-08T08:03:32.091 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-08T08:03:32.091 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_chain_nat.ko 2023-11-08T08:03:32.091 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_compat.ko 2023-11-08T08:03:32.092 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 40721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_ct.ko 2023-11-08T08:03:32.092 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-08T08:03:32.092 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_fib.ko 2023-11-08T08:03:32.092 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-08T08:03:32.092 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_hash.ko 2023-11-08T08:03:32.093 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_limit.ko 2023-11-08T08:03:32.093 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_log.ko 2023-11-08T08:03:32.093 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_masq.ko 2023-11-08T08:03:32.093 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_nat.ko 2023-11-08T08:03:32.093 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_numgen.ko 2023-11-08T08:03:32.094 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_osf.ko 2023-11-08T08:03:32.094 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_queue.ko 2023-11-08T08:03:32.094 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_redir.ko 2023-11-08T08:03:32.094 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_reject.ko 2023-11-08T08:03:32.095 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_reject_inet.ko 2023-11-08T08:03:32.095 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_socket.ko 2023-11-08T08:03:32.095 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_synproxy.ko 2023-11-08T08:03:32.095 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_tproxy.ko 2023-11-08T08:03:32.095 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 89313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/x_tables.ko 2023-11-08T08:03:32.096 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_AUDIT.ko 2023-11-08T08:03:32.096 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-08T08:03:32.096 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-08T08:03:32.096 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-08T08:03:32.096 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 18169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_CT.ko 2023-11-08T08:03:32.097 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_DSCP.ko 2023-11-08T08:03:32.097 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_HL.ko 2023-11-08T08:03:32.097 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_HMARK.ko 2023-11-08T08:03:32.097 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-08T08:03:32.097 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_LED.ko 2023-11-08T08:03:32.098 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_LOG.ko 2023-11-08T08:03:32.098 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-08T08:03:32.098 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_NETMAP.ko 2023-11-08T08:03:32.098 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_NFLOG.ko 2023-11-08T08:03:32.099 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-08T08:03:32.099 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_RATEEST.ko 2023-11-08T08:03:32.099 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-08T08:03:32.099 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 14337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_SECMARK.ko 2023-11-08T08:03:32.099 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-08T08:03:32.100 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-08T08:03:32.100 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_TEE.ko 2023-11-08T08:03:32.100 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_TPROXY.ko 2023-11-08T08:03:32.100 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_TRACE.ko 2023-11-08T08:03:32.100 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_addrtype.ko 2023-11-08T08:03:32.101 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_bpf.ko 2023-11-08T08:03:32.101 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_cgroup.ko 2023-11-08T08:03:32.101 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_cluster.ko 2023-11-08T08:03:32.101 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_comment.ko 2023-11-08T08:03:32.101 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_connbytes.ko 2023-11-08T08:03:32.102 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_connlabel.ko 2023-11-08T08:03:32.102 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_connlimit.ko 2023-11-08T08:03:32.102 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_connmark.ko 2023-11-08T08:03:32.102 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_conntrack.ko 2023-11-08T08:03:32.103 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_cpu.ko 2023-11-08T08:03:32.103 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_dccp.ko 2023-11-08T08:03:32.103 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_devgroup.ko 2023-11-08T08:03:32.103 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_dscp.ko 2023-11-08T08:03:32.103 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_ecn.ko 2023-11-08T08:03:32.104 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_esp.ko 2023-11-08T08:03:32.104 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_hashlimit.ko 2023-11-08T08:03:32.104 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_helper.ko 2023-11-08T08:03:32.104 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_hl.ko 2023-11-08T08:03:32.104 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_ipcomp.ko 2023-11-08T08:03:32.105 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_iprange.ko 2023-11-08T08:03:32.105 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_ipvs.ko 2023-11-08T08:03:32.105 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_l2tp.ko 2023-11-08T08:03:32.105 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_length.ko 2023-11-08T08:03:32.105 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_limit.ko 2023-11-08T08:03:32.106 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_mac.ko 2023-11-08T08:03:32.106 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_mark.ko 2023-11-08T08:03:32.106 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_multiport.ko 2023-11-08T08:03:32.106 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_nat.ko 2023-11-08T08:03:32.107 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_nfacct.ko 2023-11-08T08:03:32.107 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_osf.ko 2023-11-08T08:03:32.107 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_owner.ko 2023-11-08T08:03:32.107 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_physdev.ko 2023-11-08T08:03:32.107 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_pkttype.ko 2023-11-08T08:03:32.108 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_policy.ko 2023-11-08T08:03:32.108 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_quota.ko 2023-11-08T08:03:32.108 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_rateest.ko 2023-11-08T08:03:32.108 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_realm.ko 2023-11-08T08:03:32.108 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_recent.ko 2023-11-08T08:03:32.109 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_sctp.ko 2023-11-08T08:03:32.109 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_set.ko 2023-11-08T08:03:32.109 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 16089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_socket.ko 2023-11-08T08:03:32.109 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_state.ko 2023-11-08T08:03:32.110 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_statistic.ko 2023-11-08T08:03:32.110 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_string.ko 2023-11-08T08:03:32.110 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 8553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_tcpmss.ko 2023-11-08T08:03:32.110 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_tcpudp.ko 2023-11-08T08:03:32.110 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_time.ko 2023-11-08T08:03:32.111 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_u32.ko 2023-11-08T08:03:32.111 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netlink/ 2023-11-08T08:03:32.111 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netlink/netlink_diag.ko 2023-11-08T08:03:32.111 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/nsh/ 2023-11-08T08:03:32.112 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 12425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/nsh/nsh.ko 2023-11-08T08:03:32.112 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/openvswitch/ 2023-11-08T08:03:32.112 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 380033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/openvswitch/openvswitch.ko 2023-11-08T08:03:32.112 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/openvswitch/vport-geneve.ko 2023-11-08T08:03:32.112 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/openvswitch/vport-gre.ko 2023-11-08T08:03:32.113 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 11793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/openvswitch/vport-vxlan.ko 2023-11-08T08:03:32.113 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/packet/ 2023-11-08T08:03:32.113 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/packet/af_packet_diag.ko 2023-11-08T08:03:32.113 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/phonet/ 2023-11-08T08:03:32.113 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 110201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/phonet/phonet.ko 2023-11-08T08:03:32.114 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 54697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/phonet/pn_pep.ko 2023-11-08T08:03:32.114 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rds/ 2023-11-08T08:03:32.114 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 251481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rds/rds.ko 2023-11-08T08:03:32.114 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 164937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rds/rds_rdma.ko 2023-11-08T08:03:32.114 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 67377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rds/rds_tcp.ko 2023-11-08T08:03:32.115 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rxrpc/ 2023-11-08T08:03:32.115 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 819657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rxrpc/rxrpc.ko 2023-11-08T08:03:32.115 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/ 2023-11-08T08:03:32.115 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_csum.ko 2023-11-08T08:03:32.116 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_gact.ko 2023-11-08T08:03:32.116 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_ipt.ko 2023-11-08T08:03:32.116 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_mirred.ko 2023-11-08T08:03:32.116 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_nat.ko 2023-11-08T08:03:32.116 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 43449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_pedit.ko 2023-11-08T08:03:32.117 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_police.ko 2023-11-08T08:03:32.117 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_simple.ko 2023-11-08T08:03:32.117 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_skbedit.ko 2023-11-08T08:03:32.117 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_vlan.ko 2023-11-08T08:03:32.117 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_basic.ko 2023-11-08T08:03:32.118 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_bpf.ko 2023-11-08T08:03:32.118 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_cgroup.ko 2023-11-08T08:03:32.118 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_flow.ko 2023-11-08T08:03:32.118 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_fw.ko 2023-11-08T08:03:32.119 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 38489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_route.ko 2023-11-08T08:03:32.119 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 61409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_u32.ko 2023-11-08T08:03:32.119 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_cmp.ko 2023-11-08T08:03:32.119 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 13409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_ipset.ko 2023-11-08T08:03:32.119 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_meta.ko 2023-11-08T08:03:32.120 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_nbyte.ko 2023-11-08T08:03:32.120 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_text.ko 2023-11-08T08:03:32.120 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_u32.ko 2023-11-08T08:03:32.120 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 21649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_choke.ko 2023-11-08T08:03:32.121 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_codel.ko 2023-11-08T08:03:32.121 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 34633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_drr.ko 2023-11-08T08:03:32.121 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 33609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_fq.ko 2023-11-08T08:03:32.121 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_fq_codel.ko 2023-11-08T08:03:32.121 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 35433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_gred.ko 2023-11-08T08:03:32.122 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 49601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_hfsc.ko 2023-11-08T08:03:32.122 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_hhf.ko 2023-11-08T08:03:32.122 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 66393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_htb.ko 2023-11-08T08:03:32.122 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_ingress.ko 2023-11-08T08:03:32.122 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 30273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_mqprio.ko 2023-11-08T08:03:32.123 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_mqprio_lib.ko 2023-11-08T08:03:32.123 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_multiq.ko 2023-11-08T08:03:32.123 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 36137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_netem.ko 2023-11-08T08:03:32.123 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 27497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_pie.ko 2023-11-08T08:03:32.124 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_plug.ko 2023-11-08T08:03:32.124 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 24729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_prio.ko 2023-11-08T08:03:32.124 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 48793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_qfq.ko 2023-11-08T08:03:32.124 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 32905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_red.ko 2023-11-08T08:03:32.124 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 26881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_sfb.ko 2023-11-08T08:03:32.125 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 31353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_sfq.ko 2023-11-08T08:03:32.125 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_tbf.ko 2023-11-08T08:03:32.125 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 28105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_teql.ko 2023-11-08T08:03:32.125 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sctp/ 2023-11-08T08:03:32.125 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 929745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sctp/sctp.ko 2023-11-08T08:03:32.126 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 19033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sctp/sctp_diag.ko 2023-11-08T08:03:32.126 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/ 2023-11-08T08:03:32.126 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/auth_gss/ 2023-11-08T08:03:32.126 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 336529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-08T08:03:32.127 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 54737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-08T08:03:32.127 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 1520161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/sunrpc.ko 2023-11-08T08:03:32.127 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/xprtrdma/ 2023-11-08T08:03:32.127 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 707721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-08T08:03:32.127 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/tipc/ 2023-11-08T08:03:32.128 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/tipc/diag.ko 2023-11-08T08:03:32.128 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 862641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/tipc/tipc.ko 2023-11-08T08:03:32.128 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/unix/ 2023-11-08T08:03:32.128 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/unix/unix_diag.ko 2023-11-08T08:03:32.128 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/ 2023-11-08T08:03:32.129 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 85337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-08T08:03:32.129 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 68049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-08T08:03:32.129 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 96193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/vsock.ko 2023-11-08T08:03:32.129 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 10705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-08T08:03:32.130 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 17521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-08T08:03:32.130 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/x25/ 2023-11-08T08:03:32.130 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 125177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/x25/x25.ko 2023-11-08T08:03:32.130 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/xfrm/ 2023-11-08T08:03:32.130 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 22657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/xfrm/xfrm_algo.ko 2023-11-08T08:03:32.131 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 20649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-08T08:03:32.131 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 102257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/xfrm/xfrm_user.ko 2023-11-08T08:03:32.131 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/virt/ 2023-11-08T08:03:32.131 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/virt/lib/ 2023-11-08T08:03:32.131 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 15537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/virt/lib/irqbypass.ko 2023-11-08T08:03:32.132 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 639863 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.alias 2023-11-08T08:03:32.132 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 623919 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.alias.bin 2023-11-08T08:03:32.132 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 7663 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.builtin 2023-11-08T08:03:32.132 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 23428 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.builtin.alias.bin 2023-11-08T08:03:32.132 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 9821 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.builtin.bin 2023-11-08T08:03:32.133 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 70004 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.builtin.modinfo 2023-11-08T08:03:32.133 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 146313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.dep 2023-11-08T08:03:32.133 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 232548 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.dep.bin 2023-11-08T08:03:32.133 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 210 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.devname 2023-11-08T08:03:32.134 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 72606 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.order 2023-11-08T08:03:32.134 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 879 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.softdep 2023-11-08T08:03:32.134 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 254438 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.symbols 2023-11-08T08:03:32.134 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 309363 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.symbols.bin 2023-11-08T08:03:32.134 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./usr/ 2023-11-08T08:03:32.135 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./usr/share/ 2023-11-08T08:03:32.135 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./usr/share/doc/ 2023-11-08T08:03:32.135 INFO:teuthology.orchestra.run.smithi169.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./usr/share/doc/linux-image-6.6.0-rc5-g3b796cfe2969/ 2023-11-08T08:03:32.135 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 173 2023-11-07 03:19 ./usr/share/doc/linux-image-6.6.0-rc5-g3b796cfe2969/changelog.Debian.gz 2023-11-08T08:03:32.135 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r-- root/root 691 2023-11-07 03:19 ./usr/share/doc/linux-image-6.6.0-rc5-g3b796cfe2969/copyright 2023-11-08T08:03:32.137 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:32.137 INFO:teuthology.task.kernel:searching for kernel 6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:32.137 DEBUG:teuthology.orchestra.run.smithi169:> egrep '(submenu|menuentry.*6.6.0-rc5-g3b796cfe2969).*{' /boot/grub/grub.cfg 2023-11-08T08:03:32.155 INFO:teuthology.orchestra.run.smithi169.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-11-08T08:03:32.155 INFO:teuthology.orchestra.run.smithi169.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g3b796cfe2969' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g3b796cfe2969-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-11-08T08:03:32.155 INFO:teuthology.orchestra.run.smithi169.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g3b796cfe2969 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g3b796cfe2969-recovery-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-11-08T08:03:32.156 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-11-08T08:03:32.156 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:32.157 DEBUG:teuthology.orchestra.run.smithi169:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-11-08T08:03:32.164 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi169.front.sepia.ceph.com to complete... 2023-11-08T08:03:32.326 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/ 2023-11-08T08:03:32.326 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/kernel/ 2023-11-08T08:03:32.327 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/kernel/postinst.d/ 2023-11-08T08:03:32.327 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/kernel/postrm.d/ 2023-11-08T08:03:32.327 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/kernel/preinst.d/ 2023-11-08T08:03:32.327 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./etc/kernel/prerm.d/ 2023-11-08T08:03:32.328 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/ 2023-11-08T08:03:32.328 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/ 2023-11-08T08:03:32.328 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/ 2023-11-08T08:03:32.328 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/ 2023-11-08T08:03:32.328 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/ 2023-11-08T08:03:32.329 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/ 2023-11-08T08:03:32.329 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/ 2023-11-08T08:03:32.329 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 490233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/aesni-intel.ko 2023-11-08T08:03:32.329 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-08T08:03:32.330 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-08T08:03:32.330 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-08T08:03:32.330 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 93113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-08T08:03:32.330 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 106729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-08T08:03:32.330 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 152241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-08T08:03:32.331 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-08T08:03:32.331 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-08T08:03:32.331 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 83889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-08T08:03:32.331 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-08T08:03:32.332 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 47225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-08T08:03:32.332 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 47209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-08T08:03:32.332 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 51801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-08T08:03:32.332 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 38937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-08T08:03:32.332 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 40529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-08T08:03:32.333 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 58585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-08T08:03:32.333 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 50497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-08T08:03:32.333 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 32065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-08T08:03:32.333 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-08T08:03:32.333 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/ 2023-11-08T08:03:32.334 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/cpu/ 2023-11-08T08:03:32.334 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/cpu/mce/ 2023-11-08T08:03:32.334 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 40113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-08T08:03:32.334 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/cpuid.ko 2023-11-08T08:03:32.335 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kernel/msr.ko 2023-11-08T08:03:32.335 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kvm/ 2023-11-08T08:03:32.335 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 353761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-08T08:03:32.335 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 881713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-08T08:03:32.335 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 2276817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/arch/x86/kvm/kvm.ko 2023-11-08T08:03:32.336 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/ 2023-11-08T08:03:32.336 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 53761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/af_alg.ko 2023-11-08T08:03:32.336 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/algif_hash.ko 2023-11-08T08:03:32.336 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/algif_skcipher.ko 2023-11-08T08:03:32.336 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/ansi_cprng.ko 2023-11-08T08:03:32.337 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/anubis.ko 2023-11-08T08:03:32.337 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/arc4.ko 2023-11-08T08:03:32.337 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/asymmetric_keys/ 2023-11-08T08:03:32.337 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-11-08T08:03:32.338 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/ 2023-11-08T08:03:32.338 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/async_memcpy.ko 2023-11-08T08:03:32.338 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/async_pq.ko 2023-11-08T08:03:32.338 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-08T08:03:32.338 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/async_tx.ko 2023-11-08T08:03:32.339 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/async_xor.ko 2023-11-08T08:03:32.339 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/async_tx/raid6test.ko 2023-11-08T08:03:32.339 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/authenc.ko 2023-11-08T08:03:32.339 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/authencesn.ko 2023-11-08T08:03:32.339 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/blake2b_generic.ko 2023-11-08T08:03:32.340 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/blowfish_common.ko 2023-11-08T08:03:32.340 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/blowfish_generic.ko 2023-11-08T08:03:32.340 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 38873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/camellia_generic.ko 2023-11-08T08:03:32.340 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cast5_generic.ko 2023-11-08T08:03:32.340 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cast6_generic.ko 2023-11-08T08:03:32.341 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cast_common.ko 2023-11-08T08:03:32.341 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 35097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/ccm.ko 2023-11-08T08:03:32.341 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cmac.ko 2023-11-08T08:03:32.341 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/crc32_generic.ko 2023-11-08T08:03:32.342 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 47505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cryptd.ko 2023-11-08T08:03:32.342 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 39505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/crypto_engine.ko 2023-11-08T08:03:32.342 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/crypto_simd.ko 2023-11-08T08:03:32.342 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/crypto_user.ko 2023-11-08T08:03:32.343 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/cts.ko 2023-11-08T08:03:32.343 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/des_generic.ko 2023-11-08T08:03:32.343 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/dh_generic.ko 2023-11-08T08:03:32.343 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/echainiv.ko 2023-11-08T08:03:32.343 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/essiv.ko 2023-11-08T08:03:32.344 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/fcrypt.ko 2023-11-08T08:03:32.344 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/khazad.ko 2023-11-08T08:03:32.344 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/lrw.ko 2023-11-08T08:03:32.344 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/lz4.ko 2023-11-08T08:03:32.344 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/lz4hc.ko 2023-11-08T08:03:32.345 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/md4.ko 2023-11-08T08:03:32.345 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/michael_mic.ko 2023-11-08T08:03:32.345 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/pcbc.ko 2023-11-08T08:03:32.345 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/pcrypt.ko 2023-11-08T08:03:32.345 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/rmd160.ko 2023-11-08T08:03:32.346 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/seed.ko 2023-11-08T08:03:32.346 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/serpent_generic.ko 2023-11-08T08:03:32.346 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 149193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/tcrypt.ko 2023-11-08T08:03:32.346 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/tea.ko 2023-11-08T08:03:32.347 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/twofish_common.ko 2023-11-08T08:03:32.347 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/twofish_generic.ko 2023-11-08T08:03:32.347 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/vmac.ko 2023-11-08T08:03:32.347 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 36185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/wp512.ko 2023-11-08T08:03:32.347 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/xcbc.ko 2023-11-08T08:03:32.348 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/xor.ko 2023-11-08T08:03:32.348 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/xts.ko 2023-11-08T08:03:32.348 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/crypto/xxhash_generic.ko 2023-11-08T08:03:32.348 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ 2023-11-08T08:03:32.348 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/ 2023-11-08T08:03:32.349 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/acpi_extlog.ko 2023-11-08T08:03:32.349 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-08T08:03:32.349 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/acpi_pad.ko 2023-11-08T08:03:32.349 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/apei/ 2023-11-08T08:03:32.349 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 31241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/apei/einj.ko 2023-11-08T08:03:32.350 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/ec_sys.ko 2023-11-08T08:03:32.350 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/platform_profile.ko 2023-11-08T08:03:32.350 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/sbs.ko 2023-11-08T08:03:32.350 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/sbshc.ko 2023-11-08T08:03:32.351 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 103025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/acpi/video.ko 2023-11-08T08:03:32.351 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/ 2023-11-08T08:03:32.351 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/acard-ahci.ko 2023-11-08T08:03:32.351 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 88865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/ahci.ko 2023-11-08T08:03:32.351 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/ahci_platform.ko 2023-11-08T08:03:32.352 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 88289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/libahci.ko 2023-11-08T08:03:32.352 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/libahci_platform.ko 2023-11-08T08:03:32.352 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_acpi.ko 2023-11-08T08:03:32.352 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_ali.ko 2023-11-08T08:03:32.352 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_amd.ko 2023-11-08T08:03:32.353 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_artop.ko 2023-11-08T08:03:32.353 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_atiixp.ko 2023-11-08T08:03:32.353 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_atp867x.ko 2023-11-08T08:03:32.353 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_cmd640.ko 2023-11-08T08:03:32.353 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_cmd64x.ko 2023-11-08T08:03:32.354 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_cypress.ko 2023-11-08T08:03:32.354 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_efar.ko 2023-11-08T08:03:32.354 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_hpt366.ko 2023-11-08T08:03:32.354 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_hpt37x.ko 2023-11-08T08:03:32.354 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-08T08:03:32.355 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-08T08:03:32.355 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_it8213.ko 2023-11-08T08:03:32.355 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 35273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_it821x.ko 2023-11-08T08:03:32.355 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_jmicron.ko 2023-11-08T08:03:32.356 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 51481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_legacy.ko 2023-11-08T08:03:32.356 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_marvell.ko 2023-11-08T08:03:32.356 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_mpiix.ko 2023-11-08T08:03:32.356 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_netcell.ko 2023-11-08T08:03:32.356 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_ninja32.ko 2023-11-08T08:03:32.357 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_ns87410.ko 2023-11-08T08:03:32.357 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_ns87415.ko 2023-11-08T08:03:32.357 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_oldpiix.ko 2023-11-08T08:03:32.357 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_opti.ko 2023-11-08T08:03:32.358 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_optidma.ko 2023-11-08T08:03:32.358 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 32585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_pcmcia.ko 2023-11-08T08:03:32.358 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 37793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-08T08:03:32.358 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-08T08:03:32.359 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_piccolo.ko 2023-11-08T08:03:32.359 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_radisys.ko 2023-11-08T08:03:32.359 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_rdc.ko 2023-11-08T08:03:32.359 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_rz1000.ko 2023-11-08T08:03:32.359 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_sch.ko 2023-11-08T08:03:32.360 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_serverworks.ko 2023-11-08T08:03:32.360 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_sil680.ko 2023-11-08T08:03:32.360 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_sl82c105.ko 2023-11-08T08:03:32.360 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_triflex.ko 2023-11-08T08:03:32.360 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pata_via.ko 2023-11-08T08:03:32.361 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/pdc_adma.ko 2023-11-08T08:03:32.361 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_inic162x.ko 2023-11-08T08:03:32.361 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 83345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_mv.ko 2023-11-08T08:03:32.361 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 70393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_nv.ko 2023-11-08T08:03:32.362 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 37377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_promise.ko 2023-11-08T08:03:32.362 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_qstor.ko 2023-11-08T08:03:32.362 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_sil.ko 2023-11-08T08:03:32.362 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 38337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_sil24.ko 2023-11-08T08:03:32.362 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_sis.ko 2023-11-08T08:03:32.363 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_svw.ko 2023-11-08T08:03:32.363 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 44793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_sx4.ko 2023-11-08T08:03:32.363 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_uli.ko 2023-11-08T08:03:32.363 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 35065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_via.ko 2023-11-08T08:03:32.363 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ata/sata_vsc.ko 2023-11-08T08:03:32.364 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/auxdisplay/ 2023-11-08T08:03:32.364 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/auxdisplay/ks0108.ko 2023-11-08T08:03:32.364 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/bcma/ 2023-11-08T08:03:32.364 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 124473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/bcma/bcma.ko 2023-11-08T08:03:32.365 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/ 2023-11-08T08:03:32.365 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/aoe/ 2023-11-08T08:03:32.365 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 113665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/aoe/aoe.ko 2023-11-08T08:03:32.365 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/drbd/ 2023-11-08T08:03:32.365 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 965585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/drbd/drbd.ko 2023-11-08T08:03:32.366 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 173225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/floppy.ko 2023-11-08T08:03:32.366 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/mtip32xx/ 2023-11-08T08:03:32.366 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 94297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-08T08:03:32.366 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 115793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/nbd.ko 2023-11-08T08:03:32.366 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/null_blk/ 2023-11-08T08:03:32.367 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 110449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/null_blk/null_blk.ko 2023-11-08T08:03:32.367 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 98193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/pktcdvd.ko 2023-11-08T08:03:32.367 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 263769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/rbd.ko 2023-11-08T08:03:32.367 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/xen-blkback/ 2023-11-08T08:03:32.368 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 98145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-08T08:03:32.368 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/zram/ 2023-11-08T08:03:32.368 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 60217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/block/zram/zram.ko 2023-11-08T08:03:32.368 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ 2023-11-08T08:03:32.368 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/applicom.ko 2023-11-08T08:03:32.369 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hangcheck-timer.ko 2023-11-08T08:03:32.369 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/ 2023-11-08T08:03:32.369 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-08T08:03:32.369 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-08T08:03:32.369 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-08T08:03:32.370 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/via-rng.ko 2023-11-08T08:03:32.370 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-08T08:03:32.370 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ 2023-11-08T08:03:32.370 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-08T08:03:32.370 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 148601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-08T08:03:32.371 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-08T08:03:32.371 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 139313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-08T08:03:32.371 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 70921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-08T08:03:32.371 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 53513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-08T08:03:32.371 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 36321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/lp.ko 2023-11-08T08:03:32.372 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/mwave/ 2023-11-08T08:03:32.372 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 84097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/mwave/mwave.ko 2023-11-08T08:03:32.372 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/nvram.ko 2023-11-08T08:03:32.372 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 32169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/ppdev.ko 2023-11-08T08:03:32.373 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 51113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tlclk.ko 2023-11-08T08:03:32.373 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/ 2023-11-08T08:03:32.373 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-08T08:03:32.373 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-11-08T08:03:32.373 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-11-08T08:03:32.374 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 32985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-11-08T08:03:32.374 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-08T08:03:32.374 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-08T08:03:32.374 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-11-08T08:03:32.375 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/xillybus/ 2023-11-08T08:03:32.375 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/xillybus/xillybus_class.ko 2023-11-08T08:03:32.375 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 50945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/xillybus/xillybus_core.ko 2023-11-08T08:03:32.375 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-11-08T08:03:32.375 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/clk/ 2023-11-08T08:03:32.376 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/clk/clk-palmas.ko 2023-11-08T08:03:32.376 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 47849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/clk/clk-si5351.ko 2023-11-08T08:03:32.376 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/clk/clk-twl6040.ko 2023-11-08T08:03:32.376 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/clk/clk-wm831x.ko 2023-11-08T08:03:32.376 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/cpufreq/ 2023-11-08T08:03:32.377 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-08T08:03:32.377 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-08T08:03:32.377 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-08T08:03:32.377 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/ 2023-11-08T08:03:32.377 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/ccp/ 2023-11-08T08:03:32.378 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 70801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-08T08:03:32.378 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 243025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/ccp/ccp.ko 2023-11-08T08:03:32.378 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/ 2023-11-08T08:03:32.378 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/qat/ 2023-11-08T08:03:32.379 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/qat/qat_common/ 2023-11-08T08:03:32.379 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 438761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-08T08:03:32.379 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-11-08T08:03:32.379 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-08T08:03:32.379 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/padlock-aes.ko 2023-11-08T08:03:32.380 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/padlock-sha.ko 2023-11-08T08:03:32.380 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/virtio/ 2023-11-08T08:03:32.380 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 68857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-08T08:03:32.380 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dca/ 2023-11-08T08:03:32.381 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dca/dca.ko 2023-11-08T08:03:32.381 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dma/ 2023-11-08T08:03:32.381 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dma/dw/ 2023-11-08T08:03:32.381 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-08T08:03:32.382 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dma/ioat/ 2023-11-08T08:03:32.382 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 122569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-08T08:03:32.382 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/ 2023-11-08T08:03:32.382 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 79953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/amd64_edac.ko 2023-11-08T08:03:32.382 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 35769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/e752x_edac.ko 2023-11-08T08:03:32.383 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 63617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/edac_mce_amd.ko 2023-11-08T08:03:32.383 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i3000_edac.ko 2023-11-08T08:03:32.383 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i3200_edac.ko 2023-11-08T08:03:32.383 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i5100_edac.ko 2023-11-08T08:03:32.385 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i5400_edac.ko 2023-11-08T08:03:32.385 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i7300_edac.ko 2023-11-08T08:03:32.385 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 52041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i7core_edac.ko 2023-11-08T08:03:32.386 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/i82975x_edac.ko 2023-11-08T08:03:32.386 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/ie31200_edac.ko 2023-11-08T08:03:32.386 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 56825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/sb_edac.ko 2023-11-08T08:03:32.386 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/edac/x38_edac.ko 2023-11-08T08:03:32.387 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/ 2023-11-08T08:03:32.387 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-gpio.ko 2023-11-08T08:03:32.387 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-max14577.ko 2023-11-08T08:03:32.387 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-max77693.ko 2023-11-08T08:03:32.387 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-max8997.ko 2023-11-08T08:03:32.388 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-palmas.ko 2023-11-08T08:03:32.388 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-rt8973a.ko 2023-11-08T08:03:32.388 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/extcon/extcon-sm5502.ko 2023-11-08T08:03:32.388 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/firmware/ 2023-11-08T08:03:32.389 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/firmware/dmi-sysfs.ko 2023-11-08T08:03:32.389 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/firmware/efi/ 2023-11-08T08:03:32.389 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/firmware/efi/efi-pstore.ko 2023-11-08T08:03:32.389 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-08T08:03:32.389 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/ 2023-11-08T08:03:32.390 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-adp5520.ko 2023-11-08T08:03:32.390 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-amd8111.ko 2023-11-08T08:03:32.390 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-arizona.ko 2023-11-08T08:03:32.390 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-da9052.ko 2023-11-08T08:03:32.390 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-da9055.ko 2023-11-08T08:03:32.391 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-dln2.ko 2023-11-08T08:03:32.391 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 77849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-f7188x.ko 2023-11-08T08:03:32.391 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-generic.ko 2023-11-08T08:03:32.391 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-ich.ko 2023-11-08T08:03:32.392 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-11-08T08:03:32.392 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-kempld.ko 2023-11-08T08:03:32.392 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-lp3943.ko 2023-11-08T08:03:32.392 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-max7300.ko 2023-11-08T08:03:32.392 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-max730x.ko 2023-11-08T08:03:32.393 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-max732x.ko 2023-11-08T08:03:32.393 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-11-08T08:03:32.393 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 47073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-pca953x.ko 2023-11-08T08:03:32.393 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-pcf857x.ko 2023-11-08T08:03:32.394 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-rdc321x.ko 2023-11-08T08:03:32.394 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-sch.ko 2023-11-08T08:03:32.394 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-sch311x.ko 2023-11-08T08:03:32.394 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-tps65912.ko 2023-11-08T08:03:32.394 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-twl4030.ko 2023-11-08T08:03:32.395 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-twl6040.ko 2023-11-08T08:03:32.395 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-08T08:03:32.395 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-vx855.ko 2023-11-08T08:03:32.395 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-wm831x.ko 2023-11-08T08:03:32.396 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-wm8350.ko 2023-11-08T08:03:32.396 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/gpio/gpio-wm8994.ko 2023-11-08T08:03:32.396 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/ 2023-11-08T08:03:32.396 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-a4tech.ko 2023-11-08T08:03:32.396 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 39169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-apple.ko 2023-11-08T08:03:32.397 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-appleir.ko 2023-11-08T08:03:32.397 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-aureal.ko 2023-11-08T08:03:32.397 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-axff.ko 2023-11-08T08:03:32.397 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-belkin.ko 2023-11-08T08:03:32.398 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-cherry.ko 2023-11-08T08:03:32.398 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-chicony.ko 2023-11-08T08:03:32.398 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 60649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-cp2112.ko 2023-11-08T08:03:32.398 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-cypress.ko 2023-11-08T08:03:32.398 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-dr.ko 2023-11-08T08:03:32.399 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-elecom.ko 2023-11-08T08:03:32.399 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-elo.ko 2023-11-08T08:03:32.399 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-emsff.ko 2023-11-08T08:03:32.399 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-ezkey.ko 2023-11-08T08:03:32.399 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-gaff.ko 2023-11-08T08:03:32.400 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-generic.ko 2023-11-08T08:03:32.400 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-gt683r.ko 2023-11-08T08:03:32.400 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-gyration.ko 2023-11-08T08:03:32.400 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-08T08:03:32.401 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-08T08:03:32.401 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-holtekff.ko 2023-11-08T08:03:32.401 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-hyperv.ko 2023-11-08T08:03:32.401 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-icade.ko 2023-11-08T08:03:32.401 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-kensington.ko 2023-11-08T08:03:32.402 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-keytouch.ko 2023-11-08T08:03:32.402 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-kye.ko 2023-11-08T08:03:32.402 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-lcpower.ko 2023-11-08T08:03:32.402 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-led.ko 2023-11-08T08:03:32.402 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 42361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-lenovo.ko 2023-11-08T08:03:32.403 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-lg-g15.ko 2023-11-08T08:03:32.403 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 58185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-08T08:03:32.403 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 101369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-08T08:03:32.403 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 80993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-logitech.ko 2023-11-08T08:03:32.404 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-magicmouse.ko 2023-11-08T08:03:32.404 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-microsoft.ko 2023-11-08T08:03:32.404 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-monterey.ko 2023-11-08T08:03:32.404 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 50161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-multitouch.ko 2023-11-08T08:03:32.404 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 32593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-ntrig.ko 2023-11-08T08:03:32.405 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-ortek.ko 2023-11-08T08:03:32.405 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-penmount.ko 2023-11-08T08:03:32.405 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-petalynx.ko 2023-11-08T08:03:32.405 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 72937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-picolcd.ko 2023-11-08T08:03:32.405 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-pl.ko 2023-11-08T08:03:32.406 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-plantronics.ko 2023-11-08T08:03:32.406 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-primax.ko 2023-11-08T08:03:32.406 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 35457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-rmi.ko 2023-11-08T08:03:32.406 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-08T08:03:32.407 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-common.ko 2023-11-08T08:03:32.407 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 31425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-08T08:03:32.407 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-08T08:03:32.407 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 31129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-08T08:03:32.407 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-08T08:03:32.408 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-08T08:03:32.408 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-08T08:03:32.408 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-08T08:03:32.408 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-08T08:03:32.409 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-08T08:03:32.409 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-roccat.ko 2023-11-08T08:03:32.409 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-saitek.ko 2023-11-08T08:03:32.409 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-samsung.ko 2023-11-08T08:03:32.409 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 36817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-08T08:03:32.410 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-sjoy.ko 2023-11-08T08:03:32.410 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 51249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-sony.ko 2023-11-08T08:03:32.410 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-speedlink.ko 2023-11-08T08:03:32.410 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-steelseries.ko 2023-11-08T08:03:32.411 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-sunplus.ko 2023-11-08T08:03:32.411 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-08T08:03:32.411 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-tivo.ko 2023-11-08T08:03:32.411 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-tmff.ko 2023-11-08T08:03:32.411 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-topseed.ko 2023-11-08T08:03:32.412 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-twinhan.ko 2023-11-08T08:03:32.412 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 71073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-uclogic.ko 2023-11-08T08:03:32.412 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-waltop.ko 2023-11-08T08:03:32.412 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 143473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-wiimote.ko 2023-11-08T08:03:32.412 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-xinmo.ko 2023-11-08T08:03:32.413 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-zpff.ko 2023-11-08T08:03:32.413 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid-zydacron.ko 2023-11-08T08:03:32.413 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 267825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/hid.ko 2023-11-08T08:03:32.413 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 32321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/uhid.ko 2023-11-08T08:03:32.413 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/usbhid/ 2023-11-08T08:03:32.414 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 125721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/usbhid/usbhid.ko 2023-11-08T08:03:32.414 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/usbhid/usbkbd.ko 2023-11-08T08:03:32.414 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/usbhid/usbmouse.ko 2023-11-08T08:03:32.414 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 217873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hid/wacom.ko 2023-11-08T08:03:32.415 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hv/ 2023-11-08T08:03:32.415 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 64201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hv/hv_balloon.ko 2023-11-08T08:03:32.415 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 89625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hv/hv_utils.ko 2023-11-08T08:03:32.415 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 305465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hv/hv_vmbus.ko 2023-11-08T08:03:32.415 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ 2023-11-08T08:03:32.416 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 66521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/abituguru.ko 2023-11-08T08:03:32.416 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 61393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/abituguru3.ko 2023-11-08T08:03:32.416 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-08T08:03:32.416 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ad7414.ko 2023-11-08T08:03:32.416 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ad7418.ko 2023-11-08T08:03:32.417 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adc128d818.ko 2023-11-08T08:03:32.417 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 31729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adm1025.ko 2023-11-08T08:03:32.417 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 97977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adm1026.ko 2023-11-08T08:03:32.417 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adm1029.ko 2023-11-08T08:03:32.417 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 45553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adm1031.ko 2023-11-08T08:03:32.418 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adm9240.ko 2023-11-08T08:03:32.418 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ads7828.ko 2023-11-08T08:03:32.418 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7410.ko 2023-11-08T08:03:32.418 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7411.ko 2023-11-08T08:03:32.419 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 74433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7462.ko 2023-11-08T08:03:32.419 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 39585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7470.ko 2023-11-08T08:03:32.419 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 67929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7475.ko 2023-11-08T08:03:32.419 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/adt7x10.ko 2023-11-08T08:03:32.419 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 39073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/amc6821.ko 2023-11-08T08:03:32.420 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 55265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/applesmc.ko 2023-11-08T08:03:32.420 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 49913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/asb100.ko 2023-11-08T08:03:32.420 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 58617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/asc7621.ko 2023-11-08T08:03:32.420 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 40889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-08T08:03:32.420 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/atxp1.ko 2023-11-08T08:03:32.421 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/coretemp.ko 2023-11-08T08:03:32.421 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/da9052-hwmon.ko 2023-11-08T08:03:32.421 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/da9055-hwmon.ko 2023-11-08T08:03:32.421 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 44465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-08T08:03:32.421 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 99601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/dme1737.ko 2023-11-08T08:03:32.422 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ds1621.ko 2023-11-08T08:03:32.422 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ds620.ko 2023-11-08T08:03:32.422 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/emc1403.ko 2023-11-08T08:03:32.422 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/emc2103.ko 2023-11-08T08:03:32.423 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/emc6w201.ko 2023-11-08T08:03:32.423 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 66985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/f71805f.ko 2023-11-08T08:03:32.423 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 94457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/f71882fg.ko 2023-11-08T08:03:32.423 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 41401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/f75375s.ko 2023-11-08T08:03:32.424 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/fam15h_power.ko 2023-11-08T08:03:32.424 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 55449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/fschmd.ko 2023-11-08T08:03:32.424 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/g760a.ko 2023-11-08T08:03:32.424 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/g762.ko 2023-11-08T08:03:32.424 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 46801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/gl518sm.ko 2023-11-08T08:03:32.425 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 45529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/gl520sm.ko 2023-11-08T08:03:32.425 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/hih6130.ko 2023-11-08T08:03:32.425 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-08T08:03:32.425 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/i5500_temp.ko 2023-11-08T08:03:32.425 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/i5k_amb.ko 2023-11-08T08:03:32.426 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ibmaem.ko 2023-11-08T08:03:32.426 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ibmpex.ko 2023-11-08T08:03:32.426 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ina209.ko 2023-11-08T08:03:32.426 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ina2xx.ko 2023-11-08T08:03:32.427 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 130449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/it87.ko 2023-11-08T08:03:32.427 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/jc42.ko 2023-11-08T08:03:32.427 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/k10temp.ko 2023-11-08T08:03:32.427 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/k8temp.ko 2023-11-08T08:03:32.427 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lineage-pem.ko 2023-11-08T08:03:32.428 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 48601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm63.ko 2023-11-08T08:03:32.428 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm73.ko 2023-11-08T08:03:32.428 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 36777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm75.ko 2023-11-08T08:03:32.428 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm77.ko 2023-11-08T08:03:32.428 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm78.ko 2023-11-08T08:03:32.429 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 31745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm80.ko 2023-11-08T08:03:32.429 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm83.ko 2023-11-08T08:03:32.429 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 72833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm85.ko 2023-11-08T08:03:32.429 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 46153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm87.ko 2023-11-08T08:03:32.429 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 37913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm90.ko 2023-11-08T08:03:32.430 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm92.ko 2023-11-08T08:03:32.430 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 108841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm93.ko 2023-11-08T08:03:32.430 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm95234.ko 2023-11-08T08:03:32.430 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm95241.ko 2023-11-08T08:03:32.431 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/lm95245.ko 2023-11-08T08:03:32.431 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc2945.ko 2023-11-08T08:03:32.431 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4151.ko 2023-11-08T08:03:32.431 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4215.ko 2023-11-08T08:03:32.431 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4222.ko 2023-11-08T08:03:32.432 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4245.ko 2023-11-08T08:03:32.432 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4260.ko 2023-11-08T08:03:32.432 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/ltc4261.ko 2023-11-08T08:03:32.432 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max16065.ko 2023-11-08T08:03:32.432 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max1619.ko 2023-11-08T08:03:32.433 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max1668.ko 2023-11-08T08:03:32.433 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max197.ko 2023-11-08T08:03:32.433 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max6639.ko 2023-11-08T08:03:32.433 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max6650.ko 2023-11-08T08:03:32.433 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 35777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/max6697.ko 2023-11-08T08:03:32.434 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/mc13783-adc.ko 2023-11-08T08:03:32.434 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/mcp3021.ko 2023-11-08T08:03:32.434 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-11-08T08:03:32.434 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 44129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/nct6683.ko 2023-11-08T08:03:32.435 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 119817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/nct6775-core.ko 2023-11-08T08:03:32.435 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 60705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/nct6775.ko 2023-11-08T08:03:32.435 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 59737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/nct7802.ko 2023-11-08T08:03:32.435 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 82337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pc87360.ko 2023-11-08T08:03:32.435 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 54433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pc87427.ko 2023-11-08T08:03:32.436 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pcf8591.ko 2023-11-08T08:03:32.436 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/ 2023-11-08T08:03:32.436 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-08T08:03:32.436 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-08T08:03:32.436 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-08T08:03:32.437 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-08T08:03:32.437 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-08T08:03:32.437 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-08T08:03:32.437 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-08T08:03:32.438 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 82577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-08T08:03:32.438 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-08T08:03:32.438 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-08T08:03:32.438 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-08T08:03:32.438 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-08T08:03:32.439 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/powr1220.ko 2023-11-08T08:03:32.439 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sch5627.ko 2023-11-08T08:03:32.439 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sch5636.ko 2023-11-08T08:03:32.439 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-08T08:03:32.439 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 32137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sht15.ko 2023-11-08T08:03:32.440 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sht21.ko 2023-11-08T08:03:32.440 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/shtc1.ko 2023-11-08T08:03:32.440 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 40105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/sis5595.ko 2023-11-08T08:03:32.440 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/smsc47b397.ko 2023-11-08T08:03:32.441 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 39193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/smsc47m1.ko 2023-11-08T08:03:32.441 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 37385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/smsc47m192.ko 2023-11-08T08:03:32.441 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/thmc50.ko 2023-11-08T08:03:32.441 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/tmp102.ko 2023-11-08T08:03:32.441 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/tmp103.ko 2023-11-08T08:03:32.442 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/tmp401.ko 2023-11-08T08:03:32.442 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/tmp421.ko 2023-11-08T08:03:32.442 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/via-cputemp.ko 2023-11-08T08:03:32.442 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 41945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/via686a.ko 2023-11-08T08:03:32.443 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 52457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/vt1211.ko 2023-11-08T08:03:32.443 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 51545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/vt8231.ko 2023-11-08T08:03:32.443 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 57513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83627ehf.ko 2023-11-08T08:03:32.443 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 78873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83627hf.ko 2023-11-08T08:03:32.443 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 71345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83781d.ko 2023-11-08T08:03:32.444 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 63289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83791d.ko 2023-11-08T08:03:32.444 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 71321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83792d.ko 2023-11-08T08:03:32.444 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 101473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83793.ko 2023-11-08T08:03:32.444 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 119505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83795.ko 2023-11-08T08:03:32.445 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83l785ts.ko 2023-11-08T08:03:32.445 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 35625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/w83l786ng.ko 2023-11-08T08:03:32.445 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-11-08T08:03:32.445 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-11-08T08:03:32.446 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/ 2023-11-08T08:03:32.446 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/algos/ 2023-11-08T08:03:32.446 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-08T08:03:32.446 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-08T08:03:32.446 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/ 2023-11-08T08:03:32.447 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-11-08T08:03:32.447 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-11-08T08:03:32.447 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-11-08T08:03:32.447 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-08T08:03:32.448 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-08T08:03:32.448 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-08T08:03:32.448 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-11-08T08:03:32.448 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 6929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-08T08:03:32.448 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-11-08T08:03:32.449 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 50201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-08T08:03:32.449 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-08T08:03:32.449 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-08T08:03:32.449 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-08T08:03:32.450 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-11-08T08:03:32.450 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-11-08T08:03:32.450 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 62817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-08T08:03:32.450 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-08T08:03:32.450 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 47889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-08T08:03:32.451 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-11-08T08:03:32.451 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-08T08:03:32.451 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-08T08:03:32.451 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-11-08T08:03:32.452 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-08T08:03:32.452 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-08T08:03:32.452 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 47241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-08T08:03:32.452 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-11-08T08:03:32.452 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-08T08:03:32.453 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-08T08:03:32.453 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-11-08T08:03:32.453 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-11-08T08:03:32.453 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-08T08:03:32.453 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-11-08T08:03:32.454 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-08T08:03:32.454 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-08T08:03:32.454 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-08T08:03:32.454 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-08T08:03:32.455 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 47657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-11-08T08:03:32.455 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/i2c-mux.ko 2023-11-08T08:03:32.455 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/i2c-smbus.ko 2023-11-08T08:03:32.455 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/i2c-stub.ko 2023-11-08T08:03:32.455 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/muxes/ 2023-11-08T08:03:32.456 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-11-08T08:03:32.456 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-11-08T08:03:32.456 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-11-08T08:03:32.456 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ 2023-11-08T08:03:32.456 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/ 2023-11-08T08:03:32.457 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 273609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-08T08:03:32.457 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 879969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/ib_core.ko 2023-11-08T08:03:32.457 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 69337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-08T08:03:32.457 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 324217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-08T08:03:32.458 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 97985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-08T08:03:32.458 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 282273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-08T08:03:32.458 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 66201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-08T08:03:32.458 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/ 2023-11-08T08:03:32.458 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/cxgb4/ 2023-11-08T08:03:32.459 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 493073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-08T08:03:32.459 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mlx4/ 2023-11-08T08:03:32.459 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 419737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-08T08:03:32.459 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mlx5/ 2023-11-08T08:03:32.459 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 739025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-08T08:03:32.460 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mthca/ 2023-11-08T08:03:32.460 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 296625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-08T08:03:32.460 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/ocrdma/ 2023-11-08T08:03:32.460 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 153697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-11-08T08:03:32.461 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/usnic/ 2023-11-08T08:03:32.461 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 168969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-11-08T08:03:32.461 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/ 2023-11-08T08:03:32.461 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/ipoib/ 2023-11-08T08:03:32.461 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 263081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-08T08:03:32.462 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/iser/ 2023-11-08T08:03:32.462 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 100241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-08T08:03:32.462 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/isert/ 2023-11-08T08:03:32.462 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 111425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-08T08:03:32.463 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/srp/ 2023-11-08T08:03:32.463 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 128753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-08T08:03:32.463 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/srpt/ 2023-11-08T08:03:32.463 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 129529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-08T08:03:32.463 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/ 2023-11-08T08:03:32.464 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/evbug.ko 2023-11-08T08:03:32.464 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/ff-memless.ko 2023-11-08T08:03:32.464 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/gameport/ 2023-11-08T08:03:32.464 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-08T08:03:32.464 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-08T08:03:32.465 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 41241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/gameport/gameport.ko 2023-11-08T08:03:32.465 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 41801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/joydev.ko 2023-11-08T08:03:32.465 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/ 2023-11-08T08:03:32.465 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-11-08T08:03:32.465 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-11-08T08:03:32.466 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-11-08T08:03:32.466 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-11-08T08:03:32.466 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 36665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-08T08:03:32.466 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-11-08T08:03:32.466 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/lkkbd.ko 2023-11-08T08:03:32.467 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/lm8323.ko 2023-11-08T08:03:32.467 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/lm8333.ko 2023-11-08T08:03:32.467 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-11-08T08:03:32.467 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-11-08T08:03:32.468 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-11-08T08:03:32.468 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-11-08T08:03:32.468 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/newtonkbd.ko 2023-11-08T08:03:32.468 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-11-08T08:03:32.468 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/qt1070.ko 2023-11-08T08:03:32.469 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/qt2160.ko 2023-11-08T08:03:32.469 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-11-08T08:03:32.469 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/stowaway.ko 2023-11-08T08:03:32.469 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/sunkbd.ko 2023-11-08T08:03:32.469 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-11-08T08:03:32.470 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-11-08T08:03:32.470 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-11-08T08:03:32.470 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/keyboard/xtkbd.ko 2023-11-08T08:03:32.470 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/matrix-keymap.ko 2023-11-08T08:03:32.471 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/ 2023-11-08T08:03:32.471 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/appletouch.ko 2023-11-08T08:03:32.471 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/bcm5974.ko 2023-11-08T08:03:32.471 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 123681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/cyapatp.ko 2023-11-08T08:03:32.471 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 89449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-08T08:03:32.472 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/gpio_mouse.ko 2023-11-08T08:03:32.472 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 335977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/psmouse.ko 2023-11-08T08:03:32.472 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/sermouse.ko 2023-11-08T08:03:32.472 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-08T08:03:32.473 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-08T08:03:32.473 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-08T08:03:32.473 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/rmi4/ 2023-11-08T08:03:32.473 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 109873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-08T08:03:32.473 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/ 2023-11-08T08:03:32.474 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/altera_ps2.ko 2023-11-08T08:03:32.474 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/arc_ps2.ko 2023-11-08T08:03:32.474 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/ct82c710.ko 2023-11-08T08:03:32.474 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-08T08:03:32.474 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/parkbd.ko 2023-11-08T08:03:32.475 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/pcips2.ko 2023-11-08T08:03:32.475 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/ps2mult.ko 2023-11-08T08:03:32.475 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/serio_raw.ko 2023-11-08T08:03:32.475 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/serio/serport.ko 2023-11-08T08:03:32.476 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/input/sparse-keymap.ko 2023-11-08T08:03:32.476 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/iommu/ 2023-11-08T08:03:32.476 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/iommu/amd/ 2023-11-08T08:03:32.476 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 39289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-08T08:03:32.476 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/ 2023-11-08T08:03:32.477 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-88pm860x.ko 2023-11-08T08:03:32.477 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-adp5520.ko 2023-11-08T08:03:32.477 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 59753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-bd2802.ko 2023-11-08T08:03:32.477 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-blinkm.ko 2023-11-08T08:03:32.477 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-da903x.ko 2023-11-08T08:03:32.478 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-da9052.ko 2023-11-08T08:03:32.478 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-gpio.ko 2023-11-08T08:03:32.478 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lm3530.ko 2023-11-08T08:03:32.478 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lm3533.ko 2023-11-08T08:03:32.478 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lm355x.ko 2023-11-08T08:03:32.479 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lm3642.ko 2023-11-08T08:03:32.479 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lp3944.ko 2023-11-08T08:03:32.479 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-lp8788.ko 2023-11-08T08:03:32.479 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-max8997.ko 2023-11-08T08:03:32.479 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-mc13783.ko 2023-11-08T08:03:32.480 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-menf21bmc.ko 2023-11-08T08:03:32.480 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-pca9532.ko 2023-11-08T08:03:32.480 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-pca955x.ko 2023-11-08T08:03:32.480 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-pca963x.ko 2023-11-08T08:03:32.480 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-ss4200.ko 2023-11-08T08:03:32.481 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-tca6507.ko 2023-11-08T08:03:32.481 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-wm831x-status.ko 2023-11-08T08:03:32.481 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/leds-wm8350.ko 2023-11-08T08:03:32.481 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ 2023-11-08T08:03:32.482 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-08T08:03:32.482 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-08T08:03:32.482 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-08T08:03:32.482 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-08T08:03:32.482 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-08T08:03:32.483 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-08T08:03:32.483 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-08T08:03:32.483 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-08T08:03:32.483 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/ 2023-11-08T08:03:32.483 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/bcache/ 2023-11-08T08:03:32.484 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 654969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/bcache/bcache.ko 2023-11-08T08:03:32.484 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 40577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-bio-prison.ko 2023-11-08T08:03:32.484 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 89601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-bufio.ko 2023-11-08T08:03:32.484 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 47529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-cache-smq.ko 2023-11-08T08:03:32.485 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 156009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-cache.ko 2023-11-08T08:03:32.485 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 106897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-crypt.ko 2023-11-08T08:03:32.485 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-delay.ko 2023-11-08T08:03:32.485 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 53857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-era.ko 2023-11-08T08:03:32.485 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-flakey.ko 2023-11-08T08:03:32.486 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 41593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-log-userspace.ko 2023-11-08T08:03:32.486 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 32737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-log.ko 2023-11-08T08:03:32.486 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 49545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-mirror.ko 2023-11-08T08:03:32.486 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 78033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-multipath.ko 2023-11-08T08:03:32.486 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-queue-length.ko 2023-11-08T08:03:32.487 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 70825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-raid.ko 2023-11-08T08:03:32.487 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 36297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-region-hash.ko 2023-11-08T08:03:32.487 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-round-robin.ko 2023-11-08T08:03:32.487 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-service-time.ko 2023-11-08T08:03:32.488 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 109593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-snapshot.ko 2023-11-08T08:03:32.488 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-switch.ko 2023-11-08T08:03:32.488 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 181865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-thin-pool.ko 2023-11-08T08:03:32.488 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 51737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-verity.ko 2023-11-08T08:03:32.488 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/dm-zero.ko 2023-11-08T08:03:32.488 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/faulty.ko 2023-11-08T08:03:32.489 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/linear.ko 2023-11-08T08:03:32.489 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 32417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/multipath.ko 2023-11-08T08:03:32.489 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/persistent-data/ 2023-11-08T08:03:32.489 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 199937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-08T08:03:32.490 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 37337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/raid0.ko 2023-11-08T08:03:32.490 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 113073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/raid1.ko 2023-11-08T08:03:32.490 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 158681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/raid10.ko 2023-11-08T08:03:32.490 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 395721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/md/raid456.ko 2023-11-08T08:03:32.490 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/ 2023-11-08T08:03:32.491 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/ 2023-11-08T08:03:32.491 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 178137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptbase.ko 2023-11-08T08:03:32.491 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 63385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptctl.ko 2023-11-08T08:03:32.491 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 44665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptfc.ko 2023-11-08T08:03:32.492 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 37225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptlan.ko 2023-11-08T08:03:32.492 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 123321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptsas.ko 2023-11-08T08:03:32.492 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 79041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptscsih.ko 2023-11-08T08:03:32.492 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 53081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/message/fusion/mptspi.ko 2023-11-08T08:03:32.492 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/ 2023-11-08T08:03:32.493 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/88pm800.ko 2023-11-08T08:03:32.493 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/88pm805.ko 2023-11-08T08:03:32.493 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/88pm80x.ko 2023-11-08T08:03:32.493 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/arizona-i2c.ko 2023-11-08T08:03:32.493 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 123721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/arizona.ko 2023-11-08T08:03:32.494 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/bcm590xx.ko 2023-11-08T08:03:32.494 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/cros_ec_dev.ko 2023-11-08T08:03:32.494 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/dln2.ko 2023-11-08T08:03:32.494 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/janz-cmodio.ko 2023-11-08T08:03:32.495 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 53585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/kempld-core.ko 2023-11-08T08:03:32.495 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/lm3533-core.ko 2023-11-08T08:03:32.495 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-11-08T08:03:32.495 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/lp3943.ko 2023-11-08T08:03:32.495 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 54569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/lpc_ich.ko 2023-11-08T08:03:32.496 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/lpc_sch.ko 2023-11-08T08:03:32.496 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/max8907.ko 2023-11-08T08:03:32.496 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/mc13xxx-core.ko 2023-11-08T08:03:32.496 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-11-08T08:03:32.497 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/menf21bmc.ko 2023-11-08T08:03:32.497 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/pcf50633-adc.ko 2023-11-08T08:03:32.497 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/pcf50633-gpio.ko 2023-11-08T08:03:32.497 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/pcf50633.ko 2023-11-08T08:03:32.497 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-11-08T08:03:32.498 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/retu-mfd.ko 2023-11-08T08:03:32.498 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 80425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/si476x-core.ko 2023-11-08T08:03:32.498 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 48457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/sm501.ko 2023-11-08T08:03:32.498 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-11-08T08:03:32.499 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 51137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/tps65010.ko 2023-11-08T08:03:32.499 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/tps6507x.ko 2023-11-08T08:03:32.499 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/viperboard.ko 2023-11-08T08:03:32.499 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/vx855.ko 2023-11-08T08:03:32.499 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/wl1273-core.ko 2023-11-08T08:03:32.500 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 44345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/mfd/wm8994.ko 2023-11-08T08:03:32.500 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ 2023-11-08T08:03:32.500 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-11-08T08:03:32.500 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 48489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ad525x_dpot.ko 2023-11-08T08:03:32.501 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/altera-stapl/ 2023-11-08T08:03:32.501 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 46585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-08T08:03:32.501 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/apds9802als.ko 2023-11-08T08:03:32.501 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 36729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/apds990x.ko 2023-11-08T08:03:32.501 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 45593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/bh1770glc.ko 2023-11-08T08:03:32.502 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/c2port/ 2023-11-08T08:03:32.502 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-11-08T08:03:32.502 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 35281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/c2port/core.ko 2023-11-08T08:03:32.502 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/cb710/ 2023-11-08T08:03:32.503 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/cb710/cb710.ko 2023-11-08T08:03:32.503 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ds1682.ko 2023-11-08T08:03:32.503 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/dummy-irq.ko 2023-11-08T08:03:32.503 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/echo/ 2023-11-08T08:03:32.503 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/echo/echo.ko 2023-11-08T08:03:32.504 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/eeprom/ 2023-11-08T08:03:32.504 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 37849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/eeprom/at24.ko 2023-11-08T08:03:32.504 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-08T08:03:32.504 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-08T08:03:32.505 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/eeprom/max6875.ko 2023-11-08T08:03:32.505 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/enclosure.ko 2023-11-08T08:03:32.505 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/genwqe/ 2023-11-08T08:03:32.505 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 152201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-11-08T08:03:32.505 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/hmc6352.ko 2023-11-08T08:03:32.506 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 31321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/hpilo.ko 2023-11-08T08:03:32.506 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ibmasm/ 2023-11-08T08:03:32.506 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 72129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-11-08T08:03:32.506 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ics932s401.ko 2023-11-08T08:03:32.506 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/isl29003.ko 2023-11-08T08:03:32.507 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/isl29020.ko 2023-11-08T08:03:32.507 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/lis3lv02d/ 2023-11-08T08:03:32.507 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 42513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-08T08:03:32.507 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-08T08:03:32.508 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/mei/ 2023-11-08T08:03:32.508 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 125209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/mei/mei-me.ko 2023-11-08T08:03:32.508 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 56521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/mei/mei-txe.ko 2023-11-08T08:03:32.508 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 310113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/mei/mei.ko 2023-11-08T08:03:32.508 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 31329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/phantom.ko 2023-11-08T08:03:32.509 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ti-st/ 2023-11-08T08:03:32.509 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 74265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/ti-st/st_drv.ko 2023-11-08T08:03:32.509 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/tifm_7xx1.ko 2023-11-08T08:03:32.509 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/tifm_core.ko 2023-11-08T08:03:32.510 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/tsl2550.ko 2023-11-08T08:03:32.510 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 48697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/vmw_balloon.ko 2023-11-08T08:03:32.510 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/vmw_vmci/ 2023-11-08T08:03:32.510 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 183609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-08T08:03:32.510 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ 2023-11-08T08:03:32.511 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/bonding/ 2023-11-08T08:03:32.511 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 490129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/bonding/bonding.ko 2023-11-08T08:03:32.511 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/ 2023-11-08T08:03:32.511 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/b53/ 2023-11-08T08:03:32.512 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 106033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-08T08:03:32.512 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 40297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-08T08:03:32.512 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-08T08:03:32.512 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/mv88e6xxx/ 2023-11-08T08:03:32.512 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 453737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-08T08:03:32.513 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/dummy.ko 2023-11-08T08:03:32.513 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/eql.ko 2023-11-08T08:03:32.513 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/ 2023-11-08T08:03:32.513 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/3com/ 2023-11-08T08:03:32.513 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 45785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-08T08:03:32.514 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 42105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-08T08:03:32.514 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 130553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-08T08:03:32.514 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 68849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-08T08:03:32.514 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/8390/ 2023-11-08T08:03:32.515 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 37649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-08T08:03:32.515 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 59689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-08T08:03:32.515 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-08T08:03:32.515 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 99113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-08T08:03:32.515 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/adaptec/ 2023-11-08T08:03:32.516 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 54521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-08T08:03:32.516 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/agere/ 2023-11-08T08:03:32.516 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 69201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-08T08:03:32.516 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/alteon/ 2023-11-08T08:03:32.516 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 59585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-08T08:03:32.517 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/altera/ 2023-11-08T08:03:32.517 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 41409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-08T08:03:32.517 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/amd/ 2023-11-08T08:03:32.517 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 54609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-08T08:03:32.517 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 49289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-08T08:03:32.518 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 87297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-08T08:03:32.518 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/ 2023-11-08T08:03:32.518 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/alx/ 2023-11-08T08:03:32.518 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 97337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-08T08:03:32.519 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-11-08T08:03:32.519 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 104673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-08T08:03:32.519 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-11-08T08:03:32.519 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 90257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-08T08:03:32.519 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atlx/ 2023-11-08T08:03:32.520 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 81569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-08T08:03:32.520 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 69729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-08T08:03:32.520 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/ 2023-11-08T08:03:32.520 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 78401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-08T08:03:32.521 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 176761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-08T08:03:32.521 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-11-08T08:03:32.521 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 1421497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-08T08:03:32.521 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 128569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-08T08:03:32.521 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/genet/ 2023-11-08T08:03:32.522 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 132089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-08T08:03:32.522 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 312081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-08T08:03:32.522 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/brocade/ 2023-11-08T08:03:32.522 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/brocade/bna/ 2023-11-08T08:03:32.522 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 323513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-08T08:03:32.523 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/ 2023-11-08T08:03:32.523 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-11-08T08:03:32.523 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 193497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-08T08:03:32.523 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-11-08T08:03:32.524 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 347417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-08T08:03:32.524 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-11-08T08:03:32.524 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 1044433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-08T08:03:32.524 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-11-08T08:03:32.524 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 166193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-08T08:03:32.524 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-11-08T08:03:32.525 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-08T08:03:32.525 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/cisco/ 2023-11-08T08:03:32.525 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/cisco/enic/ 2023-11-08T08:03:32.525 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 204345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-08T08:03:32.525 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/ 2023-11-08T08:03:32.526 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/ 2023-11-08T08:03:32.526 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 69449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-08T08:03:32.526 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 61489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-08T08:03:32.526 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 141225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-08T08:03:32.526 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 51497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-08T08:03:32.527 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 62817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-08T08:03:32.527 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 31369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-08T08:03:32.527 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dlink/ 2023-11-08T08:03:32.527 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 60873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-08T08:03:32.527 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 62833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-08T08:03:32.528 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 45721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/dnet.ko 2023-11-08T08:03:32.528 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/ec_bhf.ko 2023-11-08T08:03:32.528 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/emulex/ 2023-11-08T08:03:32.528 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/emulex/benet/ 2023-11-08T08:03:32.529 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 281137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-08T08:03:32.529 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 49457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/ethoc.ko 2023-11-08T08:03:32.529 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 51737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/fealnx.ko 2023-11-08T08:03:32.529 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/fujitsu/ 2023-11-08T08:03:32.529 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 45345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-11-08T08:03:32.530 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/ 2023-11-08T08:03:32.530 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 90465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-08T08:03:32.530 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/e1000/ 2023-11-08T08:03:32.530 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 295577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-08T08:03:32.530 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/e1000e/ 2023-11-08T08:03:32.531 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 584497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-08T08:03:32.531 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/fm10k/ 2023-11-08T08:03:32.531 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 246209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-08T08:03:32.531 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/i40e/ 2023-11-08T08:03:32.531 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 925697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-08T08:03:32.532 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/iavf/ 2023-11-08T08:03:32.532 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 302849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-08T08:03:32.532 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/igb/ 2023-11-08T08:03:32.532 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 514681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-08T08:03:32.533 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/igbvf/ 2023-11-08T08:03:32.533 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 93593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-08T08:03:32.533 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-11-08T08:03:32.533 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 734305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-08T08:03:32.533 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-11-08T08:03:32.534 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 151769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-08T08:03:32.534 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 86921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/jme.ko 2023-11-08T08:03:32.534 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/marvell/ 2023-11-08T08:03:32.534 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-08T08:03:32.534 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 102289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-08T08:03:32.535 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 122649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-08T08:03:32.535 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/ 2023-11-08T08:03:32.535 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-11-08T08:03:32.535 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 727601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-08T08:03:32.535 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 278937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-08T08:03:32.536 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-11-08T08:03:32.536 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-11-08T08:03:32.536 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 1113993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-08T08:03:32.536 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/micrel/ 2023-11-08T08:03:32.536 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 53105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-11-08T08:03:32.537 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 40993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-11-08T08:03:32.537 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-11-08T08:03:32.537 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 95665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-08T08:03:32.537 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/myricom/ 2023-11-08T08:03:32.537 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-11-08T08:03:32.538 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 108649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-08T08:03:32.538 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/natsemi/ 2023-11-08T08:03:32.538 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 73441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-08T08:03:32.538 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 47777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-08T08:03:32.539 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/neterion/ 2023-11-08T08:03:32.539 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 150905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-08T08:03:32.539 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/nvidia/ 2023-11-08T08:03:32.539 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 136721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-08T08:03:32.539 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/packetengines/ 2023-11-08T08:03:32.540 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 50481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-08T08:03:32.540 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 43145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-08T08:03:32.540 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/ 2023-11-08T08:03:32.540 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-11-08T08:03:32.541 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 227897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-08T08:03:32.541 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 87081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-08T08:03:32.541 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-11-08T08:03:32.541 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 672049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-08T08:03:32.541 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/rdc/ 2023-11-08T08:03:32.542 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 43017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-08T08:03:32.542 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/realtek/ 2023-11-08T08:03:32.542 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 67385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-08T08:03:32.542 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 83985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-08T08:03:32.542 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-08T08:03:32.543 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 189801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-08T08:03:32.543 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/samsung/ 2023-11-08T08:03:32.543 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-11-08T08:03:32.543 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 104241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-11-08T08:03:32.544 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sfc/ 2023-11-08T08:03:32.544 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 1053921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-08T08:03:32.544 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/silan/ 2023-11-08T08:03:32.544 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 47097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-08T08:03:32.544 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sis/ 2023-11-08T08:03:32.545 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 54817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-08T08:03:32.545 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 63337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-08T08:03:32.545 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/smsc/ 2023-11-08T08:03:32.545 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 61601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-08T08:03:32.546 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 67193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-08T08:03:32.546 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 73409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-08T08:03:32.546 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 68177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-08T08:03:32.546 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/ 2023-11-08T08:03:32.546 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-11-08T08:03:32.547 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-11-08T08:03:32.547 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 39905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-08T08:03:32.547 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-11-08T08:03:32.547 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 486961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-08T08:03:32.547 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sun/ 2023-11-08T08:03:32.548 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 103337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-08T08:03:32.548 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 166161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-08T08:03:32.548 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 66329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-08T08:03:32.548 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 78801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-08T08:03:32.549 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/tehuti/ 2023-11-08T08:03:32.549 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 50057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-08T08:03:32.549 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/ti/ 2023-11-08T08:03:32.549 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 86529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-08T08:03:32.549 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/via/ 2023-11-08T08:03:32.550 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 80273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-08T08:03:32.550 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 83633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-08T08:03:32.550 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/wiznet/ 2023-11-08T08:03:32.550 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 46241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-08T08:03:32.551 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-08T08:03:32.551 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/xircom/ 2023-11-08T08:03:32.551 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 60057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-08T08:03:32.551 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/fddi/ 2023-11-08T08:03:32.552 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 41065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/fddi/defxx.ko 2023-11-08T08:03:32.552 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/fddi/skfp/ 2023-11-08T08:03:32.552 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 216985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/fddi/skfp/skfp.ko 2023-11-08T08:03:32.552 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 77345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/geneve.ko 2023-11-08T08:03:32.552 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/hyperv/ 2023-11-08T08:03:32.553 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 235521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-08T08:03:32.553 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ieee802154/ 2023-11-08T08:03:32.553 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-08T08:03:32.553 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ifb.ko 2023-11-08T08:03:32.553 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ipvlan/ 2023-11-08T08:03:32.554 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 83769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-08T08:03:32.554 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 62209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/macvlan.ko 2023-11-08T08:03:32.554 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/macvtap.ko 2023-11-08T08:03:32.554 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio/ 2023-11-08T08:03:32.555 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-08T08:03:32.555 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-08T08:03:32.555 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio/mdio-gpio.ko 2023-11-08T08:03:32.555 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-08T08:03:32.555 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mdio.ko 2023-11-08T08:03:32.556 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/mii.ko 2023-11-08T08:03:32.556 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 43953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/netconsole.ko 2023-11-08T08:03:32.556 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/nlmon.ko 2023-11-08T08:03:32.556 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/pcs/ 2023-11-08T08:03:32.557 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-08T08:03:32.557 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 38977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-08T08:03:32.557 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/ 2023-11-08T08:03:32.557 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/amd.ko 2023-11-08T08:03:32.557 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 50097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-08T08:03:32.558 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 44505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-08T08:03:32.558 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/bcm87xx.ko 2023-11-08T08:03:32.558 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 42241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/broadcom.ko 2023-11-08T08:03:32.558 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/cicada.ko 2023-11-08T08:03:32.558 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/davicom.ko 2023-11-08T08:03:32.559 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/et1011c.ko 2023-11-08T08:03:32.559 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/icplus.ko 2023-11-08T08:03:32.559 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/lxt.ko 2023-11-08T08:03:32.559 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 82049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/marvell.ko 2023-11-08T08:03:32.560 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 76177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/micrel.ko 2023-11-08T08:03:32.560 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/national.ko 2023-11-08T08:03:32.560 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 97857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/phylink.ko 2023-11-08T08:03:32.560 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/qsemi.ko 2023-11-08T08:03:32.560 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 50065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/realtek.ko 2023-11-08T08:03:32.561 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/smsc.ko 2023-11-08T08:03:32.561 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/ste10Xp.ko 2023-11-08T08:03:32.561 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/phy/vitesse.ko 2023-11-08T08:03:32.561 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/plip/ 2023-11-08T08:03:32.562 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 43569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/plip/plip.ko 2023-11-08T08:03:32.562 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/ 2023-11-08T08:03:32.562 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-08T08:03:32.562 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/ppp_async.ko 2023-11-08T08:03:32.562 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-08T08:03:32.563 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-08T08:03:32.563 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-08T08:03:32.563 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 42545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/pppoe.ko 2023-11-08T08:03:32.563 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/pppox.ko 2023-11-08T08:03:32.563 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 32689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/ppp/pptp.ko 2023-11-08T08:03:32.564 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/sb1000.ko 2023-11-08T08:03:32.564 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/slip/ 2023-11-08T08:03:32.564 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 42465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/slip/slip.ko 2023-11-08T08:03:32.564 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/sungem_phy.ko 2023-11-08T08:03:32.565 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 64385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/tap.ko 2023-11-08T08:03:32.565 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/ 2023-11-08T08:03:32.565 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 106169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team.ko 2023-11-08T08:03:32.565 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-08T08:03:32.565 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-08T08:03:32.566 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-08T08:03:32.566 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team_mode_random.ko 2023-11-08T08:03:32.566 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-08T08:03:32.566 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 87017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/veth.ko 2023-11-08T08:03:32.566 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/vmxnet3/ 2023-11-08T08:03:32.567 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 144297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-08T08:03:32.567 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/vxlan/ 2023-11-08T08:03:32.567 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 297329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/vxlan/vxlan.ko 2023-11-08T08:03:32.567 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/xen-netback/ 2023-11-08T08:03:32.567 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 144649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-08T08:03:32.568 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ntb/ 2023-11-08T08:03:32.568 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ntb/ntb.ko 2023-11-08T08:03:32.568 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/nvme/ 2023-11-08T08:03:32.568 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/nvme/host/ 2023-11-08T08:03:32.569 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 264937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/nvme/host/nvme-core.ko 2023-11-08T08:03:32.569 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 102209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/nvme/host/nvme.ko 2023-11-08T08:03:32.569 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/parport/ 2023-11-08T08:03:32.569 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 133841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/parport/parport.ko 2023-11-08T08:03:32.569 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/parport/parport_cs.ko 2023-11-08T08:03:32.570 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 84465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/parport/parport_pc.ko 2023-11-08T08:03:32.570 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/parport/parport_serial.ko 2023-11-08T08:03:32.570 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/ 2023-11-08T08:03:32.570 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/hotplug/ 2023-11-08T08:03:32.570 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-08T08:03:32.571 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-11-08T08:03:32.571 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-11-08T08:03:32.571 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/pci-stub.ko 2023-11-08T08:03:32.571 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 46529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pci/xen-pcifront.ko 2023-11-08T08:03:32.571 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/ 2023-11-08T08:03:32.572 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/i82092.ko 2023-11-08T08:03:32.572 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 145809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/pcmcia.ko 2023-11-08T08:03:32.572 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 62137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/pcmcia_core.ko 2023-11-08T08:03:32.572 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-11-08T08:03:32.573 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/pd6729.ko 2023-11-08T08:03:32.573 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 112889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-08T08:03:32.573 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/phy/ 2023-11-08T08:03:32.573 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/phy/broadcom/ 2023-11-08T08:03:32.573 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-11-08T08:03:32.574 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/phy/samsung/ 2023-11-08T08:03:32.574 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-11-08T08:03:32.574 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pinctrl/ 2023-11-08T08:03:32.574 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pinctrl/intel/ 2023-11-08T08:03:32.574 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 70153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-08T08:03:32.575 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/ 2023-11-08T08:03:32.575 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/ 2023-11-08T08:03:32.575 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 50889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-11-08T08:03:32.575 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_ec.ko 2023-11-08T08:03:32.575 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-11-08T08:03:32.576 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-11-08T08:03:32.576 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-11-08T08:03:32.576 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-11-08T08:03:32.576 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-11-08T08:03:32.577 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/ 2023-11-08T08:03:32.577 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 67097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-08T08:03:32.577 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 39145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/acerhdf.ko 2023-11-08T08:03:32.577 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 36913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-08T08:03:32.577 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 59665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-08T08:03:32.578 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 31097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-08T08:03:32.578 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 127569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-08T08:03:32.578 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 36913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-08T08:03:32.578 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 49121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-08T08:03:32.578 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-08T08:03:32.579 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 40265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-08T08:03:32.579 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-08T08:03:32.579 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 36657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/hdaps.ko 2023-11-08T08:03:32.579 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/ibm_rtl.ko 2023-11-08T08:03:32.580 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/intel/ 2023-11-08T08:03:32.580 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-08T08:03:32.580 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-11-08T08:03:32.580 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 43497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/intel_ips.ko 2023-11-08T08:03:32.580 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-08T08:03:32.581 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-08T08:03:32.581 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-08T08:03:32.581 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 42329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-08T08:03:32.581 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-08T08:03:32.581 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-08T08:03:32.582 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-08T08:03:32.582 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-08T08:03:32.582 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-08T08:03:32.582 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 57569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/platform/x86/wmi.ko 2023-11-08T08:03:32.583 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/ 2023-11-08T08:03:32.583 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/ 2023-11-08T08:03:32.583 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/88pm860x_battery.ko 2023-11-08T08:03:32.583 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/88pm860x_charger.ko 2023-11-08T08:03:32.583 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 55057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/bq2415x_charger.ko 2023-11-08T08:03:32.584 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 54081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/bq24190_charger.ko 2023-11-08T08:03:32.584 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/bq24735-charger.ko 2023-11-08T08:03:32.584 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-11-08T08:03:32.584 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/da9030_battery.ko 2023-11-08T08:03:32.584 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/da9052-battery.ko 2023-11-08T08:03:32.585 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/ds2760_battery.ko 2023-11-08T08:03:32.585 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/ds2780_battery.ko 2023-11-08T08:03:32.585 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/ds2781_battery.ko 2023-11-08T08:03:32.585 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/ds2782_battery.ko 2023-11-08T08:03:32.585 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/gpio-charger.ko 2023-11-08T08:03:32.586 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/isp1704_charger.ko 2023-11-08T08:03:32.586 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/lp8727_charger.ko 2023-11-08T08:03:32.586 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/max14577_charger.ko 2023-11-08T08:03:32.586 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/max17040_battery.ko 2023-11-08T08:03:32.586 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/max17042_battery.ko 2023-11-08T08:03:32.587 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/max8903_charger.ko 2023-11-08T08:03:32.587 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/max8925_power.ko 2023-11-08T08:03:32.587 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/pcf50633-charger.ko 2023-11-08T08:03:32.587 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 38009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/sbs-battery.ko 2023-11-08T08:03:32.587 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/test_power.ko 2023-11-08T08:03:32.588 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/tps65090-charger.ko 2023-11-08T08:03:32.588 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/wm831x_backup.ko 2023-11-08T08:03:32.588 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/wm831x_power.ko 2023-11-08T08:03:32.588 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/power/supply/wm8350_power.ko 2023-11-08T08:03:32.589 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/powercap/ 2023-11-08T08:03:32.589 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 57729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-08T08:03:32.589 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-08T08:03:32.589 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/ 2023-11-08T08:03:32.589 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/clients/ 2023-11-08T08:03:32.590 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-08T08:03:32.590 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-08T08:03:32.590 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/clients/pps_parport.ko 2023-11-08T08:03:32.590 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 41873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/pps/pps_core.ko 2023-11-08T08:03:32.591 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ptp/ 2023-11-08T08:03:32.591 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 66385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ptp/ptp.ko 2023-11-08T08:03:32.591 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ptp/ptp_kvm.ko 2023-11-08T08:03:32.591 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/ 2023-11-08T08:03:32.591 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-88pm80x.ko 2023-11-08T08:03:32.592 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-88pm860x.ko 2023-11-08T08:03:32.592 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-08T08:03:32.592 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-da9052.ko 2023-11-08T08:03:32.592 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-da9055.ko 2023-11-08T08:03:32.593 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-da9063.ko 2023-11-08T08:03:32.593 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-08T08:03:32.593 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 68577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-08T08:03:32.593 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-08T08:03:32.594 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-08T08:03:32.594 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-08T08:03:32.594 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-08T08:03:32.594 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-08T08:03:32.594 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-08T08:03:32.595 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-08T08:03:32.595 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-em3027.ko 2023-11-08T08:03:32.595 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-08T08:03:32.595 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-08T08:03:32.596 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 35969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-08T08:03:32.596 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-lp8788.ko 2023-11-08T08:03:32.596 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 38097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-08T08:03:32.596 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-08T08:03:32.596 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-08T08:03:32.597 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-m48t86.ko 2023-11-08T08:03:32.597 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-max6900.ko 2023-11-08T08:03:32.597 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-max8907.ko 2023-11-08T08:03:32.597 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-max8925.ko 2023-11-08T08:03:32.598 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-max8997.ko 2023-11-08T08:03:32.598 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-max8998.ko 2023-11-08T08:03:32.598 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-11-08T08:03:32.598 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-08T08:03:32.598 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-palmas.ko 2023-11-08T08:03:32.599 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 46633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-08T08:03:32.599 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf50633.ko 2023-11-08T08:03:32.599 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-08T08:03:32.599 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-08T08:03:32.600 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-08T08:03:32.600 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-08T08:03:32.600 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rc5t583.ko 2023-11-08T08:03:32.600 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-08T08:03:32.600 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-08T08:03:32.601 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-08T08:03:32.601 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-08T08:03:32.601 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-08T08:03:32.601 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-s35390a.ko 2023-11-08T08:03:32.601 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-08T08:03:32.602 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-tps6586x.ko 2023-11-08T08:03:32.602 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-tps65910.ko 2023-11-08T08:03:32.602 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-wm831x.ko 2023-11-08T08:03:32.602 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-wm8350.ko 2023-11-08T08:03:32.603 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/rtc/rtc-x1205.ko 2023-11-08T08:03:32.603 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ 2023-11-08T08:03:32.603 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 77617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/3w-9xxx.ko 2023-11-08T08:03:32.603 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 57665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/3w-sas.ko 2023-11-08T08:03:32.603 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 55169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/3w-xxxx.ko 2023-11-08T08:03:32.604 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 135185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/BusLogic.ko 2023-11-08T08:03:32.604 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/a100u2w.ko 2023-11-08T08:03:32.604 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aacraid/ 2023-11-08T08:03:32.604 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 272305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-08T08:03:32.604 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 100617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/advansys.ko 2023-11-08T08:03:32.605 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aic7xxx/ 2023-11-08T08:03:32.605 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 356265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-08T08:03:32.605 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 295801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-08T08:03:32.605 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aic94xx/ 2023-11-08T08:03:32.606 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 189185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-11-08T08:03:32.606 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/am53c974.ko 2023-11-08T08:03:32.606 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/arcmsr/ 2023-11-08T08:03:32.606 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 106881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-08T08:03:32.606 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 38185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/atp870u.ko 2023-11-08T08:03:32.607 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/be2iscsi/ 2023-11-08T08:03:32.607 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 246985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-08T08:03:32.607 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bfa/ 2023-11-08T08:03:32.607 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 1171945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bfa/bfa.ko 2023-11-08T08:03:32.608 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bnx2fc/ 2023-11-08T08:03:32.608 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 237273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-08T08:03:32.608 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bnx2i/ 2023-11-08T08:03:32.608 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 115369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-08T08:03:32.608 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 40585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ch.ko 2023-11-08T08:03:32.609 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/csiostor/ 2023-11-08T08:03:32.609 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 308921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-08T08:03:32.609 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/ 2023-11-08T08:03:32.609 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-11-08T08:03:32.609 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 76145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-08T08:03:32.610 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-11-08T08:03:32.610 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 101449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-08T08:03:32.610 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 146689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-08T08:03:32.610 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 75553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/dc395x.ko 2023-11-08T08:03:32.611 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 38137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/dmx3191d.ko 2023-11-08T08:03:32.611 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/esas2r/ 2023-11-08T08:03:32.611 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 212425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-08T08:03:32.611 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 60505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/esp_scsi.ko 2023-11-08T08:03:32.611 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fcoe/ 2023-11-08T08:03:32.612 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 78209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-08T08:03:32.612 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 158505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-08T08:03:32.612 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fdomain.ko 2023-11-08T08:03:32.612 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fnic/ 2023-11-08T08:03:32.612 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 263489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/fnic/fnic.ko 2023-11-08T08:03:32.613 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 201817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/hpsa.ko 2023-11-08T08:03:32.613 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 51209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/hptiop.ko 2023-11-08T08:03:32.613 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 49041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/hv_storvsc.ko 2023-11-08T08:03:32.613 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 35337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/imm.ko 2023-11-08T08:03:32.613 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 44057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/initio.ko 2023-11-08T08:03:32.614 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 266537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ipr.ko 2023-11-08T08:03:32.614 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 101881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ips.ko 2023-11-08T08:03:32.614 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/isci/ 2023-11-08T08:03:32.614 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 317153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/isci/isci.ko 2023-11-08T08:03:32.615 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-08T08:03:32.615 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 52961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-08T08:03:32.615 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libfc/ 2023-11-08T08:03:32.615 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 314865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libfc/libfc.ko 2023-11-08T08:03:32.615 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 145585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libiscsi.ko 2023-11-08T08:03:32.616 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 53521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-08T08:03:32.616 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libsas/ 2023-11-08T08:03:32.616 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 216665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/libsas/libsas.ko 2023-11-08T08:03:32.616 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/lpfc/ 2023-11-08T08:03:32.616 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 2197249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-08T08:03:32.617 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/megaraid/ 2023-11-08T08:03:32.617 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 73569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-08T08:03:32.617 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-08T08:03:32.617 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 310617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-08T08:03:32.618 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 84721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/megaraid.ko 2023-11-08T08:03:32.618 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/mpt3sas/ 2023-11-08T08:03:32.618 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 659777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-08T08:03:32.618 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/mvsas/ 2023-11-08T08:03:32.618 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 119561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-08T08:03:32.619 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 78129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/mvumi.ko 2023-11-08T08:03:32.619 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pcmcia/ 2023-11-08T08:03:32.619 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 81625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-11-08T08:03:32.619 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-11-08T08:03:32.620 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-11-08T08:03:32.620 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-11-08T08:03:32.620 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pm8001/ 2023-11-08T08:03:32.620 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 364105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-08T08:03:32.620 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 123617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/pmcraid.ko 2023-11-08T08:03:32.621 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ppa.ko 2023-11-08T08:03:32.621 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 55361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla1280.ko 2023-11-08T08:03:32.621 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla2xxx/ 2023-11-08T08:03:32.621 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 1876625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-08T08:03:32.621 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 85289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-08T08:03:32.622 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla4xxx/ 2023-11-08T08:03:32.622 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 559033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-08T08:03:32.622 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/qlogicfas408.ko 2023-11-08T08:03:32.622 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/raid_class.ko 2023-11-08T08:03:32.622 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 238825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_debug.ko 2023-11-08T08:03:32.623 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 173937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-08T08:03:32.623 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 279313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-08T08:03:32.623 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 97729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-08T08:03:32.623 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 73881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-08T08:03:32.624 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 44993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-08T08:03:32.624 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/ses.ko 2023-11-08T08:03:32.624 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 115889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/st.ko 2023-11-08T08:03:32.624 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 46825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/stex.ko 2023-11-08T08:03:32.624 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/sym53c8xx_2/ 2023-11-08T08:03:32.625 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 162121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-08T08:03:32.625 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 42297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/virtio_scsi.ko 2023-11-08T08:03:32.625 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 50321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-08T08:03:32.625 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 40025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/wd719x.ko 2023-11-08T08:03:32.625 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 50025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/scsi/xen-scsifront.ko 2023-11-08T08:03:32.626 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ssb/ 2023-11-08T08:03:32.626 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 129441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ssb/ssb.ko 2023-11-08T08:03:32.626 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/ 2023-11-08T08:03:32.626 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/iscsi/ 2023-11-08T08:03:32.626 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 736601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-08T08:03:32.627 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/loopback/ 2023-11-08T08:03:32.627 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 49985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-08T08:03:32.627 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 39713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/target_core_file.ko 2023-11-08T08:03:32.627 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 43025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/target_core_iblock.ko 2023-11-08T08:03:32.627 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 930321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/target_core_mod.ko 2023-11-08T08:03:32.628 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 44305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/target_core_pscsi.ko 2023-11-08T08:03:32.628 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 122521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/target_core_user.ko 2023-11-08T08:03:32.628 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/tcm_fc/ 2023-11-08T08:03:32.628 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 75593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-08T08:03:32.629 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/ 2023-11-08T08:03:32.629 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/ 2023-11-08T08:03:32.629 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/ 2023-11-08T08:03:32.629 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-08T08:03:32.629 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-08T08:03:32.630 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-08T08:03:32.630 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-08T08:03:32.630 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-08T08:03:32.630 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-08T08:03:32.630 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-08T08:03:32.631 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-08T08:03:32.631 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-08T08:03:32.631 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-08T08:03:32.631 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-08T08:03:32.631 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 37065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-08T08:03:32.632 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-11-08T08:03:32.632 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-08T08:03:32.632 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-08T08:03:32.632 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-08T08:03:32.633 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/ 2023-11-08T08:03:32.633 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/ipwireless/ 2023-11-08T08:03:32.633 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 78425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-08T08:03:32.633 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 57865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/moxa.ko 2023-11-08T08:03:32.633 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 50953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/mxser.ko 2023-11-08T08:03:32.634 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 108601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/n_gsm.ko 2023-11-08T08:03:32.634 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 31961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/n_hdlc.ko 2023-11-08T08:03:32.634 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 67929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/nozomi.ko 2023-11-08T08:03:32.634 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/ 2023-11-08T08:03:32.634 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/8250/ 2023-11-08T08:03:32.635 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 35753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-11-08T08:03:32.635 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 59497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-08T08:03:32.635 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-11-08T08:03:32.635 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/altera_uart.ko 2023-11-08T08:03:32.636 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/arc_uart.ko 2023-11-08T08:03:32.636 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 72425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-11-08T08:03:32.636 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/jsm/ 2023-11-08T08:03:32.636 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 103073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-08T08:03:32.636 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/rp2.ko 2023-11-08T08:03:32.637 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 52817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/tty/serial/sc16is7xx.ko 2023-11-08T08:03:32.637 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/ 2023-11-08T08:03:32.637 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/core/ 2023-11-08T08:03:32.637 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 531265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-08T08:03:32.637 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/host/ 2023-11-08T08:03:32.638 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-08T08:03:32.638 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-11-08T08:03:32.638 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/ 2023-11-08T08:03:32.638 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 44513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio.ko 2023-11-08T08:03:32.639 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_aec.ko 2023-11-08T08:03:32.639 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_cif.ko 2023-11-08T08:03:32.639 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_dmem_genirq.ko 2023-11-08T08:03:32.639 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_mf624.ko 2023-11-08T08:03:32.639 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_netx.ko 2023-11-08T08:03:32.640 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_pci_generic.ko 2023-11-08T08:03:32.640 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-11-08T08:03:32.640 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/uio/uio_sercos3.ko 2023-11-08T08:03:32.640 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/ 2023-11-08T08:03:32.640 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/c67x00/ 2023-11-08T08:03:32.641 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 73297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/c67x00/c67x00.ko 2023-11-08T08:03:32.641 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/chipidea/ 2023-11-08T08:03:32.641 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 144377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-11-08T08:03:32.641 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-11-08T08:03:32.642 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-11-08T08:03:32.642 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-11-08T08:03:32.642 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/class/ 2023-11-08T08:03:32.642 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 87089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/class/cdc-acm.ko 2023-11-08T08:03:32.642 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 51409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-08T08:03:32.643 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 47369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/class/usblp.ko 2023-11-08T08:03:32.643 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 89857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/class/usbtmc.ko 2023-11-08T08:03:32.643 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/common/ 2023-11-08T08:03:32.643 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/common/ulpi.ko 2023-11-08T08:03:32.643 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc2/ 2023-11-08T08:03:32.644 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-11-08T08:03:32.644 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc3/ 2023-11-08T08:03:32.644 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-11-08T08:03:32.644 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-11-08T08:03:32.644 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 444553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/dwc3/dwc3.ko 2023-11-08T08:03:32.645 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/ 2023-11-08T08:03:32.645 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/ 2023-11-08T08:03:32.645 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 60177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/u_ether.ko 2023-11-08T08:03:32.645 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 51881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/u_serial.ko 2023-11-08T08:03:32.645 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-11-08T08:03:32.646 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 42449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-11-08T08:03:32.646 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-11-08T08:03:32.646 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 36513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-11-08T08:03:32.646 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 92489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-11-08T08:03:32.646 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 51049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-11-08T08:03:32.647 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 122081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-11-08T08:03:32.647 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 57617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-11-08T08:03:32.647 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-11-08T08:03:32.647 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 31265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-11-08T08:03:32.648 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 57729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-11-08T08:03:32.648 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 85609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-11-08T08:03:32.648 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-11-08T08:03:32.648 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 58113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-11-08T08:03:32.648 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 67337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-11-08T08:03:32.649 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/ 2023-11-08T08:03:32.649 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-11-08T08:03:32.649 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-11-08T08:03:32.649 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-11-08T08:03:32.649 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-11-08T08:03:32.650 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-11-08T08:03:32.650 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-11-08T08:03:32.650 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-11-08T08:03:32.650 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-11-08T08:03:32.650 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 32073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-11-08T08:03:32.651 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-11-08T08:03:32.651 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-11-08T08:03:32.651 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-11-08T08:03:32.651 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 62185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-11-08T08:03:32.652 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-11-08T08:03:32.652 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 161225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/libcomposite.ko 2023-11-08T08:03:32.652 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/ 2023-11-08T08:03:32.652 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-11-08T08:03:32.652 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/bdc/ 2023-11-08T08:03:32.653 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 157681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-11-08T08:03:32.653 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 36089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-11-08T08:03:32.653 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 52721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-11-08T08:03:32.653 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 69705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-11-08T08:03:32.653 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 56033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-11-08T08:03:32.654 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 76057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/net2272.ko 2023-11-08T08:03:32.654 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 75073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/net2280.ko 2023-11-08T08:03:32.654 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 73169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-11-08T08:03:32.654 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 65809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-11-08T08:03:32.655 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 66097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-11-08T08:03:32.655 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 81505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-11-08T08:03:32.655 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 156073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-11-08T08:03:32.655 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/ 2023-11-08T08:03:32.655 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/bcma-hcd.ko 2023-11-08T08:03:32.656 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 80113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/isp116x-hcd.ko 2023-11-08T08:03:32.656 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 87353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-11-08T08:03:32.656 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 78873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-11-08T08:03:32.656 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 51329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-08T08:03:32.656 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/sl811_cs.ko 2023-11-08T08:03:32.657 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/ssb-hcd.ko 2023-11-08T08:03:32.657 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-11-08T08:03:32.657 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/image/ 2023-11-08T08:03:32.657 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/image/mdc800.ko 2023-11-08T08:03:32.657 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/image/microtek.ko 2023-11-08T08:03:32.658 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/ 2023-11-08T08:03:32.658 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 41721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/adutux.ko 2023-11-08T08:03:32.658 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/appledisplay.ko 2023-11-08T08:03:32.658 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-11-08T08:03:32.659 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/cytherm.ko 2023-11-08T08:03:32.659 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/ehset.ko 2023-11-08T08:03:32.659 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/emi26.ko 2023-11-08T08:03:32.659 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/emi62.ko 2023-11-08T08:03:32.659 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/ezusb.ko 2023-11-08T08:03:32.660 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/idmouse.ko 2023-11-08T08:03:32.660 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 36617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/iowarrior.ko 2023-11-08T08:03:32.660 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-08T08:03:32.660 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 36529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/ldusb.ko 2023-11-08T08:03:32.661 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 36265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/legousbtower.ko 2023-11-08T08:03:32.661 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/lvstest.ko 2023-11-08T08:03:32.661 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/sisusbvga/ 2023-11-08T08:03:32.661 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 64841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-08T08:03:32.661 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-08T08:03:32.662 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/usb3503.ko 2023-11-08T08:03:32.662 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/usblcd.ko 2023-11-08T08:03:32.662 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-08T08:03:32.662 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 70873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/usbtest.ko 2023-11-08T08:03:32.663 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 41969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/uss720.ko 2023-11-08T08:03:32.663 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/misc/yurex.ko 2023-11-08T08:03:32.663 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/mon/ 2023-11-08T08:03:32.663 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 72953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/mon/usbmon.ko 2023-11-08T08:03:32.663 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/musb/ 2023-11-08T08:03:32.664 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 301545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/musb/musb_hdrc.ko 2023-11-08T08:03:32.664 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/phy/ 2023-11-08T08:03:32.664 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/phy/phy-generic.ko 2023-11-08T08:03:32.664 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-11-08T08:03:32.664 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/phy/phy-isp1301.ko 2023-11-08T08:03:32.665 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/phy/phy-tahvo.ko 2023-11-08T08:03:32.665 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ 2023-11-08T08:03:32.665 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/aircable.ko 2023-11-08T08:03:32.665 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ark3116.ko 2023-11-08T08:03:32.665 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-08T08:03:32.666 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 38105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ch341.ko 2023-11-08T08:03:32.666 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 70257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/cp210x.ko 2023-11-08T08:03:32.666 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 31057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/cyberjack.ko 2023-11-08T08:03:32.666 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 58217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-08T08:03:32.667 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 49817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-08T08:03:32.667 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/empeg.ko 2023-11-08T08:03:32.667 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 44337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/f81232.ko 2023-11-08T08:03:32.667 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 175137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-08T08:03:32.667 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 53113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-08T08:03:32.668 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 137081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-08T08:03:32.668 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 124833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/io_ti.ko 2023-11-08T08:03:32.668 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 67769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ipaq.ko 2023-11-08T08:03:32.668 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ipw.ko 2023-11-08T08:03:32.668 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ir-usb.ko 2023-11-08T08:03:32.669 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 58233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-08T08:03:32.669 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 107521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/keyspan.ko 2023-11-08T08:03:32.669 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 35617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-08T08:03:32.669 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-08T08:03:32.669 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-08T08:03:32.670 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 32953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/mct_u232.ko 2023-11-08T08:03:32.670 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/metro-usb.ko 2023-11-08T08:03:32.670 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 66449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/mos7720.ko 2023-11-08T08:03:32.670 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 78217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/mos7840.ko 2023-11-08T08:03:32.670 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 56337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/mxuport.ko 2023-11-08T08:03:32.671 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/navman.ko 2023-11-08T08:03:32.671 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/omninet.ko 2023-11-08T08:03:32.671 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/opticon.ko 2023-11-08T08:03:32.671 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 190633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/option.ko 2023-11-08T08:03:32.672 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 39073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/oti6858.ko 2023-11-08T08:03:32.672 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 54353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/pl2303.ko 2023-11-08T08:03:32.672 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/qcaux.ko 2023-11-08T08:03:32.672 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 37825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/qcserial.ko 2023-11-08T08:03:32.672 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/quatech2.ko 2023-11-08T08:03:32.673 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/safe_serial.ko 2023-11-08T08:03:32.673 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 51625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/sierra.ko 2023-11-08T08:03:32.673 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-08T08:03:32.673 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ssu100.ko 2023-11-08T08:03:32.673 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/symbolserial.ko 2023-11-08T08:03:32.674 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 61489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-08T08:03:32.674 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-08T08:03:32.674 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/usb_debug.ko 2023-11-08T08:03:32.674 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-08T08:03:32.675 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 125609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/usbserial.ko 2023-11-08T08:03:32.675 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/visor.ko 2023-11-08T08:03:32.675 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 37409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/whiteheat.ko 2023-11-08T08:03:32.675 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/wishbone-serial.ko 2023-11-08T08:03:32.675 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-08T08:03:32.676 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ 2023-11-08T08:03:32.676 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 52913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/uas.ko 2023-11-08T08:03:32.676 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-08T08:03:32.676 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-08T08:03:32.676 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-08T08:03:32.677 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-08T08:03:32.677 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-08T08:03:32.677 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-08T08:03:32.677 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-08T08:03:32.677 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-karma.ko 2023-11-08T08:03:32.678 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-08T08:03:32.678 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-08T08:03:32.678 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-08T08:03:32.678 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-08T08:03:32.678 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 35337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-08T08:03:32.679 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 179969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/storage/usb-storage.ko 2023-11-08T08:03:32.679 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/usbip/ 2023-11-08T08:03:32.679 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 67481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-08T08:03:32.679 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 72177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-08T08:03:32.680 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 107001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-08T08:03:32.680 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/ 2023-11-08T08:03:32.680 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/pci/ 2023-11-08T08:03:32.680 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 154297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-08T08:03:32.680 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-08T08:03:32.681 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 108737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/vfio.ko 2023-11-08T08:03:32.681 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 77609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-08T08:03:32.681 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vhost/ 2023-11-08T08:03:32.681 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 113873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vhost/vhost.ko 2023-11-08T08:03:32.681 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-08T08:03:32.682 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 58089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vhost/vhost_net.ko 2023-11-08T08:03:32.682 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 89249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/vhost/vhost_scsi.ko 2023-11-08T08:03:32.682 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/ 2023-11-08T08:03:32.682 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/ 2023-11-08T08:03:32.683 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-11-08T08:03:32.683 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/aat2870_bl.ko 2023-11-08T08:03:32.683 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/adp5520_bl.ko 2023-11-08T08:03:32.683 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 32609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/adp8860_bl.ko 2023-11-08T08:03:32.683 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 39993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/adp8870_bl.ko 2023-11-08T08:03:32.684 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/apple_bl.ko 2023-11-08T08:03:32.684 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/as3711_bl.ko 2023-11-08T08:03:32.684 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/bd6107.ko 2023-11-08T08:03:32.684 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/da903x_bl.ko 2023-11-08T08:03:32.684 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/da9052_bl.ko 2023-11-08T08:03:32.685 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/gpio_backlight.ko 2023-11-08T08:03:32.685 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/kb3886_bl.ko 2023-11-08T08:03:32.685 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/lcd.ko 2023-11-08T08:03:32.685 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/lm3533_bl.ko 2023-11-08T08:03:32.685 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/lm3639_bl.ko 2023-11-08T08:03:32.686 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/lv5207lp.ko 2023-11-08T08:03:32.686 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/max8925_bl.ko 2023-11-08T08:03:32.686 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/pandora_bl.ko 2023-11-08T08:03:32.686 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-11-08T08:03:32.687 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-08T08:03:32.687 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/video/backlight/wm831x_bl.ko 2023-11-08T08:03:32.687 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/ 2023-11-08T08:03:32.687 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/masters/ 2023-11-08T08:03:32.687 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/masters/ds2482.ko 2023-11-08T08:03:32.688 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 38985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/masters/ds2490.ko 2023-11-08T08:03:32.688 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/masters/matrox_w1.ko 2023-11-08T08:03:32.688 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/masters/w1-gpio.ko 2023-11-08T08:03:32.688 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/ 2023-11-08T08:03:32.688 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-08T08:03:32.689 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-08T08:03:32.689 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-08T08:03:32.689 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-08T08:03:32.689 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-08T08:03:32.689 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-08T08:03:32.690 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-08T08:03:32.690 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-08T08:03:32.690 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-08T08:03:32.690 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-08T08:03:32.690 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 56425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-08T08:03:32.691 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 102673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/w1/wire.ko 2023-11-08T08:03:32.691 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/ 2023-11-08T08:03:32.691 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/acquirewdt.ko 2023-11-08T08:03:32.691 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/advantechwdt.ko 2023-11-08T08:03:32.691 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-08T08:03:32.692 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-08T08:03:32.692 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/cpu5wdt.ko 2023-11-08T08:03:32.692 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/da9052_wdt.ko 2023-11-08T08:03:32.692 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/da9055_wdt.ko 2023-11-08T08:03:32.693 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/da9063_wdt.ko 2023-11-08T08:03:32.693 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/dw_wdt.ko 2023-11-08T08:03:32.693 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/eurotechwdt.ko 2023-11-08T08:03:32.693 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-08T08:03:32.693 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/hpwdt.ko 2023-11-08T08:03:32.694 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/i6300esb.ko 2023-11-08T08:03:32.694 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-08T08:03:32.694 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-08T08:03:32.694 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/ib700wdt.ko 2023-11-08T08:03:32.694 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/ibmasr.ko 2023-11-08T08:03:32.695 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-08T08:03:32.695 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-08T08:03:32.695 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/it87_wdt.ko 2023-11-08T08:03:32.695 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/kempld_wdt.ko 2023-11-08T08:03:32.695 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/machzwd.ko 2023-11-08T08:03:32.696 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/mena21_wdt.ko 2023-11-08T08:03:32.696 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-11-08T08:03:32.696 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/nv_tco.ko 2023-11-08T08:03:32.696 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-11-08T08:03:32.696 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22265 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/pc87413_wdt.ko 2023-11-08T08:03:32.697 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 38657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-08T08:03:32.697 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 31825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-08T08:03:32.697 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/retu_wdt.ko 2023-11-08T08:03:32.697 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-11-08T08:03:32.697 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-11-08T08:03:32.698 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-08T08:03:32.698 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sc1200wdt.ko 2023-11-08T08:03:32.698 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-08T08:03:32.698 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-11-08T08:03:32.699 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/softdog.ko 2023-11-08T08:03:32.699 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-08T08:03:32.699 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/twl4030_wdt.ko 2023-11-08T08:03:32.699 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/via_wdt.ko 2023-11-08T08:03:32.699 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-08T08:03:32.700 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-08T08:03:32.700 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-08T08:03:32.700 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/wafer5823wdt.ko 2023-11-08T08:03:32.700 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/wdt_pci.ko 2023-11-08T08:03:32.700 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/wm831x_wdt.ko 2023-11-08T08:03:32.701 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/wm8350_wdt.ko 2023-11-08T08:03:32.701 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/watchdog/xen_wdt.ko 2023-11-08T08:03:32.701 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/ 2023-11-08T08:03:32.701 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-evtchn.ko 2023-11-08T08:03:32.702 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-gntalloc.ko 2023-11-08T08:03:32.702 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 47865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-gntdev.ko 2023-11-08T08:03:32.702 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-pciback/ 2023-11-08T08:03:32.702 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 176441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-08T08:03:32.703 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 40121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-privcmd.ko 2023-11-08T08:03:32.703 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 71249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xen-scsiback.ko 2023-11-08T08:03:32.703 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xenfs/ 2023-11-08T08:03:32.703 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-08T08:03:32.703 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ 2023-11-08T08:03:32.704 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/9p/ 2023-11-08T08:03:32.704 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 166425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/9p/9p.ko 2023-11-08T08:03:32.704 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/adfs/ 2023-11-08T08:03:32.704 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 61073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/adfs/adfs.ko 2023-11-08T08:03:32.705 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/affs/ 2023-11-08T08:03:32.705 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 163593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/affs/affs.ko 2023-11-08T08:03:32.705 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/afs/ 2023-11-08T08:03:32.705 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 932817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/afs/kafs.ko 2023-11-08T08:03:32.705 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/autofs/ 2023-11-08T08:03:32.706 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 101057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/autofs/autofs4.ko 2023-11-08T08:03:32.706 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/befs/ 2023-11-08T08:03:32.706 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 64545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/befs/befs.ko 2023-11-08T08:03:32.706 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/bfs/ 2023-11-08T08:03:32.707 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 41257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/bfs/bfs.ko 2023-11-08T08:03:32.707 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/binfmt_misc.ko 2023-11-08T08:03:32.707 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/btrfs/ 2023-11-08T08:03:32.707 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 3402585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/btrfs/btrfs.ko 2023-11-08T08:03:32.707 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/cachefiles/ 2023-11-08T08:03:32.708 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 306073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/cachefiles/cachefiles.ko 2023-11-08T08:03:32.708 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ceph/ 2023-11-08T08:03:32.708 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 1323945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ceph/ceph.ko 2023-11-08T08:03:32.708 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/coda/ 2023-11-08T08:03:32.709 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 100801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/coda/coda.ko 2023-11-08T08:03:32.709 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/configfs/ 2023-11-08T08:03:32.709 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 108393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/configfs/configfs.ko 2023-11-08T08:03:32.709 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/cramfs/ 2023-11-08T08:03:32.709 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 29625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/cramfs/cramfs.ko 2023-11-08T08:03:32.710 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/dlm/ 2023-11-08T08:03:32.710 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 642577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/dlm/dlm.ko 2023-11-08T08:03:32.710 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/efs/ 2023-11-08T08:03:32.710 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 36649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/efs/efs.ko 2023-11-08T08:03:32.711 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/f2fs/ 2023-11-08T08:03:32.711 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 1505025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/f2fs/f2fs.ko 2023-11-08T08:03:32.711 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fat/ 2023-11-08T08:03:32.711 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fat/msdos.ko 2023-11-08T08:03:32.711 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/freevxfs/ 2023-11-08T08:03:32.712 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 42545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/freevxfs/freevxfs.ko 2023-11-08T08:03:32.712 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fscache/ 2023-11-08T08:03:32.712 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 226481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fscache/fscache.ko 2023-11-08T08:03:32.712 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fuse/ 2023-11-08T08:03:32.713 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/fuse/cuse.ko 2023-11-08T08:03:32.713 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/gfs2/ 2023-11-08T08:03:32.713 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 1231529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/gfs2/gfs2.ko 2023-11-08T08:03:32.713 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hfs/ 2023-11-08T08:03:32.713 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 141209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hfs/hfs.ko 2023-11-08T08:03:32.714 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hfsplus/ 2023-11-08T08:03:32.714 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 222817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hfsplus/hfsplus.ko 2023-11-08T08:03:32.714 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hpfs/ 2023-11-08T08:03:32.714 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 192329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/hpfs/hpfs.ko 2023-11-08T08:03:32.715 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/isofs/ 2023-11-08T08:03:32.715 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 74065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/isofs/isofs.ko 2023-11-08T08:03:32.715 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/jfs/ 2023-11-08T08:03:32.715 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 419497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/jfs/jfs.ko 2023-11-08T08:03:32.715 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/lockd/ 2023-11-08T08:03:32.716 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 280313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/lockd/lockd.ko 2023-11-08T08:03:32.716 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/minix/ 2023-11-08T08:03:32.716 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 89193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/minix/minix.ko 2023-11-08T08:03:32.716 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/netfs/ 2023-11-08T08:03:32.717 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 124649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/netfs/netfs.ko 2023-11-08T08:03:32.717 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/ 2023-11-08T08:03:32.717 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/blocklayout/ 2023-11-08T08:03:32.717 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 81449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-08T08:03:32.717 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/filelayout/ 2023-11-08T08:03:32.718 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 59449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-08T08:03:32.718 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/flexfilelayout/ 2023-11-08T08:03:32.718 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 117513 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-08T08:03:32.718 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 1055649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/nfs.ko 2023-11-08T08:03:32.718 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 66441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/nfsv2.ko 2023-11-08T08:03:32.719 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 120177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/nfsv3.ko 2023-11-08T08:03:32.719 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 2041689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs/nfsv4.ko 2023-11-08T08:03:32.719 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs_common/ 2023-11-08T08:03:32.719 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs_common/grace.ko 2023-11-08T08:03:32.720 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfs_common/nfs_acl.ko 2023-11-08T08:03:32.720 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfsd/ 2023-11-08T08:03:32.720 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 1516809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nfsd/nfsd.ko 2023-11-08T08:03:32.720 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nilfs2/ 2023-11-08T08:03:32.720 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 592833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nilfs2/nilfs2.ko 2023-11-08T08:03:32.721 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/ 2023-11-08T08:03:32.721 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-celtic.ko 2023-11-08T08:03:32.721 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-centeuro.ko 2023-11-08T08:03:32.721 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-croatian.ko 2023-11-08T08:03:32.721 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-cyrillic.ko 2023-11-08T08:03:32.722 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-gaelic.ko 2023-11-08T08:03:32.722 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-greek.ko 2023-11-08T08:03:32.722 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-iceland.ko 2023-11-08T08:03:32.722 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-inuit.ko 2023-11-08T08:03:32.723 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-roman.ko 2023-11-08T08:03:32.723 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-romanian.ko 2023-11-08T08:03:32.723 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/mac-turkish.ko 2023-11-08T08:03:32.723 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_ascii.ko 2023-11-08T08:03:32.723 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp1250.ko 2023-11-08T08:03:32.724 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp1251.ko 2023-11-08T08:03:32.724 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp1255.ko 2023-11-08T08:03:32.724 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp737.ko 2023-11-08T08:03:32.724 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp775.ko 2023-11-08T08:03:32.724 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp850.ko 2023-11-08T08:03:32.725 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp852.ko 2023-11-08T08:03:32.725 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp855.ko 2023-11-08T08:03:32.725 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp857.ko 2023-11-08T08:03:32.725 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp860.ko 2023-11-08T08:03:32.725 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp861.ko 2023-11-08T08:03:32.726 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp862.ko 2023-11-08T08:03:32.726 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp863.ko 2023-11-08T08:03:32.726 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp864.ko 2023-11-08T08:03:32.726 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp865.ko 2023-11-08T08:03:32.727 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp866.ko 2023-11-08T08:03:32.727 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp869.ko 2023-11-08T08:03:32.727 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp874.ko 2023-11-08T08:03:32.727 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 94137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp932.ko 2023-11-08T08:03:32.727 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 142673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp936.ko 2023-11-08T08:03:32.728 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 165745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp949.ko 2023-11-08T08:03:32.728 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 118121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_cp950.ko 2023-11-08T08:03:32.728 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_euc-jp.ko 2023-11-08T08:03:32.728 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-1.ko 2023-11-08T08:03:32.728 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-13.ko 2023-11-08T08:03:32.729 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-14.ko 2023-11-08T08:03:32.729 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-15.ko 2023-11-08T08:03:32.729 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-2.ko 2023-11-08T08:03:32.729 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-3.ko 2023-11-08T08:03:32.730 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-4.ko 2023-11-08T08:03:32.730 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-5.ko 2023-11-08T08:03:32.730 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-6.ko 2023-11-08T08:03:32.730 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-7.ko 2023-11-08T08:03:32.730 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_iso8859-9.ko 2023-11-08T08:03:32.731 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_koi8-r.ko 2023-11-08T08:03:32.731 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_koi8-ru.ko 2023-11-08T08:03:32.731 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_koi8-u.ko 2023-11-08T08:03:32.731 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-08T08:03:32.732 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/nls/nls_utf8.ko 2023-11-08T08:03:32.732 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ntfs/ 2023-11-08T08:03:32.732 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 219417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ntfs/ntfs.ko 2023-11-08T08:03:32.732 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/ 2023-11-08T08:03:32.732 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/cluster/ 2023-11-08T08:03:32.733 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 244409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-08T08:03:32.733 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/dlm/ 2023-11-08T08:03:32.733 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 440737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-08T08:03:32.733 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/dlmfs/ 2023-11-08T08:03:32.733 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 54049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-08T08:03:32.734 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 2592433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/ocfs2.ko 2023-11-08T08:03:32.734 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-08T08:03:32.734 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 31729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-08T08:03:32.734 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 35897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-08T08:03:32.735 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/omfs/ 2023-11-08T08:03:32.735 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 53433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/omfs/omfs.ko 2023-11-08T08:03:32.735 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/overlayfs/ 2023-11-08T08:03:32.735 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 352153 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/overlayfs/overlay.ko 2023-11-08T08:03:32.735 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/pstore/ 2023-11-08T08:03:32.736 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 54809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/pstore/ramoops.ko 2023-11-08T08:03:32.736 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/qnx4/ 2023-11-08T08:03:32.736 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/qnx4/qnx4.ko 2023-11-08T08:03:32.736 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/qnx6/ 2023-11-08T08:03:32.736 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 46689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/qnx6/qnx6.ko 2023-11-08T08:03:32.737 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/quota/ 2023-11-08T08:03:32.737 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/quota/quota_tree.ko 2023-11-08T08:03:32.737 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/quota/quota_v1.ko 2023-11-08T08:03:32.737 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/quota/quota_v2.ko 2023-11-08T08:03:32.737 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/reiserfs/ 2023-11-08T08:03:32.738 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 532673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/reiserfs/reiserfs.ko 2023-11-08T08:03:32.738 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/romfs/ 2023-11-08T08:03:32.738 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/romfs/romfs.ko 2023-11-08T08:03:32.738 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/ 2023-11-08T08:03:32.738 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/client/ 2023-11-08T08:03:32.739 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 2668121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/client/cifs.ko 2023-11-08T08:03:32.739 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/common/ 2023-11-08T08:03:32.739 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/common/cifs_arc4.ko 2023-11-08T08:03:32.739 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/smb/common/cifs_md4.ko 2023-11-08T08:03:32.739 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/squashfs/ 2023-11-08T08:03:32.740 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 134641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/squashfs/squashfs.ko 2023-11-08T08:03:32.740 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/sysv/ 2023-11-08T08:03:32.740 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 90721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/sysv/sysv.ko 2023-11-08T08:03:32.740 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/udf/ 2023-11-08T08:03:32.741 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 247737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/udf/udf.ko 2023-11-08T08:03:32.741 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ufs/ 2023-11-08T08:03:32.741 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 162497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/ufs/ufs.ko 2023-11-08T08:03:32.741 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/xfs/ 2023-11-08T08:03:32.742 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 4010729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/fs/xfs/xfs.ko 2023-11-08T08:03:32.742 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/ 2023-11-08T08:03:32.742 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/locking/ 2023-11-08T08:03:32.742 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 75489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/locking/locktorture.ko 2023-11-08T08:03:32.742 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/time/ 2023-11-08T08:03:32.743 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/time/test_udelay.ko 2023-11-08T08:03:32.743 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 60553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/kernel/torture.ko 2023-11-08T08:03:32.743 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/ 2023-11-08T08:03:32.743 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 6833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crc-itu-t.ko 2023-11-08T08:03:32.744 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 6529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crc7.ko 2023-11-08T08:03:32.744 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crc8.ko 2023-11-08T08:03:32.744 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crypto/ 2023-11-08T08:03:32.744 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crypto/libarc4.ko 2023-11-08T08:03:32.745 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 37081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/crypto/libdes.ko 2023-11-08T08:03:32.745 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/interval_tree_test.ko 2023-11-08T08:03:32.745 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/libcrc32c.ko 2023-11-08T08:03:32.745 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/lru_cache.ko 2023-11-08T08:03:32.745 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/lz4/ 2023-11-08T08:03:32.746 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/lz4/lz4_compress.ko 2023-11-08T08:03:32.746 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/lz4/lz4hc_compress.ko 2023-11-08T08:03:32.746 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/math/ 2023-11-08T08:03:32.746 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 6617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/math/cordic.ko 2023-11-08T08:03:32.747 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/memory-notifier-error-inject.ko 2023-11-08T08:03:32.747 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/notifier-error-inject.ko 2023-11-08T08:03:32.747 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/percpu_test.ko 2023-11-08T08:03:32.747 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/pm-notifier-error-inject.ko 2023-11-08T08:03:32.747 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/raid6/ 2023-11-08T08:03:32.748 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 143433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/raid6/raid6_pq.ko 2023-11-08T08:03:32.748 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/rbtree_test.ko 2023-11-08T08:03:32.748 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/reed_solomon/ 2023-11-08T08:03:32.748 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-08T08:03:32.749 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 38841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test-kstrtox.ko 2023-11-08T08:03:32.749 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test-string_helpers.ko 2023-11-08T08:03:32.749 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 4760425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test_bpf.ko 2023-11-08T08:03:32.749 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 49089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test_firmware.ko 2023-11-08T08:03:32.749 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 6369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test_module.ko 2023-11-08T08:03:32.750 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/test_user_copy.ko 2023-11-08T08:03:32.750 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/ts_bm.ko 2023-11-08T08:03:32.750 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/ts_fsm.ko 2023-11-08T08:03:32.750 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/ts_kmp.ko 2023-11-08T08:03:32.750 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/xz/ 2023-11-08T08:03:32.751 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/lib/xz/xz_dec_test.ko 2023-11-08T08:03:32.751 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/mm/ 2023-11-08T08:03:32.751 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/mm/hwpoison-inject.ko 2023-11-08T08:03:32.751 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ 2023-11-08T08:03:32.752 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/ 2023-11-08T08:03:32.752 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 55737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/6lowpan.ko 2023-11-08T08:03:32.752 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_dest.ko 2023-11-08T08:03:32.752 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_fragment.ko 2023-11-08T08:03:32.752 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_hop.ko 2023-11-08T08:03:32.753 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 6985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-08T08:03:32.753 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_mobility.ko 2023-11-08T08:03:32.753 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_routing.ko 2023-11-08T08:03:32.753 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/6lowpan/nhc_udp.ko 2023-11-08T08:03:32.753 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/ 2023-11-08T08:03:32.754 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/garp.ko 2023-11-08T08:03:32.754 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/mrp.ko 2023-11-08T08:03:32.754 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/p8022.ko 2023-11-08T08:03:32.754 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/psnap.ko 2023-11-08T08:03:32.755 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/802/stp.ko 2023-11-08T08:03:32.755 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/8021q/ 2023-11-08T08:03:32.755 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 81737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/8021q/8021q.ko 2023-11-08T08:03:32.755 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/9p/ 2023-11-08T08:03:32.755 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 198337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/9p/9pnet.ko 2023-11-08T08:03:32.756 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 35977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/9p/9pnet_fd.ko 2023-11-08T08:03:32.756 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26345 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/9p/9pnet_rdma.ko 2023-11-08T08:03:32.756 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 32337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/9p/9pnet_virtio.ko 2023-11-08T08:03:32.756 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/batman-adv/ 2023-11-08T08:03:32.756 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 649297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/batman-adv/batman-adv.ko 2023-11-08T08:03:32.757 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/ 2023-11-08T08:03:32.757 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 63809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/br_netfilter.ko 2023-11-08T08:03:32.757 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 777689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/bridge.ko 2023-11-08T08:03:32.757 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ 2023-11-08T08:03:32.757 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-08T08:03:32.758 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-08T08:03:32.758 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-08T08:03:32.758 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-08T08:03:32.758 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-08T08:03:32.759 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-08T08:03:32.759 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-08T08:03:32.759 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-08T08:03:32.759 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13217 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-08T08:03:32.759 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-08T08:03:32.760 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-08T08:03:32.760 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8385 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-08T08:03:32.760 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-08T08:03:32.760 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-08T08:03:32.760 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-08T08:03:32.761 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-08T08:03:32.761 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-08T08:03:32.761 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-08T08:03:32.761 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-08T08:03:32.762 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-08T08:03:32.762 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 55929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/ebtables.ko 2023-11-08T08:03:32.762 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-08T08:03:32.762 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-08T08:03:32.762 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ceph/ 2023-11-08T08:03:32.763 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 1046857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ceph/libceph.ko 2023-11-08T08:03:32.763 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/core/ 2023-11-08T08:03:32.763 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 125697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/core/pktgen.ko 2023-11-08T08:03:32.763 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dccp/ 2023-11-08T08:03:32.763 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 194209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dccp/dccp.ko 2023-11-08T08:03:32.764 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dccp/dccp_diag.ko 2023-11-08T08:03:32.764 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 57881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dccp/dccp_ipv4.ko 2023-11-08T08:03:32.764 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 60337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dccp/dccp_ipv6.ko 2023-11-08T08:03:32.764 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/ 2023-11-08T08:03:32.765 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 390473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/dsa_core.ko 2023-11-08T08:03:32.765 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/tag_brcm.ko 2023-11-08T08:03:32.765 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/tag_dsa.ko 2023-11-08T08:03:32.765 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/tag_none.ko 2023-11-08T08:03:32.765 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/dsa/tag_trailer.ko 2023-11-08T08:03:32.766 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/hsr/ 2023-11-08T08:03:32.766 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 99729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/hsr/hsr.ko 2023-11-08T08:03:32.766 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ieee802154/ 2023-11-08T08:03:32.766 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ieee802154/6lowpan/ 2023-11-08T08:03:32.767 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 45233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-08T08:03:32.767 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 254625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ieee802154/ieee802154.ko 2023-11-08T08:03:32.767 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 47289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-08T08:03:32.767 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ 2023-11-08T08:03:32.767 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ah4.ko 2023-11-08T08:03:32.768 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 38905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/esp4.ko 2023-11-08T08:03:32.768 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 48401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/fou.ko 2023-11-08T08:03:32.768 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/gre.ko 2023-11-08T08:03:32.768 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 48657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/inet_diag.ko 2023-11-08T08:03:32.768 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 60633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ip_gre.ko 2023-11-08T08:03:32.769 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 61225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ip_tunnel.ko 2023-11-08T08:03:32.769 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ip_vti.ko 2023-11-08T08:03:32.769 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ipcomp.ko 2023-11-08T08:03:32.769 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/ipip.ko 2023-11-08T08:03:32.770 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ 2023-11-08T08:03:32.770 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 37641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-08T08:03:32.770 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-08T08:03:32.770 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-08T08:03:32.770 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 41673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ip_tables.ko 2023-11-08T08:03:32.771 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-08T08:03:32.771 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-11-08T08:03:32.771 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-08T08:03:32.771 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-08T08:03:32.772 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-08T08:03:32.772 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11305 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-11-08T08:03:32.772 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-08T08:03:32.772 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-08T08:03:32.772 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-08T08:03:32.773 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-08T08:03:32.773 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-08T08:03:32.773 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-08T08:03:32.773 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-08T08:03:32.773 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18225 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-08T08:03:32.774 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-08T08:03:32.774 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-11-08T08:03:32.774 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-08T08:03:32.774 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-08T08:03:32.775 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-08T08:03:32.775 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-08T08:03:32.775 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-08T08:03:32.775 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_bic.ko 2023-11-08T08:03:32.775 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_dctcp.ko 2023-11-08T08:03:32.776 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_diag.ko 2023-11-08T08:03:32.776 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_highspeed.ko 2023-11-08T08:03:32.776 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_htcp.ko 2023-11-08T08:03:32.776 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_hybla.ko 2023-11-08T08:03:32.777 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_illinois.ko 2023-11-08T08:03:32.777 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_lp.ko 2023-11-08T08:03:32.777 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_scalable.ko 2023-11-08T08:03:32.777 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_vegas.ko 2023-11-08T08:03:32.777 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_veno.ko 2023-11-08T08:03:32.778 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_westwood.ko 2023-11-08T08:03:32.778 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tcp_yeah.ko 2023-11-08T08:03:32.778 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/tunnel4.ko 2023-11-08T08:03:32.778 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/udp_diag.ko 2023-11-08T08:03:32.779 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 41665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/udp_tunnel.ko 2023-11-08T08:03:32.779 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-08T08:03:32.779 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ 2023-11-08T08:03:32.779 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ah6.ko 2023-11-08T08:03:32.779 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 39537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/esp6.ko 2023-11-08T08:03:32.780 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/fou6.ko 2023-11-08T08:03:32.780 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 80897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ip6_gre.ko 2023-11-08T08:03:32.780 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 93577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ip6_tunnel.ko 2023-11-08T08:03:32.780 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-08T08:03:32.780 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 53273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ip6_vti.ko 2023-11-08T08:03:32.781 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/ipcomp6.ko 2023-11-08T08:03:32.781 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19017 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/mip6.ko 2023-11-08T08:03:32.781 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ 2023-11-08T08:03:32.781 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 42169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-08T08:03:32.782 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-11-08T08:03:32.782 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-08T08:03:32.782 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-08T08:03:32.782 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-08T08:03:32.782 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-08T08:03:32.783 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-08T08:03:32.783 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-08T08:03:32.783 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-08T08:03:32.783 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-08T08:03:32.784 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-08T08:03:32.784 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-08T08:03:32.784 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-08T08:03:32.784 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11121 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-08T08:03:32.784 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-08T08:03:32.785 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-08T08:03:32.785 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-08T08:03:32.785 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 39113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-08T08:03:32.785 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-08T08:03:32.785 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25945 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-08T08:03:32.786 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-08T08:03:32.786 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-08T08:03:32.786 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-08T08:03:32.786 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-08T08:03:32.786 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9009 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-08T08:03:32.787 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 74073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/sit.ko 2023-11-08T08:03:32.787 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/tunnel6.ko 2023-11-08T08:03:32.787 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-08T08:03:32.787 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/key/ 2023-11-08T08:03:32.787 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 82833 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/key/af_key.ko 2023-11-08T08:03:32.788 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/ 2023-11-08T08:03:32.788 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 125281 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_core.ko 2023-11-08T08:03:32.788 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-08T08:03:32.788 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 25953 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_eth.ko 2023-11-08T08:03:32.788 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 35569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_ip.ko 2023-11-08T08:03:32.789 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 36713 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_ip6.ko 2023-11-08T08:03:32.789 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_netlink.ko 2023-11-08T08:03:32.789 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 53241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/l2tp/l2tp_ppp.ko 2023-11-08T08:03:32.789 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/lapb/ 2023-11-08T08:03:32.790 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 47977 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/lapb/lapb.ko 2023-11-08T08:03:32.790 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/llc/ 2023-11-08T08:03:32.790 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/llc/llc.ko 2023-11-08T08:03:32.790 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 267777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/llc/llc2.ko 2023-11-08T08:03:32.790 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/mac802154/ 2023-11-08T08:03:32.791 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 290417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/mac802154/mac802154.ko 2023-11-08T08:03:32.791 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/mpls/ 2023-11-08T08:03:32.791 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/mpls/mpls_gso.ko 2023-11-08T08:03:32.791 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ 2023-11-08T08:03:32.791 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ 2023-11-08T08:03:32.792 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 119585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set.ko 2023-11-08T08:03:32.792 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-08T08:03:32.792 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26913 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-08T08:03:32.792 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-08T08:03:32.792 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 100297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-08T08:03:32.793 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 101441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-08T08:03:32.793 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 102817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-08T08:03:32.793 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 103481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-08T08:03:32.793 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 112849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-08T08:03:32.794 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 60329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-08T08:03:32.794 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 109593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-08T08:03:32.794 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 113129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-08T08:03:32.794 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 112585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-08T08:03:32.795 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 111585 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-08T08:03:32.795 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 114465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-08T08:03:32.795 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-08T08:03:32.795 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ 2023-11-08T08:03:32.795 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 411729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-08T08:03:32.796 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-08T08:03:32.796 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-08T08:03:32.796 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-08T08:03:32.796 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-08T08:03:32.797 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-08T08:03:32.797 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-08T08:03:32.797 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7689 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-08T08:03:32.797 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10681 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-08T08:03:32.798 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-08T08:03:32.798 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-08T08:03:32.798 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-08T08:03:32.798 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-08T08:03:32.798 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10993 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-08T08:03:32.799 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 49257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conncount.ko 2023-11-08T08:03:32.799 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 408257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack.ko 2023-11-08T08:03:32.799 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-08T08:03:32.799 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-08T08:03:32.800 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33761 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-08T08:03:32.800 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 118145 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-08T08:03:32.800 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-08T08:03:32.800 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-08T08:03:32.801 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 107297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-08T08:03:32.801 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-08T08:03:32.801 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-08T08:03:32.801 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 66641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-08T08:03:32.801 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-08T08:03:32.802 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-08T08:03:32.802 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10113 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-08T08:03:32.802 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 89473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_flow_table.ko 2023-11-08T08:03:32.802 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9593 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-08T08:03:32.803 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 37129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_log_syslog.ko 2023-11-08T08:03:32.803 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 95417 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat.ko 2023-11-08T08:03:32.803 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-08T08:03:32.803 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-08T08:03:32.803 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11857 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat_irc.ko 2023-11-08T08:03:32.804 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28929 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat_sip.ko 2023-11-08T08:03:32.804 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-08T08:03:32.804 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 44065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-08T08:03:32.804 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 615313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nf_tables.ko 2023-11-08T08:03:32.804 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 42561 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink.ko 2023-11-08T08:03:32.805 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-08T08:03:32.805 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30569 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-11-08T08:03:32.805 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 39785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink_log.ko 2023-11-08T08:03:32.805 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-08T08:03:32.805 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 49377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-08T08:03:32.806 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10457 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_chain_nat.ko 2023-11-08T08:03:32.806 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_compat.ko 2023-11-08T08:03:32.806 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 40721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_ct.ko 2023-11-08T08:03:32.806 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-08T08:03:32.807 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_fib.ko 2023-11-08T08:03:32.807 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-08T08:03:32.807 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14185 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_hash.ko 2023-11-08T08:03:32.807 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_limit.ko 2023-11-08T08:03:32.807 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_log.ko 2023-11-08T08:03:32.808 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_masq.ko 2023-11-08T08:03:32.808 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14329 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_nat.ko 2023-11-08T08:03:32.808 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13673 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_numgen.ko 2023-11-08T08:03:32.808 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11841 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_osf.ko 2023-11-08T08:03:32.809 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_queue.ko 2023-11-08T08:03:32.809 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_redir.ko 2023-11-08T08:03:32.809 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_reject.ko 2023-11-08T08:03:32.809 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_reject_inet.ko 2023-11-08T08:03:32.809 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_socket.ko 2023-11-08T08:03:32.810 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20369 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_synproxy.ko 2023-11-08T08:03:32.810 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/nft_tproxy.ko 2023-11-08T08:03:32.810 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 89313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/x_tables.ko 2023-11-08T08:03:32.810 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_AUDIT.ko 2023-11-08T08:03:32.811 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-08T08:03:32.811 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-08T08:03:32.811 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-08T08:03:32.811 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 18169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_CT.ko 2023-11-08T08:03:32.812 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_DSCP.ko 2023-11-08T08:03:32.812 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_HL.ko 2023-11-08T08:03:32.812 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_HMARK.ko 2023-11-08T08:03:32.812 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 38873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-08T08:03:32.812 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13889 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_LED.ko 2023-11-08T08:03:32.813 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_LOG.ko 2023-11-08T08:03:32.813 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11753 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-08T08:03:32.813 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_NETMAP.ko 2023-11-08T08:03:32.813 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_NFLOG.ko 2023-11-08T08:03:32.814 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-08T08:03:32.814 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_RATEEST.ko 2023-11-08T08:03:32.814 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-08T08:03:32.814 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 14337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_SECMARK.ko 2023-11-08T08:03:32.815 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-08T08:03:32.815 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-08T08:03:32.815 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17361 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_TEE.ko 2023-11-08T08:03:32.815 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_TPROXY.ko 2023-11-08T08:03:32.815 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_TRACE.ko 2023-11-08T08:03:32.816 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_addrtype.ko 2023-11-08T08:03:32.816 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_bpf.ko 2023-11-08T08:03:32.816 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_cgroup.ko 2023-11-08T08:03:32.816 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11209 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_cluster.ko 2023-11-08T08:03:32.816 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_comment.ko 2023-11-08T08:03:32.817 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10465 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_connbytes.ko 2023-11-08T08:03:32.817 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11081 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_connlabel.ko 2023-11-08T08:03:32.817 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_connlimit.ko 2023-11-08T08:03:32.817 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_connmark.ko 2023-11-08T08:03:32.817 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_conntrack.ko 2023-11-08T08:03:32.818 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_cpu.ko 2023-11-08T08:03:32.818 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_dccp.ko 2023-11-08T08:03:32.818 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_devgroup.ko 2023-11-08T08:03:32.818 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8801 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_dscp.ko 2023-11-08T08:03:32.819 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_ecn.ko 2023-11-08T08:03:32.819 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_esp.ko 2023-11-08T08:03:32.819 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 43289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_hashlimit.ko 2023-11-08T08:03:32.819 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_helper.ko 2023-11-08T08:03:32.819 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7865 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_hl.ko 2023-11-08T08:03:32.820 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11921 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_ipcomp.ko 2023-11-08T08:03:32.820 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_iprange.ko 2023-11-08T08:03:32.820 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11785 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_ipvs.ko 2023-11-08T08:03:32.820 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16041 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_l2tp.ko 2023-11-08T08:03:32.820 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8001 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_length.ko 2023-11-08T08:03:32.821 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_limit.ko 2023-11-08T08:03:32.821 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7545 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_mac.ko 2023-11-08T08:03:32.821 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_mark.ko 2023-11-08T08:03:32.821 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11825 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_multiport.ko 2023-11-08T08:03:32.822 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13817 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_nat.ko 2023-11-08T08:03:32.822 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_nfacct.ko 2023-11-08T08:03:32.822 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_osf.ko 2023-11-08T08:03:32.822 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_owner.ko 2023-11-08T08:03:32.822 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_physdev.ko 2023-11-08T08:03:32.823 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_pkttype.ko 2023-11-08T08:03:32.823 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11985 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_policy.ko 2023-11-08T08:03:32.823 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_quota.ko 2023-11-08T08:03:32.823 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_rateest.ko 2023-11-08T08:03:32.824 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_realm.ko 2023-11-08T08:03:32.824 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 36497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_recent.ko 2023-11-08T08:03:32.824 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_sctp.ko 2023-11-08T08:03:32.824 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_set.ko 2023-11-08T08:03:32.824 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 16089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_socket.ko 2023-11-08T08:03:32.825 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9505 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_state.ko 2023-11-08T08:03:32.825 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9473 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_statistic.ko 2023-11-08T08:03:32.825 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8809 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_string.ko 2023-11-08T08:03:32.825 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 8553 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_tcpmss.ko 2023-11-08T08:03:32.825 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15777 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_tcpudp.ko 2023-11-08T08:03:32.826 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11129 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_time.ko 2023-11-08T08:03:32.826 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netfilter/xt_u32.ko 2023-11-08T08:03:32.826 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netlink/ 2023-11-08T08:03:32.826 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13969 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/netlink/netlink_diag.ko 2023-11-08T08:03:32.827 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/nsh/ 2023-11-08T08:03:32.827 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 12425 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/nsh/nsh.ko 2023-11-08T08:03:32.827 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/openvswitch/ 2023-11-08T08:03:32.827 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 380033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/openvswitch/openvswitch.ko 2023-11-08T08:03:32.827 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10321 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/openvswitch/vport-geneve.ko 2023-11-08T08:03:32.828 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/openvswitch/vport-gre.ko 2023-11-08T08:03:32.828 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 11793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/openvswitch/vport-vxlan.ko 2023-11-08T08:03:32.828 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/packet/ 2023-11-08T08:03:32.828 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13057 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/packet/af_packet_diag.ko 2023-11-08T08:03:32.829 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/phonet/ 2023-11-08T08:03:32.829 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 110201 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/phonet/phonet.ko 2023-11-08T08:03:32.829 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 54697 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/phonet/pn_pep.ko 2023-11-08T08:03:32.829 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rds/ 2023-11-08T08:03:32.829 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 251481 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rds/rds.ko 2023-11-08T08:03:32.830 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 164937 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rds/rds_rdma.ko 2023-11-08T08:03:32.830 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 67377 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rds/rds_tcp.ko 2023-11-08T08:03:32.830 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rxrpc/ 2023-11-08T08:03:32.830 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 819657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/rxrpc/rxrpc.ko 2023-11-08T08:03:32.830 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/ 2023-11-08T08:03:32.831 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 32241 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_csum.ko 2023-11-08T08:03:32.831 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23577 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_gact.ko 2023-11-08T08:03:32.831 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26873 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_ipt.ko 2023-11-08T08:03:32.831 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34097 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_mirred.ko 2023-11-08T08:03:32.832 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_nat.ko 2023-11-08T08:03:32.832 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 43449 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_pedit.ko 2023-11-08T08:03:32.832 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 31961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_police.ko 2023-11-08T08:03:32.832 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_simple.ko 2023-11-08T08:03:32.832 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 35297 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_skbedit.ko 2023-11-08T08:03:32.833 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 36665 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/act_vlan.ko 2023-11-08T08:03:32.833 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26169 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_basic.ko 2023-11-08T08:03:32.833 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33961 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_bpf.ko 2023-11-08T08:03:32.833 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19233 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_cgroup.ko 2023-11-08T08:03:32.833 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 31401 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_flow.ko 2023-11-08T08:03:32.834 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 31769 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_fw.ko 2023-11-08T08:03:32.834 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 38489 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_route.ko 2023-11-08T08:03:32.834 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 61409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/cls_u32.ko 2023-11-08T08:03:32.834 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7441 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_cmp.ko 2023-11-08T08:03:32.834 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 13409 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_ipset.ko 2023-11-08T08:03:32.835 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33289 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_meta.ko 2023-11-08T08:03:32.835 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_nbyte.ko 2023-11-08T08:03:32.835 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_text.ko 2023-11-08T08:03:32.835 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7249 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/em_u32.ko 2023-11-08T08:03:32.835 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 21649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_choke.ko 2023-11-08T08:03:32.836 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20849 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_codel.ko 2023-11-08T08:03:32.836 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 34633 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_drr.ko 2023-11-08T08:03:32.836 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 33609 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_fq.ko 2023-11-08T08:03:32.836 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 31617 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_fq_codel.ko 2023-11-08T08:03:32.837 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 35433 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_gred.ko 2023-11-08T08:03:32.837 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 49601 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_hfsc.ko 2023-11-08T08:03:32.837 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22089 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_hhf.ko 2023-11-08T08:03:32.837 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 66393 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_htb.ko 2023-11-08T08:03:32.837 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26073 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_ingress.ko 2023-11-08T08:03:32.838 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 30273 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_mqprio.ko 2023-11-08T08:03:32.838 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_mqprio_lib.ko 2023-11-08T08:03:32.838 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26065 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_multiq.ko 2023-11-08T08:03:32.838 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 36137 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_netem.ko 2023-11-08T08:03:32.838 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 27497 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_pie.ko 2023-11-08T08:03:32.839 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10625 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_plug.ko 2023-11-08T08:03:32.839 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 24729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_prio.ko 2023-11-08T08:03:32.839 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 48793 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_qfq.ko 2023-11-08T08:03:32.839 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 32905 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_red.ko 2023-11-08T08:03:32.840 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 26881 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_sfb.ko 2023-11-08T08:03:32.840 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 31353 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_sfq.ko 2023-11-08T08:03:32.840 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28897 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_tbf.ko 2023-11-08T08:03:32.840 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 28105 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sched/sch_teql.ko 2023-11-08T08:03:32.840 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sctp/ 2023-11-08T08:03:32.841 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 929745 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sctp/sctp.ko 2023-11-08T08:03:32.841 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 19033 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sctp/sctp_diag.ko 2023-11-08T08:03:32.841 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/ 2023-11-08T08:03:32.841 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/auth_gss/ 2023-11-08T08:03:32.841 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 336529 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-08T08:03:32.842 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 54737 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-08T08:03:32.842 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 1520161 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/sunrpc.ko 2023-11-08T08:03:32.842 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/xprtrdma/ 2023-11-08T08:03:32.842 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 707721 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-08T08:03:32.843 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/tipc/ 2023-11-08T08:03:32.843 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10025 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/tipc/diag.ko 2023-11-08T08:03:32.843 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 862641 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/tipc/tipc.ko 2023-11-08T08:03:32.843 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/unix/ 2023-11-08T08:03:32.843 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15729 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/unix/unix_diag.ko 2023-11-08T08:03:32.844 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/ 2023-11-08T08:03:32.844 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 85337 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-08T08:03:32.844 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 68049 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-08T08:03:32.844 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 96193 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/vsock.ko 2023-11-08T08:03:32.845 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 10705 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-08T08:03:32.845 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 17521 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-08T08:03:32.845 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/x25/ 2023-11-08T08:03:32.845 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 125177 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/x25/x25.ko 2023-11-08T08:03:32.845 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/xfrm/ 2023-11-08T08:03:32.846 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 22657 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/xfrm/xfrm_algo.ko 2023-11-08T08:03:32.846 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 20649 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-08T08:03:32.846 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 102257 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/net/xfrm/xfrm_user.ko 2023-11-08T08:03:32.846 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/virt/ 2023-11-08T08:03:32.847 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/virt/lib/ 2023-11-08T08:03:32.847 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 15537 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/kernel/virt/lib/irqbypass.ko 2023-11-08T08:03:32.847 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 639863 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.alias 2023-11-08T08:03:32.847 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 623919 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.alias.bin 2023-11-08T08:03:32.847 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 7663 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.builtin 2023-11-08T08:03:32.848 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 23428 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.builtin.alias.bin 2023-11-08T08:03:32.848 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 9821 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.builtin.bin 2023-11-08T08:03:32.848 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 70004 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.builtin.modinfo 2023-11-08T08:03:32.848 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 146313 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.dep 2023-11-08T08:03:32.848 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 232548 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.dep.bin 2023-11-08T08:03:32.849 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 210 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.devname 2023-11-08T08:03:32.849 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 72606 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.order 2023-11-08T08:03:32.849 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 879 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.softdep 2023-11-08T08:03:32.849 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 254438 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.symbols 2023-11-08T08:03:32.850 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 309363 2023-11-07 03:19 ./lib/modules/6.6.0-rc5-g3b796cfe2969/modules.symbols.bin 2023-11-08T08:03:32.850 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./usr/ 2023-11-08T08:03:32.850 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./usr/share/ 2023-11-08T08:03:32.850 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./usr/share/doc/ 2023-11-08T08:03:32.850 INFO:teuthology.orchestra.run.smithi040.stdout:drwxr-xr-x root/root 0 2023-11-07 03:19 ./usr/share/doc/linux-image-6.6.0-rc5-g3b796cfe2969/ 2023-11-08T08:03:32.851 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 173 2023-11-07 03:19 ./usr/share/doc/linux-image-6.6.0-rc5-g3b796cfe2969/changelog.Debian.gz 2023-11-08T08:03:32.851 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r-- root/root 691 2023-11-07 03:19 ./usr/share/doc/linux-image-6.6.0-rc5-g3b796cfe2969/copyright 2023-11-08T08:03:32.852 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:32.852 INFO:teuthology.task.kernel:searching for kernel 6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:32.852 DEBUG:teuthology.orchestra.run.smithi040:> egrep '(submenu|menuentry.*6.6.0-rc5-g3b796cfe2969).*{' /boot/grub/grub.cfg 2023-11-08T08:03:32.862 INFO:teuthology.orchestra.run.smithi040.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-11-08T08:03:32.862 INFO:teuthology.orchestra.run.smithi040.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g3b796cfe2969' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g3b796cfe2969-advanced-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-11-08T08:03:32.862 INFO:teuthology.orchestra.run.smithi040.stdout: menuentry 'Ubuntu, with Linux 6.6.0-rc5-g3b796cfe2969 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-rc5-g3b796cfe2969-recovery-1168b92c-82ac-47e1-87d9-5d77414c7738' { 2023-11-08T08:03:32.863 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-11-08T08:03:32.863 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:32.864 DEBUG:teuthology.orchestra.run.smithi040:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-11-08T08:03:32.866 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi040.front.sepia.ceph.com to complete... 2023-11-08T08:03:32.877 INFO:teuthology.orchestra.run.smithi169.stderr:Sourcing file `/etc/default/grub' 2023-11-08T08:03:32.886 INFO:teuthology.orchestra.run.smithi169.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-08T08:03:32.898 INFO:teuthology.orchestra.run.smithi169.stderr:Generating grub configuration file ... 2023-11-08T08:03:33.154 INFO:teuthology.orchestra.run.smithi169.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:33.165 INFO:teuthology.orchestra.run.smithi169.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:33.565 INFO:teuthology.orchestra.run.smithi169.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-11-08T08:03:33.577 INFO:teuthology.orchestra.run.smithi169.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-11-08T08:03:33.580 INFO:teuthology.orchestra.run.smithi040.stderr:Sourcing file `/etc/default/grub' 2023-11-08T08:03:33.583 INFO:teuthology.orchestra.run.smithi040.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-08T08:03:33.597 INFO:teuthology.orchestra.run.smithi040.stderr:Generating grub configuration file ... 2023-11-08T08:03:33.878 INFO:teuthology.orchestra.run.smithi040.stderr:Found linux image: /boot/vmlinuz-6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:33.890 INFO:teuthology.orchestra.run.smithi040.stderr:Found initrd image: /boot/initrd.img-6.6.0-rc5-g3b796cfe2969 2023-11-08T08:03:34.302 INFO:teuthology.orchestra.run.smithi040.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-11-08T08:03:34.313 INFO:teuthology.orchestra.run.smithi040.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-11-08T08:03:34.595 INFO:teuthology.orchestra.run.smithi169.stderr:done 2023-11-08T08:03:35.303 INFO:teuthology.orchestra.run.smithi040.stderr:done 2023-11-08T08:04:05.736 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-11-08T08:04:05.737 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-11-08T08:04:05.738 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T08:04:06.445 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-11-08T08:04:06.445 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-11-08T08:04:06.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T08:04:24.059 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2023-11-08T08:04:24.830 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.40 2023-11-08T08:04:33.069 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-11-08T08:04:33.070 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T08:04:33.832 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-11-08T08:04:33.832 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T08:04:36.122 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2023-11-08T08:04:48.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-11-08T08:04:48.135 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T08:05:33.879 DEBUG:teuthology.orchestra.remote:timed out 2023-11-08T08:05:45.890 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-11-08T08:05:45.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T08:05:46.138 DEBUG:teuthology.orchestra.run.smithi040:> true 2023-11-08T08:05:47.019 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-11-08T08:05:47.019 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-rc5-g3b796cfe2969"... 2023-11-08T08:05:47.020 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2023-11-08T08:05:47.080 INFO:teuthology.orchestra.run.smithi040.stdout:6.6.0-rc5-g3b796cfe2969 2023-11-08T08:05:47.081 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g3b796cfe2969 vs 6.6.0-rc5-g3b796cfe2969 2023-11-08T08:05:47.081 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-08T08:05:47.081 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-11-08T08:05:48.083 DEBUG:teuthology.parallel:result is None 2023-11-08T08:05:48.137 DEBUG:teuthology.orchestra.remote:timed out 2023-11-08T08:06:03.138 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-11-08T08:06:03.140 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T08:06:03.389 DEBUG:teuthology.orchestra.run.smithi169:> true 2023-11-08T08:06:04.141 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-11-08T08:06:04.141 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.6.0-rc5-g3b796cfe2969"... 2023-11-08T08:06:04.142 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2023-11-08T08:06:04.199 INFO:teuthology.orchestra.run.smithi169.stdout:6.6.0-rc5-g3b796cfe2969 2023-11-08T08:06:04.199 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g3b796cfe2969 vs 6.6.0-rc5-g3b796cfe2969 2023-11-08T08:06:04.200 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-08T08:06:04.200 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-11-08T08:06:05.201 DEBUG:teuthology.parallel:result is None 2023-11-08T08:06:05.202 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-08T08:06:05.210 INFO:teuthology.task.internal:Creating test directory... 2023-11-08T08:06:05.210 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-08T08:06:05.218 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-08T08:06:05.240 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-08T08:06:05.248 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-08T08:06:05.261 INFO:teuthology.task.internal:Creating archive directory... 2023-11-08T08:06:05.261 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-08T08:06:05.281 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-08T08:06:05.331 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-08T08:06:05.338 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-08T08:06:05.339 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-08T08:06:05.354 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-08T08:06:05.486 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T08:06:05.488 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T08:06:05.511 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T08:06:05.517 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T08:06:05.521 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-08T08:06:05.530 INFO:teuthology.task.internal:Configuring sudo... 2023-11-08T08:06:05.530 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-08T08:06:05.537 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-08T08:06:05.573 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-08T08:06:05.583 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-08T08:06:05.583 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-08T08:06:05.621 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-08T08:06:05.640 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-08T08:06:05.700 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-08T08:06:05.764 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:06:05.764 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-08T08:06:05.849 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-08T08:06:05.869 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-08T08:06:05.930 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:06:05.931 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-08T08:06:06.010 DEBUG:teuthology.orchestra.run.smithi040:> sudo service rsyslog restart 2023-11-08T08:06:06.018 DEBUG:teuthology.orchestra.run.smithi169:> sudo service rsyslog restart 2023-11-08T08:06:06.171 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-08T08:06:06.180 INFO:teuthology.task.internal:Starting timer... 2023-11-08T08:06:06.181 INFO:teuthology.run_tasks:Running task pcp... 2023-11-08T08:06:06.193 INFO:teuthology.run_tasks:Running task selinux... 2023-11-08T08:06:06.204 DEBUG:teuthology.task.selinux:Excluding smithi040: OS 'ubuntu' does not support SELinux 2023-11-08T08:06:06.204 DEBUG:teuthology.task.selinux:Excluding smithi169: OS 'ubuntu' does not support SELinux 2023-11-08T08:06:06.205 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-08T08:06:06.205 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-11-08T08:06:06.205 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-08T08:06:06.205 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-08T08:06:06.299 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-08T08:06:06.300 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-08T08:06:06.320 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-08T08:06:06.321 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi040.front.sepia.ceph.com,smithi169.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-08T08:14:05.405 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi040.front.sepia.ceph.com'), Remote(name='ubuntu@smithi169.front.sepia.ceph.com')] 2023-11-08T08:14:05.407 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-11-08T08:14:05.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T08:14:05.520 DEBUG:teuthology.orchestra.run.smithi040:> true 2023-11-08T08:14:05.751 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-11-08T08:14:05.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-11-08T08:14:05.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T08:14:05.858 DEBUG:teuthology.orchestra.run.smithi169:> true 2023-11-08T08:14:06.082 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-11-08T08:14:06.082 INFO:teuthology.run_tasks:Running task clock... 2023-11-08T08:14:06.093 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-08T08:14:06.094 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-08T08:14:06.094 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-08T08:14:06.102 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-08T08:14:06.102 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-08T08:14:06.171 INFO:teuthology.orchestra.run.smithi040.stdout: 8 Nov 08:14:06 ntpd[22629]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-11-08T08:14:06.171 INFO:teuthology.orchestra.run.smithi040.stdout: 8 Nov 08:14:06 ntpd[22629]: Command line: ntpd -gq 2023-11-08T08:14:06.172 INFO:teuthology.orchestra.run.smithi040.stdout: 8 Nov 08:14:06 ntpd[22629]: proto: precision = 0.043 usec (-24) 2023-11-08T08:14:06.172 INFO:teuthology.orchestra.run.smithi040.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-08T08:14:06.173 INFO:teuthology.orchestra.run.smithi040.stdout: 8 Nov 08:14:06 ntpd[22629]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-08T08:14:06.173 INFO:teuthology.orchestra.run.smithi040.stdout: 8 Nov 08:14:06 ntpd[22629]: restrict ::: KOD does nothing without LIMITED. 2023-11-08T08:14:06.173 INFO:teuthology.orchestra.run.smithi040.stderr:restrict ::: KOD does nothing without LIMITED. 2023-11-08T08:14:06.174 INFO:teuthology.orchestra.run.smithi040.stdout: 8 Nov 08:14:06 ntpd[22629]: Listen and drop on 0 v6wildcard [::]:123 2023-11-08T08:14:06.175 INFO:teuthology.orchestra.run.smithi040.stdout: 8 Nov 08:14:06 ntpd[22629]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-11-08T08:14:06.175 INFO:teuthology.orchestra.run.smithi040.stdout: 8 Nov 08:14:06 ntpd[22629]: Listen normally on 2 lo 127.0.0.1:123 2023-11-08T08:14:06.175 INFO:teuthology.orchestra.run.smithi040.stdout: 8 Nov 08:14:06 ntpd[22629]: Listen normally on 3 ens1f0 172.21.15.40:123 2023-11-08T08:14:06.175 INFO:teuthology.orchestra.run.smithi040.stdout: 8 Nov 08:14:06 ntpd[22629]: Listen normally on 4 lo [::1]:123 2023-11-08T08:14:06.176 INFO:teuthology.orchestra.run.smithi040.stdout: 8 Nov 08:14:06 ntpd[22629]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:12cc%4]:123 2023-11-08T08:14:06.176 INFO:teuthology.orchestra.run.smithi040.stdout: 8 Nov 08:14:06 ntpd[22629]: Listening on routing socket on fd #22 for interface updates 2023-11-08T08:14:06.194 INFO:teuthology.orchestra.run.smithi169.stdout: 8 Nov 08:14:06 ntpd[22754]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-11-08T08:14:06.194 INFO:teuthology.orchestra.run.smithi169.stdout: 8 Nov 08:14:06 ntpd[22754]: Command line: ntpd -gq 2023-11-08T08:14:06.195 INFO:teuthology.orchestra.run.smithi169.stdout: 8 Nov 08:14:06 ntpd[22754]: proto: precision = 0.041 usec (-24) 2023-11-08T08:14:06.195 INFO:teuthology.orchestra.run.smithi169.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-08T08:14:06.196 INFO:teuthology.orchestra.run.smithi169.stdout: 8 Nov 08:14:06 ntpd[22754]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-08T08:14:06.196 INFO:teuthology.orchestra.run.smithi169.stderr:restrict ::: KOD does nothing without LIMITED. 2023-11-08T08:14:06.197 INFO:teuthology.orchestra.run.smithi169.stdout: 8 Nov 08:14:06 ntpd[22754]: restrict ::: KOD does nothing without LIMITED. 2023-11-08T08:14:06.197 INFO:teuthology.orchestra.run.smithi169.stdout: 8 Nov 08:14:06 ntpd[22754]: Listen and drop on 0 v6wildcard [::]:123 2023-11-08T08:14:06.197 INFO:teuthology.orchestra.run.smithi169.stdout: 8 Nov 08:14:06 ntpd[22754]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-11-08T08:14:06.198 INFO:teuthology.orchestra.run.smithi169.stdout: 8 Nov 08:14:06 ntpd[22754]: Listen normally on 2 lo 127.0.0.1:123 2023-11-08T08:14:06.198 INFO:teuthology.orchestra.run.smithi169.stdout: 8 Nov 08:14:06 ntpd[22754]: Listen normally on 3 enp3s0f1 172.21.15.169:123 2023-11-08T08:14:06.198 INFO:teuthology.orchestra.run.smithi169.stdout: 8 Nov 08:14:06 ntpd[22754]: Listen normally on 4 lo [::1]:123 2023-11-08T08:14:06.198 INFO:teuthology.orchestra.run.smithi169.stdout: 8 Nov 08:14:06 ntpd[22754]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f4d%5]:123 2023-11-08T08:14:06.198 INFO:teuthology.orchestra.run.smithi169.stdout: 8 Nov 08:14:06 ntpd[22754]: Listening on routing socket on fd #22 for interface updates 2023-11-08T08:14:07.172 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:07 ntpd[22629]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:07.172 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:07 ntpd[22629]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:07.172 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:07 ntpd[22629]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:07.173 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:07 ntpd[22629]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:07.173 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:07 ntpd[22629]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:07.173 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:07 ntpd[22629]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:07.174 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:07 ntpd[22629]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:07.175 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:07 ntpd[22629]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:07.175 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:07 ntpd[22629]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-08T08:14:07.175 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:07 ntpd[22629]: can't open /var/log/ntpstats/peerstats.20231108: Permission denied 2023-11-08T08:14:07.175 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:07 ntpd[22629]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:07.175 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:07 ntpd[22629]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:07.175 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:07 ntpd[22629]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-08T08:14:07.176 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:07 ntpd[22629]: can't open /var/log/ntpstats/peerstats.20231108: Permission denied 2023-11-08T08:14:07.194 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:07 ntpd[22754]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:07.194 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:07 ntpd[22754]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:07.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:07 ntpd[22754]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:07.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:07 ntpd[22754]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:07.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:07 ntpd[22754]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-08T08:14:07.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:07 ntpd[22754]: can't open /var/log/ntpstats/peerstats.20231108: Permission denied 2023-11-08T08:14:08.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:08 ntpd[22754]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:08.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:08 ntpd[22754]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:08.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:08 ntpd[22754]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:08.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:08 ntpd[22754]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:08.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:08 ntpd[22754]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-08T08:14:08.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:08 ntpd[22754]: can't open /var/log/ntpstats/peerstats.20231108: Permission denied 2023-11-08T08:14:09.172 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:09 ntpd[22629]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:09.173 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:09 ntpd[22629]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:09.173 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:09 ntpd[22629]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:09.173 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:09 ntpd[22629]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:09.173 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:09 ntpd[22629]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:09.174 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:09 ntpd[22629]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:09.174 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:09 ntpd[22629]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-08T08:14:09.174 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:09 ntpd[22629]: can't open /var/log/ntpstats/peerstats.20231108: Permission denied 2023-11-08T08:14:09.174 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:09 ntpd[22629]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:09.174 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:09 ntpd[22629]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:09.175 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:09 ntpd[22629]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-08T08:14:09.175 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:09 ntpd[22629]: can't open /var/log/ntpstats/peerstats.20231108: Permission denied 2023-11-08T08:14:09.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:09 ntpd[22754]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:09.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:09 ntpd[22754]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:09.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:09 ntpd[22754]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:09.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:09 ntpd[22754]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:09.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:09 ntpd[22754]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-08T08:14:09.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:09 ntpd[22754]: can't open /var/log/ntpstats/peerstats.20231108: Permission denied 2023-11-08T08:14:10.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:10 ntpd[22754]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:10.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:10 ntpd[22754]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:10.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:10 ntpd[22754]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:10.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:10 ntpd[22754]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:10.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:10 ntpd[22754]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:10.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:10 ntpd[22754]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:10.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:10 ntpd[22754]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-08T08:14:10.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:10 ntpd[22754]: can't open /var/log/ntpstats/peerstats.20231108: Permission denied 2023-11-08T08:14:11.173 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:11 ntpd[22629]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:11.173 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:11 ntpd[22629]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:11.174 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:11 ntpd[22629]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:11.174 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:11 ntpd[22629]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:11.174 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:11 ntpd[22629]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:11.174 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:11 ntpd[22629]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:11.175 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:11 ntpd[22629]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:11.175 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:11 ntpd[22629]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:11.175 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:11 ntpd[22629]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-08T08:14:11.175 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:11 ntpd[22629]: can't open /var/log/ntpstats/peerstats.20231108: Permission denied 2023-11-08T08:14:11.175 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:11 ntpd[22629]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:11.176 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:11 ntpd[22629]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:11.176 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:11 ntpd[22629]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-08T08:14:11.176 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:11 ntpd[22629]: can't open /var/log/ntpstats/peerstats.20231108: Permission denied 2023-11-08T08:14:11.176 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:11 ntpd[22629]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:11.177 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:11 ntpd[22629]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:11.177 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:11 ntpd[22629]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-08T08:14:11.177 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:11 ntpd[22629]: can't open /var/log/ntpstats/peerstats.20231108: Permission denied 2023-11-08T08:14:11.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:11 ntpd[22754]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:11.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:11 ntpd[22754]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:11.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:11 ntpd[22754]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:11.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:11 ntpd[22754]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:11.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:11 ntpd[22754]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:11.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:11 ntpd[22754]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:11.197 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:11 ntpd[22754]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-08T08:14:11.197 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:11 ntpd[22754]: can't open /var/log/ntpstats/peerstats.20231108: Permission denied 2023-11-08T08:14:11.197 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:11 ntpd[22754]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:11.197 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:11 ntpd[22754]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:11.197 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:11 ntpd[22754]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-08T08:14:11.198 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:11 ntpd[22754]: can't open /var/log/ntpstats/peerstats.20231108: Permission denied 2023-11-08T08:14:12.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:12 ntpd[22754]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:12.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:12 ntpd[22754]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:12.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:12 ntpd[22754]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:12.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:12 ntpd[22754]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:12.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:12 ntpd[22754]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-08T08:14:12.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:12 ntpd[22754]: can't open /var/log/ntpstats/peerstats.20231108: Permission denied 2023-11-08T08:14:13.173 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:13 ntpd[22629]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:13.173 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:13 ntpd[22629]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:13.173 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:13 ntpd[22629]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:13.173 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:13 ntpd[22629]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:13.174 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:13 ntpd[22629]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:13.174 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:13 ntpd[22629]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:13.174 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:13 ntpd[22629]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:13.174 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:13 ntpd[22629]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:13.175 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:13 ntpd[22629]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-08T08:14:13.175 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:13 ntpd[22629]: can't open /var/log/ntpstats/peerstats.20231108: Permission denied 2023-11-08T08:14:13.175 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:13 ntpd[22629]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:13.175 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:13 ntpd[22629]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:13.175 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:13 ntpd[22629]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-08T08:14:13.176 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:13 ntpd[22629]: can't open /var/log/ntpstats/peerstats.20231108: Permission denied 2023-11-08T08:14:13.176 INFO:teuthology.orchestra.run.smithi040.stdout: 8 Nov 08:14:13 ntpd[22629]: ntpd: time slew +0.001989 s 2023-11-08T08:14:13.176 INFO:teuthology.orchestra.run.smithi040.stdout:ntpd: time slew +0.001989s 2023-11-08T08:14:13.177 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:13 ntpd[22629]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-11-08T08:14:13.177 INFO:teuthology.orchestra.run.smithi040.stderr: 8 Nov 08:14:13 ntpd[22629]: can't open /var/log/ntpstats/loopstats.20231108: Permission denied 2023-11-08T08:14:13.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:13 ntpd[22754]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:13.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:13 ntpd[22754]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:13.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:13 ntpd[22754]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:13.195 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:13 ntpd[22754]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:13.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:13 ntpd[22754]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-08T08:14:13.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:13 ntpd[22754]: can't open /var/log/ntpstats/rawstats.20231108: Permission denied 2023-11-08T08:14:13.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:13 ntpd[22754]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-08T08:14:13.196 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:13 ntpd[22754]: can't open /var/log/ntpstats/peerstats.20231108: Permission denied 2023-11-08T08:14:13.197 INFO:teuthology.orchestra.run.smithi169.stdout: 8 Nov 08:14:13 ntpd[22754]: ntpd: time slew +0.001344 s 2023-11-08T08:14:13.197 INFO:teuthology.orchestra.run.smithi169.stdout:ntpd: time slew +0.001344s 2023-11-08T08:14:13.197 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:13 ntpd[22754]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-11-08T08:14:13.198 INFO:teuthology.orchestra.run.smithi169.stderr: 8 Nov 08:14:13 ntpd[22754]: can't open /var/log/ntpstats/loopstats.20231108: Permission denied 2023-11-08T08:14:13.312 INFO:teuthology.orchestra.run.smithi040.stdout: remote refid st t when poll reach delay offset jitter 2023-11-08T08:14:13.312 INFO:teuthology.orchestra.run.smithi040.stdout:============================================================================== 2023-11-08T08:14:13.313 INFO:teuthology.orchestra.run.smithi040.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-11-08T08:14:13.313 INFO:teuthology.orchestra.run.smithi040.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-11-08T08:14:13.313 INFO:teuthology.orchestra.run.smithi040.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-11-08T08:14:13.313 INFO:teuthology.orchestra.run.smithi040.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-11-08T08:14:13.357 INFO:teuthology.orchestra.run.smithi169.stdout: remote refid st t when poll reach delay offset jitter 2023-11-08T08:14:13.357 INFO:teuthology.orchestra.run.smithi169.stdout:============================================================================== 2023-11-08T08:14:13.357 INFO:teuthology.orchestra.run.smithi169.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-11-08T08:14:13.357 INFO:teuthology.orchestra.run.smithi169.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-11-08T08:14:13.358 INFO:teuthology.orchestra.run.smithi169.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-11-08T08:14:13.358 INFO:teuthology.orchestra.run.smithi169.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-11-08T08:14:13.359 INFO:teuthology.run_tasks:Running task install... 2023-11-08T08:14:13.368 DEBUG:teuthology.task.install:project ceph 2023-11-08T08:14:13.369 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b820560f30d310ab2570761b9a294523225d6998'}} 2023-11-08T08:14:13.369 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b820560f30d310ab2570761b9a294523225d6998'} 2023-11-08T08:14:13.369 INFO:teuthology.task.install:Using flavor: default 2023-11-08T08:14:13.375 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-08T08:14:13.375 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-08T08:14:13.376 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-key list | grep Ceph 2023-11-08T08:14:13.377 DEBUG:teuthology.orchestra.run.smithi169:> sudo apt-key list | grep Ceph 2023-11-08T08:14:13.596 INFO:teuthology.orchestra.run.smithi169.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-11-08T08:14:13.621 INFO:teuthology.orchestra.run.smithi040.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-11-08T08:14:13.739 INFO:teuthology.orchestra.run.smithi169.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-11-08T08:14:13.739 INFO:teuthology.orchestra.run.smithi169.stdout:uid [ unknown] Ceph.com (release key) 2023-11-08T08:14:13.741 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-11-08T08:14:13.741 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-11-08T08:14:13.742 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=b820560f30d310ab2570761b9a294523225d6998 2023-11-08T08:14:13.774 INFO:teuthology.orchestra.run.smithi040.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-11-08T08:14:13.775 INFO:teuthology.orchestra.run.smithi040.stdout:uid [ unknown] Ceph.com (release key) 2023-11-08T08:14:13.775 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-11-08T08:14:13.775 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-11-08T08:14:13.775 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=b820560f30d310ab2570761b9a294523225d6998 2023-11-08T08:14:13.892 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default/ 2023-11-08T08:14:13.893 INFO:teuthology.task.install.deb:Package version is 18.0.0-2698-gb820560f-1focal 2023-11-08T08:14:14.002 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default/ 2023-11-08T08:14:14.003 INFO:teuthology.task.install.deb:Package version is 18.0.0-2698-gb820560f-1focal 2023-11-08T08:14:14.077 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:14:14.078 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-11-08T08:14:14.112 DEBUG:teuthology.orchestra.run.smithi169:> sudo apt-get update 2023-11-08T08:14:14.169 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:14:14.169 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-11-08T08:14:14.206 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get update 2023-11-08T08:14:14.422 INFO:teuthology.orchestra.run.smithi169.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-11-08T08:14:14.494 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-11-08T08:14:14.494 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-11-08T08:14:14.500 INFO:teuthology.orchestra.run.smithi169.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-11-08T08:14:14.500 INFO:teuthology.orchestra.run.smithi169.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal InRelease 2023-11-08T08:14:14.509 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-11-08T08:14:14.537 INFO:teuthology.orchestra.run.smithi169.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-11-08T08:14:14.576 INFO:teuthology.orchestra.run.smithi169.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal Release.gpg 2023-11-08T08:14:14.582 INFO:teuthology.orchestra.run.smithi169.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-11-08T08:14:14.601 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-11-08T08:14:14.601 INFO:teuthology.orchestra.run.smithi040.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal InRelease 2023-11-08T08:14:14.613 INFO:teuthology.orchestra.run.smithi169.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 Packages [17.3 kB] 2023-11-08T08:14:14.638 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-11-08T08:14:14.669 INFO:teuthology.orchestra.run.smithi169.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main i386 Packages [3,421 B] 2023-11-08T08:14:14.677 INFO:teuthology.orchestra.run.smithi040.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal Release.gpg 2023-11-08T08:14:14.713 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 Packages [17.3 kB] 2023-11-08T08:14:14.807 INFO:teuthology.orchestra.run.smithi040.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main i386 Packages [3,421 B] 2023-11-08T08:14:14.934 INFO:teuthology.orchestra.run.smithi169.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-11-08T08:14:15.828 INFO:teuthology.orchestra.run.smithi169.stdout:Fetched 142 kB in 1s (104 kB/s) 2023-11-08T08:14:15.969 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 28.4 kB in 1s (20.1 kB/s) 2023-11-08T08:14:17.047 INFO:teuthology.orchestra.run.smithi169.stdout:Reading package lists... 2023-11-08T08:14:17.083 DEBUG:teuthology.orchestra.run.smithi169:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-2698-gb820560f-1focal cephadm=18.0.0-2698-gb820560f-1focal ceph-mds=18.0.0-2698-gb820560f-1focal ceph-mgr=18.0.0-2698-gb820560f-1focal ceph-common=18.0.0-2698-gb820560f-1focal ceph-fuse=18.0.0-2698-gb820560f-1focal ceph-test=18.0.0-2698-gb820560f-1focal ceph-volume=18.0.0-2698-gb820560f-1focal radosgw=18.0.0-2698-gb820560f-1focal python3-rados=18.0.0-2698-gb820560f-1focal python3-rgw=18.0.0-2698-gb820560f-1focal python3-cephfs=18.0.0-2698-gb820560f-1focal python3-rbd=18.0.0-2698-gb820560f-1focal libcephfs2=18.0.0-2698-gb820560f-1focal libcephfs-dev=18.0.0-2698-gb820560f-1focal librados2=18.0.0-2698-gb820560f-1focal librbd1=18.0.0-2698-gb820560f-1focal rbd-fuse=18.0.0-2698-gb820560f-1focal python3-cephfs=18.0.0-2698-gb820560f-1focal cephfs-shell=18.0.0-2698-gb820560f-1focal cephfs-top=18.0.0-2698-gb820560f-1focal cephfs-mirror=18.0.0-2698-gb820560f-1focal 2023-11-08T08:14:17.207 INFO:teuthology.orchestra.run.smithi169.stdout:Reading package lists... 2023-11-08T08:14:17.229 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2023-11-08T08:14:17.277 DEBUG:teuthology.orchestra.run.smithi040:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-2698-gb820560f-1focal cephadm=18.0.0-2698-gb820560f-1focal ceph-mds=18.0.0-2698-gb820560f-1focal ceph-mgr=18.0.0-2698-gb820560f-1focal ceph-common=18.0.0-2698-gb820560f-1focal ceph-fuse=18.0.0-2698-gb820560f-1focal ceph-test=18.0.0-2698-gb820560f-1focal ceph-volume=18.0.0-2698-gb820560f-1focal radosgw=18.0.0-2698-gb820560f-1focal python3-rados=18.0.0-2698-gb820560f-1focal python3-rgw=18.0.0-2698-gb820560f-1focal python3-cephfs=18.0.0-2698-gb820560f-1focal python3-rbd=18.0.0-2698-gb820560f-1focal libcephfs2=18.0.0-2698-gb820560f-1focal libcephfs-dev=18.0.0-2698-gb820560f-1focal librados2=18.0.0-2698-gb820560f-1focal librbd1=18.0.0-2698-gb820560f-1focal rbd-fuse=18.0.0-2698-gb820560f-1focal python3-cephfs=18.0.0-2698-gb820560f-1focal cephfs-shell=18.0.0-2698-gb820560f-1focal cephfs-top=18.0.0-2698-gb820560f-1focal cephfs-mirror=18.0.0-2698-gb820560f-1focal 2023-11-08T08:14:17.387 INFO:teuthology.orchestra.run.smithi169.stdout:Building dependency tree... 2023-11-08T08:14:17.387 INFO:teuthology.orchestra.run.smithi169.stdout:Reading state information... 2023-11-08T08:14:17.404 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2023-11-08T08:14:17.559 INFO:teuthology.orchestra.run.smithi169.stdout:The following packages were automatically installed and are no longer required: 2023-11-08T08:14:17.560 INFO:teuthology.orchestra.run.smithi169.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-11-08T08:14:17.560 INFO:teuthology.orchestra.run.smithi169.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-08T08:14:17.561 INFO:teuthology.orchestra.run.smithi169.stdout:The following additional packages will be installed: 2023-11-08T08:14:17.561 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-11-08T08:14:17.562 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-11-08T08:14:17.562 INFO:teuthology.orchestra.run.smithi169.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-11-08T08:14:17.562 INFO:teuthology.orchestra.run.smithi169.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-11-08T08:14:17.562 INFO:teuthology.orchestra.run.smithi169.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-11-08T08:14:17.563 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-11-08T08:14:17.563 INFO:teuthology.orchestra.run.smithi169.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-11-08T08:14:17.563 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-11-08T08:14:17.563 INFO:teuthology.orchestra.run.smithi169.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-11-08T08:14:17.563 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2023-11-08T08:14:17.564 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2023-11-08T08:14:17.564 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2023-11-08T08:14:17.564 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2023-11-08T08:14:17.564 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2023-11-08T08:14:17.564 INFO:teuthology.orchestra.run.smithi169.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-11-08T08:14:17.565 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2023-11-08T08:14:17.565 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2023-11-08T08:14:17.565 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2023-11-08T08:14:17.565 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-11-08T08:14:17.565 INFO:teuthology.orchestra.run.smithi169.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-11-08T08:14:17.566 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-11-08T08:14:17.566 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-11-08T08:14:17.566 INFO:teuthology.orchestra.run.smithi169.stdout: qttranslations5-l10n socat xmlstarlet 2023-11-08T08:14:17.567 INFO:teuthology.orchestra.run.smithi169.stdout:Suggested packages: 2023-11-08T08:14:17.567 INFO:teuthology.orchestra.run.smithi169.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-11-08T08:14:17.567 INFO:teuthology.orchestra.run.smithi169.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-11-08T08:14:17.567 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-11-08T08:14:17.567 INFO:teuthology.orchestra.run.smithi169.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-11-08T08:14:17.568 INFO:teuthology.orchestra.run.smithi169.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-11-08T08:14:17.568 INFO:teuthology.orchestra.run.smithi169.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-11-08T08:14:17.568 INFO:teuthology.orchestra.run.smithi169.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-11-08T08:14:17.568 INFO:teuthology.orchestra.run.smithi169.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-11-08T08:14:17.568 INFO:teuthology.orchestra.run.smithi169.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-11-08T08:14:17.568 INFO:teuthology.orchestra.run.smithi169.stdout:Recommended packages: 2023-11-08T08:14:17.569 INFO:teuthology.orchestra.run.smithi169.stdout: btrfs-tools 2023-11-08T08:14:17.598 INFO:teuthology.orchestra.run.smithi040.stdout:Building dependency tree... 2023-11-08T08:14:17.599 INFO:teuthology.orchestra.run.smithi040.stdout:Reading state information... 2023-11-08T08:14:17.770 INFO:teuthology.orchestra.run.smithi169.stdout:The following NEW packages will be installed: 2023-11-08T08:14:17.770 INFO:teuthology.orchestra.run.smithi169.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-11-08T08:14:17.771 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-11-08T08:14:17.771 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-11-08T08:14:17.771 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-11-08T08:14:17.771 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-11-08T08:14:17.771 INFO:teuthology.orchestra.run.smithi169.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-11-08T08:14:17.772 INFO:teuthology.orchestra.run.smithi169.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-11-08T08:14:17.773 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-11-08T08:14:17.773 INFO:teuthology.orchestra.run.smithi169.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-11-08T08:14:17.773 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-11-08T08:14:17.773 INFO:teuthology.orchestra.run.smithi169.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-11-08T08:14:17.774 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2023-11-08T08:14:17.774 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-11-08T08:14:17.774 INFO:teuthology.orchestra.run.smithi169.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-11-08T08:14:17.774 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-11-08T08:14:17.774 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-11-08T08:14:17.775 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-11-08T08:14:17.775 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-11-08T08:14:17.775 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-11-08T08:14:17.775 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-11-08T08:14:17.775 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-11-08T08:14:17.775 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-11-08T08:14:17.776 INFO:teuthology.orchestra.run.smithi169.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-11-08T08:14:17.776 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-11-08T08:14:17.776 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-11-08T08:14:17.776 INFO:teuthology.orchestra.run.smithi169.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-11-08T08:14:17.776 INFO:teuthology.orchestra.run.smithi169.stdout:The following packages will be upgraded: 2023-11-08T08:14:17.777 INFO:teuthology.orchestra.run.smithi169.stdout: librados2 librbd1 2023-11-08T08:14:17.791 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages were automatically installed and are no longer required: 2023-11-08T08:14:17.793 INFO:teuthology.orchestra.run.smithi040.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-11-08T08:14:17.793 INFO:teuthology.orchestra.run.smithi040.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-08T08:14:17.794 INFO:teuthology.orchestra.run.smithi040.stdout:The following additional packages will be installed: 2023-11-08T08:14:17.794 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-11-08T08:14:17.794 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-11-08T08:14:17.794 INFO:teuthology.orchestra.run.smithi040.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-11-08T08:14:17.794 INFO:teuthology.orchestra.run.smithi040.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-11-08T08:14:17.795 INFO:teuthology.orchestra.run.smithi040.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-11-08T08:14:17.795 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-11-08T08:14:17.796 INFO:teuthology.orchestra.run.smithi040.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-11-08T08:14:17.796 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-11-08T08:14:17.796 INFO:teuthology.orchestra.run.smithi040.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-11-08T08:14:17.796 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2023-11-08T08:14:17.796 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2023-11-08T08:14:17.797 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2023-11-08T08:14:17.797 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2023-11-08T08:14:17.797 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2023-11-08T08:14:17.797 INFO:teuthology.orchestra.run.smithi040.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-11-08T08:14:17.797 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2023-11-08T08:14:17.798 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2023-11-08T08:14:17.798 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2023-11-08T08:14:17.798 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-11-08T08:14:17.798 INFO:teuthology.orchestra.run.smithi040.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-11-08T08:14:17.799 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-11-08T08:14:17.799 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-11-08T08:14:17.799 INFO:teuthology.orchestra.run.smithi040.stdout: qttranslations5-l10n socat xmlstarlet 2023-11-08T08:14:17.799 INFO:teuthology.orchestra.run.smithi040.stdout:Suggested packages: 2023-11-08T08:14:17.799 INFO:teuthology.orchestra.run.smithi040.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-11-08T08:14:17.800 INFO:teuthology.orchestra.run.smithi040.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-11-08T08:14:17.800 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-11-08T08:14:17.800 INFO:teuthology.orchestra.run.smithi040.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-11-08T08:14:17.800 INFO:teuthology.orchestra.run.smithi040.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-11-08T08:14:17.800 INFO:teuthology.orchestra.run.smithi040.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-11-08T08:14:17.801 INFO:teuthology.orchestra.run.smithi040.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-11-08T08:14:17.801 INFO:teuthology.orchestra.run.smithi040.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-11-08T08:14:17.801 INFO:teuthology.orchestra.run.smithi040.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-11-08T08:14:17.801 INFO:teuthology.orchestra.run.smithi040.stdout:Recommended packages: 2023-11-08T08:14:17.801 INFO:teuthology.orchestra.run.smithi040.stdout: btrfs-tools 2023-11-08T08:14:17.993 INFO:teuthology.orchestra.run.smithi169.stdout:2 upgraded, 116 newly installed, 0 to remove and 258 not upgraded. 2023-11-08T08:14:17.993 INFO:teuthology.orchestra.run.smithi169.stdout:Need to get 207 MB of archives. 2023-11-08T08:14:17.993 INFO:teuthology.orchestra.run.smithi169.stdout:After this operation, 1,026 MB of additional disk space will be used. 2023-11-08T08:14:17.993 INFO:teuthology.orchestra.run.smithi169.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-11-08T08:14:18.010 INFO:teuthology.orchestra.run.smithi040.stdout:The following NEW packages will be installed: 2023-11-08T08:14:18.010 INFO:teuthology.orchestra.run.smithi040.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-11-08T08:14:18.010 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-11-08T08:14:18.011 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-11-08T08:14:18.011 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-11-08T08:14:18.011 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-11-08T08:14:18.011 INFO:teuthology.orchestra.run.smithi040.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-11-08T08:14:18.011 INFO:teuthology.orchestra.run.smithi040.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-11-08T08:14:18.012 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-11-08T08:14:18.013 INFO:teuthology.orchestra.run.smithi040.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-11-08T08:14:18.013 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-11-08T08:14:18.013 INFO:teuthology.orchestra.run.smithi040.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-11-08T08:14:18.013 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2023-11-08T08:14:18.014 INFO:teuthology.orchestra.run.smithi040.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-11-08T08:14:18.014 INFO:teuthology.orchestra.run.smithi040.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-11-08T08:14:18.014 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-11-08T08:14:18.014 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-11-08T08:14:18.014 INFO:teuthology.orchestra.run.smithi040.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-11-08T08:14:18.015 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-11-08T08:14:18.015 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-11-08T08:14:18.015 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-11-08T08:14:18.015 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-11-08T08:14:18.015 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-11-08T08:14:18.015 INFO:teuthology.orchestra.run.smithi040.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-11-08T08:14:18.016 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-11-08T08:14:18.016 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-11-08T08:14:18.016 INFO:teuthology.orchestra.run.smithi040.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-11-08T08:14:18.016 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages will be upgraded: 2023-11-08T08:14:18.016 INFO:teuthology.orchestra.run.smithi040.stdout: librados2 librbd1 2023-11-08T08:14:18.077 INFO:teuthology.orchestra.run.smithi169.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-11-08T08:14:18.085 INFO:teuthology.orchestra.run.smithi169.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.0.0-2698-gb820560f-1focal [3,433 kB] 2023-11-08T08:14:18.112 INFO:teuthology.orchestra.run.smithi169.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-11-08T08:14:18.113 INFO:teuthology.orchestra.run.smithi040.stdout:2 upgraded, 116 newly installed, 0 to remove and 258 not upgraded. 2023-11-08T08:14:18.113 INFO:teuthology.orchestra.run.smithi040.stdout:Need to get 207 MB of archives. 2023-11-08T08:14:18.113 INFO:teuthology.orchestra.run.smithi040.stdout:After this operation, 1,026 MB of additional disk space will be used. 2023-11-08T08:14:18.113 INFO:teuthology.orchestra.run.smithi040.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-11-08T08:14:18.138 INFO:teuthology.orchestra.run.smithi040.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-11-08T08:14:18.149 INFO:teuthology.orchestra.run.smithi040.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-11-08T08:14:18.189 INFO:teuthology.orchestra.run.smithi040.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-11-08T08:14:18.233 INFO:teuthology.orchestra.run.smithi040.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-11-08T08:14:18.239 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-11-08T08:14:18.251 INFO:teuthology.orchestra.run.smithi169.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-11-08T08:14:18.261 INFO:teuthology.orchestra.run.smithi040.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-11-08T08:14:18.314 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.0.0-2698-gb820560f-1focal [3,433 kB] 2023-11-08T08:14:18.372 INFO:teuthology.orchestra.run.smithi040.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-11-08T08:14:18.375 INFO:teuthology.orchestra.run.smithi040.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-11-08T08:14:18.387 INFO:teuthology.orchestra.run.smithi169.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-11-08T08:14:18.398 INFO:teuthology.orchestra.run.smithi040.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-11-08T08:14:18.404 INFO:teuthology.orchestra.run.smithi040.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-11-08T08:14:18.405 INFO:teuthology.orchestra.run.smithi169.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-11-08T08:14:18.410 INFO:teuthology.orchestra.run.smithi040.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-11-08T08:14:18.414 INFO:teuthology.orchestra.run.smithi040.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-11-08T08:14:18.415 INFO:teuthology.orchestra.run.smithi040.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-11-08T08:14:18.426 INFO:teuthology.orchestra.run.smithi040.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-11-08T08:14:18.428 INFO:teuthology.orchestra.run.smithi040.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-11-08T08:14:18.430 INFO:teuthology.orchestra.run.smithi040.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-11-08T08:14:18.466 INFO:teuthology.orchestra.run.smithi169.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-11-08T08:14:18.470 INFO:teuthology.orchestra.run.smithi040.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-11-08T08:14:18.471 INFO:teuthology.orchestra.run.smithi040.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-11-08T08:14:18.472 INFO:teuthology.orchestra.run.smithi040.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-11-08T08:14:18.475 INFO:teuthology.orchestra.run.smithi040.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-11-08T08:14:18.475 INFO:teuthology.orchestra.run.smithi040.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-11-08T08:14:18.476 INFO:teuthology.orchestra.run.smithi040.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-11-08T08:14:18.478 INFO:teuthology.orchestra.run.smithi040.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-11-08T08:14:18.480 INFO:teuthology.orchestra.run.smithi040.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-11-08T08:14:18.510 INFO:teuthology.orchestra.run.smithi040.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-11-08T08:14:18.512 INFO:teuthology.orchestra.run.smithi040.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-11-08T08:14:18.547 INFO:teuthology.orchestra.run.smithi040.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-11-08T08:14:18.558 INFO:teuthology.orchestra.run.smithi040.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-11-08T08:14:18.559 INFO:teuthology.orchestra.run.smithi040.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-11-08T08:14:18.560 INFO:teuthology.orchestra.run.smithi040.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-11-08T08:14:18.562 INFO:teuthology.orchestra.run.smithi040.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-11-08T08:14:18.564 INFO:teuthology.orchestra.run.smithi040.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-11-08T08:14:18.566 INFO:teuthology.orchestra.run.smithi040.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-11-08T08:14:18.568 INFO:teuthology.orchestra.run.smithi040.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-11-08T08:14:18.591 INFO:teuthology.orchestra.run.smithi040.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2023-11-08T08:14:18.594 INFO:teuthology.orchestra.run.smithi040.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-11-08T08:14:18.633 INFO:teuthology.orchestra.run.smithi040.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-11-08T08:14:18.635 INFO:teuthology.orchestra.run.smithi040.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-11-08T08:14:18.636 INFO:teuthology.orchestra.run.smithi040.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-11-08T08:14:18.639 INFO:teuthology.orchestra.run.smithi040.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-11-08T08:14:18.640 INFO:teuthology.orchestra.run.smithi040.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-11-08T08:14:18.643 INFO:teuthology.orchestra.run.smithi040.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-11-08T08:14:18.644 INFO:teuthology.orchestra.run.smithi040.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-11-08T08:14:18.645 INFO:teuthology.orchestra.run.smithi040.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-11-08T08:14:18.668 INFO:teuthology.orchestra.run.smithi169.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-11-08T08:14:18.675 INFO:teuthology.orchestra.run.smithi169.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-11-08T08:14:18.676 INFO:teuthology.orchestra.run.smithi040.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-11-08T08:14:18.677 INFO:teuthology.orchestra.run.smithi040.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-11-08T08:14:18.702 INFO:teuthology.orchestra.run.smithi169.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-11-08T08:14:18.711 INFO:teuthology.orchestra.run.smithi169.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-11-08T08:14:18.718 INFO:teuthology.orchestra.run.smithi040.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-11-08T08:14:18.722 INFO:teuthology.orchestra.run.smithi040.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-11-08T08:14:18.725 INFO:teuthology.orchestra.run.smithi040.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-11-08T08:14:18.726 INFO:teuthology.orchestra.run.smithi040.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-11-08T08:14:18.728 INFO:teuthology.orchestra.run.smithi040.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-11-08T08:14:18.876 INFO:teuthology.orchestra.run.smithi169.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-11-08T08:14:18.974 INFO:teuthology.orchestra.run.smithi040.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-11-08T08:14:18.984 INFO:teuthology.orchestra.run.smithi169.stdout:Get:14 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.0.0-2698-gb820560f-1focal [3,982 kB] 2023-11-08T08:14:19.007 INFO:teuthology.orchestra.run.smithi040.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-11-08T08:14:19.009 INFO:teuthology.orchestra.run.smithi040.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-11-08T08:14:19.034 INFO:teuthology.orchestra.run.smithi040.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-11-08T08:14:19.034 INFO:teuthology.orchestra.run.smithi040.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-11-08T08:14:19.035 INFO:teuthology.orchestra.run.smithi040.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-11-08T08:14:19.035 INFO:teuthology.orchestra.run.smithi040.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-11-08T08:14:19.036 INFO:teuthology.orchestra.run.smithi040.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-11-08T08:14:19.037 INFO:teuthology.orchestra.run.smithi040.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-11-08T08:14:19.037 INFO:teuthology.orchestra.run.smithi040.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-11-08T08:14:19.049 INFO:teuthology.orchestra.run.smithi040.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-11-08T08:14:19.050 INFO:teuthology.orchestra.run.smithi040.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-11-08T08:14:19.069 INFO:teuthology.orchestra.run.smithi040.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-11-08T08:14:19.076 INFO:teuthology.orchestra.run.smithi040.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-11-08T08:14:19.081 INFO:teuthology.orchestra.run.smithi040.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-11-08T08:14:19.083 INFO:teuthology.orchestra.run.smithi040.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-11-08T08:14:19.084 INFO:teuthology.orchestra.run.smithi040.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-11-08T08:14:19.087 INFO:teuthology.orchestra.run.smithi040.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-11-08T08:14:19.090 INFO:teuthology.orchestra.run.smithi040.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-11-08T08:14:19.091 INFO:teuthology.orchestra.run.smithi040.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-11-08T08:14:19.110 INFO:teuthology.orchestra.run.smithi040.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-11-08T08:14:19.113 INFO:teuthology.orchestra.run.smithi040.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2023-11-08T08:14:19.130 INFO:teuthology.orchestra.run.smithi040.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-11-08T08:14:19.208 INFO:teuthology.orchestra.run.smithi040.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-11-08T08:14:19.209 INFO:teuthology.orchestra.run.smithi040.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-11-08T08:14:19.210 INFO:teuthology.orchestra.run.smithi040.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-11-08T08:14:19.210 INFO:teuthology.orchestra.run.smithi040.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-11-08T08:14:19.229 INFO:teuthology.orchestra.run.smithi040.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-11-08T08:14:19.229 INFO:teuthology.orchestra.run.smithi040.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-11-08T08:14:19.231 INFO:teuthology.orchestra.run.smithi040.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-11-08T08:14:19.232 INFO:teuthology.orchestra.run.smithi040.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-11-08T08:14:19.232 INFO:teuthology.orchestra.run.smithi040.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-11-08T08:14:19.246 INFO:teuthology.orchestra.run.smithi169.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-11-08T08:14:19.268 INFO:teuthology.orchestra.run.smithi169.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-11-08T08:14:19.446 INFO:teuthology.orchestra.run.smithi169.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-11-08T08:14:19.456 INFO:teuthology.orchestra.run.smithi169.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-11-08T08:14:19.494 INFO:teuthology.orchestra.run.smithi169.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-11-08T08:14:19.499 INFO:teuthology.orchestra.run.smithi169.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-11-08T08:14:19.501 INFO:teuthology.orchestra.run.smithi169.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-11-08T08:14:19.504 INFO:teuthology.orchestra.run.smithi169.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-11-08T08:14:19.513 INFO:teuthology.orchestra.run.smithi169.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-11-08T08:14:19.514 INFO:teuthology.orchestra.run.smithi169.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-11-08T08:14:19.597 INFO:teuthology.orchestra.run.smithi169.stdout:Get:25 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.0.0-2698-gb820560f-1focal [722 kB] 2023-11-08T08:14:19.672 INFO:teuthology.orchestra.run.smithi169.stdout:Get:26 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.0.0-2698-gb820560f-1focal [376 kB] 2023-11-08T08:14:19.679 INFO:teuthology.orchestra.run.smithi169.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-11-08T08:14:19.736 INFO:teuthology.orchestra.run.smithi169.stdout:Get:28 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.0.0-2698-gb820560f-1focal [22.4 kB] 2023-11-08T08:14:19.738 INFO:teuthology.orchestra.run.smithi169.stdout:Get:29 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.0.0-2698-gb820560f-1focal [184 kB] 2023-11-08T08:14:19.744 INFO:teuthology.orchestra.run.smithi169.stdout:Get:30 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.0.0-2698-gb820560f-1focal [47.7 kB] 2023-11-08T08:14:19.757 INFO:teuthology.orchestra.run.smithi040.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.0.0-2698-gb820560f-1focal [3,982 kB] 2023-11-08T08:14:19.771 INFO:teuthology.orchestra.run.smithi169.stdout:Get:31 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.0.0-2698-gb820560f-1focal [355 kB] 2023-11-08T08:14:19.811 INFO:teuthology.orchestra.run.smithi169.stdout:Get:32 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.0.0-2698-gb820560f-1focal [6,531 kB] 2023-11-08T08:14:20.017 INFO:teuthology.orchestra.run.smithi169.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-11-08T08:14:20.040 INFO:teuthology.orchestra.run.smithi169.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-11-08T08:14:20.110 INFO:teuthology.orchestra.run.smithi169.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-11-08T08:14:20.173 INFO:teuthology.orchestra.run.smithi169.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-11-08T08:14:20.180 INFO:teuthology.orchestra.run.smithi169.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-11-08T08:14:20.229 INFO:teuthology.orchestra.run.smithi040.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.0.0-2698-gb820560f-1focal [722 kB] 2023-11-08T08:14:20.296 INFO:teuthology.orchestra.run.smithi040.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.0.0-2698-gb820560f-1focal [376 kB] 2023-11-08T08:14:20.300 INFO:teuthology.orchestra.run.smithi169.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-11-08T08:14:20.301 INFO:teuthology.orchestra.run.smithi169.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-11-08T08:14:20.305 INFO:teuthology.orchestra.run.smithi169.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-11-08T08:14:20.308 INFO:teuthology.orchestra.run.smithi040.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.0.0-2698-gb820560f-1focal [22.4 kB] 2023-11-08T08:14:20.309 INFO:teuthology.orchestra.run.smithi169.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-11-08T08:14:20.310 INFO:teuthology.orchestra.run.smithi040.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.0.0-2698-gb820560f-1focal [184 kB] 2023-11-08T08:14:20.321 INFO:teuthology.orchestra.run.smithi169.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-11-08T08:14:20.336 INFO:teuthology.orchestra.run.smithi040.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.0.0-2698-gb820560f-1focal [47.7 kB] 2023-11-08T08:14:20.338 INFO:teuthology.orchestra.run.smithi040.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.0.0-2698-gb820560f-1focal [355 kB] 2023-11-08T08:14:20.370 INFO:teuthology.orchestra.run.smithi040.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.0.0-2698-gb820560f-1focal [6,531 kB] 2023-11-08T08:14:20.507 INFO:teuthology.orchestra.run.smithi169.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-11-08T08:14:20.596 INFO:teuthology.orchestra.run.smithi169.stdout:Get:44 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.0.0-2698-gb820560f-1focal [103 kB] 2023-11-08T08:14:20.604 INFO:teuthology.orchestra.run.smithi169.stdout:Get:45 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.0.0-2698-gb820560f-1focal [467 kB] 2023-11-08T08:14:20.646 INFO:teuthology.orchestra.run.smithi169.stdout:Get:46 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.0.0-2698-gb820560f-1focal [25.0 MB] 2023-11-08T08:14:20.744 INFO:teuthology.orchestra.run.smithi040.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.0.0-2698-gb820560f-1focal [103 kB] 2023-11-08T08:14:20.746 INFO:teuthology.orchestra.run.smithi040.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.0.0-2698-gb820560f-1focal [467 kB] 2023-11-08T08:14:20.766 INFO:teuthology.orchestra.run.smithi040.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.0.0-2698-gb820560f-1focal [25.0 MB] 2023-11-08T08:14:20.958 INFO:teuthology.orchestra.run.smithi169.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-11-08T08:14:20.971 INFO:teuthology.orchestra.run.smithi169.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2023-11-08T08:14:21.018 INFO:teuthology.orchestra.run.smithi169.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-11-08T08:14:21.060 INFO:teuthology.orchestra.run.smithi169.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-11-08T08:14:21.069 INFO:teuthology.orchestra.run.smithi169.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-11-08T08:14:21.075 INFO:teuthology.orchestra.run.smithi169.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-11-08T08:14:21.085 INFO:teuthology.orchestra.run.smithi169.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-11-08T08:14:21.090 INFO:teuthology.orchestra.run.smithi169.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-11-08T08:14:21.107 INFO:teuthology.orchestra.run.smithi169.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-11-08T08:14:21.109 INFO:teuthology.orchestra.run.smithi169.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-11-08T08:14:21.280 INFO:teuthology.orchestra.run.smithi169.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-11-08T08:14:21.705 INFO:teuthology.orchestra.run.smithi169.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-11-08T08:14:21.709 INFO:teuthology.orchestra.run.smithi169.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-11-08T08:14:21.742 INFO:teuthology.orchestra.run.smithi169.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-11-08T08:14:21.791 INFO:teuthology.orchestra.run.smithi040.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.0.0-2698-gb820560f-1focal [10.9 MB] 2023-11-08T08:14:21.793 INFO:teuthology.orchestra.run.smithi169.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-11-08T08:14:21.822 INFO:teuthology.orchestra.run.smithi169.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-11-08T08:14:21.825 INFO:teuthology.orchestra.run.smithi169.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-11-08T08:14:21.835 INFO:teuthology.orchestra.run.smithi169.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-11-08T08:14:22.197 INFO:teuthology.orchestra.run.smithi040.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.0.0-2698-gb820560f-1focal [221 kB] 2023-11-08T08:14:22.200 INFO:teuthology.orchestra.run.smithi040.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.0.0-2698-gb820560f-1focal [133 kB] 2023-11-08T08:14:22.204 INFO:teuthology.orchestra.run.smithi040.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.0.0-2698-gb820560f-1focal [1,548 kB] 2023-11-08T08:14:22.315 INFO:teuthology.orchestra.run.smithi169.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-11-08T08:14:22.343 INFO:teuthology.orchestra.run.smithi040.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.0.0-2698-gb820560f-1focal [9,376 kB] 2023-11-08T08:14:22.373 INFO:teuthology.orchestra.run.smithi169.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-11-08T08:14:22.375 INFO:teuthology.orchestra.run.smithi169.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-11-08T08:14:22.605 INFO:teuthology.orchestra.run.smithi169.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-11-08T08:14:22.647 INFO:teuthology.orchestra.run.smithi040.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.0.0-2698-gb820560f-1focal [31.3 MB] 2023-11-08T08:14:22.690 INFO:teuthology.orchestra.run.smithi169.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-11-08T08:14:22.814 INFO:teuthology.orchestra.run.smithi169.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-11-08T08:14:22.877 INFO:teuthology.orchestra.run.smithi169.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-11-08T08:14:22.902 INFO:teuthology.orchestra.run.smithi169.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-11-08T08:14:22.941 INFO:teuthology.orchestra.run.smithi169.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-11-08T08:14:22.954 INFO:teuthology.orchestra.run.smithi169.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-11-08T08:14:23.005 INFO:teuthology.orchestra.run.smithi169.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-11-08T08:14:23.014 INFO:teuthology.orchestra.run.smithi169.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-11-08T08:14:23.027 INFO:teuthology.orchestra.run.smithi169.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-11-08T08:14:23.107 INFO:teuthology.orchestra.run.smithi169.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-11-08T08:14:23.297 INFO:teuthology.orchestra.run.smithi169.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-11-08T08:14:23.670 INFO:teuthology.orchestra.run.smithi169.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-11-08T08:14:23.698 INFO:teuthology.orchestra.run.smithi169.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-11-08T08:14:23.850 INFO:teuthology.orchestra.run.smithi169.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-11-08T08:14:23.863 INFO:teuthology.orchestra.run.smithi040.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.0.0-2698-gb820560f-1focal [3,816 B] 2023-11-08T08:14:23.863 INFO:teuthology.orchestra.run.smithi040.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.0.0-2698-gb820560f-1focal [830 kB] 2023-11-08T08:14:23.879 INFO:teuthology.orchestra.run.smithi040.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.0.0-2698-gb820560f-1focal [2,285 kB] 2023-11-08T08:14:23.908 INFO:teuthology.orchestra.run.smithi169.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-11-08T08:14:23.912 INFO:teuthology.orchestra.run.smithi169.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-11-08T08:14:23.967 INFO:teuthology.orchestra.run.smithi040.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.0.0-2698-gb820560f-1focal [96.8 kB] 2023-11-08T08:14:23.969 INFO:teuthology.orchestra.run.smithi040.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.0.0-2698-gb820560f-1focal [117 kB] 2023-11-08T08:14:23.972 INFO:teuthology.orchestra.run.smithi040.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.0.0-2698-gb820560f-1focal [1,795 kB] 2023-11-08T08:14:24.044 INFO:teuthology.orchestra.run.smithi040.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.0.0-2698-gb820560f-1focal [7,890 kB] 2023-11-08T08:14:24.079 INFO:teuthology.orchestra.run.smithi169.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-11-08T08:14:24.087 INFO:teuthology.orchestra.run.smithi169.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2023-11-08T08:14:24.097 INFO:teuthology.orchestra.run.smithi169.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-11-08T08:14:24.260 INFO:teuthology.orchestra.run.smithi169.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-11-08T08:14:24.261 INFO:teuthology.orchestra.run.smithi169.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-11-08T08:14:24.348 INFO:teuthology.orchestra.run.smithi040.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.0.0-2698-gb820560f-1focal [3,936 B] 2023-11-08T08:14:24.349 INFO:teuthology.orchestra.run.smithi040.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.0.0-2698-gb820560f-1focal [49.9 MB] 2023-11-08T08:14:24.426 INFO:teuthology.orchestra.run.smithi169.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-11-08T08:14:24.762 INFO:teuthology.orchestra.run.smithi169.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-11-08T08:14:25.128 INFO:teuthology.orchestra.run.smithi169.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-11-08T08:14:25.131 INFO:teuthology.orchestra.run.smithi169.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-11-08T08:14:25.143 INFO:teuthology.orchestra.run.smithi169.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-11-08T08:14:25.145 INFO:teuthology.orchestra.run.smithi169.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-11-08T08:14:25.149 INFO:teuthology.orchestra.run.smithi169.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-11-08T08:14:25.964 INFO:teuthology.orchestra.run.smithi169.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.0.0-2698-gb820560f-1focal [10.9 MB] 2023-11-08T08:14:26.255 INFO:teuthology.orchestra.run.smithi040.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.0.0-2698-gb820560f-1focal [225 kB] 2023-11-08T08:14:26.257 INFO:teuthology.orchestra.run.smithi040.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.0.0-2698-gb820560f-1focal [253 kB] 2023-11-08T08:14:26.259 INFO:teuthology.orchestra.run.smithi040.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.0.0-2698-gb820560f-1focal [23.3 kB] 2023-11-08T08:14:26.260 INFO:teuthology.orchestra.run.smithi040.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.0.0-2698-gb820560f-1focal [13.0 kB] 2023-11-08T08:14:26.260 INFO:teuthology.orchestra.run.smithi040.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.0.0-2698-gb820560f-1focal [27.4 kB] 2023-11-08T08:14:26.261 INFO:teuthology.orchestra.run.smithi040.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.0.0-2698-gb820560f-1focal [13.3 MB] 2023-11-08T08:14:26.757 INFO:teuthology.orchestra.run.smithi040.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.0.0-2698-gb820560f-1focal [97.3 kB] 2023-11-08T08:14:27.531 INFO:teuthology.orchestra.run.smithi169.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.0.0-2698-gb820560f-1focal [221 kB] 2023-11-08T08:14:27.537 INFO:teuthology.orchestra.run.smithi169.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.0.0-2698-gb820560f-1focal [133 kB] 2023-11-08T08:14:27.563 INFO:teuthology.orchestra.run.smithi169.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.0.0-2698-gb820560f-1focal [1,548 kB] 2023-11-08T08:14:27.714 INFO:teuthology.orchestra.run.smithi169.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.0.0-2698-gb820560f-1focal [9,376 kB] 2023-11-08T08:14:27.726 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 207 MB in 9s (23.7 MB/s) 2023-11-08T08:14:27.907 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pyperclip. 2023-11-08T08:14:27.980 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 107758 files and directories currently installed.) 2023-11-08T08:14:27.983 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-11-08T08:14:28.028 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-11-08T08:14:28.339 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-wcwidth. 2023-11-08T08:14:28.360 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-11-08T08:14:28.389 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-11-08T08:14:28.608 INFO:teuthology.orchestra.run.smithi169.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.0.0-2698-gb820560f-1focal [31.3 MB] 2023-11-08T08:14:28.751 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cmd2. 2023-11-08T08:14:28.770 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-11-08T08:14:28.809 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-11-08T08:14:29.229 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-11-08T08:14:29.250 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-11-08T08:14:29.354 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-11-08T08:14:29.767 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-11-08T08:14:29.784 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-11-08T08:14:29.816 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-11-08T08:14:30.228 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-11-08T08:14:30.249 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-11-08T08:14:30.294 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-11-08T08:14:30.823 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-11-08T08:14:30.836 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-11-08T08:14:30.895 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-11-08T08:14:31.248 INFO:teuthology.orchestra.run.smithi169.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.0.0-2698-gb820560f-1focal [3,816 B] 2023-11-08T08:14:31.249 INFO:teuthology.orchestra.run.smithi169.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.0.0-2698-gb820560f-1focal [830 kB] 2023-11-08T08:14:31.292 INFO:teuthology.orchestra.run.smithi169.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.0.0-2698-gb820560f-1focal [2,285 kB] 2023-11-08T08:14:31.385 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-11-08T08:14:31.407 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-11-08T08:14:31.451 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-11-08T08:14:31.467 INFO:teuthology.orchestra.run.smithi169.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.0.0-2698-gb820560f-1focal [96.8 kB] 2023-11-08T08:14:31.470 INFO:teuthology.orchestra.run.smithi169.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.0.0-2698-gb820560f-1focal [117 kB] 2023-11-08T08:14:31.472 INFO:teuthology.orchestra.run.smithi169.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.0.0-2698-gb820560f-1focal [1,795 kB] 2023-11-08T08:14:31.606 INFO:teuthology.orchestra.run.smithi169.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.0.0-2698-gb820560f-1focal [7,890 kB] 2023-11-08T08:14:31.888 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-11-08T08:14:31.909 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-11-08T08:14:31.956 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-11-08T08:14:32.116 INFO:teuthology.orchestra.run.smithi169.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.0.0-2698-gb820560f-1focal [3,936 B] 2023-11-08T08:14:32.116 INFO:teuthology.orchestra.run.smithi169.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.0.0-2698-gb820560f-1focal [49.9 MB] 2023-11-08T08:14:32.392 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-11-08T08:14:32.410 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-11-08T08:14:32.450 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-11-08T08:14:33.039 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../010-librbd1_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:33.196 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librbd1 (18.0.0-2698-gb820560f-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-11-08T08:14:33.971 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../011-librados2_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:34.084 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librados2 (18.0.0-2698-gb820560f-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-11-08T08:14:34.847 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libcephfs2. 2023-11-08T08:14:34.868 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../012-libcephfs2_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:34.904 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libcephfs2 (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:35.187 INFO:teuthology.orchestra.run.smithi169.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.0.0-2698-gb820560f-1focal [225 kB] 2023-11-08T08:14:35.189 INFO:teuthology.orchestra.run.smithi169.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.0.0-2698-gb820560f-1focal [253 kB] 2023-11-08T08:14:35.192 INFO:teuthology.orchestra.run.smithi169.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.0.0-2698-gb820560f-1focal [23.3 kB] 2023-11-08T08:14:35.193 INFO:teuthology.orchestra.run.smithi169.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.0.0-2698-gb820560f-1focal [13.0 kB] 2023-11-08T08:14:35.193 INFO:teuthology.orchestra.run.smithi169.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.0.0-2698-gb820560f-1focal [27.4 kB] 2023-11-08T08:14:35.194 INFO:teuthology.orchestra.run.smithi169.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.0.0-2698-gb820560f-1focal [13.3 MB] 2023-11-08T08:14:35.274 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-rados. 2023-11-08T08:14:35.294 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../013-python3-rados_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:35.332 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-rados (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:35.705 INFO:teuthology.orchestra.run.smithi169.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-54557/b820560f30d310ab2570761b9a294523225d6998/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.0.0-2698-gb820560f-1focal [97.3 kB] 2023-11-08T08:14:35.751 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-11-08T08:14:35.774 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.0.0-2698-gb820560f-1focal_all.deb ... 2023-11-08T08:14:35.818 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-ceph-argparse (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:36.146 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cephfs. 2023-11-08T08:14:36.164 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../015-python3-cephfs_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:36.204 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cephfs (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:36.541 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-ceph-common. 2023-11-08T08:14:36.561 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../016-python3-ceph-common_18.0.0-2698-gb820560f-1focal_all.deb ... 2023-11-08T08:14:36.599 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-ceph-common (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:36.666 INFO:teuthology.orchestra.run.smithi169.stdout:Fetched 207 MB in 18s (11.6 MB/s) 2023-11-08T08:14:36.853 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pyperclip. 2023-11-08T08:14:36.925 INFO:teuthology.orchestra.run.smithi169.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 107758 files and directories currently installed.) 2023-11-08T08:14:36.928 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-11-08T08:14:36.960 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-prettytable. 2023-11-08T08:14:36.974 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-11-08T08:14:36.978 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2023-11-08T08:14:37.018 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-11-08T08:14:37.310 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-wcwidth. 2023-11-08T08:14:37.330 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-11-08T08:14:37.338 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-rbd. 2023-11-08T08:14:37.359 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../018-python3-rbd_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:37.360 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-11-08T08:14:37.404 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-rbd (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:37.746 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-cmd2. 2023-11-08T08:14:37.766 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-11-08T08:14:37.774 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-11-08T08:14:37.794 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-11-08T08:14:37.831 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-11-08T08:14:37.832 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-11-08T08:14:38.202 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-11-08T08:14:38.223 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2023-11-08T08:14:38.235 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-11-08T08:14:38.255 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-11-08T08:14:38.260 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-11-08T08:14:38.335 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-11-08T08:14:38.663 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-11-08T08:14:38.682 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-11-08T08:14:38.721 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-11-08T08:14:38.730 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-11-08T08:14:38.751 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-11-08T08:14:38.788 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-11-08T08:14:39.141 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package librgw2. 2023-11-08T08:14:39.159 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../022-librgw2_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:39.199 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librgw2 (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:39.200 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-11-08T08:14:39.219 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-11-08T08:14:39.258 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-11-08T08:14:39.695 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-11-08T08:14:39.715 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-11-08T08:14:39.805 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-11-08T08:14:40.127 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-rgw. 2023-11-08T08:14:40.144 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../023-python3-rgw_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:40.185 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-rgw (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:40.373 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-11-08T08:14:40.393 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-11-08T08:14:40.431 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-11-08T08:14:40.588 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package liboath0. 2023-11-08T08:14:40.608 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-11-08T08:14:40.646 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-11-08T08:14:40.851 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-11-08T08:14:40.868 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-11-08T08:14:40.934 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-11-08T08:14:41.016 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libradosstriper1. 2023-11-08T08:14:41.034 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../025-libradosstriper1_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:41.074 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libradosstriper1 (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:41.372 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-11-08T08:14:41.395 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-11-08T08:14:41.438 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-11-08T08:14:41.561 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-common. 2023-11-08T08:14:41.580 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../026-ceph-common_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:41.619 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-common (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:41.859 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../010-librbd1_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:41.975 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking librbd1 (18.0.0-2698-gb820560f-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-11-08T08:14:42.758 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../011-librados2_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:42.913 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking librados2 (18.0.0-2698-gb820560f-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-11-08T08:14:43.684 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libcephfs2. 2023-11-08T08:14:43.707 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../012-libcephfs2_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:43.742 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libcephfs2 (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:44.103 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-rados. 2023-11-08T08:14:44.119 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../013-python3-rados_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:44.161 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-rados (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:44.216 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-base. 2023-11-08T08:14:44.237 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../027-ceph-base_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:44.304 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-base (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:44.555 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-11-08T08:14:44.572 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.0.0-2698-gb820560f-1focal_all.deb ... 2023-11-08T08:14:44.613 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-ceph-argparse (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:44.916 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-cephfs. 2023-11-08T08:14:44.939 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../015-python3-cephfs_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:44.986 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-cephfs (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:45.353 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-ceph-common. 2023-11-08T08:14:45.373 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../016-python3-ceph-common_18.0.0-2698-gb820560f-1focal_all.deb ... 2023-11-08T08:14:45.411 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-ceph-common (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:45.789 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-prettytable. 2023-11-08T08:14:45.798 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cherrypy3. 2023-11-08T08:14:45.809 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2023-11-08T08:14:45.820 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2023-11-08T08:14:45.847 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-11-08T08:14:45.864 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-11-08T08:14:46.150 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-rbd. 2023-11-08T08:14:46.171 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../018-python3-rbd_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:46.216 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-rbd (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:46.276 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-natsort. 2023-11-08T08:14:46.299 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2023-11-08T08:14:46.342 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-11-08T08:14:46.594 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-11-08T08:14:46.614 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-11-08T08:14:46.694 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-11-08T08:14:46.704 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-logutils. 2023-11-08T08:14:46.725 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2023-11-08T08:14:46.770 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-11-08T08:14:47.089 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-11-08T08:14:47.109 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2023-11-08T08:14:47.146 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-11-08T08:14:47.157 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-markupsafe. 2023-11-08T08:14:47.175 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-11-08T08:14:47.215 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-11-08T08:14:47.533 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-11-08T08:14:47.551 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-mako. 2023-11-08T08:14:47.553 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-11-08T08:14:47.570 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-11-08T08:14:47.591 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-11-08T08:14:47.601 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-11-08T08:14:47.921 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-simplegeneric. 2023-11-08T08:14:47.941 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2023-11-08T08:14:47.979 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-11-08T08:14:48.653 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package librgw2. 2023-11-08T08:14:48.674 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../022-librgw2_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:48.711 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking librgw2 (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:49.008 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-singledispatch. 2023-11-08T08:14:49.030 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-11-08T08:14:49.091 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-11-08T08:14:49.386 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-webob. 2023-11-08T08:14:49.407 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2_all.deb ... 2023-11-08T08:14:49.477 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-11-08T08:14:49.647 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-rgw. 2023-11-08T08:14:49.668 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../023-python3-rgw_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:49.705 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-rgw (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:49.788 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-soupsieve. 2023-11-08T08:14:49.806 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-11-08T08:14:49.838 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-11-08T08:14:50.148 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package liboath0. 2023-11-08T08:14:50.166 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-11-08T08:14:50.200 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-bs4. 2023-11-08T08:14:50.216 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-11-08T08:14:50.220 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2023-11-08T08:14:50.257 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-11-08T08:14:50.569 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libradosstriper1. 2023-11-08T08:14:50.588 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../025-libradosstriper1_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:50.627 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libradosstriper1 (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:50.645 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-waitress. 2023-11-08T08:14:50.662 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-11-08T08:14:50.726 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-11-08T08:14:51.089 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-tempita. 2023-11-08T08:14:51.110 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2023-11-08T08:14:51.122 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-common. 2023-11-08T08:14:51.143 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../026-ceph-common_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:51.163 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-11-08T08:14:51.188 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-common (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:51.500 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-paste. 2023-11-08T08:14:51.523 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-11-08T08:14:51.591 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-11-08T08:14:51.995 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-11-08T08:14:52.012 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-11-08T08:14:52.052 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-11-08T08:14:52.406 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pastedeploy. 2023-11-08T08:14:52.424 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2023-11-08T08:14:52.472 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-11-08T08:14:52.833 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-webtest. 2023-11-08T08:14:52.848 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2023-11-08T08:14:52.891 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-11-08T08:14:53.270 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pecan. 2023-11-08T08:14:53.287 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-11-08T08:14:53.327 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-11-08T08:14:53.741 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-werkzeug. 2023-11-08T08:14:53.760 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-11-08T08:14:53.769 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-base. 2023-11-08T08:14:53.791 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../027-ceph-base_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:53.801 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-11-08T08:14:53.857 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-base (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:54.152 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-11-08T08:14:54.173 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.0.0-2698-gb820560f-1focal_all.deb ... 2023-11-08T08:14:54.218 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-modules-core (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:54.657 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-11-08T08:14:54.675 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:54.715 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:55.110 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cryptography. 2023-11-08T08:14:55.127 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../048-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-11-08T08:14:55.168 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-11-08T08:14:55.357 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-cherrypy3. 2023-11-08T08:14:55.379 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2023-11-08T08:14:55.423 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-11-08T08:14:55.588 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-openssl. 2023-11-08T08:14:55.609 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../049-python3-openssl_19.0.0-1build1_all.deb ... 2023-11-08T08:14:55.646 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-11-08T08:14:55.810 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-natsort. 2023-11-08T08:14:55.830 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2023-11-08T08:14:55.860 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-11-08T08:14:56.058 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr. 2023-11-08T08:14:56.079 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../050-ceph-mgr_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:56.157 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:56.230 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-logutils. 2023-11-08T08:14:56.248 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2023-11-08T08:14:56.288 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-11-08T08:14:56.649 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-markupsafe. 2023-11-08T08:14:56.670 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-11-08T08:14:56.694 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mon. 2023-11-08T08:14:56.707 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-11-08T08:14:56.712 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../051-ceph-mon_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:56.752 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mon (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:57.060 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-mako. 2023-11-08T08:14:57.080 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-11-08T08:14:57.143 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-11-08T08:14:57.547 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-simplegeneric. 2023-11-08T08:14:57.570 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2023-11-08T08:14:57.613 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-11-08T08:14:57.975 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-singledispatch. 2023-11-08T08:14:57.996 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-11-08T08:14:58.031 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-osd. 2023-11-08T08:14:58.041 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-11-08T08:14:58.048 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../052-ceph-osd_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:14:58.088 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-osd (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:14:58.386 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-webob. 2023-11-08T08:14:58.408 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2_all.deb ... 2023-11-08T08:14:58.452 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-11-08T08:14:58.847 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-soupsieve. 2023-11-08T08:14:58.866 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-11-08T08:14:58.904 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-11-08T08:14:59.283 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-bs4. 2023-11-08T08:14:59.302 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2023-11-08T08:14:59.342 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-11-08T08:14:59.744 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-waitress. 2023-11-08T08:14:59.763 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-11-08T08:14:59.809 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-11-08T08:15:00.155 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-tempita. 2023-11-08T08:15:00.176 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2023-11-08T08:15:00.238 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-11-08T08:15:00.575 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-paste. 2023-11-08T08:15:00.595 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-11-08T08:15:00.649 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-11-08T08:15:00.934 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph. 2023-11-08T08:15:00.954 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../053-ceph_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:00.992 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:01.053 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-11-08T08:15:01.070 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-11-08T08:15:01.110 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-11-08T08:15:01.378 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-fuse. 2023-11-08T08:15:01.399 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../054-ceph-fuse_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:01.436 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-fuse (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:01.472 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pastedeploy. 2023-11-08T08:15:01.492 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2023-11-08T08:15:01.530 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-11-08T08:15:01.875 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-webtest. 2023-11-08T08:15:01.895 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2023-11-08T08:15:01.923 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mds. 2023-11-08T08:15:01.933 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-11-08T08:15:01.944 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../055-ceph-mds_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:01.982 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mds (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:02.303 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pecan. 2023-11-08T08:15:02.320 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-11-08T08:15:02.361 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-11-08T08:15:02.542 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package cephadm. 2023-11-08T08:15:02.563 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../056-cephadm_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:02.600 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking cephadm (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:02.817 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-werkzeug. 2023-11-08T08:15:02.838 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-11-08T08:15:02.882 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-11-08T08:15:02.978 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-asyncssh. 2023-11-08T08:15:02.996 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../057-python3-asyncssh_1.12.2-1_all.deb ... 2023-11-08T08:15:03.036 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2023-11-08T08:15:03.219 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-11-08T08:15:03.241 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.0.0-2698-gb820560f-1focal_all.deb ... 2023-11-08T08:15:03.318 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mgr-modules-core (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:03.440 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-jinja2. 2023-11-08T08:15:03.458 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../058-python3-jinja2_2.10.1-2_all.deb ... 2023-11-08T08:15:03.498 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-11-08T08:15:03.892 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-11-08T08:15:03.913 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../059-ceph-mgr-cephadm_18.0.0-2698-gb820560f-1focal_all.deb ... 2023-11-08T08:15:03.960 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-cephadm (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:04.057 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-11-08T08:15:04.074 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:04.114 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:04.493 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-cryptography. 2023-11-08T08:15:04.512 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../048-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-11-08T08:15:04.550 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-11-08T08:15:04.554 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-bcrypt. 2023-11-08T08:15:04.575 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../060-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-11-08T08:15:04.611 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-11-08T08:15:05.013 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-openssl. 2023-11-08T08:15:05.031 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../049-python3-openssl_19.0.0-1build1_all.deb ... 2023-11-08T08:15:05.040 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-jwt. 2023-11-08T08:15:05.061 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-11-08T08:15:05.070 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-11-08T08:15:05.131 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-11-08T08:15:05.458 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mgr. 2023-11-08T08:15:05.476 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-attr. 2023-11-08T08:15:05.478 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../050-ceph-mgr_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:05.495 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../062-python3-attr_19.3.0-2_all.deb ... 2023-11-08T08:15:05.507 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mgr (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:05.559 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-11-08T08:15:05.912 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pluggy. 2023-11-08T08:15:05.933 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../063-python3-pluggy_0.13.0-2_all.deb ... 2023-11-08T08:15:05.987 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-11-08T08:15:06.086 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mon. 2023-11-08T08:15:06.106 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../051-ceph-mon_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:06.177 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mon (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:06.382 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-py. 2023-11-08T08:15:06.398 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../064-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-11-08T08:15:06.439 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-11-08T08:15:06.810 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-atomicwrites. 2023-11-08T08:15:06.828 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../065-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-11-08T08:15:06.868 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-11-08T08:15:07.220 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-packaging. 2023-11-08T08:15:07.242 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../066-python3-packaging_20.3-1_all.deb ... 2023-11-08T08:15:07.278 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-packaging (20.3-1) ... 2023-11-08T08:15:07.514 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-osd. 2023-11-08T08:15:07.538 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../052-ceph-osd_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:07.580 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-osd (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:07.640 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pytest. 2023-11-08T08:15:07.661 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../067-python3-pytest_4.6.9-1_all.deb ... 2023-11-08T08:15:07.706 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-11-08T08:15:08.110 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-repoze.lru. 2023-11-08T08:15:08.128 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../068-python3-repoze.lru_0.7-2_all.deb ... 2023-11-08T08:15:08.168 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-11-08T08:15:08.446 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libjs-underscore. 2023-11-08T08:15:08.465 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../069-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-11-08T08:15:08.503 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-11-08T08:15:08.815 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-11-08T08:15:08.835 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../070-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-11-08T08:15:08.889 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-11-08T08:15:09.226 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-routes. 2023-11-08T08:15:09.246 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../071-python3-routes_2.4.1-2_all.deb ... 2023-11-08T08:15:09.317 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-11-08T08:15:09.762 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-11-08T08:15:09.782 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.0.0-2698-gb820560f-1focal_all.deb ... 2023-11-08T08:15:09.819 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-dashboard (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:10.300 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph. 2023-11-08T08:15:10.320 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../053-ceph_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:10.349 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:10.391 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-decorator. 2023-11-08T08:15:10.410 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../073-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-11-08T08:15:10.448 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-11-08T08:15:10.711 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-fuse. 2023-11-08T08:15:10.732 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../054-ceph-fuse_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:10.777 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-fuse (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:10.981 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-11-08T08:15:10.998 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../074-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-11-08T08:15:11.050 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-11-08T08:15:11.263 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mds. 2023-11-08T08:15:11.283 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../055-ceph-mds_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:11.321 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mds (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:11.420 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-scipy. 2023-11-08T08:15:11.440 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../075-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-11-08T08:15:11.478 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-11-08T08:15:11.925 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package cephadm. 2023-11-08T08:15:11.945 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../056-cephadm_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:11.982 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking cephadm (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:12.345 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-asyncssh. 2023-11-08T08:15:12.367 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../057-python3-asyncssh_1.12.2-1_all.deb ... 2023-11-08T08:15:12.402 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2023-11-08T08:15:12.813 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-jinja2. 2023-11-08T08:15:12.831 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../058-python3-jinja2_2.10.1-2_all.deb ... 2023-11-08T08:15:12.903 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-11-08T08:15:12.977 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-11-08T08:15:12.998 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../076-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-11-08T08:15:13.068 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-11-08T08:15:13.299 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-11-08T08:15:13.314 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../059-ceph-mgr-cephadm_18.0.0-2698-gb820560f-1focal_all.deb ... 2023-11-08T08:15:13.357 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mgr-cephadm (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:13.664 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-joblib. 2023-11-08T08:15:13.681 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../077-python3-joblib_0.14.0-3_all.deb ... 2023-11-08T08:15:13.713 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-11-08T08:15:13.753 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-bcrypt. 2023-11-08T08:15:13.770 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../060-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-11-08T08:15:13.810 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-11-08T08:15:14.159 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-sklearn. 2023-11-08T08:15:14.171 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-jwt. 2023-11-08T08:15:14.180 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../078-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-11-08T08:15:14.192 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-11-08T08:15:14.216 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-11-08T08:15:14.238 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-11-08T08:15:14.616 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-attr. 2023-11-08T08:15:14.637 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../062-python3-attr_19.3.0-2_all.deb ... 2023-11-08T08:15:14.682 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-11-08T08:15:15.051 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-11-08T08:15:15.072 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../079-ceph-mgr-diskprediction-local_18.0.0-2698-gb820560f-1focal_all.deb ... 2023-11-08T08:15:15.102 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pluggy. 2023-11-08T08:15:15.120 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../063-python3-pluggy_0.13.0-2_all.deb ... 2023-11-08T08:15:15.150 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:15.159 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-11-08T08:15:15.763 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-py. 2023-11-08T08:15:15.783 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../064-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-11-08T08:15:15.821 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-11-08T08:15:16.225 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-atomicwrites. 2023-11-08T08:15:16.246 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../065-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-11-08T08:15:16.291 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-11-08T08:15:16.437 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cachetools. 2023-11-08T08:15:16.458 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../080-python3-cachetools_4.0.0-1_all.deb ... 2023-11-08T08:15:16.514 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-11-08T08:15:16.644 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-packaging. 2023-11-08T08:15:16.665 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../066-python3-packaging_20.3-1_all.deb ... 2023-11-08T08:15:16.742 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-packaging (20.3-1) ... 2023-11-08T08:15:16.971 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pyasn1. 2023-11-08T08:15:16.992 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../081-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-11-08T08:15:17.028 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-11-08T08:15:17.121 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pytest. 2023-11-08T08:15:17.134 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../067-python3-pytest_4.6.9-1_all.deb ... 2023-11-08T08:15:17.179 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-11-08T08:15:17.415 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-11-08T08:15:17.435 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../082-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-11-08T08:15:17.473 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-11-08T08:15:17.582 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-repoze.lru. 2023-11-08T08:15:17.595 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../068-python3-repoze.lru_0.7-2_all.deb ... 2023-11-08T08:15:17.641 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-11-08T08:15:17.827 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-rsa. 2023-11-08T08:15:17.845 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../083-python3-rsa_4.0-3_all.deb ... 2023-11-08T08:15:17.884 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-rsa (4.0-3) ... 2023-11-08T08:15:17.944 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libjs-underscore. 2023-11-08T08:15:17.963 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../069-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-11-08T08:15:18.001 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-11-08T08:15:18.263 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-google-auth. 2023-11-08T08:15:18.284 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../084-python3-google-auth_1.5.1-2_all.deb ... 2023-11-08T08:15:18.321 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-11-08T08:15:18.329 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-11-08T08:15:18.349 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../070-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-11-08T08:15:18.386 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-11-08T08:15:18.709 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-blinker. 2023-11-08T08:15:18.728 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../085-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-11-08T08:15:18.767 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-11-08T08:15:18.772 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-routes. 2023-11-08T08:15:18.786 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../071-python3-routes_2.4.1-2_all.deb ... 2023-11-08T08:15:18.831 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-11-08T08:15:19.137 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-oauthlib. 2023-11-08T08:15:19.159 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../086-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-11-08T08:15:19.201 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-11-08T08:15:19.202 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-11-08T08:15:19.220 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.0.0-2698-gb820560f-1focal_all.deb ... 2023-11-08T08:15:19.259 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mgr-dashboard (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:19.615 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-11-08T08:15:19.636 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-11-08T08:15:19.681 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-11-08T08:15:19.846 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-decorator. 2023-11-08T08:15:19.864 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../073-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-11-08T08:15:19.903 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-11-08T08:15:20.060 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-websocket. 2023-11-08T08:15:20.080 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../088-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-11-08T08:15:20.126 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-11-08T08:15:20.273 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-11-08T08:15:20.294 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../074-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-11-08T08:15:20.339 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-11-08T08:15:20.504 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-kubernetes. 2023-11-08T08:15:20.517 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../089-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-11-08T08:15:20.561 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-11-08T08:15:20.676 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-scipy. 2023-11-08T08:15:20.693 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../075-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-11-08T08:15:20.733 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-11-08T08:15:21.301 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-11-08T08:15:21.324 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.0.0-2698-gb820560f-1focal_all.deb ... 2023-11-08T08:15:21.366 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:21.761 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libonig5:amd64. 2023-11-08T08:15:21.780 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../091-libonig5_6.9.4-1_amd64.deb ... 2023-11-08T08:15:21.819 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-11-08T08:15:22.157 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-11-08T08:15:22.179 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../076-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-11-08T08:15:22.222 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-11-08T08:15:22.223 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libjq1:amd64. 2023-11-08T08:15:22.245 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../092-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-11-08T08:15:22.314 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-11-08T08:15:22.626 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package jq. 2023-11-08T08:15:22.647 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../093-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-11-08T08:15:22.683 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-11-08T08:15:22.828 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-joblib. 2023-11-08T08:15:22.851 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../077-python3-joblib_0.14.0-3_all.deb ... 2023-11-08T08:15:22.920 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-11-08T08:15:22.986 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package socat. 2023-11-08T08:15:23.007 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../094-socat_1.7.3.3-2_amd64.deb ... 2023-11-08T08:15:23.052 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking socat (1.7.3.3-2) ... 2023-11-08T08:15:23.357 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-sklearn. 2023-11-08T08:15:23.376 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../078-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-11-08T08:15:23.414 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-11-08T08:15:23.423 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package xmlstarlet. 2023-11-08T08:15:23.443 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-11-08T08:15:23.480 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-11-08T08:15:23.931 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-test. 2023-11-08T08:15:23.947 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../096-ceph-test_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:24.005 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-test (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:24.215 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-11-08T08:15:24.233 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../079-ceph-mgr-diskprediction-local_18.0.0-2698-gb820560f-1focal_all.deb ... 2023-11-08T08:15:24.289 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:25.509 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-cachetools. 2023-11-08T08:15:25.528 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../080-python3-cachetools_4.0.0-1_all.deb ... 2023-11-08T08:15:25.567 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-11-08T08:15:26.051 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pyasn1. 2023-11-08T08:15:26.070 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../081-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-11-08T08:15:26.110 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-11-08T08:15:26.522 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-11-08T08:15:26.540 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../082-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-11-08T08:15:26.578 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-11-08T08:15:26.957 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-rsa. 2023-11-08T08:15:26.977 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../083-python3-rsa_4.0-3_all.deb ... 2023-11-08T08:15:27.015 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-rsa (4.0-3) ... 2023-11-08T08:15:27.385 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-google-auth. 2023-11-08T08:15:27.405 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../084-python3-google-auth_1.5.1-2_all.deb ... 2023-11-08T08:15:27.443 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-11-08T08:15:27.822 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-blinker. 2023-11-08T08:15:27.842 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../085-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-11-08T08:15:27.904 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-11-08T08:15:28.242 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-oauthlib. 2023-11-08T08:15:28.262 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../086-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-11-08T08:15:28.267 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-volume. 2023-11-08T08:15:28.285 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../097-ceph-volume_18.0.0-2698-gb820560f-1focal_all.deb ... 2023-11-08T08:15:28.324 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-11-08T08:15:28.325 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-volume (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:28.704 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-11-08T08:15:28.721 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-11-08T08:15:28.755 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package cephfs-mirror. 2023-11-08T08:15:28.761 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-11-08T08:15:28.778 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../098-cephfs-mirror_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:28.821 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking cephfs-mirror (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:29.148 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-websocket. 2023-11-08T08:15:29.167 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../088-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-11-08T08:15:29.206 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-11-08T08:15:29.224 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-colorama. 2023-11-08T08:15:29.247 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../099-python3-colorama_0.4.3-1build1_all.deb ... 2023-11-08T08:15:29.290 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-11-08T08:15:29.584 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-kubernetes. 2023-11-08T08:15:29.607 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../089-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-11-08T08:15:29.627 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package cephfs-shell. 2023-11-08T08:15:29.648 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../100-cephfs-shell_18.0.0-2698-gb820560f-1focal_all.deb ... 2023-11-08T08:15:29.650 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-11-08T08:15:29.685 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking cephfs-shell (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:30.021 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package cephfs-top. 2023-11-08T08:15:30.044 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../101-cephfs-top_18.0.0-2698-gb820560f-1focal_all.deb ... 2023-11-08T08:15:30.087 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking cephfs-top (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:30.289 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-11-08T08:15:30.310 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.0.0-2698-gb820560f-1focal_all.deb ... 2023-11-08T08:15:30.355 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:30.382 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package fonts-mathjax. 2023-11-08T08:15:30.403 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../102-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-11-08T08:15:30.448 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-11-08T08:15:30.742 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libonig5:amd64. 2023-11-08T08:15:30.762 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../091-libonig5_6.9.4-1_amd64.deb ... 2023-11-08T08:15:30.807 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-11-08T08:15:30.977 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libcephfs-dev. 2023-11-08T08:15:31.000 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../103-libcephfs-dev_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:31.081 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libcephfs-dev (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:31.228 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libjq1:amd64. 2023-11-08T08:15:31.245 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../092-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-11-08T08:15:31.285 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-11-08T08:15:31.614 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package jq. 2023-11-08T08:15:31.630 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package nvme-cli. 2023-11-08T08:15:31.636 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../093-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-11-08T08:15:31.652 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../104-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-11-08T08:15:31.680 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-11-08T08:15:31.729 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-11-08T08:15:31.983 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package socat. 2023-11-08T08:15:32.003 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../094-socat_1.7.3.3-2_amd64.deb ... 2023-11-08T08:15:32.049 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking socat (1.7.3.3-2) ... 2023-11-08T08:15:32.118 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-11-08T08:15:32.141 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../105-python-asyncssh-doc_1.12.2-1_all.deb ... 2023-11-08T08:15:32.184 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2023-11-08T08:15:32.411 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package xmlstarlet. 2023-11-08T08:15:32.433 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-11-08T08:15:32.468 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-11-08T08:15:32.612 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libjs-mathjax. 2023-11-08T08:15:32.633 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../106-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-11-08T08:15:32.670 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-11-08T08:15:32.884 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-test. 2023-11-08T08:15:32.901 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../096-ceph-test_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:32.965 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-test (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:35.110 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python-natsort-doc. 2023-11-08T08:15:35.132 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../107-python-natsort-doc_7.0.1-1_all.deb ... 2023-11-08T08:15:35.176 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-11-08T08:15:35.572 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-webencodings. 2023-11-08T08:15:35.593 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../108-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-11-08T08:15:35.638 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-11-08T08:15:36.008 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-html5lib. 2023-11-08T08:15:36.029 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../109-python3-html5lib_1.0.1-2_all.deb ... 2023-11-08T08:15:36.073 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-11-08T08:15:36.477 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-11-08T08:15:36.499 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../110-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-11-08T08:15:36.543 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-11-08T08:15:37.080 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pastescript. 2023-11-08T08:15:37.100 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../111-python3-pastescript_2.0.2-4_all.deb ... 2023-11-08T08:15:37.138 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-11-08T08:15:37.152 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ceph-volume. 2023-11-08T08:15:37.166 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../097-ceph-volume_18.0.0-2698-gb820560f-1focal_all.deb ... 2023-11-08T08:15:37.202 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ceph-volume (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:37.534 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-psutil. 2023-11-08T08:15:37.556 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../112-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-11-08T08:15:37.599 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package cephfs-mirror. 2023-11-08T08:15:37.619 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../098-cephfs-mirror_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:37.625 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-11-08T08:15:37.673 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking cephfs-mirror (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:38.039 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pyinotify. 2023-11-08T08:15:38.060 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../113-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-11-08T08:15:38.085 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-colorama. 2023-11-08T08:15:38.105 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-11-08T08:15:38.107 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../099-python3-colorama_0.4.3-1build1_all.deb ... 2023-11-08T08:15:38.151 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-11-08T08:15:38.462 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package cephfs-shell. 2023-11-08T08:15:38.484 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-simplejson. 2023-11-08T08:15:38.485 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../100-cephfs-shell_18.0.0-2698-gb820560f-1focal_all.deb ... 2023-11-08T08:15:38.505 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-11-08T08:15:38.520 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking cephfs-shell (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:38.549 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-11-08T08:15:38.831 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package cephfs-top. 2023-11-08T08:15:38.852 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../101-cephfs-top_18.0.0-2698-gb820560f-1focal_all.deb ... 2023-11-08T08:15:38.889 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking cephfs-top (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:38.911 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-11-08T08:15:38.932 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-11-08T08:15:38.977 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-11-08T08:15:39.208 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package fonts-mathjax. 2023-11-08T08:15:39.232 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../102-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-11-08T08:15:39.274 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-11-08T08:15:39.760 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package radosgw. 2023-11-08T08:15:39.779 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../116-radosgw_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:39.817 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking radosgw (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:39.829 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libcephfs-dev. 2023-11-08T08:15:39.850 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../103-libcephfs-dev_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:39.895 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libcephfs-dev (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:40.348 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package nvme-cli. 2023-11-08T08:15:40.367 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../104-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-11-08T08:15:40.470 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-11-08T08:15:40.811 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-11-08T08:15:40.832 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../105-python-asyncssh-doc_1.12.2-1_all.deb ... 2023-11-08T08:15:40.877 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2023-11-08T08:15:41.237 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package rbd-fuse. 2023-11-08T08:15:41.261 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../117-rbd-fuse_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:41.289 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libjs-mathjax. 2023-11-08T08:15:41.303 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking rbd-fuse (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:41.312 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../106-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-11-08T08:15:41.355 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-11-08T08:15:41.700 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-attr (19.3.0-2) ... 2023-11-08T08:15:42.119 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-11-08T08:15:42.270 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-11-08T08:15:42.404 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-11-08T08:15:42.642 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up cephadm (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:43.373 INFO:teuthology.orchestra.run.smithi040.stdout:Adding system user cephadm....done 2023-11-08T08:15:43.393 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python-natsort-doc. 2023-11-08T08:15:43.410 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../107-python-natsort-doc_7.0.1-1_all.deb ... 2023-11-08T08:15:43.452 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-11-08T08:15:43.578 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-11-08T08:15:43.713 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-11-08T08:15:43.830 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-webencodings. 2023-11-08T08:15:43.850 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../108-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-11-08T08:15:43.888 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-11-08T08:15:44.148 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-11-08T08:15:44.258 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-html5lib. 2023-11-08T08:15:44.279 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../109-python3-html5lib_1.0.1-2_all.deb ... 2023-11-08T08:15:44.357 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-11-08T08:15:44.499 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-11-08T08:15:44.625 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-11-08T08:15:44.769 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-11-08T08:15:44.776 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-11-08T08:15:44.790 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../110-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-11-08T08:15:44.835 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-11-08T08:15:45.178 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-11-08T08:15:45.347 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pastescript. 2023-11-08T08:15:45.370 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../111-python3-pastescript_2.0.2-4_all.deb ... 2023-11-08T08:15:45.413 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-11-08T08:15:45.680 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-11-08T08:15:45.809 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-psutil. 2023-11-08T08:15:45.830 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../112-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-11-08T08:15:45.900 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-11-08T08:15:46.056 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-11-08T08:15:46.288 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-pyinotify. 2023-11-08T08:15:46.307 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../113-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-11-08T08:15:46.346 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-11-08T08:15:46.441 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-11-08T08:15:46.717 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package python3-simplejson. 2023-11-08T08:15:46.736 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-11-08T08:15:46.774 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-11-08T08:15:46.842 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-11-08T08:15:47.111 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-11-08T08:15:47.132 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-11-08T08:15:47.176 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-11-08T08:15:47.184 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-11-08T08:15:47.586 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-ceph-argparse (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:47.934 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package radosgw. 2023-11-08T08:15:47.955 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../116-radosgw_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:47.970 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-11-08T08:15:48.000 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking radosgw (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:48.139 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-11-08T08:15:48.506 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-11-08T08:15:48.857 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-11-08T08:15:49.241 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-11-08T08:15:49.412 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package rbd-fuse. 2023-11-08T08:15:49.425 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-11-08T08:15:49.432 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../117-rbd-fuse_18.0.0-2698-gb820560f-1focal_amd64.deb ... 2023-11-08T08:15:49.499 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking rbd-fuse (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:49.868 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-attr (19.3.0-2) ... 2023-11-08T08:15:49.893 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-11-08T08:15:50.243 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-11-08T08:15:50.253 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-11-08T08:15:50.394 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-11-08T08:15:50.520 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-11-08T08:15:50.604 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-11-08T08:15:50.764 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up cephadm (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:51.047 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-11-08T08:15:51.423 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-11-08T08:15:51.522 INFO:teuthology.orchestra.run.smithi169.stdout:Adding system user cephadm....done 2023-11-08T08:15:51.711 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-11-08T08:15:51.849 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-11-08T08:15:51.881 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-11-08T08:15:52.233 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-packaging (20.3-1) ... 2023-11-08T08:15:52.324 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-11-08T08:15:52.601 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-11-08T08:15:52.692 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-11-08T08:15:52.743 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-11-08T08:15:52.818 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-11-08T08:15:52.869 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-11-08T08:15:52.978 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-11-08T08:15:53.238 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-11-08T08:15:53.387 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-11-08T08:15:53.739 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-11-08T08:15:53.873 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-11-08T08:15:54.132 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-11-08T08:15:54.241 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-11-08T08:15:54.591 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-11-08T08:15:54.592 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-11-08T08:15:54.952 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-11-08T08:15:55.051 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up socat (1.7.3.3-2) ... 2023-11-08T08:15:55.227 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-ceph-common (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:55.328 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-11-08T08:15:55.637 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-11-08T08:15:55.721 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-ceph-argparse (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:15:55.780 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-11-08T08:15:56.114 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-11-08T08:15:56.223 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-11-08T08:15:56.240 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-11-08T08:15:56.341 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-11-08T08:15:56.500 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-11-08T08:15:56.574 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-11-08T08:15:56.860 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-11-08T08:15:56.925 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-11-08T08:15:57.244 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-11-08T08:15:57.285 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-11-08T08:15:57.370 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-11-08T08:15:57.410 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-11-08T08:15:57.505 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-11-08T08:15:57.862 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-11-08T08:15:57.931 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-11-08T08:15:58.230 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-11-08T08:15:58.332 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-11-08T08:15:58.573 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-11-08T08:15:58.695 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-11-08T08:15:58.819 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-11-08T08:15:58.965 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-11-08T08:15:59.202 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-11-08T08:15:59.308 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-11-08T08:15:59.603 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-11-08T08:15:59.709 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-11-08T08:16:00.021 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-11-08T08:16:00.077 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-packaging (20.3-1) ... 2023-11-08T08:16:00.463 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-11-08T08:16:00.485 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-11-08T08:16:00.596 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-11-08T08:16:00.763 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-11-08T08:16:00.840 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-11-08T08:16:01.000 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-11-08T08:16:01.140 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-11-08T08:16:01.421 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-11-08T08:16:01.616 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-11-08T08:16:01.855 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-11-08T08:16:01.981 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-11-08T08:16:02.009 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-11-08T08:16:02.107 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-11-08T08:16:02.461 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-11-08T08:16:02.550 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-11-08T08:16:02.896 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up socat (1.7.3.3-2) ... 2023-11-08T08:16:02.943 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-11-08T08:16:03.047 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-ceph-common (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:03.448 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-11-08T08:16:03.591 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-11-08T08:16:04.009 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-11-08T08:16:04.180 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-11-08T08:16:04.311 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-11-08T08:16:04.671 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-11-08T08:16:04.850 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-11-08T08:16:04.976 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-11-08T08:16:05.063 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-11-08T08:16:05.198 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-11-08T08:16:05.349 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-11-08T08:16:05.654 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-11-08T08:16:05.751 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-11-08T08:16:06.110 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-11-08T08:16:06.140 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-11-08T08:16:06.503 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-11-08T08:16:06.541 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-11-08T08:16:06.629 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-11-08T08:16:06.659 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-11-08T08:16:06.972 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-11-08T08:16:07.085 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-11-08T08:16:07.340 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-11-08T08:16:07.487 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2023-11-08T08:16:07.725 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-11-08T08:16:07.963 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-11-08T08:16:08.117 INFO:teuthology.orchestra.run.smithi169.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-11-08T08:16:08.390 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-11-08T08:16:08.460 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-11-08T08:16:08.587 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-11-08T08:16:08.974 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-11-08T08:16:09.409 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-11-08T08:16:09.560 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-11-08T08:16:09.671 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-11-08T08:16:09.685 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-11-08T08:16:09.872 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-11-08T08:16:09.999 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-11-08T08:16:10.087 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-11-08T08:16:10.124 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-rsa (4.0-3) ... 2023-11-08T08:16:10.484 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-11-08T08:16:10.627 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-11-08T08:16:10.696 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-11-08T08:16:11.036 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-11-08T08:16:11.437 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-11-08T08:16:11.864 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-11-08T08:16:12.382 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librados2 (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:12.510 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-11-08T08:16:12.602 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2023-11-08T08:16:12.670 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-11-08T08:16:12.728 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librgw2 (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:12.879 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libsqlite3-mod-ceph (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:13.006 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libcephfs2 (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:13.129 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-11-08T08:16:13.131 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-routes (2.4.1-2) ... 2023-11-08T08:16:13.516 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-11-08T08:16:13.533 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libradosstriper1 (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:13.693 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-11-08T08:16:13.885 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-11-08T08:16:14.010 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-11-08T08:16:14.070 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librbd1 (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:14.195 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-modules-core (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:14.321 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-fuse (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:14.428 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-11-08T08:16:14.601 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-08T08:16:14.602 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-08T08:16:14.865 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2023-11-08T08:16:15.365 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-11-08T08:16:15.490 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libcephfs-dev (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:15.616 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-rados (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:15.766 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up cephfs-top (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:15.792 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-11-08T08:16:15.893 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-11-08T08:16:17.033 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-11-08T08:16:17.158 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-11-08T08:16:17.265 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-rbd (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:17.317 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-11-08T08:16:17.416 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up rbd-fuse (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:17.443 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-rsa (4.0-3) ... 2023-11-08T08:16:17.543 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-rgw (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:17.668 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cephfs (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:17.819 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up cephfs-shell (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:17.820 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-11-08T08:16:17.945 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-common (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:17.945 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-11-08T08:16:18.338 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-11-08T08:16:18.551 INFO:teuthology.orchestra.run.smithi040.stdout:Adding group ceph....done 2023-11-08T08:16:18.716 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-11-08T08:16:19.030 INFO:teuthology.orchestra.run.smithi040.stdout:Adding system user ceph....done 2023-11-08T08:16:19.157 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-11-08T08:16:19.255 INFO:teuthology.orchestra.run.smithi040.stdout:Setting system user ceph properties....done 2023-11-08T08:16:19.290 INFO:teuthology.orchestra.run.smithi040.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-11-08T08:16:19.504 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-11-08T08:16:19.601 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up librados2 (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:19.838 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2023-11-08T08:16:19.989 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up librgw2 (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:20.084 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-11-08T08:16:20.115 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libsqlite3-mod-ceph (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:20.241 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libcephfs2 (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:20.400 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-routes (2.4.1-2) ... 2023-11-08T08:16:20.769 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libradosstriper1 (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:20.894 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-11-08T08:16:21.104 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up cephfs-mirror (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:21.240 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-test (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:21.271 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up librbd1 (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:21.366 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up radosgw (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:21.406 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mgr-modules-core (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:21.557 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-fuse (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:21.710 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-08T08:16:21.711 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-08T08:16:21.810 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-08T08:16:21.810 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-08T08:16:22.667 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libcephfs-dev (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:22.793 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-rados (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:22.919 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up cephfs-top (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:22.992 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-base (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:23.078 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-11-08T08:16:23.528 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-11-08T08:16:24.200 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-rbd (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:24.326 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up rbd-fuse (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:24.450 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-rgw (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:24.494 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mds (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:24.610 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up python3-cephfs (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:24.736 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up cephfs-shell (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:24.797 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-08T08:16:24.797 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-08T08:16:24.862 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-common (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:25.513 INFO:teuthology.orchestra.run.smithi169.stdout:Adding group ceph....done 2023-11-08T08:16:25.696 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:25.981 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-08T08:16:25.982 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-08T08:16:26.168 INFO:teuthology.orchestra.run.smithi169.stdout:Adding system user ceph....done 2023-11-08T08:16:26.392 INFO:teuthology.orchestra.run.smithi169.stdout:Setting system user ceph properties....done 2023-11-08T08:16:26.428 INFO:teuthology.orchestra.run.smithi169.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-11-08T08:16:26.617 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-11-08T08:16:26.873 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-osd (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:27.147 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-11-08T08:16:27.664 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-08T08:16:27.664 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-08T08:16:28.049 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up cephfs-mirror (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:28.175 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-test (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:28.301 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up radosgw (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:28.590 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-k8sevents (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:28.633 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-08T08:16:28.633 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-08T08:16:28.725 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:28.876 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mon (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:29.160 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-08T08:16:29.161 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-08T08:16:29.900 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-base (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:30.053 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-cephadm (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:30.203 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:30.363 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-dashboard (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:30.386 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-11-08T08:16:30.514 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-volume (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:30.678 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-11-08T08:16:30.843 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-11-08T08:16:31.263 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mds (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:31.328 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-11-08T08:16:31.546 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-08T08:16:31.547 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-08T08:16:32.432 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mgr (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:32.703 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-08T08:16:32.703 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-08T08:16:33.525 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-osd (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:34.314 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-08T08:16:34.315 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-08T08:16:35.160 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mgr-k8sevents (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:35.335 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:35.486 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mon (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:35.747 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-08T08:16:35.748 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-08T08:16:36.538 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mgr-cephadm (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:36.689 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:36.823 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-11-08T08:16:36.849 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-mgr-dashboard (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:37.025 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ceph-volume (18.0.0-2698-gb820560f-1focal) ... 2023-11-08T08:16:37.152 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-11-08T08:16:37.337 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-11-08T08:16:37.588 INFO:teuthology.orchestra.run.smithi040.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-08T08:16:37.594 DEBUG:teuthology.orchestra.run.smithi040:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-11-08T08:16:37.723 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2023-11-08T08:16:37.805 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-11-08T08:16:37.939 INFO:teuthology.orchestra.run.smithi040.stdout:Building dependency tree... 2023-11-08T08:16:37.940 INFO:teuthology.orchestra.run.smithi040.stdout:Reading state information... 2023-11-08T08:16:38.162 INFO:teuthology.orchestra.run.smithi040.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-11-08T08:16:38.163 INFO:teuthology.orchestra.run.smithi040.stdout:flex is already the newest version (2.6.4-6.2). 2023-11-08T08:16:38.163 INFO:teuthology.orchestra.run.smithi040.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-11-08T08:16:38.163 INFO:teuthology.orchestra.run.smithi040.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-11-08T08:16:38.163 INFO:teuthology.orchestra.run.smithi040.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-11-08T08:16:38.163 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages were automatically installed and are no longer required: 2023-11-08T08:16:38.164 INFO:teuthology.orchestra.run.smithi040.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-11-08T08:16:38.164 INFO:teuthology.orchestra.run.smithi040.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-08T08:16:38.165 INFO:teuthology.orchestra.run.smithi040.stdout:The following additional packages will be installed: 2023-11-08T08:16:38.165 INFO:teuthology.orchestra.run.smithi040.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2023-11-08T08:16:38.165 INFO:teuthology.orchestra.run.smithi040.stdout: libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 2023-11-08T08:16:38.166 INFO:teuthology.orchestra.run.smithi040.stdout: libteamdctl0 libuuid1 modemmanager network-manager-pptp postgresql-12 2023-11-08T08:16:38.166 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-client-12 postgresql-client-common ppp pptp-linux usb-modeswitch 2023-11-08T08:16:38.166 INFO:teuthology.orchestra.run.smithi040.stdout: usb-modeswitch-data wpasupplicant 2023-11-08T08:16:38.167 INFO:teuthology.orchestra.run.smithi040.stdout:Suggested packages: 2023-11-08T08:16:38.167 INFO:teuthology.orchestra.run.smithi040.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2023-11-08T08:16:38.168 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-doc-12 libjson-perl comgt wvdial util-linux-locales wpagui 2023-11-08T08:16:38.168 INFO:teuthology.orchestra.run.smithi040.stdout: libengine-pkcs11-openssl 2023-11-08T08:16:38.276 INFO:teuthology.orchestra.run.smithi040.stdout:The following NEW packages will be installed: 2023-11-08T08:16:38.277 INFO:teuthology.orchestra.run.smithi040.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-11-08T08:16:38.277 INFO:teuthology.orchestra.run.smithi040.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2023-11-08T08:16:38.278 INFO:teuthology.orchestra.run.smithi040.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-11-08T08:16:38.278 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2023-11-08T08:16:38.278 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2023-11-08T08:16:38.278 INFO:teuthology.orchestra.run.smithi040.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-08T08:16:38.279 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages will be upgraded: 2023-11-08T08:16:38.279 INFO:teuthology.orchestra.run.smithi040.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2023-11-08T08:16:38.480 INFO:teuthology.orchestra.run.smithi040.stdout:6 upgraded, 31 newly installed, 0 to remove and 252 not upgraded. 2023-11-08T08:16:38.480 INFO:teuthology.orchestra.run.smithi040.stdout:Need to get 25.3 MB of archives. 2023-11-08T08:16:38.480 INFO:teuthology.orchestra.run.smithi040.stdout:After this operation, 79.9 MB of additional disk space will be used. 2023-11-08T08:16:38.481 INFO:teuthology.orchestra.run.smithi040.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-11-08T08:16:39.118 INFO:teuthology.orchestra.run.smithi040.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-11-08T08:16:39.120 INFO:teuthology.orchestra.run.smithi040.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-11-08T08:16:39.122 INFO:teuthology.orchestra.run.smithi040.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2023-11-08T08:16:39.134 INFO:teuthology.orchestra.run.smithi040.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2023-11-08T08:16:39.136 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.20 [1,321 kB] 2023-11-08T08:16:39.235 INFO:teuthology.orchestra.run.smithi040.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-11-08T08:16:39.236 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-11-08T08:16:39.244 INFO:teuthology.orchestra.run.smithi040.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2023-11-08T08:16:39.249 INFO:teuthology.orchestra.run.smithi040.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-11-08T08:16:39.250 INFO:teuthology.orchestra.run.smithi040.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-11-08T08:16:39.417 INFO:teuthology.orchestra.run.smithi040.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2023-11-08T08:16:39.688 INFO:teuthology.orchestra.run.smithi040.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-11-08T08:16:39.826 INFO:teuthology.orchestra.run.smithi040.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-11-08T08:16:39.833 INFO:teuthology.orchestra.run.smithi040.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-11-08T08:16:39.933 INFO:teuthology.orchestra.run.smithi040.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-11-08T08:16:39.937 INFO:teuthology.orchestra.run.smithi040.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-11-08T08:16:40.018 INFO:teuthology.orchestra.run.smithi040.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.16-0ubuntu0.20.04.1 [115 kB] 2023-11-08T08:16:40.034 INFO:teuthology.orchestra.run.smithi040.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-11-08T08:16:40.102 INFO:teuthology.orchestra.run.smithi040.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-11-08T08:16:40.102 INFO:teuthology.orchestra.run.smithi040.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.20 [1,584 kB] 2023-11-08T08:16:40.186 INFO:teuthology.orchestra.run.smithi040.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-11-08T08:16:40.350 INFO:teuthology.orchestra.run.smithi040.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-11-08T08:16:40.979 INFO:teuthology.orchestra.run.smithi040.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-11-08T08:16:41.079 INFO:teuthology.orchestra.run.smithi040.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-11-08T08:16:41.169 INFO:teuthology.orchestra.run.smithi040.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-11-08T08:16:41.178 INFO:teuthology.orchestra.run.smithi040.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-11-08T08:16:41.179 INFO:teuthology.orchestra.run.smithi040.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-11-08T08:16:41.180 INFO:teuthology.orchestra.run.smithi040.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2023-11-08T08:16:41.180 INFO:teuthology.orchestra.run.smithi040.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.16-0ubuntu0.20.04.1 [1,053 kB] 2023-11-08T08:16:41.213 INFO:teuthology.orchestra.run.smithi040.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2023-11-08T08:16:41.218 INFO:teuthology.orchestra.run.smithi040.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.16-0ubuntu0.20.04.1 [13.5 MB] 2023-11-08T08:16:41.597 INFO:teuthology.orchestra.run.smithi040.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2023-11-08T08:16:41.761 INFO:teuthology.orchestra.run.smithi040.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2023-11-08T08:16:41.845 INFO:teuthology.orchestra.run.smithi040.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2023-11-08T08:16:41.845 INFO:teuthology.orchestra.run.smithi040.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-11-08T08:16:41.961 INFO:teuthology.orchestra.run.smithi040.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-11-08T08:16:42.650 INFO:teuthology.orchestra.run.smithi040.stdout:Preconfiguring packages ... 2023-11-08T08:16:42.899 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 25.3 MB in 4s (6,719 kB/s) 2023-11-08T08:16:42.958 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-11-08T08:16:43.256 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118860 files and directories currently installed.) 2023-11-08T08:16:43.260 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-11-08T08:16:43.451 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-11-08T08:16:43.771 INFO:teuthology.orchestra.run.smithi169.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-08T08:16:43.777 DEBUG:teuthology.orchestra.run.smithi169:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-11-08T08:16:43.896 INFO:teuthology.orchestra.run.smithi169.stdout:Reading package lists... 2023-11-08T08:16:44.099 INFO:teuthology.orchestra.run.smithi169.stdout:Building dependency tree... 2023-11-08T08:16:44.100 INFO:teuthology.orchestra.run.smithi169.stdout:Reading state information... 2023-11-08T08:16:44.303 INFO:teuthology.orchestra.run.smithi169.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-11-08T08:16:44.303 INFO:teuthology.orchestra.run.smithi169.stdout:flex is already the newest version (2.6.4-6.2). 2023-11-08T08:16:44.304 INFO:teuthology.orchestra.run.smithi169.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-11-08T08:16:44.304 INFO:teuthology.orchestra.run.smithi169.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-11-08T08:16:44.304 INFO:teuthology.orchestra.run.smithi169.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-11-08T08:16:44.304 INFO:teuthology.orchestra.run.smithi169.stdout:The following packages were automatically installed and are no longer required: 2023-11-08T08:16:44.305 INFO:teuthology.orchestra.run.smithi169.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-11-08T08:16:44.306 INFO:teuthology.orchestra.run.smithi169.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-08T08:16:44.306 INFO:teuthology.orchestra.run.smithi169.stdout:The following additional packages will be installed: 2023-11-08T08:16:44.306 INFO:teuthology.orchestra.run.smithi169.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2023-11-08T08:16:44.306 INFO:teuthology.orchestra.run.smithi169.stdout: libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 2023-11-08T08:16:44.307 INFO:teuthology.orchestra.run.smithi169.stdout: libteamdctl0 libuuid1 modemmanager network-manager-pptp postgresql-12 2023-11-08T08:16:44.307 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-client-12 postgresql-client-common ppp pptp-linux usb-modeswitch 2023-11-08T08:16:44.307 INFO:teuthology.orchestra.run.smithi169.stdout: usb-modeswitch-data wpasupplicant 2023-11-08T08:16:44.307 INFO:teuthology.orchestra.run.smithi169.stdout:Suggested packages: 2023-11-08T08:16:44.308 INFO:teuthology.orchestra.run.smithi169.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2023-11-08T08:16:44.308 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-doc-12 libjson-perl comgt wvdial util-linux-locales wpagui 2023-11-08T08:16:44.308 INFO:teuthology.orchestra.run.smithi169.stdout: libengine-pkcs11-openssl 2023-11-08T08:16:44.413 INFO:teuthology.orchestra.run.smithi169.stdout:The following NEW packages will be installed: 2023-11-08T08:16:44.414 INFO:teuthology.orchestra.run.smithi169.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-11-08T08:16:44.414 INFO:teuthology.orchestra.run.smithi169.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2023-11-08T08:16:44.415 INFO:teuthology.orchestra.run.smithi169.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-11-08T08:16:44.415 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2023-11-08T08:16:44.415 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2023-11-08T08:16:44.416 INFO:teuthology.orchestra.run.smithi169.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-08T08:16:44.416 INFO:teuthology.orchestra.run.smithi169.stdout:The following packages will be upgraded: 2023-11-08T08:16:44.417 INFO:teuthology.orchestra.run.smithi169.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2023-11-08T08:16:44.487 INFO:teuthology.orchestra.run.smithi169.stdout:6 upgraded, 31 newly installed, 0 to remove and 252 not upgraded. 2023-11-08T08:16:44.488 INFO:teuthology.orchestra.run.smithi169.stdout:Need to get 25.3 MB of archives. 2023-11-08T08:16:44.488 INFO:teuthology.orchestra.run.smithi169.stdout:After this operation, 79.9 MB of additional disk space will be used. 2023-11-08T08:16:44.489 INFO:teuthology.orchestra.run.smithi169.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-11-08T08:16:44.620 INFO:teuthology.orchestra.run.smithi169.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-11-08T08:16:44.622 INFO:teuthology.orchestra.run.smithi169.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-11-08T08:16:44.623 INFO:teuthology.orchestra.run.smithi169.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2023-11-08T08:16:44.629 INFO:teuthology.orchestra.run.smithi169.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2023-11-08T08:16:44.629 INFO:teuthology.orchestra.run.smithi169.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.20 [1,321 kB] 2023-11-08T08:16:44.646 INFO:teuthology.orchestra.run.smithi169.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-11-08T08:16:44.647 INFO:teuthology.orchestra.run.smithi169.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-11-08T08:16:44.648 INFO:teuthology.orchestra.run.smithi169.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2023-11-08T08:16:44.650 INFO:teuthology.orchestra.run.smithi169.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-11-08T08:16:44.650 INFO:teuthology.orchestra.run.smithi169.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-11-08T08:16:44.653 INFO:teuthology.orchestra.run.smithi169.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2023-11-08T08:16:44.655 INFO:teuthology.orchestra.run.smithi169.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-11-08T08:16:44.656 INFO:teuthology.orchestra.run.smithi169.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-11-08T08:16:44.657 INFO:teuthology.orchestra.run.smithi169.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-11-08T08:16:44.671 INFO:teuthology.orchestra.run.smithi169.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-11-08T08:16:44.671 INFO:teuthology.orchestra.run.smithi169.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-11-08T08:16:44.677 INFO:teuthology.orchestra.run.smithi169.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.16-0ubuntu0.20.04.1 [115 kB] 2023-11-08T08:16:44.679 INFO:teuthology.orchestra.run.smithi169.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-11-08T08:16:44.689 INFO:teuthology.orchestra.run.smithi169.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-11-08T08:16:44.689 INFO:teuthology.orchestra.run.smithi169.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.20 [1,584 kB] 2023-11-08T08:16:44.706 INFO:teuthology.orchestra.run.smithi169.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-11-08T08:16:44.706 INFO:teuthology.orchestra.run.smithi169.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-11-08T08:16:44.715 INFO:teuthology.orchestra.run.smithi169.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-11-08T08:16:44.727 INFO:teuthology.orchestra.run.smithi169.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-11-08T08:16:44.746 INFO:teuthology.orchestra.run.smithi169.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-11-08T08:16:44.749 INFO:teuthology.orchestra.run.smithi169.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-11-08T08:16:44.749 INFO:teuthology.orchestra.run.smithi169.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-11-08T08:16:44.750 INFO:teuthology.orchestra.run.smithi169.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2023-11-08T08:16:44.751 INFO:teuthology.orchestra.run.smithi169.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.16-0ubuntu0.20.04.1 [1,053 kB] 2023-11-08T08:16:44.761 INFO:teuthology.orchestra.run.smithi169.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2023-11-08T08:16:44.763 INFO:teuthology.orchestra.run.smithi169.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.16-0ubuntu0.20.04.1 [13.5 MB] 2023-11-08T08:16:44.894 INFO:teuthology.orchestra.run.smithi169.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2023-11-08T08:16:44.894 INFO:teuthology.orchestra.run.smithi169.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2023-11-08T08:16:44.894 INFO:teuthology.orchestra.run.smithi169.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2023-11-08T08:16:44.894 INFO:teuthology.orchestra.run.smithi169.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-11-08T08:16:44.895 INFO:teuthology.orchestra.run.smithi169.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-11-08T08:16:45.166 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-11-08T08:16:45.469 INFO:teuthology.orchestra.run.smithi169.stdout:Preconfiguring packages ... 2023-11-08T08:16:45.749 INFO:teuthology.orchestra.run.smithi169.stdout:Fetched 25.3 MB in 0s (54.1 MB/s) 2023-11-08T08:16:46.081 INFO:teuthology.orchestra.run.smithi169.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118860 files and directories currently installed.) 2023-11-08T08:16:46.084 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-11-08T08:16:46.473 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-11-08T08:16:47.274 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118860 files and directories currently installed.) 2023-11-08T08:16:47.278 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-11-08T08:16:47.408 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-11-08T08:16:47.862 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-11-08T08:16:48.031 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-11-08T08:16:48.142 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-11-08T08:16:48.548 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-11-08T08:16:49.086 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118860 files and directories currently installed.) 2023-11-08T08:16:49.090 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-11-08T08:16:49.223 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-11-08T08:16:49.757 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-11-08T08:16:49.928 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-11-08T08:16:50.234 INFO:teuthology.orchestra.run.smithi169.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118860 files and directories currently installed.) 2023-11-08T08:16:50.237 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-11-08T08:16:50.385 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-11-08T08:16:50.437 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.20_amd64.deb ... 2023-11-08T08:16:50.575 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.20) over (1.1.1f-1ubuntu2.16) ... 2023-11-08T08:16:50.856 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-11-08T08:16:51.024 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-11-08T08:16:51.145 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-11-08T08:16:51.168 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2023-11-08T08:16:51.220 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-11-08T08:16:51.559 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-11-08T08:16:51.624 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package dump. 2023-11-08T08:16:51.642 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2023-11-08T08:16:51.682 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking dump (0.4b46-6) ... 2023-11-08T08:16:52.058 INFO:teuthology.orchestra.run.smithi169.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 118860 files and directories currently installed.) 2023-11-08T08:16:52.062 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-11-08T08:16:52.209 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-11-08T08:16:52.279 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package indent. 2023-11-08T08:16:52.302 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2023-11-08T08:16:52.345 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2023-11-08T08:16:52.732 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-11-08T08:16:52.745 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-11-08T08:16:52.755 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2023-11-08T08:16:52.798 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-11-08T08:16:52.873 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-11-08T08:16:53.210 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-11-08T08:16:53.232 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-11-08T08:16:53.310 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-11-08T08:16:53.443 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.20_amd64.deb ... 2023-11-08T08:16:53.586 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.20) over (1.1.1f-1ubuntu2.16) ... 2023-11-08T08:16:53.621 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-11-08T08:16:53.645 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2023-11-08T08:16:53.729 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-11-08T08:16:54.115 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-11-08T08:16:54.134 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2023-11-08T08:16:54.141 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-11-08T08:16:54.163 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-11-08T08:16:54.207 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-11-08T08:16:54.207 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-11-08T08:16:54.560 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libmbim-proxy. 2023-11-08T08:16:54.579 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-11-08T08:16:54.602 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package dump. 2023-11-08T08:16:54.618 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-11-08T08:16:54.623 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2023-11-08T08:16:54.685 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking dump (0.4b46-6) ... 2023-11-08T08:16:54.998 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-11-08T08:16:55.021 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-11-08T08:16:55.064 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-11-08T08:16:55.065 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package indent. 2023-11-08T08:16:55.085 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2023-11-08T08:16:55.122 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2023-11-08T08:16:55.450 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-11-08T08:16:55.472 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2023-11-08T08:16:55.491 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libndp0:amd64. 2023-11-08T08:16:55.508 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-11-08T08:16:55.514 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-11-08T08:16:55.557 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-11-08T08:16:55.912 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-11-08T08:16:55.933 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-11-08T08:16:55.961 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libnm0:amd64. 2023-11-08T08:16:55.978 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-11-08T08:16:55.982 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-11-08T08:16:56.027 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-11-08T08:16:56.323 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-11-08T08:16:56.345 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2023-11-08T08:16:56.389 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-11-08T08:16:56.439 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libpq5:amd64. 2023-11-08T08:16:56.458 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../14-libpq5_12.16-0ubuntu0.20.04.1_amd64.deb ... 2023-11-08T08:16:56.496 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libpq5:amd64 (12.16-0ubuntu0.20.04.1) ... 2023-11-08T08:16:56.893 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-11-08T08:16:56.913 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../15-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-11-08T08:16:56.958 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-11-08T08:16:57.036 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-11-08T08:16:57.057 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-11-08T08:16:57.102 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-11-08T08:16:57.361 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqmi-proxy. 2023-11-08T08:16:57.385 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../16-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-11-08T08:16:57.438 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libmbim-proxy. 2023-11-08T08:16:57.452 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-11-08T08:16:57.460 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-11-08T08:16:57.496 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-11-08T08:16:57.722 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-11-08T08:16:57.746 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../17-libssl-dev_1.1.1f-1ubuntu2.20_amd64.deb ... 2023-11-08T08:16:57.788 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.20) ... 2023-11-08T08:16:57.883 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-11-08T08:16:57.902 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-11-08T08:16:57.941 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-11-08T08:16:58.344 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libndp0:amd64. 2023-11-08T08:16:58.364 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-11-08T08:16:58.383 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-11-08T08:16:58.402 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-11-08T08:16:58.403 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../18-libteamdctl0_1.30-1_amd64.deb ... 2023-11-08T08:16:58.441 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-11-08T08:16:58.782 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libnm0:amd64. 2023-11-08T08:16:58.802 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-11-08T08:16:58.865 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-11-08T08:16:58.878 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package modemmanager. 2023-11-08T08:16:58.900 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../19-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-11-08T08:16:58.969 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-11-08T08:16:59.293 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libpq5:amd64. 2023-11-08T08:16:59.313 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../14-libpq5_12.16-0ubuntu0.20.04.1_amd64.deb ... 2023-11-08T08:16:59.351 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libpq5:amd64 (12.16-0ubuntu0.20.04.1) ... 2023-11-08T08:16:59.651 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package wpasupplicant. 2023-11-08T08:16:59.671 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../20-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-11-08T08:16:59.709 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-11-08T08:16:59.763 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-11-08T08:16:59.784 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../15-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-11-08T08:16:59.829 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-11-08T08:17:00.216 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libqmi-proxy. 2023-11-08T08:17:00.238 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../16-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-11-08T08:17:00.282 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-11-08T08:17:00.298 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package network-manager. 2023-11-08T08:17:00.318 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../21-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-11-08T08:17:00.434 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-11-08T08:17:00.601 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-11-08T08:17:00.623 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../17-libssl-dev_1.1.1f-1ubuntu2.20_amd64.deb ... 2023-11-08T08:17:00.667 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.20) ... 2023-11-08T08:17:01.280 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-11-08T08:17:01.281 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ppp. 2023-11-08T08:17:01.302 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../18-libteamdctl0_1.30-1_amd64.deb ... 2023-11-08T08:17:01.305 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../22-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-11-08T08:17:01.346 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-11-08T08:17:01.792 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-11-08T08:17:01.825 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package modemmanager. 2023-11-08T08:17:01.847 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../19-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-11-08T08:17:01.912 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-11-08T08:17:02.278 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package pptp-linux. 2023-11-08T08:17:02.301 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../23-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-11-08T08:17:02.344 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-11-08T08:17:02.598 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package wpasupplicant. 2023-11-08T08:17:02.619 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../20-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-11-08T08:17:02.664 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-11-08T08:17:02.781 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package network-manager-pptp. 2023-11-08T08:17:02.804 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../24-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-11-08T08:17:02.906 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-11-08T08:17:03.253 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package network-manager. 2023-11-08T08:17:03.275 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../21-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-11-08T08:17:03.276 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-client-common. 2023-11-08T08:17:03.297 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../25-postgresql-client-common_214ubuntu0.1_all.deb ... 2023-11-08T08:17:03.342 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2023-11-08T08:17:03.370 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-11-08T08:17:03.681 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-client-12. 2023-11-08T08:17:03.705 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../26-postgresql-client-12_12.16-0ubuntu0.20.04.1_amd64.deb ... 2023-11-08T08:17:03.755 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-client-12 (12.16-0ubuntu0.20.04.1) ... 2023-11-08T08:17:04.193 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package ppp. 2023-11-08T08:17:04.215 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../22-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-11-08T08:17:04.426 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-common. 2023-11-08T08:17:04.449 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../27-postgresql-common_214ubuntu0.1_all.deb ... 2023-11-08T08:17:04.566 INFO:teuthology.orchestra.run.smithi040.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-11-08T08:17:04.597 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2023-11-08T08:17:04.721 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-11-08T08:17:05.099 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package pptp-linux. 2023-11-08T08:17:05.117 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../23-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-11-08T08:17:05.125 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-12. 2023-11-08T08:17:05.140 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../28-postgresql-12_12.16-0ubuntu0.20.04.1_amd64.deb ... 2023-11-08T08:17:05.156 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-11-08T08:17:05.190 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-12 (12.16-0ubuntu0.20.04.1) ... 2023-11-08T08:17:05.560 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package network-manager-pptp. 2023-11-08T08:17:05.580 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../24-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-11-08T08:17:05.654 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-11-08T08:17:06.013 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql-client-common. 2023-11-08T08:17:06.035 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../25-postgresql-client-common_214ubuntu0.1_all.deb ... 2023-11-08T08:17:06.079 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2023-11-08T08:17:06.459 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql-client-12. 2023-11-08T08:17:06.478 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../26-postgresql-client-12_12.16-0ubuntu0.20.04.1_amd64.deb ... 2023-11-08T08:17:06.517 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql-client-12 (12.16-0ubuntu0.20.04.1) ... 2023-11-08T08:17:06.813 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql. 2023-11-08T08:17:06.835 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../29-postgresql_12+214ubuntu0.1_all.deb ... 2023-11-08T08:17:06.880 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2023-11-08T08:17:07.130 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql-common. 2023-11-08T08:17:07.152 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../27-postgresql-common_214ubuntu0.1_all.deb ... 2023-11-08T08:17:07.198 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-client. 2023-11-08T08:17:07.213 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../30-postgresql-client_12+214ubuntu0.1_all.deb ... 2023-11-08T08:17:07.227 INFO:teuthology.orchestra.run.smithi169.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-11-08T08:17:07.257 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2023-11-08T08:17:07.258 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2023-11-08T08:17:07.829 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql-12. 2023-11-08T08:17:07.852 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../28-postgresql-12_12.16-0ubuntu0.20.04.1_amd64.deb ... 2023-11-08T08:17:07.902 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql-12 (12.16-0ubuntu0.20.04.1) ... 2023-11-08T08:17:08.268 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-contrib. 2023-11-08T08:17:08.291 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../31-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2023-11-08T08:17:08.334 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2023-11-08T08:17:08.645 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-11-08T08:17:08.667 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../32-usb-modeswitch-data_20191128-3_all.deb ... 2023-11-08T08:17:08.711 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-11-08T08:17:09.140 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package usb-modeswitch. 2023-11-08T08:17:09.161 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../33-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-11-08T08:17:09.229 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-11-08T08:17:09.456 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql. 2023-11-08T08:17:09.481 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../29-postgresql_12+214ubuntu0.1_all.deb ... 2023-11-08T08:17:09.523 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2023-11-08T08:17:09.606 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2023-11-08T08:17:09.834 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql-client. 2023-11-08T08:17:09.857 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../30-postgresql-client_12+214ubuntu0.1_all.deb ... 2023-11-08T08:17:09.887 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.20) ... 2023-11-08T08:17:09.900 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2023-11-08T08:17:10.219 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package postgresql-contrib. 2023-11-08T08:17:10.241 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../31-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2023-11-08T08:17:10.280 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libpq5:amd64 (12.16-0ubuntu0.20.04.1) ... 2023-11-08T08:17:10.291 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2023-11-08T08:17:10.406 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-11-08T08:17:10.532 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-11-08T08:17:10.604 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-11-08T08:17:10.627 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../32-usb-modeswitch-data_20191128-3_all.deb ... 2023-11-08T08:17:10.671 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-11-08T08:17:10.700 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-11-08T08:17:11.099 INFO:teuthology.orchestra.run.smithi169.stdout:Selecting previously unselected package usb-modeswitch. 2023-11-08T08:17:11.118 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing to unpack .../33-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-11-08T08:17:11.143 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-08T08:17:11.144 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-08T08:17:11.179 INFO:teuthology.orchestra.run.smithi169.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-11-08T08:17:11.523 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2023-11-08T08:17:12.046 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.20) ... 2023-11-08T08:17:12.069 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-client-12 (12.16-0ubuntu0.20.04.1) ... 2023-11-08T08:17:12.590 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libpq5:amd64 (12.16-0ubuntu0.20.04.1) ... 2023-11-08T08:17:12.715 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-11-08T08:17:12.833 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-11-08T08:17:12.993 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-11-08T08:17:13.470 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-08T08:17:13.470 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-08T08:17:14.267 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-11-08T08:17:14.462 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql-client-12 (12.16-0ubuntu0.20.04.1) ... 2023-11-08T08:17:14.517 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-11-08T08:17:14.643 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-11-08T08:17:14.769 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2023-11-08T08:17:16.066 INFO:teuthology.orchestra.run.smithi040.stdout:Adding user postgres to group ssl-cert 2023-11-08T08:17:16.458 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:16.458 INFO:teuthology.orchestra.run.smithi040.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-11-08T08:17:16.571 INFO:teuthology.orchestra.run.smithi169.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-11-08T08:17:16.709 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-11-08T08:17:16.852 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-11-08T08:17:16.965 INFO:teuthology.orchestra.run.smithi040.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-11-08T08:17:16.965 INFO:teuthology.orchestra.run.smithi040.stdout:Removing obsolete dictionary files: 2023-11-08T08:17:16.978 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2023-11-08T08:17:17.548 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-11-08T08:17:18.660 INFO:teuthology.orchestra.run.smithi169.stdout:Adding user postgres to group ssl-cert 2023-11-08T08:17:19.087 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.20) ... 2023-11-08T08:17:19.097 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:19.098 INFO:teuthology.orchestra.run.smithi169.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-11-08T08:17:19.213 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-11-08T08:17:19.339 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-11-08T08:17:19.507 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-11-08T08:17:19.574 INFO:teuthology.orchestra.run.smithi169.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-11-08T08:17:19.574 INFO:teuthology.orchestra.run.smithi169.stdout:Removing obsolete dictionary files: 2023-11-08T08:17:20.146 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-11-08T08:17:20.735 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-11-08T08:17:21.162 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up dump (0.4b46-6) ... 2023-11-08T08:17:21.392 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-11-08T08:17:21.455 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-11-08T08:17:21.605 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.20) ... 2023-11-08T08:17:21.623 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-11-08T08:17:21.749 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2023-11-08T08:17:21.778 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-11-08T08:17:21.875 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-11-08T08:17:21.907 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-11-08T08:17:22.025 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-11-08T08:17:22.093 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-11-08T08:17:22.219 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-12 (12.16-0ubuntu0.20.04.1) ... 2023-11-08T08:17:22.777 INFO:teuthology.orchestra.run.smithi040.stdout:Creating new PostgreSQL cluster 12/main ... 2023-11-08T08:17:22.814 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2023-11-08T08:17:22.897 INFO:teuthology.orchestra.run.smithi040.stdout:The files belonging to this database system will be owned by user "postgres". 2023-11-08T08:17:22.897 INFO:teuthology.orchestra.run.smithi040.stdout:This user must also own the server process. 2023-11-08T08:17:22.897 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:22.898 INFO:teuthology.orchestra.run.smithi040.stdout:The database cluster will be initialized with locale "en_US". 2023-11-08T08:17:22.898 INFO:teuthology.orchestra.run.smithi040.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-11-08T08:17:22.898 INFO:teuthology.orchestra.run.smithi040.stdout:The default text search configuration will be set to "english". 2023-11-08T08:17:22.898 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:22.898 INFO:teuthology.orchestra.run.smithi040.stdout:Data page checksums are disabled. 2023-11-08T08:17:22.899 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:22.899 INFO:teuthology.orchestra.run.smithi040.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2023-11-08T08:17:22.900 INFO:teuthology.orchestra.run.smithi040.stdout:creating subdirectories ... ok 2023-11-08T08:17:22.901 INFO:teuthology.orchestra.run.smithi040.stdout:selecting dynamic shared memory implementation ... posix 2023-11-08T08:17:22.938 INFO:teuthology.orchestra.run.smithi040.stdout:selecting default max_connections ... 100 2023-11-08T08:17:22.987 INFO:teuthology.orchestra.run.smithi040.stdout:selecting default shared_buffers ... 128MB 2023-11-08T08:17:23.010 INFO:teuthology.orchestra.run.smithi040.stdout:selecting default time zone ... Etc/UTC 2023-11-08T08:17:23.011 INFO:teuthology.orchestra.run.smithi040.stdout:creating configuration files ... ok 2023-11-08T08:17:23.198 INFO:teuthology.orchestra.run.smithi040.stdout:running bootstrap script ... ok 2023-11-08T08:17:23.308 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-11-08T08:17:23.746 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up dump (0.4b46-6) ... 2023-11-08T08:17:23.832 INFO:teuthology.orchestra.run.smithi040.stdout:performing post-bootstrap initialization ... ok 2023-11-08T08:17:23.862 INFO:teuthology.orchestra.run.smithi169.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-11-08T08:17:23.932 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-11-08T08:17:24.058 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-11-08T08:17:24.217 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2023-11-08T08:17:24.343 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-11-08T08:17:24.440 INFO:teuthology.orchestra.run.smithi040.stdout:syncing data to disk ... ok 2023-11-08T08:17:24.440 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:24.441 INFO:teuthology.orchestra.run.smithi040.stdout:Success. You can now start the database server using: 2023-11-08T08:17:24.441 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:24.441 INFO:teuthology.orchestra.run.smithi040.stdout: pg_ctlcluster 12 main start 2023-11-08T08:17:24.441 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:24.561 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-11-08T08:17:24.687 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql-12 (12.16-0ubuntu0.20.04.1) ... 2023-11-08T08:17:25.039 INFO:teuthology.orchestra.run.smithi040.stdout:Ver Cluster Port Status Owner Data directory Log file 2023-11-08T08:17:25.040 INFO:teuthology.orchestra.run.smithi040.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2023-11-08T08:17:25.200 INFO:teuthology.orchestra.run.smithi169.stdout:Creating new PostgreSQL cluster 12/main ... 2023-11-08T08:17:25.228 INFO:teuthology.orchestra.run.smithi169.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2023-11-08T08:17:25.267 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-11-08T08:17:25.319 INFO:teuthology.orchestra.run.smithi169.stdout:The files belonging to this database system will be owned by user "postgres". 2023-11-08T08:17:25.320 INFO:teuthology.orchestra.run.smithi169.stdout:This user must also own the server process. 2023-11-08T08:17:25.320 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:25.320 INFO:teuthology.orchestra.run.smithi169.stdout:The database cluster will be initialized with locale "en_US". 2023-11-08T08:17:25.320 INFO:teuthology.orchestra.run.smithi169.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-11-08T08:17:25.320 INFO:teuthology.orchestra.run.smithi169.stdout:The default text search configuration will be set to "english". 2023-11-08T08:17:25.321 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:25.321 INFO:teuthology.orchestra.run.smithi169.stdout:Data page checksums are disabled. 2023-11-08T08:17:25.321 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:25.321 INFO:teuthology.orchestra.run.smithi169.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2023-11-08T08:17:25.323 INFO:teuthology.orchestra.run.smithi169.stdout:creating subdirectories ... ok 2023-11-08T08:17:25.323 INFO:teuthology.orchestra.run.smithi169.stdout:selecting dynamic shared memory implementation ... posix 2023-11-08T08:17:25.359 INFO:teuthology.orchestra.run.smithi169.stdout:selecting default max_connections ... 100 2023-11-08T08:17:25.409 INFO:teuthology.orchestra.run.smithi169.stdout:selecting default shared_buffers ... 128MB 2023-11-08T08:17:25.433 INFO:teuthology.orchestra.run.smithi169.stdout:selecting default time zone ... Etc/UTC 2023-11-08T08:17:25.434 INFO:teuthology.orchestra.run.smithi169.stdout:creating configuration files ... ok 2023-11-08T08:17:25.621 INFO:teuthology.orchestra.run.smithi169.stdout:running bootstrap script ... ok 2023-11-08T08:17:26.234 INFO:teuthology.orchestra.run.smithi169.stdout:performing post-bootstrap initialization ... ok 2023-11-08T08:17:26.769 INFO:teuthology.orchestra.run.smithi169.stdout:syncing data to disk ... ok 2023-11-08T08:17:26.769 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:26.769 INFO:teuthology.orchestra.run.smithi169.stdout:Success. You can now start the database server using: 2023-11-08T08:17:26.769 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:26.770 INFO:teuthology.orchestra.run.smithi169.stdout: pg_ctlcluster 12 main start 2023-11-08T08:17:26.770 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:27.341 INFO:teuthology.orchestra.run.smithi169.stdout:Ver Cluster Port Status Owner Data directory Log file 2023-11-08T08:17:27.341 INFO:teuthology.orchestra.run.smithi169.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2023-11-08T08:17:27.559 INFO:teuthology.orchestra.run.smithi169.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-11-08T08:17:29.068 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2023-11-08T08:17:29.194 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-11-08T08:17:29.321 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-11-08T08:17:29.514 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2023-11-08T08:17:29.640 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2023-11-08T08:17:29.766 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-11-08T08:17:31.015 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:31.015 INFO:teuthology.orchestra.run.smithi040.stdout:The following network interfaces were found in /etc/network/interfaces 2023-11-08T08:17:31.015 INFO:teuthology.orchestra.run.smithi040.stdout:which means they are currently configured by ifupdown: 2023-11-08T08:17:31.015 INFO:teuthology.orchestra.run.smithi040.stdout:- ens1f0 2023-11-08T08:17:31.015 INFO:teuthology.orchestra.run.smithi040.stdout:If you want to manage those interfaces with NetworkManager instead 2023-11-08T08:17:31.016 INFO:teuthology.orchestra.run.smithi040.stdout:remove their configuration from /etc/network/interfaces. 2023-11-08T08:17:31.016 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:31.145 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2023-11-08T08:17:31.264 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-11-08T08:17:31.296 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-11-08T08:17:31.422 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-11-08T08:17:31.632 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2023-11-08T08:17:31.758 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2023-11-08T08:17:31.821 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-11-08T08:17:31.884 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-11-08T08:17:32.853 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-11-08T08:17:33.058 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:33.058 INFO:teuthology.orchestra.run.smithi169.stdout:The following network interfaces were found in /etc/network/interfaces 2023-11-08T08:17:33.058 INFO:teuthology.orchestra.run.smithi169.stdout:which means they are currently configured by ifupdown: 2023-11-08T08:17:33.059 INFO:teuthology.orchestra.run.smithi169.stdout:- enp3s0f1 2023-11-08T08:17:33.059 INFO:teuthology.orchestra.run.smithi169.stdout:If you want to manage those interfaces with NetworkManager instead 2023-11-08T08:17:33.059 INFO:teuthology.orchestra.run.smithi169.stdout:remove their configuration from /etc/network/interfaces. 2023-11-08T08:17:33.059 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:33.284 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-11-08T08:17:33.810 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-11-08T08:17:33.812 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-11-08T08:17:33.945 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-11-08T08:17:34.113 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-11-08T08:17:34.239 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2023-11-08T08:17:34.365 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2023-11-08T08:17:34.524 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-11-08T08:17:34.650 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-11-08T08:17:34.763 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-11-08T08:17:34.926 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-11-08T08:17:35.252 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-11-08T08:17:35.253 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-11-08T08:17:35.737 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-11-08T08:17:35.896 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-11-08T08:17:36.064 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-11-08T08:17:36.232 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2023-11-08T08:17:36.278 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-11-08T08:17:36.358 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2023-11-08T08:17:36.397 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-11-08T08:17:36.484 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-11-08T08:17:36.531 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-11-08T08:17:36.643 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-11-08T08:17:36.733 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-11-08T08:17:36.769 INFO:teuthology.orchestra.run.smithi169.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-11-08T08:17:37.090 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-11-08T08:17:37.091 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-11-08T08:17:37.276 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-11-08T08:17:38.005 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-11-08T08:17:38.148 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-11-08T08:17:38.299 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-11-08T08:17:38.458 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-11-08T08:17:39.026 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-11-08T08:17:51.329 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-11-08T08:17:51.437 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-11-08T08:17:52.687 INFO:teuthology.orchestra.run.smithi040.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-08T08:17:52.694 DEBUG:teuthology.parallel:result is None 2023-11-08T08:17:52.747 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-11-08T08:17:52.863 INFO:teuthology.orchestra.run.smithi169.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-11-08T08:17:54.132 INFO:teuthology.orchestra.run.smithi169.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-08T08:17:54.136 DEBUG:teuthology.parallel:result is None 2023-11-08T08:17:54.137 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=b820560f30d310ab2570761b9a294523225d6998 2023-11-08T08:17:54.358 DEBUG:teuthology.orchestra.run.smithi040:> dpkg-query -W -f '${Version}' ceph 2023-11-08T08:17:54.396 INFO:teuthology.orchestra.run.smithi040.stdout:18.0.0-2698-gb820560f-1focal 2023-11-08T08:17:54.396 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2698-gb820560f-1focal 2023-11-08T08:17:54.397 INFO:teuthology.task.install:The correct ceph version 18.0.0-2698-gb820560f-1focal is installed. 2023-11-08T08:17:54.397 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=b820560f30d310ab2570761b9a294523225d6998 2023-11-08T08:17:54.622 DEBUG:teuthology.orchestra.run.smithi169:> dpkg-query -W -f '${Version}' ceph 2023-11-08T08:17:54.660 INFO:teuthology.orchestra.run.smithi169.stdout:18.0.0-2698-gb820560f-1focal 2023-11-08T08:17:54.661 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2698-gb820560f-1focal 2023-11-08T08:17:54.661 INFO:teuthology.task.install:The correct ceph version 18.0.0-2698-gb820560f-1focal is installed. 2023-11-08T08:17:54.662 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-08T08:17:54.662 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:17:54.662 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-08T08:17:54.699 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:17:54.700 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-08T08:17:54.742 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-08T08:17:54.743 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:17:54.743 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/daemon-helper 2023-11-08T08:17:54.778 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-08T08:17:54.857 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:17:54.858 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/daemon-helper 2023-11-08T08:17:54.898 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-08T08:17:54.971 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-08T08:17:54.971 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:17:54.972 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-08T08:17:55.006 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-08T08:17:55.086 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:17:55.086 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-08T08:17:55.122 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-08T08:17:55.195 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-08T08:17:55.196 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:17:55.196 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/stdin-killer 2023-11-08T08:17:55.232 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-08T08:17:55.312 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:17:55.313 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/stdin-killer 2023-11-08T08:17:55.353 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-08T08:17:55.427 INFO:teuthology.run_tasks:Running task ceph... 2023-11-08T08:17:55.492 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-08T08:17:55.493 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 777 /var/log/ceph 2023-11-08T08:17:55.500 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 777 /var/log/ceph 2023-11-08T08:17:55.534 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-08T08:17:55.535 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-08T08:17:55.582 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-08T08:17:55.615 INFO:tasks.ceph:Creating extra log directories... 2023-11-08T08:17:55.615 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-08T08:17:55.662 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-08T08:17:55.707 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-08T08:17:55.707 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-08T08:17:55.708 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-08T08:17:55.708 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:17:55.709 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-08T08:17:55.754 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-08T08:17:55.887 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:17:55.887 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-08T08:17:55.905 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-08T08:17:56.042 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-08T08:17:56.043 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-08T08:17:56.043 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2023-11-08_07:51:45-fs-wip-neesingh-testing-54557-distro-default-smithi/7451825', 'branch': 'wip-neesingh-testing-54557', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7451825', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2023-11-08_07:51:45-fs-wip-neesingh-testing-54557-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-54557'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'b820560f30d310ab2570761b9a294523225d6998'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b820560f30d310ab2570761b9a294523225d6998'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-neesingh-testing-54557', 'sha1': 'b820560f30d310ab2570761b9a294523225d6998'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 10, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5135, 'sha1': 'b820560f30d310ab2570761b9a294523225d6998', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-neesingh-testing-54557', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_b820560f30d310ab2570761b9a294523225d6998/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'b820560f30d310ab2570761b9a294523225d6998', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi040.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi169.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b820560f30d310ab2570761b9a294523225d6998'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '6899cd26fceddb2fec83dc1a1349394b28c8998e', 'timestamp': '2023-11-08_07:51:45', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3590714'} 2023-11-08T08:17:56.044 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-08T08:17:56.051 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-08T08:17:56.071 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/run/ceph 2023-11-08T08:17:56.118 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m0777 -- /var/run/ceph 2023-11-08T08:17:56.155 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:17:56.155 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/scratch_devs of=/dev/stdout 2023-11-08T08:17:56.213 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-08T08:17:56.214 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_1 2023-11-08T08:17:56.276 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-08T08:17:56.277 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T08:17:56.277 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 496 Links: 1 2023-11-08T08:17:56.277 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T08:17:56.277 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-11-08 08:12:08.399785762 +0000 2023-11-08T08:17:56.278 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-11-08 08:12:07.923763704 +0000 2023-11-08T08:17:56.278 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-11-08 08:12:07.923763704 +0000 2023-11-08T08:17:56.278 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2023-11-08T08:17:56.278 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-08T08:17:56.353 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2023-11-08T08:17:56.353 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2023-11-08T08:17:56.353 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000701823 s, 730 kB/s 2023-11-08T08:17:56.355 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-08T08:17:56.423 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_2 2023-11-08T08:17:56.490 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-08T08:17:56.490 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T08:17:56.491 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 500 Links: 1 2023-11-08T08:17:56.491 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T08:17:56.491 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-11-08 08:12:09.167820760 +0000 2023-11-08T08:17:56.491 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-11-08 08:12:08.679798639 +0000 2023-11-08T08:17:56.491 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-11-08 08:12:08.679798639 +0000 2023-11-08T08:17:56.492 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2023-11-08T08:17:56.492 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-08T08:17:56.574 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2023-11-08T08:17:56.574 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2023-11-08T08:17:56.574 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.00072096 s, 710 kB/s 2023-11-08T08:17:56.575 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-08T08:17:56.643 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_3 2023-11-08T08:17:56.710 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-08T08:17:56.710 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T08:17:56.710 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 508 Links: 1 2023-11-08T08:17:56.711 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T08:17:56.711 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-11-08 08:12:09.983856655 +0000 2023-11-08T08:17:56.711 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-11-08 08:12:09.503835540 +0000 2023-11-08T08:17:56.711 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-11-08 08:12:09.503835540 +0000 2023-11-08T08:17:56.711 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2023-11-08T08:17:56.712 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-08T08:17:56.794 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2023-11-08T08:17:56.794 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2023-11-08T08:17:56.795 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.00065554 s, 781 kB/s 2023-11-08T08:17:56.797 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-08T08:17:56.863 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_4 2023-11-08T08:17:56.933 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-08T08:17:56.933 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T08:17:56.933 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 513 Links: 1 2023-11-08T08:17:56.934 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T08:17:56.934 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-11-08 08:12:10.759889284 +0000 2023-11-08T08:17:56.934 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-11-08 08:12:10.275868962 +0000 2023-11-08T08:17:56.934 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-11-08 08:12:10.275868962 +0000 2023-11-08T08:17:56.934 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2023-11-08T08:17:56.935 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-08T08:17:57.014 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2023-11-08T08:17:57.014 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2023-11-08T08:17:57.014 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000713446 s, 718 kB/s 2023-11-08T08:17:57.017 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-08T08:17:57.089 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-08T08:17:57.090 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:17:57.090 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/scratch_devs of=/dev/stdout 2023-11-08T08:17:57.107 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-08T08:17:57.108 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_1 2023-11-08T08:17:57.176 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-08T08:17:57.177 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T08:17:57.177 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 494 Links: 1 2023-11-08T08:17:57.177 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T08:17:57.177 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2023-11-08 08:12:10.240107788 +0000 2023-11-08T08:17:57.177 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2023-11-08 08:12:09.756100075 +0000 2023-11-08T08:17:57.178 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2023-11-08 08:12:09.756100075 +0000 2023-11-08T08:17:57.178 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: - 2023-11-08T08:17:57.178 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-08T08:17:57.249 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2023-11-08T08:17:57.250 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2023-11-08T08:17:57.250 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000525891 s, 974 kB/s 2023-11-08T08:17:57.251 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-08T08:17:57.318 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_2 2023-11-08T08:17:57.378 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-08T08:17:57.379 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T08:17:57.379 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 500 Links: 1 2023-11-08T08:17:57.379 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T08:17:57.379 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2023-11-08 08:12:11.032120355 +0000 2023-11-08T08:17:57.379 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2023-11-08 08:12:10.560112889 +0000 2023-11-08T08:17:57.380 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2023-11-08 08:12:10.560112889 +0000 2023-11-08T08:17:57.380 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: - 2023-11-08T08:17:57.380 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-08T08:17:57.462 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2023-11-08T08:17:57.462 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2023-11-08T08:17:57.463 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000723488 s, 708 kB/s 2023-11-08T08:17:57.464 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-08T08:17:57.536 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_3 2023-11-08T08:17:57.598 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-08T08:17:57.598 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T08:17:57.599 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 508 Links: 1 2023-11-08T08:17:57.599 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T08:17:57.599 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2023-11-08 08:12:11.784130837 +0000 2023-11-08T08:17:57.599 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2023-11-08 08:12:11.320124369 +0000 2023-11-08T08:17:57.600 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2023-11-08 08:12:11.320124369 +0000 2023-11-08T08:17:57.600 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: - 2023-11-08T08:17:57.600 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-08T08:17:57.678 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2023-11-08T08:17:57.678 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2023-11-08T08:17:57.678 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000695912 s, 736 kB/s 2023-11-08T08:17:57.680 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-08T08:17:57.752 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_4 2023-11-08T08:17:57.816 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-08T08:17:57.816 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T08:17:57.817 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 513 Links: 1 2023-11-08T08:17:57.817 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T08:17:57.817 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2023-11-08 08:12:12.548140399 +0000 2023-11-08T08:17:57.817 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2023-11-08 08:12:12.080134810 +0000 2023-11-08T08:17:57.817 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2023-11-08 08:12:12.080134810 +0000 2023-11-08T08:17:57.818 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: - 2023-11-08T08:17:57.818 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-08T08:17:57.891 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2023-11-08T08:17:57.891 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2023-11-08T08:17:57.892 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000699566 s, 732 kB/s 2023-11-08T08:17:57.895 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-08T08:17:57.964 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-08T08:17:57.965 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi169.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-08T08:17:57.965 INFO:tasks.ceph:Generating config... 2023-11-08T08:17:57.967 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-08T08:17:57.967 INFO:tasks.ceph:[client] debug client = 20 2023-11-08T08:17:57.967 INFO:tasks.ceph:[client] debug ms = 1 2023-11-08T08:17:57.968 INFO:tasks.ceph:[client] pid file = 2023-11-08T08:17:57.968 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-08T08:17:57.968 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-08T08:17:57.968 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-08T08:17:57.969 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-08T08:17:57.969 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-08T08:17:57.969 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-08T08:17:57.969 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-08T08:17:57.970 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-08T08:17:57.970 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-08T08:17:57.970 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-08T08:17:57.971 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-08T08:17:57.971 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-08T08:17:57.971 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-08T08:17:57.972 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-08T08:17:57.972 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-08T08:17:57.972 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-08T08:17:57.972 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-08T08:17:57.973 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-08T08:17:57.973 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-08T08:17:57.973 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-08T08:17:57.973 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-08T08:17:57.974 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-11-08T08:17:57.974 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-11-08T08:17:57.974 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-08T08:17:57.975 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-08T08:17:57.975 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-11-08T08:17:57.975 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-08T08:17:57.975 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-08T08:17:57.976 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-08T08:17:57.976 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-08T08:17:57.976 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-08T08:17:57.977 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-08T08:17:57.977 INFO:tasks.ceph:Setting up mon.a... 2023-11-08T08:17:57.977 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-08T08:17:58.054 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.keyring 2023-11-08T08:17:58.068 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-08T08:17:58.146 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-08T08:17:58.177 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.40'), ('mon.b', '172.21.15.169'), ('mon.c', '[v2:172.21.15.169:3301,v1:172.21.15.169:6790]')] 2023-11-08T08:17:58.178 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.40,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-08T08:17:58.179 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:17:58.179 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-08T08:17:58.240 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.40 --add b 172.21.15.169 --addv c '[v2:172.21.15.169:3301,v1:172.21.15.169:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-08T08:17:58.362 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-08T08:17:58.363 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: generated fsid 54ed45f4-d5b0-41c3-8c3b-b5a18b844f25 2023-11-08T08:17:58.363 INFO:teuthology.orchestra.run.smithi040.stdout:setting min_mon_release = pacific 2023-11-08T08:17:58.363 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 0 2023-11-08T08:17:58.363 INFO:teuthology.orchestra.run.smithi040.stdout:fsid 54ed45f4-d5b0-41c3-8c3b-b5a18b844f25 2023-11-08T08:17:58.364 INFO:teuthology.orchestra.run.smithi040.stdout:last_changed 2023-11-08T08:17:58.362436+0000 2023-11-08T08:17:58.364 INFO:teuthology.orchestra.run.smithi040.stdout:created 2023-11-08T08:17:58.362436+0000 2023-11-08T08:17:58.364 INFO:teuthology.orchestra.run.smithi040.stdout:min_mon_release 16 (pacific) 2023-11-08T08:17:58.364 INFO:teuthology.orchestra.run.smithi040.stdout:election_strategy: 1 2023-11-08T08:17:58.364 INFO:teuthology.orchestra.run.smithi040.stdout:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-11-08T08:17:58.365 INFO:teuthology.orchestra.run.smithi040.stdout:1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.b 2023-11-08T08:17:58.365 INFO:teuthology.orchestra.run.smithi040.stdout:2: [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] mon.c 2023-11-08T08:17:58.365 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-08T08:17:58.366 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-08T08:17:58.383 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 54ed45f4-d5b0-41c3-8c3b-b5a18b844f25... 2023-11-08T08:17:58.384 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T08:17:58.434 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T08:17:58.497 INFO:teuthology.orchestra.run.smithi040.stdout:[global] 2023-11-08T08:17:58.497 INFO:teuthology.orchestra.run.smithi040.stdout: chdir = "" 2023-11-08T08:17:58.497 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T08:17:58.498 INFO:teuthology.orchestra.run.smithi040.stdout: auth supported = cephx 2023-11-08T08:17:58.498 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.498 INFO:teuthology.orchestra.run.smithi040.stdout: filestore xattr use omap = true 2023-11-08T08:17:58.499 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.499 INFO:teuthology.orchestra.run.smithi040.stdout: mon clock drift allowed = 1.000 2023-11-08T08:17:58.499 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.499 INFO:teuthology.orchestra.run.smithi040.stdout: osd crush chooseleaf type = 0 2023-11-08T08:17:58.500 INFO:teuthology.orchestra.run.smithi040.stdout: auth debug = true 2023-11-08T08:17:58.500 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.500 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on old message = true 2023-11-08T08:17:58.500 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on bug = true 2023-11-08T08:17:58.501 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.501 INFO:teuthology.orchestra.run.smithi040.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T08:17:58.501 INFO:teuthology.orchestra.run.smithi040.stdout: mon pg warn max object skew = 0 2023-11-08T08:17:58.501 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.502 INFO:teuthology.orchestra.run.smithi040.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T08:17:58.502 INFO:teuthology.orchestra.run.smithi040.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T08:17:58.502 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.502 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default size = 2 2023-11-08T08:17:58.502 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.503 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow primary affinity = true 2023-11-08T08:17:58.503 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow pg remap = true 2023-11-08T08:17:58.503 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on legacy crush tunables = false 2023-11-08T08:17:58.503 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on crush straw calc version zero = false 2023-11-08T08:17:58.503 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on no sortbitwise = false 2023-11-08T08:17:58.504 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on osd down out interval zero = false 2023-11-08T08:17:58.504 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on too few osds = false 2023-11-08T08:17:58.504 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T08:17:58.504 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T08:17:58.504 INFO:teuthology.orchestra.run.smithi040.stdout: mon_allow_pool_size_one = true 2023-11-08T08:17:58.505 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.505 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T08:17:58.505 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.505 INFO:teuthology.orchestra.run.smithi040.stdout: osd default data pool replay window = 5 2023-11-08T08:17:58.505 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.506 INFO:teuthology.orchestra.run.smithi040.stdout: mon allow pool delete = true 2023-11-08T08:17:58.506 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.506 INFO:teuthology.orchestra.run.smithi040.stdout: mon cluster log file level = debug 2023-11-08T08:17:58.506 INFO:teuthology.orchestra.run.smithi040.stdout: debug asserts on shutdown = true 2023-11-08T08:17:58.507 INFO:teuthology.orchestra.run.smithi040.stdout: mon health detail to clog = false 2023-11-08T08:17:58.507 INFO:teuthology.orchestra.run.smithi040.stdout: mon host = "172.21.15.40,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2023-11-08T08:17:58.507 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on skipped message = False 2023-11-08T08:17:58.507 INFO:teuthology.orchestra.run.smithi040.stdout: fsid = 54ed45f4-d5b0-41c3-8c3b-b5a18b844f25 2023-11-08T08:17:58.507 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.508 INFO:teuthology.orchestra.run.smithi040.stdout:[osd] 2023-11-08T08:17:58.508 INFO:teuthology.orchestra.run.smithi040.stdout: osd journal size = 100 2023-11-08T08:17:58.508 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.508 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub load threshold = 5.0 2023-11-08T08:17:58.508 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub max interval = 600 2023-11-08T08:17:58.509 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.509 INFO:teuthology.orchestra.run.smithi040.stdout: osd recover clone overlap = true 2023-11-08T08:17:58.509 INFO:teuthology.orchestra.run.smithi040.stdout: osd recovery max chunk = 1048576 2023-11-08T08:17:58.509 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.509 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug shutdown = true 2023-11-08T08:17:58.510 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug op order = true 2023-11-08T08:17:58.510 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug verify stray on activate = true 2023-11-08T08:17:58.510 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.511 INFO:teuthology.orchestra.run.smithi040.stdout: osd open classes on start = true 2023-11-08T08:17:58.511 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug pg log writeout = true 2023-11-08T08:17:58.511 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.511 INFO:teuthology.orchestra.run.smithi040.stdout: osd deep scrub update digest min age = 30 2023-11-08T08:17:58.511 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.511 INFO:teuthology.orchestra.run.smithi040.stdout: osd map max advance = 10 2023-11-08T08:17:58.512 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.512 INFO:teuthology.orchestra.run.smithi040.stdout: journal zero on create = true 2023-11-08T08:17:58.512 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.512 INFO:teuthology.orchestra.run.smithi040.stdout: filestore ondisk finisher threads = 3 2023-11-08T08:17:58.513 INFO:teuthology.orchestra.run.smithi040.stdout: filestore apply finisher threads = 3 2023-11-08T08:17:58.513 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.513 INFO:teuthology.orchestra.run.smithi040.stdout: bdev debug aio = true 2023-11-08T08:17:58.514 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug misdirected ops = true 2023-11-08T08:17:58.514 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore block size = 96636764160 2023-11-08T08:17:58.514 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore fsck on mount = True 2023-11-08T08:17:58.514 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluefs = 20 2023-11-08T08:17:58.514 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluestore = 20 2023-11-08T08:17:58.515 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2023-11-08T08:17:58.515 INFO:teuthology.orchestra.run.smithi040.stdout: debug osd = 20 2023-11-08T08:17:58.515 INFO:teuthology.orchestra.run.smithi040.stdout: debug rocksdb = 10 2023-11-08T08:17:58.515 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T08:17:58.515 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd full ratio = 0.9 2023-11-08T08:17:58.516 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd nearfull ratio = 0.8 2023-11-08T08:17:58.516 INFO:teuthology.orchestra.run.smithi040.stdout: osd failsafe full ratio = 0.95 2023-11-08T08:17:58.516 INFO:teuthology.orchestra.run.smithi040.stdout: osd objectstore = bluestore 2023-11-08T08:17:58.516 INFO:teuthology.orchestra.run.smithi040.stdout: osd op complaint time = 180 2023-11-08T08:17:58.516 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.517 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr] 2023-11-08T08:17:58.517 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2023-11-08T08:17:58.517 INFO:teuthology.orchestra.run.smithi040.stdout: debug mgr = 20 2023-11-08T08:17:58.517 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2023-11-08T08:17:58.517 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2023-11-08T08:17:58.518 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2023-11-08T08:17:58.518 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2023-11-08T08:17:58.518 INFO:teuthology.orchestra.run.smithi040.stdout: mgr/telemetry/nag = false 2023-11-08T08:17:58.518 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.518 INFO:teuthology.orchestra.run.smithi040.stdout:[mon] 2023-11-08T08:17:58.519 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2023-11-08T08:17:58.519 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2023-11-08T08:17:58.519 INFO:teuthology.orchestra.run.smithi040.stdout: debug paxos = 20 2023-11-08T08:17:58.519 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2023-11-08T08:17:58.520 INFO:teuthology.orchestra.run.smithi040.stdout: mon data avail warn = 5 2023-11-08T08:17:58.520 INFO:teuthology.orchestra.run.smithi040.stdout: mon mgr mkfs grace = 240 2023-11-08T08:17:58.520 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2023-11-08T08:17:58.520 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd reporter subtree level = osd 2023-11-08T08:17:58.520 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd prime pg temp = true 2023-11-08T08:17:58.521 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2023-11-08T08:17:58.521 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.521 INFO:teuthology.orchestra.run.smithi040.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T08:17:58.521 INFO:teuthology.orchestra.run.smithi040.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T08:17:58.522 INFO:teuthology.orchestra.run.smithi040.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T08:17:58.522 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.522 INFO:teuthology.orchestra.run.smithi040.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T08:17:58.522 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T08:17:58.522 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T08:17:58.523 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.523 INFO:teuthology.orchestra.run.smithi040.stdout: # 1m isn't quite enough 2023-11-08T08:17:58.523 INFO:teuthology.orchestra.run.smithi040.stdout: mon_down_mkfs_grace = 2m 2023-11-08T08:17:58.523 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.524 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_filestore_osds = false 2023-11-08T08:17:58.524 INFO:teuthology.orchestra.run.smithi040.stdout: mon op complaint time = 120 2023-11-08T08:17:58.524 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:17:58.524 INFO:teuthology.orchestra.run.smithi040.stdout:[client] 2023-11-08T08:17:58.524 INFO:teuthology.orchestra.run.smithi040.stdout: rgw cache enabled = true 2023-11-08T08:17:58.525 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable ops log = true 2023-11-08T08:17:58.525 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable usage log = true 2023-11-08T08:17:58.525 INFO:teuthology.orchestra.run.smithi040.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T08:17:58.525 INFO:teuthology.orchestra.run.smithi040.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T08:17:58.526 INFO:teuthology.orchestra.run.smithi040.stdout: client mount timeout = 600 2023-11-08T08:17:58.526 INFO:teuthology.orchestra.run.smithi040.stdout: debug client = 20 2023-11-08T08:17:58.526 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2023-11-08T08:17:58.526 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = "" 2023-11-08T08:17:58.526 INFO:teuthology.orchestra.run.smithi040.stdout: rados mon op timeout = 900 2023-11-08T08:17:58.527 INFO:teuthology.orchestra.run.smithi040.stdout: rados osd op timeout = 900 2023-11-08T08:17:58.527 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.a] 2023-11-08T08:17:58.527 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.b] 2023-11-08T08:17:58.527 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.c] 2023-11-08T08:17:58.528 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a] 2023-11-08T08:17:58.528 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.c] 2023-11-08T08:17:58.528 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b] 2023-11-08T08:17:58.528 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.d] 2023-11-08T08:17:58.529 INFO:teuthology.orchestra.run.smithi040.stdout:[mds] 2023-11-08T08:17:58.529 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds = 20 2023-11-08T08:17:58.529 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds balancer = 20 2023-11-08T08:17:58.529 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2023-11-08T08:17:58.530 INFO:teuthology.orchestra.run.smithi040.stdout: mds debug frag = True 2023-11-08T08:17:58.530 INFO:teuthology.orchestra.run.smithi040.stdout: mds debug scatterstat = True 2023-11-08T08:17:58.530 INFO:teuthology.orchestra.run.smithi040.stdout: mds op complaint time = 180 2023-11-08T08:17:58.530 INFO:teuthology.orchestra.run.smithi040.stdout: mds verify scatter = True 2023-11-08T08:17:58.531 INFO:teuthology.orchestra.run.smithi040.stdout: osd op complaint time = 180 2023-11-08T08:17:58.531 INFO:teuthology.orchestra.run.smithi040.stdout: rados mon op timeout = 900 2023-11-08T08:17:58.531 INFO:teuthology.orchestra.run.smithi040.stdout: rados osd op timeout = 900 2023-11-08T08:17:58.532 INFO:teuthology.orchestra.run.smithi169.stdout:[global] 2023-11-08T08:17:58.532 INFO:teuthology.orchestra.run.smithi169.stdout: chdir = "" 2023-11-08T08:17:58.533 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T08:17:58.533 INFO:teuthology.orchestra.run.smithi169.stdout: auth supported = cephx 2023-11-08T08:17:58.533 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.534 INFO:teuthology.orchestra.run.smithi169.stdout: filestore xattr use omap = true 2023-11-08T08:17:58.534 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.534 INFO:teuthology.orchestra.run.smithi169.stdout: mon clock drift allowed = 1.000 2023-11-08T08:17:58.534 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.535 INFO:teuthology.orchestra.run.smithi169.stdout: osd crush chooseleaf type = 0 2023-11-08T08:17:58.535 INFO:teuthology.orchestra.run.smithi169.stdout: auth debug = true 2023-11-08T08:17:58.535 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.535 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on old message = true 2023-11-08T08:17:58.535 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on bug = true 2023-11-08T08:17:58.536 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.536 INFO:teuthology.orchestra.run.smithi169.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T08:17:58.536 INFO:teuthology.orchestra.run.smithi169.stdout: mon pg warn max object skew = 0 2023-11-08T08:17:58.536 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.536 INFO:teuthology.orchestra.run.smithi169.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T08:17:58.537 INFO:teuthology.orchestra.run.smithi169.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T08:17:58.537 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.537 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default size = 2 2023-11-08T08:17:58.537 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.537 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow primary affinity = true 2023-11-08T08:17:58.538 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow pg remap = true 2023-11-08T08:17:58.538 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on legacy crush tunables = false 2023-11-08T08:17:58.538 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on crush straw calc version zero = false 2023-11-08T08:17:58.538 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on no sortbitwise = false 2023-11-08T08:17:58.538 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on osd down out interval zero = false 2023-11-08T08:17:58.539 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on too few osds = false 2023-11-08T08:17:58.539 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T08:17:58.539 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T08:17:58.539 INFO:teuthology.orchestra.run.smithi169.stdout: mon_allow_pool_size_one = true 2023-11-08T08:17:58.539 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.540 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T08:17:58.540 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.540 INFO:teuthology.orchestra.run.smithi169.stdout: osd default data pool replay window = 5 2023-11-08T08:17:58.540 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.540 INFO:teuthology.orchestra.run.smithi169.stdout: mon allow pool delete = true 2023-11-08T08:17:58.540 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.541 INFO:teuthology.orchestra.run.smithi169.stdout: mon cluster log file level = debug 2023-11-08T08:17:58.541 INFO:teuthology.orchestra.run.smithi169.stdout: debug asserts on shutdown = true 2023-11-08T08:17:58.541 INFO:teuthology.orchestra.run.smithi169.stdout: mon health detail to clog = false 2023-11-08T08:17:58.541 INFO:teuthology.orchestra.run.smithi169.stdout: mon host = "172.21.15.40,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2023-11-08T08:17:58.541 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on skipped message = False 2023-11-08T08:17:58.542 INFO:teuthology.orchestra.run.smithi169.stdout: fsid = 54ed45f4-d5b0-41c3-8c3b-b5a18b844f25 2023-11-08T08:17:58.542 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.542 INFO:teuthology.orchestra.run.smithi169.stdout:[osd] 2023-11-08T08:17:58.542 INFO:teuthology.orchestra.run.smithi169.stdout: osd journal size = 100 2023-11-08T08:17:58.542 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.543 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub load threshold = 5.0 2023-11-08T08:17:58.543 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub max interval = 600 2023-11-08T08:17:58.543 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.543 INFO:teuthology.orchestra.run.smithi169.stdout: osd recover clone overlap = true 2023-11-08T08:17:58.543 INFO:teuthology.orchestra.run.smithi169.stdout: osd recovery max chunk = 1048576 2023-11-08T08:17:58.544 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.544 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug shutdown = true 2023-11-08T08:17:58.544 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug op order = true 2023-11-08T08:17:58.544 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug verify stray on activate = true 2023-11-08T08:17:58.544 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.545 INFO:teuthology.orchestra.run.smithi169.stdout: osd open classes on start = true 2023-11-08T08:17:58.545 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug pg log writeout = true 2023-11-08T08:17:58.545 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.545 INFO:teuthology.orchestra.run.smithi169.stdout: osd deep scrub update digest min age = 30 2023-11-08T08:17:58.545 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.546 INFO:teuthology.orchestra.run.smithi169.stdout: osd map max advance = 10 2023-11-08T08:17:58.546 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.546 INFO:teuthology.orchestra.run.smithi169.stdout: journal zero on create = true 2023-11-08T08:17:58.546 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.546 INFO:teuthology.orchestra.run.smithi169.stdout: filestore ondisk finisher threads = 3 2023-11-08T08:17:58.547 INFO:teuthology.orchestra.run.smithi169.stdout: filestore apply finisher threads = 3 2023-11-08T08:17:58.547 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.547 INFO:teuthology.orchestra.run.smithi169.stdout: bdev debug aio = true 2023-11-08T08:17:58.547 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug misdirected ops = true 2023-11-08T08:17:58.547 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore block size = 96636764160 2023-11-08T08:17:58.548 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore fsck on mount = True 2023-11-08T08:17:58.548 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluefs = 20 2023-11-08T08:17:58.548 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluestore = 20 2023-11-08T08:17:58.548 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-11-08T08:17:58.548 INFO:teuthology.orchestra.run.smithi169.stdout: debug osd = 20 2023-11-08T08:17:58.549 INFO:teuthology.orchestra.run.smithi169.stdout: debug rocksdb = 10 2023-11-08T08:17:58.549 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T08:17:58.549 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd full ratio = 0.9 2023-11-08T08:17:58.549 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd nearfull ratio = 0.8 2023-11-08T08:17:58.549 INFO:teuthology.orchestra.run.smithi169.stdout: osd failsafe full ratio = 0.95 2023-11-08T08:17:58.550 INFO:teuthology.orchestra.run.smithi169.stdout: osd objectstore = bluestore 2023-11-08T08:17:58.550 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2023-11-08T08:17:58.550 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.550 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr] 2023-11-08T08:17:58.550 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-11-08T08:17:58.551 INFO:teuthology.orchestra.run.smithi169.stdout: debug mgr = 20 2023-11-08T08:17:58.551 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2023-11-08T08:17:58.551 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2023-11-08T08:17:58.551 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2023-11-08T08:17:58.551 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2023-11-08T08:17:58.552 INFO:teuthology.orchestra.run.smithi169.stdout: mgr/telemetry/nag = false 2023-11-08T08:17:58.552 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.552 INFO:teuthology.orchestra.run.smithi169.stdout:[mon] 2023-11-08T08:17:58.552 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-11-08T08:17:58.552 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2023-11-08T08:17:58.553 INFO:teuthology.orchestra.run.smithi169.stdout: debug paxos = 20 2023-11-08T08:17:58.553 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2023-11-08T08:17:58.553 INFO:teuthology.orchestra.run.smithi169.stdout: mon data avail warn = 5 2023-11-08T08:17:58.553 INFO:teuthology.orchestra.run.smithi169.stdout: mon mgr mkfs grace = 240 2023-11-08T08:17:58.553 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2023-11-08T08:17:58.554 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd reporter subtree level = osd 2023-11-08T08:17:58.554 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd prime pg temp = true 2023-11-08T08:17:58.554 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2023-11-08T08:17:58.554 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.554 INFO:teuthology.orchestra.run.smithi169.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T08:17:58.555 INFO:teuthology.orchestra.run.smithi169.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T08:17:58.555 INFO:teuthology.orchestra.run.smithi169.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T08:17:58.555 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.555 INFO:teuthology.orchestra.run.smithi169.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T08:17:58.556 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T08:17:58.556 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T08:17:58.556 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.556 INFO:teuthology.orchestra.run.smithi169.stdout: # 1m isn't quite enough 2023-11-08T08:17:58.557 INFO:teuthology.orchestra.run.smithi169.stdout: mon_down_mkfs_grace = 2m 2023-11-08T08:17:58.557 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.557 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_filestore_osds = false 2023-11-08T08:17:58.557 INFO:teuthology.orchestra.run.smithi169.stdout: mon op complaint time = 120 2023-11-08T08:17:58.557 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-08T08:17:58.558 INFO:teuthology.orchestra.run.smithi169.stdout:[client] 2023-11-08T08:17:58.558 INFO:teuthology.orchestra.run.smithi169.stdout: rgw cache enabled = true 2023-11-08T08:17:58.558 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable ops log = true 2023-11-08T08:17:58.558 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable usage log = true 2023-11-08T08:17:58.559 INFO:teuthology.orchestra.run.smithi169.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T08:17:58.559 INFO:teuthology.orchestra.run.smithi169.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T08:17:58.559 INFO:teuthology.orchestra.run.smithi169.stdout: client mount timeout = 600 2023-11-08T08:17:58.559 INFO:teuthology.orchestra.run.smithi169.stdout: debug client = 20 2023-11-08T08:17:58.559 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-11-08T08:17:58.560 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = "" 2023-11-08T08:17:58.560 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2023-11-08T08:17:58.560 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2023-11-08T08:17:58.560 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.a] 2023-11-08T08:17:58.561 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.b] 2023-11-08T08:17:58.561 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.c] 2023-11-08T08:17:58.561 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2023-11-08T08:17:58.561 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2023-11-08T08:17:58.561 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2023-11-08T08:17:58.562 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2023-11-08T08:17:58.562 INFO:teuthology.orchestra.run.smithi169.stdout:[mds] 2023-11-08T08:17:58.562 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds = 20 2023-11-08T08:17:58.562 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds balancer = 20 2023-11-08T08:17:58.562 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-11-08T08:17:58.563 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug frag = True 2023-11-08T08:17:58.563 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug scatterstat = True 2023-11-08T08:17:58.563 INFO:teuthology.orchestra.run.smithi169.stdout: mds op complaint time = 180 2023-11-08T08:17:58.563 INFO:teuthology.orchestra.run.smithi169.stdout: mds verify scatter = True 2023-11-08T08:17:58.563 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2023-11-08T08:17:58.564 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2023-11-08T08:17:58.564 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2023-11-08T08:17:58.566 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-08T08:17:58.566 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-08T08:17:58.706 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-08T08:17:58.706 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:17:58.707 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-08T08:17:58.725 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:17:58.725 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-08T08:17:58.788 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi040.front.sepia.ceph.com 2023-11-08T08:17:58.788 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:17:58.788 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-08T08:17:58.789 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-08T08:17:58.894 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:17:58.894 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-08T08:17:58.952 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi169.front.sepia.ceph.com 2023-11-08T08:17:58.952 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:17:58.953 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-08T08:17:58.953 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-08T08:17:59.018 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:17:59.018 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-08T08:17:59.076 INFO:tasks.ceph:Setting up mon nodes... 2023-11-08T08:17:59.076 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-08T08:17:59.076 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-08T08:17:59.178 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-08T08:17:59.188 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-08T08:17:59.293 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-08T08:17:59.314 INFO:tasks.ceph:Setting up mds nodes... 2023-11-08T08:17:59.315 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-08T08:17:59.421 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-08T08:17:59.428 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-08T08:17:59.463 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-08T08:17:59.614 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-08T08:17:59.621 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-08T08:17:59.654 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-08T08:17:59.763 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-08T08:17:59.771 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-08T08:17:59.807 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-08T08:17:59.949 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-08T08:17:59.956 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-08T08:17:59.996 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-08T08:17:59.997 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-08T08:18:00.078 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-08T08:18:00.107 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-08T08:18:00.181 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-08T08:18:00.211 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-08T08:18:00.289 INFO:teuthology.orchestra.run.smithi169.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-08T08:18:00.315 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-08T08:18:00.385 INFO:teuthology.orchestra.run.smithi169.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-08T08:18:00.413 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-08T08:18:00.414 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi169.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-08T08:18:00.415 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-08T08:18:00.453 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-08T08:18:00.454 INFO:tasks.ceph:role: osd.0 2023-11-08T08:18:00.454 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com 2023-11-08T08:18:00.454 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-08T08:18:01.025 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T08:18:01.025 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T08:18:01.025 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T08:18:01.026 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 2023-11-08T08:18:01.026 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T08:18:01.026 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2023-11-08T08:18:01.026 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T08:18:01.026 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-08T08:18:01.027 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T08:18:01.027 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T08:18:01.031 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2023-11-08T08:18:01.031 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-08T08:18:01.163 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-08T08:18:01.185 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: /sbin/restorecon: command not found 2023-11-08T08:18:01.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T08:18:01.187 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-08T08:18:01.275 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-08T08:18:01.275 INFO:tasks.ceph:role: osd.1 2023-11-08T08:18:01.275 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com 2023-11-08T08:18:01.276 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-08T08:18:01.800 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T08:18:01.801 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T08:18:01.801 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T08:18:01.801 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 2023-11-08T08:18:01.801 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T08:18:01.802 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2023-11-08T08:18:01.802 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T08:18:01.802 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-08T08:18:01.802 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T08:18:01.803 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T08:18:01.804 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2023-11-08T08:18:01.804 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-08T08:18:01.880 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-08T08:18:01.902 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: /sbin/restorecon: command not found 2023-11-08T08:18:01.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T08:18:01.904 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-08T08:18:01.986 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-08T08:18:01.986 INFO:tasks.ceph:role: osd.2 2023-11-08T08:18:01.986 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com 2023-11-08T08:18:01.987 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-08T08:18:02.552 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T08:18:02.552 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T08:18:02.553 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T08:18:02.553 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 2023-11-08T08:18:02.553 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T08:18:02.553 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2023-11-08T08:18:02.553 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T08:18:02.554 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-08T08:18:02.554 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T08:18:02.554 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T08:18:02.555 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2023-11-08T08:18:02.555 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-08T08:18:02.667 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-08T08:18:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: /sbin/restorecon: command not found 2023-11-08T08:18:02.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T08:18:02.694 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-08T08:18:02.770 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-08T08:18:02.771 INFO:tasks.ceph:role: osd.3 2023-11-08T08:18:02.771 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com 2023-11-08T08:18:02.771 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-08T08:18:03.363 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T08:18:03.363 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T08:18:03.363 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T08:18:03.364 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 2023-11-08T08:18:03.364 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T08:18:03.364 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2023-11-08T08:18:03.364 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T08:18:03.364 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-08T08:18:03.365 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T08:18:03.365 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T08:18:03.368 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2023-11-08T08:18:03.368 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-08T08:18:03.540 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-08T08:18:03.568 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: /sbin/restorecon: command not found 2023-11-08T08:18:03.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T08:18:03.570 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T08:18:03.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:03.736+0000 7fc235cc7700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-08T08:18:03.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:03.740+0000 7fc235cc7700 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-08T08:18:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:03.740+0000 7fc235cc7700 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-08T08:18:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:03.740+0000 7fc235cc7700 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-08T08:18:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:03.740+0000 7fc235cc7700 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-08T08:18:03.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:03.744+0000 7fc235cc7700 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-08T08:18:06.811 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-08T08:18:06.844 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T08:18:06.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:06.968+0000 7fcbab268700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-08T08:18:06.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:06.968+0000 7fcbab268700 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-08T08:18:06.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:06.968+0000 7fcbab268700 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-08T08:18:06.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:06.968+0000 7fcbab268700 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-08T08:18:06.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:06.968+0000 7fcbab268700 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-08T08:18:06.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:06.972+0000 7fcbab268700 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-08T08:18:10.039 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-08T08:18:10.075 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T08:18:10.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:10.200+0000 7f2755cd5700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-08T08:18:10.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:10.204+0000 7f2755cd5700 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-08T08:18:10.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:10.204+0000 7f2755cd5700 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-08T08:18:10.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:10.204+0000 7f2755cd5700 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-08T08:18:10.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:10.204+0000 7f2755cd5700 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-08T08:18:10.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:10.204+0000 7f2755cd5700 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-08T08:18:13.288 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-08T08:18:13.320 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T08:18:13.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:13.448+0000 7fefe0ca8700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-08T08:18:13.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:13.448+0000 7fefe0ca8700 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-08T08:18:13.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:13.448+0000 7fefe0ca8700 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-08T08:18:13.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:13.448+0000 7fefe0ca8700 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-08T08:18:13.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:13.448+0000 7fefe0ca8700 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-08T08:18:13.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:13.452+0000 7fefe0ca8700 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-08T08:18:16.529 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-08T08:18:16.568 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-08T08:18:16.604 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-08T08:18:16.605 INFO:tasks.ceph:role: osd.4 2023-11-08T08:18:16.605 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi169.front.sepia.ceph.com 2023-11-08T08:18:16.605 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-08T08:18:17.208 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T08:18:17.209 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T08:18:17.209 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T08:18:17.209 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 2023-11-08T08:18:17.209 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T08:18:17.209 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2023-11-08T08:18:17.210 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T08:18:17.210 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-08T08:18:17.210 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T08:18:17.210 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T08:18:17.214 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2023-11-08T08:18:17.214 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-08T08:18:17.321 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-08T08:18:17.342 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: /sbin/restorecon: command not found 2023-11-08T08:18:17.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T08:18:17.344 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-08T08:18:17.428 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-08T08:18:17.428 INFO:tasks.ceph:role: osd.5 2023-11-08T08:18:17.429 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi169.front.sepia.ceph.com 2023-11-08T08:18:17.429 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-08T08:18:17.965 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T08:18:17.965 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T08:18:17.966 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T08:18:17.966 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 2023-11-08T08:18:17.966 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T08:18:17.966 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2023-11-08T08:18:17.966 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T08:18:17.967 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-08T08:18:17.967 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T08:18:17.967 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T08:18:17.972 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2023-11-08T08:18:17.972 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-08T08:18:18.035 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-08T08:18:18.056 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: /sbin/restorecon: command not found 2023-11-08T08:18:18.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T08:18:18.058 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-08T08:18:18.139 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-08T08:18:18.139 INFO:tasks.ceph:role: osd.6 2023-11-08T08:18:18.140 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi169.front.sepia.ceph.com 2023-11-08T08:18:18.140 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-08T08:18:18.714 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T08:18:18.714 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T08:18:18.714 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T08:18:18.715 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 2023-11-08T08:18:18.715 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T08:18:18.715 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2023-11-08T08:18:18.715 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T08:18:18.715 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-08T08:18:18.716 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T08:18:18.716 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T08:18:18.717 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2023-11-08T08:18:18.717 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-08T08:18:18.886 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-08T08:18:18.908 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: /sbin/restorecon: command not found 2023-11-08T08:18:18.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T08:18:18.910 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-08T08:18:18.991 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-08T08:18:18.991 INFO:tasks.ceph:role: osd.7 2023-11-08T08:18:18.992 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi169.front.sepia.ceph.com 2023-11-08T08:18:18.992 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-08T08:18:19.586 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T08:18:19.586 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T08:18:19.587 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T08:18:19.587 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 2023-11-08T08:18:19.587 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T08:18:19.587 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2023-11-08T08:18:19.587 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T08:18:19.588 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-08T08:18:19.588 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T08:18:19.588 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T08:18:19.591 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2023-11-08T08:18:19.591 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-08T08:18:19.697 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-08T08:18:19.720 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: /sbin/restorecon: command not found 2023-11-08T08:18:19.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T08:18:19.721 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T08:18:19.898 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:19.894+0000 7f0eff239700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-08T08:18:19.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:19.898+0000 7f0eff239700 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-08T08:18:19.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:19.898+0000 7f0eff239700 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-08T08:18:19.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:19.898+0000 7f0eff239700 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-08T08:18:19.903 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:19.898+0000 7f0eff239700 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-08T08:18:19.905 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:19.902+0000 7f0eff239700 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-08T08:18:22.997 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-08T08:18:23.030 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T08:18:23.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:23.158+0000 7fee5b632700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-08T08:18:23.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:23.158+0000 7fee5b632700 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-08T08:18:23.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:23.158+0000 7fee5b632700 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-08T08:18:23.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:23.158+0000 7fee5b632700 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-08T08:18:23.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:23.158+0000 7fee5b632700 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-08T08:18:23.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:23.162+0000 7fee5b632700 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-08T08:18:26.045 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:18:26.053 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:18:26.260 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-08T08:18:26.295 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T08:18:26.430 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:26.425+0000 7f145a462700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-08T08:18:26.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:26.429+0000 7f145a462700 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-08T08:18:26.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:26.429+0000 7f145a462700 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-08T08:18:26.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:26.429+0000 7f145a462700 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-08T08:18:26.432 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:26.429+0000 7f145a462700 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-08T08:18:26.434 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:26.429+0000 7f145a462700 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-08T08:18:29.501 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-08T08:18:29.541 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T08:18:29.669 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:29.665+0000 7efd7d2c3700 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-08T08:18:29.670 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:29.665+0000 7efd7d2c3700 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-08T08:18:29.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:29.665+0000 7efd7d2c3700 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-08T08:18:29.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:29.665+0000 7efd7d2c3700 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-08T08:18:29.671 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:29.665+0000 7efd7d2c3700 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-08T08:18:29.673 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:29.669+0000 7efd7d2c3700 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-08T08:18:32.731 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-08T08:18:32.766 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-08T08:18:32.766 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:18:32.767 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-08T08:18:32.802 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:18:32.803 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-08T08:18:32.883 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:18:32.883 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-08T08:18:32.962 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:18:32.962 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-08T08:18:33.038 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:18:33.038 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-08T08:18:33.119 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:18:33.119 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-08T08:18:33.202 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:18:33.203 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-08T08:18:33.283 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:18:33.283 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-08T08:18:33.320 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:18:33.321 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-08T08:18:33.395 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:18:33.396 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-08T08:18:33.472 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:18:33.473 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-08T08:18:33.552 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:18:33.552 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-08T08:18:33.640 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:18:33.640 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-08T08:18:33.717 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:18:33.718 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-08T08:18:33.792 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:18:33.792 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-08T08:18:33.810 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:18:33.811 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-08T08:18:33.867 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:18:33.868 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-08T08:18:33.888 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:18:33.888 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-08T08:18:33.950 INFO:tasks.ceph:Adding keys to all mons... 2023-11-08T08:18:33.951 DEBUG:teuthology.orchestra.run.smithi040:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-08T08:18:33.958 DEBUG:teuthology.orchestra.run.smithi169:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-08T08:18:34.006 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.y] 2023-11-08T08:18:34.006 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQA3REtlmuaiChAAsI2681k5qbe8I0hdIXHJGg== 2023-11-08T08:18:34.007 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a] 2023-11-08T08:18:34.007 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQA3REtlj6YZGRAABMVq4pKyrp6U3tbqShtmuw== 2023-11-08T08:18:34.007 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.c] 2023-11-08T08:18:34.007 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQA3REtlRseiJBAAZFNQO+PK+kioVBn+S/54Hw== 2023-11-08T08:18:34.008 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.0] 2023-11-08T08:18:34.008 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQA7REtlvWsaLBAAws4XZXHoYl9KnhBp6LEh0w== 2023-11-08T08:18:34.008 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.1] 2023-11-08T08:18:34.008 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQA+REtl8FYFOhAAOY5q9LJAl6PS+essHcF8cw== 2023-11-08T08:18:34.008 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.2] 2023-11-08T08:18:34.009 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBCREtljU1ODBAA1gVE4sX2QNdGHIjdRue/rA== 2023-11-08T08:18:34.009 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.3] 2023-11-08T08:18:34.009 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBFREtlRZsSGxAAh7v5f7TVZNQozESV6k641g== 2023-11-08T08:18:34.009 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.x] 2023-11-08T08:18:34.009 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQA3REtl6YeRERAAKW6fA2vM7HOpZGCulFFLzQ== 2023-11-08T08:18:34.009 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b] 2023-11-08T08:18:34.010 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQA3REtlAfqKLRAAmNg22Rw+FWmzeRDBKQtjfw== 2023-11-08T08:18:34.010 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.d] 2023-11-08T08:18:34.010 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQA3REtlZHCiOBAAW8bJst1bqTZ2jPOr+wXcpg== 2023-11-08T08:18:34.010 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.4] 2023-11-08T08:18:34.011 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBLREtla+qaNRAA0jaCFarD/RgWGp1ltYXQdA== 2023-11-08T08:18:34.011 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.5] 2023-11-08T08:18:34.011 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBPREtltHKNCRAA/TKgyxOGRkN5y/RHWr8vSQ== 2023-11-08T08:18:34.011 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.6] 2023-11-08T08:18:34.011 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBSREtlpXC4GRAAZl6YWPVnbX3o1tyfEkR7CA== 2023-11-08T08:18:34.012 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.7] 2023-11-08T08:18:34.012 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBVREtlvL/4JxAAZtGdthYoFzdMRs3DMcRXNw== 2023-11-08T08:18:34.012 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2023-11-08T08:18:34.012 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQA4REtl0YCpBBAAtap/irtK+GkSmR8g37HRVQ== 2023-11-08T08:18:34.012 INFO:teuthology.orchestra.run.smithi040.stdout:[client.2] 2023-11-08T08:18:34.013 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQA4REtldlrMChAAC9+syjsFWWFwJGXPw82zwQ== 2023-11-08T08:18:34.013 INFO:teuthology.orchestra.run.smithi040.stdout:[client.1] 2023-11-08T08:18:34.013 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQA4REtlgapFERAAxg8GS7iiH06t2mCF5ImFXA== 2023-11-08T08:18:34.013 INFO:teuthology.orchestra.run.smithi040.stdout:[client.3] 2023-11-08T08:18:34.013 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQA4REtllHEDFxAACJSjTZQaOTnyA0K3X/igXQ== 2023-11-08T08:18:34.031 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr.y] 2023-11-08T08:18:34.031 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA3REtlmuaiChAAsI2681k5qbe8I0hdIXHJGg== 2023-11-08T08:18:34.032 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2023-11-08T08:18:34.032 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA3REtlj6YZGRAABMVq4pKyrp6U3tbqShtmuw== 2023-11-08T08:18:34.033 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2023-11-08T08:18:34.033 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA3REtlRseiJBAAZFNQO+PK+kioVBn+S/54Hw== 2023-11-08T08:18:34.033 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.0] 2023-11-08T08:18:34.033 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA7REtlvWsaLBAAws4XZXHoYl9KnhBp6LEh0w== 2023-11-08T08:18:34.033 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.1] 2023-11-08T08:18:34.034 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA+REtl8FYFOhAAOY5q9LJAl6PS+essHcF8cw== 2023-11-08T08:18:34.034 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.2] 2023-11-08T08:18:34.034 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBCREtljU1ODBAA1gVE4sX2QNdGHIjdRue/rA== 2023-11-08T08:18:34.034 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.3] 2023-11-08T08:18:34.034 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBFREtlRZsSGxAAh7v5f7TVZNQozESV6k641g== 2023-11-08T08:18:34.035 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr.x] 2023-11-08T08:18:34.035 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA3REtl6YeRERAAKW6fA2vM7HOpZGCulFFLzQ== 2023-11-08T08:18:34.035 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2023-11-08T08:18:34.035 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA3REtlAfqKLRAAmNg22Rw+FWmzeRDBKQtjfw== 2023-11-08T08:18:34.035 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2023-11-08T08:18:34.035 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA3REtlZHCiOBAAW8bJst1bqTZ2jPOr+wXcpg== 2023-11-08T08:18:34.036 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.4] 2023-11-08T08:18:34.036 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBLREtla+qaNRAA0jaCFarD/RgWGp1ltYXQdA== 2023-11-08T08:18:34.036 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.5] 2023-11-08T08:18:34.036 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBPREtltHKNCRAA/TKgyxOGRkN5y/RHWr8vSQ== 2023-11-08T08:18:34.036 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.6] 2023-11-08T08:18:34.037 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBSREtlpXC4GRAAZl6YWPVnbX3o1tyfEkR7CA== 2023-11-08T08:18:34.037 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.7] 2023-11-08T08:18:34.037 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQBVREtlvL/4JxAAZtGdthYoFzdMRs3DMcRXNw== 2023-11-08T08:18:34.037 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2023-11-08T08:18:34.037 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA4REtl0YCpBBAAtap/irtK+GkSmR8g37HRVQ== 2023-11-08T08:18:34.038 INFO:teuthology.orchestra.run.smithi169.stdout:[client.2] 2023-11-08T08:18:34.038 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA4REtldlrMChAAC9+syjsFWWFwJGXPw82zwQ== 2023-11-08T08:18:34.038 INFO:teuthology.orchestra.run.smithi169.stdout:[client.1] 2023-11-08T08:18:34.038 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA4REtlgapFERAAxg8GS7iiH06t2mCF5ImFXA== 2023-11-08T08:18:34.038 INFO:teuthology.orchestra.run.smithi169.stdout:[client.3] 2023-11-08T08:18:34.039 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQA4REtllHEDFxAACJSjTZQaOTnyA0K3X/igXQ== 2023-11-08T08:18:34.040 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-08T08:18:34.069 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-08T08:18:34.172 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T08:18:34.179 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T08:18:34.266 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T08:18:34.273 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T08:18:34.360 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T08:18:34.367 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T08:18:34.461 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T08:18:34.469 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T08:18:34.553 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T08:18:34.560 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T08:18:34.648 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T08:18:34.655 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T08:18:34.738 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-08T08:18:34.744 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-08T08:18:34.826 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T08:18:34.833 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T08:18:34.919 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T08:18:34.926 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T08:18:35.016 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T08:18:35.023 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T08:18:35.111 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T08:18:35.118 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T08:18:35.203 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T08:18:35.211 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T08:18:35.295 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T08:18:35.302 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T08:18:35.395 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T08:18:35.403 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T08:18:35.491 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T08:18:35.498 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T08:18:35.587 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T08:18:35.594 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T08:18:35.681 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T08:18:35.686 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T08:18:35.776 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-08T08:18:35.776 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-08T08:18:35.813 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-08T08:18:35.979 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-08T08:18:36.019 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-08T08:18:36.055 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-08T08:18:36.209 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-08T08:18:36.248 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-08T08:18:36.330 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-08T08:18:36.474 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-08T08:18:36.513 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-08T08:18:36.522 DEBUG:teuthology.orchestra.run.smithi169:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-08T08:18:36.577 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-08T08:18:36.578 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-08T08:18:36.578 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-08T08:18:36.584 INFO:tasks.ceph.mon.a:Started 2023-11-08T08:18:36.585 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-08T08:18:36.585 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-08T08:18:36.625 INFO:tasks.ceph.mon.b:Started 2023-11-08T08:18:36.626 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-08T08:18:36.626 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-08T08:18:36.630 INFO:tasks.ceph.mon.c:Started 2023-11-08T08:18:36.631 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-08T08:18:36.631 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-08T08:18:36.632 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-08T08:18:36.636 INFO:tasks.ceph.mgr.y:Started 2023-11-08T08:18:36.636 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-08T08:18:36.637 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-08T08:18:36.642 INFO:tasks.ceph.mgr.x:Started 2023-11-08T08:18:36.643 INFO:tasks.ceph:Setting crush tunables to default 2023-11-08T08:18:36.643 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd crush tunables default 2023-11-08T08:18:36.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.808+0000 7f7fe2e61700 1 Processor -- start 2023-11-08T08:18:36.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.808+0000 7f7fe2e61700 1 -- start start 2023-11-08T08:18:36.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.808+0000 7f7fe2e61700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fdc070080 0x7f7fdc0704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:36.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.808+0000 7f7fe2e61700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7fdc164250 0x7f7fdc1646d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:36.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.808+0000 7f7fe2e61700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fdc0ad6c0 con 0x7f7fdc160710 2023-11-08T08:18:36.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.808+0000 7f7fe2e61700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fdc06b1c0 con 0x7f7fdc164250 2023-11-08T08:18:36.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.808+0000 7f7fe2e61700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc0adc60 con 0x7f7fdc070080 2023-11-08T08:18:36.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.812+0000 7f7fe2660700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7fdc164250 0x7f7fdc1646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:36.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.812+0000 7f7fe165e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fdc070080 0x7f7fdc0704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:36.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.812+0000 7f7fe165e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fdc070080 0x7f7fdc0704b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:44798/0 (socket says 172.21.15.40:44798) 2023-11-08T08:18:36.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.812+0000 7f7fe2660700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7fdc164250 0x7f7fdc1646d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46762/0 (socket says 172.21.15.40:46762) 2023-11-08T08:18:36.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.812+0000 7f7fe165e700 1 -- 172.21.15.40:0/432434960 learned_addr learned my addr 172.21.15.40:0/432434960 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:18:36.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.812+0000 7f7fe2660700 1 -- 172.21.15.40:0/432434960 >> v2:172.21.15.40:3300/0 conn(0x7f7fdc164250 msgr2=0x7f7fdc1646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-08T08:18:36.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.812+0000 7f7fe2660700 1 -- 172.21.15.40:0/432434960 >> v2:172.21.15.40:3300/0 conn(0x7f7fdc164250 msgr2=0x7f7fdc1646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-08T08:18:36.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.812+0000 7f7fe2660700 1 --2- 172.21.15.40:0/432434960 >> v2:172.21.15.40:3300/0 conn(0x7f7fdc164250 0x7f7fdc1646d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-08T08:18:36.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.812+0000 7f7fe165e700 1 -- 172.21.15.40:0/432434960 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fdc070080 msgr2=0x7f7fdc0704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-08T08:18:36.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.812+0000 7f7fe165e700 1 -- 172.21.15.40:0/432434960 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fdc070080 msgr2=0x7f7fdc0704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-08T08:18:36.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.812+0000 7f7fe165e700 1 --2- 172.21.15.40:0/432434960 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fdc070080 0x7f7fdc0704b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-08T08:18:36.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.812+0000 7f7fe165e700 1 --2- 172.21.15.40:0/432434960 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fdc070080 0x7f7fdc0704b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-08T08:18:36.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.812+0000 7f7fe2660700 1 --2- 172.21.15.40:0/432434960 >> v2:172.21.15.40:3300/0 conn(0x7f7fdc164250 0x7f7fdc1646d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-08T08:18:36.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.992+0000 7f7fe0e5d700 1 -- 172.21.15.40:0/432434960 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4257627988 0 0) 0x7f7fdc0ad6c0 con 0x7f7fdc160710 2023-11-08T08:18:36.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.992+0000 7f7fe0e5d700 1 -- 172.21.15.40:0/432434960 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fc80035d0 con 0x7f7fdc160710 2023-11-08T08:18:37.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe0e5d700 1 -- 172.21.15.40:0/432434960 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3451836435 0 0) 0x7f7fc80035d0 con 0x7f7fdc160710 2023-11-08T08:18:37.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe0e5d700 1 -- 172.21.15.40:0/432434960 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fdc070080 msgr2=0x7f7fdc0704b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-08T08:18:37.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe0e5d700 1 --2- 172.21.15.40:0/432434960 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fdc070080 0x7f7fdc0704b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:37.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe0e5d700 1 -- 172.21.15.40:0/432434960 >> v2:172.21.15.40:3300/0 conn(0x7f7fdc164250 msgr2=0x7f7fdc1646d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-08T08:18:37.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe0e5d700 1 --2- 172.21.15.40:0/432434960 >> v2:172.21.15.40:3300/0 conn(0x7f7fdc164250 0x7f7fdc1646d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:37.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe0e5d700 1 -- 172.21.15.40:0/432434960 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc0709f0 con 0x7f7fdc160710 2023-11-08T08:18:37.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe0e5d700 1 -- 172.21.15.40:0/432434960 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f7fd8001670 con 0x7f7fdc160710 2023-11-08T08:18:37.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe0e5d700 1 -- 172.21.15.40:0/432434960 >> v1:172.21.15.40:6789/0 conn(0x7f7fdc160710 legacy=0x7f7fdc0cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:37.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe0e5d700 1 --2- 172.21.15.40:0/432434960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fc8003a20 0x7f7fc8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:37.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe0e5d700 1 --2- 172.21.15.40:0/432434960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fc8004870 0x7f7fc8004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:37.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe0e5d700 1 --2- 172.21.15.40:0/432434960 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fdc070080 0x7f7fc8005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:37.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe0e5d700 1 -- 172.21.15.40:0/432434960 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc06b1c0 con 0x7f7fc8003a20 2023-11-08T08:18:37.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe0e5d700 1 -- 172.21.15.40:0/432434960 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc0adc60 con 0x7f7fc8004870 2023-11-08T08:18:37.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe0e5d700 1 -- 172.21.15.40:0/432434960 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc800bae0 con 0x7f7fdc070080 2023-11-08T08:18:37.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe1e5f700 1 --2- 172.21.15.40:0/432434960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fc8003a20 0x7f7fc8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:37.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe165e700 1 --2- 172.21.15.40:0/432434960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fc8004870 0x7f7fc8004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:37.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe2660700 1 --2- 172.21.15.40:0/432434960 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fdc070080 0x7f7fc8005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:37.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe1e5f700 1 -- 172.21.15.40:0/432434960 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fdc070080 msgr2=0x7f7fc8005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:37.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe1e5f700 1 --2- 172.21.15.40:0/432434960 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fdc070080 0x7f7fc8005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:37.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe1e5f700 1 -- 172.21.15.40:0/432434960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fc8004870 msgr2=0x7f7fc8004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:37.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe1e5f700 1 --2- 172.21.15.40:0/432434960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fc8004870 0x7f7fc8004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:37.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:36.996+0000 7f7fe1e5f700 1 -- 172.21.15.40:0/432434960 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc80035d0 con 0x7f7fc8003a20 2023-11-08T08:18:37.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.000+0000 7f7fe1e5f700 1 --2- 172.21.15.40:0/432434960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fc8003a20 0x7f7fc8003e30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800c240 tx=0x7f7fc80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6666979299397921 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:37.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.004+0000 7f7fe0e5d700 1 -- 172.21.15.40:0/432434960 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8007b80 con 0x7f7fc8003a20 2023-11-08T08:18:37.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.004+0000 7f7fe0e5d700 1 -- 172.21.15.40:0/432434960 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd800cce0 con 0x7f7fc8003a20 2023-11-08T08:18:37.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.004+0000 7f7fe0e5d700 1 -- 172.21.15.40:0/432434960 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8012660 con 0x7f7fc8003a20 2023-11-08T08:18:37.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.004+0000 7f7fe2e61700 1 -- 172.21.15.40:0/432434960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fc8003a20 msgr2=0x7f7fc8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:37.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.004+0000 7f7fe2e61700 1 --2- 172.21.15.40:0/432434960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fc8003a20 0x7f7fc8003e30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800c240 tx=0x7f7fc80058a0 comp rx=0 tx=0).stop 2023-11-08T08:18:37.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.004+0000 7f7fe2660700 1 -- 172.21.15.40:0/432434960 reap_dead start 2023-11-08T08:18:37.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.004+0000 7f7fe2e61700 1 -- 172.21.15.40:0/432434960 shutdown_connections 2023-11-08T08:18:37.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.004+0000 7f7fe2e61700 1 -- 172.21.15.40:0/432434960 >> 172.21.15.40:0/432434960 conn(0x7f7fdc0b0f80 msgr2=0x7f7fdc0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:37.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.004+0000 7f7fe2e61700 1 -- 172.21.15.40:0/432434960 shutdown_connections 2023-11-08T08:18:37.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.004+0000 7f7fe2e61700 1 -- 172.21.15.40:0/432434960 wait complete. 2023-11-08T08:18:37.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.008+0000 7f7fe2e61700 1 Processor -- start 2023-11-08T08:18:37.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.008+0000 7f7fe2e61700 1 -- start start 2023-11-08T08:18:37.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.008+0000 7f7fe2e61700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fc8003a20 0x7f7fdc1f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:37.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.008+0000 7f7fe2e61700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fdc151ec0 0x7f7fdc0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:37.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.008+0000 7f7fe2e61700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fdc15bef0 0x7f7fdc0b3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:37.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.008+0000 7f7fe2e61700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc163990 con 0x7f7fdc151ec0 2023-11-08T08:18:37.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.008+0000 7f7fe2e61700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc0c2910 con 0x7f7fc8003a20 2023-11-08T08:18:37.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.008+0000 7f7fe2e61700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc0c2b60 con 0x7f7fdc15bef0 2023-11-08T08:18:37.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.008+0000 7f7fe165e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fdc151ec0 0x7f7fdc0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:37.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.008+0000 7f7fe165e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fdc151ec0 0x7f7fdc0b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46788/0 (socket says 172.21.15.40:46788) 2023-11-08T08:18:37.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.008+0000 7f7fe165e700 1 -- 172.21.15.40:0/3590527220 learned_addr learned my addr 172.21.15.40:0/3590527220 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:18:37.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.008+0000 7f7fe2660700 1 --2- 172.21.15.40:0/3590527220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fdc15bef0 0x7f7fdc0b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:37.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.008+0000 7f7fe165e700 1 -- 172.21.15.40:0/3590527220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fdc15bef0 msgr2=0x7f7fdc0b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:37.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.008+0000 7f7fe165e700 1 --2- 172.21.15.40:0/3590527220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fdc15bef0 0x7f7fdc0b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:37.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.008+0000 7f7fe165e700 1 -- 172.21.15.40:0/3590527220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fc8003a20 msgr2=0x7f7fdc1f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:37.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.008+0000 7f7fe165e700 1 --2- 172.21.15.40:0/3590527220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fc8003a20 0x7f7fdc1f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:37.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.008+0000 7f7fe165e700 1 -- 172.21.15.40:0/3590527220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc06b3d0 con 0x7f7fdc151ec0 2023-11-08T08:18:37.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.008+0000 7f7fe165e700 1 --2- 172.21.15.40:0/3590527220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fdc151ec0 0x7f7fdc0b2e00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc010820 tx=0x7f7fcc010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:37.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.012+0000 7f7fd27fc700 1 -- 172.21.15.40:0/3590527220 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc015940 con 0x7f7fdc151ec0 2023-11-08T08:18:37.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.012+0000 7f7fd27fc700 1 -- 172.21.15.40:0/3590527220 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fcc015aa0 con 0x7f7fdc151ec0 2023-11-08T08:18:37.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.012+0000 7f7fd27fc700 1 -- 172.21.15.40:0/3590527220 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc015e50 con 0x7f7fdc151ec0 2023-11-08T08:18:37.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.012+0000 7f7fe2e61700 1 -- 172.21.15.40:0/3590527220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fdc0cca30 con 0x7f7fdc151ec0 2023-11-08T08:18:37.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.012+0000 7f7fe2e61700 1 -- 172.21.15.40:0/3590527220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fdc06b6b0 con 0x7f7fdc151ec0 2023-11-08T08:18:37.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.012+0000 7f7fd27fc700 1 -- 172.21.15.40:0/3590527220 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f7fcc026070 con 0x7f7fdc151ec0 2023-11-08T08:18:37.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.012+0000 7f7fd27fc700 1 -- 172.21.15.40:0/3590527220 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7fcc00e450 con 0x7f7fdc151ec0 2023-11-08T08:18:37.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.012+0000 7f7fe2e61700 1 -- 172.21.15.40:0/3590527220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fdc073cb0 con 0x7f7fdc151ec0 2023-11-08T08:18:37.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.016+0000 7f7fd27fc700 1 -- 172.21.15.40:0/3590527220 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f7fcc01a020 con 0x7f7fdc151ec0 2023-11-08T08:18:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.092+0000 7f7fe2e61700 1 -- 172.21.15.40:0/3590527220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f7fdc0cf2c0 con 0x7f7fdc151ec0 2023-11-08T08:18:37.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.100+0000 7f7fd27fc700 1 -- 172.21.15.40:0/3590527220 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7fcc00f040 con 0x7f7fdc151ec0 2023-11-08T08:18:37.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.104+0000 7f7fe2e61700 1 -- 172.21.15.40:0/3590527220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fdc151ec0 msgr2=0x7f7fdc0b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:37.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.104+0000 7f7fe2e61700 1 --2- 172.21.15.40:0/3590527220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fdc151ec0 0x7f7fdc0b2e00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc010820 tx=0x7f7fcc010b30 comp rx=0 tx=0).stop 2023-11-08T08:18:37.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.104+0000 7f7fe2e61700 1 -- 172.21.15.40:0/3590527220 shutdown_connections 2023-11-08T08:18:37.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.104+0000 7f7fe2e61700 1 --2- 172.21.15.40:0/3590527220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fdc15bef0 0x7f7fdc0b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:37.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.104+0000 7f7fe2e61700 1 --2- 172.21.15.40:0/3590527220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fdc151ec0 0x7f7fdc0b2e00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:37.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.104+0000 7f7fe2e61700 1 --2- 172.21.15.40:0/3590527220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fc8003a20 0x7f7fdc1f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:37.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.104+0000 7f7fe2e61700 1 -- 172.21.15.40:0/3590527220 >> 172.21.15.40:0/3590527220 conn(0x7f7fdc0b0f80 msgr2=0x7f7fdc060660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:37.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.104+0000 7f7fe2e61700 1 -- 172.21.15.40:0/3590527220 shutdown_connections 2023-11-08T08:18:37.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:37.104+0000 7f7fe2e61700 1 -- 172.21.15.40:0/3590527220 wait complete. 2023-11-08T08:18:37.110 INFO:teuthology.orchestra.run.smithi040.stderr:adjusted tunables profile to default 2023-11-08T08:18:37.133 INFO:tasks.ceph:check_enable_crimson: False 2023-11-08T08:18:37.133 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-08T08:18:37.133 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:18:37.134 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-08T08:18:37.162 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:18:37.163 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-08T08:18:37.238 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:37.232+0000 7fee8dd942c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-08T08:18:37.241 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:18:37.241 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-08T08:18:37.248 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:37.245+0000 7f364a3552c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-08T08:18:37.324 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:18:37.324 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-08T08:18:37.403 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:18:37.403 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-08T08:18:37.437 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:18:37.437 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-08T08:18:37.522 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:18:37.522 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-08T08:18:37.597 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:18:37.597 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-08T08:18:37.672 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 99dd46c2-e710-426f-9a2b-481865dbe5f0 0 2023-11-08T08:18:37.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.873+0000 7f427359e700 1 Processor -- start 2023-11-08T08:18:37.878 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.873+0000 7f427359e700 1 -- start start 2023-11-08T08:18:37.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.873+0000 7f427359e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f42740b43a0 0x7f427415e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:37.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.873+0000 7f427359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4274070790 0x7f427406bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:37.879 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.873+0000 7f427359e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f427415a550 con 0x7f42740b1640 2023-11-08T08:18:37.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.873+0000 7f427359e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f427406b0a0 con 0x7f42740b43a0 2023-11-08T08:18:37.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.873+0000 7f427359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42740caac0 con 0x7f4274070790 2023-11-08T08:18:37.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.873+0000 7f4272d9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4274070790 0x7f427406bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:37.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.873+0000 7f4272d9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4274070790 0x7f427406bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:41902/0 (socket says 172.21.15.169:41902) 2023-11-08T08:18:37.880 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.873+0000 7f4272d9d700 1 -- 172.21.15.169:0/1658526604 learned_addr learned my addr 172.21.15.169:0/1658526604 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-08T08:18:37.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.873+0000 7f427259c700 1 --2- 172.21.15.169:0/1658526604 >> v2:172.21.15.169:3300/0 conn(0x7f42740b43a0 0x7f427415e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:37.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.877+0000 7f4272d9d700 1 -- 172.21.15.169:0/1658526604 >> v2:172.21.15.169:3300/0 conn(0x7f42740b43a0 msgr2=0x7f427415e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:37.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.877+0000 7f4272d9d700 1 --2- 172.21.15.169:0/1658526604 >> v2:172.21.15.169:3300/0 conn(0x7f42740b43a0 0x7f427415e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:37.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.877+0000 7f4272d9d700 1 -- 172.21.15.169:0/1658526604 >> v1:172.21.15.40:6789/0 conn(0x7f42740b1640 legacy=0x7f4274070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:37.881 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.877+0000 7f4272d9d700 1 -- 172.21.15.169:0/1658526604 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42740cada0 con 0x7f4274070790 2023-11-08T08:18:37.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.877+0000 7f4272d9d700 1 --2- 172.21.15.169:0/1658526604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4274070790 0x7f427406bfd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f426c009e50 tx=0x7f426c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66e67d3230b2783f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:37.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.877+0000 7f4271d9b700 1 -- 172.21.15.169:0/1658526604 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c00e070 con 0x7f4274070790 2023-11-08T08:18:37.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.877+0000 7f4271d9b700 1 -- 172.21.15.169:0/1658526604 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c003a40 con 0x7f4274070790 2023-11-08T08:18:37.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.877+0000 7f4271d9b700 1 -- 172.21.15.169:0/1658526604 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c003df0 con 0x7f4274070790 2023-11-08T08:18:37.883 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.877+0000 7f427359e700 1 -- 172.21.15.169:0/1658526604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4274070790 msgr2=0x7f427406bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:37.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.877+0000 7f427359e700 1 --2- 172.21.15.169:0/1658526604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4274070790 0x7f427406bfd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f426c009e50 tx=0x7f426c00b330 comp rx=0 tx=0).stop 2023-11-08T08:18:37.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.877+0000 7f427359e700 1 -- 172.21.15.169:0/1658526604 shutdown_connections 2023-11-08T08:18:37.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.877+0000 7f427359e700 1 --2- 172.21.15.169:0/1658526604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4274070790 0x7f427406bfd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:37.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.877+0000 7f427359e700 1 --2- 172.21.15.169:0/1658526604 >> v2:172.21.15.169:3300/0 conn(0x7f42740b43a0 0x7f427415e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:37.884 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.877+0000 7f427359e700 1 -- 172.21.15.169:0/1658526604 >> 172.21.15.169:0/1658526604 conn(0x7f42740b0f80 msgr2=0x7f42740ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:37.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.877+0000 7f427359e700 1 -- 172.21.15.169:0/1658526604 shutdown_connections 2023-11-08T08:18:37.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.877+0000 7f427359e700 1 -- 172.21.15.169:0/1658526604 wait complete. 2023-11-08T08:18:37.885 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.881+0000 7f427359e700 1 Processor -- start 2023-11-08T08:18:37.886 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.881+0000 7f427359e700 1 -- start start 2023-11-08T08:18:37.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.881+0000 7f427359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4274070790 0x7f42741f5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:37.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.881+0000 7f427359e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42740b1640 0x7f42740cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:37.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.881+0000 7f427359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42740b43a0 0x7f42740cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:37.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.881+0000 7f427359e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42740c2570 con 0x7f4274070790 2023-11-08T08:18:37.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.881+0000 7f427359e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42741f5f80 con 0x7f42740b1640 2023-11-08T08:18:37.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.881+0000 7f427359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f427414bf70 con 0x7f42740b43a0 2023-11-08T08:18:37.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.885+0000 7f4269d9b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42740b1640 0x7f42740cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:37.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.885+0000 7f4269d9b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42740b1640 0x7f42740cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55122/0 (socket says 172.21.15.169:55122) 2023-11-08T08:18:37.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.885+0000 7f4269d9b700 1 -- 172.21.15.169:0/206692841 learned_addr learned my addr 172.21.15.169:0/206692841 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-08T08:18:37.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.885+0000 7f4272d9d700 1 --2- 172.21.15.169:0/206692841 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42740b43a0 0x7f42740cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:37.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.885+0000 7f427259c700 1 --2- 172.21.15.169:0/206692841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4274070790 0x7f42741f5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:37.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.885+0000 7f4272d9d700 1 -- 172.21.15.169:0/206692841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42740b1640 msgr2=0x7f42740cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:37.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.885+0000 7f4272d9d700 1 --2- 172.21.15.169:0/206692841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42740b1640 0x7f42740cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:37.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.885+0000 7f4272d9d700 1 -- 172.21.15.169:0/206692841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4274070790 msgr2=0x7f42741f5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:37.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.885+0000 7f4272d9d700 1 --2- 172.21.15.169:0/206692841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4274070790 0x7f42741f5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:37.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.885+0000 7f4272d9d700 1 -- 172.21.15.169:0/206692841 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c009b30 con 0x7f42740b43a0 2023-11-08T08:18:37.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.885+0000 7f4272d9d700 1 --2- 172.21.15.169:0/206692841 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42740b43a0 0x7f42740cb520 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f426c014000 tx=0x7f426c004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:37.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.885+0000 7f426b7fe700 1 -- 172.21.15.169:0/206692841 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c004440 con 0x7f42740b43a0 2023-11-08T08:18:37.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.885+0000 7f426b7fe700 1 -- 172.21.15.169:0/206692841 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c0092e0 con 0x7f42740b43a0 2023-11-08T08:18:37.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.885+0000 7f427359e700 1 -- 172.21.15.169:0/206692841 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427406b5d0 con 0x7f42740b43a0 2023-11-08T08:18:37.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.885+0000 7f426b7fe700 1 -- 172.21.15.169:0/206692841 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c004800 con 0x7f42740b43a0 2023-11-08T08:18:37.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.885+0000 7f427359e700 1 -- 172.21.15.169:0/206692841 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427406b2f0 con 0x7f42740b43a0 2023-11-08T08:18:37.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.885+0000 7f426b7fe700 1 -- 172.21.15.169:0/206692841 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f426c01c440 con 0x7f42740b43a0 2023-11-08T08:18:37.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.885+0000 7f426b7fe700 1 -- 172.21.15.169:0/206692841 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f426c012430 con 0x7f42740b43a0 2023-11-08T08:18:37.893 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.885+0000 7f427359e700 1 -- 172.21.15.169:0/206692841 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f427415eb10 con 0x7f42740b43a0 2023-11-08T08:18:37.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.893+0000 7f426b7fe700 1 -- 172.21.15.169:0/206692841 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f426c020020 con 0x7f42740b43a0 2023-11-08T08:18:37.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.969+0000 7f427359e700 1 -- 172.21.15.169:0/206692841 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "99dd46c2-e710-426f-9a2b-481865dbe5f0", "id": 0} v 0) v1 -- 0x7f4274073cb0 con 0x7f42740b43a0 2023-11-08T08:18:37.983 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.981+0000 7f426b7fe700 1 -- 172.21.15.169:0/206692841 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "99dd46c2-e710-426f-9a2b-481865dbe5f0", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f426c017070 con 0x7f42740b43a0 2023-11-08T08:18:37.984 INFO:teuthology.orchestra.run.smithi169.stdout:0 2023-11-08T08:18:37.985 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.981+0000 7f427359e700 1 -- 172.21.15.169:0/206692841 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42740b43a0 msgr2=0x7f42740cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:37.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.981+0000 7f427359e700 1 --2- 172.21.15.169:0/206692841 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42740b43a0 0x7f42740cb520 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f426c014000 tx=0x7f426c004af0 comp rx=0 tx=0).stop 2023-11-08T08:18:37.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.981+0000 7f427359e700 1 -- 172.21.15.169:0/206692841 shutdown_connections 2023-11-08T08:18:37.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.981+0000 7f427359e700 1 --2- 172.21.15.169:0/206692841 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42740b43a0 0x7f42740cb520 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:37.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.981+0000 7f427359e700 1 --2- 172.21.15.169:0/206692841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42740b1640 0x7f42740cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:37.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.981+0000 7f427359e700 1 --2- 172.21.15.169:0/206692841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4274070790 0x7f42741f5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:37.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.981+0000 7f427359e700 1 -- 172.21.15.169:0/206692841 >> 172.21.15.169:0/206692841 conn(0x7f42740b0f80 msgr2=0x7f42740aba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:37.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.981+0000 7f427359e700 1 -- 172.21.15.169:0/206692841 shutdown_connections 2023-11-08T08:18:37.987 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:37.981+0000 7f427359e700 1 -- 172.21.15.169:0/206692841 wait complete. 2023-11-08T08:18:38.007 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 62815141-84f4-42c1-b665-f88c1f0561a4 1 2023-11-08T08:18:38.013 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:38.008+0000 7fee8dd942c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-08T08:18:38.014 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:38.009+0000 7f364a3552c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-08T08:18:38.100 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:38.097+0000 7f364a3552c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-08T08:18:38.103 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:38.100+0000 7fee8dd942c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-08T08:18:38.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.157+0000 7f55f4b58700 1 Processor -- start 2023-11-08T08:18:38.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.157+0000 7f55f4b58700 1 -- start start 2023-11-08T08:18:38.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.157+0000 7f55f4b58700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55f00b43a0 0x7f55f015e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:38.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.157+0000 7f55f4b58700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f55f00b1640 0x7f55f0070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:38.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.157+0000 7f55f4b58700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f015a550 con 0x7f55f00c9040 2023-11-08T08:18:38.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.157+0000 7f55f4b58700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f006b0a0 con 0x7f55f00b1640 2023-11-08T08:18:38.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.157+0000 7f55f4b58700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00caac0 con 0x7f55f00b43a0 2023-11-08T08:18:38.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.157+0000 7f55eeffd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f55f00b1640 0x7f55f0070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:38.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.157+0000 7f55eeffd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f55f00b1640 0x7f55f0070080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55130/0 (socket says 172.21.15.169:55130) 2023-11-08T08:18:38.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.157+0000 7f55eeffd700 1 -- 172.21.15.169:0/748944493 learned_addr learned my addr 172.21.15.169:0/748944493 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-08T08:18:38.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.157+0000 7f55ef7fe700 1 --2- 172.21.15.169:0/748944493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55f00b43a0 0x7f55f015e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:38.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.161+0000 7f55eeffd700 1 -- 172.21.15.169:0/748944493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55f00b43a0 msgr2=0x7f55f015e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:38.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.161+0000 7f55eeffd700 1 --2- 172.21.15.169:0/748944493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55f00b43a0 0x7f55f015e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.161+0000 7f55eeffd700 1 -- 172.21.15.169:0/748944493 >> v1:172.21.15.40:6789/0 conn(0x7f55f00c9040 legacy=0x7f55f0069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:38.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.161+0000 7f55eeffd700 1 -- 172.21.15.169:0/748944493 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f00cada0 con 0x7f55f00b1640 2023-11-08T08:18:38.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.161+0000 7f55eeffd700 1 --2- 172.21.15.169:0/748944493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55f00b1640 0x7f55f0070080 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009910 tx=0x7f55e4009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=306bcb67283f35bd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:38.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.161+0000 7f55ee7fc700 1 -- 172.21.15.169:0/748944493 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4011830 con 0x7f55f00b1640 2023-11-08T08:18:38.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.161+0000 7f55ee7fc700 1 -- 172.21.15.169:0/748944493 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e4011990 con 0x7f55f00b1640 2023-11-08T08:18:38.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.161+0000 7f55ee7fc700 1 -- 172.21.15.169:0/748944493 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4011d40 con 0x7f55f00b1640 2023-11-08T08:18:38.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.161+0000 7f55f4b58700 1 -- 172.21.15.169:0/748944493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55f00b1640 msgr2=0x7f55f0070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:38.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.161+0000 7f55f4b58700 1 --2- 172.21.15.169:0/748944493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55f00b1640 0x7f55f0070080 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009910 tx=0x7f55e4009c20 comp rx=0 tx=0).stop 2023-11-08T08:18:38.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.161+0000 7f55f4b58700 1 -- 172.21.15.169:0/748944493 shutdown_connections 2023-11-08T08:18:38.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.161+0000 7f55f4b58700 1 --2- 172.21.15.169:0/748944493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55f00b1640 0x7f55f0070080 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.161+0000 7f55f4b58700 1 --2- 172.21.15.169:0/748944493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55f00b43a0 0x7f55f015e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.161+0000 7f55f4b58700 1 -- 172.21.15.169:0/748944493 >> 172.21.15.169:0/748944493 conn(0x7f55f00b0f80 msgr2=0x7f55f00ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:38.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.161+0000 7f55f4b58700 1 -- 172.21.15.169:0/748944493 shutdown_connections 2023-11-08T08:18:38.168 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.161+0000 7f55f4b58700 1 -- 172.21.15.169:0/748944493 wait complete. 2023-11-08T08:18:38.169 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.161+0000 7f55f4b58700 1 Processor -- start 2023-11-08T08:18:38.170 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55f4b58700 1 -- start start 2023-11-08T08:18:38.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55f4b58700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55f00b1640 0x7f55f00c4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:38.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55f4b58700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55f00b43a0 0x7f55f00af030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:38.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55f4b58700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55f00c9040 0x7f55f00af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:38.172 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55f4b58700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00c4db0 con 0x7f55f00c9040 2023-11-08T08:18:38.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55f4b58700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f015b090 con 0x7f55f00b1640 2023-11-08T08:18:38.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55f4b58700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f015aa90 con 0x7f55f00b43a0 2023-11-08T08:18:38.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55ef7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55f00b1640 0x7f55f00c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:38.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55eeffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55f00b43a0 0x7f55f00af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:38.173 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55ef7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55f00b1640 0x7f55f00c4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55138/0 (socket says 172.21.15.169:55138) 2023-11-08T08:18:38.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55ef7fe700 1 -- 172.21.15.169:0/3591977449 learned_addr learned my addr 172.21.15.169:0/3591977449 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-08T08:18:38.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55effff700 1 --2- 172.21.15.169:0/3591977449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55f00c9040 0x7f55f00af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:38.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55ef7fe700 1 -- 172.21.15.169:0/3591977449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55f00b43a0 msgr2=0x7f55f00af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:38.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55ef7fe700 1 --2- 172.21.15.169:0/3591977449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55f00b43a0 0x7f55f00af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.174 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55ef7fe700 1 -- 172.21.15.169:0/3591977449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55f00c9040 msgr2=0x7f55f00af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:38.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55ef7fe700 1 --2- 172.21.15.169:0/3591977449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55f00c9040 0x7f55f00af570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55ef7fe700 1 -- 172.21.15.169:0/3591977449 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e4009570 con 0x7f55f00b1640 2023-11-08T08:18:38.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55ef7fe700 1 --2- 172.21.15.169:0/3591977449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55f00b1640 0x7f55f00c4870 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f55e000e9d0 tx=0x7f55e000ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:38.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55d7fff700 1 -- 172.21.15.169:0/3591977449 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0009720 con 0x7f55f00b1640 2023-11-08T08:18:38.175 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55d7fff700 1 -- 172.21.15.169:0/3591977449 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e000c850 con 0x7f55f00b1640 2023-11-08T08:18:38.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55f4b58700 1 -- 172.21.15.169:0/3591977449 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f006b5d0 con 0x7f55f00b1640 2023-11-08T08:18:38.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55d7fff700 1 -- 172.21.15.169:0/3591977449 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e000cbc0 con 0x7f55f00b1640 2023-11-08T08:18:38.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55f4b58700 1 -- 172.21.15.169:0/3591977449 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f006b2f0 con 0x7f55f00b1640 2023-11-08T08:18:38.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55d7fff700 1 -- 172.21.15.169:0/3591977449 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f55e0019030 con 0x7f55f00b1640 2023-11-08T08:18:38.176 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.165+0000 7f55d7fff700 1 -- 172.21.15.169:0/3591977449 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f55e0006710 con 0x7f55f00b1640 2023-11-08T08:18:38.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.169+0000 7f55f4b58700 1 -- 172.21.15.169:0/3591977449 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f015eb10 con 0x7f55f00b1640 2023-11-08T08:18:38.177 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.173+0000 7f55d7fff700 1 -- 172.21.15.169:0/3591977449 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f55e0021060 con 0x7f55f00b1640 2023-11-08T08:18:38.249 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:38.245+0000 7f364a3552c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-08T08:18:38.256 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.253+0000 7f55f4b58700 1 -- 172.21.15.169:0/3591977449 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "62815141-84f4-42c1-b665-f88c1f0561a4", "id": 1} v 0) v1 -- 0x7f55f0073cb0 con 0x7f55f00b1640 2023-11-08T08:18:38.256 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:38.252+0000 7fee8dd942c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-08T08:18:38.268 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.265+0000 7f55d7fff700 1 -- 172.21.15.169:0/3591977449 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "62815141-84f4-42c1-b665-f88c1f0561a4", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f55e0017070 con 0x7f55f00b1640 2023-11-08T08:18:38.268 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-11-08T08:18:38.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.265+0000 7f55f4b58700 1 -- 172.21.15.169:0/3591977449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55f00b1640 msgr2=0x7f55f00c4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:38.269 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.265+0000 7f55f4b58700 1 --2- 172.21.15.169:0/3591977449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55f00b1640 0x7f55f00c4870 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f55e000e9d0 tx=0x7f55e000ece0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.265+0000 7f55f4b58700 1 -- 172.21.15.169:0/3591977449 shutdown_connections 2023-11-08T08:18:38.270 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.265+0000 7f55f4b58700 1 --2- 172.21.15.169:0/3591977449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55f00c9040 0x7f55f00af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.265+0000 7f55f4b58700 1 --2- 172.21.15.169:0/3591977449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55f00b43a0 0x7f55f00af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.265+0000 7f55f4b58700 1 --2- 172.21.15.169:0/3591977449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55f00b1640 0x7f55f00c4870 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.271 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.265+0000 7f55f4b58700 1 -- 172.21.15.169:0/3591977449 >> 172.21.15.169:0/3591977449 conn(0x7f55f00b0f80 msgr2=0x7f55f00ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:38.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.265+0000 7f55f4b58700 1 -- 172.21.15.169:0/3591977449 shutdown_connections 2023-11-08T08:18:38.272 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.265+0000 7f55f4b58700 1 -- 172.21.15.169:0/3591977449 wait complete. 2023-11-08T08:18:38.289 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 70af5f1b-66ab-4534-bd9b-d786aa50fb22 2 2023-11-08T08:18:38.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.501+0000 7f2c709ef700 1 Processor -- start 2023-11-08T08:18:38.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.501+0000 7f2c709ef700 1 -- start start 2023-11-08T08:18:38.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.501+0000 7f2c709ef700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6c15c6d0 0x7f2c6c0b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:38.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.501+0000 7f2c709ef700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2c6c06a300 0x7f2c6c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:38.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.501+0000 7f2c709ef700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c6c072270 con 0x7f2c6c0cfc40 2023-11-08T08:18:38.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.501+0000 7f2c709ef700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c6c0d0400 con 0x7f2c6c06a300 2023-11-08T08:18:38.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.501+0000 7f2c709ef700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c0d0120 con 0x7f2c6c15c6d0 2023-11-08T08:18:38.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.501+0000 7f2c6b7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6c15c6d0 0x7f2c6c0b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:38.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.501+0000 7f2c6b7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6c15c6d0 0x7f2c6c0b17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:41946/0 (socket says 172.21.15.169:41946) 2023-11-08T08:18:38.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.501+0000 7f2c6b7fe700 1 -- 172.21.15.169:0/1553715040 learned_addr learned my addr 172.21.15.169:0/1553715040 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-08T08:18:38.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.501+0000 7f2c6bfff700 1 --2- 172.21.15.169:0/1553715040 >> v2:172.21.15.40:3300/0 conn(0x7f2c6c06a300 0x7f2c6c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:38.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.501+0000 7f2c6b7fe700 1 -- 172.21.15.169:0/1553715040 >> v2:172.21.15.40:3300/0 conn(0x7f2c6c06a300 msgr2=0x7f2c6c06c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:38.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.501+0000 7f2c6b7fe700 1 --2- 172.21.15.169:0/1553715040 >> v2:172.21.15.40:3300/0 conn(0x7f2c6c06a300 0x7f2c6c06c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.501+0000 7f2c6b7fe700 1 -- 172.21.15.169:0/1553715040 >> v1:172.21.15.40:6789/0 conn(0x7f2c6c0cfc40 legacy=0x7f2c6c069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:38.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.501+0000 7f2c6b7fe700 1 -- 172.21.15.169:0/1553715040 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6c071c20 con 0x7f2c6c15c6d0 2023-11-08T08:18:38.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.505+0000 7f2c6b7fe700 1 --2- 172.21.15.169:0/1553715040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6c15c6d0 0x7f2c6c0b17a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c009910 tx=0x7f2c5c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe9a0617e9a7e23b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:38.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.505+0000 7f2c6a7fc700 1 -- 172.21.15.169:0/1553715040 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c011830 con 0x7f2c6c15c6d0 2023-11-08T08:18:38.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.505+0000 7f2c6a7fc700 1 -- 172.21.15.169:0/1553715040 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5c011990 con 0x7f2c6c15c6d0 2023-11-08T08:18:38.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.505+0000 7f2c6a7fc700 1 -- 172.21.15.169:0/1553715040 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c011d40 con 0x7f2c6c15c6d0 2023-11-08T08:18:38.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.505+0000 7f2c709ef700 1 -- 172.21.15.169:0/1553715040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6c15c6d0 msgr2=0x7f2c6c0b17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:38.510 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.505+0000 7f2c709ef700 1 --2- 172.21.15.169:0/1553715040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6c15c6d0 0x7f2c6c0b17a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c009910 tx=0x7f2c5c009c20 comp rx=0 tx=0).stop 2023-11-08T08:18:38.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.505+0000 7f2c709ef700 1 -- 172.21.15.169:0/1553715040 shutdown_connections 2023-11-08T08:18:38.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.505+0000 7f2c709ef700 1 --2- 172.21.15.169:0/1553715040 >> v2:172.21.15.40:3300/0 conn(0x7f2c6c06a300 0x7f2c6c06c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.505+0000 7f2c709ef700 1 --2- 172.21.15.169:0/1553715040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6c15c6d0 0x7f2c6c0b17a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.505+0000 7f2c709ef700 1 -- 172.21.15.169:0/1553715040 >> 172.21.15.169:0/1553715040 conn(0x7f2c6c0b6520 msgr2=0x7f2c6c071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:38.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.505+0000 7f2c709ef700 1 -- 172.21.15.169:0/1553715040 shutdown_connections 2023-11-08T08:18:38.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.505+0000 7f2c709ef700 1 -- 172.21.15.169:0/1553715040 wait complete. 2023-11-08T08:18:38.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.509+0000 7f2c709ef700 1 Processor -- start 2023-11-08T08:18:38.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.509+0000 7f2c709ef700 1 -- start start 2023-11-08T08:18:38.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.509+0000 7f2c709ef700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c6c06a300 0x7f2c6c0d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:38.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.509+0000 7f2c709ef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c6c0cfc40 0x7f2c6c1e6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:38.514 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.509+0000 7f2c709ef700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6c15c6d0 0x7f2c6c1e68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:38.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.509+0000 7f2c709ef700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c0c9d90 con 0x7f2c6c0cfc40 2023-11-08T08:18:38.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.509+0000 7f2c709ef700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c0c9fe0 con 0x7f2c6c06a300 2023-11-08T08:18:38.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.509+0000 7f2c709ef700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c0ca2c0 con 0x7f2c6c15c6d0 2023-11-08T08:18:38.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.509+0000 7f2c6b7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c6c06a300 0x7f2c6c0d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:38.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.509+0000 7f2c6bfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6c15c6d0 0x7f2c6c1e68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:38.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.509+0000 7f2c6b7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c6c06a300 0x7f2c6c0d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55150/0 (socket says 172.21.15.169:55150) 2023-11-08T08:18:38.516 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.509+0000 7f2c6b7fe700 1 -- 172.21.15.169:0/3956557629 learned_addr learned my addr 172.21.15.169:0/3956557629 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-08T08:18:38.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.509+0000 7f2c6affd700 1 --2- 172.21.15.169:0/3956557629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c6c0cfc40 0x7f2c6c1e6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:38.517 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.513+0000 7f2c6b7fe700 1 -- 172.21.15.169:0/3956557629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6c15c6d0 msgr2=0x7f2c6c1e68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:38.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.513+0000 7f2c6b7fe700 1 --2- 172.21.15.169:0/3956557629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6c15c6d0 0x7f2c6c1e68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.513+0000 7f2c6b7fe700 1 -- 172.21.15.169:0/3956557629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c6c0cfc40 msgr2=0x7f2c6c1e6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:38.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.513+0000 7f2c6b7fe700 1 --2- 172.21.15.169:0/3956557629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c6c0cfc40 0x7f2c6c1e6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.513+0000 7f2c6b7fe700 1 -- 172.21.15.169:0/3956557629 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c009570 con 0x7f2c6c06a300 2023-11-08T08:18:38.518 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.513+0000 7f2c6bfff700 1 --2- 172.21.15.169:0/3956557629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6c15c6d0 0x7f2c6c1e68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:18:38.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.513+0000 7f2c6b7fe700 1 --2- 172.21.15.169:0/3956557629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c6c06a300 0x7f2c6c0d08b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c000c00 tx=0x7f2c5c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:38.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.513+0000 7f2c53fff700 1 -- 172.21.15.169:0/3956557629 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c00b970 con 0x7f2c6c06a300 2023-11-08T08:18:38.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.513+0000 7f2c53fff700 1 -- 172.21.15.169:0/3956557629 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5c01e070 con 0x7f2c6c06a300 2023-11-08T08:18:38.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.513+0000 7f2c709ef700 1 -- 172.21.15.169:0/3956557629 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6c15caa0 con 0x7f2c6c06a300 2023-11-08T08:18:38.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.513+0000 7f2c53fff700 1 -- 172.21.15.169:0/3956557629 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c00bcd0 con 0x7f2c6c06a300 2023-11-08T08:18:38.519 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.513+0000 7f2c709ef700 1 -- 172.21.15.169:0/3956557629 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6c071f00 con 0x7f2c6c06a300 2023-11-08T08:18:38.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.513+0000 7f2c53fff700 1 -- 172.21.15.169:0/3956557629 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f2c5c00f050 con 0x7f2c6c06a300 2023-11-08T08:18:38.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.513+0000 7f2c53fff700 1 -- 172.21.15.169:0/3956557629 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f2c5c00be30 con 0x7f2c6c06a300 2023-11-08T08:18:38.520 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.513+0000 7f2c709ef700 1 -- 172.21.15.169:0/3956557629 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6c073cb0 con 0x7f2c6c06a300 2023-11-08T08:18:38.521 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.517+0000 7f2c53fff700 1 -- 172.21.15.169:0/3956557629 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f2c5c020070 con 0x7f2c6c06a300 2023-11-08T08:18:38.581 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:38.576+0000 7fee8dd942c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-08T08:18:38.582 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:38.577+0000 7f364a3552c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-08T08:18:38.600 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.597+0000 7f2c709ef700 1 -- 172.21.15.169:0/3956557629 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "70af5f1b-66ab-4534-bd9b-d786aa50fb22", "id": 2} v 0) v1 -- 0x7f2c6c0cd7f0 con 0x7f2c6c06a300 2023-11-08T08:18:38.610 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.605+0000 7f2c53fff700 1 -- 172.21.15.169:0/3956557629 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "70af5f1b-66ab-4534-bd9b-d786aa50fb22", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2c5c015020 con 0x7f2c6c06a300 2023-11-08T08:18:38.610 INFO:teuthology.orchestra.run.smithi169.stdout:2 2023-11-08T08:18:38.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.609+0000 7f2c709ef700 1 -- 172.21.15.169:0/3956557629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c6c06a300 msgr2=0x7f2c6c0d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:38.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.609+0000 7f2c709ef700 1 --2- 172.21.15.169:0/3956557629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c6c06a300 0x7f2c6c0d08b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c000c00 tx=0x7f2c5c004770 comp rx=0 tx=0).stop 2023-11-08T08:18:38.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.609+0000 7f2c709ef700 1 -- 172.21.15.169:0/3956557629 shutdown_connections 2023-11-08T08:18:38.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.609+0000 7f2c709ef700 1 --2- 172.21.15.169:0/3956557629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c6c15c6d0 0x7f2c6c1e68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.609+0000 7f2c709ef700 1 --2- 172.21.15.169:0/3956557629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c6c0cfc40 0x7f2c6c1e6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.609+0000 7f2c709ef700 1 --2- 172.21.15.169:0/3956557629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c6c06a300 0x7f2c6c0d08b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.609+0000 7f2c709ef700 1 -- 172.21.15.169:0/3956557629 >> 172.21.15.169:0/3956557629 conn(0x7f2c6c0b6520 msgr2=0x7f2c6c15a790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:38.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.609+0000 7f2c709ef700 1 -- 172.21.15.169:0/3956557629 shutdown_connections 2023-11-08T08:18:38.614 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.609+0000 7f2c709ef700 1 -- 172.21.15.169:0/3956557629 wait complete. 2023-11-08T08:18:38.633 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new aed310e9-dc5d-4c80-a1fa-da02ebfc8945 3 2023-11-08T08:18:38.656 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:38.653+0000 7f364a3552c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-08T08:18:38.662 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:38.656+0000 7fee8dd942c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-08T08:18:38.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.793+0000 7efc385e6700 1 Processor -- start 2023-11-08T08:18:38.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.793+0000 7efc385e6700 1 -- start start 2023-11-08T08:18:38.799 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc385e6700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efc3015c700 0x7efc300702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:38.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc385e6700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3006cf00 con 0x7efc300b1be0 2023-11-08T08:18:38.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc385e6700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3006ac00 con 0x7efc300b2130 2023-11-08T08:18:38.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc385e6700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc300cd360 con 0x7efc3015c700 2023-11-08T08:18:38.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc35b81700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7efc300b2130 0x7efc3006c0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:44238/0 (socket says 172.21.15.169:44238) 2023-11-08T08:18:38.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc35b81700 1 -- 172.21.15.169:0/4230177738 learned_addr learned my addr 172.21.15.169:0/4230177738 (peer_addr_for_me v1:172.21.15.169:0/0) 2023-11-08T08:18:38.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc36382700 1 --2- 172.21.15.169:0/4230177738 >> v2:172.21.15.40:3300/0 conn(0x7efc3015c700 0x7efc300702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:38.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc35380700 1 -- 172.21.15.169:0/4230177738 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 835402145 0 0) 0x7efc3006ac00 con 0x7efc300b2130 2023-11-08T08:18:38.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc35380700 1 -- 172.21.15.169:0/4230177738 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc1c0035d0 con 0x7efc300b2130 2023-11-08T08:18:38.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc35380700 1 -- 172.21.15.169:0/4230177738 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2045924553 0 0) 0x7efc1c0035d0 con 0x7efc300b2130 2023-11-08T08:18:38.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc35380700 1 -- 172.21.15.169:0/4230177738 >> v2:172.21.15.40:3300/0 conn(0x7efc3015c700 msgr2=0x7efc300702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:38.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc35380700 1 --2- 172.21.15.169:0/4230177738 >> v2:172.21.15.40:3300/0 conn(0x7efc3015c700 0x7efc300702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc35380700 1 -- 172.21.15.169:0/4230177738 >> v1:172.21.15.40:6789/0 conn(0x7efc300b1be0 legacy=0x7efc3006c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:38.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc35380700 1 -- 172.21.15.169:0/4230177738 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc30160190 con 0x7efc300b2130 2023-11-08T08:18:38.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc35380700 1 -- 172.21.15.169:0/4230177738 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7efc20001670 con 0x7efc300b2130 2023-11-08T08:18:38.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc35380700 1 -- 172.21.15.169:0/4230177738 >> v1:172.21.15.169:6789/0 conn(0x7efc300b2130 legacy=0x7efc3006c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:38.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc35380700 1 --2- 172.21.15.169:0/4230177738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc1c003a20 0x7efc1c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:38.805 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc35380700 1 --2- 172.21.15.169:0/4230177738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc1c004870 0x7efc1c004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:38.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc35380700 1 --2- 172.21.15.169:0/4230177738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc1c005220 0x7efc1c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:38.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc35380700 1 -- 172.21.15.169:0/4230177738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc300cd360 con 0x7efc1c004870 2023-11-08T08:18:38.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc35380700 1 -- 172.21.15.169:0/4230177738 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c00c0a0 con 0x7efc1c003a20 2023-11-08T08:18:38.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc35380700 1 -- 172.21.15.169:0/4230177738 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c00c260 con 0x7efc1c005220 2023-11-08T08:18:38.806 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc36b83700 1 --2- 172.21.15.169:0/4230177738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc1c005220 0x7efc1c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:38.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc35b81700 1 --2- 172.21.15.169:0/4230177738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc1c004870 0x7efc1c004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:38.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc36382700 1 --2- 172.21.15.169:0/4230177738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc1c003a20 0x7efc1c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:38.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc36b83700 1 -- 172.21.15.169:0/4230177738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc1c003a20 msgr2=0x7efc1c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:38.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc36b83700 1 --2- 172.21.15.169:0/4230177738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc1c003a20 0x7efc1c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.807 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc36b83700 1 -- 172.21.15.169:0/4230177738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc1c004870 msgr2=0x7efc1c004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:38.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc36b83700 1 --2- 172.21.15.169:0/4230177738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc1c004870 0x7efc1c004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc36b83700 1 -- 172.21.15.169:0/4230177738 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c0035d0 con 0x7efc1c005220 2023-11-08T08:18:38.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.797+0000 7efc36382700 1 --2- 172.21.15.169:0/4230177738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc1c003a20 0x7efc1c003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:18:38.808 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.801+0000 7efc36b83700 1 -- 172.21.15.169:0/4230177738 reap_dead start 2023-11-08T08:18:38.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.801+0000 7efc36b83700 1 --2- 172.21.15.169:0/4230177738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc1c005220 0x7efc1c00ba30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efc2800c190 tx=0x7efc1c0058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c862feba94fa6f72 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:38.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.801+0000 7efc35380700 1 -- 172.21.15.169:0/4230177738 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc28001db0 con 0x7efc1c005220 2023-11-08T08:18:38.811 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.801+0000 7efc35380700 1 -- 172.21.15.169:0/4230177738 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2800ccf0 con 0x7efc1c005220 2023-11-08T08:18:38.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.801+0000 7efc35380700 1 -- 172.21.15.169:0/4230177738 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc28003c60 con 0x7efc1c005220 2023-11-08T08:18:38.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.801+0000 7efc385e6700 1 -- 172.21.15.169:0/4230177738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc1c005220 msgr2=0x7efc1c00ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:38.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.801+0000 7efc385e6700 1 --2- 172.21.15.169:0/4230177738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc1c005220 0x7efc1c00ba30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efc2800c190 tx=0x7efc1c0058a0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.801+0000 7efc385e6700 1 -- 172.21.15.169:0/4230177738 shutdown_connections 2023-11-08T08:18:38.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.801+0000 7efc385e6700 1 --2- 172.21.15.169:0/4230177738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc1c005220 0x7efc1c00ba30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.812 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.801+0000 7efc385e6700 1 -- 172.21.15.169:0/4230177738 >> 172.21.15.169:0/4230177738 conn(0x7efc300d2d10 msgr2=0x7efc300d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:38.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.801+0000 7efc385e6700 1 -- 172.21.15.169:0/4230177738 shutdown_connections 2023-11-08T08:18:38.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.801+0000 7efc385e6700 1 -- 172.21.15.169:0/4230177738 wait complete. 2023-11-08T08:18:38.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.805+0000 7efc385e6700 1 Processor -- start 2023-11-08T08:18:38.813 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.805+0000 7efc385e6700 1 -- start start 2023-11-08T08:18:38.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.805+0000 7efc385e6700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc1c005220 0x7efc3014d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:38.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.805+0000 7efc385e6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc3015dac0 0x7efc300b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:38.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.805+0000 7efc385e6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc3015df30 0x7efc300b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:38.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.805+0000 7efc385e6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc300cbbd0 con 0x7efc3015df30 2023-11-08T08:18:38.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.805+0000 7efc385e6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc300cad40 con 0x7efc3015dac0 2023-11-08T08:18:38.814 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.805+0000 7efc385e6700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc300cb020 con 0x7efc1c005220 2023-11-08T08:18:38.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.805+0000 7efc36382700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc1c005220 0x7efc3014d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:38.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.805+0000 7efc35b81700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc3015dac0 0x7efc300b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:38.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.805+0000 7efc35b81700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc3015dac0 0x7efc300b2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55168/0 (socket says 172.21.15.169:55168) 2023-11-08T08:18:38.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.805+0000 7efc35b81700 1 -- 172.21.15.169:0/3428421110 learned_addr learned my addr 172.21.15.169:0/3428421110 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-08T08:18:38.815 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.805+0000 7efc36b83700 1 --2- 172.21.15.169:0/3428421110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc3015df30 0x7efc300b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:38.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.805+0000 7efc36382700 1 -- 172.21.15.169:0/3428421110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc3015dac0 msgr2=0x7efc300b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:38.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.805+0000 7efc36382700 1 --2- 172.21.15.169:0/3428421110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc3015dac0 0x7efc300b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.805+0000 7efc36382700 1 -- 172.21.15.169:0/3428421110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc3015df30 msgr2=0x7efc300b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:38.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.805+0000 7efc36382700 1 --2- 172.21.15.169:0/3428421110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc3015df30 0x7efc300b2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.816 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.805+0000 7efc36382700 1 -- 172.21.15.169:0/3428421110 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc30160750 con 0x7efc1c005220 2023-11-08T08:18:38.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.805+0000 7efc36382700 1 --2- 172.21.15.169:0/3428421110 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc1c005220 0x7efc3014d0c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efc2c011d00 tx=0x7efc2c00f520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:38.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.809+0000 7efc26ffd700 1 -- 172.21.15.169:0/3428421110 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c00fb30 con 0x7efc1c005220 2023-11-08T08:18:38.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.809+0000 7efc26ffd700 1 -- 172.21.15.169:0/3428421110 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2c00fc90 con 0x7efc1c005220 2023-11-08T08:18:38.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.809+0000 7efc385e6700 1 -- 172.21.15.169:0/3428421110 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc30160470 con 0x7efc1c005220 2023-11-08T08:18:38.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.809+0000 7efc26ffd700 1 -- 172.21.15.169:0/3428421110 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c004020 con 0x7efc1c005220 2023-11-08T08:18:38.817 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.809+0000 7efc385e6700 1 -- 172.21.15.169:0/3428421110 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc30160a30 con 0x7efc1c005220 2023-11-08T08:18:38.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.809+0000 7efc26ffd700 1 -- 172.21.15.169:0/3428421110 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7efc2c024070 con 0x7efc1c005220 2023-11-08T08:18:38.818 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.809+0000 7efc26ffd700 1 -- 172.21.15.169:0/3428421110 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7efc2c003d80 con 0x7efc1c005220 2023-11-08T08:18:38.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.809+0000 7efc385e6700 1 -- 172.21.15.169:0/3428421110 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc300cbe40 con 0x7efc1c005220 2023-11-08T08:18:38.819 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.813+0000 7efc26ffd700 1 -- 172.21.15.169:0/3428421110 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7efc2c022060 con 0x7efc1c005220 2023-11-08T08:18:38.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.893+0000 7efc385e6700 1 -- 172.21.15.169:0/3428421110 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "aed310e9-dc5d-4c80-a1fa-da02ebfc8945", "id": 3} v 0) v1 -- 0x7efc300726c0 con 0x7efc1c005220 2023-11-08T08:18:38.906 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.901+0000 7efc26ffd700 1 -- 172.21.15.169:0/3428421110 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "aed310e9-dc5d-4c80-a1fa-da02ebfc8945", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7efc2c02c0b0 con 0x7efc1c005220 2023-11-08T08:18:38.906 INFO:teuthology.orchestra.run.smithi169.stdout:3 2023-11-08T08:18:38.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.905+0000 7efc385e6700 1 -- 172.21.15.169:0/3428421110 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc1c005220 msgr2=0x7efc3014d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:38.907 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.905+0000 7efc385e6700 1 --2- 172.21.15.169:0/3428421110 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc1c005220 0x7efc3014d0c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efc2c011d00 tx=0x7efc2c00f520 comp rx=0 tx=0).stop 2023-11-08T08:18:38.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.905+0000 7efc385e6700 1 -- 172.21.15.169:0/3428421110 shutdown_connections 2023-11-08T08:18:38.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.905+0000 7efc385e6700 1 --2- 172.21.15.169:0/3428421110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc3015df30 0x7efc300b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.905+0000 7efc385e6700 1 --2- 172.21.15.169:0/3428421110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc3015dac0 0x7efc300b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.905+0000 7efc385e6700 1 --2- 172.21.15.169:0/3428421110 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc1c005220 0x7efc3014d0c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:38.908 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.905+0000 7efc385e6700 1 -- 172.21.15.169:0/3428421110 >> 172.21.15.169:0/3428421110 conn(0x7efc300d2d10 msgr2=0x7efc300ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:38.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.905+0000 7efc385e6700 1 -- 172.21.15.169:0/3428421110 shutdown_connections 2023-11-08T08:18:38.909 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:38.905+0000 7efc385e6700 1 -- 172.21.15.169:0/3428421110 wait complete. 2023-11-08T08:18:38.919 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:38.917+0000 7f364a3552c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-08T08:18:38.921 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:38.916+0000 7fee8dd942c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-08T08:18:38.931 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 03ad6509-7f6f-4c1e-8580-dc3371e85213 4 2023-11-08T08:18:39.020 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:39.017+0000 7f364a3552c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-08T08:18:39.030 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:39.024+0000 7fee8dd942c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-08T08:18:39.126 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:39.121+0000 7f364a3552c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-08T08:18:39.132 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:39.128+0000 7fee8dd942c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-08T08:18:39.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.145+0000 7fa2bd7aa700 1 Processor -- start 2023-11-08T08:18:39.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.145+0000 7fa2bd7aa700 1 -- start start 2023-11-08T08:18:39.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.149+0000 7fa2bd7aa700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b815c3c0 0x7fa2b80ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:39.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.149+0000 7fa2bd7aa700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa2b8071210 0x7fa2b815fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:39.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.149+0000 7fa2bd7aa700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b80cd030 con 0x7fa2b815d290 2023-11-08T08:18:39.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.149+0000 7fa2bd7aa700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b815d850 con 0x7fa2b8071210 2023-11-08T08:18:39.154 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.149+0000 7fa2bd7aa700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b8061510 con 0x7fa2b815c3c0 2023-11-08T08:18:39.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.149+0000 7fa2b6ffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b815c3c0 0x7fa2b80ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:39.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.149+0000 7fa2b67fc700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fa2b815d290 0x7fa2b8070b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:44242/0 (socket says 172.21.15.169:44242) 2023-11-08T08:18:39.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.149+0000 7fa2b67fc700 1 -- 172.21.15.169:0/1324338245 learned_addr learned my addr 172.21.15.169:0/1324338245 (peer_addr_for_me v1:172.21.15.169:0/0) 2023-11-08T08:18:39.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.149+0000 7fa2b77fe700 1 --2- 172.21.15.169:0/1324338245 >> v2:172.21.15.40:3300/0 conn(0x7fa2b8071210 0x7fa2b815fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:39.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.149+0000 7fa2b6ffd700 1 -- 172.21.15.169:0/1324338245 >> v2:172.21.15.40:3300/0 conn(0x7fa2b8071210 msgr2=0x7fa2b815fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:39.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.149+0000 7fa2b6ffd700 1 --2- 172.21.15.169:0/1324338245 >> v2:172.21.15.40:3300/0 conn(0x7fa2b8071210 0x7fa2b815fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.149+0000 7fa2b6ffd700 1 -- 172.21.15.169:0/1324338245 >> v1:172.21.15.169:6789/0 conn(0x7fa2b815d290 legacy=0x7fa2b8070b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:39.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.149+0000 7fa2b6ffd700 1 -- 172.21.15.169:0/1324338245 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b80719c0 con 0x7fa2b815c3c0 2023-11-08T08:18:39.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.149+0000 7fa2b5ffb700 1 -- 172.21.15.169:0/1324338245 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 51080084 0 0) 0x7fa2b80cd030 con 0x7fa2b815d290 2023-11-08T08:18:39.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.149+0000 7fa2b6ffd700 1 --2- 172.21.15.169:0/1324338245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b815c3c0 0x7fa2b80ce800 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009850 tx=0x7fa2a8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8baed9aeda27f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:39.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.149+0000 7fa2b5ffb700 1 -- 172.21.15.169:0/1324338245 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8011830 con 0x7fa2b815c3c0 2023-11-08T08:18:39.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.149+0000 7fa2b5ffb700 1 -- 172.21.15.169:0/1324338245 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a8011990 con 0x7fa2b815c3c0 2023-11-08T08:18:39.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.149+0000 7fa2b5ffb700 1 -- 172.21.15.169:0/1324338245 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8011d40 con 0x7fa2b815c3c0 2023-11-08T08:18:39.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.149+0000 7fa2bd7aa700 1 -- 172.21.15.169:0/1324338245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b815c3c0 msgr2=0x7fa2b80ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:39.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.149+0000 7fa2bd7aa700 1 --2- 172.21.15.169:0/1324338245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b815c3c0 0x7fa2b80ce800 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009850 tx=0x7fa2a8009b60 comp rx=0 tx=0).stop 2023-11-08T08:18:39.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.153+0000 7fa2bd7aa700 1 -- 172.21.15.169:0/1324338245 shutdown_connections 2023-11-08T08:18:39.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.153+0000 7fa2bd7aa700 1 --2- 172.21.15.169:0/1324338245 >> v2:172.21.15.40:3300/0 conn(0x7fa2b8071210 0x7fa2b815fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.153+0000 7fa2bd7aa700 1 --2- 172.21.15.169:0/1324338245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b815c3c0 0x7fa2b80ce800 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.159 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.153+0000 7fa2bd7aa700 1 -- 172.21.15.169:0/1324338245 >> 172.21.15.169:0/1324338245 conn(0x7fa2b80cbb70 msgr2=0x7fa2b80cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:39.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.153+0000 7fa2bd7aa700 1 -- 172.21.15.169:0/1324338245 shutdown_connections 2023-11-08T08:18:39.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.153+0000 7fa2bd7aa700 1 -- 172.21.15.169:0/1324338245 wait complete. 2023-11-08T08:18:39.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.153+0000 7fa2bd7aa700 1 Processor -- start 2023-11-08T08:18:39.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.153+0000 7fa2bd7aa700 1 -- start start 2023-11-08T08:18:39.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.153+0000 7fa2bd7aa700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b8071210 0x7fa2b80ac270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:39.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.153+0000 7fa2bd7aa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b815c3c0 0x7fa2b80c3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:39.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.153+0000 7fa2bd7aa700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2b815d290 0x7fa2b80c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:39.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa2bd7aa700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b80c2c20 con 0x7fa2b815c3c0 2023-11-08T08:18:39.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa2bd7aa700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b80ac850 con 0x7fa2b815d290 2023-11-08T08:18:39.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa2bd7aa700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b80acb30 con 0x7fa2b8071210 2023-11-08T08:18:39.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa2b77fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2b815d290 0x7fa2b80c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:39.163 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa2b77fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2b815d290 0x7fa2b80c35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55180/0 (socket says 172.21.15.169:55180) 2023-11-08T08:18:39.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa2b77fe700 1 -- 172.21.15.169:0/41215789 learned_addr learned my addr 172.21.15.169:0/41215789 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-08T08:18:39.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa2b6ffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b8071210 0x7fa2b80ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:39.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa2b67fc700 1 --2- 172.21.15.169:0/41215789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b815c3c0 0x7fa2b80c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:39.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa2b77fe700 1 -- 172.21.15.169:0/41215789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b8071210 msgr2=0x7fa2b80ac270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:39.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa2b77fe700 1 --2- 172.21.15.169:0/41215789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b8071210 0x7fa2b80ac270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.164 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa2b77fe700 1 -- 172.21.15.169:0/41215789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b815c3c0 msgr2=0x7fa2b80c3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:39.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa2b77fe700 1 --2- 172.21.15.169:0/41215789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b815c3c0 0x7fa2b80c3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa2b77fe700 1 -- 172.21.15.169:0/41215789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a80094b0 con 0x7fa2b815d290 2023-11-08T08:18:39.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa2b77fe700 1 --2- 172.21.15.169:0/41215789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2b815d290 0x7fa2b80c35b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000ef90 tx=0x7fa2b0009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:39.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa29f7fe700 1 -- 172.21.15.169:0/41215789 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0009b20 con 0x7fa2b815d290 2023-11-08T08:18:39.165 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa29f7fe700 1 -- 172.21.15.169:0/41215789 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b000f040 con 0x7fa2b815d290 2023-11-08T08:18:39.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa29f7fe700 1 -- 172.21.15.169:0/41215789 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b000cd20 con 0x7fa2b815d290 2023-11-08T08:18:39.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa2bd7aa700 1 -- 172.21.15.169:0/41215789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b80cccc0 con 0x7fa2b815d290 2023-11-08T08:18:39.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa2bd7aa700 1 -- 172.21.15.169:0/41215789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b8071ca0 con 0x7fa2b815d290 2023-11-08T08:18:39.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa29f7fe700 1 -- 172.21.15.169:0/41215789 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fa2b0014480 con 0x7fa2b815d290 2023-11-08T08:18:39.166 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa29f7fe700 1 -- 172.21.15.169:0/41215789 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fa2b0014c20 con 0x7fa2b815d290 2023-11-08T08:18:39.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.157+0000 7fa2bd7aa700 1 -- 172.21.15.169:0/41215789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b8069bf0 con 0x7fa2b815d290 2023-11-08T08:18:39.167 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.165+0000 7fa29f7fe700 1 -- 172.21.15.169:0/41215789 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fa2b003bb00 con 0x7fa2b815d290 2023-11-08T08:18:39.247 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.245+0000 7fa2bd7aa700 1 -- 172.21.15.169:0/41215789 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "03ad6509-7f6f-4c1e-8580-dc3371e85213", "id": 4} v 0) v1 -- 0x7fa2b80c2d90 con 0x7fa2b815d290 2023-11-08T08:18:39.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.253+0000 7fa29f7fe700 1 -- 172.21.15.169:0/41215789 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "03ad6509-7f6f-4c1e-8580-dc3371e85213", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa2b80c2d90 con 0x7fa2b815d290 2023-11-08T08:18:39.257 INFO:teuthology.orchestra.run.smithi169.stdout:4 2023-11-08T08:18:39.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.253+0000 7fa2bd7aa700 1 -- 172.21.15.169:0/41215789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2b815d290 msgr2=0x7fa2b80c35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:39.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.253+0000 7fa2bd7aa700 1 --2- 172.21.15.169:0/41215789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2b815d290 0x7fa2b80c35b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000ef90 tx=0x7fa2b0009f60 comp rx=0 tx=0).stop 2023-11-08T08:18:39.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.253+0000 7fa2bd7aa700 1 -- 172.21.15.169:0/41215789 shutdown_connections 2023-11-08T08:18:39.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.253+0000 7fa2bd7aa700 1 --2- 172.21.15.169:0/41215789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2b815d290 0x7fa2b80c35b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.253+0000 7fa2bd7aa700 1 --2- 172.21.15.169:0/41215789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b815c3c0 0x7fa2b80c3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.253+0000 7fa2bd7aa700 1 --2- 172.21.15.169:0/41215789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b8071210 0x7fa2b80ac270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.253+0000 7fa2bd7aa700 1 -- 172.21.15.169:0/41215789 >> 172.21.15.169:0/41215789 conn(0x7fa2b80cbb70 msgr2=0x7fa2b815c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:39.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.253+0000 7fa2bd7aa700 1 -- 172.21.15.169:0/41215789 shutdown_connections 2023-11-08T08:18:39.260 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.253+0000 7fa2bd7aa700 1 -- 172.21.15.169:0/41215789 wait complete. 2023-11-08T08:18:39.282 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 3f69a76b-e285-4741-99ef-5be9617754f4 5 2023-11-08T08:18:39.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.481+0000 7fbf071bc700 1 Processor -- start 2023-11-08T08:18:39.485 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.481+0000 7fbf071bc700 1 -- start start 2023-11-08T08:18:39.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf071bc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbf00071aa0 0x7fbf0006d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:39.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf071bc700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf0006c600 con 0x7fbf0015c640 2023-11-08T08:18:39.488 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf071bc700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf000722f0 con 0x7fbf00069e00 2023-11-08T08:18:39.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf071bc700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf0006cef0 con 0x7fbf00071aa0 2023-11-08T08:18:39.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf069bb700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fbf00069e00 0x7fbf000d0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:44246/0 (socket says 172.21.15.169:44246) 2023-11-08T08:18:39.489 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf069bb700 1 -- 172.21.15.169:0/4169211607 learned_addr learned my addr 172.21.15.169:0/4169211607 (peer_addr_for_me v1:172.21.15.169:0/0) 2023-11-08T08:18:39.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf059b9700 1 --2- 172.21.15.169:0/4169211607 >> v2:172.21.15.40:3300/0 conn(0x7fbf00071aa0 0x7fbf0006d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:39.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf051b8700 1 -- 172.21.15.169:0/4169211607 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372537014 0 0) 0x7fbf000722f0 con 0x7fbf00069e00 2023-11-08T08:18:39.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf051b8700 1 -- 172.21.15.169:0/4169211607 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbee80035d0 con 0x7fbf00069e00 2023-11-08T08:18:39.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf051b8700 1 -- 172.21.15.169:0/4169211607 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2741256395 0 0) 0x7fbee80035d0 con 0x7fbf00069e00 2023-11-08T08:18:39.492 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf051b8700 1 -- 172.21.15.169:0/4169211607 >> v2:172.21.15.40:3300/0 conn(0x7fbf00071aa0 msgr2=0x7fbf0006d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:39.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf051b8700 1 --2- 172.21.15.169:0/4169211607 >> v2:172.21.15.40:3300/0 conn(0x7fbf00071aa0 0x7fbf0006d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf051b8700 1 -- 172.21.15.169:0/4169211607 >> v1:172.21.15.40:6789/0 conn(0x7fbf0015c640 legacy=0x7fbf000d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:39.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf051b8700 1 -- 172.21.15.169:0/4169211607 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0015ca10 con 0x7fbf00069e00 2023-11-08T08:18:39.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf051b8700 1 -- 172.21.15.169:0/4169211607 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7fbef80019f0 con 0x7fbf00069e00 2023-11-08T08:18:39.493 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf059b9700 1 --2- 172.21.15.169:0/4169211607 >> v2:172.21.15.40:3300/0 conn(0x7fbf00071aa0 0x7fbf0006d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:18:39.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf051b8700 1 -- 172.21.15.169:0/4169211607 >> v1:172.21.15.169:6789/0 conn(0x7fbf00069e00 legacy=0x7fbf000d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:39.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf051b8700 1 --2- 172.21.15.169:0/4169211607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbee8003a20 0x7fbee8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:39.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf051b8700 1 --2- 172.21.15.169:0/4169211607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbee8004870 0x7fbee8004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:39.494 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf051b8700 1 --2- 172.21.15.169:0/4169211607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbee8005220 0x7fbee800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:39.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf051b8700 1 -- 172.21.15.169:0/4169211607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0006cef0 con 0x7fbee8005220 2023-11-08T08:18:39.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf051b8700 1 -- 172.21.15.169:0/4169211607 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee800c080 con 0x7fbee8004870 2023-11-08T08:18:39.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf051b8700 1 -- 172.21.15.169:0/4169211607 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee800c240 con 0x7fbee8003a20 2023-11-08T08:18:39.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf061ba700 1 --2- 172.21.15.169:0/4169211607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbee8003a20 0x7fbee8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:39.495 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf059b9700 1 --2- 172.21.15.169:0/4169211607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbee8004870 0x7fbee8004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:39.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf069bb700 1 --2- 172.21.15.169:0/4169211607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbee8005220 0x7fbee800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:39.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf061ba700 1 -- 172.21.15.169:0/4169211607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbee8004870 msgr2=0x7fbee8004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:39.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf061ba700 1 --2- 172.21.15.169:0/4169211607 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbee8004870 0x7fbee8004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf061ba700 1 -- 172.21.15.169:0/4169211607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbee8005220 msgr2=0x7fbee800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:39.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf061ba700 1 --2- 172.21.15.169:0/4169211607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbee8005220 0x7fbee800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.496 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf061ba700 1 -- 172.21.15.169:0/4169211607 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee80035d0 con 0x7fbee8003a20 2023-11-08T08:18:39.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.485+0000 7fbf069bb700 1 -- 172.21.15.169:0/4169211607 reap_dead start 2023-11-08T08:18:39.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.489+0000 7fbf061ba700 1 --2- 172.21.15.169:0/4169211607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbee8003a20 0x7fbee8003e10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbef0003ef0 tx=0x7fbee80058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96b3098d2da69187 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:39.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.489+0000 7fbf051b8700 1 -- 172.21.15.169:0/4169211607 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0016070 con 0x7fbee8003a20 2023-11-08T08:18:39.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.489+0000 7fbf051b8700 1 -- 172.21.15.169:0/4169211607 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef0006880 con 0x7fbee8003a20 2023-11-08T08:18:39.497 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.489+0000 7fbf051b8700 1 -- 172.21.15.169:0/4169211607 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0003990 con 0x7fbee8003a20 2023-11-08T08:18:39.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.489+0000 7fbf071bc700 1 -- 172.21.15.169:0/4169211607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbee8003a20 msgr2=0x7fbee8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:39.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.489+0000 7fbf071bc700 1 --2- 172.21.15.169:0/4169211607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbee8003a20 0x7fbee8003e10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbef0003ef0 tx=0x7fbee80058a0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.489+0000 7fbf071bc700 1 -- 172.21.15.169:0/4169211607 shutdown_connections 2023-11-08T08:18:39.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.489+0000 7fbf071bc700 1 --2- 172.21.15.169:0/4169211607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbee8003a20 0x7fbee8003e10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.501 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.489+0000 7fbf071bc700 1 -- 172.21.15.169:0/4169211607 >> 172.21.15.169:0/4169211607 conn(0x7fbf000b6520 msgr2=0x7fbf00071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:39.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.489+0000 7fbf071bc700 1 -- 172.21.15.169:0/4169211607 shutdown_connections 2023-11-08T08:18:39.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.489+0000 7fbf071bc700 1 -- 172.21.15.169:0/4169211607 wait complete. 2023-11-08T08:18:39.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.493+0000 7fbf071bc700 1 Processor -- start 2023-11-08T08:18:39.502 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.493+0000 7fbf071bc700 1 -- start start 2023-11-08T08:18:39.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.493+0000 7fbf071bc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbee8003a20 0x7fbf000c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:39.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.493+0000 7fbf071bc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf000d0c00 0x7fbf0009b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:39.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.493+0000 7fbf071bc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf0015f360 0x7fbf0009be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:39.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.493+0000 7fbf071bc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000d38f0 con 0x7fbee8003a20 2023-11-08T08:18:39.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.493+0000 7fbf071bc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0015f1d0 con 0x7fbf000d0c00 2023-11-08T08:18:39.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.493+0000 7fbf071bc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000c7d10 con 0x7fbf0015f360 2023-11-08T08:18:39.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.493+0000 7fbf059b9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf000d0c00 0x7fbf0009b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:39.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.493+0000 7fbf069bb700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf0015f360 0x7fbf0009be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:39.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.493+0000 7fbf059b9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf000d0c00 0x7fbf0009b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55204/0 (socket says 172.21.15.169:55204) 2023-11-08T08:18:39.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.493+0000 7fbf059b9700 1 -- 172.21.15.169:0/1254315057 learned_addr learned my addr 172.21.15.169:0/1254315057 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-08T08:18:39.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.493+0000 7fbf069bb700 1 -- 172.21.15.169:0/1254315057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf000d0c00 msgr2=0x7fbf0009b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:39.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.493+0000 7fbf069bb700 1 --2- 172.21.15.169:0/1254315057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf000d0c00 0x7fbf0009b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.493+0000 7fbf069bb700 1 -- 172.21.15.169:0/1254315057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbee8003a20 msgr2=0x7fbf000c6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:39.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.493+0000 7fbf069bb700 1 --2- 172.21.15.169:0/1254315057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbee8003a20 0x7fbf000c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.493+0000 7fbf069bb700 1 -- 172.21.15.169:0/1254315057 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0006c830 con 0x7fbf0015f360 2023-11-08T08:18:39.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.493+0000 7fbf059b9700 1 --2- 172.21.15.169:0/1254315057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf000d0c00 0x7fbf0009b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:18:39.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.497+0000 7fbf069bb700 1 --2- 172.21.15.169:0/1254315057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf0015f360 0x7fbf0009be50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbf0006fda0 tx=0x7fbef8001a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:39.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.497+0000 7fbefeffd700 1 -- 172.21.15.169:0/1254315057 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef80077a0 con 0x7fbf0015f360 2023-11-08T08:18:39.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.497+0000 7fbefeffd700 1 -- 172.21.15.169:0/1254315057 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef8007900 con 0x7fbf0015f360 2023-11-08T08:18:39.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.497+0000 7fbf071bc700 1 -- 172.21.15.169:0/1254315057 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf00072010 con 0x7fbf0015f360 2023-11-08T08:18:39.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.497+0000 7fbefeffd700 1 -- 172.21.15.169:0/1254315057 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef80133f0 con 0x7fbf0015f360 2023-11-08T08:18:39.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.497+0000 7fbf071bc700 1 -- 172.21.15.169:0/1254315057 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0006cb10 con 0x7fbf0015f360 2023-11-08T08:18:39.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.497+0000 7fbefeffd700 1 -- 172.21.15.169:0/1254315057 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fbef8013840 con 0x7fbf0015f360 2023-11-08T08:18:39.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.497+0000 7fbefeffd700 1 -- 172.21.15.169:0/1254315057 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fbef801d460 con 0x7fbf0015f360 2023-11-08T08:18:39.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.497+0000 7fbf071bc700 1 -- 172.21.15.169:0/1254315057 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf00073cb0 con 0x7fbf0015f360 2023-11-08T08:18:39.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.501+0000 7fbefeffd700 1 -- 172.21.15.169:0/1254315057 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fbef8013b80 con 0x7fbf0015f360 2023-11-08T08:18:39.584 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.581+0000 7fbf071bc700 1 -- 172.21.15.169:0/1254315057 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3f69a76b-e285-4741-99ef-5be9617754f4", "id": 5} v 0) v1 -- 0x7fbf000b4bc0 con 0x7fbf0015f360 2023-11-08T08:18:39.593 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.589+0000 7fbefeffd700 1 -- 172.21.15.169:0/1254315057 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3f69a76b-e285-4741-99ef-5be9617754f4", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbef802c0b0 con 0x7fbf0015f360 2023-11-08T08:18:39.594 INFO:teuthology.orchestra.run.smithi169.stdout:5 2023-11-08T08:18:39.595 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.589+0000 7fbf071bc700 1 -- 172.21.15.169:0/1254315057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf0015f360 msgr2=0x7fbf0009be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:39.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.589+0000 7fbf071bc700 1 --2- 172.21.15.169:0/1254315057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf0015f360 0x7fbf0009be50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbf0006fda0 tx=0x7fbef8001a90 comp rx=0 tx=0).stop 2023-11-08T08:18:39.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.589+0000 7fbf071bc700 1 -- 172.21.15.169:0/1254315057 shutdown_connections 2023-11-08T08:18:39.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.589+0000 7fbf071bc700 1 --2- 172.21.15.169:0/1254315057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf0015f360 0x7fbf0009be50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.596 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.589+0000 7fbf071bc700 1 --2- 172.21.15.169:0/1254315057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf000d0c00 0x7fbf0009b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.589+0000 7fbf071bc700 1 --2- 172.21.15.169:0/1254315057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbee8003a20 0x7fbf000c6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.593+0000 7fbf071bc700 1 -- 172.21.15.169:0/1254315057 >> 172.21.15.169:0/1254315057 conn(0x7fbf000b6520 msgr2=0x7fbf000d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:39.597 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.593+0000 7fbf071bc700 1 -- 172.21.15.169:0/1254315057 shutdown_connections 2023-11-08T08:18:39.598 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.593+0000 7fbf071bc700 1 -- 172.21.15.169:0/1254315057 wait complete. 2023-11-08T08:18:39.615 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new f11bd326-3279-413b-84db-ceb549995008 6 2023-11-08T08:18:39.670 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:39.664+0000 7fee8dd942c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-08T08:18:39.675 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:39.669+0000 7f364a3552c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-08T08:18:39.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.777+0000 7f5974cfc700 1 Processor -- start 2023-11-08T08:18:39.782 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.777+0000 7f5974cfc700 1 -- start start 2023-11-08T08:18:39.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.781+0000 7f5974cfc700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f597006cb00 0x7f59700714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:39.786 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.781+0000 7f5974cfc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5970070970 0x7f5970069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:39.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.781+0000 7f5974cfc700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59700c9b20 con 0x7f5970159ff0 2023-11-08T08:18:39.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.781+0000 7f5974cfc700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59700cba40 con 0x7f5970070970 2023-11-08T08:18:39.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.781+0000 7f5974cfc700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59700cc2e0 con 0x7f597006cb00 2023-11-08T08:18:39.787 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.781+0000 7f596f7fe700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f5970159ff0 0x7f597006c3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:44256/0 (socket says 172.21.15.169:44256) 2023-11-08T08:18:39.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.781+0000 7f596f7fe700 1 -- 172.21.15.169:0/1669837222 learned_addr learned my addr 172.21.15.169:0/1669837222 (peer_addr_for_me v1:172.21.15.169:0/0) 2023-11-08T08:18:39.788 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.781+0000 7f596effd700 1 --2- 172.21.15.169:0/1669837222 >> v2:172.21.15.169:3300/0 conn(0x7f597006cb00 0x7f59700714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:39.790 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.781+0000 7f596e7fc700 1 -- 172.21.15.169:0/1669837222 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3145306593 0 0) 0x7f59700c9b20 con 0x7f5970159ff0 2023-11-08T08:18:39.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.781+0000 7f596e7fc700 1 -- 172.21.15.169:0/1669837222 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59580035d0 con 0x7f5970159ff0 2023-11-08T08:18:39.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.781+0000 7f596ffff700 1 --2- 172.21.15.169:0/1669837222 >> v2:172.21.15.40:3300/0 conn(0x7f5970070970 0x7f5970069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:39.791 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.785+0000 7f596effd700 1 -- 172.21.15.169:0/1669837222 >> v2:172.21.15.40:3300/0 conn(0x7f5970070970 msgr2=0x7f5970069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:39.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.785+0000 7f596effd700 1 --2- 172.21.15.169:0/1669837222 >> v2:172.21.15.40:3300/0 conn(0x7f5970070970 0x7f5970069ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.785+0000 7f596effd700 1 -- 172.21.15.169:0/1669837222 >> v1:172.21.15.169:6789/0 conn(0x7f5970159ff0 legacy=0x7f597006c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:39.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.785+0000 7f596effd700 1 -- 172.21.15.169:0/1669837222 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597015a3c0 con 0x7f597006cb00 2023-11-08T08:18:39.792 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.785+0000 7f596effd700 1 --2- 172.21.15.169:0/1669837222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f597006cb00 0x7f59700714c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f59700c29f0 tx=0x7f5964009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd96bd4ff0767f61 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:39.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.785+0000 7f596e7fc700 1 -- 172.21.15.169:0/1669837222 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5964011830 con 0x7f597006cb00 2023-11-08T08:18:39.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.785+0000 7f596e7fc700 1 -- 172.21.15.169:0/1669837222 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5964011990 con 0x7f597006cb00 2023-11-08T08:18:39.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.785+0000 7f596e7fc700 1 -- 172.21.15.169:0/1669837222 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5964011d00 con 0x7f597006cb00 2023-11-08T08:18:39.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.785+0000 7f5974cfc700 1 -- 172.21.15.169:0/1669837222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f597006cb00 msgr2=0x7f59700714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:39.793 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.785+0000 7f5974cfc700 1 --2- 172.21.15.169:0/1669837222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f597006cb00 0x7f59700714c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f59700c29f0 tx=0x7f5964009b30 comp rx=0 tx=0).stop 2023-11-08T08:18:39.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.785+0000 7f5974cfc700 1 -- 172.21.15.169:0/1669837222 shutdown_connections 2023-11-08T08:18:39.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.785+0000 7f5974cfc700 1 --2- 172.21.15.169:0/1669837222 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f597006cb00 0x7f59700714c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.785+0000 7f5974cfc700 1 --2- 172.21.15.169:0/1669837222 >> v2:172.21.15.40:3300/0 conn(0x7f5970070970 0x7f5970069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.785+0000 7f5974cfc700 1 -- 172.21.15.169:0/1669837222 >> 172.21.15.169:0/1669837222 conn(0x7f59700b0a20 msgr2=0x7f59700b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:39.794 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.785+0000 7f5974cfc700 1 -- 172.21.15.169:0/1669837222 shutdown_connections 2023-11-08T08:18:39.795 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.785+0000 7f5974cfc700 1 -- 172.21.15.169:0/1669837222 wait complete. 2023-11-08T08:18:39.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.789+0000 7f5974cfc700 1 Processor -- start 2023-11-08T08:18:39.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.789+0000 7f5974cfc700 1 -- start start 2023-11-08T08:18:39.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f5974cfc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f597006cb00 0x7f59700ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:39.796 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f5974cfc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5970070970 0x7f59700b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:39.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f5974cfc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5970159ff0 0x7f5970153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:39.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f5974cfc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59700b2ec0 con 0x7f597006cb00 2023-11-08T08:18:39.797 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f5974cfc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59700b2280 con 0x7f5970070970 2023-11-08T08:18:39.798 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f5974cfc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5970153bc0 con 0x7f5970159ff0 2023-11-08T08:18:39.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f596ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5970159ff0 0x7f5970153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:39.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f596effd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5970070970 0x7f59700b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:39.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f596ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5970159ff0 0x7f5970153680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:42038/0 (socket says 172.21.15.169:42038) 2023-11-08T08:18:39.800 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f596ffff700 1 -- 172.21.15.169:0/3503315888 learned_addr learned my addr 172.21.15.169:0/3503315888 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-08T08:18:39.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f596f7fe700 1 --2- 172.21.15.169:0/3503315888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f597006cb00 0x7f59700ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:39.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f596ffff700 1 -- 172.21.15.169:0/3503315888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5970070970 msgr2=0x7f59700b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:39.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f596ffff700 1 --2- 172.21.15.169:0/3503315888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5970070970 0x7f59700b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f596ffff700 1 -- 172.21.15.169:0/3503315888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f597006cb00 msgr2=0x7f59700ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:39.801 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f596ffff700 1 --2- 172.21.15.169:0/3503315888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f597006cb00 0x7f59700ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f596ffff700 1 -- 172.21.15.169:0/3503315888 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59640094b0 con 0x7f5970159ff0 2023-11-08T08:18:39.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f596ffff700 1 --2- 172.21.15.169:0/3503315888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5970159ff0 0x7f5970153680 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5968007e30 tx=0x7f596800e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:39.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f5957fff700 1 -- 172.21.15.169:0/3503315888 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596800ede0 con 0x7f5970159ff0 2023-11-08T08:18:39.802 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f5957fff700 1 -- 172.21.15.169:0/3503315888 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5968014420 con 0x7f5970159ff0 2023-11-08T08:18:39.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f5974cfc700 1 -- 172.21.15.169:0/3503315888 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59700c9840 con 0x7f5970159ff0 2023-11-08T08:18:39.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f5957fff700 1 -- 172.21.15.169:0/3503315888 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59680147d0 con 0x7f5970159ff0 2023-11-08T08:18:39.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f5974cfc700 1 -- 172.21.15.169:0/3503315888 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59700c9560 con 0x7f5970159ff0 2023-11-08T08:18:39.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f5957fff700 1 -- 172.21.15.169:0/3503315888 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f5968012050 con 0x7f5970159ff0 2023-11-08T08:18:39.803 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.793+0000 7f5957fff700 1 -- 172.21.15.169:0/3503315888 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f5968006e40 con 0x7f5970159ff0 2023-11-08T08:18:39.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.797+0000 7f5974cfc700 1 -- 172.21.15.169:0/3503315888 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59700cbc90 con 0x7f5970159ff0 2023-11-08T08:18:39.804 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.797+0000 7f5957fff700 1 -- 172.21.15.169:0/3503315888 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f596801e510 con 0x7f5970159ff0 2023-11-08T08:18:39.858 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:39.852+0000 7fee8dd942c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-08T08:18:39.865 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:39.861+0000 7f364a3552c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-08T08:18:39.877 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.873+0000 7f5974cfc700 1 -- 172.21.15.169:0/3503315888 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f11bd326-3279-413b-84db-ceb549995008", "id": 6} v 0) v1 -- 0x7f5970071ac0 con 0x7f5970159ff0 2023-11-08T08:18:39.888 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.885+0000 7f5957fff700 1 -- 172.21.15.169:0/3503315888 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f11bd326-3279-413b-84db-ceb549995008", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5968014c20 con 0x7f5970159ff0 2023-11-08T08:18:39.888 INFO:teuthology.orchestra.run.smithi169.stdout:6 2023-11-08T08:18:39.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.885+0000 7f5974cfc700 1 -- 172.21.15.169:0/3503315888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5970159ff0 msgr2=0x7f5970153680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:39.890 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.885+0000 7f5974cfc700 1 --2- 172.21.15.169:0/3503315888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5970159ff0 0x7f5970153680 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5968007e30 tx=0x7f596800e3c0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.885+0000 7f5974cfc700 1 -- 172.21.15.169:0/3503315888 shutdown_connections 2023-11-08T08:18:39.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.885+0000 7f5974cfc700 1 --2- 172.21.15.169:0/3503315888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5970159ff0 0x7f5970153680 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.885+0000 7f5974cfc700 1 --2- 172.21.15.169:0/3503315888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5970070970 0x7f59700b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.885+0000 7f5974cfc700 1 --2- 172.21.15.169:0/3503315888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f597006cb00 0x7f59700ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:39.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.885+0000 7f5974cfc700 1 -- 172.21.15.169:0/3503315888 >> 172.21.15.169:0/3503315888 conn(0x7f59700b0a20 msgr2=0x7f59700b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:39.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.885+0000 7f5974cfc700 1 -- 172.21.15.169:0/3503315888 shutdown_connections 2023-11-08T08:18:39.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:39.885+0000 7f5974cfc700 1 -- 172.21.15.169:0/3503315888 wait complete. 2023-11-08T08:18:39.910 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 4d36f429-63d5-4d0c-a6b1-0b43f54d7225 7 2023-11-08T08:18:40.086 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.081+0000 7fedd1066700 1 Processor -- start 2023-11-08T08:18:40.087 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.081+0000 7fedd1066700 1 -- start start 2023-11-08T08:18:40.090 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.085+0000 7fedd1066700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fedcc158840 0x7fedcc0ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:40.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.085+0000 7fedd1066700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedcc070790 0x7fedcc06a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:40.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.085+0000 7fedd1066700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedcc069bf0 con 0x7fedcc15c3b0 2023-11-08T08:18:40.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.085+0000 7fedd1066700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedcc15c890 con 0x7fedcc158840 2023-11-08T08:18:40.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.085+0000 7fedd1066700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc15cb70 con 0x7fedcc070790 2023-11-08T08:18:40.091 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.085+0000 7fedcbfff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fedcc158840 0x7fedcc0ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:40.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.085+0000 7fedd0865700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedcc070790 0x7fedcc06a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:40.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.085+0000 7fedcb7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fedcc15c3b0 0x7fedcc070080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.169:58494/0 (socket says 172.21.15.169:58494) 2023-11-08T08:18:40.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.085+0000 7fedcb7fe700 1 -- 172.21.15.169:0/2200785220 learned_addr learned my addr 172.21.15.169:0/2200785220 (peer_addr_for_me v1:172.21.15.169:0/0) 2023-11-08T08:18:40.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.085+0000 7fedcaffd700 1 -- 172.21.15.169:0/2200785220 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777809180 0 0) 0x7fedcc069bf0 con 0x7fedcc15c3b0 2023-11-08T08:18:40.092 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.085+0000 7fedcaffd700 1 -- 172.21.15.169:0/2200785220 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedb40057e0 con 0x7fedcc15c3b0 2023-11-08T08:18:40.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.085+0000 7fedd0865700 1 -- 172.21.15.169:0/2200785220 >> v2:172.21.15.169:3300/0 conn(0x7fedcc158840 msgr2=0x7fedcc0ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:40.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.085+0000 7fedd0865700 1 --2- 172.21.15.169:0/2200785220 >> v2:172.21.15.169:3300/0 conn(0x7fedcc158840 0x7fedcc0ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:40.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.085+0000 7fedd0865700 1 -- 172.21.15.169:0/2200785220 >> v1:172.21.15.40:6789/0 conn(0x7fedcc15c3b0 legacy=0x7fedcc070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:40.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.085+0000 7fedd0865700 1 -- 172.21.15.169:0/2200785220 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc15ce50 con 0x7fedcc070790 2023-11-08T08:18:40.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.085+0000 7fedcbfff700 1 --2- 172.21.15.169:0/2200785220 >> v2:172.21.15.169:3300/0 conn(0x7fedcc158840 0x7fedcc0ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:18:40.093 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.085+0000 7fedd0865700 1 --2- 172.21.15.169:0/2200785220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedcc070790 0x7fedcc06a080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fedc4009d90 tx=0x7fedc400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=427eb2358dd22ec5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:40.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.085+0000 7fedcaffd700 1 -- 172.21.15.169:0/2200785220 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc400e070 con 0x7fedcc070790 2023-11-08T08:18:40.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.089+0000 7fedcaffd700 1 -- 172.21.15.169:0/2200785220 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc400bb20 con 0x7fedcc070790 2023-11-08T08:18:40.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.089+0000 7fedcaffd700 1 -- 172.21.15.169:0/2200785220 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc400be90 con 0x7fedcc070790 2023-11-08T08:18:40.095 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.089+0000 7fedd1066700 1 -- 172.21.15.169:0/2200785220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedcc070790 msgr2=0x7fedcc06a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:40.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.089+0000 7fedd1066700 1 --2- 172.21.15.169:0/2200785220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedcc070790 0x7fedcc06a080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fedc4009d90 tx=0x7fedc400b330 comp rx=0 tx=0).stop 2023-11-08T08:18:40.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.089+0000 7fedd1066700 1 -- 172.21.15.169:0/2200785220 shutdown_connections 2023-11-08T08:18:40.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.089+0000 7fedd1066700 1 --2- 172.21.15.169:0/2200785220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedcc070790 0x7fedcc06a080 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:40.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.089+0000 7fedd1066700 1 --2- 172.21.15.169:0/2200785220 >> v2:172.21.15.169:3300/0 conn(0x7fedcc158840 0x7fedcc0ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:40.096 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.089+0000 7fedd1066700 1 -- 172.21.15.169:0/2200785220 >> 172.21.15.169:0/2200785220 conn(0x7fedcc0b0f80 msgr2=0x7fedcc0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:40.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.089+0000 7fedd1066700 1 -- 172.21.15.169:0/2200785220 shutdown_connections 2023-11-08T08:18:40.097 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.089+0000 7fedd1066700 1 -- 172.21.15.169:0/2200785220 wait complete. 2023-11-08T08:18:40.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.093+0000 7fedd1066700 1 Processor -- start 2023-11-08T08:18:40.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.093+0000 7fedd1066700 1 -- start start 2023-11-08T08:18:40.098 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.093+0000 7fedd1066700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedcc070790 0x7fedcc159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:40.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.093+0000 7fedd1066700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedcc158840 0x7fedcc0c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:40.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.093+0000 7fedd1066700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedcc15c3b0 0x7fedcc1578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:40.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.093+0000 7fedd1066700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0b45b0 con 0x7fedcc070790 2023-11-08T08:18:40.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.093+0000 7fedd1066700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0b1640 con 0x7fedcc15c3b0 2023-11-08T08:18:40.099 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.093+0000 7fedd1066700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc159980 con 0x7fedcc158840 2023-11-08T08:18:40.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.093+0000 7fedd0865700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedcc15c3b0 0x7fedcc1578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:40.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.093+0000 7fedd0865700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedcc15c3b0 0x7fedcc1578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:55232/0 (socket says 172.21.15.169:55232) 2023-11-08T08:18:40.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.093+0000 7fedd0865700 1 -- 172.21.15.169:0/2090255622 learned_addr learned my addr 172.21.15.169:0/2090255622 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-08T08:18:40.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.093+0000 7fedcb7fe700 1 --2- 172.21.15.169:0/2090255622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedcc158840 0x7fedcc0c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:40.100 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.093+0000 7fedcbfff700 1 --2- 172.21.15.169:0/2090255622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedcc070790 0x7fedcc159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:40.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.093+0000 7fedd0865700 1 -- 172.21.15.169:0/2090255622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedcc158840 msgr2=0x7fedcc0c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:40.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.093+0000 7fedd0865700 1 --2- 172.21.15.169:0/2090255622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedcc158840 0x7fedcc0c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:40.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.093+0000 7fedd0865700 1 -- 172.21.15.169:0/2090255622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedcc070790 msgr2=0x7fedcc159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:40.102 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.093+0000 7fedd0865700 1 --2- 172.21.15.169:0/2090255622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedcc070790 0x7fedcc159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:40.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.093+0000 7fedd0865700 1 -- 172.21.15.169:0/2090255622 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedbc00a650 con 0x7fedcc15c3b0 2023-11-08T08:18:40.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.097+0000 7fedd0865700 1 --2- 172.21.15.169:0/2090255622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedcc15c3b0 0x7fedcc1578e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fedc4014040 tx=0x7fedc4004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:40.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.097+0000 7fedc8ff9700 1 -- 172.21.15.169:0/2090255622 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc4004800 con 0x7fedcc15c3b0 2023-11-08T08:18:40.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.097+0000 7fedc8ff9700 1 -- 172.21.15.169:0/2090255622 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc4020070 con 0x7fedcc15c3b0 2023-11-08T08:18:40.103 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.097+0000 7fedd1066700 1 -- 172.21.15.169:0/2090255622 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc4009a70 con 0x7fedcc15c3b0 2023-11-08T08:18:40.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.097+0000 7fedc8ff9700 1 -- 172.21.15.169:0/2090255622 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc40276c0 con 0x7fedcc15c3b0 2023-11-08T08:18:40.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.097+0000 7fedd1066700 1 -- 172.21.15.169:0/2090255622 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedcc158c10 con 0x7fedcc15c3b0 2023-11-08T08:18:40.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.097+0000 7fedc8ff9700 1 -- 172.21.15.169:0/2090255622 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fedc402e070 con 0x7fedcc15c3b0 2023-11-08T08:18:40.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.097+0000 7fedc8ff9700 1 -- 172.21.15.169:0/2090255622 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fedc4022070 con 0x7fedcc15c3b0 2023-11-08T08:18:40.104 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.097+0000 7fedd1066700 1 -- 172.21.15.169:0/2090255622 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedcc15d130 con 0x7fedcc15c3b0 2023-11-08T08:18:40.105 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.101+0000 7fedc8ff9700 1 -- 172.21.15.169:0/2090255622 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fedc402c060 con 0x7fedcc15c3b0 2023-11-08T08:18:40.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.181+0000 7fedd1066700 1 -- 172.21.15.169:0/2090255622 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4d36f429-63d5-4d0c-a6b1-0b43f54d7225", "id": 7} v 0) v1 -- 0x7fedcc0adf30 con 0x7fedcc15c3b0 2023-11-08T08:18:40.197 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.193+0000 7fedc8ff9700 1 -- 172.21.15.169:0/2090255622 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4d36f429-63d5-4d0c-a6b1-0b43f54d7225", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fedc4027b10 con 0x7fedcc15c3b0 2023-11-08T08:18:40.198 INFO:teuthology.orchestra.run.smithi169.stdout:7 2023-11-08T08:18:40.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.197+0000 7fedd1066700 1 -- 172.21.15.169:0/2090255622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedcc15c3b0 msgr2=0x7fedcc1578e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:40.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.197+0000 7fedd1066700 1 --2- 172.21.15.169:0/2090255622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedcc15c3b0 0x7fedcc1578e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fedc4014040 tx=0x7fedc4004af0 comp rx=0 tx=0).stop 2023-11-08T08:18:40.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.197+0000 7fedd1066700 1 -- 172.21.15.169:0/2090255622 shutdown_connections 2023-11-08T08:18:40.200 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.197+0000 7fedd1066700 1 --2- 172.21.15.169:0/2090255622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedcc15c3b0 0x7fedcc1578e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:40.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.197+0000 7fedd1066700 1 --2- 172.21.15.169:0/2090255622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedcc158840 0x7fedcc0c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:40.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.197+0000 7fedd1066700 1 --2- 172.21.15.169:0/2090255622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedcc070790 0x7fedcc159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:40.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.197+0000 7fedd1066700 1 -- 172.21.15.169:0/2090255622 >> 172.21.15.169:0/2090255622 conn(0x7fedcc0b0f80 msgr2=0x7fedcc0aea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:40.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.197+0000 7fedd1066700 1 -- 172.21.15.169:0/2090255622 shutdown_connections 2023-11-08T08:18:40.201 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-08T08:18:40.197+0000 7fedd1066700 1 -- 172.21.15.169:0/2090255622 wait complete. 2023-11-08T08:18:40.220 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-08T08:18:40.221 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-08T08:18:40.228 INFO:tasks.ceph.osd.0:Started 2023-11-08T08:18:40.229 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-08T08:18:40.229 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-08T08:18:40.233 INFO:tasks.ceph.osd.1:Started 2023-11-08T08:18:40.234 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-08T08:18:40.234 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-08T08:18:40.238 INFO:tasks.ceph.osd.2:Started 2023-11-08T08:18:40.239 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-08T08:18:40.239 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-08T08:18:40.248 INFO:tasks.ceph.osd.3:Started 2023-11-08T08:18:40.248 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-08T08:18:40.248 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-08T08:18:40.255 INFO:tasks.ceph.osd.4:Started 2023-11-08T08:18:40.256 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-08T08:18:40.256 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-08T08:18:40.263 INFO:tasks.ceph.osd.5:Started 2023-11-08T08:18:40.263 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-08T08:18:40.264 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-08T08:18:40.267 INFO:tasks.ceph.osd.6:Started 2023-11-08T08:18:40.268 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-08T08:18:40.268 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-08T08:18:40.275 INFO:tasks.ceph.osd.7:Started 2023-11-08T08:18:40.276 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:18:40.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.468+0000 7f99b428a700 1 Processor -- start 2023-11-08T08:18:40.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.468+0000 7f99b428a700 1 -- start start 2023-11-08T08:18:40.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.472+0000 7f99b428a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99ac1534a0 0x7f99ac15d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:40.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.472+0000 7f99b428a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f99ac160920 0x7f99ac0b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:40.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.472+0000 7f99b428a700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99ac15d080 con 0x7f99ac0b6030 2023-11-08T08:18:40.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.472+0000 7f99b428a700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99ac153870 con 0x7f99ac160920 2023-11-08T08:18:40.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.472+0000 7f99b428a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac0b6450 con 0x7f99ac1534a0 2023-11-08T08:18:40.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.472+0000 7f99b2026700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99ac1534a0 0x7f99ac15d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:40.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.472+0000 7f99b2026700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99ac1534a0 0x7f99ac15d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:48784/0 (socket says 172.21.15.40:48784) 2023-11-08T08:18:40.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.472+0000 7f99b2026700 1 -- 172.21.15.40:0/767031523 learned_addr learned my addr 172.21.15.40:0/767031523 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:18:40.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.472+0000 7f99b1825700 1 --2- 172.21.15.40:0/767031523 >> v2:172.21.15.169:3300/0 conn(0x7f99ac160920 0x7f99ac0b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:40.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.472+0000 7f99b2026700 1 -- 172.21.15.40:0/767031523 >> v2:172.21.15.169:3300/0 conn(0x7f99ac160920 msgr2=0x7f99ac0b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:40.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.472+0000 7f99b2026700 1 --2- 172.21.15.40:0/767031523 >> v2:172.21.15.169:3300/0 conn(0x7f99ac160920 0x7f99ac0b5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:40.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.472+0000 7f99b2026700 1 -- 172.21.15.40:0/767031523 >> v1:172.21.15.169:6789/0 conn(0x7f99ac0b6030 legacy=0x7f99ac070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:40.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.472+0000 7f99b2026700 1 -- 172.21.15.40:0/767031523 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac0b53c0 con 0x7f99ac1534a0 2023-11-08T08:18:40.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.472+0000 7f99b2026700 1 --2- 172.21.15.40:0/767031523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99ac1534a0 0x7f99ac15d510 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009910 tx=0x7f99a8009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=496c5185a7083300 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:40.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.476+0000 7f99b1024700 1 -- 172.21.15.40:0/767031523 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8011830 con 0x7f99ac1534a0 2023-11-08T08:18:40.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.476+0000 7f99b1024700 1 -- 172.21.15.40:0/767031523 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a8011990 con 0x7f99ac1534a0 2023-11-08T08:18:40.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.476+0000 7f99b1024700 1 -- 172.21.15.40:0/767031523 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8011d40 con 0x7f99ac1534a0 2023-11-08T08:18:40.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.476+0000 7f99b428a700 1 -- 172.21.15.40:0/767031523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99ac1534a0 msgr2=0x7f99ac15d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:40.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.476+0000 7f99b428a700 1 --2- 172.21.15.40:0/767031523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99ac1534a0 0x7f99ac15d510 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009910 tx=0x7f99a8009c20 comp rx=0 tx=0).stop 2023-11-08T08:18:40.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.476+0000 7f99b428a700 1 -- 172.21.15.40:0/767031523 shutdown_connections 2023-11-08T08:18:40.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.476+0000 7f99b428a700 1 --2- 172.21.15.40:0/767031523 >> v2:172.21.15.169:3300/0 conn(0x7f99ac160920 0x7f99ac0b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:40.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.476+0000 7f99b428a700 1 --2- 172.21.15.40:0/767031523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99ac1534a0 0x7f99ac15d510 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:40.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.476+0000 7f99b428a700 1 -- 172.21.15.40:0/767031523 >> 172.21.15.40:0/767031523 conn(0x7f99ac06bfc0 msgr2=0x7f99ac06c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:40.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.476+0000 7f99b428a700 1 -- 172.21.15.40:0/767031523 shutdown_connections 2023-11-08T08:18:40.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.476+0000 7f99b428a700 1 -- 172.21.15.40:0/767031523 wait complete. 2023-11-08T08:18:40.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99b428a700 1 Processor -- start 2023-11-08T08:18:40.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99b428a700 1 -- start start 2023-11-08T08:18:40.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99b428a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99ac0b6030 0x7f99ac1455d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:40.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99b428a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99ac1534a0 0x7f99ac1e6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:40.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99b428a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99ac160920 0x7f99ac1561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:40.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99b428a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac145b10 con 0x7f99ac0b6030 2023-11-08T08:18:40.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99b428a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac1e7690 con 0x7f99ac160920 2023-11-08T08:18:40.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99b428a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac1e7970 con 0x7f99ac1534a0 2023-11-08T08:18:40.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99b2026700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99ac0b6030 0x7f99ac1455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:40.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99b2026700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99ac0b6030 0x7f99ac1455d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39446/0 (socket says 172.21.15.40:39446) 2023-11-08T08:18:40.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99b2026700 1 -- 172.21.15.40:0/2678541002 learned_addr learned my addr 172.21.15.40:0/2678541002 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:18:40.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99b1825700 1 --2- 172.21.15.40:0/2678541002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99ac1534a0 0x7f99ac1e6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:40.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99b2026700 1 -- 172.21.15.40:0/2678541002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99ac1534a0 msgr2=0x7f99ac1e6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:40.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99b2026700 1 --2- 172.21.15.40:0/2678541002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99ac1534a0 0x7f99ac1e6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:40.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99b2026700 1 -- 172.21.15.40:0/2678541002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99ac160920 msgr2=0x7f99ac1561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:40.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99b2026700 1 --2- 172.21.15.40:0/2678541002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99ac160920 0x7f99ac1561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:40.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99b2026700 1 -- 172.21.15.40:0/2678541002 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a8009570 con 0x7f99ac0b6030 2023-11-08T08:18:40.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99b2026700 1 --2- 172.21.15.40:0/2678541002 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99ac0b6030 0x7f99ac1455d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f99a8000c00 tx=0x7f99a8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:40.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99a2ffd700 1 -- 172.21.15.40:0/2678541002 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8004610 con 0x7f99ac0b6030 2023-11-08T08:18:40.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99b428a700 1 -- 172.21.15.40:0/2678541002 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ac15ca30 con 0x7f99ac0b6030 2023-11-08T08:18:40.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99a2ffd700 1 -- 172.21.15.40:0/2678541002 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a80036c0 con 0x7f99ac0b6030 2023-11-08T08:18:40.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99b428a700 1 -- 172.21.15.40:0/2678541002 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ac0b56a0 con 0x7f99ac0b6030 2023-11-08T08:18:40.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99a2ffd700 1 -- 172.21.15.40:0/2678541002 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8003a30 con 0x7f99ac0b6030 2023-11-08T08:18:40.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99a2ffd700 1 -- 172.21.15.40:0/2678541002 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f99a802d070 con 0x7f99ac0b6030 2023-11-08T08:18:40.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.480+0000 7f99a2ffd700 1 -- 172.21.15.40:0/2678541002 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f99a8021030 con 0x7f99ac0b6030 2023-11-08T08:18:40.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.484+0000 7f99b428a700 1 -- 172.21.15.40:0/2678541002 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99ac15cd10 con 0x7f99ac0b6030 2023-11-08T08:18:40.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.488+0000 7f99a2ffd700 1 -- 172.21.15.40:0/2678541002 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f99a8045b00 con 0x7f99ac0b6030 2023-11-08T08:18:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.556+0000 7f99b428a700 1 -- 172.21.15.40:0/2678541002 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99ac0726c0 con 0x7f99ac0b6030 2023-11-08T08:18:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.556+0000 7f99a2ffd700 1 -- 172.21.15.40:0/2678541002 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f99a8034070 con 0x7f99ac0b6030 2023-11-08T08:18:40.563 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:18:40.564 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":10,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:18:40.187770+0000","last_up_change":"0.000000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:18:40.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.560+0000 7f99b428a700 1 -- 172.21.15.40:0/2678541002 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99ac0b6030 msgr2=0x7f99ac1455d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:40.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.560+0000 7f99b428a700 1 --2- 172.21.15.40:0/2678541002 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99ac0b6030 0x7f99ac1455d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f99a8000c00 tx=0x7f99a8004770 comp rx=0 tx=0).stop 2023-11-08T08:18:40.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.560+0000 7f99b428a700 1 -- 172.21.15.40:0/2678541002 shutdown_connections 2023-11-08T08:18:40.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.560+0000 7f99b428a700 1 --2- 172.21.15.40:0/2678541002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99ac160920 0x7f99ac1561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:40.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.560+0000 7f99b428a700 1 --2- 172.21.15.40:0/2678541002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99ac1534a0 0x7f99ac1e6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:40.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.560+0000 7f99b428a700 1 --2- 172.21.15.40:0/2678541002 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99ac0b6030 0x7f99ac1455d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:40.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.560+0000 7f99b428a700 1 -- 172.21.15.40:0/2678541002 >> 172.21.15.40:0/2678541002 conn(0x7f99ac06bfc0 msgr2=0x7f99ac06a480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:40.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.560+0000 7f99b428a700 1 -- 172.21.15.40:0/2678541002 shutdown_connections 2023-11-08T08:18:40.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:40.560+0000 7f99b428a700 1 -- 172.21.15.40:0/2678541002 wait complete. 2023-11-08T08:18:40.585 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-08T08:18:40.586 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-08T08:18:40.641 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:40.636+0000 7fee8dd942c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-08T08:18:40.664 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:40.661+0000 7f364a3552c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-08T08:18:40.878 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:40.872+0000 7fee8dd942c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-08T08:18:40.898 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-11-08T08:18:40.892+0000 7f6980aee700 -1 Falling back to public interface 2023-11-08T08:18:40.911 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:40.909+0000 7f364a3552c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-08T08:18:40.923 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-11-08T08:18:40.916+0000 7f51c321a700 -1 Falling back to public interface 2023-11-08T08:18:40.933 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-11-08T08:18:40.928+0000 7fa765407700 -1 Falling back to public interface 2023-11-08T08:18:40.938 INFO:tasks.ceph.osd.6.smithi169.stderr:2023-11-08T08:18:40.933+0000 7f979b3df700 -1 Falling back to public interface 2023-11-08T08:18:40.950 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-11-08T08:18:40.944+0000 7fab5501a700 -1 Falling back to public interface 2023-11-08T08:18:40.953 INFO:tasks.ceph.osd.7.smithi169.stderr:2023-11-08T08:18:40.949+0000 7f9c3b988700 -1 Falling back to public interface 2023-11-08T08:18:40.954 INFO:tasks.ceph.osd.5.smithi169.stderr:2023-11-08T08:18:40.949+0000 7f5305738700 -1 Falling back to public interface 2023-11-08T08:18:40.954 INFO:tasks.ceph.osd.4.smithi169.stderr:2023-11-08T08:18:40.949+0000 7fbf231f1700 -1 Falling back to public interface 2023-11-08T08:18:41.109 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:41.104+0000 7fee8dd942c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-08T08:18:41.192 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:41.188+0000 7fee8dd942c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-08T08:18:41.193 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:41.189+0000 7f364a3552c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-08T08:18:41.278 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:41.273+0000 7f364a3552c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-08T08:18:41.392 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-08T08:18:41.541 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.536+0000 7f2428cd5700 1 Processor -- start 2023-11-08T08:18:41.542 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.536+0000 7f2428cd5700 1 -- start start 2023-11-08T08:18:41.543 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.536+0000 7f2428cd5700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f241c038560 0x7f241c03ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:41.543 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.536+0000 7f2428cd5700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f241c007360 con 0x7f241c03b160 2023-11-08T08:18:41.543 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.536+0000 7f2428cd5700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f241c053f70 con 0x7f241c0e29f0 2023-11-08T08:18:41.543 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.536+0000 7f2428cd5700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f241c0e2dc0 con 0x7f241c038560 2023-11-08T08:18:41.545 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f2423fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f241c03b160 0x7f241c057210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52288/0 (socket says 172.21.15.40:52288) 2023-11-08T08:18:41.545 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f2423fff700 1 -- 172.21.15.40:0/3326465919 learned_addr learned my addr 172.21.15.40:0/3326465919 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:18:41.545 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f2422ffd700 1 --2- 172.21.15.40:0/3326465919 >> v2:172.21.15.169:3300/0 conn(0x7f241c038560 0x7f241c03ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:41.545 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24227fc700 1 -- 172.21.15.40:0/3326465919 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3306921927 0 0) 0x7f241c007360 con 0x7f241c03b160 2023-11-08T08:18:41.546 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24227fc700 1 -- 172.21.15.40:0/3326465919 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24140035d0 con 0x7f241c03b160 2023-11-08T08:18:41.546 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24227fc700 1 -- 172.21.15.40:0/3326465919 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2952029784 0 0) 0x7f24140035d0 con 0x7f241c03b160 2023-11-08T08:18:41.546 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24227fc700 1 -- 172.21.15.40:0/3326465919 >> v2:172.21.15.169:3300/0 conn(0x7f241c038560 msgr2=0x7f241c03ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:41.546 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24227fc700 1 --2- 172.21.15.40:0/3326465919 >> v2:172.21.15.169:3300/0 conn(0x7f241c038560 0x7f241c03ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:41.546 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24227fc700 1 -- 172.21.15.40:0/3326465919 >> v1:172.21.15.169:6789/0 conn(0x7f241c0e29f0 legacy=0x7f241c055490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:41.547 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24227fc700 1 -- 172.21.15.40:0/3326465919 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c03b5a0 con 0x7f241c03b160 2023-11-08T08:18:41.547 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24227fc700 1 -- 172.21.15.40:0/3326465919 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f24240008d0 con 0x7f241c03b160 2023-11-08T08:18:41.547 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24227fc700 1 -- 172.21.15.40:0/3326465919 >> v1:172.21.15.40:6789/0 conn(0x7f241c03b160 legacy=0x7f241c057210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:41.547 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24227fc700 1 --2- 172.21.15.40:0/3326465919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2414003a20 0x7f2414003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:41.547 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24227fc700 1 --2- 172.21.15.40:0/3326465919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24140043b0 0x7f2414004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:41.548 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24227fc700 1 --2- 172.21.15.40:0/3326465919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2414005110 0x7f241400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:41.548 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24227fc700 1 -- 172.21.15.40:0/3326465919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c0e2dc0 con 0x7f2414003a20 2023-11-08T08:18:41.548 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24227fc700 1 -- 172.21.15.40:0/3326465919 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241400c050 con 0x7f24140043b0 2023-11-08T08:18:41.548 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24227fc700 1 -- 172.21.15.40:0/3326465919 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241400c210 con 0x7f2414005110 2023-11-08T08:18:41.548 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24237fe700 1 --2- 172.21.15.40:0/3326465919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2414003a20 0x7f2414003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:41.550 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f2422ffd700 1 --2- 172.21.15.40:0/3326465919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24140043b0 0x7f2414004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:41.550 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f2423fff700 1 --2- 172.21.15.40:0/3326465919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2414005110 0x7f241400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:41.551 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24237fe700 1 -- 172.21.15.40:0/3326465919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2414005110 msgr2=0x7f241400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:41.551 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24237fe700 1 --2- 172.21.15.40:0/3326465919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2414005110 0x7f241400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:41.551 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24237fe700 1 -- 172.21.15.40:0/3326465919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24140043b0 msgr2=0x7f2414004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:41.551 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24237fe700 1 --2- 172.21.15.40:0/3326465919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24140043b0 0x7f2414004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:41.552 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24237fe700 1 -- 172.21.15.40:0/3326465919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24140035d0 con 0x7f2414003a20 2023-11-08T08:18:41.552 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f2423fff700 1 --2- 172.21.15.40:0/3326465919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2414005110 0x7f241400ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:18:41.552 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f2423fff700 1 -- 172.21.15.40:0/3326465919 reap_dead start 2023-11-08T08:18:41.552 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24237fe700 1 --2- 172.21.15.40:0/3326465919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2414003a20 0x7f2414003e30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f24180017e0 tx=0x7f24140058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2c927a49d87b71 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:41.552 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24227fc700 1 -- 172.21.15.40:0/3326465919 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24180052d0 con 0x7f2414003a20 2023-11-08T08:18:41.552 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24227fc700 1 -- 172.21.15.40:0/3326465919 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2418005430 con 0x7f2414003a20 2023-11-08T08:18:41.553 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.540+0000 7f24227fc700 1 -- 172.21.15.40:0/3326465919 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24180055a0 con 0x7f2414003a20 2023-11-08T08:18:41.553 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.544+0000 7f2428cd5700 1 -- 172.21.15.40:0/3326465919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2414003a20 msgr2=0x7f2414003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:41.553 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.544+0000 7f2428cd5700 1 --2- 172.21.15.40:0/3326465919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2414003a20 0x7f2414003e30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f24180017e0 tx=0x7f24140058a0 comp rx=0 tx=0).stop 2023-11-08T08:18:41.553 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.544+0000 7f2428cd5700 1 -- 172.21.15.40:0/3326465919 shutdown_connections 2023-11-08T08:18:41.553 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.544+0000 7f2428cd5700 1 --2- 172.21.15.40:0/3326465919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2414003a20 0x7f2414003e30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:41.554 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.544+0000 7f2428cd5700 1 -- 172.21.15.40:0/3326465919 >> 172.21.15.40:0/3326465919 conn(0x7f241c055c20 msgr2=0x7f241c056020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:41.554 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.544+0000 7f2428cd5700 1 -- 172.21.15.40:0/3326465919 shutdown_connections 2023-11-08T08:18:41.554 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.544+0000 7f2428cd5700 1 -- 172.21.15.40:0/3326465919 wait complete. 2023-11-08T08:18:41.554 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.544+0000 7f2428cd5700 1 Processor -- start 2023-11-08T08:18:41.556 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.544+0000 7f2428cd5700 1 -- start start 2023-11-08T08:18:41.557 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f2428cd5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2414003a20 0x7f241c0e33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:41.557 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f2428cd5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f241c04cfe0 0x7f241c177850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:41.557 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f2428cd5700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f241c0e55a0 0x7f241c0cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:41.557 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f2428cd5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c051bb0 con 0x7f2414003a20 2023-11-08T08:18:41.557 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f2428cd5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c0512c0 con 0x7f241c04cfe0 2023-11-08T08:18:41.558 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f2428cd5700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241c0e5f40 con 0x7f241c0e55a0 2023-11-08T08:18:41.558 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f24237fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2414003a20 0x7f241c0e33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:41.558 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f24237fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2414003a20 0x7f241c0e33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39470/0 (socket says 172.21.15.40:39470) 2023-11-08T08:18:41.558 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f24237fe700 1 -- 172.21.15.40:0/42636380 learned_addr learned my addr 172.21.15.40:0/42636380 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:18:41.558 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f2422ffd700 1 --2- 172.21.15.40:0/42636380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f241c04cfe0 0x7f241c177850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:41.559 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f2423fff700 1 --2- 172.21.15.40:0/42636380 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f241c0e55a0 0x7f241c0cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:41.559 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f24237fe700 1 -- 172.21.15.40:0/42636380 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f241c0e55a0 msgr2=0x7f241c0cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:41.559 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f24237fe700 1 --2- 172.21.15.40:0/42636380 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f241c0e55a0 0x7f241c0cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:41.559 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f24237fe700 1 -- 172.21.15.40:0/42636380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f241c04cfe0 msgr2=0x7f241c177850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:41.559 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f24237fe700 1 --2- 172.21.15.40:0/42636380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f241c04cfe0 0x7f241c177850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:41.560 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f24237fe700 1 -- 172.21.15.40:0/42636380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c007080 con 0x7f2414003a20 2023-11-08T08:18:41.560 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f24237fe700 1 --2- 172.21.15.40:0/42636380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2414003a20 0x7f241c0e33a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2418015040 tx=0x7f2418001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:41.560 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f240ffff700 1 -- 172.21.15.40:0/42636380 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24180055a0 con 0x7f2414003a20 2023-11-08T08:18:41.560 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f240ffff700 1 -- 172.21.15.40:0/42636380 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2418003e60 con 0x7f2414003a20 2023-11-08T08:18:41.560 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f2428cd5700 1 -- 172.21.15.40:0/42636380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241c006dc0 con 0x7f2414003a20 2023-11-08T08:18:41.560 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f240ffff700 1 -- 172.21.15.40:0/42636380 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241801d6c0 con 0x7f2414003a20 2023-11-08T08:18:41.561 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f2428cd5700 1 -- 172.21.15.40:0/42636380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241c053920 con 0x7f2414003a20 2023-11-08T08:18:41.561 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.548+0000 7f240ffff700 1 -- 172.21.15.40:0/42636380 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f241802f070 con 0x7f2414003a20 2023-11-08T08:18:41.561 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.552+0000 7f240ffff700 1 -- 172.21.15.40:0/42636380 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f241801d820 con 0x7f2414003a20 2023-11-08T08:18:41.561 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.552+0000 7f2428cd5700 1 -- 172.21.15.40:0/42636380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f241c005570 con 0x7f2414003a20 2023-11-08T08:18:41.562 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.556+0000 7f240ffff700 1 -- 172.21.15.40:0/42636380 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f241802d060 con 0x7f2414003a20 2023-11-08T08:18:41.628 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.624+0000 7f2428cd5700 1 -- 172.21.15.40:0/42636380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f241c0077b0 con 0x7f2414003a20 2023-11-08T08:18:41.630 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.624+0000 7f240ffff700 1 -- 172.21.15.40:0/42636380 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f2418023070 con 0x7f2414003a20 2023-11-08T08:18:41.630 INFO:teuthology.misc.health.smithi040.stdout: 2023-11-08T08:18:41.631 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":10,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:18:40.187770+0000","last_up_change":"0.000000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:18:41.632 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.628+0000 7f2428cd5700 1 -- 172.21.15.40:0/42636380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2414003a20 msgr2=0x7f241c0e33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:41.632 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.628+0000 7f2428cd5700 1 --2- 172.21.15.40:0/42636380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2414003a20 0x7f241c0e33a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2418015040 tx=0x7f2418001120 comp rx=0 tx=0).stop 2023-11-08T08:18:41.632 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.628+0000 7f2428cd5700 1 -- 172.21.15.40:0/42636380 shutdown_connections 2023-11-08T08:18:41.632 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.628+0000 7f2428cd5700 1 --2- 172.21.15.40:0/42636380 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f241c0e55a0 0x7f241c0cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:41.632 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.628+0000 7f2428cd5700 1 --2- 172.21.15.40:0/42636380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f241c04cfe0 0x7f241c177850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:41.633 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.628+0000 7f2428cd5700 1 --2- 172.21.15.40:0/42636380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2414003a20 0x7f241c0e33a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:41.633 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.628+0000 7f2428cd5700 1 -- 172.21.15.40:0/42636380 >> 172.21.15.40:0/42636380 conn(0x7f241c055c20 msgr2=0x7f241c03e150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:41.633 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.628+0000 7f2428cd5700 1 -- 172.21.15.40:0/42636380 shutdown_connections 2023-11-08T08:18:41.634 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:41.628+0000 7f2428cd5700 1 -- 172.21.15.40:0/42636380 wait complete. 2023-11-08T08:18:41.649 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-08T08:18:41.975 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:41.968+0000 7fee8dd942c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-08T08:18:42.002 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:41.997+0000 7f364a3552c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-08T08:18:42.077 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:42.072+0000 7fee8dd942c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-08T08:18:42.102 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:42.097+0000 7f364a3552c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-08T08:18:42.255 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:42.248+0000 7fee8dd942c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-08T08:18:42.272 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:42.269+0000 7f364a3552c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-08T08:18:42.350 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:42.344+0000 7fee8dd942c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-08T08:18:42.363 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:42.361+0000 7f364a3552c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-08T08:18:42.628 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:42.624+0000 7fee8dd942c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-08T08:18:42.643 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:42.641+0000 7f364a3552c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-08T08:18:42.856 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:42.853+0000 7f364a3552c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-08T08:18:42.862 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:42.856+0000 7fee8dd942c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-08T08:18:42.950 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:42.945+0000 7f364a3552c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-08T08:18:42.967 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:42.960+0000 7fee8dd942c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-08T08:18:43.852 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:43.849+0000 7f364a3552c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-08T08:18:43.992 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-11-08T08:18:43.988+0000 7fee8dd942c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-08T08:18:44.892 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:44.889+0000 7f3630bbd700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-08T08:18:44.894 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-08T08:18:44.889+0000 7f3630bbd700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-08T08:18:44.927 INFO:tasks.ceph.osd.5.smithi169.stderr:2023-11-08T08:18:44.925+0000 7f5305738700 -1 osd.5 0 log_to_monitors true 2023-11-08T08:18:44.928 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-11-08T08:18:44.924+0000 7fab5501a700 -1 osd.2 0 log_to_monitors true 2023-11-08T08:18:44.952 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-11-08T08:18:44.948+0000 7f51c321a700 -1 osd.0 0 log_to_monitors true 2023-11-08T08:18:44.958 INFO:tasks.ceph.osd.7.smithi169.stderr:2023-11-08T08:18:44.953+0000 7f9c3b988700 -1 osd.7 0 log_to_monitors true 2023-11-08T08:18:45.162 INFO:tasks.ceph.osd.6.smithi169.stderr:2023-11-08T08:18:45.157+0000 7f979b3df700 -1 osd.6 0 log_to_monitors true 2023-11-08T08:18:45.163 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-11-08T08:18:45.156+0000 7fa765407700 -1 osd.3 0 log_to_monitors true 2023-11-08T08:18:45.165 INFO:tasks.ceph.osd.4.smithi169.stderr:2023-11-08T08:18:45.161+0000 7fbf231f1700 -1 osd.4 0 log_to_monitors true 2023-11-08T08:18:45.168 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-11-08T08:18:45.164+0000 7f6980aee700 -1 osd.1 0 log_to_monitors true 2023-11-08T08:18:48.456 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-08T08:18:48.660 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.655+0000 7fdd3135d700 1 Processor -- start 2023-11-08T08:18:48.660 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.655+0000 7fdd3135d700 1 -- start start 2023-11-08T08:18:48.663 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.655+0000 7fdd3135d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdd2400cb20 0x7fdd24053020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:48.664 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.655+0000 7fdd2bfff700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fdd240e0750 0x7fdd24039d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:35916/0 (socket says 172.21.15.40:35916) 2023-11-08T08:18:48.664 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.655+0000 7fdd2b7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdd240514c0 0x7fdd2400c410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41082/0 (socket says 172.21.15.40:41082) 2023-11-08T08:18:48.664 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.655+0000 7fdd2bfff700 1 -- 172.21.15.40:0/3754706316 learned_addr learned my addr 172.21.15.40:0/3754706316 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:18:48.664 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.655+0000 7fdd30b5c700 1 --2- 172.21.15.40:0/3754706316 >> v2:172.21.15.169:3300/0 conn(0x7fdd2400cb20 0x7fdd24053020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:48.664 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.659+0000 7fdd3135d700 1 -- 172.21.15.40:0/3754706316 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd240554f0 con 0x7fdd240514c0 2023-11-08T08:18:48.665 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.659+0000 7fdd2affd700 1 -- 172.21.15.40:0/3754706316 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3919288870 0 0) 0x7fdd240554f0 con 0x7fdd240514c0 2023-11-08T08:18:48.666 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.659+0000 7fdd3135d700 1 -- 172.21.15.40:0/3754706316 --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd240e0b20 con 0x7fdd240e0750 2023-11-08T08:18:48.666 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.659+0000 7fdd3135d700 1 -- 172.21.15.40:0/3754706316 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd2403a460 con 0x7fdd2400cb20 2023-11-08T08:18:48.666 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.659+0000 7fdd2affd700 1 -- 172.21.15.40:0/3754706316 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd1c0035d0 con 0x7fdd240514c0 2023-11-08T08:18:48.666 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.659+0000 7fdd2affd700 1 -- 172.21.15.40:0/3754706316 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 368960829 0 0) 0x7fdd240e0b20 con 0x7fdd240e0750 2023-11-08T08:18:48.666 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.659+0000 7fdd2affd700 1 -- 172.21.15.40:0/3754706316 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd240554f0 con 0x7fdd240e0750 2023-11-08T08:18:48.667 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.659+0000 7fdd30b5c700 1 -- 172.21.15.40:0/3754706316 >> v1:172.21.15.169:6789/0 conn(0x7fdd240e0750 legacy=0x7fdd24039d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:48.667 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.659+0000 7fdd30b5c700 1 -- 172.21.15.40:0/3754706316 >> v1:172.21.15.40:6789/0 conn(0x7fdd240514c0 legacy=0x7fdd2400c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:48.667 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.659+0000 7fdd30b5c700 1 -- 172.21.15.40:0/3754706316 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd24055790 con 0x7fdd2400cb20 2023-11-08T08:18:48.668 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.659+0000 7fdd30b5c700 1 --2- 172.21.15.40:0/3754706316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2400cb20 0x7fdd24053020 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c04ed50 tx=0x7fdd2c0008d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a0faec12ef5f53b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:48.669 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.663+0000 7fdd2affd700 1 -- 172.21.15.40:0/3754706316 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c0729f0 con 0x7fdd2400cb20 2023-11-08T08:18:48.669 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.663+0000 7fdd2affd700 1 -- 172.21.15.40:0/3754706316 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2c072b50 con 0x7fdd2400cb20 2023-11-08T08:18:48.669 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.663+0000 7fdd2affd700 1 -- 172.21.15.40:0/3754706316 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c066b40 con 0x7fdd2400cb20 2023-11-08T08:18:48.670 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.663+0000 7fdd3135d700 1 -- 172.21.15.40:0/3754706316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2400cb20 msgr2=0x7fdd24053020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:48.670 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.663+0000 7fdd3135d700 1 --2- 172.21.15.40:0/3754706316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2400cb20 0x7fdd24053020 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c04ed50 tx=0x7fdd2c0008d0 comp rx=0 tx=0).stop 2023-11-08T08:18:48.671 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.663+0000 7fdd3135d700 1 -- 172.21.15.40:0/3754706316 shutdown_connections 2023-11-08T08:18:48.671 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.663+0000 7fdd3135d700 1 --2- 172.21.15.40:0/3754706316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2400cb20 0x7fdd24053020 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:48.671 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.663+0000 7fdd3135d700 1 -- 172.21.15.40:0/3754706316 >> 172.21.15.40:0/3754706316 conn(0x7fdd24056f00 msgr2=0x7fdd24057300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:48.671 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.663+0000 7fdd3135d700 1 -- 172.21.15.40:0/3754706316 shutdown_connections 2023-11-08T08:18:48.672 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.663+0000 7fdd3135d700 1 -- 172.21.15.40:0/3754706316 wait complete. 2023-11-08T08:18:48.673 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.667+0000 7fdd3135d700 1 Processor -- start 2023-11-08T08:18:48.673 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.667+0000 7fdd3135d700 1 -- start start 2023-11-08T08:18:48.676 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.671+0000 7fdd3135d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2400cb20 0x7fdd24050280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:48.676 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.671+0000 7fdd3135d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd240514c0 0x7fdd24038bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:48.676 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.671+0000 7fdd3135d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd240e0750 0x7fdd240390f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:48.676 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.671+0000 7fdd3135d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2404e130 con 0x7fdd240e0750 2023-11-08T08:18:48.677 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.671+0000 7fdd3135d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd240507c0 con 0x7fdd2400cb20 2023-11-08T08:18:48.677 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.671+0000 7fdd3135d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd24173f50 con 0x7fdd240514c0 2023-11-08T08:18:48.678 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.671+0000 7fdd30b5c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd240e0750 0x7fdd240390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:48.678 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.671+0000 7fdd2bfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2400cb20 0x7fdd24050280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:48.678 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.671+0000 7fdd2bfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2400cb20 0x7fdd24050280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:60354/0 (socket says 172.21.15.40:60354) 2023-11-08T08:18:48.678 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.671+0000 7fdd2bfff700 1 -- 172.21.15.40:0/2919866916 learned_addr learned my addr 172.21.15.40:0/2919866916 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:18:48.678 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.671+0000 7fdd2b7fe700 1 --2- 172.21.15.40:0/2919866916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd240514c0 0x7fdd24038bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:48.679 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.671+0000 7fdd2bfff700 1 -- 172.21.15.40:0/2919866916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd240514c0 msgr2=0x7fdd24038bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:48.680 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.671+0000 7fdd2bfff700 1 --2- 172.21.15.40:0/2919866916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd240514c0 0x7fdd24038bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:48.680 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.671+0000 7fdd2bfff700 1 -- 172.21.15.40:0/2919866916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd240e0750 msgr2=0x7fdd240390f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:48.680 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.671+0000 7fdd2bfff700 1 --2- 172.21.15.40:0/2919866916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd240e0750 0x7fdd240390f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:48.680 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.671+0000 7fdd2bfff700 1 -- 172.21.15.40:0/2919866916 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd24055d50 con 0x7fdd2400cb20 2023-11-08T08:18:48.681 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.671+0000 7fdd30b5c700 1 --2- 172.21.15.40:0/2919866916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd240e0750 0x7fdd240390f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:18:48.681 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.671+0000 7fdd2bfff700 1 --2- 172.21.15.40:0/2919866916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2400cb20 0x7fdd24050280 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000c700 tx=0x7fdd2000ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:48.681 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.675+0000 7fdd28ff9700 1 -- 172.21.15.40:0/2919866916 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20005d90 con 0x7fdd2400cb20 2023-11-08T08:18:48.681 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.675+0000 7fdd28ff9700 1 -- 172.21.15.40:0/2919866916 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd200070d0 con 0x7fdd2400cb20 2023-11-08T08:18:48.681 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.675+0000 7fdd28ff9700 1 -- 172.21.15.40:0/2919866916 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20007440 con 0x7fdd2400cb20 2023-11-08T08:18:48.681 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.675+0000 7fdd3135d700 1 -- 172.21.15.40:0/2919866916 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd24055a70 con 0x7fdd2400cb20 2023-11-08T08:18:48.682 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.675+0000 7fdd3135d700 1 -- 172.21.15.40:0/2919866916 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2400e740 con 0x7fdd2400cb20 2023-11-08T08:18:48.686 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.675+0000 7fdd3135d700 1 -- 172.21.15.40:0/2919866916 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd24005570 con 0x7fdd2400cb20 2023-11-08T08:18:48.687 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.679+0000 7fdd28ff9700 1 -- 172.21.15.40:0/2919866916 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 4) v1 ==== 83919+0+0 (secure 0 0 0) 0x7fdd20019030 con 0x7fdd2400cb20 2023-11-08T08:18:48.690 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.683+0000 7fdd28ff9700 1 --2- 172.21.15.40:0/2919866916 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdd10064e90 0x7fdd10067340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:48.690 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.683+0000 7fdd2b7fe700 1 --2- 172.21.15.40:0/2919866916 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdd10064e90 0x7fdd10067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:48.691 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.683+0000 7fdd28ff9700 1 -- 172.21.15.40:0/2919866916 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fdd200802f0 con 0x7fdd2400cb20 2023-11-08T08:18:48.700 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.695+0000 7fdd2b7fe700 1 --2- 172.21.15.40:0/2919866916 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdd10064e90 0x7fdd10067340 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdd240db020 tx=0x7fdd18007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:48.707 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:48.703+0000 7fdd28ff9700 1 -- 172.21.15.40:0/2919866916 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fdd2004f410 con 0x7fdd2400cb20 2023-11-08T08:18:49.037 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:49.031+0000 7fdd3135d700 1 -- 172.21.15.40:0/2919866916 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd24005c70 con 0x7fdd2400cb20 2023-11-08T08:18:49.041 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:49.035+0000 7fdd28ff9700 1 -- 172.21.15.40:0/2919866916 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fdd20017090 con 0x7fdd2400cb20 2023-11-08T08:18:49.041 INFO:teuthology.misc.health.smithi040.stdout: 2023-11-08T08:18:49.041 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":12,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:18:46.940813+0000","last_up_change":"0.000000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:18:49.045 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:49.039+0000 7fdd3135d700 1 -- 172.21.15.40:0/2919866916 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdd10064e90 msgr2=0x7fdd10067340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:49.045 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:49.039+0000 7fdd3135d700 1 --2- 172.21.15.40:0/2919866916 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdd10064e90 0x7fdd10067340 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdd240db020 tx=0x7fdd18007de0 comp rx=0 tx=0).stop 2023-11-08T08:18:49.046 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:49.039+0000 7fdd3135d700 1 -- 172.21.15.40:0/2919866916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2400cb20 msgr2=0x7fdd24050280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:49.046 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:49.039+0000 7fdd3135d700 1 --2- 172.21.15.40:0/2919866916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2400cb20 0x7fdd24050280 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000c700 tx=0x7fdd2000ca10 comp rx=0 tx=0).stop 2023-11-08T08:18:49.046 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:49.039+0000 7fdd3135d700 1 -- 172.21.15.40:0/2919866916 shutdown_connections 2023-11-08T08:18:49.047 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:49.039+0000 7fdd3135d700 1 --2- 172.21.15.40:0/2919866916 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdd10064e90 0x7fdd10067340 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:49.047 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:49.039+0000 7fdd3135d700 1 --2- 172.21.15.40:0/2919866916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd240e0750 0x7fdd240390f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:49.048 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:49.039+0000 7fdd3135d700 1 --2- 172.21.15.40:0/2919866916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd240514c0 0x7fdd24038bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:49.048 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:49.039+0000 7fdd3135d700 1 --2- 172.21.15.40:0/2919866916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2400cb20 0x7fdd24050280 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:49.049 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:49.039+0000 7fdd3135d700 1 -- 172.21.15.40:0/2919866916 >> 172.21.15.40:0/2919866916 conn(0x7fdd24056f00 msgr2=0x7fdd240dc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:49.050 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:49.039+0000 7fdd3135d700 1 -- 172.21.15.40:0/2919866916 shutdown_connections 2023-11-08T08:18:49.050 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:49.039+0000 7fdd3135d700 1 -- 172.21.15.40:0/2919866916 wait complete. 2023-11-08T08:18:49.071 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-08T08:18:49.367 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-11-08T08:18:49.363+0000 7f697d2b4700 -1 osd.1 0 waiting for initial osdmap 2023-11-08T08:18:49.378 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-11-08T08:18:49.371+0000 7f69743dc700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T08:18:49.443 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-11-08T08:18:49.435+0000 7fa761246700 -1 osd.3 0 waiting for initial osdmap 2023-11-08T08:18:49.451 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-11-08T08:18:49.443+0000 7fa758bf0700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T08:18:49.469 INFO:tasks.ceph.osd.5.smithi169.stderr:2023-11-08T08:18:49.465+0000 7f5301efe700 -1 osd.5 0 waiting for initial osdmap 2023-11-08T08:18:49.474 INFO:tasks.ceph.osd.4.smithi169.stderr:2023-11-08T08:18:49.469+0000 7fbf1f9b7700 -1 osd.4 0 waiting for initial osdmap 2023-11-08T08:18:49.477 INFO:tasks.ceph.osd.5.smithi169.stderr:2023-11-08T08:18:49.473+0000 7f52f8ff0700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T08:18:49.482 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-11-08T08:18:49.475+0000 7f51bf059700 -1 osd.0 0 waiting for initial osdmap 2023-11-08T08:18:49.483 INFO:tasks.ceph.osd.7.smithi169.stderr:2023-11-08T08:18:49.481+0000 7f9c377c7700 -1 osd.7 0 waiting for initial osdmap 2023-11-08T08:18:49.485 INFO:tasks.ceph.osd.4.smithi169.stderr:2023-11-08T08:18:49.481+0000 7fbf169f0700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T08:18:49.489 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-11-08T08:18:49.483+0000 7f51b69f0700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T08:18:49.490 INFO:tasks.ceph.osd.6.smithi169.stderr:2023-11-08T08:18:49.485+0000 7f979721e700 -1 osd.6 0 waiting for initial osdmap 2023-11-08T08:18:49.493 INFO:tasks.ceph.osd.7.smithi169.stderr:2023-11-08T08:18:49.489+0000 7f9c2f1f0700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T08:18:49.496 INFO:tasks.ceph.osd.6.smithi169.stderr:2023-11-08T08:18:49.493+0000 7f978ebf0700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T08:18:49.525 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-11-08T08:18:49.519+0000 7fab517e0700 -1 osd.2 0 waiting for initial osdmap 2023-11-08T08:18:49.535 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-11-08T08:18:49.527+0000 7fab487f0700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T08:18:55.878 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-08T08:18:56.037 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.031+0000 7f49aaabc700 1 Processor -- start 2023-11-08T08:18:56.038 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.031+0000 7f49aaabc700 1 -- start start 2023-11-08T08:18:56.039 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.031+0000 7f49aaabc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49a415acd0 0x7f49a406ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:56.039 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.031+0000 7f49aaabc700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f49a406cad0 0x7f49a40ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:56.039 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.035+0000 7f49aaabc700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a406a890 con 0x7f49a406b260 2023-11-08T08:18:56.040 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.035+0000 7f49aaabc700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49a415b0a0 con 0x7f49a415acd0 2023-11-08T08:18:56.040 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.035+0000 7f49aaabc700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49a4161b10 con 0x7f49a406cad0 2023-11-08T08:18:56.041 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.035+0000 7f49a9aba700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49a415acd0 0x7f49a406ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:56.041 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.035+0000 7f49a9aba700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49a415acd0 0x7f49a406ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49954/0 (socket says 172.21.15.40:49954) 2023-11-08T08:18:56.041 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.035+0000 7f49a9aba700 1 -- 172.21.15.40:0/3175212130 learned_addr learned my addr 172.21.15.40:0/3175212130 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:18:56.041 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.035+0000 7f49aa2bb700 1 --2- 172.21.15.40:0/3175212130 >> v2:172.21.15.169:3300/0 conn(0x7f49a406cad0 0x7f49a40ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:56.042 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.035+0000 7f49a9aba700 1 -- 172.21.15.40:0/3175212130 >> v2:172.21.15.169:3300/0 conn(0x7f49a406cad0 msgr2=0x7f49a40ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:56.042 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.035+0000 7f49a9aba700 1 --2- 172.21.15.40:0/3175212130 >> v2:172.21.15.169:3300/0 conn(0x7f49a406cad0 0x7f49a40ad7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:56.042 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.035+0000 7f49a9aba700 1 -- 172.21.15.40:0/3175212130 >> v1:172.21.15.169:6789/0 conn(0x7f49a406b260 legacy=0x7f49a406c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:56.042 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.035+0000 7f49a9aba700 1 -- 172.21.15.40:0/3175212130 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a4069c80 con 0x7f49a415acd0 2023-11-08T08:18:56.043 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.035+0000 7f49a9aba700 1 --2- 172.21.15.40:0/3175212130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49a415acd0 0x7f49a406ad20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f49a406be10 tx=0x7f49a0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5714a9453e98de83 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:56.044 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.035+0000 7f49a8ab8700 1 -- 172.21.15.40:0/3175212130 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a000b8a0 con 0x7f49a415acd0 2023-11-08T08:18:56.044 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.035+0000 7f49a8ab8700 1 -- 172.21.15.40:0/3175212130 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a000ba00 con 0x7f49a415acd0 2023-11-08T08:18:56.044 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.035+0000 7f49a8ab8700 1 -- 172.21.15.40:0/3175212130 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a000bd90 con 0x7f49a415acd0 2023-11-08T08:18:56.044 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.035+0000 7f49aaabc700 1 -- 172.21.15.40:0/3175212130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49a415acd0 msgr2=0x7f49a406ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:56.045 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.035+0000 7f49aaabc700 1 --2- 172.21.15.40:0/3175212130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49a415acd0 0x7f49a406ad20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f49a406be10 tx=0x7f49a0009bf0 comp rx=0 tx=0).stop 2023-11-08T08:18:56.045 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.039+0000 7f49aaabc700 1 -- 172.21.15.40:0/3175212130 shutdown_connections 2023-11-08T08:18:56.045 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.039+0000 7f49aaabc700 1 --2- 172.21.15.40:0/3175212130 >> v2:172.21.15.169:3300/0 conn(0x7f49a406cad0 0x7f49a40ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:56.045 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.039+0000 7f49aaabc700 1 --2- 172.21.15.40:0/3175212130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49a415acd0 0x7f49a406ad20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:56.045 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.039+0000 7f49aaabc700 1 -- 172.21.15.40:0/3175212130 >> 172.21.15.40:0/3175212130 conn(0x7f49a40cb0c0 msgr2=0x7f49a40cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:56.045 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.039+0000 7f49aaabc700 1 -- 172.21.15.40:0/3175212130 shutdown_connections 2023-11-08T08:18:56.046 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.039+0000 7f49aaabc700 1 -- 172.21.15.40:0/3175212130 wait complete. 2023-11-08T08:18:56.046 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.039+0000 7f49aaabc700 1 Processor -- start 2023-11-08T08:18:56.046 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.039+0000 7f49aaabc700 1 -- start start 2023-11-08T08:18:56.048 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f49aaabc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49a406b260 0x7f49a415b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:56.048 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f49aaabc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49a406cad0 0x7f49a4154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:56.048 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f49aaabc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49a415acd0 0x7f49a415f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:56.048 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f49aaabc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a41f9600 con 0x7f49a406b260 2023-11-08T08:18:56.049 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f49aaabc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a4154f90 con 0x7f49a415acd0 2023-11-08T08:18:56.049 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f49aaabc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a41f9850 con 0x7f49a406cad0 2023-11-08T08:18:56.049 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f49a9aba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49a406b260 0x7f49a415b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:56.049 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f49a9aba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49a406b260 0x7f49a415b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49956/0 (socket says 172.21.15.40:49956) 2023-11-08T08:18:56.049 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f49a9aba700 1 -- 172.21.15.40:0/985617400 learned_addr learned my addr 172.21.15.40:0/985617400 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:18:56.051 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f49a92b9700 1 --2- 172.21.15.40:0/985617400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49a406cad0 0x7f49a4154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:56.051 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f49aa2bb700 1 --2- 172.21.15.40:0/985617400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49a415acd0 0x7f49a415f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:56.051 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f49a9aba700 1 -- 172.21.15.40:0/985617400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49a406cad0 msgr2=0x7f49a4154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:56.051 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f49a9aba700 1 --2- 172.21.15.40:0/985617400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49a406cad0 0x7f49a4154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:56.051 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f49a9aba700 1 -- 172.21.15.40:0/985617400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49a415acd0 msgr2=0x7f49a415f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:56.052 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f49a9aba700 1 --2- 172.21.15.40:0/985617400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49a415acd0 0x7f49a415f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:56.052 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f49a9aba700 1 -- 172.21.15.40:0/985617400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a0009570 con 0x7f49a406b260 2023-11-08T08:18:56.052 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f49a9aba700 1 --2- 172.21.15.40:0/985617400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49a406b260 0x7f49a415b1e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f49a40ce280 tx=0x7f49a0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:56.052 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f499a7fc700 1 -- 172.21.15.40:0/985617400 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a00038c0 con 0x7f49a406b260 2023-11-08T08:18:56.052 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f499a7fc700 1 -- 172.21.15.40:0/985617400 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a001e070 con 0x7f49a406b260 2023-11-08T08:18:56.053 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f499a7fc700 1 -- 172.21.15.40:0/985617400 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a0003c80 con 0x7f49a406b260 2023-11-08T08:18:56.053 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f49aaabc700 1 -- 172.21.15.40:0/985617400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a406a240 con 0x7f49a406b260 2023-11-08T08:18:56.053 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.043+0000 7f49aaabc700 1 -- 172.21.15.40:0/985617400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a4069f60 con 0x7f49a406b260 2023-11-08T08:18:56.053 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.047+0000 7f49aaabc700 1 -- 172.21.15.40:0/985617400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49a406a520 con 0x7f49a406b260 2023-11-08T08:18:56.054 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.047+0000 7f499a7fc700 1 -- 172.21.15.40:0/985617400 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 83919+0+0 (secure 0 0 0) 0x7f49a000f050 con 0x7f49a406b260 2023-11-08T08:18:56.055 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.047+0000 7f499a7fc700 1 --2- 172.21.15.40:0/985617400 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4988064e90 0x7f4988067340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:56.055 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.051+0000 7f499a7fc700 1 -- 172.21.15.40:0/985617400 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f49a0020070 con 0x7f49a406b260 2023-11-08T08:18:56.062 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.055+0000 7f499a7fc700 1 -- 172.21.15.40:0/985617400 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f49a0051810 con 0x7f49a406b260 2023-11-08T08:18:56.100 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.095+0000 7f49a92b9700 1 --2- 172.21.15.40:0/985617400 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4988064e90 0x7f4988067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:56.101 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.095+0000 7f49a92b9700 1 --2- 172.21.15.40:0/985617400 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4988064e90 0x7f4988067340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f49a4160810 tx=0x7f4994009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:56.152 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:18:56.158 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:18:56.245 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.239+0000 7f49aaabc700 1 -- 172.21.15.40:0/985617400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49a4067250 con 0x7f49a406b260 2023-11-08T08:18:56.314 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.307+0000 7f499a7fc700 1 -- 172.21.15.40:0/985617400 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12382 (secure 0 0 0) 0x7f49a0054e30 con 0x7f49a406b260 2023-11-08T08:18:56.315 INFO:teuthology.misc.health.smithi040.stdout: 2023-11-08T08:18:56.315 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":17,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:18:54.775494+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:18:56.319 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.311+0000 7f49aaabc700 1 -- 172.21.15.40:0/985617400 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4988064e90 msgr2=0x7f4988067340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:56.319 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.311+0000 7f49aaabc700 1 --2- 172.21.15.40:0/985617400 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4988064e90 0x7f4988067340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f49a4160810 tx=0x7f4994009040 comp rx=0 tx=0).stop 2023-11-08T08:18:56.320 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.311+0000 7f49aaabc700 1 -- 172.21.15.40:0/985617400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49a406b260 msgr2=0x7f49a415b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:56.320 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.311+0000 7f49aaabc700 1 --2- 172.21.15.40:0/985617400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49a406b260 0x7f49a415b1e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f49a40ce280 tx=0x7f49a0004770 comp rx=0 tx=0).stop 2023-11-08T08:18:56.320 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.311+0000 7f49aaabc700 1 -- 172.21.15.40:0/985617400 shutdown_connections 2023-11-08T08:18:56.320 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.311+0000 7f49aaabc700 1 --2- 172.21.15.40:0/985617400 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4988064e90 0x7f4988067340 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:56.320 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.311+0000 7f49aaabc700 1 --2- 172.21.15.40:0/985617400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49a415acd0 0x7f49a415f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:56.321 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.311+0000 7f49aaabc700 1 --2- 172.21.15.40:0/985617400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49a406cad0 0x7f49a4154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:56.321 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.311+0000 7f49aaabc700 1 --2- 172.21.15.40:0/985617400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49a406b260 0x7f49a415b1e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:56.321 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.311+0000 7f49aaabc700 1 -- 172.21.15.40:0/985617400 >> 172.21.15.40:0/985617400 conn(0x7f49a40cb0c0 msgr2=0x7f49a4163f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:56.321 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.315+0000 7f49aaabc700 1 -- 172.21.15.40:0/985617400 shutdown_connections 2023-11-08T08:18:56.321 INFO:teuthology.misc.health.smithi040.stderr:2023-11-08T08:18:56.315+0000 7f49aaabc700 1 -- 172.21.15.40:0/985617400 wait complete. 2023-11-08T08:18:56.338 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-08T08:18:56.338 INFO:tasks.ceph:Creating RBD pool 2023-11-08T08:18:56.338 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd pool create rbd 8 2023-11-08T08:18:56.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.551+0000 7fe932177700 1 Processor -- start 2023-11-08T08:18:56.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.551+0000 7fe932177700 1 -- start start 2023-11-08T08:18:56.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.551+0000 7fe932177700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe92c159cd0 0x7fe92c15d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:56.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.551+0000 7fe932177700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe92c06ff10 0x7fe92c0703b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:56.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.551+0000 7fe932177700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92c15d650 con 0x7fe92c0cc1a0 2023-11-08T08:18:56.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.551+0000 7fe932177700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe92c0708f0 con 0x7fe92c159cd0 2023-11-08T08:18:56.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.551+0000 7fe932177700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c06be80 con 0x7fe92c06ff10 2023-11-08T08:18:56.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe930974700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe92c0cc1a0 0x7fe92c06c890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41088/0 (socket says 172.21.15.40:41088) 2023-11-08T08:18:56.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe930974700 1 -- 172.21.15.40:0/1502236778 learned_addr learned my addr 172.21.15.40:0/1502236778 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:18:56.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe931175700 1 --2- 172.21.15.40:0/1502236778 >> v2:172.21.15.169:3300/0 conn(0x7fe92c159cd0 0x7fe92c15d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:56.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe923fff700 1 -- 172.21.15.40:0/1502236778 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575348734 0 0) 0x7fe92c15d650 con 0x7fe92c0cc1a0 2023-11-08T08:18:56.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe923fff700 1 -- 172.21.15.40:0/1502236778 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9180035d0 con 0x7fe92c0cc1a0 2023-11-08T08:18:56.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe931976700 1 --2- 172.21.15.40:0/1502236778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe92c06ff10 0x7fe92c0703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:56.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe923fff700 1 -- 172.21.15.40:0/1502236778 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3233437190 0 0) 0x7fe9180035d0 con 0x7fe92c0cc1a0 2023-11-08T08:18:56.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe923fff700 1 -- 172.21.15.40:0/1502236778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe92c06ff10 msgr2=0x7fe92c0703b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:56.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe923fff700 1 --2- 172.21.15.40:0/1502236778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe92c06ff10 0x7fe92c0703b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:56.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe923fff700 1 -- 172.21.15.40:0/1502236778 >> v2:172.21.15.169:3300/0 conn(0x7fe92c159cd0 msgr2=0x7fe92c15d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:56.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe923fff700 1 --2- 172.21.15.40:0/1502236778 >> v2:172.21.15.169:3300/0 conn(0x7fe92c159cd0 0x7fe92c15d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:56.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe923fff700 1 -- 172.21.15.40:0/1502236778 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c15a0a0 con 0x7fe92c0cc1a0 2023-11-08T08:18:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe923fff700 1 -- 172.21.15.40:0/1502236778 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7fe91c001690 con 0x7fe92c0cc1a0 2023-11-08T08:18:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe923fff700 1 -- 172.21.15.40:0/1502236778 >> v1:172.21.15.40:6789/0 conn(0x7fe92c0cc1a0 legacy=0x7fe92c06c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe923fff700 1 --2- 172.21.15.40:0/1502236778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe92c06ff10 0x7fe918003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe923fff700 1 --2- 172.21.15.40:0/1502236778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe918003fa0 0x7fe918004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe923fff700 1 --2- 172.21.15.40:0/1502236778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe918004db0 0x7fe918005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:56.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe923fff700 1 -- 172.21.15.40:0/1502236778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c06be80 con 0x7fe918003fa0 2023-11-08T08:18:56.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe923fff700 1 -- 172.21.15.40:0/1502236778 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91800ba30 con 0x7fe918004db0 2023-11-08T08:18:56.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe923fff700 1 -- 172.21.15.40:0/1502236778 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91800bba0 con 0x7fe92c06ff10 2023-11-08T08:18:56.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe931175700 1 --2- 172.21.15.40:0/1502236778 >> v2:172.21.15.169:3300/0 conn(0x7fe92c159cd0 0x7fe92c15d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:18:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe930974700 1 --2- 172.21.15.40:0/1502236778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe918003fa0 0x7fe918004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe931976700 1 --2- 172.21.15.40:0/1502236778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe918004db0 0x7fe918005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe931175700 1 --2- 172.21.15.40:0/1502236778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe92c06ff10 0x7fe918003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe930974700 1 -- 172.21.15.40:0/1502236778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe92c06ff10 msgr2=0x7fe918003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:56.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe930974700 1 --2- 172.21.15.40:0/1502236778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe92c06ff10 0x7fe918003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:56.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe930974700 1 -- 172.21.15.40:0/1502236778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe918004db0 msgr2=0x7fe918005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:56.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe930974700 1 --2- 172.21.15.40:0/1502236778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe918004db0 0x7fe918005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:56.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.555+0000 7fe930974700 1 -- 172.21.15.40:0/1502236778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9180035d0 con 0x7fe918003fa0 2023-11-08T08:18:56.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.559+0000 7fe930974700 1 --2- 172.21.15.40:0/1502236778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe918003fa0 0x7fe918004870 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00c270 tx=0x7fe9180058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b93c78957d4d140 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:56.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.559+0000 7fe923fff700 1 -- 172.21.15.40:0/1502236778 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c007c80 con 0x7fe918003fa0 2023-11-08T08:18:56.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.559+0000 7fe923fff700 1 -- 172.21.15.40:0/1502236778 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91c00ce30 con 0x7fe918003fa0 2023-11-08T08:18:56.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.559+0000 7fe923fff700 1 -- 172.21.15.40:0/1502236778 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c005bc0 con 0x7fe918003fa0 2023-11-08T08:18:56.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.559+0000 7fe932177700 1 -- 172.21.15.40:0/1502236778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe918003fa0 msgr2=0x7fe918004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:56.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.559+0000 7fe932177700 1 --2- 172.21.15.40:0/1502236778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe918003fa0 0x7fe918004870 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00c270 tx=0x7fe9180058a0 comp rx=0 tx=0).stop 2023-11-08T08:18:56.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.559+0000 7fe931976700 1 -- 172.21.15.40:0/1502236778 reap_dead start 2023-11-08T08:18:56.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.559+0000 7fe932177700 1 -- 172.21.15.40:0/1502236778 shutdown_connections 2023-11-08T08:18:56.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.559+0000 7fe932177700 1 -- 172.21.15.40:0/1502236778 >> 172.21.15.40:0/1502236778 conn(0x7fe92c0cf280 msgr2=0x7fe92c0cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:56.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.559+0000 7fe932177700 1 -- 172.21.15.40:0/1502236778 shutdown_connections 2023-11-08T08:18:56.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.559+0000 7fe932177700 1 -- 172.21.15.40:0/1502236778 wait complete. 2023-11-08T08:18:56.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.563+0000 7fe932177700 1 Processor -- start 2023-11-08T08:18:56.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.563+0000 7fe932177700 1 -- start start 2023-11-08T08:18:56.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.563+0000 7fe932177700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe92c145060 0x7fe92c152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:56.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.563+0000 7fe932177700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe92c152680 0x7fe92c153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:56.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.563+0000 7fe932177700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe92c0c7dd0 0x7fe92c1542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:56.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.563+0000 7fe932177700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0c8670 con 0x7fe92c152680 2023-11-08T08:18:56.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.563+0000 7fe932177700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0c8390 con 0x7fe92c145060 2023-11-08T08:18:56.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.563+0000 7fe932177700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c157180 con 0x7fe92c0c7dd0 2023-11-08T08:18:56.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.563+0000 7fe930974700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe92c152680 0x7fe92c153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:56.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.563+0000 7fe930974700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe92c152680 0x7fe92c153d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49972/0 (socket says 172.21.15.40:49972) 2023-11-08T08:18:56.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.563+0000 7fe930974700 1 -- 172.21.15.40:0/1926947862 learned_addr learned my addr 172.21.15.40:0/1926947862 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:18:56.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.563+0000 7fe931976700 1 --2- 172.21.15.40:0/1926947862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe92c0c7dd0 0x7fe92c1542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:56.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.563+0000 7fe931175700 1 --2- 172.21.15.40:0/1926947862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe92c145060 0x7fe92c152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:56.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.563+0000 7fe930974700 1 -- 172.21.15.40:0/1926947862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe92c0c7dd0 msgr2=0x7fe92c1542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:56.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.563+0000 7fe930974700 1 --2- 172.21.15.40:0/1926947862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe92c0c7dd0 0x7fe92c1542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:56.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.563+0000 7fe930974700 1 -- 172.21.15.40:0/1926947862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe92c145060 msgr2=0x7fe92c152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:56.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.563+0000 7fe930974700 1 --2- 172.21.15.40:0/1926947862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe92c145060 0x7fe92c152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:56.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.563+0000 7fe930974700 1 -- 172.21.15.40:0/1926947862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c0ce260 con 0x7fe92c152680 2023-11-08T08:18:56.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.563+0000 7fe930974700 1 --2- 172.21.15.40:0/1926947862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe92c152680 0x7fe92c153d80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe91c011000 tx=0x7fe91c001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:56.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.563+0000 7fe921ffb700 1 -- 172.21.15.40:0/1926947862 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c004d60 con 0x7fe92c152680 2023-11-08T08:18:56.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.567+0000 7fe921ffb700 1 -- 172.21.15.40:0/1926947862 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91c0070d0 con 0x7fe92c152680 2023-11-08T08:18:56.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.567+0000 7fe932177700 1 -- 172.21.15.40:0/1926947862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92c0cdf80 con 0x7fe92c152680 2023-11-08T08:18:56.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.567+0000 7fe921ffb700 1 -- 172.21.15.40:0/1926947862 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c007460 con 0x7fe92c152680 2023-11-08T08:18:56.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.567+0000 7fe932177700 1 -- 172.21.15.40:0/1926947862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92c06c090 con 0x7fe92c152680 2023-11-08T08:18:56.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.567+0000 7fe932177700 1 -- 172.21.15.40:0/1926947862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe92c0cca80 con 0x7fe92c152680 2023-11-08T08:18:56.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.567+0000 7fe921ffb700 1 -- 172.21.15.40:0/1926947862 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 83919+0+0 (secure 0 0 0) 0x7fe91c010070 con 0x7fe92c152680 2023-11-08T08:18:56.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.571+0000 7fe921ffb700 1 --2- 172.21.15.40:0/1926947862 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe910064e90 0x7fe910067340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:56.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.571+0000 7fe921ffb700 1 -- 172.21.15.40:0/1926947862 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe91c083490 con 0x7fe92c152680 2023-11-08T08:18:56.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.571+0000 7fe931175700 1 --2- 172.21.15.40:0/1926947862 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe910064e90 0x7fe910067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:56.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.571+0000 7fe931175700 1 --2- 172.21.15.40:0/1926947862 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe910064e90 0x7fe910067340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe928009e70 tx=0x7fe928009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:56.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.575+0000 7fe921ffb700 1 -- 172.21.15.40:0/1926947862 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fe91c051bd0 con 0x7fe92c152680 2023-11-08T08:18:56.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.759+0000 7fe932177700 1 -- 172.21.15.40:0/1926947862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fe92c073d20 con 0x7fe92c152680 2023-11-08T08:18:56.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:56.847+0000 7fe921ffb700 1 -- 172.21.15.40:0/1926947862 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fe91c051fa0 con 0x7fe92c152680 2023-11-08T08:18:57.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.751+0000 7fe921ffb700 1 -- 172.21.15.40:0/1926947862 <== mon.0 v2:172.21.15.40:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v18) v1 ==== 109+0+0 (secure 0 0 0) 0x7fe91c0553d0 con 0x7fe92c152680 2023-11-08T08:18:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.755+0000 7fe932177700 1 -- 172.21.15.40:0/1926947862 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe910064e90 msgr2=0x7fe910067340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.755+0000 7fe932177700 1 --2- 172.21.15.40:0/1926947862 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe910064e90 0x7fe910067340 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe928009e70 tx=0x7fe928009240 comp rx=0 tx=0).stop 2023-11-08T08:18:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.755+0000 7fe932177700 1 -- 172.21.15.40:0/1926947862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe92c152680 msgr2=0x7fe92c153d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:57.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.755+0000 7fe932177700 1 --2- 172.21.15.40:0/1926947862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe92c152680 0x7fe92c153d80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe91c011000 tx=0x7fe91c001a20 comp rx=0 tx=0).stop 2023-11-08T08:18:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.755+0000 7fe932177700 1 -- 172.21.15.40:0/1926947862 shutdown_connections 2023-11-08T08:18:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.755+0000 7fe932177700 1 --2- 172.21.15.40:0/1926947862 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe910064e90 0x7fe910067340 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.755+0000 7fe932177700 1 --2- 172.21.15.40:0/1926947862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe92c0c7dd0 0x7fe92c1542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.755+0000 7fe932177700 1 --2- 172.21.15.40:0/1926947862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe92c152680 0x7fe92c153d80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.755+0000 7fe932177700 1 --2- 172.21.15.40:0/1926947862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe92c145060 0x7fe92c152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:57.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.755+0000 7fe932177700 1 -- 172.21.15.40:0/1926947862 >> 172.21.15.40:0/1926947862 conn(0x7fe92c0cf280 msgr2=0x7fe92c0afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:57.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.755+0000 7fe932177700 1 -- 172.21.15.40:0/1926947862 shutdown_connections 2023-11-08T08:18:57.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.755+0000 7fe932177700 1 -- 172.21.15.40:0/1926947862 wait complete. 2023-11-08T08:18:57.764 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'rbd' created 2023-11-08T08:18:57.784 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-11-08T08:18:57.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.963+0000 7f3ef4c28700 1 Processor -- start 2023-11-08T08:18:57.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.963+0000 7f3ef4c28700 1 -- start start 2023-11-08T08:18:57.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.963+0000 7f3ef4c28700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3ef00ca940 0x7f3ef00c1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:57.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.963+0000 7f3ef4c28700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ef00c1ac0 0x7f3ef00c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:57.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.963+0000 7f3ef4c28700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef0165140 con 0x7f3ef00ccb00 2023-11-08T08:18:57.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.963+0000 7f3ef4c28700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef0160da0 con 0x7f3ef00ca940 2023-11-08T08:18:57.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.963+0000 7f3ef4c28700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00c1f00 con 0x7f3ef00c1ac0 2023-11-08T08:18:57.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.963+0000 7f3eef7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3ef00ccb00 0x7f3ef00ad8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42876/0 (socket says 172.21.15.40:42876) 2023-11-08T08:18:57.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.963+0000 7f3eef7fe700 1 -- 172.21.15.40:0/938712665 learned_addr learned my addr 172.21.15.40:0/938712665 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:18:57.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.963+0000 7f3eeeffd700 1 --2- 172.21.15.40:0/938712665 >> v2:172.21.15.169:3300/0 conn(0x7f3ef00ca940 0x7f3ef00c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:57.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.963+0000 7f3eeffff700 1 --2- 172.21.15.40:0/938712665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ef00c1ac0 0x7f3ef00c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:57.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.963+0000 7f3eee7fc700 1 -- 172.21.15.40:0/938712665 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1577643353 0 0) 0x7f3ef0165140 con 0x7f3ef00ccb00 2023-11-08T08:18:57.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.967+0000 7f3eee7fc700 1 -- 172.21.15.40:0/938712665 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ed80035d0 con 0x7f3ef00ccb00 2023-11-08T08:18:57.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.967+0000 7f3eeeffd700 1 -- 172.21.15.40:0/938712665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ef00c1ac0 msgr2=0x7f3ef00c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:57.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.967+0000 7f3eeeffd700 1 --2- 172.21.15.40:0/938712665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ef00c1ac0 0x7f3ef00c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:57.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.967+0000 7f3eeeffd700 1 -- 172.21.15.40:0/938712665 >> v1:172.21.15.40:6789/0 conn(0x7f3ef00ccb00 legacy=0x7f3ef00ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:57.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.967+0000 7f3eeeffd700 1 -- 172.21.15.40:0/938712665 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef00cced0 con 0x7f3ef00ca940 2023-11-08T08:18:57.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.967+0000 7f3eeeffd700 1 --2- 172.21.15.40:0/938712665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ef00ca940 0x7f3ef00c1580 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009850 tx=0x7f3ee4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56826865bd1fbed2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:57.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.967+0000 7f3eee7fc700 1 -- 172.21.15.40:0/938712665 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4011830 con 0x7f3ef00ca940 2023-11-08T08:18:57.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.967+0000 7f3eee7fc700 1 -- 172.21.15.40:0/938712665 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee4011990 con 0x7f3ef00ca940 2023-11-08T08:18:57.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.967+0000 7f3eee7fc700 1 -- 172.21.15.40:0/938712665 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4011d40 con 0x7f3ef00ca940 2023-11-08T08:18:57.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.967+0000 7f3ef4c28700 1 -- 172.21.15.40:0/938712665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ef00ca940 msgr2=0x7f3ef00c1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:57.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.967+0000 7f3ef4c28700 1 --2- 172.21.15.40:0/938712665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ef00ca940 0x7f3ef00c1580 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009850 tx=0x7f3ee4009b60 comp rx=0 tx=0).stop 2023-11-08T08:18:57.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.967+0000 7f3ef4c28700 1 -- 172.21.15.40:0/938712665 shutdown_connections 2023-11-08T08:18:57.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.967+0000 7f3ef4c28700 1 --2- 172.21.15.40:0/938712665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ef00c1ac0 0x7f3ef00c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:57.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.967+0000 7f3ef4c28700 1 --2- 172.21.15.40:0/938712665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ef00ca940 0x7f3ef00c1580 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:57.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.967+0000 7f3ef4c28700 1 -- 172.21.15.40:0/938712665 >> 172.21.15.40:0/938712665 conn(0x7f3ef00ce5b0 msgr2=0x7f3ef00ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:57.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.967+0000 7f3ef4c28700 1 -- 172.21.15.40:0/938712665 shutdown_connections 2023-11-08T08:18:57.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.967+0000 7f3ef4c28700 1 -- 172.21.15.40:0/938712665 wait complete. 2023-11-08T08:18:57.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.971+0000 7f3ef4c28700 1 Processor -- start 2023-11-08T08:18:57.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.971+0000 7f3ef4c28700 1 -- start start 2023-11-08T08:18:57.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.971+0000 7f3ef4c28700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ef00c1ac0 0x7f3ef00c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:57.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.971+0000 7f3ef4c28700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ef00ca940 0x7f3ef01632f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:57.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.971+0000 7f3ef4c28700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ef00ccb00 0x7f3ef01f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:57.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.971+0000 7f3ef4c28700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef0163830 con 0x7f3ef00ccb00 2023-11-08T08:18:57.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.971+0000 7f3ef4c28700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef00c2770 con 0x7f3ef00ca940 2023-11-08T08:18:57.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.971+0000 7f3ef4c28700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef01f5e20 con 0x7f3ef00c1ac0 2023-11-08T08:18:57.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.975+0000 7f3eeffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ef00ccb00 0x7f3ef01f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:57.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.975+0000 7f3eeffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ef00ccb00 0x7f3ef01f58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55958/0 (socket says 172.21.15.40:55958) 2023-11-08T08:18:57.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.975+0000 7f3eeffff700 1 -- 172.21.15.40:0/2378142252 learned_addr learned my addr 172.21.15.40:0/2378142252 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:18:57.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.975+0000 7f3eeeffd700 1 --2- 172.21.15.40:0/2378142252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ef00ca940 0x7f3ef01632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:57.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.975+0000 7f3eef7fe700 1 --2- 172.21.15.40:0/2378142252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ef00c1ac0 0x7f3ef00c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:57.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.975+0000 7f3eeeffd700 1 -- 172.21.15.40:0/2378142252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ef00c1ac0 msgr2=0x7f3ef00c20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:57.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.975+0000 7f3eeeffd700 1 --2- 172.21.15.40:0/2378142252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ef00c1ac0 0x7f3ef00c20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:57.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.975+0000 7f3eeeffd700 1 -- 172.21.15.40:0/2378142252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ef00ccb00 msgr2=0x7f3ef01f58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:57.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.975+0000 7f3eeeffd700 1 --2- 172.21.15.40:0/2378142252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ef00ccb00 0x7f3ef01f58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:57.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.975+0000 7f3eeeffd700 1 -- 172.21.15.40:0/2378142252 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee40094b0 con 0x7f3ef00ca940 2023-11-08T08:18:57.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.975+0000 7f3eeffff700 1 --2- 172.21.15.40:0/2378142252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ef00ccb00 0x7f3ef01f58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:18:57.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.975+0000 7f3eeeffd700 1 --2- 172.21.15.40:0/2378142252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ef00ca940 0x7f3ef01632f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4016000 tx=0x7f3ee4004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:57.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.975+0000 7f3ed7fff700 1 -- 172.21.15.40:0/2378142252 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4004590 con 0x7f3ef00ca940 2023-11-08T08:18:57.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.975+0000 7f3ed7fff700 1 -- 172.21.15.40:0/2378142252 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee401e070 con 0x7f3ef00ca940 2023-11-08T08:18:57.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.975+0000 7f3ed7fff700 1 -- 172.21.15.40:0/2378142252 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40038e0 con 0x7f3ef00ca940 2023-11-08T08:18:57.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.975+0000 7f3ef4c28700 1 -- 172.21.15.40:0/2378142252 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef0164810 con 0x7f3ef00ca940 2023-11-08T08:18:57.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.975+0000 7f3ef4c28700 1 -- 172.21.15.40:0/2378142252 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef0164530 con 0x7f3ef00ca940 2023-11-08T08:18:57.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.979+0000 7f3ef4c28700 1 -- 172.21.15.40:0/2378142252 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef0164af0 con 0x7f3ef00ca940 2023-11-08T08:18:57.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.979+0000 7f3ed7fff700 1 -- 172.21.15.40:0/2378142252 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f3ee402a020 con 0x7f3ef00ca940 2023-11-08T08:18:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.979+0000 7f3ed7fff700 1 --2- 172.21.15.40:0/2378142252 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3edc064ee0 0x7f3edc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.979+0000 7f3ed7fff700 1 -- 172.21.15.40:0/2378142252 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f3ee4082b70 con 0x7f3ef00ca940 2023-11-08T08:18:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.979+0000 7f3eef7fe700 1 --2- 172.21.15.40:0/2378142252 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3edc064ee0 0x7f3edc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.979+0000 7f3eef7fe700 1 --2- 172.21.15.40:0/2378142252 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3edc064ee0 0x7f3edc067390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00010e0 tx=0x7f3ee0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:57.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:57.991+0000 7f3ed7fff700 1 -- 172.21.15.40:0/2378142252 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f3ee4050f80 con 0x7f3ef00ca940 2023-11-08T08:18:58.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:58.171+0000 7f3ef4c28700 1 -- 172.21.15.40:0/2378142252 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3ef0073cb0 con 0x7f3ef00ca940 2023-11-08T08:18:58.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:58.755+0000 7f3ed7fff700 1 -- 172.21.15.40:0/2378142252 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v19) v1 ==== 173+0+0 (secure 0 0 0) 0x7f3ee40545a0 con 0x7f3ef00ca940 2023-11-08T08:18:58.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:58.759+0000 7f3ef4c28700 1 -- 172.21.15.40:0/2378142252 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3edc064ee0 msgr2=0x7f3edc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:58.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:58.759+0000 7f3ef4c28700 1 --2- 172.21.15.40:0/2378142252 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3edc064ee0 0x7f3edc067390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00010e0 tx=0x7f3ee0009040 comp rx=0 tx=0).stop 2023-11-08T08:18:58.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:58.759+0000 7f3ef4c28700 1 -- 172.21.15.40:0/2378142252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ef00ca940 msgr2=0x7f3ef01632f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:58.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:58.759+0000 7f3ef4c28700 1 --2- 172.21.15.40:0/2378142252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ef00ca940 0x7f3ef01632f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4016000 tx=0x7f3ee4004770 comp rx=0 tx=0).stop 2023-11-08T08:18:58.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:58.759+0000 7f3ef4c28700 1 -- 172.21.15.40:0/2378142252 shutdown_connections 2023-11-08T08:18:58.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:58.759+0000 7f3ef4c28700 1 --2- 172.21.15.40:0/2378142252 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3edc064ee0 0x7f3edc067390 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:58.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:58.759+0000 7f3ef4c28700 1 --2- 172.21.15.40:0/2378142252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ef00ccb00 0x7f3ef01f58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:58.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:58.759+0000 7f3ef4c28700 1 --2- 172.21.15.40:0/2378142252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ef00ca940 0x7f3ef01632f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:58.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:58.759+0000 7f3ef4c28700 1 --2- 172.21.15.40:0/2378142252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ef00c1ac0 0x7f3ef00c20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:58.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:58.759+0000 7f3ef4c28700 1 -- 172.21.15.40:0/2378142252 >> 172.21.15.40:0/2378142252 conn(0x7f3ef00ce5b0 msgr2=0x7f3ef015ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:58.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:58.763+0000 7f3ef4c28700 1 -- 172.21.15.40:0/2378142252 shutdown_connections 2023-11-08T08:18:58.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:58.763+0000 7f3ef4c28700 1 -- 172.21.15.40:0/2378142252 wait complete. 2023-11-08T08:18:58.770 INFO:teuthology.orchestra.run.smithi040.stderr:enabled application 'rbd' on pool 'rbd' 2023-11-08T08:18:58.790 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-08T08:18:58.791 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T08:18:58.791 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T08:18:58.798 INFO:tasks.ceph.mds.a:Started 2023-11-08T08:18:58.798 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T08:18:58.799 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T08:18:58.803 INFO:tasks.ceph.mds.c:Started 2023-11-08T08:18:58.803 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T08:18:58.803 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T08:18:58.810 INFO:tasks.ceph.mds.b:Started 2023-11-08T08:18:58.810 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T08:18:58.811 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T08:18:58.817 INFO:tasks.ceph.mds.d:Started 2023-11-08T08:18:58.817 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-08T08:18:58.818 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:18:58.920 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-11-08T08:18:58.921 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2023-11-08T08:18:58.927 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2023-11-08T08:18:58.956 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2023-11-08T08:18:59.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:58.999+0000 7f288cf4f700 1 Processor -- start 2023-11-08T08:18:59.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.003+0000 7f288cf4f700 1 -- start start 2023-11-08T08:18:59.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.003+0000 7f288cf4f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2888071aa0 0x7f288806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:59.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.003+0000 7f288cf4f700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2888069e00 0x7f288806a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:59.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.003+0000 7f288cf4f700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f288806c600 con 0x7f288815c640 2023-11-08T08:18:59.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.003+0000 7f288cf4f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f288806cef0 con 0x7f2888071aa0 2023-11-08T08:18:59.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.003+0000 7f288cf4f700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28880d38f0 con 0x7f2888069e00 2023-11-08T08:18:59.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.003+0000 7f287ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2888071aa0 0x7f288806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:59.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.003+0000 7f2887fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2888069e00 0x7f288806a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:59.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.003+0000 7f28877fe700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f288815c640 0x7f28880d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:60336/0 (socket says 172.21.15.40:60336) 2023-11-08T08:18:59.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.003+0000 7f28877fe700 1 -- 172.21.15.40:0/2028769562 learned_addr learned my addr 172.21.15.40:0/2028769562 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:18:59.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.003+0000 7f2886ffd700 1 -- 172.21.15.40:0/2028769562 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 37160710 0 0) 0x7f288806c600 con 0x7f288815c640 2023-11-08T08:18:59.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.003+0000 7f2886ffd700 1 -- 172.21.15.40:0/2028769562 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28740035d0 con 0x7f288815c640 2023-11-08T08:18:59.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.003+0000 7f2887fff700 1 -- 172.21.15.40:0/2028769562 >> v2:172.21.15.40:3300/0 conn(0x7f2888071aa0 msgr2=0x7f288806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:59.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.007+0000 7f2887fff700 1 --2- 172.21.15.40:0/2028769562 >> v2:172.21.15.40:3300/0 conn(0x7f2888071aa0 0x7f288806d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.007+0000 7f2887fff700 1 -- 172.21.15.40:0/2028769562 >> v1:172.21.15.169:6789/0 conn(0x7f288815c640 legacy=0x7f28880d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:59.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.007+0000 7f2887fff700 1 -- 172.21.15.40:0/2028769562 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28880722f0 con 0x7f2888069e00 2023-11-08T08:18:59.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.007+0000 7f287ffff700 1 --2- 172.21.15.40:0/2028769562 >> v2:172.21.15.40:3300/0 conn(0x7f2888071aa0 0x7f288806d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:18:59.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.007+0000 7f2887fff700 1 --2- 172.21.15.40:0/2028769562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2888069e00 0x7f288806a280 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f288806ed60 tx=0x7f288000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=282811e61cbe8af9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:59.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.007+0000 7f2886ffd700 1 -- 172.21.15.40:0/2028769562 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288000e070 con 0x7f2888069e00 2023-11-08T08:18:59.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.007+0000 7f2886ffd700 1 -- 172.21.15.40:0/2028769562 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288000bb20 con 0x7f2888069e00 2023-11-08T08:18:59.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.007+0000 7f2886ffd700 1 -- 172.21.15.40:0/2028769562 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880012430 con 0x7f2888069e00 2023-11-08T08:18:59.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.007+0000 7f288cf4f700 1 -- 172.21.15.40:0/2028769562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2888069e00 msgr2=0x7f288806a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:59.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.007+0000 7f288cf4f700 1 --2- 172.21.15.40:0/2028769562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2888069e00 0x7f288806a280 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f288806ed60 tx=0x7f288000b330 comp rx=0 tx=0).stop 2023-11-08T08:18:59.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.007+0000 7f288cf4f700 1 -- 172.21.15.40:0/2028769562 shutdown_connections 2023-11-08T08:18:59.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.007+0000 7f288cf4f700 1 --2- 172.21.15.40:0/2028769562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2888069e00 0x7f288806a280 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.007+0000 7f288cf4f700 1 --2- 172.21.15.40:0/2028769562 >> v2:172.21.15.40:3300/0 conn(0x7f2888071aa0 0x7f288806d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.007+0000 7f288cf4f700 1 -- 172.21.15.40:0/2028769562 >> 172.21.15.40:0/2028769562 conn(0x7f28880b6520 msgr2=0x7f2888071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:59.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.007+0000 7f288cf4f700 1 -- 172.21.15.40:0/2028769562 shutdown_connections 2023-11-08T08:18:59.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.007+0000 7f288cf4f700 1 -- 172.21.15.40:0/2028769562 wait complete. 2023-11-08T08:18:59.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.011+0000 7f288cf4f700 1 Processor -- start 2023-11-08T08:18:59.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.011+0000 7f288cf4f700 1 -- start start 2023-11-08T08:18:59.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.011+0000 7f288cf4f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2888069e00 0x7f28880c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:59.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.011+0000 7f288cf4f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2888071aa0 0x7f2888156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:59.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.011+0000 7f288cf4f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f288815c640 0x7f2888156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:59.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.011+0000 7f288cf4f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28881f4fc0 con 0x7f288815c640 2023-11-08T08:18:59.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.011+0000 7f288cf4f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2888157390 con 0x7f2888069e00 2023-11-08T08:18:59.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.011+0000 7f288cf4f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28880c9ed0 con 0x7f2888071aa0 2023-11-08T08:18:59.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.011+0000 7f2887fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f288815c640 0x7f2888156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:59.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.011+0000 7f2887fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f288815c640 0x7f2888156e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56020/0 (socket says 172.21.15.40:56020) 2023-11-08T08:18:59.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.011+0000 7f2887fff700 1 -- 172.21.15.40:0/521468948 learned_addr learned my addr 172.21.15.40:0/521468948 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:18:59.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.011+0000 7f28877fe700 1 --2- 172.21.15.40:0/521468948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2888069e00 0x7f28880c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:59.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.011+0000 7f287ffff700 1 --2- 172.21.15.40:0/521468948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2888071aa0 0x7f2888156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:59.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.011+0000 7f2887fff700 1 -- 172.21.15.40:0/521468948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2888071aa0 msgr2=0x7f2888156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:59.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.011+0000 7f2887fff700 1 --2- 172.21.15.40:0/521468948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2888071aa0 0x7f2888156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.011+0000 7f2887fff700 1 -- 172.21.15.40:0/521468948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2888069e00 msgr2=0x7f28880c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:59.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.011+0000 7f2887fff700 1 --2- 172.21.15.40:0/521468948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2888069e00 0x7f28880c9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.011+0000 7f2887fff700 1 -- 172.21.15.40:0/521468948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28700094b0 con 0x7f288815c640 2023-11-08T08:18:59.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.011+0000 7f28877fe700 1 --2- 172.21.15.40:0/521468948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2888069e00 0x7f28880c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:18:59.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.011+0000 7f287ffff700 1 --2- 172.21.15.40:0/521468948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2888071aa0 0x7f2888156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:18:59.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.015+0000 7f2887fff700 1 --2- 172.21.15.40:0/521468948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f288815c640 0x7f2888156e50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f288000b370 tx=0x7f2880004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:59.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.015+0000 7f2884ff9700 1 -- 172.21.15.40:0/521468948 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880012430 con 0x7f288815c640 2023-11-08T08:18:59.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.015+0000 7f288cf4f700 1 -- 172.21.15.40:0/521468948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2880009a70 con 0x7f288815c640 2023-11-08T08:18:59.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.015+0000 7f2884ff9700 1 -- 172.21.15.40:0/521468948 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2880020070 con 0x7f288815c640 2023-11-08T08:18:59.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.015+0000 7f2884ff9700 1 -- 172.21.15.40:0/521468948 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880004280 con 0x7f288815c640 2023-11-08T08:18:59.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.015+0000 7f288cf4f700 1 -- 172.21.15.40:0/521468948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2888072010 con 0x7f288815c640 2023-11-08T08:18:59.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.015+0000 7f288cf4f700 1 -- 172.21.15.40:0/521468948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f288815ca10 con 0x7f288815c640 2023-11-08T08:18:59.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.019+0000 7f2884ff9700 1 -- 172.21.15.40:0/521468948 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f288002e070 con 0x7f288815c640 2023-11-08T08:18:59.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.019+0000 7f2884ff9700 1 --2- 172.21.15.40:0/521468948 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2868064f10 0x7f28680673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:59.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.019+0000 7f2884ff9700 1 -- 172.21.15.40:0/521468948 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2880022070 con 0x7f288815c640 2023-11-08T08:18:59.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.019+0000 7f28877fe700 1 --2- 172.21.15.40:0/521468948 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2868064f10 0x7f28680673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:59.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.019+0000 7f28877fe700 1 --2- 172.21.15.40:0/521468948 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2868064f10 0x7f28680673c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f28780011e0 tx=0x7f2878009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:59.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.027+0000 7f2884ff9700 1 -- 172.21.15.40:0/521468948 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f2880051cb0 con 0x7f288815c640 2023-11-08T08:18:59.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.203+0000 7f288cf4f700 1 -- 172.21.15.40:0/521468948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f288806c830 con 0x7f288815c640 2023-11-08T08:18:59.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.203+0000 7f2884ff9700 1 -- 172.21.15.40:0/521468948 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14181 (secure 0 0 0) 0x7f28800552d0 con 0x7f288815c640 2023-11-08T08:18:59.211 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:18:59.211 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":19,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:18:58.749176+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:18:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.207+0000 7f288cf4f700 1 -- 172.21.15.40:0/521468948 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2868064f10 msgr2=0x7f28680673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.207+0000 7f288cf4f700 1 --2- 172.21.15.40:0/521468948 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2868064f10 0x7f28680673c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f28780011e0 tx=0x7f2878009040 comp rx=0 tx=0).stop 2023-11-08T08:18:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.207+0000 7f288cf4f700 1 -- 172.21.15.40:0/521468948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f288815c640 msgr2=0x7f2888156e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.207+0000 7f288cf4f700 1 --2- 172.21.15.40:0/521468948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f288815c640 0x7f2888156e50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f288000b370 tx=0x7f2880004af0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.211+0000 7f288cf4f700 1 -- 172.21.15.40:0/521468948 shutdown_connections 2023-11-08T08:18:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.211+0000 7f288cf4f700 1 --2- 172.21.15.40:0/521468948 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2868064f10 0x7f28680673c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.211+0000 7f288cf4f700 1 --2- 172.21.15.40:0/521468948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f288815c640 0x7f2888156e50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.211+0000 7f288cf4f700 1 --2- 172.21.15.40:0/521468948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2888071aa0 0x7f2888156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.211+0000 7f288cf4f700 1 --2- 172.21.15.40:0/521468948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2888069e00 0x7f28880c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.211+0000 7f288cf4f700 1 -- 172.21.15.40:0/521468948 >> 172.21.15.40:0/521468948 conn(0x7f28880b6520 msgr2=0x7f28880d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.211+0000 7f288cf4f700 1 -- 172.21.15.40:0/521468948 shutdown_connections 2023-11-08T08:18:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.211+0000 7f288cf4f700 1 -- 172.21.15.40:0/521468948 wait complete. 2023-11-08T08:18:59.239 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T08:18:59.240 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:18:59.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.415+0000 7fc6c9dbc700 1 Processor -- start 2023-11-08T08:18:59.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.415+0000 7fc6c9dbc700 1 -- start start 2023-11-08T08:18:59.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.419+0000 7fc6c9dbc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc6c40aeb80 0x7fc6c415e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:59.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.419+0000 7fc6c9dbc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6c40b1330 0x7fc6c40b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:59.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.419+0000 7fc6c9dbc700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc6c40711e0 0x7fc6c4071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:59.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.419+0000 7fc6c9dbc700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c40b1ce0 con 0x7fc6c40aeb80 2023-11-08T08:18:59.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.419+0000 7fc6c9dbc700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c415ed70 con 0x7fc6c40711e0 2023-11-08T08:18:59.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.419+0000 7fc6c9dbc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40cacc0 con 0x7fc6c40b1330 2023-11-08T08:18:59.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.419+0000 7fc6c8dba700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc6c40aeb80 0x7fc6c415e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:59.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.419+0000 7fc6c8dba700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc6c40aeb80 0x7fc6c415e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56024/0 (socket says 172.21.15.40:56024) 2023-11-08T08:18:59.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.419+0000 7fc6c8dba700 1 -- 172.21.15.40:0/1828563412 learned_addr learned my addr 172.21.15.40:0/1828563412 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:18:59.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.419+0000 7fc6c95bb700 1 --2- 172.21.15.40:0/1828563412 >> v2:172.21.15.169:3300/0 conn(0x7fc6c40711e0 0x7fc6c4071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:59.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.419+0000 7fc6c3fff700 1 --2- 172.21.15.40:0/1828563412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6c40b1330 0x7fc6c40b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:59.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.419+0000 7fc6c8dba700 1 -- 172.21.15.40:0/1828563412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6c40b1330 msgr2=0x7fc6c40b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:59.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.419+0000 7fc6c8dba700 1 --2- 172.21.15.40:0/1828563412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6c40b1330 0x7fc6c40b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.419+0000 7fc6c8dba700 1 -- 172.21.15.40:0/1828563412 >> v2:172.21.15.169:3300/0 conn(0x7fc6c40711e0 msgr2=0x7fc6c4071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:18:59.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.419+0000 7fc6c8dba700 1 --2- 172.21.15.40:0/1828563412 >> v2:172.21.15.169:3300/0 conn(0x7fc6c40711e0 0x7fc6c4071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.419+0000 7fc6c8dba700 1 -- 172.21.15.40:0/1828563412 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c406a9e0 con 0x7fc6c40aeb80 2023-11-08T08:18:59.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.419+0000 7fc6c8dba700 1 --2- 172.21.15.40:0/1828563412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6c40aeb80 0x7fc6c415e830 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc6c406a0f0 tx=0x7fc6b8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfbc9c1a3091616 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:59.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.423+0000 7fc6c37fe700 1 -- 172.21.15.40:0/1828563412 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8011830 con 0x7fc6c40aeb80 2023-11-08T08:18:59.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.423+0000 7fc6c37fe700 1 -- 172.21.15.40:0/1828563412 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b8011990 con 0x7fc6c40aeb80 2023-11-08T08:18:59.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.423+0000 7fc6c37fe700 1 -- 172.21.15.40:0/1828563412 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8011d20 con 0x7fc6c40aeb80 2023-11-08T08:18:59.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.423+0000 7fc6c9dbc700 1 -- 172.21.15.40:0/1828563412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6c40aeb80 msgr2=0x7fc6c415e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.423+0000 7fc6c9dbc700 1 --2- 172.21.15.40:0/1828563412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6c40aeb80 0x7fc6c415e830 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc6c406a0f0 tx=0x7fc6b8009bf0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.423+0000 7fc6c9dbc700 1 -- 172.21.15.40:0/1828563412 shutdown_connections 2023-11-08T08:18:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.423+0000 7fc6c9dbc700 1 --2- 172.21.15.40:0/1828563412 >> v2:172.21.15.169:3300/0 conn(0x7fc6c40711e0 0x7fc6c4071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.423+0000 7fc6c9dbc700 1 --2- 172.21.15.40:0/1828563412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6c40b1330 0x7fc6c40b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.423+0000 7fc6c9dbc700 1 --2- 172.21.15.40:0/1828563412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6c40aeb80 0x7fc6c415e830 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.423+0000 7fc6c9dbc700 1 -- 172.21.15.40:0/1828563412 >> 172.21.15.40:0/1828563412 conn(0x7fc6c40cc330 msgr2=0x7fc6c40cc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.423+0000 7fc6c9dbc700 1 -- 172.21.15.40:0/1828563412 shutdown_connections 2023-11-08T08:18:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.423+0000 7fc6c9dbc700 1 -- 172.21.15.40:0/1828563412 wait complete. 2023-11-08T08:18:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.423+0000 7fc6c9dbc700 1 Processor -- start 2023-11-08T08:18:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c9dbc700 1 -- start start 2023-11-08T08:18:59.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c9dbc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6c40711e0 0x7fc6c4158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:59.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c9dbc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6c40aeb80 0x7fc6c414e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:59.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c9dbc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6c40b1330 0x7fc6c40b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:59.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c9dbc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40c8520 con 0x7fc6c40711e0 2023-11-08T08:18:59.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c9dbc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c415bc60 con 0x7fc6c40aeb80 2023-11-08T08:18:59.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c9dbc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c414ec00 con 0x7fc6c40b1330 2023-11-08T08:18:59.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c8dba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6c40711e0 0x7fc6c4158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:59.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c8dba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6c40711e0 0x7fc6c4158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56034/0 (socket says 172.21.15.40:56034) 2023-11-08T08:18:59.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c8dba700 1 -- 172.21.15.40:0/3890152299 learned_addr learned my addr 172.21.15.40:0/3890152299 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:18:59.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c3fff700 1 --2- 172.21.15.40:0/3890152299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6c40aeb80 0x7fc6c414e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:59.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c95bb700 1 --2- 172.21.15.40:0/3890152299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6c40b1330 0x7fc6c40b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c8dba700 1 -- 172.21.15.40:0/3890152299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6c40b1330 msgr2=0x7fc6c40b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c8dba700 1 --2- 172.21.15.40:0/3890152299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6c40b1330 0x7fc6c40b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c8dba700 1 -- 172.21.15.40:0/3890152299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6c40aeb80 msgr2=0x7fc6c414e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c8dba700 1 --2- 172.21.15.40:0/3890152299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6c40aeb80 0x7fc6c414e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c8dba700 1 -- 172.21.15.40:0/3890152299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b8009570 con 0x7fc6c40711e0 2023-11-08T08:18:59.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c8dba700 1 --2- 172.21.15.40:0/3890152299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6c40711e0 0x7fc6c4158d30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8000c00 tx=0x7fc6b8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:59.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c17fa700 1 -- 172.21.15.40:0/3890152299 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8004450 con 0x7fc6c40711e0 2023-11-08T08:18:59.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c17fa700 1 -- 172.21.15.40:0/3890152299 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b80045b0 con 0x7fc6c40711e0 2023-11-08T08:18:59.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.427+0000 7fc6c17fa700 1 -- 172.21.15.40:0/3890152299 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8003900 con 0x7fc6c40711e0 2023-11-08T08:18:59.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.431+0000 7fc6c9dbc700 1 -- 172.21.15.40:0/3890152299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c40ca950 con 0x7fc6c40711e0 2023-11-08T08:18:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.431+0000 7fc6c9dbc700 1 -- 172.21.15.40:0/3890152299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c40ca670 con 0x7fc6c40711e0 2023-11-08T08:18:59.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.431+0000 7fc6c9dbc700 1 -- 172.21.15.40:0/3890152299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c406acc0 con 0x7fc6c40711e0 2023-11-08T08:18:59.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.431+0000 7fc6c17fa700 1 -- 172.21.15.40:0/3890152299 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc6b802d070 con 0x7fc6c40711e0 2023-11-08T08:18:59.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.435+0000 7fc6c17fa700 1 --2- 172.21.15.40:0/3890152299 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc6b0064ee0 0x7fc6b0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:59.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.435+0000 7fc6c17fa700 1 -- 172.21.15.40:0/3890152299 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc6b8021030 con 0x7fc6c40711e0 2023-11-08T08:18:59.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.435+0000 7fc6c3fff700 1 --2- 172.21.15.40:0/3890152299 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc6b0064ee0 0x7fc6b0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:59.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.435+0000 7fc6c3fff700 1 --2- 172.21.15.40:0/3890152299 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc6b0064ee0 0x7fc6b0067390 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc6c41528b0 tx=0x7fc6b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:59.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.439+0000 7fc6c17fa700 1 -- 172.21.15.40:0/3890152299 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc6b8051570 con 0x7fc6c40711e0 2023-11-08T08:18:59.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.615+0000 7fc6c9dbc700 1 -- 172.21.15.40:0/3890152299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc6c415bda0 con 0x7fc6c40711e0 2023-11-08T08:18:59.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.615+0000 7fc6c17fa700 1 -- 172.21.15.40:0/3890152299 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc6b802b0a0 con 0x7fc6c40711e0 2023-11-08T08:18:59.621 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:18:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.615+0000 7fc6c9dbc700 1 -- 172.21.15.40:0/3890152299 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc6b0064ee0 msgr2=0x7fc6b0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.619+0000 7fc6c9dbc700 1 --2- 172.21.15.40:0/3890152299 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc6b0064ee0 0x7fc6b0067390 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc6c41528b0 tx=0x7fc6b4008040 comp rx=0 tx=0).stop 2023-11-08T08:18:59.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.619+0000 7fc6c9dbc700 1 -- 172.21.15.40:0/3890152299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6c40711e0 msgr2=0x7fc6c4158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:59.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.619+0000 7fc6c9dbc700 1 --2- 172.21.15.40:0/3890152299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6c40711e0 0x7fc6c4158d30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8000c00 tx=0x7fc6b8004770 comp rx=0 tx=0).stop 2023-11-08T08:18:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.619+0000 7fc6c9dbc700 1 -- 172.21.15.40:0/3890152299 shutdown_connections 2023-11-08T08:18:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.619+0000 7fc6c9dbc700 1 --2- 172.21.15.40:0/3890152299 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc6b0064ee0 0x7fc6b0067390 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.619+0000 7fc6c9dbc700 1 --2- 172.21.15.40:0/3890152299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6c40b1330 0x7fc6c40b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.619+0000 7fc6c9dbc700 1 --2- 172.21.15.40:0/3890152299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6c40aeb80 0x7fc6c414e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.619+0000 7fc6c9dbc700 1 --2- 172.21.15.40:0/3890152299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6c40711e0 0x7fc6c4158d30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.619+0000 7fc6c9dbc700 1 -- 172.21.15.40:0/3890152299 >> 172.21.15.40:0/3890152299 conn(0x7fc6c40cc330 msgr2=0x7fc6c4159e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:59.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.619+0000 7fc6c9dbc700 1 -- 172.21.15.40:0/3890152299 shutdown_connections 2023-11-08T08:18:59.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.619+0000 7fc6c9dbc700 1 -- 172.21.15.40:0/3890152299 wait complete. 2023-11-08T08:18:59.645 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:18:59.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.827+0000 7f1d4ca15700 1 Processor -- start 2023-11-08T08:18:59.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.827+0000 7f1d4ca15700 1 -- start start 2023-11-08T08:18:59.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.831+0000 7f1d4ca15700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1d4815c380 0x7f1d4806aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:59.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.831+0000 7f1d4ca15700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d4806b570 con 0x7f1d480d1af0 2023-11-08T08:18:59.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.831+0000 7f1d4ca15700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d4815c750 con 0x7f1d48069f50 2023-11-08T08:18:59.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.831+0000 7f1d4ca15700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d480b5360 con 0x7f1d4815c380 2023-11-08T08:18:59.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.831+0000 7f1d4759e700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1d480d1af0 0x7f1d480ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42910/0 (socket says 172.21.15.40:42910) 2023-11-08T08:18:59.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.831+0000 7f1d4759e700 1 -- 172.21.15.40:0/2473696588 learned_addr learned my addr 172.21.15.40:0/2473696588 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:18:59.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.831+0000 7f1d46d9d700 1 --2- 172.21.15.40:0/2473696588 >> v2:172.21.15.169:3300/0 conn(0x7f1d4815c380 0x7f1d4806aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:59.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.831+0000 7f1d45d9b700 1 -- 172.21.15.40:0/2473696588 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2372337726 0 0) 0x7f1d4806b570 con 0x7f1d480d1af0 2023-11-08T08:18:59.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.831+0000 7f1d45d9b700 1 -- 172.21.15.40:0/2473696588 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d300035d0 con 0x7f1d480d1af0 2023-11-08T08:18:59.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.831+0000 7f1d45d9b700 1 -- 172.21.15.40:0/2473696588 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3099814847 0 0) 0x7f1d300035d0 con 0x7f1d480d1af0 2023-11-08T08:18:59.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.831+0000 7f1d46d9d700 1 -- 172.21.15.40:0/2473696588 >> v1:172.21.15.169:6789/0 conn(0x7f1d48069f50 legacy=0x7f1d480d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:59.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.831+0000 7f1d46d9d700 1 -- 172.21.15.40:0/2473696588 >> v1:172.21.15.40:6789/0 conn(0x7f1d480d1af0 legacy=0x7f1d480ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:59.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.831+0000 7f1d46d9d700 1 -- 172.21.15.40:0/2473696588 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d480d0810 con 0x7f1d4815c380 2023-11-08T08:18:59.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.831+0000 7f1d46d9d700 1 --2- 172.21.15.40:0/2473696588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4815c380 0x7f1d4806aeb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009880 tx=0x7f1d38009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aff60b3044dbcf34 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:59.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.831+0000 7f1d45d9b700 1 -- 172.21.15.40:0/2473696588 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d400019f0 con 0x7f1d4815c380 2023-11-08T08:18:59.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.831+0000 7f1d45d9b700 1 -- 172.21.15.40:0/2473696588 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d38004030 con 0x7f1d4815c380 2023-11-08T08:18:59.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.831+0000 7f1d45d9b700 1 -- 172.21.15.40:0/2473696588 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d380043e0 con 0x7f1d4815c380 2023-11-08T08:18:59.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.835+0000 7f1d4ca15700 1 -- 172.21.15.40:0/2473696588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4815c380 msgr2=0x7f1d4806aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:59.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.835+0000 7f1d4ca15700 1 --2- 172.21.15.40:0/2473696588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4815c380 0x7f1d4806aeb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009880 tx=0x7f1d38009b90 comp rx=0 tx=0).stop 2023-11-08T08:18:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.835+0000 7f1d4ca15700 1 -- 172.21.15.40:0/2473696588 shutdown_connections 2023-11-08T08:18:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.835+0000 7f1d4ca15700 1 --2- 172.21.15.40:0/2473696588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4815c380 0x7f1d4806aeb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.835+0000 7f1d4ca15700 1 -- 172.21.15.40:0/2473696588 >> 172.21.15.40:0/2473696588 conn(0x7f1d4806cb80 msgr2=0x7f1d480d2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:18:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.835+0000 7f1d4ca15700 1 -- 172.21.15.40:0/2473696588 shutdown_connections 2023-11-08T08:18:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.835+0000 7f1d4ca15700 1 -- 172.21.15.40:0/2473696588 wait complete. 2023-11-08T08:18:59.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.839+0000 7f1d4ca15700 1 Processor -- start 2023-11-08T08:18:59.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.839+0000 7f1d4ca15700 1 -- start start 2023-11-08T08:18:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.839+0000 7f1d4ca15700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d48069f50 0x7f1d481588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:59.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.839+0000 7f1d4ca15700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d480d1af0 0x7f1d480ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:59.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.839+0000 7f1d4ca15700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4815c380 0x7f1d480ce610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:59.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.839+0000 7f1d4ca15700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4815b250 con 0x7f1d480d1af0 2023-11-08T08:18:59.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.839+0000 7f1d4ca15700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d480b6400 con 0x7f1d4815c380 2023-11-08T08:18:59.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.839+0000 7f1d4ca15700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d480b66e0 con 0x7f1d48069f50 2023-11-08T08:18:59.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.839+0000 7f1d4659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d480d1af0 0x7f1d480ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:59.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.839+0000 7f1d46d9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d48069f50 0x7f1d481588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:59.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.839+0000 7f1d4659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d480d1af0 0x7f1d480ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56048/0 (socket says 172.21.15.40:56048) 2023-11-08T08:18:59.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.839+0000 7f1d46d9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d48069f50 0x7f1d481588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:33948/0 (socket says 172.21.15.40:33948) 2023-11-08T08:18:59.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.839+0000 7f1d4659c700 1 -- 172.21.15.40:0/116854319 learned_addr learned my addr 172.21.15.40:0/116854319 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:18:59.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.843+0000 7f1d4759e700 1 --2- 172.21.15.40:0/116854319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4815c380 0x7f1d480ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:59.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.843+0000 7f1d4659c700 1 -- 172.21.15.40:0/116854319 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d48069f50 msgr2=0x7f1d481588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:59.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.843+0000 7f1d4659c700 1 --2- 172.21.15.40:0/116854319 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d48069f50 0x7f1d481588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.843+0000 7f1d4659c700 1 -- 172.21.15.40:0/116854319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4815c380 msgr2=0x7f1d480ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:18:59.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.843+0000 7f1d4659c700 1 --2- 172.21.15.40:0/116854319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4815c380 0x7f1d480ce610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:18:59.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.843+0000 7f1d4659c700 1 -- 172.21.15.40:0/116854319 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d380094b0 con 0x7f1d480d1af0 2023-11-08T08:18:59.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.843+0000 7f1d46d9d700 1 --2- 172.21.15.40:0/116854319 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d48069f50 0x7f1d481588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:18:59.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.843+0000 7f1d4659c700 1 --2- 172.21.15.40:0/116854319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d480d1af0 0x7f1d480ce0d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00c4d0 tx=0x7f1d3c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:59.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.843+0000 7f1d2f7fe700 1 -- 172.21.15.40:0/116854319 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c0052d0 con 0x7f1d480d1af0 2023-11-08T08:18:59.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.843+0000 7f1d2f7fe700 1 -- 172.21.15.40:0/116854319 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3c005430 con 0x7f1d480d1af0 2023-11-08T08:18:59.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.843+0000 7f1d4ca15700 1 -- 172.21.15.40:0/116854319 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d480d0dd0 con 0x7f1d480d1af0 2023-11-08T08:18:59.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.843+0000 7f1d2f7fe700 1 -- 172.21.15.40:0/116854319 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c0055a0 con 0x7f1d480d1af0 2023-11-08T08:18:59.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.843+0000 7f1d4ca15700 1 -- 172.21.15.40:0/116854319 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d480d0af0 con 0x7f1d480d1af0 2023-11-08T08:18:59.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.843+0000 7f1d4ca15700 1 -- 172.21.15.40:0/116854319 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d480cdb40 con 0x7f1d480d1af0 2023-11-08T08:18:59.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.847+0000 7f1d2f7fe700 1 -- 172.21.15.40:0/116854319 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1d3c019030 con 0x7f1d480d1af0 2023-11-08T08:18:59.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.847+0000 7f1d2f7fe700 1 --2- 172.21.15.40:0/116854319 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d34064f10 0x7f1d340673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:18:59.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.847+0000 7f1d2f7fe700 1 -- 172.21.15.40:0/116854319 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1d3c080cb0 con 0x7f1d480d1af0 2023-11-08T08:18:59.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.847+0000 7f1d46d9d700 1 --2- 172.21.15.40:0/116854319 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d34064f10 0x7f1d340673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:18:59.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.847+0000 7f1d46d9d700 1 --2- 172.21.15.40:0/116854319 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d34064f10 0x7f1d340673c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1d380090b0 tx=0x7f1d38010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:18:59.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:18:59.855+0000 7f1d2f7fe700 1 -- 172.21.15.40:0/116854319 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1d3c04f1a0 con 0x7f1d480d1af0 2023-11-08T08:19:00.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.031+0000 7f1d4ca15700 1 -- 172.21.15.40:0/116854319 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1d480b6ba0 con 0x7f1d480d1af0 2023-11-08T08:19:00.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.031+0000 7f1d2f7fe700 1 -- 172.21.15.40:0/116854319 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1d3c0527c0 con 0x7f1d480d1af0 2023-11-08T08:19:00.039 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:19:00.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.035+0000 7f1d4ca15700 1 -- 172.21.15.40:0/116854319 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d34064f10 msgr2=0x7f1d340673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:00.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.035+0000 7f1d4ca15700 1 --2- 172.21.15.40:0/116854319 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d34064f10 0x7f1d340673c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1d380090b0 tx=0x7f1d38010000 comp rx=0 tx=0).stop 2023-11-08T08:19:00.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.035+0000 7f1d4ca15700 1 -- 172.21.15.40:0/116854319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d480d1af0 msgr2=0x7f1d480ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:00.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.035+0000 7f1d4ca15700 1 --2- 172.21.15.40:0/116854319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d480d1af0 0x7f1d480ce0d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00c4d0 tx=0x7f1d3c00c7e0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.035+0000 7f1d4ca15700 1 -- 172.21.15.40:0/116854319 shutdown_connections 2023-11-08T08:19:00.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.035+0000 7f1d4ca15700 1 --2- 172.21.15.40:0/116854319 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d34064f10 0x7f1d340673c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.035+0000 7f1d4ca15700 1 --2- 172.21.15.40:0/116854319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4815c380 0x7f1d480ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.035+0000 7f1d4ca15700 1 --2- 172.21.15.40:0/116854319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d480d1af0 0x7f1d480ce0d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.035+0000 7f1d4ca15700 1 --2- 172.21.15.40:0/116854319 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d48069f50 0x7f1d481588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.035+0000 7f1d4ca15700 1 -- 172.21.15.40:0/116854319 >> 172.21.15.40:0/116854319 conn(0x7f1d4806cb80 msgr2=0x7f1d481e7770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:00.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.035+0000 7f1d4ca15700 1 -- 172.21.15.40:0/116854319 shutdown_connections 2023-11-08T08:19:00.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.035+0000 7f1d4ca15700 1 -- 172.21.15.40:0/116854319 wait complete. 2023-11-08T08:19:00.063 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:19:00.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.235+0000 7f4944f0e700 1 Processor -- start 2023-11-08T08:19:00.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.235+0000 7f4944f0e700 1 -- start start 2023-11-08T08:19:00.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.235+0000 7f4944f0e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f494015c640 0x7f49400d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:00.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.235+0000 7f4944f0e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4940071aa0 0x7f494006d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:00.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.235+0000 7f4944f0e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f494006c600 con 0x7f4940069e00 2023-11-08T08:19:00.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.235+0000 7f4944f0e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f494006cef0 con 0x7f4940071aa0 2023-11-08T08:19:00.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.235+0000 7f4944f0e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49400722f0 con 0x7f494015c640 2023-11-08T08:19:00.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.235+0000 7f493ffff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4940069e00 0x7f49400d0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42914/0 (socket says 172.21.15.40:42914) 2023-11-08T08:19:00.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.235+0000 7f493ffff700 1 -- 172.21.15.40:0/1514514129 learned_addr learned my addr 172.21.15.40:0/1514514129 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:00.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.235+0000 7f493effd700 1 --2- 172.21.15.40:0/1514514129 >> v2:172.21.15.40:3300/0 conn(0x7f4940071aa0 0x7f494006d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:00.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.235+0000 7f493f7fe700 1 --2- 172.21.15.40:0/1514514129 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f494015c640 0x7f49400d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:00.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.239+0000 7f493effd700 1 -- 172.21.15.40:0/1514514129 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f494015c640 msgr2=0x7f49400d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:00.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.239+0000 7f493effd700 1 --2- 172.21.15.40:0/1514514129 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f494015c640 0x7f49400d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.239+0000 7f493effd700 1 -- 172.21.15.40:0/1514514129 >> v1:172.21.15.40:6789/0 conn(0x7f4940069e00 legacy=0x7f49400d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:00.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.239+0000 7f493effd700 1 -- 172.21.15.40:0/1514514129 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494015ca10 con 0x7f4940071aa0 2023-11-08T08:19:00.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.239+0000 7f493effd700 1 --2- 172.21.15.40:0/1514514129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4940071aa0 0x7f494006d310 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4934009850 tx=0x7f4934009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748316acc280d4c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:00.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.239+0000 7f493e7fc700 1 -- 172.21.15.40:0/1514514129 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934011830 con 0x7f4940071aa0 2023-11-08T08:19:00.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.239+0000 7f493e7fc700 1 -- 172.21.15.40:0/1514514129 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4934011990 con 0x7f4940071aa0 2023-11-08T08:19:00.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.239+0000 7f493e7fc700 1 -- 172.21.15.40:0/1514514129 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934011d40 con 0x7f4940071aa0 2023-11-08T08:19:00.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.239+0000 7f4944f0e700 1 -- 172.21.15.40:0/1514514129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4940071aa0 msgr2=0x7f494006d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:00.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.239+0000 7f4944f0e700 1 --2- 172.21.15.40:0/1514514129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4940071aa0 0x7f494006d310 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4934009850 tx=0x7f4934009b60 comp rx=0 tx=0).stop 2023-11-08T08:19:00.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.239+0000 7f4944f0e700 1 -- 172.21.15.40:0/1514514129 shutdown_connections 2023-11-08T08:19:00.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.239+0000 7f4944f0e700 1 --2- 172.21.15.40:0/1514514129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4940071aa0 0x7f494006d310 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.239+0000 7f4944f0e700 1 --2- 172.21.15.40:0/1514514129 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f494015c640 0x7f49400d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.239+0000 7f4944f0e700 1 -- 172.21.15.40:0/1514514129 >> 172.21.15.40:0/1514514129 conn(0x7f49400b6520 msgr2=0x7f4940071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:00.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.239+0000 7f4944f0e700 1 -- 172.21.15.40:0/1514514129 shutdown_connections 2023-11-08T08:19:00.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.239+0000 7f4944f0e700 1 -- 172.21.15.40:0/1514514129 wait complete. 2023-11-08T08:19:00.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.243+0000 7f4944f0e700 1 Processor -- start 2023-11-08T08:19:00.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.243+0000 7f4944f0e700 1 -- start start 2023-11-08T08:19:00.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.243+0000 7f4944f0e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4940069e00 0x7f494009bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:00.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.243+0000 7f4944f0e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4940071aa0 0x7f4940155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:00.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.243+0000 7f4944f0e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f494015c640 0x7f4940155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:00.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.243+0000 7f4944f0e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49400cb680 con 0x7f494015c640 2023-11-08T08:19:00.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.243+0000 7f4944f0e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49400d1280 con 0x7f4940069e00 2023-11-08T08:19:00.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.243+0000 7f4944f0e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49400cb920 con 0x7f4940071aa0 2023-11-08T08:19:00.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.243+0000 7f493ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f494015c640 0x7f4940155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:00.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.243+0000 7f493f7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4940069e00 0x7f494009bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:00.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.243+0000 7f493effd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4940071aa0 0x7f4940155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:00.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.243+0000 7f493f7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4940069e00 0x7f494009bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:37122/0 (socket says 172.21.15.40:37122) 2023-11-08T08:19:00.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.243+0000 7f493f7fe700 1 -- 172.21.15.40:0/1838078276 learned_addr learned my addr 172.21.15.40:0/1838078276 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:00.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.243+0000 7f493effd700 1 -- 172.21.15.40:0/1838078276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4940069e00 msgr2=0x7f494009bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:00.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.243+0000 7f493effd700 1 --2- 172.21.15.40:0/1838078276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4940069e00 0x7f494009bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.243+0000 7f493effd700 1 -- 172.21.15.40:0/1838078276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f494015c640 msgr2=0x7f4940155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:00.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.243+0000 7f493effd700 1 --2- 172.21.15.40:0/1838078276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f494015c640 0x7f4940155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.243+0000 7f493effd700 1 -- 172.21.15.40:0/1838078276 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49340094b0 con 0x7f4940071aa0 2023-11-08T08:19:00.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.243+0000 7f493ffff700 1 --2- 172.21.15.40:0/1838078276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f494015c640 0x7f4940155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:00.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.247+0000 7f493effd700 1 --2- 172.21.15.40:0/1838078276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4940071aa0 0x7f4940155760 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4934016000 tx=0x7f4934004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:00.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.247+0000 7f4927fff700 1 -- 172.21.15.40:0/1838078276 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934004610 con 0x7f4940071aa0 2023-11-08T08:19:00.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.247+0000 7f4927fff700 1 -- 172.21.15.40:0/1838078276 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493401e070 con 0x7f4940071aa0 2023-11-08T08:19:00.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.247+0000 7f4944f0e700 1 -- 172.21.15.40:0/1838078276 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494006c830 con 0x7f4940071aa0 2023-11-08T08:19:00.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.247+0000 7f4927fff700 1 -- 172.21.15.40:0/1838078276 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49340038e0 con 0x7f4940071aa0 2023-11-08T08:19:00.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.247+0000 7f4944f0e700 1 -- 172.21.15.40:0/1838078276 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4940072010 con 0x7f4940071aa0 2023-11-08T08:19:00.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.247+0000 7f4944f0e700 1 -- 172.21.15.40:0/1838078276 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494006cb10 con 0x7f4940071aa0 2023-11-08T08:19:00.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.247+0000 7f4927fff700 1 -- 172.21.15.40:0/1838078276 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f493400f050 con 0x7f4940071aa0 2023-11-08T08:19:00.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.251+0000 7f4927fff700 1 --2- 172.21.15.40:0/1838078276 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f492c064f10 0x7f492c0673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:00.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.251+0000 7f4927fff700 1 -- 172.21.15.40:0/1838078276 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4934082950 con 0x7f4940071aa0 2023-11-08T08:19:00.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.251+0000 7f493f7fe700 1 --2- 172.21.15.40:0/1838078276 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f492c064f10 0x7f492c0673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:00.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.251+0000 7f493f7fe700 1 --2- 172.21.15.40:0/1838078276 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f492c064f10 0x7f492c0673c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4930009cb0 tx=0x7f4930009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:00.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.259+0000 7f4927fff700 1 -- 172.21.15.40:0/1838078276 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4934050d50 con 0x7f4940071aa0 2023-11-08T08:19:00.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.467+0000 7f4944f0e700 1 -- 172.21.15.40:0/1838078276 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4940073cb0 con 0x7f4940071aa0 2023-11-08T08:19:00.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.467+0000 7f4927fff700 1 -- 172.21.15.40:0/1838078276 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3521 (secure 0 0 0) 0x7f4934020de0 con 0x7f4940071aa0 2023-11-08T08:19:00.475 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:00.476 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/196211087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":196211087},{"type":"v1","addr":"172.21.15.40:6834","nonce":196211087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/67127106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":67127106},{"type":"v1","addr":"172.21.15.40:6835","nonce":67127106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1231240563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1231240563},{"type":"v1","addr":"172.21.15.169:6835","nonce":1231240563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1989495204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1989495204},{"type":"v1","addr":"172.21.15.169:6837","nonce":1989495204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-08T08:19:00.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.475+0000 7f4944f0e700 1 -- 172.21.15.40:0/1838078276 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f492c064f10 msgr2=0x7f492c0673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:00.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.475+0000 7f4944f0e700 1 --2- 172.21.15.40:0/1838078276 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f492c064f10 0x7f492c0673c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4930009cb0 tx=0x7f4930009240 comp rx=0 tx=0).stop 2023-11-08T08:19:00.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.475+0000 7f4944f0e700 1 -- 172.21.15.40:0/1838078276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4940071aa0 msgr2=0x7f4940155760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:00.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.475+0000 7f4944f0e700 1 --2- 172.21.15.40:0/1838078276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4940071aa0 0x7f4940155760 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4934016000 tx=0x7f4934004770 comp rx=0 tx=0).stop 2023-11-08T08:19:00.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.475+0000 7f4944f0e700 1 -- 172.21.15.40:0/1838078276 shutdown_connections 2023-11-08T08:19:00.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.475+0000 7f4944f0e700 1 --2- 172.21.15.40:0/1838078276 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f492c064f10 0x7f492c0673c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.475+0000 7f4944f0e700 1 --2- 172.21.15.40:0/1838078276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f494015c640 0x7f4940155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.475+0000 7f4944f0e700 1 --2- 172.21.15.40:0/1838078276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4940071aa0 0x7f4940155760 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.475+0000 7f4944f0e700 1 --2- 172.21.15.40:0/1838078276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4940069e00 0x7f494009bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.475+0000 7f4944f0e700 1 -- 172.21.15.40:0/1838078276 >> 172.21.15.40:0/1838078276 conn(0x7f49400b6520 msgr2=0x7f49400b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:00.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.475+0000 7f4944f0e700 1 -- 172.21.15.40:0/1838078276 shutdown_connections 2023-11-08T08:19:00.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.475+0000 7f4944f0e700 1 -- 172.21.15.40:0/1838078276 wait complete. 2023-11-08T08:19:00.482 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 3 2023-11-08T08:19:00.507 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:19:00.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.679+0000 7fd7ecc45700 1 Processor -- start 2023-11-08T08:19:00.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.679+0000 7fd7ecc45700 1 -- start start 2023-11-08T08:19:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.679+0000 7fd7ecc45700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd7e815c1a0 0x7fd7e806c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.679+0000 7fd7ecc45700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7e8070790 0x7fd7e806a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7ecc45700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e80adf90 con 0x7fd7e80cdeb0 2023-11-08T08:19:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7ecc45700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e8069bf0 con 0x7fd7e815c1a0 2023-11-08T08:19:00.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7ecc45700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e80ce370 con 0x7fd7e8070790 2023-11-08T08:19:00.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7e77fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd7e815c1a0 0x7fd7e806c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:00.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7e6ffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd7e80cdeb0 0x7fd7e8070080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42926/0 (socket says 172.21.15.40:42926) 2023-11-08T08:19:00.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7e77fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd7e815c1a0 0x7fd7e806c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56078/0 (socket says 172.21.15.40:56078) 2023-11-08T08:19:00.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7e6ffd700 1 -- 172.21.15.40:0/1011819884 learned_addr learned my addr 172.21.15.40:0/1011819884 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:00.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7e7fff700 1 --2- 172.21.15.40:0/1011819884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7e8070790 0x7fd7e806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:00.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7e77fe700 1 -- 172.21.15.40:0/1011819884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7e8070790 msgr2=0x7fd7e806a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:00.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7e77fe700 1 --2- 172.21.15.40:0/1011819884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7e8070790 0x7fd7e806a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7e77fe700 1 -- 172.21.15.40:0/1011819884 >> v1:172.21.15.40:6789/0 conn(0x7fd7e80cdeb0 legacy=0x7fd7e8070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:00.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7e77fe700 1 -- 172.21.15.40:0/1011819884 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e80cc4a0 con 0x7fd7e815c1a0 2023-11-08T08:19:00.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7e7fff700 1 --2- 172.21.15.40:0/1011819884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7e8070790 0x7fd7e806a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:00.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7e67fc700 1 -- 172.21.15.40:0/1011819884 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 589930313 0 0) 0x7fd7e80adf90 con 0x7fd7e80cdeb0 2023-11-08T08:19:00.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7e77fe700 1 --2- 172.21.15.40:0/1011819884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7e815c1a0 0x7fd7e806c8b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8009850 tx=0x7fd7d8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fff617a2ee82d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:00.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7e67fc700 1 -- 172.21.15.40:0/1011819884 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8011830 con 0x7fd7e815c1a0 2023-11-08T08:19:00.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7e67fc700 1 -- 172.21.15.40:0/1011819884 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d8011990 con 0x7fd7e815c1a0 2023-11-08T08:19:00.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7e67fc700 1 -- 172.21.15.40:0/1011819884 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8011d20 con 0x7fd7e815c1a0 2023-11-08T08:19:00.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7ecc45700 1 -- 172.21.15.40:0/1011819884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7e815c1a0 msgr2=0x7fd7e806c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:00.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7ecc45700 1 --2- 172.21.15.40:0/1011819884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7e815c1a0 0x7fd7e806c8b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8009850 tx=0x7fd7d8009b60 comp rx=0 tx=0).stop 2023-11-08T08:19:00.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7ecc45700 1 -- 172.21.15.40:0/1011819884 shutdown_connections 2023-11-08T08:19:00.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7ecc45700 1 --2- 172.21.15.40:0/1011819884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7e8070790 0x7fd7e806a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7ecc45700 1 --2- 172.21.15.40:0/1011819884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7e815c1a0 0x7fd7e806c8b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7ecc45700 1 -- 172.21.15.40:0/1011819884 >> 172.21.15.40:0/1011819884 conn(0x7fd7e80b0f80 msgr2=0x7fd7e80ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:00.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7ecc45700 1 -- 172.21.15.40:0/1011819884 shutdown_connections 2023-11-08T08:19:00.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.683+0000 7fd7ecc45700 1 -- 172.21.15.40:0/1011819884 wait complete. 2023-11-08T08:19:00.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.687+0000 7fd7ecc45700 1 Processor -- start 2023-11-08T08:19:00.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.687+0000 7fd7ecc45700 1 -- start start 2023-11-08T08:19:00.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7ecc45700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7e8070790 0x7fd7e80b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:00.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7ecc45700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7e80cdeb0 0x7fd7e815ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:00.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7ecc45700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7e815c1a0 0x7fd7e815ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:00.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7ecc45700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e81565a0 con 0x7fd7e80cdeb0 2023-11-08T08:19:00.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7ecc45700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e80c1ea0 con 0x7fd7e815c1a0 2023-11-08T08:19:00.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7ecc45700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e81f2380 con 0x7fd7e8070790 2023-11-08T08:19:00.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7e6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7e80cdeb0 0x7fd7e815ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:00.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7e6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7e80cdeb0 0x7fd7e815ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56082/0 (socket says 172.21.15.40:56082) 2023-11-08T08:19:00.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7e6ffd700 1 -- 172.21.15.40:0/1878375250 learned_addr learned my addr 172.21.15.40:0/1878375250 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:00.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7e77fe700 1 --2- 172.21.15.40:0/1878375250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7e8070790 0x7fd7e80b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:00.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7e7fff700 1 --2- 172.21.15.40:0/1878375250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7e815c1a0 0x7fd7e815ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:00.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7e6ffd700 1 -- 172.21.15.40:0/1878375250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7e8070790 msgr2=0x7fd7e80b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:00.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7e6ffd700 1 --2- 172.21.15.40:0/1878375250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7e8070790 0x7fd7e80b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7e6ffd700 1 -- 172.21.15.40:0/1878375250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7e815c1a0 msgr2=0x7fd7e815ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:00.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7e6ffd700 1 --2- 172.21.15.40:0/1878375250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7e815c1a0 0x7fd7e815ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7e6ffd700 1 -- 172.21.15.40:0/1878375250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d80094b0 con 0x7fd7e80cdeb0 2023-11-08T08:19:00.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7e6ffd700 1 --2- 172.21.15.40:0/1878375250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7e80cdeb0 0x7fd7e815ea20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0011e0 tx=0x7fd7dc00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:00.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7cffff700 1 -- 172.21.15.40:0/1878375250 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc005d90 con 0x7fd7e80cdeb0 2023-11-08T08:19:00.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7cffff700 1 -- 172.21.15.40:0/1878375250 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7dc0070d0 con 0x7fd7e80cdeb0 2023-11-08T08:19:00.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7cffff700 1 -- 172.21.15.40:0/1878375250 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc007460 con 0x7fd7e80cdeb0 2023-11-08T08:19:00.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7ecc45700 1 -- 172.21.15.40:0/1878375250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e806cdf0 con 0x7fd7e80cdeb0 2023-11-08T08:19:00.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.691+0000 7fd7ecc45700 1 -- 172.21.15.40:0/1878375250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e80cc780 con 0x7fd7e80cdeb0 2023-11-08T08:19:00.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.695+0000 7fd7ecc45700 1 -- 172.21.15.40:0/1878375250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e815c570 con 0x7fd7e80cdeb0 2023-11-08T08:19:00.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.695+0000 7fd7cffff700 1 -- 172.21.15.40:0/1878375250 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd7dc019030 con 0x7fd7e80cdeb0 2023-11-08T08:19:00.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.699+0000 7fd7cffff700 1 --2- 172.21.15.40:0/1878375250 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd7d4064ee0 0x7fd7d4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:00.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.699+0000 7fd7cffff700 1 -- 172.21.15.40:0/1878375250 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd7dc080ef0 con 0x7fd7e80cdeb0 2023-11-08T08:19:00.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.699+0000 7fd7e77fe700 1 --2- 172.21.15.40:0/1878375250 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd7d4064ee0 0x7fd7d4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:00.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.699+0000 7fd7e77fe700 1 --2- 172.21.15.40:0/1878375250 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd7d4064ee0 0x7fd7d4067390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8014040 tx=0x7fd7d8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:00.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.703+0000 7fd7cffff700 1 -- 172.21.15.40:0/1878375250 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd7dc04f2f0 con 0x7fd7e80cdeb0 2023-11-08T08:19:00.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.875+0000 7fd7ecc45700 1 -- 172.21.15.40:0/1878375250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7e8073cb0 con 0x7fd7e80cdeb0 2023-11-08T08:19:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.879+0000 7fd7cffff700 1 -- 172.21.15.40:0/1878375250 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+14168 (secure 0 0 0) 0x7fd7dc052910 con 0x7fd7e80cdeb0 2023-11-08T08:19:00.884 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:00.885 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:18:59.753552+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:19:00.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.883+0000 7fd7ecc45700 1 -- 172.21.15.40:0/1878375250 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd7d4064ee0 msgr2=0x7fd7d4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:00.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.883+0000 7fd7ecc45700 1 --2- 172.21.15.40:0/1878375250 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd7d4064ee0 0x7fd7d4067390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8014040 tx=0x7fd7d8008030 comp rx=0 tx=0).stop 2023-11-08T08:19:00.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.883+0000 7fd7ecc45700 1 -- 172.21.15.40:0/1878375250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7e80cdeb0 msgr2=0x7fd7e815ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:00.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.883+0000 7fd7ecc45700 1 --2- 172.21.15.40:0/1878375250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7e80cdeb0 0x7fd7e815ea20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0011e0 tx=0x7fd7dc00ca30 comp rx=0 tx=0).stop 2023-11-08T08:19:00.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.883+0000 7fd7ecc45700 1 -- 172.21.15.40:0/1878375250 shutdown_connections 2023-11-08T08:19:00.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.883+0000 7fd7ecc45700 1 --2- 172.21.15.40:0/1878375250 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd7d4064ee0 0x7fd7d4067390 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.883+0000 7fd7ecc45700 1 --2- 172.21.15.40:0/1878375250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7e815c1a0 0x7fd7e815ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.883+0000 7fd7ecc45700 1 --2- 172.21.15.40:0/1878375250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7e80cdeb0 0x7fd7e815ea20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.883+0000 7fd7ecc45700 1 --2- 172.21.15.40:0/1878375250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7e8070790 0x7fd7e80b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:00.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.883+0000 7fd7ecc45700 1 -- 172.21.15.40:0/1878375250 >> 172.21.15.40:0/1878375250 conn(0x7fd7e80b0f80 msgr2=0x7fd7e8157530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:00.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.883+0000 7fd7ecc45700 1 -- 172.21.15.40:0/1878375250 shutdown_connections 2023-11-08T08:19:00.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:00.883+0000 7fd7ecc45700 1 -- 172.21.15.40:0/1878375250 wait complete. 2023-11-08T08:19:00.911 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T08:19:00.912 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:19:01.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.095+0000 7f823d3aa700 1 Processor -- start 2023-11-08T08:19:01.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.095+0000 7f823d3aa700 1 -- start start 2023-11-08T08:19:01.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.095+0000 7f823d3aa700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f823815c150 0x7f82380b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:01.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.095+0000 7f823d3aa700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f823806ff10 0x7f8238070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:01.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.095+0000 7f823d3aa700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823815c520 con 0x7f823815fc30 2023-11-08T08:19:01.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f823d3aa700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82380b4240 con 0x7f823815c150 2023-11-08T08:19:01.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f823d3aa700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82380708a0 con 0x7f823806ff10 2023-11-08T08:19:01.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f8236ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f823815c150 0x7f82380b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:01.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f82377fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f823815fc30 0x7f82381600b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42936/0 (socket says 172.21.15.40:42936) 2023-11-08T08:19:01.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f82377fe700 1 -- 172.21.15.40:0/2320053819 learned_addr learned my addr 172.21.15.40:0/2320053819 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:01.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f82367fc700 1 --2- 172.21.15.40:0/2320053819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f823806ff10 0x7f8238070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:01.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f8236ffd700 1 -- 172.21.15.40:0/2320053819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f823806ff10 msgr2=0x7f8238070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:01.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f8236ffd700 1 --2- 172.21.15.40:0/2320053819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f823806ff10 0x7f8238070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f8236ffd700 1 -- 172.21.15.40:0/2320053819 >> v1:172.21.15.40:6789/0 conn(0x7f823815fc30 legacy=0x7f82381600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:01.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f8236ffd700 1 -- 172.21.15.40:0/2320053819 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823806ac00 con 0x7f823815c150 2023-11-08T08:19:01.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f8235ffb700 1 -- 172.21.15.40:0/2320053819 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 65133259 0 0) 0x7f823815c520 con 0x7f823815fc30 2023-11-08T08:19:01.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f8236ffd700 1 --2- 172.21.15.40:0/2320053819 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f823815c150 0x7f82380b3d00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8228009850 tx=0x7f8228009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bb217e543ed6d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:01.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f8235ffb700 1 -- 172.21.15.40:0/2320053819 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228011830 con 0x7f823815c150 2023-11-08T08:19:01.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f8235ffb700 1 -- 172.21.15.40:0/2320053819 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8228011990 con 0x7f823815c150 2023-11-08T08:19:01.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f8235ffb700 1 -- 172.21.15.40:0/2320053819 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228011d40 con 0x7f823815c150 2023-11-08T08:19:01.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f823d3aa700 1 -- 172.21.15.40:0/2320053819 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f823815c150 msgr2=0x7f82380b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:01.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f823d3aa700 1 --2- 172.21.15.40:0/2320053819 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f823815c150 0x7f82380b3d00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8228009850 tx=0x7f8228009b60 comp rx=0 tx=0).stop 2023-11-08T08:19:01.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f823d3aa700 1 -- 172.21.15.40:0/2320053819 shutdown_connections 2023-11-08T08:19:01.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f823d3aa700 1 --2- 172.21.15.40:0/2320053819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f823806ff10 0x7f8238070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f823d3aa700 1 --2- 172.21.15.40:0/2320053819 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f823815c150 0x7f82380b3d00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.099+0000 7f823d3aa700 1 -- 172.21.15.40:0/2320053819 >> 172.21.15.40:0/2320053819 conn(0x7f82380d3680 msgr2=0x7f82380cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:01.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.103+0000 7f823d3aa700 1 -- 172.21.15.40:0/2320053819 shutdown_connections 2023-11-08T08:19:01.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.103+0000 7f823d3aa700 1 -- 172.21.15.40:0/2320053819 wait complete. 2023-11-08T08:19:01.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.103+0000 7f823d3aa700 1 Processor -- start 2023-11-08T08:19:01.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.103+0000 7f823d3aa700 1 -- start start 2023-11-08T08:19:01.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f823d3aa700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f823806ff10 0x7f8238156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:01.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f823d3aa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f823815c150 0x7f82381f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:01.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f823d3aa700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f823815fc30 0x7f82381f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:01.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f823d3aa700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82380cc330 con 0x7f823815c150 2023-11-08T08:19:01.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f823d3aa700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8238156ff0 con 0x7f823815fc30 2023-11-08T08:19:01.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f823d3aa700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82381f4e10 con 0x7f823806ff10 2023-11-08T08:19:01.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f82367fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f823815c150 0x7f82381f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:01.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f82367fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f823815c150 0x7f82381f17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56110/0 (socket says 172.21.15.40:56110) 2023-11-08T08:19:01.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f82367fc700 1 -- 172.21.15.40:0/222594751 learned_addr learned my addr 172.21.15.40:0/222594751 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:01.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f82377fe700 1 --2- 172.21.15.40:0/222594751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f823815fc30 0x7f82381f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:01.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f8236ffd700 1 --2- 172.21.15.40:0/222594751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f823806ff10 0x7f8238156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:01.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f82367fc700 1 -- 172.21.15.40:0/222594751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f823806ff10 msgr2=0x7f8238156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:01.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f82367fc700 1 --2- 172.21.15.40:0/222594751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f823806ff10 0x7f8238156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f82367fc700 1 -- 172.21.15.40:0/222594751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f823815fc30 msgr2=0x7f82381f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:01.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f82367fc700 1 --2- 172.21.15.40:0/222594751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f823815fc30 0x7f82381f5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f82367fc700 1 -- 172.21.15.40:0/222594751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82280094b0 con 0x7f823815c150 2023-11-08T08:19:01.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f8236ffd700 1 --2- 172.21.15.40:0/222594751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f823806ff10 0x7f8238156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:01.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f82367fc700 1 --2- 172.21.15.40:0/222594751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f823815c150 0x7f82381f17f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f822c00e9e0 tx=0x7f822c00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:01.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f821f7fe700 1 -- 172.21.15.40:0/222594751 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c009710 con 0x7f823815c150 2023-11-08T08:19:01.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f821f7fe700 1 -- 172.21.15.40:0/222594751 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c00c850 con 0x7f823815c150 2023-11-08T08:19:01.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f823d3aa700 1 -- 172.21.15.40:0/222594751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82380d2450 con 0x7f823815c150 2023-11-08T08:19:01.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f821f7fe700 1 -- 172.21.15.40:0/222594751 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c00cbc0 con 0x7f823815c150 2023-11-08T08:19:01.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.107+0000 7f823d3aa700 1 -- 172.21.15.40:0/222594751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823806cc50 con 0x7f823815c150 2023-11-08T08:19:01.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.111+0000 7f823d3aa700 1 -- 172.21.15.40:0/222594751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82380d2730 con 0x7f823815c150 2023-11-08T08:19:01.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.111+0000 7f821f7fe700 1 -- 172.21.15.40:0/222594751 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f822c023070 con 0x7f823815c150 2023-11-08T08:19:01.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.115+0000 7f821f7fe700 1 --2- 172.21.15.40:0/222594751 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8224064ee0 0x7f8224067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:01.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.115+0000 7f821f7fe700 1 -- 172.21.15.40:0/222594751 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f822c081f40 con 0x7f823815c150 2023-11-08T08:19:01.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.115+0000 7f8236ffd700 1 --2- 172.21.15.40:0/222594751 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8224064ee0 0x7f8224067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:01.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.115+0000 7f8236ffd700 1 --2- 172.21.15.40:0/222594751 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8224064ee0 0x7f8224067390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f82280090b0 tx=0x7f8228008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:01.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.119+0000 7f821f7fe700 1 -- 172.21.15.40:0/222594751 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f822c0502e0 con 0x7f823815c150 2023-11-08T08:19:01.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.295+0000 7f823d3aa700 1 -- 172.21.15.40:0/222594751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8238073d20 con 0x7f823815c150 2023-11-08T08:19:01.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.295+0000 7f821f7fe700 1 -- 172.21.15.40:0/222594751 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f822c021020 con 0x7f823815c150 2023-11-08T08:19:01.303 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:19:01.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.303+0000 7f823d3aa700 1 -- 172.21.15.40:0/222594751 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8224064ee0 msgr2=0x7f8224067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:01.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.303+0000 7f823d3aa700 1 --2- 172.21.15.40:0/222594751 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8224064ee0 0x7f8224067390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f82280090b0 tx=0x7f8228008030 comp rx=0 tx=0).stop 2023-11-08T08:19:01.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.303+0000 7f823d3aa700 1 -- 172.21.15.40:0/222594751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f823815c150 msgr2=0x7f82381f17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:01.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.303+0000 7f823d3aa700 1 --2- 172.21.15.40:0/222594751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f823815c150 0x7f82381f17f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f822c00e9e0 tx=0x7f822c00ecf0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.303+0000 7f823d3aa700 1 -- 172.21.15.40:0/222594751 shutdown_connections 2023-11-08T08:19:01.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.303+0000 7f823d3aa700 1 --2- 172.21.15.40:0/222594751 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8224064ee0 0x7f8224067390 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.303+0000 7f823d3aa700 1 --2- 172.21.15.40:0/222594751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f823815fc30 0x7f82381f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.303+0000 7f823d3aa700 1 --2- 172.21.15.40:0/222594751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f823815c150 0x7f82381f17f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.303+0000 7f823d3aa700 1 --2- 172.21.15.40:0/222594751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f823806ff10 0x7f8238156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.303+0000 7f823d3aa700 1 -- 172.21.15.40:0/222594751 >> 172.21.15.40:0/222594751 conn(0x7f82380d3680 msgr2=0x7f82380c8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:01.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.303+0000 7f823d3aa700 1 -- 172.21.15.40:0/222594751 shutdown_connections 2023-11-08T08:19:01.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.303+0000 7f823d3aa700 1 -- 172.21.15.40:0/222594751 wait complete. 2023-11-08T08:19:01.329 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:19:01.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.511+0000 7fe35ab3e700 1 Processor -- start 2023-11-08T08:19:01.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.511+0000 7fe35ab3e700 1 -- start start 2023-11-08T08:19:01.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.511+0000 7fe35ab3e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe35415c150 0x7fe3540b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:01.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.511+0000 7fe35ab3e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe35406ff10 0x7fe354070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:01.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.511+0000 7fe35ab3e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35415c520 con 0x7fe35415fc30 2023-11-08T08:19:01.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.511+0000 7fe35ab3e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3540b4240 con 0x7fe35415c150 2023-11-08T08:19:01.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.511+0000 7fe35ab3e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3540708a0 con 0x7fe35406ff10 2023-11-08T08:19:01.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.511+0000 7fe359b3c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe35415c150 0x7fe3540b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:01.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.511+0000 7fe359b3c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe35415c150 0x7fe3540b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56124/0 (socket says 172.21.15.40:56124) 2023-11-08T08:19:01.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.511+0000 7fe359b3c700 1 -- 172.21.15.40:0/1181633577 learned_addr learned my addr 172.21.15.40:0/1181633577 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:01.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.515+0000 7fe35933b700 1 --2- 172.21.15.40:0/1181633577 >> v2:172.21.15.169:3300/0 conn(0x7fe35406ff10 0x7fe354070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:01.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.515+0000 7fe359b3c700 1 -- 172.21.15.40:0/1181633577 >> v2:172.21.15.169:3300/0 conn(0x7fe35406ff10 msgr2=0x7fe354070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:01.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.515+0000 7fe359b3c700 1 --2- 172.21.15.40:0/1181633577 >> v2:172.21.15.169:3300/0 conn(0x7fe35406ff10 0x7fe354070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.515+0000 7fe359b3c700 1 -- 172.21.15.40:0/1181633577 >> v1:172.21.15.169:6789/0 conn(0x7fe35415fc30 legacy=0x7fe354160050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:01.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.515+0000 7fe359b3c700 1 -- 172.21.15.40:0/1181633577 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35406ac00 con 0x7fe35415c150 2023-11-08T08:19:01.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.515+0000 7fe35933b700 1 --2- 172.21.15.40:0/1181633577 >> v2:172.21.15.169:3300/0 conn(0x7fe35406ff10 0x7fe354070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:01.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.515+0000 7fe359b3c700 1 --2- 172.21.15.40:0/1181633577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35415c150 0x7fe3540b3d00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe354066d30 tx=0x7fe350009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746616fce347dbc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:01.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.515+0000 7fe358b3a700 1 -- 172.21.15.40:0/1181633577 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35000b8a0 con 0x7fe35415c150 2023-11-08T08:19:01.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.515+0000 7fe358b3a700 1 -- 172.21.15.40:0/1181633577 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35000ba00 con 0x7fe35415c150 2023-11-08T08:19:01.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.515+0000 7fe358b3a700 1 -- 172.21.15.40:0/1181633577 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35000bd90 con 0x7fe35415c150 2023-11-08T08:19:01.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.515+0000 7fe35ab3e700 1 -- 172.21.15.40:0/1181633577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35415c150 msgr2=0x7fe3540b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:01.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.515+0000 7fe35ab3e700 1 --2- 172.21.15.40:0/1181633577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35415c150 0x7fe3540b3d00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe354066d30 tx=0x7fe350009bf0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.515+0000 7fe35ab3e700 1 -- 172.21.15.40:0/1181633577 shutdown_connections 2023-11-08T08:19:01.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.515+0000 7fe35ab3e700 1 --2- 172.21.15.40:0/1181633577 >> v2:172.21.15.169:3300/0 conn(0x7fe35406ff10 0x7fe354070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.515+0000 7fe35ab3e700 1 --2- 172.21.15.40:0/1181633577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35415c150 0x7fe3540b3d00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.515+0000 7fe35ab3e700 1 -- 172.21.15.40:0/1181633577 >> 172.21.15.40:0/1181633577 conn(0x7fe3540d3680 msgr2=0x7fe3540cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:01.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.515+0000 7fe35ab3e700 1 -- 172.21.15.40:0/1181633577 shutdown_connections 2023-11-08T08:19:01.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.515+0000 7fe35ab3e700 1 -- 172.21.15.40:0/1181633577 wait complete. 2023-11-08T08:19:01.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.519+0000 7fe35ab3e700 1 Processor -- start 2023-11-08T08:19:01.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.519+0000 7fe35ab3e700 1 -- start start 2023-11-08T08:19:01.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.519+0000 7fe35ab3e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe35406ff10 0x7fe35415ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:01.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.519+0000 7fe35ab3e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35415c150 0x7fe3541f0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:01.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.519+0000 7fe35ab3e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe35415fc30 0x7fe3541f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:01.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.519+0000 7fe35ab3e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35415db60 con 0x7fe35415c150 2023-11-08T08:19:01.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.519+0000 7fe35ab3e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35415de40 con 0x7fe35415fc30 2023-11-08T08:19:01.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.519+0000 7fe35ab3e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35415e120 con 0x7fe35406ff10 2023-11-08T08:19:01.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.519+0000 7fe35933b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35415c150 0x7fe3541f0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:01.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.519+0000 7fe35933b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35415c150 0x7fe3541f0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56128/0 (socket says 172.21.15.40:56128) 2023-11-08T08:19:01.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.519+0000 7fe35933b700 1 -- 172.21.15.40:0/187003085 learned_addr learned my addr 172.21.15.40:0/187003085 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:01.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.519+0000 7fe359b3c700 1 --2- 172.21.15.40:0/187003085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe35406ff10 0x7fe35415ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:01.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.519+0000 7fe35a33d700 1 --2- 172.21.15.40:0/187003085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe35415fc30 0x7fe3541f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:01.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.519+0000 7fe35933b700 1 -- 172.21.15.40:0/187003085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe35406ff10 msgr2=0x7fe35415ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:01.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.519+0000 7fe35933b700 1 --2- 172.21.15.40:0/187003085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe35406ff10 0x7fe35415ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.519+0000 7fe35933b700 1 -- 172.21.15.40:0/187003085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe35415fc30 msgr2=0x7fe3541f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:01.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.519+0000 7fe35933b700 1 --2- 172.21.15.40:0/187003085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe35415fc30 0x7fe3541f4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.519+0000 7fe35933b700 1 -- 172.21.15.40:0/187003085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe350009570 con 0x7fe35415c150 2023-11-08T08:19:01.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.519+0000 7fe35933b700 1 --2- 172.21.15.40:0/187003085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35415c150 0x7fe3541f0fc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe34400e810 tx=0x7fe34400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:01.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.523+0000 7fe34a7fc700 1 -- 172.21.15.40:0/187003085 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe344005c40 con 0x7fe35415c150 2023-11-08T08:19:01.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.523+0000 7fe34a7fc700 1 -- 172.21.15.40:0/187003085 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe344005da0 con 0x7fe35415c150 2023-11-08T08:19:01.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.523+0000 7fe35ab3e700 1 -- 172.21.15.40:0/187003085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3540d2450 con 0x7fe35415c150 2023-11-08T08:19:01.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.523+0000 7fe34a7fc700 1 -- 172.21.15.40:0/187003085 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe344004250 con 0x7fe35415c150 2023-11-08T08:19:01.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.523+0000 7fe35ab3e700 1 -- 172.21.15.40:0/187003085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35406cc50 con 0x7fe35415c150 2023-11-08T08:19:01.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.523+0000 7fe35ab3e700 1 -- 172.21.15.40:0/187003085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3540d2730 con 0x7fe35415c150 2023-11-08T08:19:01.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.527+0000 7fe34a7fc700 1 -- 172.21.15.40:0/187003085 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fe344022070 con 0x7fe35415c150 2023-11-08T08:19:01.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.527+0000 7fe34a7fc700 1 --2- 172.21.15.40:0/187003085 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe338064ee0 0x7fe338067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:01.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.527+0000 7fe34a7fc700 1 -- 172.21.15.40:0/187003085 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe344081b30 con 0x7fe35415c150 2023-11-08T08:19:01.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.527+0000 7fe359b3c700 1 --2- 172.21.15.40:0/187003085 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe338064ee0 0x7fe338067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:01.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.531+0000 7fe359b3c700 1 --2- 172.21.15.40:0/187003085 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe338064ee0 0x7fe338067390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe350000c00 tx=0x7fe350008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:01.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.535+0000 7fe34a7fc700 1 -- 172.21.15.40:0/187003085 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fe34404ff30 con 0x7fe35415c150 2023-11-08T08:19:01.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.711+0000 7fe35ab3e700 1 -- 172.21.15.40:0/187003085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe354073d20 con 0x7fe35415c150 2023-11-08T08:19:01.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.711+0000 7fe34a7fc700 1 -- 172.21.15.40:0/187003085 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe344053550 con 0x7fe35415c150 2023-11-08T08:19:01.718 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:19:01.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.715+0000 7fe35ab3e700 1 -- 172.21.15.40:0/187003085 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe338064ee0 msgr2=0x7fe338067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:01.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.715+0000 7fe35ab3e700 1 --2- 172.21.15.40:0/187003085 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe338064ee0 0x7fe338067390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe350000c00 tx=0x7fe350008030 comp rx=0 tx=0).stop 2023-11-08T08:19:01.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.715+0000 7fe35ab3e700 1 -- 172.21.15.40:0/187003085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35415c150 msgr2=0x7fe3541f0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:01.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.715+0000 7fe35ab3e700 1 --2- 172.21.15.40:0/187003085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35415c150 0x7fe3541f0fc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe34400e810 tx=0x7fe34400ebd0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.715+0000 7fe35ab3e700 1 -- 172.21.15.40:0/187003085 shutdown_connections 2023-11-08T08:19:01.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.715+0000 7fe35ab3e700 1 --2- 172.21.15.40:0/187003085 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe338064ee0 0x7fe338067390 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.715+0000 7fe35ab3e700 1 --2- 172.21.15.40:0/187003085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe35415fc30 0x7fe3541f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.715+0000 7fe35ab3e700 1 --2- 172.21.15.40:0/187003085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe35415c150 0x7fe3541f0fc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.715+0000 7fe35ab3e700 1 --2- 172.21.15.40:0/187003085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe35406ff10 0x7fe35415ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.715+0000 7fe35ab3e700 1 -- 172.21.15.40:0/187003085 >> 172.21.15.40:0/187003085 conn(0x7fe3540d3680 msgr2=0x7fe3540c8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:01.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.715+0000 7fe35ab3e700 1 -- 172.21.15.40:0/187003085 shutdown_connections 2023-11-08T08:19:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.715+0000 7fe35ab3e700 1 -- 172.21.15.40:0/187003085 wait complete. 2023-11-08T08:19:01.743 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T08:19:01.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.919+0000 7fa4c4bb4700 1 Processor -- start 2023-11-08T08:19:01.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.919+0000 7fa4c4bb4700 1 -- start start 2023-11-08T08:19:01.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.919+0000 7fa4c4bb4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa4c00704b0 0x7fa4c0164960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:01.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.919+0000 7fa4c4bb4700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c00ad6c0 con 0x7fa4c0070080 2023-11-08T08:19:01.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.919+0000 7fa4c4bb4700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c00adc60 con 0x7fa4c0160710 2023-11-08T08:19:01.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.919+0000 7fa4c4bb4700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c006b1c0 con 0x7fa4c00704b0 2023-11-08T08:19:01.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.919+0000 7fa4b6ffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa4c0070080 0x7fa4c0164250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42940/0 (socket says 172.21.15.40:42940) 2023-11-08T08:19:01.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.919+0000 7fa4b6ffd700 1 -- 172.21.15.40:0/2641997410 learned_addr learned my addr 172.21.15.40:0/2641997410 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:01.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.919+0000 7fa4bffff700 1 --2- 172.21.15.40:0/2641997410 >> v2:172.21.15.40:3300/0 conn(0x7fa4c00704b0 0x7fa4c0164960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:01.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.923+0000 7fa4beffd700 1 -- 172.21.15.40:0/2641997410 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3683998942 0 0) 0x7fa4c00adc60 con 0x7fa4c0160710 2023-11-08T08:19:01.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.923+0000 7fa4beffd700 1 -- 172.21.15.40:0/2641997410 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4ac0035d0 con 0x7fa4c0160710 2023-11-08T08:19:01.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.923+0000 7fa4beffd700 1 -- 172.21.15.40:0/2641997410 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508766934 0 0) 0x7fa4c00ad6c0 con 0x7fa4c0070080 2023-11-08T08:19:01.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.923+0000 7fa4beffd700 1 -- 172.21.15.40:0/2641997410 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4c00adc60 con 0x7fa4c0070080 2023-11-08T08:19:01.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.923+0000 7fa4bffff700 1 -- 172.21.15.40:0/2641997410 >> v1:172.21.15.169:6789/0 conn(0x7fa4c0160710 legacy=0x7fa4c00cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:01.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.923+0000 7fa4bffff700 1 -- 172.21.15.40:0/2641997410 >> v1:172.21.15.40:6789/0 conn(0x7fa4c0070080 legacy=0x7fa4c0164250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:01.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.923+0000 7fa4bffff700 1 -- 172.21.15.40:0/2641997410 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c00cca30 con 0x7fa4c00704b0 2023-11-08T08:19:01.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.923+0000 7fa4bffff700 1 --2- 172.21.15.40:0/2641997410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c00704b0 0x7fa4c0164960 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009d90 tx=0x7fa4b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892a3608bfdc5032 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:01.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.923+0000 7fa4beffd700 1 -- 172.21.15.40:0/2641997410 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b800e070 con 0x7fa4c00704b0 2023-11-08T08:19:01.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.923+0000 7fa4beffd700 1 -- 172.21.15.40:0/2641997410 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b800bb20 con 0x7fa4c00704b0 2023-11-08T08:19:01.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.923+0000 7fa4beffd700 1 -- 172.21.15.40:0/2641997410 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8012430 con 0x7fa4c00704b0 2023-11-08T08:19:01.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.923+0000 7fa4c4bb4700 1 -- 172.21.15.40:0/2641997410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c00704b0 msgr2=0x7fa4c0164960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:01.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.923+0000 7fa4c4bb4700 1 --2- 172.21.15.40:0/2641997410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c00704b0 0x7fa4c0164960 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009d90 tx=0x7fa4b800b330 comp rx=0 tx=0).stop 2023-11-08T08:19:01.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.923+0000 7fa4c4bb4700 1 -- 172.21.15.40:0/2641997410 shutdown_connections 2023-11-08T08:19:01.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.923+0000 7fa4c4bb4700 1 --2- 172.21.15.40:0/2641997410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c00704b0 0x7fa4c0164960 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.923+0000 7fa4c4bb4700 1 -- 172.21.15.40:0/2641997410 >> 172.21.15.40:0/2641997410 conn(0x7fa4c00b0f80 msgr2=0x7fa4c00ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:01.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.923+0000 7fa4c4bb4700 1 -- 172.21.15.40:0/2641997410 shutdown_connections 2023-11-08T08:19:01.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.923+0000 7fa4c4bb4700 1 -- 172.21.15.40:0/2641997410 wait complete. 2023-11-08T08:19:01.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.927+0000 7fa4c4bb4700 1 Processor -- start 2023-11-08T08:19:01.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.927+0000 7fa4c4bb4700 1 -- start start 2023-11-08T08:19:01.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4c4bb4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c0070080 0x7fa4c01f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:01.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4c4bb4700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4c00704b0 0x7fa4c015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:01.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4c4bb4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4c0160710 0x7fa4c015f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:01.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4c4bb4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c01f9650 con 0x7fa4c0070080 2023-11-08T08:19:01.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4c4bb4700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c01f9930 con 0x7fa4c0160710 2023-11-08T08:19:01.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4c4bb4700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c01f9c10 con 0x7fa4c00704b0 2023-11-08T08:19:01.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4bf7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c0070080 0x7fa4c01f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:01.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4bf7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c0070080 0x7fa4c01f57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56148/0 (socket says 172.21.15.40:56148) 2023-11-08T08:19:01.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4bf7fe700 1 -- 172.21.15.40:0/825272872 learned_addr learned my addr 172.21.15.40:0/825272872 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:01.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4bffff700 1 --2- 172.21.15.40:0/825272872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4c0160710 0x7fa4c015f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:01.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4b6ffd700 1 --2- 172.21.15.40:0/825272872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4c00704b0 0x7fa4c015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:01.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4bf7fe700 1 -- 172.21.15.40:0/825272872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4c00704b0 msgr2=0x7fa4c015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:01.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4bf7fe700 1 --2- 172.21.15.40:0/825272872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4c00704b0 0x7fa4c015ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4bf7fe700 1 -- 172.21.15.40:0/825272872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4c0160710 msgr2=0x7fa4c015f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:01.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4bf7fe700 1 --2- 172.21.15.40:0/825272872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4c0160710 0x7fa4c015f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:01.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4bf7fe700 1 -- 172.21.15.40:0/825272872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b8009a70 con 0x7fa4c0070080 2023-11-08T08:19:01.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4b6ffd700 1 --2- 172.21.15.40:0/825272872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4c00704b0 0x7fa4c015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:01.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4bf7fe700 1 --2- 172.21.15.40:0/825272872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c0070080 0x7fa4c01f57a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00010e0 tx=0x7fa4b000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:01.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4bcff9700 1 -- 172.21.15.40:0/825272872 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b00052d0 con 0x7fa4c0070080 2023-11-08T08:19:01.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4bcff9700 1 -- 172.21.15.40:0/825272872 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b0005430 con 0x7fa4c0070080 2023-11-08T08:19:01.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4bcff9700 1 -- 172.21.15.40:0/825272872 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b00055a0 con 0x7fa4c0070080 2023-11-08T08:19:01.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4c4bb4700 1 -- 172.21.15.40:0/825272872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c006b6b0 con 0x7fa4c0070080 2023-11-08T08:19:01.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.931+0000 7fa4c4bb4700 1 -- 172.21.15.40:0/825272872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c006b3d0 con 0x7fa4c0070080 2023-11-08T08:19:01.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.935+0000 7fa4c4bb4700 1 -- 172.21.15.40:0/825272872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c0073cb0 con 0x7fa4c0070080 2023-11-08T08:19:01.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.935+0000 7fa4bcff9700 1 -- 172.21.15.40:0/825272872 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa4b0019030 con 0x7fa4c0070080 2023-11-08T08:19:01.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.935+0000 7fa4bcff9700 1 --2- 172.21.15.40:0/825272872 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa4a0064ee0 0x7fa4a0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:01.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.935+0000 7fa4bcff9700 1 -- 172.21.15.40:0/825272872 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa4b0080f50 con 0x7fa4c0070080 2023-11-08T08:19:01.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.935+0000 7fa4b6ffd700 1 --2- 172.21.15.40:0/825272872 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa4a0064ee0 0x7fa4a0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:01.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.939+0000 7fa4b6ffd700 1 --2- 172.21.15.40:0/825272872 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa4a0064ee0 0x7fa4a0067390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa4c01f9410 tx=0x7fa4a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:01.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:01.943+0000 7fa4bcff9700 1 -- 172.21.15.40:0/825272872 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa4b004f350 con 0x7fa4c0070080 2023-11-08T08:19:02.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.119+0000 7fa4c4bb4700 1 -- 172.21.15.40:0/825272872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa4c015fec0 con 0x7fa4c0070080 2023-11-08T08:19:02.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.119+0000 7fa4bcff9700 1 -- 172.21.15.40:0/825272872 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v20) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa4b0052970 con 0x7fa4c0070080 2023-11-08T08:19:02.125 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2023-11-08T08:19:02.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:19:02.126 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2023-11-08T08:19:02.126 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2023-11-08T08:19:02.127 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:19:02.127 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:19:02.127 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 2, 2023-11-08T08:19:02.127 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": "rbd" 2023-11-08T08:19:02.128 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:19:02.128 INFO:teuthology.orchestra.run.smithi040.stdout:] 2023-11-08T08:19:02.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.123+0000 7fa4c4bb4700 1 -- 172.21.15.40:0/825272872 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa4a0064ee0 msgr2=0x7fa4a0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:02.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.123+0000 7fa4c4bb4700 1 --2- 172.21.15.40:0/825272872 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa4a0064ee0 0x7fa4a0067390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa4c01f9410 tx=0x7fa4a8009040 comp rx=0 tx=0).stop 2023-11-08T08:19:02.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.123+0000 7fa4c4bb4700 1 -- 172.21.15.40:0/825272872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c0070080 msgr2=0x7fa4c01f57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:02.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.123+0000 7fa4c4bb4700 1 --2- 172.21.15.40:0/825272872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c0070080 0x7fa4c01f57a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00010e0 tx=0x7fa4b000c7b0 comp rx=0 tx=0).stop 2023-11-08T08:19:02.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.123+0000 7fa4c4bb4700 1 -- 172.21.15.40:0/825272872 shutdown_connections 2023-11-08T08:19:02.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.123+0000 7fa4c4bb4700 1 --2- 172.21.15.40:0/825272872 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa4a0064ee0 0x7fa4a0067390 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:02.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.123+0000 7fa4c4bb4700 1 --2- 172.21.15.40:0/825272872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4c0160710 0x7fa4c015f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:02.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.123+0000 7fa4c4bb4700 1 --2- 172.21.15.40:0/825272872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4c00704b0 0x7fa4c015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:02.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.123+0000 7fa4c4bb4700 1 --2- 172.21.15.40:0/825272872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c0070080 0x7fa4c01f57a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:02.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.123+0000 7fa4c4bb4700 1 -- 172.21.15.40:0/825272872 >> 172.21.15.40:0/825272872 conn(0x7fa4c00b0f80 msgr2=0x7fa4c0069bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:02.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.123+0000 7fa4c4bb4700 1 -- 172.21.15.40:0/825272872 shutdown_connections 2023-11-08T08:19:02.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.123+0000 7fa4c4bb4700 1 -- 172.21.15.40:0/825272872 wait complete. 2023-11-08T08:19:02.154 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T08:19:02.157 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T08:19:02.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.351+0000 7fd2754a7700 1 Processor -- start 2023-11-08T08:19:02.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.351+0000 7fd2754a7700 1 -- start start 2023-11-08T08:19:02.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.351+0000 7fd2754a7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd27015c310 0x7fd270069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:02.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.351+0000 7fd2754a7700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd27006ad80 0x7fd27006c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:02.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.351+0000 7fd2754a7700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2700cd2f0 con 0x7fd27006a1e0 2023-11-08T08:19:02.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.351+0000 7fd2754a7700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2700cd5d0 con 0x7fd27015c310 2023-11-08T08:19:02.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.351+0000 7fd2754a7700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2700cd8b0 con 0x7fd27006ad80 2023-11-08T08:19:02.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.355+0000 7fd26f7fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd27006ad80 0x7fd27006c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:02.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.355+0000 7fd26f7fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd27006ad80 0x7fd27006c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:37170/0 (socket says 172.21.15.40:37170) 2023-11-08T08:19:02.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.355+0000 7fd26f7fe700 1 -- 172.21.15.40:0/1137381339 learned_addr learned my addr 172.21.15.40:0/1137381339 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:02.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.355+0000 7fd26effd700 1 --2- 172.21.15.40:0/1137381339 >> v2:172.21.15.40:3300/0 conn(0x7fd27015c310 0x7fd270069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:02.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.355+0000 7fd26f7fe700 1 -- 172.21.15.40:0/1137381339 >> v2:172.21.15.40:3300/0 conn(0x7fd27015c310 msgr2=0x7fd270069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:02.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.355+0000 7fd26f7fe700 1 --2- 172.21.15.40:0/1137381339 >> v2:172.21.15.40:3300/0 conn(0x7fd27015c310 0x7fd270069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:02.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.355+0000 7fd26f7fe700 1 -- 172.21.15.40:0/1137381339 >> v1:172.21.15.40:6789/0 conn(0x7fd27006a1e0 legacy=0x7fd27006a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:02.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.355+0000 7fd26dffb700 1 -- 172.21.15.40:0/1137381339 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1684184206 0 0) 0x7fd2700cd2f0 con 0x7fd27006a1e0 2023-11-08T08:19:02.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.355+0000 7fd26f7fe700 1 -- 172.21.15.40:0/1137381339 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27006bb10 con 0x7fd27006ad80 2023-11-08T08:19:02.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.355+0000 7fd26f7fe700 1 --2- 172.21.15.40:0/1137381339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd27006ad80 0x7fd27006c310 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd268009e50 tx=0x7fd26800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb3aa94e839bdf9d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:02.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.355+0000 7fd26dffb700 1 -- 172.21.15.40:0/1137381339 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26800e070 con 0x7fd27006ad80 2023-11-08T08:19:02.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.355+0000 7fd26dffb700 1 -- 172.21.15.40:0/1137381339 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26800b8d0 con 0x7fd27006ad80 2023-11-08T08:19:02.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.355+0000 7fd26dffb700 1 -- 172.21.15.40:0/1137381339 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26800bc80 con 0x7fd27006ad80 2023-11-08T08:19:02.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.355+0000 7fd2754a7700 1 -- 172.21.15.40:0/1137381339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd27006ad80 msgr2=0x7fd27006c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:02.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.355+0000 7fd2754a7700 1 --2- 172.21.15.40:0/1137381339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd27006ad80 0x7fd27006c310 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd268009e50 tx=0x7fd26800b330 comp rx=0 tx=0).stop 2023-11-08T08:19:02.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.355+0000 7fd2754a7700 1 -- 172.21.15.40:0/1137381339 shutdown_connections 2023-11-08T08:19:02.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.355+0000 7fd2754a7700 1 --2- 172.21.15.40:0/1137381339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd27006ad80 0x7fd27006c310 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.355+0000 7fd2754a7700 1 --2- 172.21.15.40:0/1137381339 >> v2:172.21.15.40:3300/0 conn(0x7fd27015c310 0x7fd270069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.355+0000 7fd2754a7700 1 -- 172.21.15.40:0/1137381339 >> 172.21.15.40:0/1137381339 conn(0x7fd2700ce5b0 msgr2=0x7fd2700ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.359+0000 7fd2754a7700 1 -- 172.21.15.40:0/1137381339 shutdown_connections 2023-11-08T08:19:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.359+0000 7fd2754a7700 1 -- 172.21.15.40:0/1137381339 wait complete. 2023-11-08T08:19:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.359+0000 7fd2754a7700 1 Processor -- start 2023-11-08T08:19:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.359+0000 7fd2754a7700 1 -- start start 2023-11-08T08:19:02.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd2754a7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd27006a1e0 0x7fd27015f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:02.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd2754a7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd27006ad80 0x7fd2701f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:02.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd2754a7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd27015c310 0x7fd2700b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:02.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd2754a7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2700c6110 con 0x7fd27015c310 2023-11-08T08:19:02.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd2754a7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27015f850 con 0x7fd27006ad80 2023-11-08T08:19:02.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd2754a7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2700c57b0 con 0x7fd27006a1e0 2023-11-08T08:19:02.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd26f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd27015c310 0x7fd2700b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:02.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd26f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd27015c310 0x7fd2700b0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56166/0 (socket says 172.21.15.40:56166) 2023-11-08T08:19:02.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd26f7fe700 1 -- 172.21.15.40:0/2294111235 learned_addr learned my addr 172.21.15.40:0/2294111235 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:02.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd26effd700 1 --2- 172.21.15.40:0/2294111235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd27006a1e0 0x7fd27015f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd26e7fc700 1 --2- 172.21.15.40:0/2294111235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd27006ad80 0x7fd2701f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd26f7fe700 1 -- 172.21.15.40:0/2294111235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd27006a1e0 msgr2=0x7fd27015f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd26f7fe700 1 --2- 172.21.15.40:0/2294111235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd27006a1e0 0x7fd27015f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd26f7fe700 1 -- 172.21.15.40:0/2294111235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd27006ad80 msgr2=0x7fd2701f54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd26f7fe700 1 --2- 172.21.15.40:0/2294111235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd27006ad80 0x7fd2701f54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:02.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd26f7fe700 1 -- 172.21.15.40:0/2294111235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd268009b30 con 0x7fd27015c310 2023-11-08T08:19:02.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd26f7fe700 1 --2- 172.21.15.40:0/2294111235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd27015c310 0x7fd2700b0080 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd268012000 tx=0x7fd268004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:02.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd2577fe700 1 -- 172.21.15.40:0/2294111235 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd268004880 con 0x7fd27015c310 2023-11-08T08:19:02.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd2577fe700 1 -- 172.21.15.40:0/2294111235 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2680099b0 con 0x7fd27015c310 2023-11-08T08:19:02.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd2754a7700 1 -- 172.21.15.40:0/2294111235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27015c6e0 con 0x7fd27015c310 2023-11-08T08:19:02.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd2577fe700 1 -- 172.21.15.40:0/2294111235 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26801c640 con 0x7fd27015c310 2023-11-08T08:19:02.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.363+0000 7fd2754a7700 1 -- 172.21.15.40:0/2294111235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27006bdf0 con 0x7fd27015c310 2023-11-08T08:19:02.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.367+0000 7fd2754a7700 1 -- 172.21.15.40:0/2294111235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2700ccf50 con 0x7fd27015c310 2023-11-08T08:19:02.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.367+0000 7fd2577fe700 1 -- 172.21.15.40:0/2294111235 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd268022030 con 0x7fd27015c310 2023-11-08T08:19:02.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.371+0000 7fd2577fe700 1 --2- 172.21.15.40:0/2294111235 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd25c064ee0 0x7fd25c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:02.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.371+0000 7fd2577fe700 1 -- 172.21.15.40:0/2294111235 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd268083300 con 0x7fd27015c310 2023-11-08T08:19:02.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.371+0000 7fd26effd700 1 --2- 172.21.15.40:0/2294111235 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd25c064ee0 0x7fd25c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:02.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.371+0000 7fd26effd700 1 --2- 172.21.15.40:0/2294111235 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd25c064ee0 0x7fd25c067390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd260009b40 tx=0x7fd2600091c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:02.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.375+0000 7fd2577fe700 1 -- 172.21.15.40:0/2294111235 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd268051700 con 0x7fd27015c310 2023-11-08T08:19:02.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.551+0000 7fd2754a7700 1 -- 172.21.15.40:0/2294111235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd2700c6250 con 0x7fd27015c310 2023-11-08T08:19:02.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.967+0000 7fd2577fe700 1 -- 172.21.15.40:0/2294111235 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v21) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd2700c6250 con 0x7fd27015c310 2023-11-08T08:19:02.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.971+0000 7fd2754a7700 1 -- 172.21.15.40:0/2294111235 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd25c064ee0 msgr2=0x7fd25c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:02.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.971+0000 7fd2754a7700 1 --2- 172.21.15.40:0/2294111235 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd25c064ee0 0x7fd25c067390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd260009b40 tx=0x7fd2600091c0 comp rx=0 tx=0).stop 2023-11-08T08:19:02.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.971+0000 7fd2754a7700 1 -- 172.21.15.40:0/2294111235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd27015c310 msgr2=0x7fd2700b0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:02.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.971+0000 7fd2754a7700 1 --2- 172.21.15.40:0/2294111235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd27015c310 0x7fd2700b0080 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd268012000 tx=0x7fd268004af0 comp rx=0 tx=0).stop 2023-11-08T08:19:02.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.975+0000 7fd2754a7700 1 -- 172.21.15.40:0/2294111235 shutdown_connections 2023-11-08T08:19:02.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.975+0000 7fd2754a7700 1 --2- 172.21.15.40:0/2294111235 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd25c064ee0 0x7fd25c067390 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:02.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.975+0000 7fd2754a7700 1 --2- 172.21.15.40:0/2294111235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd27015c310 0x7fd2700b0080 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:02.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.975+0000 7fd2754a7700 1 --2- 172.21.15.40:0/2294111235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd27006ad80 0x7fd2701f54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:02.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.975+0000 7fd2754a7700 1 --2- 172.21.15.40:0/2294111235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd27006a1e0 0x7fd27015f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:02.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.975+0000 7fd2754a7700 1 -- 172.21.15.40:0/2294111235 >> 172.21.15.40:0/2294111235 conn(0x7fd2700ce5b0 msgr2=0x7fd27015b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:02.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.975+0000 7fd2754a7700 1 -- 172.21.15.40:0/2294111235 shutdown_connections 2023-11-08T08:19:02.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:02.975+0000 7fd2754a7700 1 -- 172.21.15.40:0/2294111235 wait complete. 2023-11-08T08:19:02.983 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2023-11-08T08:19:03.004 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-11-08T08:19:03.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.203+0000 7f7295416700 1 Processor -- start 2023-11-08T08:19:03.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.203+0000 7f7295416700 1 -- start start 2023-11-08T08:19:03.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.207+0000 7f7295416700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7290161030 0x7f729006cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:03.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.207+0000 7f7295416700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f72900cddf0 0x7f72900ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:03.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.207+0000 7f7295416700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72900ce730 con 0x7f729015c7c0 2023-11-08T08:19:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.207+0000 7f7295416700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72900cdba0 con 0x7f7290161030 2023-11-08T08:19:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.207+0000 7f7295416700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72900cd8d0 con 0x7f72900cddf0 2023-11-08T08:19:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.207+0000 7f728e7fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7290161030 0x7f729006cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.207+0000 7f728e7fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7290161030 0x7f729006cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56174/0 (socket says 172.21.15.40:56174) 2023-11-08T08:19:03.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.207+0000 7f728effd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f729015c7c0 0x7f72900cd1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42952/0 (socket says 172.21.15.40:42952) 2023-11-08T08:19:03.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.207+0000 7f728e7fc700 1 -- 172.21.15.40:0/3830268182 learned_addr learned my addr 172.21.15.40:0/3830268182 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:03.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.207+0000 7f728f7fe700 1 --2- 172.21.15.40:0/3830268182 >> v2:172.21.15.169:3300/0 conn(0x7f72900cddf0 0x7f72900ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:03.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.207+0000 7f728e7fc700 1 -- 172.21.15.40:0/3830268182 >> v2:172.21.15.169:3300/0 conn(0x7f72900cddf0 msgr2=0x7f72900ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:03.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.207+0000 7f728e7fc700 1 --2- 172.21.15.40:0/3830268182 >> v2:172.21.15.169:3300/0 conn(0x7f72900cddf0 0x7f72900ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:03.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.207+0000 7f728e7fc700 1 -- 172.21.15.40:0/3830268182 >> v1:172.21.15.40:6789/0 conn(0x7f729015c7c0 legacy=0x7f72900cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:03.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.207+0000 7f728e7fc700 1 -- 172.21.15.40:0/3830268182 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729006c010 con 0x7f7290161030 2023-11-08T08:19:03.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.207+0000 7f728f7fe700 1 --2- 172.21.15.40:0/3830268182 >> v2:172.21.15.169:3300/0 conn(0x7f72900cddf0 0x7f72900ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:03.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.207+0000 7f728e7fc700 1 --2- 172.21.15.40:0/3830268182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7290161030 0x7f729006cdd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f72900b2c40 tx=0x7f7284009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3aff91add5a4cd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:03.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.207+0000 7f728dffb700 1 -- 172.21.15.40:0/3830268182 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728400b8a0 con 0x7f7290161030 2023-11-08T08:19:03.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.207+0000 7f728dffb700 1 -- 172.21.15.40:0/3830268182 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728400ba00 con 0x7f7290161030 2023-11-08T08:19:03.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.207+0000 7f728dffb700 1 -- 172.21.15.40:0/3830268182 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728400bd90 con 0x7f7290161030 2023-11-08T08:19:03.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.211+0000 7f7295416700 1 -- 172.21.15.40:0/3830268182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7290161030 msgr2=0x7f729006cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:03.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.211+0000 7f7295416700 1 --2- 172.21.15.40:0/3830268182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7290161030 0x7f729006cdd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f72900b2c40 tx=0x7f7284009bf0 comp rx=0 tx=0).stop 2023-11-08T08:19:03.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.211+0000 7f7295416700 1 -- 172.21.15.40:0/3830268182 shutdown_connections 2023-11-08T08:19:03.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.211+0000 7f7295416700 1 --2- 172.21.15.40:0/3830268182 >> v2:172.21.15.169:3300/0 conn(0x7f72900cddf0 0x7f72900ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:03.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.211+0000 7f7295416700 1 --2- 172.21.15.40:0/3830268182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7290161030 0x7f729006cdd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:03.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.211+0000 7f7295416700 1 -- 172.21.15.40:0/3830268182 >> 172.21.15.40:0/3830268182 conn(0x7f72900d3800 msgr2=0x7f72900cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:03.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.211+0000 7f7295416700 1 -- 172.21.15.40:0/3830268182 shutdown_connections 2023-11-08T08:19:03.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.211+0000 7f7295416700 1 -- 172.21.15.40:0/3830268182 wait complete. 2023-11-08T08:19:03.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.215+0000 7f7295416700 1 Processor -- start 2023-11-08T08:19:03.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.215+0000 7f7295416700 1 -- start start 2023-11-08T08:19:03.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.215+0000 7f7295416700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72900cddf0 0x7f729014cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:03.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.215+0000 7f7295416700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f729015c7c0 0x7f72900c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:03.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.215+0000 7f7295416700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72900b4cc0 0x7f72900cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:03.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.215+0000 7f7295416700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72900b74c0 con 0x7f72900cddf0 2023-11-08T08:19:03.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.215+0000 7f7295416700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72900c69c0 con 0x7f72900b4cc0 2023-11-08T08:19:03.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.215+0000 7f7295416700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f729014d4f0 con 0x7f729015c7c0 2023-11-08T08:19:03.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.215+0000 7f728effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72900cddf0 0x7f729014cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:03.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.215+0000 7f728effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72900cddf0 0x7f729014cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56178/0 (socket says 172.21.15.40:56178) 2023-11-08T08:19:03.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.215+0000 7f728effd700 1 -- 172.21.15.40:0/3573002655 learned_addr learned my addr 172.21.15.40:0/3573002655 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:03.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.215+0000 7f728e7fc700 1 --2- 172.21.15.40:0/3573002655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f729015c7c0 0x7f72900c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:03.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.215+0000 7f728f7fe700 1 --2- 172.21.15.40:0/3573002655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72900b4cc0 0x7f72900cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:03.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.215+0000 7f728effd700 1 -- 172.21.15.40:0/3573002655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f729015c7c0 msgr2=0x7f72900c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:03.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.215+0000 7f728effd700 1 --2- 172.21.15.40:0/3573002655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f729015c7c0 0x7f72900c7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:03.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.215+0000 7f728effd700 1 -- 172.21.15.40:0/3573002655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72900b4cc0 msgr2=0x7f72900cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:03.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.215+0000 7f728effd700 1 --2- 172.21.15.40:0/3573002655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72900b4cc0 0x7f72900cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:03.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.215+0000 7f728effd700 1 -- 172.21.15.40:0/3573002655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7284009570 con 0x7f72900cddf0 2023-11-08T08:19:03.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.219+0000 7f728effd700 1 --2- 172.21.15.40:0/3573002655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72900cddf0 0x7f729014cfb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f728000c500 tx=0x7f728000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.219+0000 7f72777fe700 1 -- 172.21.15.40:0/3573002655 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7280005cd0 con 0x7f72900cddf0 2023-11-08T08:19:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.219+0000 7f72777fe700 1 -- 172.21.15.40:0/3573002655 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7280005e30 con 0x7f72900cddf0 2023-11-08T08:19:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.219+0000 7f72777fe700 1 -- 172.21.15.40:0/3573002655 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72800072f0 con 0x7f72900cddf0 2023-11-08T08:19:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.219+0000 7f7295416700 1 -- 172.21.15.40:0/3573002655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729006c5d0 con 0x7f72900cddf0 2023-11-08T08:19:03.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.219+0000 7f7295416700 1 -- 172.21.15.40:0/3573002655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729006c2f0 con 0x7f72900cddf0 2023-11-08T08:19:03.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.219+0000 7f7295416700 1 -- 172.21.15.40:0/3573002655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729006c8b0 con 0x7f72900cddf0 2023-11-08T08:19:03.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.223+0000 7f72777fe700 1 -- 172.21.15.40:0/3573002655 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7280019030 con 0x7f72900cddf0 2023-11-08T08:19:03.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.223+0000 7f72777fe700 1 --2- 172.21.15.40:0/3573002655 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f727c064ee0 0x7f727c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:03.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.223+0000 7f72777fe700 1 -- 172.21.15.40:0/3573002655 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f7280081140 con 0x7f72900cddf0 2023-11-08T08:19:03.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.223+0000 7f728e7fc700 1 --2- 172.21.15.40:0/3573002655 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f727c064ee0 0x7f727c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:03.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.227+0000 7f728e7fc700 1 --2- 172.21.15.40:0/3573002655 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f727c064ee0 0x7f727c067390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7284000c00 tx=0x7f7284008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:03.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.231+0000 7f72777fe700 1 -- 172.21.15.40:0/3573002655 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f728004f3d0 con 0x7f72900cddf0 2023-11-08T08:19:03.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.403+0000 7f7295416700 1 -- 172.21.15.40:0/3573002655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f72900b7600 con 0x7f72900cddf0 2023-11-08T08:19:03.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.987+0000 7f72777fe700 1 -- 172.21.15.40:0/3573002655 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v22) v1 ==== 171+0+0 (secure 0 0 0) 0x7f72800170d0 con 0x7f72900cddf0 2023-11-08T08:19:04.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.995+0000 7f7295416700 1 -- 172.21.15.40:0/3573002655 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f727c064ee0 msgr2=0x7f727c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:04.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.995+0000 7f7295416700 1 --2- 172.21.15.40:0/3573002655 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f727c064ee0 0x7f727c067390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7284000c00 tx=0x7f7284008030 comp rx=0 tx=0).stop 2023-11-08T08:19:04.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.995+0000 7f7295416700 1 -- 172.21.15.40:0/3573002655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72900cddf0 msgr2=0x7f729014cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:04.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.995+0000 7f7295416700 1 --2- 172.21.15.40:0/3573002655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72900cddf0 0x7f729014cfb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f728000c500 tx=0x7f728000c8c0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.999+0000 7f7295416700 1 -- 172.21.15.40:0/3573002655 shutdown_connections 2023-11-08T08:19:04.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.999+0000 7f7295416700 1 --2- 172.21.15.40:0/3573002655 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f727c064ee0 0x7f727c067390 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.999+0000 7f7295416700 1 --2- 172.21.15.40:0/3573002655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72900b4cc0 0x7f72900cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.999+0000 7f7295416700 1 --2- 172.21.15.40:0/3573002655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f729015c7c0 0x7f72900c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.999+0000 7f7295416700 1 --2- 172.21.15.40:0/3573002655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72900cddf0 0x7f729014cfb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.999+0000 7f7295416700 1 -- 172.21.15.40:0/3573002655 >> 172.21.15.40:0/3573002655 conn(0x7f72900d3800 msgr2=0x7f7290060660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:04.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.999+0000 7f7295416700 1 -- 172.21.15.40:0/3573002655 shutdown_connections 2023-11-08T08:19:04.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:03.999+0000 7f7295416700 1 -- 172.21.15.40:0/3573002655 wait complete. 2023-11-08T08:19:04.009 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2023-11-08T08:19:04.033 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T08:19:04.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.211+0000 7fed25d77700 1 Processor -- start 2023-11-08T08:19:04.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.211+0000 7fed25d77700 1 -- start start 2023-11-08T08:19:04.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.215+0000 7fed25d77700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fed2015e190 0x7fed200cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:04.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.215+0000 7fed25d77700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed2006ff10 0x7fed20070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:04.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.215+0000 7fed25d77700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed200708b0 con 0x7fed20161560 2023-11-08T08:19:04.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.215+0000 7fed25d77700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed2015e560 con 0x7fed2006ff10 2023-11-08T08:19:04.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.215+0000 7fed25d77700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed2006be80 con 0x7fed2015e190 2023-11-08T08:19:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.215+0000 7fed25576700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed2006ff10 0x7fed20070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.215+0000 7fed25576700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed2006ff10 0x7fed20070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56184/0 (socket says 172.21.15.40:56184) 2023-11-08T08:19:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.215+0000 7fed25576700 1 -- 172.21.15.40:0/2793651191 learned_addr learned my addr 172.21.15.40:0/2793651191 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.215+0000 7fed24d75700 1 --2- 172.21.15.40:0/2793651191 >> v2:172.21.15.169:3300/0 conn(0x7fed2015e190 0x7fed200cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.215+0000 7fed1f7fe700 1 -- 172.21.15.40:0/2793651191 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675285508 0 0) 0x7fed200708b0 con 0x7fed20161560 2023-11-08T08:19:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.215+0000 7fed1f7fe700 1 -- 172.21.15.40:0/2793651191 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed100035d0 con 0x7fed20161560 2023-11-08T08:19:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.215+0000 7fed25576700 1 -- 172.21.15.40:0/2793651191 >> v2:172.21.15.169:3300/0 conn(0x7fed2015e190 msgr2=0x7fed200cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.215+0000 7fed25576700 1 --2- 172.21.15.40:0/2793651191 >> v2:172.21.15.169:3300/0 conn(0x7fed2015e190 0x7fed200cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.215+0000 7fed25576700 1 -- 172.21.15.40:0/2793651191 >> v1:172.21.15.169:6789/0 conn(0x7fed20161560 legacy=0x7fed20069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.215+0000 7fed25576700 1 -- 172.21.15.40:0/2793651191 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed200ce330 con 0x7fed2006ff10 2023-11-08T08:19:04.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.215+0000 7fed25576700 1 --2- 172.21.15.40:0/2793651191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed2006ff10 0x7fed20070370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fed200cb9b0 tx=0x7fed1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef54afa2045e7275 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:04.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.215+0000 7fed1f7fe700 1 -- 172.21.15.40:0/2793651191 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1800e070 con 0x7fed2006ff10 2023-11-08T08:19:04.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.215+0000 7fed1f7fe700 1 -- 172.21.15.40:0/2793651191 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1800b910 con 0x7fed2006ff10 2023-11-08T08:19:04.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.215+0000 7fed1f7fe700 1 -- 172.21.15.40:0/2793651191 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1800bc80 con 0x7fed2006ff10 2023-11-08T08:19:04.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.219+0000 7fed25d77700 1 -- 172.21.15.40:0/2793651191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed2006ff10 msgr2=0x7fed20070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:04.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.219+0000 7fed25d77700 1 --2- 172.21.15.40:0/2793651191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed2006ff10 0x7fed20070370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fed200cb9b0 tx=0x7fed1800b330 comp rx=0 tx=0).stop 2023-11-08T08:19:04.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.219+0000 7fed25d77700 1 -- 172.21.15.40:0/2793651191 shutdown_connections 2023-11-08T08:19:04.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.219+0000 7fed25d77700 1 --2- 172.21.15.40:0/2793651191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed2006ff10 0x7fed20070370 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.219+0000 7fed25d77700 1 --2- 172.21.15.40:0/2793651191 >> v2:172.21.15.169:3300/0 conn(0x7fed2015e190 0x7fed200cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.219+0000 7fed25d77700 1 -- 172.21.15.40:0/2793651191 >> 172.21.15.40:0/2793651191 conn(0x7fed200cf280 msgr2=0x7fed200cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:04.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.219+0000 7fed25d77700 1 -- 172.21.15.40:0/2793651191 shutdown_connections 2023-11-08T08:19:04.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.219+0000 7fed25d77700 1 -- 172.21.15.40:0/2793651191 wait complete. 2023-11-08T08:19:04.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.223+0000 7fed25d77700 1 Processor -- start 2023-11-08T08:19:04.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.223+0000 7fed25d77700 1 -- start start 2023-11-08T08:19:04.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.223+0000 7fed25d77700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed2006ff10 0x7fed2013b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:04.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.223+0000 7fed25d77700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed2015e190 0x7fed201623c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:04.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.223+0000 7fed25d77700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed20161560 0x7fed2014d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:04.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.223+0000 7fed25d77700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed20139350 con 0x7fed20161560 2023-11-08T08:19:04.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.223+0000 7fed25d77700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2013b920 con 0x7fed2006ff10 2023-11-08T08:19:04.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.223+0000 7fed25d77700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2013bc00 con 0x7fed2015e190 2023-11-08T08:19:04.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.223+0000 7fed25576700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed20161560 0x7fed2014d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:04.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.223+0000 7fed25576700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed20161560 0x7fed2014d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56190/0 (socket says 172.21.15.40:56190) 2023-11-08T08:19:04.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.223+0000 7fed25576700 1 -- 172.21.15.40:0/3790056678 learned_addr learned my addr 172.21.15.40:0/3790056678 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:04.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.223+0000 7fed1ffff700 1 --2- 172.21.15.40:0/3790056678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed2015e190 0x7fed201623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:04.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.223+0000 7fed24d75700 1 --2- 172.21.15.40:0/3790056678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed2006ff10 0x7fed2013b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:04.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.227+0000 7fed25576700 1 -- 172.21.15.40:0/3790056678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed2015e190 msgr2=0x7fed201623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:04.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.227+0000 7fed25576700 1 --2- 172.21.15.40:0/3790056678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed2015e190 0x7fed201623c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.227+0000 7fed25576700 1 -- 172.21.15.40:0/3790056678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed2006ff10 msgr2=0x7fed2013b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:04.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.227+0000 7fed25576700 1 --2- 172.21.15.40:0/3790056678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed2006ff10 0x7fed2013b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.227+0000 7fed25576700 1 -- 172.21.15.40:0/3790056678 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed18009b30 con 0x7fed20161560 2023-11-08T08:19:04.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.227+0000 7fed25576700 1 --2- 172.21.15.40:0/3790056678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed20161560 0x7fed2014d0e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fed200717e0 tx=0x7fed18004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:04.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.227+0000 7fed1d7fa700 1 -- 172.21.15.40:0/3790056678 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18004880 con 0x7fed20161560 2023-11-08T08:19:04.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.227+0000 7fed1d7fa700 1 -- 172.21.15.40:0/3790056678 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed180099b0 con 0x7fed20161560 2023-11-08T08:19:04.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.227+0000 7fed25d77700 1 -- 172.21.15.40:0/3790056678 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed200afc70 con 0x7fed20161560 2023-11-08T08:19:04.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.227+0000 7fed1d7fa700 1 -- 172.21.15.40:0/3790056678 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1801b670 con 0x7fed20161560 2023-11-08T08:19:04.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.227+0000 7fed25d77700 1 -- 172.21.15.40:0/3790056678 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed200af990 con 0x7fed20161560 2023-11-08T08:19:04.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.227+0000 7fed25d77700 1 -- 172.21.15.40:0/3790056678 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed200aff50 con 0x7fed20161560 2023-11-08T08:19:04.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.231+0000 7fed1d7fa700 1 -- 172.21.15.40:0/3790056678 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fed18021030 con 0x7fed20161560 2023-11-08T08:19:04.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.231+0000 7fed1d7fa700 1 --2- 172.21.15.40:0/3790056678 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fed04064f60 0x7fed04067410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:04.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.231+0000 7fed1d7fa700 1 -- 172.21.15.40:0/3790056678 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fed180831b0 con 0x7fed20161560 2023-11-08T08:19:04.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.235+0000 7fed24d75700 1 --2- 172.21.15.40:0/3790056678 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fed04064f60 0x7fed04067410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:04.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.235+0000 7fed24d75700 1 --2- 172.21.15.40:0/3790056678 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fed04064f60 0x7fed04067410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fed2014e4d0 tx=0x7fed14009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:04.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.239+0000 7fed1d7fa700 1 -- 172.21.15.40:0/3790056678 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fed18051340 con 0x7fed20161560 2023-11-08T08:19:04.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.447+0000 7fed25d77700 1 -- 172.21.15.40:0/3790056678 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fed20163c10 con 0x7fed20161560 2023-11-08T08:19:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.463+0000 7fed1d7fa700 1 -- 172.21.15.40:0/3790056678 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fed18054960 con 0x7fed20161560 2023-11-08T08:19:04.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.471+0000 7fed25d77700 1 -- 172.21.15.40:0/3790056678 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fed04064f60 msgr2=0x7fed04067410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:04.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.471+0000 7fed25d77700 1 --2- 172.21.15.40:0/3790056678 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fed04064f60 0x7fed04067410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fed2014e4d0 tx=0x7fed14009240 comp rx=0 tx=0).stop 2023-11-08T08:19:04.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.471+0000 7fed25d77700 1 -- 172.21.15.40:0/3790056678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed20161560 msgr2=0x7fed2014d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:04.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.471+0000 7fed25d77700 1 --2- 172.21.15.40:0/3790056678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed20161560 0x7fed2014d0e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fed200717e0 tx=0x7fed18004af0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.471+0000 7fed25d77700 1 -- 172.21.15.40:0/3790056678 shutdown_connections 2023-11-08T08:19:04.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.471+0000 7fed25d77700 1 --2- 172.21.15.40:0/3790056678 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fed04064f60 0x7fed04067410 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.471+0000 7fed25d77700 1 --2- 172.21.15.40:0/3790056678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed20161560 0x7fed2014d0e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.471+0000 7fed25d77700 1 --2- 172.21.15.40:0/3790056678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed2015e190 0x7fed201623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.471+0000 7fed25d77700 1 --2- 172.21.15.40:0/3790056678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed2006ff10 0x7fed2013b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.471+0000 7fed25d77700 1 -- 172.21.15.40:0/3790056678 >> 172.21.15.40:0/3790056678 conn(0x7fed200cf280 msgr2=0x7fed200702e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:04.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.471+0000 7fed25d77700 1 -- 172.21.15.40:0/3790056678 shutdown_connections 2023-11-08T08:19:04.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.475+0000 7fed25d77700 1 -- 172.21.15.40:0/3790056678 wait complete. 2023-11-08T08:19:04.489 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 3 and data pool 4 2023-11-08T08:19:04.522 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-11-08T08:19:04.523 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-11-08T08:19:04.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.695+0000 7f03ca6b5700 1 Processor -- start 2023-11-08T08:19:04.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.699+0000 7f03ca6b5700 1 -- start start 2023-11-08T08:19:04.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.699+0000 7f03ca6b5700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f03c4142650 0x7f03c4142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:04.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.699+0000 7f03ca6b5700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f03c40b5a00 0x7f03c40b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:04.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.699+0000 7f03ca6b5700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c40b63a0 con 0x7f03c406aab0 2023-11-08T08:19:04.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.699+0000 7f03ca6b5700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03c406b5d0 con 0x7f03c40b5a00 2023-11-08T08:19:04.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.699+0000 7f03ca6b5700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03c406b8d0 con 0x7f03c4142650 2023-11-08T08:19:04.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.699+0000 7f03c37fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f03c40b5a00 0x7f03c40b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:04.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.699+0000 7f03c37fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f03c40b5a00 0x7f03c40b5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56198/0 (socket says 172.21.15.40:56198) 2023-11-08T08:19:04.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.699+0000 7f03c37fe700 1 -- 172.21.15.40:0/3153800122 learned_addr learned my addr 172.21.15.40:0/3153800122 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:04.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.699+0000 7f03c3fff700 1 --2- 172.21.15.40:0/3153800122 >> v2:172.21.15.169:3300/0 conn(0x7f03c4142650 0x7f03c4142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:04.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.699+0000 7f03c37fe700 1 -- 172.21.15.40:0/3153800122 >> v2:172.21.15.169:3300/0 conn(0x7f03c4142650 msgr2=0x7f03c4142fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:04.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.699+0000 7f03c37fe700 1 --2- 172.21.15.40:0/3153800122 >> v2:172.21.15.169:3300/0 conn(0x7f03c4142650 0x7f03c4142fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.699+0000 7f03c37fe700 1 -- 172.21.15.40:0/3153800122 >> v1:172.21.15.169:6789/0 conn(0x7f03c406aab0 legacy=0x7f03c406aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:04.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.699+0000 7f03c37fe700 1 -- 172.21.15.40:0/3153800122 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c406bb70 con 0x7f03c40b5a00 2023-11-08T08:19:04.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.699+0000 7f03c2ffd700 1 -- 172.21.15.40:0/3153800122 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 940107773 0 0) 0x7f03c40b63a0 con 0x7f03c406aab0 2023-11-08T08:19:04.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.703+0000 7f03c37fe700 1 --2- 172.21.15.40:0/3153800122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03c40b5a00 0x7f03c40b5e60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f03c406fe90 tx=0x7f03b8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d140975c0a233bb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:04.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.703+0000 7f03c2ffd700 1 -- 172.21.15.40:0/3153800122 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b8011830 con 0x7f03c40b5a00 2023-11-08T08:19:04.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.703+0000 7f03c2ffd700 1 -- 172.21.15.40:0/3153800122 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b8011990 con 0x7f03c40b5a00 2023-11-08T08:19:04.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.703+0000 7f03c2ffd700 1 -- 172.21.15.40:0/3153800122 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b8011d20 con 0x7f03c40b5a00 2023-11-08T08:19:04.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.703+0000 7f03ca6b5700 1 -- 172.21.15.40:0/3153800122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03c40b5a00 msgr2=0x7f03c40b5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:04.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.703+0000 7f03ca6b5700 1 --2- 172.21.15.40:0/3153800122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03c40b5a00 0x7f03c40b5e60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f03c406fe90 tx=0x7f03b8009bf0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.703+0000 7f03ca6b5700 1 -- 172.21.15.40:0/3153800122 shutdown_connections 2023-11-08T08:19:04.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.703+0000 7f03ca6b5700 1 --2- 172.21.15.40:0/3153800122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03c40b5a00 0x7f03c40b5e60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.703+0000 7f03ca6b5700 1 --2- 172.21.15.40:0/3153800122 >> v2:172.21.15.169:3300/0 conn(0x7f03c4142650 0x7f03c4142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.703+0000 7f03ca6b5700 1 -- 172.21.15.40:0/3153800122 >> 172.21.15.40:0/3153800122 conn(0x7f03c40b5390 msgr2=0x7f03c40d2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:04.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.703+0000 7f03ca6b5700 1 -- 172.21.15.40:0/3153800122 shutdown_connections 2023-11-08T08:19:04.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.703+0000 7f03ca6b5700 1 -- 172.21.15.40:0/3153800122 wait complete. 2023-11-08T08:19:04.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.707+0000 7f03ca6b5700 1 Processor -- start 2023-11-08T08:19:04.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.707+0000 7f03ca6b5700 1 -- start start 2023-11-08T08:19:04.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.707+0000 7f03ca6b5700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03c406aab0 0x7f03c4158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:04.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.707+0000 7f03ca6b5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03c40b5a00 0x7f03c413f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:04.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.707+0000 7f03ca6b5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03c4142650 0x7f03c41595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:04.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.707+0000 7f03ca6b5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c4140870 con 0x7f03c4142650 2023-11-08T08:19:04.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.707+0000 7f03ca6b5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c40d0560 con 0x7f03c40b5a00 2023-11-08T08:19:04.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.707+0000 7f03ca6b5700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c413fd00 con 0x7f03c406aab0 2023-11-08T08:19:04.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.711+0000 7f03c8c52700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03c4142650 0x7f03c41595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:04.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.711+0000 7f03c37fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03c40b5a00 0x7f03c413f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:04.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.711+0000 7f03c8c52700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03c4142650 0x7f03c41595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56210/0 (socket says 172.21.15.40:56210) 2023-11-08T08:19:04.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.711+0000 7f03c8c52700 1 -- 172.21.15.40:0/2647546578 learned_addr learned my addr 172.21.15.40:0/2647546578 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:04.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.711+0000 7f03c3fff700 1 --2- 172.21.15.40:0/2647546578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03c406aab0 0x7f03c4158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:04.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.711+0000 7f03c8c52700 1 -- 172.21.15.40:0/2647546578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03c406aab0 msgr2=0x7f03c4158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:04.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.711+0000 7f03c8c52700 1 --2- 172.21.15.40:0/2647546578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03c406aab0 0x7f03c4158850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.711+0000 7f03c8c52700 1 -- 172.21.15.40:0/2647546578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03c40b5a00 msgr2=0x7f03c413f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:04.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.711+0000 7f03c8c52700 1 --2- 172.21.15.40:0/2647546578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03c40b5a00 0x7f03c413f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:04.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.711+0000 7f03c8c52700 1 -- 172.21.15.40:0/2647546578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b8009570 con 0x7f03c4142650 2023-11-08T08:19:04.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.711+0000 7f03c8c52700 1 --2- 172.21.15.40:0/2647546578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03c4142650 0x7f03c41595b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00cc70 tx=0x7f03bc00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:04.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.711+0000 7f03c0ff9700 1 -- 172.21.15.40:0/2647546578 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc003e50 con 0x7f03c4142650 2023-11-08T08:19:04.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.711+0000 7f03c0ff9700 1 -- 172.21.15.40:0/2647546578 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03bc00a6f0 con 0x7f03c4142650 2023-11-08T08:19:04.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.711+0000 7f03ca6b5700 1 -- 172.21.15.40:0/2647546578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c406c130 con 0x7f03c4142650 2023-11-08T08:19:04.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.711+0000 7f03c0ff9700 1 -- 172.21.15.40:0/2647546578 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc00aaa0 con 0x7f03c4142650 2023-11-08T08:19:04.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.711+0000 7f03ca6b5700 1 -- 172.21.15.40:0/2647546578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c406be50 con 0x7f03c4142650 2023-11-08T08:19:04.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.715+0000 7f03ca6b5700 1 -- 172.21.15.40:0/2647546578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c406c410 con 0x7f03c4142650 2023-11-08T08:19:04.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.715+0000 7f03c0ff9700 1 -- 172.21.15.40:0/2647546578 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f03bc019030 con 0x7f03c4142650 2023-11-08T08:19:04.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.715+0000 7f03c0ff9700 1 --2- 172.21.15.40:0/2647546578 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f03b0064f10 0x7f03b00673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:04.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.715+0000 7f03c0ff9700 1 -- 172.21.15.40:0/2647546578 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f03bc081910 con 0x7f03c4142650 2023-11-08T08:19:04.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.715+0000 7f03c3fff700 1 --2- 172.21.15.40:0/2647546578 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f03b0064f10 0x7f03b00673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:04.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.719+0000 7f03c3fff700 1 --2- 172.21.15.40:0/2647546578 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f03b0064f10 0x7f03b00673c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f03c415f880 tx=0x7f03b4009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:04.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.723+0000 7f03c0ff9700 1 -- 172.21.15.40:0/2647546578 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f03bc04f940 con 0x7f03c4142650 2023-11-08T08:19:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:04.911+0000 7f03ca6b5700 1 -- 172.21.15.40:0/2647546578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f03c40d0330 con 0x7f03c4142650 2023-11-08T08:19:05.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.471+0000 7f03c0ff9700 1 -- 172.21.15.40:0/2647546578 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v24) v1 ==== 157+0+0 (secure 0 0 0) 0x7f03c40d0330 con 0x7f03c4142650 2023-11-08T08:19:05.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.475+0000 7f03ca6b5700 1 -- 172.21.15.40:0/2647546578 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f03b0064f10 msgr2=0x7f03b00673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:05.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.475+0000 7f03ca6b5700 1 --2- 172.21.15.40:0/2647546578 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f03b0064f10 0x7f03b00673c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f03c415f880 tx=0x7f03b4009240 comp rx=0 tx=0).stop 2023-11-08T08:19:05.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.475+0000 7f03ca6b5700 1 -- 172.21.15.40:0/2647546578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03c4142650 msgr2=0x7f03c41595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:05.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.475+0000 7f03ca6b5700 1 --2- 172.21.15.40:0/2647546578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03c4142650 0x7f03c41595b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00cc70 tx=0x7f03bc00cf80 comp rx=0 tx=0).stop 2023-11-08T08:19:05.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.475+0000 7f03ca6b5700 1 -- 172.21.15.40:0/2647546578 shutdown_connections 2023-11-08T08:19:05.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.475+0000 7f03ca6b5700 1 --2- 172.21.15.40:0/2647546578 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f03b0064f10 0x7f03b00673c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:05.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.475+0000 7f03ca6b5700 1 --2- 172.21.15.40:0/2647546578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03c4142650 0x7f03c41595b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:05.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.475+0000 7f03ca6b5700 1 --2- 172.21.15.40:0/2647546578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03c40b5a00 0x7f03c413f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:05.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.475+0000 7f03ca6b5700 1 --2- 172.21.15.40:0/2647546578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03c406aab0 0x7f03c4158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:05.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.475+0000 7f03ca6b5700 1 -- 172.21.15.40:0/2647546578 >> 172.21.15.40:0/2647546578 conn(0x7f03c40b5390 msgr2=0x7f03c4142a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:05.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.475+0000 7f03ca6b5700 1 -- 172.21.15.40:0/2647546578 shutdown_connections 2023-11-08T08:19:05.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.475+0000 7f03ca6b5700 1 -- 172.21.15.40:0/2647546578 wait complete. 2023-11-08T08:19:05.503 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T08:19:05.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.691+0000 7ff5acee4700 1 Processor -- start 2023-11-08T08:19:05.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.691+0000 7ff5acee4700 1 -- start start 2023-11-08T08:19:05.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.691+0000 7ff5acee4700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff5a8159ea0 0x7ff5a815d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:05.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.691+0000 7ff5acee4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5a806a360 0x7ff5a8070130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:05.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5acee4700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a80d0810 con 0x7ff5a8159ad0 2023-11-08T08:19:05.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5acee4700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a815d780 con 0x7ff5a806a360 2023-11-08T08:19:05.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5acee4700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a8069bf0 con 0x7ff5a8159ea0 2023-11-08T08:19:05.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5a7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5a806a360 0x7ff5a8070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:05.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5a7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff5a806a360 0x7ff5a8070130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56222/0 (socket says 172.21.15.40:56222) 2023-11-08T08:19:05.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5a7fff700 1 -- 172.21.15.40:0/2119888518 learned_addr learned my addr 172.21.15.40:0/2119888518 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:05.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5a6ffd700 1 --2- 172.21.15.40:0/2119888518 >> v2:172.21.15.169:3300/0 conn(0x7ff5a8159ea0 0x7ff5a815d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:05.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5a7fff700 1 -- 172.21.15.40:0/2119888518 >> v2:172.21.15.169:3300/0 conn(0x7ff5a8159ea0 msgr2=0x7ff5a815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:05.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5a7fff700 1 --2- 172.21.15.40:0/2119888518 >> v2:172.21.15.169:3300/0 conn(0x7ff5a8159ea0 0x7ff5a815d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:05.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5a7fff700 1 -- 172.21.15.40:0/2119888518 >> v1:172.21.15.169:6789/0 conn(0x7ff5a8159ad0 legacy=0x7ff5a80d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:05.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5a7fff700 1 -- 172.21.15.40:0/2119888518 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a80d0a60 con 0x7ff5a806a360 2023-11-08T08:19:05.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5a7fff700 1 --2- 172.21.15.40:0/2119888518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a806a360 0x7ff5a8070130 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8070cd0 tx=0x7ff5a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ab6171846898a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:05.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5a67fc700 1 -- 172.21.15.40:0/2119888518 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a000e070 con 0x7ff5a806a360 2023-11-08T08:19:05.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5a67fc700 1 -- 172.21.15.40:0/2119888518 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a000b910 con 0x7ff5a806a360 2023-11-08T08:19:05.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5a67fc700 1 -- 172.21.15.40:0/2119888518 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a000bcc0 con 0x7ff5a806a360 2023-11-08T08:19:05.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5acee4700 1 -- 172.21.15.40:0/2119888518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a806a360 msgr2=0x7ff5a8070130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:05.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5acee4700 1 --2- 172.21.15.40:0/2119888518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a806a360 0x7ff5a8070130 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8070cd0 tx=0x7ff5a000b330 comp rx=0 tx=0).stop 2023-11-08T08:19:05.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5acee4700 1 -- 172.21.15.40:0/2119888518 shutdown_connections 2023-11-08T08:19:05.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5acee4700 1 --2- 172.21.15.40:0/2119888518 >> v2:172.21.15.169:3300/0 conn(0x7ff5a8159ea0 0x7ff5a815d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:05.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5acee4700 1 --2- 172.21.15.40:0/2119888518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a806a360 0x7ff5a8070130 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:05.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5acee4700 1 -- 172.21.15.40:0/2119888518 >> 172.21.15.40:0/2119888518 conn(0x7ff5a80d0350 msgr2=0x7ff5a80d2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:05.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5acee4700 1 -- 172.21.15.40:0/2119888518 shutdown_connections 2023-11-08T08:19:05.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.695+0000 7ff5acee4700 1 -- 172.21.15.40:0/2119888518 wait complete. 2023-11-08T08:19:05.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.699+0000 7ff5acee4700 1 Processor -- start 2023-11-08T08:19:05.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.699+0000 7ff5acee4700 1 -- start start 2023-11-08T08:19:05.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff5acee4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a806a360 0x7ff5a80cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:05.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff5acee4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a8159ad0 0x7ff5a8158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:05.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff5acee4700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5a8159ea0 0x7ff5a81584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:05.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff5acee4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80c77d0 con 0x7ff5a8159ad0 2023-11-08T08:19:05.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff5acee4700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80c6bc0 con 0x7ff5a806a360 2023-11-08T08:19:05.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff5acee4700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80c6ea0 con 0x7ff5a8159ea0 2023-11-08T08:19:05.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff5a6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a8159ad0 0x7ff5a8158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:05.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff5a6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a8159ad0 0x7ff5a8158ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56228/0 (socket says 172.21.15.40:56228) 2023-11-08T08:19:05.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff5a6ffd700 1 -- 172.21.15.40:0/701930087 learned_addr learned my addr 172.21.15.40:0/701930087 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:05.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff5a77fe700 1 --2- 172.21.15.40:0/701930087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a806a360 0x7ff5a80cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:05.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff5a7fff700 1 --2- 172.21.15.40:0/701930087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5a8159ea0 0x7ff5a81584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:05.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff5a6ffd700 1 -- 172.21.15.40:0/701930087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5a8159ea0 msgr2=0x7ff5a81584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:05.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff5a6ffd700 1 --2- 172.21.15.40:0/701930087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5a8159ea0 0x7ff5a81584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:05.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff5a6ffd700 1 -- 172.21.15.40:0/701930087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a806a360 msgr2=0x7ff5a80cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:05.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff5a6ffd700 1 --2- 172.21.15.40:0/701930087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a806a360 0x7ff5a80cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:05.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff5a6ffd700 1 -- 172.21.15.40:0/701930087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a0009b30 con 0x7ff5a8159ad0 2023-11-08T08:19:05.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff5a6ffd700 1 --2- 172.21.15.40:0/701930087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a8159ad0 0x7ff5a8158ae0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009fd0 tx=0x7ff59c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:05.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff58ffff700 1 -- 172.21.15.40:0/701930087 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0095a0 con 0x7ff5a8159ad0 2023-11-08T08:19:05.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff58ffff700 1 -- 172.21.15.40:0/701930087 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c013680 con 0x7ff5a8159ad0 2023-11-08T08:19:05.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff5acee4700 1 -- 172.21.15.40:0/701930087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a8069e40 con 0x7ff5a8159ad0 2023-11-08T08:19:05.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff58ffff700 1 -- 172.21.15.40:0/701930087 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c013a10 con 0x7ff5a8159ad0 2023-11-08T08:19:05.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.703+0000 7ff5acee4700 1 -- 172.21.15.40:0/701930087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a80d0d40 con 0x7ff5a8159ad0 2023-11-08T08:19:05.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.707+0000 7ff5acee4700 1 -- 172.21.15.40:0/701930087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a806a7e0 con 0x7ff5a8159ad0 2023-11-08T08:19:05.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.707+0000 7ff58ffff700 1 -- 172.21.15.40:0/701930087 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff59c023070 con 0x7ff5a8159ad0 2023-11-08T08:19:05.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.711+0000 7ff58ffff700 1 --2- 172.21.15.40:0/701930087 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff594064ee0 0x7ff594067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:05.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.711+0000 7ff58ffff700 1 -- 172.21.15.40:0/701930087 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6288+0+0 (secure 0 0 0) 0x7ff59c0821d0 con 0x7ff5a8159ad0 2023-11-08T08:19:05.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.711+0000 7ff5a77fe700 1 --2- 172.21.15.40:0/701930087 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff594064ee0 0x7ff594067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:05.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.711+0000 7ff5a77fe700 1 --2- 172.21.15.40:0/701930087 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff594064ee0 0x7ff594067390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff598000f60 tx=0x7ff598009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:05.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.715+0000 7ff58ffff700 1 -- 172.21.15.40:0/701930087 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ff59c0501b0 con 0x7ff5a8159ad0 2023-11-08T08:19:05.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:05.895+0000 7ff5acee4700 1 -- 172.21.15.40:0/701930087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff5a8073d20 con 0x7ff5a8159ad0 2023-11-08T08:19:07.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.487+0000 7ff58ffff700 1 -- 172.21.15.40:0/701930087 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v26) v1 ==== 228+0+0 (secure 0 0 0) 0x7ff59c0537d0 con 0x7ff5a8159ad0 2023-11-08T08:19:07.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.491+0000 7ff5acee4700 1 -- 172.21.15.40:0/701930087 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff594064ee0 msgr2=0x7ff594067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:07.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.491+0000 7ff5acee4700 1 --2- 172.21.15.40:0/701930087 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff594064ee0 0x7ff594067390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff598000f60 tx=0x7ff598009040 comp rx=0 tx=0).stop 2023-11-08T08:19:07.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.491+0000 7ff5acee4700 1 -- 172.21.15.40:0/701930087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a8159ad0 msgr2=0x7ff5a8158ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:07.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.491+0000 7ff5acee4700 1 --2- 172.21.15.40:0/701930087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a8159ad0 0x7ff5a8158ae0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009fd0 tx=0x7ff59c00ebd0 comp rx=0 tx=0).stop 2023-11-08T08:19:07.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.495+0000 7ff5acee4700 1 -- 172.21.15.40:0/701930087 shutdown_connections 2023-11-08T08:19:07.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.495+0000 7ff5acee4700 1 --2- 172.21.15.40:0/701930087 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff594064ee0 0x7ff594067390 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:07.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.495+0000 7ff5acee4700 1 --2- 172.21.15.40:0/701930087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5a8159ea0 0x7ff5a81584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:07.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.495+0000 7ff5acee4700 1 --2- 172.21.15.40:0/701930087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5a8159ad0 0x7ff5a8158ae0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:07.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.495+0000 7ff5acee4700 1 --2- 172.21.15.40:0/701930087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5a806a360 0x7ff5a80cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:07.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.495+0000 7ff5acee4700 1 -- 172.21.15.40:0/701930087 >> 172.21.15.40:0/701930087 conn(0x7ff5a80d0350 msgr2=0x7ff5a80b81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:07.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.495+0000 7ff5acee4700 1 -- 172.21.15.40:0/701930087 shutdown_connections 2023-11-08T08:19:07.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.495+0000 7ff5acee4700 1 -- 172.21.15.40:0/701930087 wait complete. 2023-11-08T08:19:07.505 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data_ec' created 2023-11-08T08:19:07.528 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-11-08T08:19:07.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.719+0000 7f98f64d8700 1 Processor -- start 2023-11-08T08:19:07.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.719+0000 7f98f64d8700 1 -- start start 2023-11-08T08:19:07.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98f64d8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98f0142650 0x7f98f0142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:07.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98f64d8700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f006b8d0 con 0x7f98f006b1c0 2023-11-08T08:19:07.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98f64d8700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f006bb70 con 0x7f98f00b5a00 2023-11-08T08:19:07.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98f64d8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f006be50 con 0x7f98f0142650 2023-11-08T08:19:07.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98f4a75700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f98f006b1c0 0x7f98f006c930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51444/0 (socket says 172.21.15.40:51444) 2023-11-08T08:19:07.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98f4a75700 1 -- 172.21.15.40:0/4083073471 learned_addr learned my addr 172.21.15.40:0/4083073471 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:07.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98effff700 1 --2- 172.21.15.40:0/4083073471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98f0142650 0x7f98f0142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:07.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98eeffd700 1 -- 172.21.15.40:0/4083073471 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 817485312 0 0) 0x7f98f006b8d0 con 0x7f98f006b1c0 2023-11-08T08:19:07.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98eeffd700 1 -- 172.21.15.40:0/4083073471 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98d80035d0 con 0x7f98f006b1c0 2023-11-08T08:19:07.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98eeffd700 1 -- 172.21.15.40:0/4083073471 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 179731484 0 0) 0x7f98d80035d0 con 0x7f98f006b1c0 2023-11-08T08:19:07.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98eeffd700 1 -- 172.21.15.40:0/4083073471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98f0142650 msgr2=0x7f98f0142fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:07.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98eeffd700 1 --2- 172.21.15.40:0/4083073471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98f0142650 0x7f98f0142fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:07.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98eeffd700 1 -- 172.21.15.40:0/4083073471 >> v1:172.21.15.169:6789/0 conn(0x7f98f00b5a00 legacy=0x7f98f006aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:07.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98eeffd700 1 -- 172.21.15.40:0/4083073471 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f006c130 con 0x7f98f006b1c0 2023-11-08T08:19:07.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98eeffd700 1 -- 172.21.15.40:0/4083073471 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f98e8001a10 con 0x7f98f006b1c0 2023-11-08T08:19:07.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98eeffd700 1 -- 172.21.15.40:0/4083073471 >> v1:172.21.15.40:6789/0 conn(0x7f98f006b1c0 legacy=0x7f98f006c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:07.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98eeffd700 1 --2- 172.21.15.40:0/4083073471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98d8003a50 0x7f98d8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:07.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98eeffd700 1 --2- 172.21.15.40:0/4083073471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d80043f0 0x7f98d80048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:07.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98eeffd700 1 --2- 172.21.15.40:0/4083073471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98f0142650 0x7f98d8005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:07.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98eeffd700 1 -- 172.21.15.40:0/4083073471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f006be50 con 0x7f98d80043f0 2023-11-08T08:19:07.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98eeffd700 1 -- 172.21.15.40:0/4083073471 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d800ba30 con 0x7f98d8003a50 2023-11-08T08:19:07.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.723+0000 7f98eeffd700 1 -- 172.21.15.40:0/4083073471 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d800bbc0 con 0x7f98f0142650 2023-11-08T08:19:07.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.727+0000 7f98ef7fe700 1 --2- 172.21.15.40:0/4083073471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d80043f0 0x7f98d80048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:07.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.727+0000 7f98effff700 1 --2- 172.21.15.40:0/4083073471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98d8003a50 0x7f98d8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:07.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.727+0000 7f98f4a75700 1 --2- 172.21.15.40:0/4083073471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98f0142650 0x7f98d8005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:07.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.727+0000 7f98ef7fe700 1 -- 172.21.15.40:0/4083073471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98f0142650 msgr2=0x7f98d8005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:07.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.727+0000 7f98ef7fe700 1 --2- 172.21.15.40:0/4083073471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98f0142650 0x7f98d8005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:07.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.727+0000 7f98ef7fe700 1 -- 172.21.15.40:0/4083073471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98d8003a50 msgr2=0x7f98d8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:07.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.727+0000 7f98ef7fe700 1 --2- 172.21.15.40:0/4083073471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98d8003a50 0x7f98d8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:07.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.727+0000 7f98ef7fe700 1 -- 172.21.15.40:0/4083073471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d80035d0 con 0x7f98d80043f0 2023-11-08T08:19:07.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.727+0000 7f98ef7fe700 1 --2- 172.21.15.40:0/4083073471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d80043f0 0x7f98d80048b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f98e40018f0 tx=0x7f98d80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fb23221f954232 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:07.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.727+0000 7f98eeffd700 1 -- 172.21.15.40:0/4083073471 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e4001390 con 0x7f98d80043f0 2023-11-08T08:19:07.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.727+0000 7f98eeffd700 1 -- 172.21.15.40:0/4083073471 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e400c690 con 0x7f98d80043f0 2023-11-08T08:19:07.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.727+0000 7f98eeffd700 1 -- 172.21.15.40:0/4083073471 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e400ca40 con 0x7f98d80043f0 2023-11-08T08:19:07.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.727+0000 7f98f64d8700 1 -- 172.21.15.40:0/4083073471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d80043f0 msgr2=0x7f98d80048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:07.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.727+0000 7f98f64d8700 1 --2- 172.21.15.40:0/4083073471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d80043f0 0x7f98d80048b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f98e40018f0 tx=0x7f98d80058a0 comp rx=0 tx=0).stop 2023-11-08T08:19:07.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.727+0000 7f98f4a75700 1 -- 172.21.15.40:0/4083073471 reap_dead start 2023-11-08T08:19:07.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.727+0000 7f98f64d8700 1 -- 172.21.15.40:0/4083073471 shutdown_connections 2023-11-08T08:19:07.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.727+0000 7f98f64d8700 1 -- 172.21.15.40:0/4083073471 >> 172.21.15.40:0/4083073471 conn(0x7f98f00b5390 msgr2=0x7f98f00d2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:07.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.727+0000 7f98f64d8700 1 -- 172.21.15.40:0/4083073471 shutdown_connections 2023-11-08T08:19:07.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.727+0000 7f98f64d8700 1 -- 172.21.15.40:0/4083073471 wait complete. 2023-11-08T08:19:07.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.731+0000 7f98f64d8700 1 Processor -- start 2023-11-08T08:19:07.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.731+0000 7f98f64d8700 1 -- start start 2023-11-08T08:19:07.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98f64d8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98d80043f0 0x7f98f01576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:07.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98f64d8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98f0156d40 0x7f98f01407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:07.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98f64d8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f015bfa0 0x7f98f01438c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:07.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98f64d8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f015bb10 con 0x7f98f015bfa0 2023-11-08T08:19:07.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98f64d8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f015a2c0 con 0x7f98d80043f0 2023-11-08T08:19:07.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98f64d8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f0157dd0 con 0x7f98f0156d40 2023-11-08T08:19:07.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98f4a75700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f015bfa0 0x7f98f01438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:07.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98f4a75700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f015bfa0 0x7f98f01438c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43264/0 (socket says 172.21.15.40:43264) 2023-11-08T08:19:07.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98f4a75700 1 -- 172.21.15.40:0/3245411523 learned_addr learned my addr 172.21.15.40:0/3245411523 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:07.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98effff700 1 --2- 172.21.15.40:0/3245411523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98d80043f0 0x7f98f01576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:07.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98ef7fe700 1 --2- 172.21.15.40:0/3245411523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98f0156d40 0x7f98f01407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:07.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98f4a75700 1 -- 172.21.15.40:0/3245411523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98f0156d40 msgr2=0x7f98f01407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:07.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98f4a75700 1 --2- 172.21.15.40:0/3245411523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98f0156d40 0x7f98f01407e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:07.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98f4a75700 1 -- 172.21.15.40:0/3245411523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98d80043f0 msgr2=0x7f98f01576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:07.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98f4a75700 1 --2- 172.21.15.40:0/3245411523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98d80043f0 0x7f98f01576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:07.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98f4a75700 1 -- 172.21.15.40:0/3245411523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f0142a20 con 0x7f98f015bfa0 2023-11-08T08:19:07.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98ef7fe700 1 --2- 172.21.15.40:0/3245411523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98f0156d40 0x7f98f01407e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:07.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98f4a75700 1 --2- 172.21.15.40:0/3245411523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f015bfa0 0x7f98f01438c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f98e8000f80 tx=0x7f98e8007060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:07.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98ecff9700 1 -- 172.21.15.40:0/3245411523 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e80077d0 con 0x7f98f015bfa0 2023-11-08T08:19:07.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98ecff9700 1 -- 172.21.15.40:0/3245411523 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e8007950 con 0x7f98f015bfa0 2023-11-08T08:19:07.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98ecff9700 1 -- 172.21.15.40:0/3245411523 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e80133f0 con 0x7f98f015bfa0 2023-11-08T08:19:07.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98f64d8700 1 -- 172.21.15.40:0/3245411523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f006c410 con 0x7f98f015bfa0 2023-11-08T08:19:07.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.735+0000 7f98f64d8700 1 -- 172.21.15.40:0/3245411523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f0142fa0 con 0x7f98f015bfa0 2023-11-08T08:19:07.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.739+0000 7f98f64d8700 1 -- 172.21.15.40:0/3245411523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f00726c0 con 0x7f98f015bfa0 2023-11-08T08:19:07.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.743+0000 7f98ecff9700 1 -- 172.21.15.40:0/3245411523 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f98e8023070 con 0x7f98f015bfa0 2023-11-08T08:19:07.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.743+0000 7f98ecff9700 1 --2- 172.21.15.40:0/3245411523 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98dc064f10 0x7f98dc0673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:07.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.743+0000 7f98ecff9700 1 -- 172.21.15.40:0/3245411523 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f98e8082e10 con 0x7f98f015bfa0 2023-11-08T08:19:07.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.743+0000 7f98effff700 1 --2- 172.21.15.40:0/3245411523 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98dc064f10 0x7f98dc0673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:07.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.743+0000 7f98effff700 1 --2- 172.21.15.40:0/3245411523 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98dc064f10 0x7f98dc0673c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009bf0 tx=0x7f98e0009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:07.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.747+0000 7f98ecff9700 1 -- 172.21.15.40:0/3245411523 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f98e8050c70 con 0x7f98f015bfa0 2023-11-08T08:19:07.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:07.923+0000 7f98f64d8700 1 -- 172.21.15.40:0/3245411523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f98f015fa80 con 0x7f98f015bfa0 2023-11-08T08:19:08.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.491+0000 7f98ecff9700 1 -- 172.21.15.40:0/3245411523 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7f98e80170c0 con 0x7f98f015bfa0 2023-11-08T08:19:08.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.499+0000 7f98f64d8700 1 -- 172.21.15.40:0/3245411523 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98dc064f10 msgr2=0x7f98dc0673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:08.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.499+0000 7f98f64d8700 1 --2- 172.21.15.40:0/3245411523 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98dc064f10 0x7f98dc0673c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009bf0 tx=0x7f98e0009240 comp rx=0 tx=0).stop 2023-11-08T08:19:08.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.499+0000 7f98f64d8700 1 -- 172.21.15.40:0/3245411523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f015bfa0 msgr2=0x7f98f01438c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:08.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.499+0000 7f98f64d8700 1 --2- 172.21.15.40:0/3245411523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f015bfa0 0x7f98f01438c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f98e8000f80 tx=0x7f98e8007060 comp rx=0 tx=0).stop 2023-11-08T08:19:08.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.499+0000 7f98f64d8700 1 -- 172.21.15.40:0/3245411523 shutdown_connections 2023-11-08T08:19:08.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.499+0000 7f98f64d8700 1 --2- 172.21.15.40:0/3245411523 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98dc064f10 0x7f98dc0673c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:08.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.499+0000 7f98f64d8700 1 --2- 172.21.15.40:0/3245411523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98f015bfa0 0x7f98f01438c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:08.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.499+0000 7f98f64d8700 1 --2- 172.21.15.40:0/3245411523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98f0156d40 0x7f98f01407e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:08.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.499+0000 7f98f64d8700 1 --2- 172.21.15.40:0/3245411523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98d80043f0 0x7f98f01576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:08.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.499+0000 7f98f64d8700 1 -- 172.21.15.40:0/3245411523 >> 172.21.15.40:0/3245411523 conn(0x7f98f00b5390 msgr2=0x7f98f00b5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:08.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.503+0000 7f98f64d8700 1 -- 172.21.15.40:0/3245411523 shutdown_connections 2023-11-08T08:19:08.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.507+0000 7f98f64d8700 1 -- 172.21.15.40:0/3245411523 wait complete. 2023-11-08T08:19:08.525 INFO:teuthology.orchestra.run.smithi040.stderr:set pool 5 allow_ec_overwrites to true 2023-11-08T08:19:08.554 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-11-08T08:19:08.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.727+0000 7f68102bc700 1 Processor -- start 2023-11-08T08:19:08.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.727+0000 7f68102bc700 1 -- start start 2023-11-08T08:19:08.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.731+0000 7f68102bc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f680815c700 0x7f68080702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:08.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.731+0000 7f68102bc700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f68080b2130 0x7f6808070830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:08.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.731+0000 7f68102bc700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680806ac00 con 0x7f68080b1be0 2023-11-08T08:19:08.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.731+0000 7f68102bc700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68080cd360 con 0x7f680815c700 2023-11-08T08:19:08.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.731+0000 7f68102bc700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6808160190 con 0x7f68080b2130 2023-11-08T08:19:08.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.731+0000 7f680e058700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f680815c700 0x7f68080702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:08.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.731+0000 7f680e058700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f680815c700 0x7f68080702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43266/0 (socket says 172.21.15.40:43266) 2023-11-08T08:19:08.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.731+0000 7f680e058700 1 -- 172.21.15.40:0/2117775623 learned_addr learned my addr 172.21.15.40:0/2117775623 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:08.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.731+0000 7f680d857700 1 --2- 172.21.15.40:0/2117775623 >> v2:172.21.15.169:3300/0 conn(0x7f68080b2130 0x7f6808070830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:08.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.731+0000 7f680e058700 1 -- 172.21.15.40:0/2117775623 >> v2:172.21.15.169:3300/0 conn(0x7f68080b2130 msgr2=0x7f6808070830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:08.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.731+0000 7f680e058700 1 --2- 172.21.15.40:0/2117775623 >> v2:172.21.15.169:3300/0 conn(0x7f68080b2130 0x7f6808070830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:08.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.731+0000 7f680e058700 1 -- 172.21.15.40:0/2117775623 >> v1:172.21.15.169:6789/0 conn(0x7f68080b1be0 legacy=0x7f680806c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:08.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.731+0000 7f680e058700 1 -- 172.21.15.40:0/2117775623 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6808160470 con 0x7f680815c700 2023-11-08T08:19:08.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.731+0000 7f680e058700 1 --2- 172.21.15.40:0/2117775623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f680815c700 0x7f68080702f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f68080700f0 tx=0x7f6804009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae9069c85c0aae server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:08.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.731+0000 7f680d056700 1 -- 172.21.15.40:0/2117775623 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680400b8a0 con 0x7f680815c700 2023-11-08T08:19:08.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.731+0000 7f680d056700 1 -- 172.21.15.40:0/2117775623 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680400ba00 con 0x7f680815c700 2023-11-08T08:19:08.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.731+0000 7f680d056700 1 -- 172.21.15.40:0/2117775623 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680400bd90 con 0x7f680815c700 2023-11-08T08:19:08.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.735+0000 7f68102bc700 1 -- 172.21.15.40:0/2117775623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f680815c700 msgr2=0x7f68080702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:08.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.735+0000 7f68102bc700 1 --2- 172.21.15.40:0/2117775623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f680815c700 0x7f68080702f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f68080700f0 tx=0x7f6804009bf0 comp rx=0 tx=0).stop 2023-11-08T08:19:08.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.735+0000 7f68102bc700 1 -- 172.21.15.40:0/2117775623 shutdown_connections 2023-11-08T08:19:08.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.735+0000 7f68102bc700 1 --2- 172.21.15.40:0/2117775623 >> v2:172.21.15.169:3300/0 conn(0x7f68080b2130 0x7f6808070830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:08.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.735+0000 7f68102bc700 1 --2- 172.21.15.40:0/2117775623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f680815c700 0x7f68080702f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:08.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.735+0000 7f68102bc700 1 -- 172.21.15.40:0/2117775623 >> 172.21.15.40:0/2117775623 conn(0x7f68080d2d10 msgr2=0x7f68080d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:08.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.735+0000 7f68102bc700 1 -- 172.21.15.40:0/2117775623 shutdown_connections 2023-11-08T08:19:08.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.735+0000 7f68102bc700 1 -- 172.21.15.40:0/2117775623 wait complete. 2023-11-08T08:19:08.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.739+0000 7f68102bc700 1 Processor -- start 2023-11-08T08:19:08.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.739+0000 7f68102bc700 1 -- start start 2023-11-08T08:19:08.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.739+0000 7f68102bc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68080b1be0 0x7f680815f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:08.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.739+0000 7f68102bc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68080b2130 0x7f68081f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:08.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.739+0000 7f68102bc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f680815c700 0x7f68081eef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:08.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.739+0000 7f68102bc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680815cad0 con 0x7f68080b1be0 2023-11-08T08:19:08.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.739+0000 7f68102bc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680815cd20 con 0x7f680815c700 2023-11-08T08:19:08.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.739+0000 7f68102bc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680815d000 con 0x7f68080b2130 2023-11-08T08:19:08.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.739+0000 7f680e058700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68080b1be0 0x7f680815f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:08.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.739+0000 7f680e058700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68080b1be0 0x7f680815f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43268/0 (socket says 172.21.15.40:43268) 2023-11-08T08:19:08.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.739+0000 7f680e058700 1 -- 172.21.15.40:0/1378778298 learned_addr learned my addr 172.21.15.40:0/1378778298 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:08.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.739+0000 7f680d857700 1 --2- 172.21.15.40:0/1378778298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68080b2130 0x7f68081f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:08.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.739+0000 7f680e058700 1 -- 172.21.15.40:0/1378778298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68080b2130 msgr2=0x7f68081f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:08.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.739+0000 7f680e058700 1 --2- 172.21.15.40:0/1378778298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68080b2130 0x7f68081f56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:08.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.739+0000 7f680e058700 1 -- 172.21.15.40:0/1378778298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f680815c700 msgr2=0x7f68081eef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:08.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.739+0000 7f680e859700 1 --2- 172.21.15.40:0/1378778298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f680815c700 0x7f68081eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:08.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.739+0000 7f680e058700 1 --2- 172.21.15.40:0/1378778298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f680815c700 0x7f68081eef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:08.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.739+0000 7f680e058700 1 -- 172.21.15.40:0/1378778298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6804009570 con 0x7f68080b1be0 2023-11-08T08:19:08.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.739+0000 7f680e859700 1 --2- 172.21.15.40:0/1378778298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f680815c700 0x7f68081eef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:08.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.743+0000 7f680e058700 1 --2- 172.21.15.40:0/1378778298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68080b1be0 0x7f680815f280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f68080d3a50 tx=0x7f6804004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:08.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.743+0000 7f67feffd700 1 -- 172.21.15.40:0/1378778298 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68040038c0 con 0x7f68080b1be0 2023-11-08T08:19:08.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.743+0000 7f67feffd700 1 -- 172.21.15.40:0/1378778298 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680401f070 con 0x7f68080b1be0 2023-11-08T08:19:08.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.743+0000 7f68102bc700 1 -- 172.21.15.40:0/1378778298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6808160a30 con 0x7f68080b1be0 2023-11-08T08:19:08.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.743+0000 7f67feffd700 1 -- 172.21.15.40:0/1378778298 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804003c60 con 0x7f68080b1be0 2023-11-08T08:19:08.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.743+0000 7f68102bc700 1 -- 172.21.15.40:0/1378778298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6808160750 con 0x7f68080b1be0 2023-11-08T08:19:08.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.743+0000 7f68102bc700 1 -- 172.21.15.40:0/1378778298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680806b530 con 0x7f68080b1be0 2023-11-08T08:19:08.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.743+0000 7f67feffd700 1 -- 172.21.15.40:0/1378778298 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f680400f050 con 0x7f68080b1be0 2023-11-08T08:19:08.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.747+0000 7f67feffd700 1 --2- 172.21.15.40:0/1378778298 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f67ec064ee0 0x7f67ec067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:08.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.747+0000 7f67feffd700 1 -- 172.21.15.40:0/1378778298 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f6804021070 con 0x7f68080b1be0 2023-11-08T08:19:08.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.747+0000 7f680d857700 1 --2- 172.21.15.40:0/1378778298 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f67ec064ee0 0x7f67ec067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:08.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.747+0000 7f680d857700 1 --2- 172.21.15.40:0/1378778298 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f67ec064ee0 0x7f67ec067390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6808157d20 tx=0x7f67f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:08.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.755+0000 7f67feffd700 1 -- 172.21.15.40:0/1378778298 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f6804051b80 con 0x7f68080b1be0 2023-11-08T08:19:08.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.975+0000 7f68102bc700 1 -- 172.21.15.40:0/1378778298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f68080726c0 con 0x7f68080b1be0 2023-11-08T08:19:08.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.983+0000 7f67feffd700 1 -- 172.21.15.40:0/1378778298 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f68040551a0 con 0x7f68080b1be0 2023-11-08T08:19:08.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.987+0000 7f68102bc700 1 -- 172.21.15.40:0/1378778298 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f67ec064ee0 msgr2=0x7f67ec067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:08.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.987+0000 7f68102bc700 1 --2- 172.21.15.40:0/1378778298 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f67ec064ee0 0x7f67ec067390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6808157d20 tx=0x7f67f8008040 comp rx=0 tx=0).stop 2023-11-08T08:19:08.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.987+0000 7f68102bc700 1 -- 172.21.15.40:0/1378778298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68080b1be0 msgr2=0x7f680815f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:08.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.987+0000 7f68102bc700 1 --2- 172.21.15.40:0/1378778298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68080b1be0 0x7f680815f280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f68080d3a50 tx=0x7f6804004770 comp rx=0 tx=0).stop 2023-11-08T08:19:08.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.987+0000 7f68102bc700 1 -- 172.21.15.40:0/1378778298 shutdown_connections 2023-11-08T08:19:08.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.987+0000 7f68102bc700 1 --2- 172.21.15.40:0/1378778298 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f67ec064ee0 0x7f67ec067390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:08.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.987+0000 7f68102bc700 1 --2- 172.21.15.40:0/1378778298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f680815c700 0x7f68081eef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:08.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.987+0000 7f68102bc700 1 --2- 172.21.15.40:0/1378778298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68080b2130 0x7f68081f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:08.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.987+0000 7f68102bc700 1 --2- 172.21.15.40:0/1378778298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68080b1be0 0x7f680815f280 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:08.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.987+0000 7f68102bc700 1 -- 172.21.15.40:0/1378778298 >> 172.21.15.40:0/1378778298 conn(0x7f68080d2d10 msgr2=0x7f68080b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:08.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.987+0000 7f68102bc700 1 -- 172.21.15.40:0/1378778298 shutdown_connections 2023-11-08T08:19:08.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:08.987+0000 7f68102bc700 1 -- 172.21.15.40:0/1378778298 wait complete. 2023-11-08T08:19:08.996 INFO:teuthology.orchestra.run.smithi040.stderr:added data pool 5 to fsmap 2023-11-08T08:19:09.015 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:19:09.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.195+0000 7f77d6662700 1 Processor -- start 2023-11-08T08:19:09.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.195+0000 7f77d6662700 1 -- start start 2023-11-08T08:19:09.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.199+0000 7f77d6662700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f77d00cba00 0x7f77d015d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:09.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.199+0000 7f77d6662700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d015c750 0x7f77d015cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:09.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.199+0000 7f77d6662700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d015d790 con 0x7f77d00b1b30 2023-11-08T08:19:09.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.199+0000 7f77d6662700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d015d1d0 con 0x7f77d00cba00 2023-11-08T08:19:09.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.199+0000 7f77d6662700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d015d4b0 con 0x7f77d015c750 2023-11-08T08:19:09.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.199+0000 7f77cffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f77d00cba00 0x7f77d015d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:09.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.199+0000 7f77d5400700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f77d00b1b30 0x7f77d00cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:58604/0 (socket says 172.21.15.40:58604) 2023-11-08T08:19:09.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.199+0000 7f77d5400700 1 -- 172.21.15.40:0/1978143841 learned_addr learned my addr 172.21.15.40:0/1978143841 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:09.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.199+0000 7f77c7fff700 1 --2- 172.21.15.40:0/1978143841 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d015c750 0x7f77d015cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:09.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.199+0000 7f77cffff700 1 -- 172.21.15.40:0/1978143841 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d015c750 msgr2=0x7f77d015cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:09.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.199+0000 7f77cffff700 1 --2- 172.21.15.40:0/1978143841 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d015c750 0x7f77d015cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:09.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.199+0000 7f77cffff700 1 -- 172.21.15.40:0/1978143841 >> v1:172.21.15.169:6789/0 conn(0x7f77d00b1b30 legacy=0x7f77d00cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:09.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.199+0000 7f77cffff700 1 -- 172.21.15.40:0/1978143841 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d00cebc0 con 0x7f77d00cba00 2023-11-08T08:19:09.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.199+0000 7f77cf7fe700 1 -- 172.21.15.40:0/1978143841 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331341763 0 0) 0x7f77d015d790 con 0x7f77d00b1b30 2023-11-08T08:19:09.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.199+0000 7f77cffff700 1 --2- 172.21.15.40:0/1978143841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77d00cba00 0x7f77d015d960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009850 tx=0x7f77c0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43593a14a084931b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:09.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.199+0000 7f77cf7fe700 1 -- 172.21.15.40:0/1978143841 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0011830 con 0x7f77d00cba00 2023-11-08T08:19:09.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.199+0000 7f77cf7fe700 1 -- 172.21.15.40:0/1978143841 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c0011990 con 0x7f77d00cba00 2023-11-08T08:19:09.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.199+0000 7f77cf7fe700 1 -- 172.21.15.40:0/1978143841 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0011d20 con 0x7f77d00cba00 2023-11-08T08:19:09.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.203+0000 7f77d6662700 1 -- 172.21.15.40:0/1978143841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77d00cba00 msgr2=0x7f77d015d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:09.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.203+0000 7f77d6662700 1 --2- 172.21.15.40:0/1978143841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77d00cba00 0x7f77d015d960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009850 tx=0x7f77c0009b60 comp rx=0 tx=0).stop 2023-11-08T08:19:09.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.203+0000 7f77d6662700 1 -- 172.21.15.40:0/1978143841 shutdown_connections 2023-11-08T08:19:09.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.203+0000 7f77d6662700 1 --2- 172.21.15.40:0/1978143841 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d015c750 0x7f77d015cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:09.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.203+0000 7f77d6662700 1 --2- 172.21.15.40:0/1978143841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77d00cba00 0x7f77d015d960 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:09.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.203+0000 7f77d6662700 1 -- 172.21.15.40:0/1978143841 >> 172.21.15.40:0/1978143841 conn(0x7f77d00ccb30 msgr2=0x7f77d00ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:09.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.203+0000 7f77d6662700 1 -- 172.21.15.40:0/1978143841 shutdown_connections 2023-11-08T08:19:09.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.203+0000 7f77d6662700 1 -- 172.21.15.40:0/1978143841 wait complete. 2023-11-08T08:19:09.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.203+0000 7f77d6662700 1 Processor -- start 2023-11-08T08:19:09.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.203+0000 7f77d6662700 1 -- start start 2023-11-08T08:19:09.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.207+0000 7f77d6662700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d00b1b30 0x7f77d00c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:09.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.207+0000 7f77d6662700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d00cba00 0x7f77d01f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:09.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.207+0000 7f77d6662700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77d015c750 0x7f77d0150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:09.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.207+0000 7f77d6662700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d00c5960 con 0x7f77d015c750 2023-11-08T08:19:09.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.207+0000 7f77d6662700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d0150c10 con 0x7f77d00cba00 2023-11-08T08:19:09.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.207+0000 7f77d6662700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d01f5c40 con 0x7f77d00b1b30 2023-11-08T08:19:09.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.207+0000 7f77d5400700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77d015c750 0x7f77d0150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:09.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.207+0000 7f77c7fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d00cba00 0x7f77d01f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:09.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.207+0000 7f77d5400700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77d015c750 0x7f77d0150100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43286/0 (socket says 172.21.15.40:43286) 2023-11-08T08:19:09.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.207+0000 7f77cffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d00b1b30 0x7f77d00c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:09.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.207+0000 7f77d5400700 1 -- 172.21.15.40:0/1664655486 learned_addr learned my addr 172.21.15.40:0/1664655486 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:09.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.207+0000 7f77cffff700 1 -- 172.21.15.40:0/1664655486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d00cba00 msgr2=0x7f77d01f5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:09.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.207+0000 7f77cffff700 1 --2- 172.21.15.40:0/1664655486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d00cba00 0x7f77d01f5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:09.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.207+0000 7f77cffff700 1 -- 172.21.15.40:0/1664655486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77d015c750 msgr2=0x7f77d0150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:09.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.207+0000 7f77cffff700 1 --2- 172.21.15.40:0/1664655486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77d015c750 0x7f77d0150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:09.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.207+0000 7f77cffff700 1 -- 172.21.15.40:0/1664655486 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c00094b0 con 0x7f77d00b1b30 2023-11-08T08:19:09.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.207+0000 7f77d5400700 1 --2- 172.21.15.40:0/1664655486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77d015c750 0x7f77d0150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:09.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.207+0000 7f77cffff700 1 --2- 172.21.15.40:0/1664655486 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d00b1b30 0x7f77d00c3be0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f77c0016000 tx=0x7f77c0004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:09.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.211+0000 7f77cd7fa700 1 -- 172.21.15.40:0/1664655486 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0004480 con 0x7f77d00b1b30 2023-11-08T08:19:09.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.211+0000 7f77cd7fa700 1 -- 172.21.15.40:0/1664655486 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c001f070 con 0x7f77d00b1b30 2023-11-08T08:19:09.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.211+0000 7f77cd7fa700 1 -- 172.21.15.40:0/1664655486 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0003900 con 0x7f77d00b1b30 2023-11-08T08:19:09.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.211+0000 7f77d6662700 1 -- 172.21.15.40:0/1664655486 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d00caa60 con 0x7f77d00b1b30 2023-11-08T08:19:09.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.211+0000 7f77d6662700 1 -- 172.21.15.40:0/1664655486 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d00ceea0 con 0x7f77d00b1b30 2023-11-08T08:19:09.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.211+0000 7f77d6662700 1 -- 172.21.15.40:0/1664655486 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d00cabf0 con 0x7f77d00b1b30 2023-11-08T08:19:09.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.215+0000 7f77cd7fa700 1 -- 172.21.15.40:0/1664655486 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f77c002a020 con 0x7f77d00b1b30 2023-11-08T08:19:09.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.215+0000 7f77cd7fa700 1 --2- 172.21.15.40:0/1664655486 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f77b0064f10 0x7f77b00673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:09.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.215+0000 7f77cd7fa700 1 -- 172.21.15.40:0/1664655486 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f77c0083b20 con 0x7f77d00b1b30 2023-11-08T08:19:09.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.215+0000 7f77c7fff700 1 --2- 172.21.15.40:0/1664655486 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f77b0064f10 0x7f77b00673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:09.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.219+0000 7f77c7fff700 1 --2- 172.21.15.40:0/1664655486 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f77b0064f10 0x7f77b00673c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f77d0159b00 tx=0x7f77b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:09.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.223+0000 7f77cd7fa700 1 -- 172.21.15.40:0/1664655486 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f77c0051a70 con 0x7f77d00b1b30 2023-11-08T08:19:09.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.439+0000 7f77d6662700 1 -- 172.21.15.40:0/1664655486 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77d00726c0 con 0x7f77d00b1b30 2023-11-08T08:19:09.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.443+0000 7f77cd7fa700 1 -- 172.21.15.40:0/1664655486 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4641 (secure 0 0 0) 0x7f77c0055090 con 0x7f77d00b1b30 2023-11-08T08:19:09.448 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:09.448 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/196211087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":196211087},{"type":"v1","addr":"172.21.15.40:6834","nonce":196211087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/67127106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":67127106},{"type":"v1","addr":"172.21.15.40:6835","nonce":67127106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1231240563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1231240563},{"type":"v1","addr":"172.21.15.169:6835","nonce":1231240563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:19:04.456508+0000","modified":"2023-11-08T08:19:08.980644+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6837/1989495204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1989495204},{"type":"v1","addr":"172.21.15.169:6837","nonce":1989495204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-08T08:19:09.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.447+0000 7f77d6662700 1 -- 172.21.15.40:0/1664655486 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f77b0064f10 msgr2=0x7f77b00673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:09.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.447+0000 7f77d6662700 1 --2- 172.21.15.40:0/1664655486 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f77b0064f10 0x7f77b00673c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f77d0159b00 tx=0x7f77b8008040 comp rx=0 tx=0).stop 2023-11-08T08:19:09.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.447+0000 7f77d6662700 1 -- 172.21.15.40:0/1664655486 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d00b1b30 msgr2=0x7f77d00c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.447+0000 7f77d6662700 1 --2- 172.21.15.40:0/1664655486 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d00b1b30 0x7f77d00c3be0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f77c0016000 tx=0x7f77c0004770 comp rx=0 tx=0).stop 2023-11-08T08:19:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.447+0000 7f77d6662700 1 -- 172.21.15.40:0/1664655486 shutdown_connections 2023-11-08T08:19:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.447+0000 7f77d6662700 1 --2- 172.21.15.40:0/1664655486 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f77b0064f10 0x7f77b00673c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.447+0000 7f77d6662700 1 --2- 172.21.15.40:0/1664655486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77d015c750 0x7f77d0150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.447+0000 7f77d6662700 1 --2- 172.21.15.40:0/1664655486 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d00cba00 0x7f77d01f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.447+0000 7f77d6662700 1 --2- 172.21.15.40:0/1664655486 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d00b1b30 0x7f77d00c3be0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.447+0000 7f77d6662700 1 -- 172.21.15.40:0/1664655486 >> 172.21.15.40:0/1664655486 conn(0x7f77d00ccb30 msgr2=0x7f77d00ccf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.447+0000 7f77d6662700 1 -- 172.21.15.40:0/1664655486 shutdown_connections 2023-11-08T08:19:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.447+0000 7f77d6662700 1 -- 172.21.15.40:0/1664655486 wait complete. 2023-11-08T08:19:09.455 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2023-11-08T08:19:09.476 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:19:09.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.651+0000 7f5b51a6d700 1 Processor -- start 2023-11-08T08:19:09.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.651+0000 7f5b51a6d700 1 -- start start 2023-11-08T08:19:09.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.655+0000 7f5b51a6d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15c650 0x7f5b4c06b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:09.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.655+0000 7f5b51a6d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5b4c0cd0d0 0x7f5b4c0b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:09.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.655+0000 7f5b51a6d700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b4c0c9d40 con 0x7f5b4c0b1b80 2023-11-08T08:19:09.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.655+0000 7f5b51a6d700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b4c06b140 con 0x7f5b4c0cd0d0 2023-11-08T08:19:09.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.655+0000 7f5b51a6d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c06b820 con 0x7f5b4c15c650 2023-11-08T08:19:09.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.655+0000 7f5b43fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5b4c0cd0d0 0x7f5b4c0b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:09.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.655+0000 7f5b50a6b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15c650 0x7f5b4c06b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:09.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.655+0000 7f5b50a6b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15c650 0x7f5b4c06b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:41256/0 (socket says 172.21.15.40:41256) 2023-11-08T08:19:09.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.655+0000 7f5b43fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5b4c0cd0d0 0x7f5b4c0b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43290/0 (socket says 172.21.15.40:43290) 2023-11-08T08:19:09.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.655+0000 7f5b50a6b700 1 -- 172.21.15.40:0/3817650605 learned_addr learned my addr 172.21.15.40:0/3817650605 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:09.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.655+0000 7f5b5126c700 1 --1- 172.21.15.40:0/3817650605 >> v1:172.21.15.169:6789/0 conn(0x7f5b4c0b1b80 0x7f5b4c06bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:58614/0 (socket says 172.21.15.40:58614) 2023-11-08T08:19:09.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.655+0000 7f5b50a6b700 1 -- 172.21.15.40:0/3817650605 >> v2:172.21.15.40:3300/0 conn(0x7f5b4c0cd0d0 msgr2=0x7f5b4c0b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:09.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.655+0000 7f5b50a6b700 1 --2- 172.21.15.40:0/3817650605 >> v2:172.21.15.40:3300/0 conn(0x7f5b4c0cd0d0 0x7f5b4c0b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:09.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.655+0000 7f5b50a6b700 1 -- 172.21.15.40:0/3817650605 >> v1:172.21.15.169:6789/0 conn(0x7f5b4c0b1b80 legacy=0x7f5b4c06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:09.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.655+0000 7f5b50a6b700 1 -- 172.21.15.40:0/3817650605 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c0cd5d0 con 0x7f5b4c15c650 2023-11-08T08:19:09.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.655+0000 7f5b43fff700 1 --2- 172.21.15.40:0/3817650605 >> v2:172.21.15.40:3300/0 conn(0x7f5b4c0cd0d0 0x7f5b4c0b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:09.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.655+0000 7f5b50a6b700 1 --2- 172.21.15.40:0/3817650605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15c650 0x7f5b4c06b2e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009e50 tx=0x7f5b4800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eefe351d2aacf8aa server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:09.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.659+0000 7f5b437fe700 1 -- 172.21.15.40:0/3817650605 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4800e070 con 0x7f5b4c15c650 2023-11-08T08:19:09.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.659+0000 7f5b437fe700 1 -- 172.21.15.40:0/3817650605 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4800b910 con 0x7f5b4c15c650 2023-11-08T08:19:09.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.659+0000 7f5b437fe700 1 -- 172.21.15.40:0/3817650605 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4800bcc0 con 0x7f5b4c15c650 2023-11-08T08:19:09.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.659+0000 7f5b51a6d700 1 -- 172.21.15.40:0/3817650605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15c650 msgr2=0x7f5b4c06b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:09.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.659+0000 7f5b51a6d700 1 --2- 172.21.15.40:0/3817650605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15c650 0x7f5b4c06b2e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009e50 tx=0x7f5b4800b330 comp rx=0 tx=0).stop 2023-11-08T08:19:09.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.659+0000 7f5b51a6d700 1 -- 172.21.15.40:0/3817650605 shutdown_connections 2023-11-08T08:19:09.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.659+0000 7f5b51a6d700 1 --2- 172.21.15.40:0/3817650605 >> v2:172.21.15.40:3300/0 conn(0x7f5b4c0cd0d0 0x7f5b4c0b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:09.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.659+0000 7f5b51a6d700 1 --2- 172.21.15.40:0/3817650605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15c650 0x7f5b4c06b2e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:09.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.659+0000 7f5b51a6d700 1 -- 172.21.15.40:0/3817650605 >> 172.21.15.40:0/3817650605 conn(0x7f5b4c0b0f80 msgr2=0x7f5b4c0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:09.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.659+0000 7f5b51a6d700 1 -- 172.21.15.40:0/3817650605 shutdown_connections 2023-11-08T08:19:09.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.659+0000 7f5b51a6d700 1 -- 172.21.15.40:0/3817650605 wait complete. 2023-11-08T08:19:09.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.663+0000 7f5b51a6d700 1 Processor -- start 2023-11-08T08:19:09.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.663+0000 7f5b51a6d700 1 -- start start 2023-11-08T08:19:09.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.663+0000 7f5b51a6d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b4c0b1b80 0x7f5b4c157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:09.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.663+0000 7f5b51a6d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b4c0cd0d0 0x7f5b4c070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:09.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.663+0000 7f5b51a6d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15c650 0x7f5b4c149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:09.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.663+0000 7f5b51a6d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c0ab860 con 0x7f5b4c0b1b80 2023-11-08T08:19:09.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.663+0000 7f5b51a6d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c1580c0 con 0x7f5b4c0cd0d0 2023-11-08T08:19:09.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.663+0000 7f5b51a6d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c14cbe0 con 0x7f5b4c15c650 2023-11-08T08:19:09.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.663+0000 7f5b50a6b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b4c0b1b80 0x7f5b4c157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:09.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.663+0000 7f5b50a6b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b4c0b1b80 0x7f5b4c157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43298/0 (socket says 172.21.15.40:43298) 2023-11-08T08:19:09.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.663+0000 7f5b50a6b700 1 -- 172.21.15.40:0/2903021369 learned_addr learned my addr 172.21.15.40:0/2903021369 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:09.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.663+0000 7f5b43fff700 1 --2- 172.21.15.40:0/2903021369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b4c0cd0d0 0x7f5b4c070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:09.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.663+0000 7f5b5126c700 1 --2- 172.21.15.40:0/2903021369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15c650 0x7f5b4c149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:09.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.663+0000 7f5b50a6b700 1 -- 172.21.15.40:0/2903021369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15c650 msgr2=0x7f5b4c149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:09.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.663+0000 7f5b50a6b700 1 --2- 172.21.15.40:0/2903021369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15c650 0x7f5b4c149770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:09.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.663+0000 7f5b50a6b700 1 -- 172.21.15.40:0/2903021369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b4c0cd0d0 msgr2=0x7f5b4c070500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:09.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.663+0000 7f5b50a6b700 1 --2- 172.21.15.40:0/2903021369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b4c0cd0d0 0x7f5b4c070500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:09.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.663+0000 7f5b50a6b700 1 -- 172.21.15.40:0/2903021369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3c0094b0 con 0x7f5b4c0b1b80 2023-11-08T08:19:09.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.667+0000 7f5b50a6b700 1 --2- 172.21.15.40:0/2903021369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b4c0b1b80 0x7f5b4c157b80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800b370 tx=0x7f5b48004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:09.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.667+0000 7f5b417fa700 1 -- 172.21.15.40:0/2903021369 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b48004800 con 0x7f5b4c0b1b80 2023-11-08T08:19:09.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.667+0000 7f5b417fa700 1 -- 172.21.15.40:0/2903021369 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b480092e0 con 0x7f5b4c0b1b80 2023-11-08T08:19:09.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.667+0000 7f5b417fa700 1 -- 172.21.15.40:0/2903021369 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4801c650 con 0x7f5b4c0b1b80 2023-11-08T08:19:09.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.667+0000 7f5b51a6d700 1 -- 172.21.15.40:0/2903021369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b48009b30 con 0x7f5b4c0b1b80 2023-11-08T08:19:09.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.667+0000 7f5b51a6d700 1 -- 172.21.15.40:0/2903021369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4c0c99d0 con 0x7f5b4c0b1b80 2023-11-08T08:19:09.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.667+0000 7f5b51a6d700 1 -- 172.21.15.40:0/2903021369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b4c0cd8b0 con 0x7f5b4c0b1b80 2023-11-08T08:19:09.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.671+0000 7f5b417fa700 1 -- 172.21.15.40:0/2903021369 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5b4802b070 con 0x7f5b4c0b1b80 2023-11-08T08:19:09.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.671+0000 7f5b417fa700 1 --2- 172.21.15.40:0/2903021369 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5b38064f10 0x7f5b380673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:09.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.671+0000 7f5b417fa700 1 -- 172.21.15.40:0/2903021369 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5b48083830 con 0x7f5b4c0b1b80 2023-11-08T08:19:09.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.671+0000 7f5b43fff700 1 --2- 172.21.15.40:0/2903021369 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5b38064f10 0x7f5b380673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:09.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.671+0000 7f5b43fff700 1 --2- 172.21.15.40:0/2903021369 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5b38064f10 0x7f5b380673c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c06d1e0 tx=0x7f5b3c009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:09.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.675+0000 7f5b417fa700 1 -- 172.21.15.40:0/2903021369 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5b480516e0 con 0x7f5b4c0b1b80 2023-11-08T08:19:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.855+0000 7f5b51a6d700 1 -- 172.21.15.40:0/2903021369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b4c0c9680 con 0x7f5b4c0b1b80 2023-11-08T08:19:09.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.859+0000 7f5b417fa700 1 -- 172.21.15.40:0/2903021369 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19652 (secure 0 0 0) 0x7f5b48054d00 con 0x7f5b4c0b1b80 2023-11-08T08:19:09.865 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:09.865 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":28,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:08.980158+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:19:09.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.859+0000 7f5b51a6d700 1 -- 172.21.15.40:0/2903021369 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5b38064f10 msgr2=0x7f5b380673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:09.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.859+0000 7f5b51a6d700 1 --2- 172.21.15.40:0/2903021369 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5b38064f10 0x7f5b380673c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c06d1e0 tx=0x7f5b3c009240 comp rx=0 tx=0).stop 2023-11-08T08:19:09.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.859+0000 7f5b51a6d700 1 -- 172.21.15.40:0/2903021369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b4c0b1b80 msgr2=0x7f5b4c157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:09.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.859+0000 7f5b51a6d700 1 --2- 172.21.15.40:0/2903021369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b4c0b1b80 0x7f5b4c157b80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800b370 tx=0x7f5b48004af0 comp rx=0 tx=0).stop 2023-11-08T08:19:09.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.863+0000 7f5b51a6d700 1 -- 172.21.15.40:0/2903021369 shutdown_connections 2023-11-08T08:19:09.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.863+0000 7f5b51a6d700 1 --2- 172.21.15.40:0/2903021369 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5b38064f10 0x7f5b380673c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:09.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.863+0000 7f5b51a6d700 1 --2- 172.21.15.40:0/2903021369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b4c15c650 0x7f5b4c149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:09.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.863+0000 7f5b51a6d700 1 --2- 172.21.15.40:0/2903021369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b4c0cd0d0 0x7f5b4c070500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:09.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.863+0000 7f5b51a6d700 1 --2- 172.21.15.40:0/2903021369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b4c0b1b80 0x7f5b4c157b80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:09.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.863+0000 7f5b51a6d700 1 -- 172.21.15.40:0/2903021369 >> 172.21.15.40:0/2903021369 conn(0x7f5b4c0b0f80 msgr2=0x7f5b4c0cc960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:09.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.863+0000 7f5b51a6d700 1 -- 172.21.15.40:0/2903021369 shutdown_connections 2023-11-08T08:19:09.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:09.863+0000 7f5b51a6d700 1 -- 172.21.15.40:0/2903021369 wait complete. 2023-11-08T08:19:09.891 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:19:09.964 INFO:tasks.ceph.mds.d.smithi169.stderr:2023-11-08T08:19:09.960+0000 7fabd785f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T08:19:10.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.059+0000 7feb748ce700 1 Processor -- start 2023-11-08T08:19:10.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.059+0000 7feb748ce700 1 -- start start 2023-11-08T08:19:10.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb748ce700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7feb700b33d0 0x7feb700ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:10.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb748ce700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feb7006cc30 0x7feb7006ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:10.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb748ce700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb7006bf00 con 0x7feb7006a600 2023-11-08T08:19:10.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb748ce700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb7006c7a0 con 0x7feb7006cc30 2023-11-08T08:19:10.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb748ce700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb7006aa80 con 0x7feb700b33d0 2023-11-08T08:19:10.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6e59c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7feb7006a600 0x7feb700b37a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51452/0 (socket says 172.21.15.40:51452) 2023-11-08T08:19:10.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6e59c700 1 -- 172.21.15.40:0/715351715 learned_addr learned my addr 172.21.15.40:0/715351715 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:10.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6f59e700 1 --2- 172.21.15.40:0/715351715 >> v2:172.21.15.40:3300/0 conn(0x7feb7006cc30 0x7feb7006ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:10.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6ed9d700 1 --2- 172.21.15.40:0/715351715 >> v2:172.21.15.169:3300/0 conn(0x7feb700b33d0 0x7feb700ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:10.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6dd9b700 1 -- 172.21.15.40:0/715351715 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 650027469 0 0) 0x7feb7006bf00 con 0x7feb7006a600 2023-11-08T08:19:10.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6dd9b700 1 -- 172.21.15.40:0/715351715 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb580035d0 con 0x7feb7006a600 2023-11-08T08:19:10.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6dd9b700 1 -- 172.21.15.40:0/715351715 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4134929580 0 0) 0x7feb580035d0 con 0x7feb7006a600 2023-11-08T08:19:10.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6dd9b700 1 -- 172.21.15.40:0/715351715 >> v2:172.21.15.169:3300/0 conn(0x7feb700b33d0 msgr2=0x7feb700ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:10.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6dd9b700 1 --2- 172.21.15.40:0/715351715 >> v2:172.21.15.169:3300/0 conn(0x7feb700b33d0 0x7feb700ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6dd9b700 1 -- 172.21.15.40:0/715351715 >> v2:172.21.15.40:3300/0 conn(0x7feb7006cc30 msgr2=0x7feb7006ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:10.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6dd9b700 1 --2- 172.21.15.40:0/715351715 >> v2:172.21.15.40:3300/0 conn(0x7feb7006cc30 0x7feb7006ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6dd9b700 1 -- 172.21.15.40:0/715351715 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7006ad00 con 0x7feb7006a600 2023-11-08T08:19:10.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6dd9b700 1 -- 172.21.15.40:0/715351715 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7feb64001670 con 0x7feb7006a600 2023-11-08T08:19:10.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6dd9b700 1 -- 172.21.15.40:0/715351715 >> v1:172.21.15.40:6789/0 conn(0x7feb7006a600 legacy=0x7feb700b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:10.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6dd9b700 1 --2- 172.21.15.40:0/715351715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb58003a70 0x7feb58003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:10.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6dd9b700 1 --2- 172.21.15.40:0/715351715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb58004870 0x7feb58004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:10.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6dd9b700 1 --2- 172.21.15.40:0/715351715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb58005220 0x7feb5800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:10.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6dd9b700 1 -- 172.21.15.40:0/715351715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7006c7a0 con 0x7feb58003a70 2023-11-08T08:19:10.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6dd9b700 1 -- 172.21.15.40:0/715351715 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7006aa80 con 0x7feb58004870 2023-11-08T08:19:10.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6dd9b700 1 -- 172.21.15.40:0/715351715 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5800c070 con 0x7feb58005220 2023-11-08T08:19:10.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6f59e700 1 --2- 172.21.15.40:0/715351715 >> v2:172.21.15.40:3300/0 conn(0x7feb7006cc30 0x7feb7006ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:10.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.063+0000 7feb6ed9d700 1 --2- 172.21.15.40:0/715351715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb58003a70 0x7feb58003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:10.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb6f59e700 1 --2- 172.21.15.40:0/715351715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb58005220 0x7feb5800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:10.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb6e59c700 1 --2- 172.21.15.40:0/715351715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb58004870 0x7feb58004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:10.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb6ed9d700 1 -- 172.21.15.40:0/715351715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb58005220 msgr2=0x7feb5800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:10.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb6ed9d700 1 --2- 172.21.15.40:0/715351715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb58005220 0x7feb5800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb6ed9d700 1 -- 172.21.15.40:0/715351715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb58004870 msgr2=0x7feb58004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:10.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb6ed9d700 1 --2- 172.21.15.40:0/715351715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb58004870 0x7feb58004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb6ed9d700 1 -- 172.21.15.40:0/715351715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb580035d0 con 0x7feb58003a70 2023-11-08T08:19:10.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb6f59e700 1 --2- 172.21.15.40:0/715351715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb58005220 0x7feb5800ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:10.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb6e59c700 1 --2- 172.21.15.40:0/715351715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb58004870 0x7feb58004ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:10.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb6f59e700 1 -- 172.21.15.40:0/715351715 reap_dead start 2023-11-08T08:19:10.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb6ed9d700 1 --2- 172.21.15.40:0/715351715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb58003a70 0x7feb58003e80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feb6000e170 tx=0x7feb580058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5927c4db1f7441c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:10.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb6dd9b700 1 -- 172.21.15.40:0/715351715 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6000ec90 con 0x7feb58003a70 2023-11-08T08:19:10.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb6dd9b700 1 -- 172.21.15.40:0/715351715 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6000f040 con 0x7feb58003a70 2023-11-08T08:19:10.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb6dd9b700 1 -- 172.21.15.40:0/715351715 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb60014480 con 0x7feb58003a70 2023-11-08T08:19:10.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb748ce700 1 -- 172.21.15.40:0/715351715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb58003a70 msgr2=0x7feb58003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:10.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb748ce700 1 --2- 172.21.15.40:0/715351715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb58003a70 0x7feb58003e80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feb6000e170 tx=0x7feb580058a0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb748ce700 1 -- 172.21.15.40:0/715351715 shutdown_connections 2023-11-08T08:19:10.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb748ce700 1 --2- 172.21.15.40:0/715351715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb58003a70 0x7feb58003e80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb748ce700 1 -- 172.21.15.40:0/715351715 >> 172.21.15.40:0/715351715 conn(0x7feb700ce510 msgr2=0x7feb700ce910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:10.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb748ce700 1 -- 172.21.15.40:0/715351715 shutdown_connections 2023-11-08T08:19:10.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.067+0000 7feb748ce700 1 -- 172.21.15.40:0/715351715 wait complete. 2023-11-08T08:19:10.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.071+0000 7feb748ce700 1 Processor -- start 2023-11-08T08:19:10.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.071+0000 7feb748ce700 1 -- start start 2023-11-08T08:19:10.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.071+0000 7feb748ce700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb58003a70 0x7feb70097a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:10.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.071+0000 7feb748ce700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb700c5650 0x7feb700c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:10.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.071+0000 7feb748ce700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb70152370 0x7feb701527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:10.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.071+0000 7feb748ce700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb70152d20 con 0x7feb58003a70 2023-11-08T08:19:10.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.071+0000 7feb748ce700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb700cc890 con 0x7feb700c5650 2023-11-08T08:19:10.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.071+0000 7feb748ce700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb700c8330 con 0x7feb70152370 2023-11-08T08:19:10.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.075+0000 7feb6ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb58003a70 0x7feb70097a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:10.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.075+0000 7feb6ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb58003a70 0x7feb70097a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43334/0 (socket says 172.21.15.40:43334) 2023-11-08T08:19:10.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.075+0000 7feb6ed9d700 1 -- 172.21.15.40:0/3257453682 learned_addr learned my addr 172.21.15.40:0/3257453682 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:10.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.075+0000 7feb6f59e700 1 --2- 172.21.15.40:0/3257453682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb70152370 0x7feb701527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:10.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.075+0000 7feb6e59c700 1 --2- 172.21.15.40:0/3257453682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb700c5650 0x7feb700c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:10.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.075+0000 7feb6ed9d700 1 -- 172.21.15.40:0/3257453682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb70152370 msgr2=0x7feb701527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:10.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.075+0000 7feb6ed9d700 1 --2- 172.21.15.40:0/3257453682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb70152370 0x7feb701527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.075+0000 7feb6ed9d700 1 -- 172.21.15.40:0/3257453682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb700c5650 msgr2=0x7feb700c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:10.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.075+0000 7feb6ed9d700 1 --2- 172.21.15.40:0/3257453682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb700c5650 0x7feb700c7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.075+0000 7feb6ed9d700 1 -- 172.21.15.40:0/3257453682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7006c430 con 0x7feb58003a70 2023-11-08T08:19:10.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.075+0000 7feb6f59e700 1 --2- 172.21.15.40:0/3257453682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb70152370 0x7feb701527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:10.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.075+0000 7feb6e59c700 1 --2- 172.21.15.40:0/3257453682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb700c5650 0x7feb700c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:10.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.075+0000 7feb6ed9d700 1 --2- 172.21.15.40:0/3257453682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb58003a70 0x7feb70097a30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feb6000e760 tx=0x7feb60009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:10.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.075+0000 7feb577fe700 1 -- 172.21.15.40:0/3257453682 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb60007000 con 0x7feb58003a70 2023-11-08T08:19:10.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.075+0000 7feb577fe700 1 -- 172.21.15.40:0/3257453682 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb60003c40 con 0x7feb58003a70 2023-11-08T08:19:10.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.075+0000 7feb577fe700 1 -- 172.21.15.40:0/3257453682 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6000eec0 con 0x7feb58003a70 2023-11-08T08:19:10.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.075+0000 7feb748ce700 1 -- 172.21.15.40:0/3257453682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7006c150 con 0x7feb58003a70 2023-11-08T08:19:10.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.075+0000 7feb748ce700 1 -- 172.21.15.40:0/3257453682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7015d840 con 0x7feb58003a70 2023-11-08T08:19:10.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.075+0000 7feb748ce700 1 -- 172.21.15.40:0/3257453682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb70073cb0 con 0x7feb58003a70 2023-11-08T08:19:10.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.079+0000 7feb577fe700 1 -- 172.21.15.40:0/3257453682 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7feb60014a40 con 0x7feb58003a70 2023-11-08T08:19:10.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.079+0000 7feb577fe700 1 --2- 172.21.15.40:0/3257453682 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feb5c064f10 0x7feb5c0673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:10.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.079+0000 7feb577fe700 1 -- 172.21.15.40:0/3257453682 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7feb60083d70 con 0x7feb58003a70 2023-11-08T08:19:10.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.079+0000 7feb6e59c700 1 --2- 172.21.15.40:0/3257453682 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feb5c064f10 0x7feb5c0673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:10.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.083+0000 7feb6e59c700 1 --2- 172.21.15.40:0/3257453682 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feb5c064f10 0x7feb5c0673c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feb700b45c0 tx=0x7feb64009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:10.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.083+0000 7feb577fe700 1 -- 172.21.15.40:0/3257453682 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7feb60051b30 con 0x7feb58003a70 2023-11-08T08:19:10.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.267+0000 7feb748ce700 1 -- 172.21.15.40:0/3257453682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb70072280 con 0x7feb58003a70 2023-11-08T08:19:10.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.267+0000 7feb577fe700 1 -- 172.21.15.40:0/3257453682 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19652 (secure 0 0 0) 0x7feb60018090 con 0x7feb58003a70 2023-11-08T08:19:10.275 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:10.275 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":28,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:08.980158+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:19:10.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.271+0000 7feb748ce700 1 -- 172.21.15.40:0/3257453682 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feb5c064f10 msgr2=0x7feb5c0673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:10.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.271+0000 7feb748ce700 1 --2- 172.21.15.40:0/3257453682 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feb5c064f10 0x7feb5c0673c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feb700b45c0 tx=0x7feb64009040 comp rx=0 tx=0).stop 2023-11-08T08:19:10.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.271+0000 7feb748ce700 1 -- 172.21.15.40:0/3257453682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb58003a70 msgr2=0x7feb70097a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:10.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.271+0000 7feb748ce700 1 --2- 172.21.15.40:0/3257453682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb58003a70 0x7feb70097a30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feb6000e760 tx=0x7feb60009750 comp rx=0 tx=0).stop 2023-11-08T08:19:10.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.271+0000 7feb748ce700 1 -- 172.21.15.40:0/3257453682 shutdown_connections 2023-11-08T08:19:10.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.271+0000 7feb748ce700 1 --2- 172.21.15.40:0/3257453682 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feb5c064f10 0x7feb5c0673c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.271+0000 7feb748ce700 1 --2- 172.21.15.40:0/3257453682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb70152370 0x7feb701527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.271+0000 7feb748ce700 1 --2- 172.21.15.40:0/3257453682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb700c5650 0x7feb700c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.271+0000 7feb748ce700 1 --2- 172.21.15.40:0/3257453682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb58003a70 0x7feb70097a30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.271+0000 7feb748ce700 1 -- 172.21.15.40:0/3257453682 >> 172.21.15.40:0/3257453682 conn(0x7feb700ce510 msgr2=0x7feb7006d090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:10.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.271+0000 7feb748ce700 1 -- 172.21.15.40:0/3257453682 shutdown_connections 2023-11-08T08:19:10.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.271+0000 7feb748ce700 1 -- 172.21.15.40:0/3257453682 wait complete. 2023-11-08T08:19:10.312 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:19:10.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.495+0000 7fda01301700 1 Processor -- start 2023-11-08T08:19:10.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.495+0000 7fda01301700 1 -- start start 2023-11-08T08:19:10.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.495+0000 7fda01301700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9fc15a1a0 0x7fd9fc0b5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:10.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.495+0000 7fda01301700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd9fc0b5dc0 0x7fd9fc0b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:10.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.495+0000 7fda01301700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd9fc0b3f50 0x7fd9fc06f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:10.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.495+0000 7fda01301700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9fc15e990 con 0x7fd9fc0b3f50 2023-11-08T08:19:10.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.495+0000 7fda01301700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9fc0b43f0 con 0x7fd9fc0b5dc0 2023-11-08T08:19:10.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.495+0000 7fda01301700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc15dd10 con 0x7fd9fc15a1a0 2023-11-08T08:19:10.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.495+0000 7fd9fbfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd9fc0b3f50 0x7fd9fc06f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:10.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.495+0000 7fd9fbfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd9fc0b3f50 0x7fd9fc06f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43340/0 (socket says 172.21.15.40:43340) 2023-11-08T08:19:10.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.495+0000 7fd9fbfff700 1 -- 172.21.15.40:0/868340598 learned_addr learned my addr 172.21.15.40:0/868340598 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:10.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.495+0000 7fd9fa7fc700 1 --2- 172.21.15.40:0/868340598 >> v2:172.21.15.169:3300/0 conn(0x7fd9fc0b5dc0 0x7fd9fc0b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:10.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.495+0000 7fd9faffd700 1 --2- 172.21.15.40:0/868340598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9fc15a1a0 0x7fd9fc0b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:10.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.495+0000 7fd9fbfff700 1 -- 172.21.15.40:0/868340598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9fc15a1a0 msgr2=0x7fd9fc0b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:10.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.495+0000 7fd9fbfff700 1 --2- 172.21.15.40:0/868340598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9fc15a1a0 0x7fd9fc0b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.495+0000 7fd9fbfff700 1 -- 172.21.15.40:0/868340598 >> v2:172.21.15.169:3300/0 conn(0x7fd9fc0b5dc0 msgr2=0x7fd9fc0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:10.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.495+0000 7fd9fbfff700 1 --2- 172.21.15.40:0/868340598 >> v2:172.21.15.169:3300/0 conn(0x7fd9fc0b5dc0 0x7fd9fc0b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.495+0000 7fd9fbfff700 1 -- 172.21.15.40:0/868340598 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc15dff0 con 0x7fd9fc0b3f50 2023-11-08T08:19:10.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.499+0000 7fd9faffd700 1 --2- 172.21.15.40:0/868340598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9fc15a1a0 0x7fd9fc0b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:10.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.499+0000 7fd9fa7fc700 1 --2- 172.21.15.40:0/868340598 >> v2:172.21.15.169:3300/0 conn(0x7fd9fc0b5dc0 0x7fd9fc0b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:10.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.499+0000 7fd9fbfff700 1 --2- 172.21.15.40:0/868340598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9fc0b3f50 0x7fd9fc06f320 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0d0010 tx=0x7fd9f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de22a73df1a01d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:10.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.499+0000 7fd9f9ffb700 1 -- 172.21.15.40:0/868340598 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f400e070 con 0x7fd9fc0b3f50 2023-11-08T08:19:10.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.499+0000 7fd9f9ffb700 1 -- 172.21.15.40:0/868340598 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f400b9c0 con 0x7fd9fc0b3f50 2023-11-08T08:19:10.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.499+0000 7fd9f9ffb700 1 -- 172.21.15.40:0/868340598 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f400bd70 con 0x7fd9fc0b3f50 2023-11-08T08:19:10.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.499+0000 7fda01301700 1 -- 172.21.15.40:0/868340598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9fc0b3f50 msgr2=0x7fd9fc06f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:10.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.499+0000 7fda01301700 1 --2- 172.21.15.40:0/868340598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9fc0b3f50 0x7fd9fc06f320 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0d0010 tx=0x7fd9f400b330 comp rx=0 tx=0).stop 2023-11-08T08:19:10.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.499+0000 7fda01301700 1 -- 172.21.15.40:0/868340598 shutdown_connections 2023-11-08T08:19:10.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.499+0000 7fda01301700 1 --2- 172.21.15.40:0/868340598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9fc0b3f50 0x7fd9fc06f320 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.499+0000 7fda01301700 1 --2- 172.21.15.40:0/868340598 >> v2:172.21.15.169:3300/0 conn(0x7fd9fc0b5dc0 0x7fd9fc0b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.499+0000 7fda01301700 1 --2- 172.21.15.40:0/868340598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9fc15a1a0 0x7fd9fc0b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.499+0000 7fda01301700 1 -- 172.21.15.40:0/868340598 >> 172.21.15.40:0/868340598 conn(0x7fd9fc0ccb30 msgr2=0x7fd9fc0ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:10.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.499+0000 7fda01301700 1 -- 172.21.15.40:0/868340598 shutdown_connections 2023-11-08T08:19:10.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.499+0000 7fda01301700 1 -- 172.21.15.40:0/868340598 wait complete. 2023-11-08T08:19:10.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.503+0000 7fda01301700 1 Processor -- start 2023-11-08T08:19:10.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.503+0000 7fda01301700 1 -- start start 2023-11-08T08:19:10.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.503+0000 7fda01301700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9fc0b3f50 0x7fd9fc158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:10.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.503+0000 7fda01301700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9fc0b5dc0 0x7fd9fc15c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:10.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.503+0000 7fda01301700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9fc15a1a0 0x7fd9fc15cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:10.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.503+0000 7fda01301700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc15c070 con 0x7fd9fc0b5dc0 2023-11-08T08:19:10.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.503+0000 7fda01301700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc0c4690 con 0x7fd9fc0b3f50 2023-11-08T08:19:10.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.503+0000 7fda01301700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc15a8a0 con 0x7fd9fc15a1a0 2023-11-08T08:19:10.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.503+0000 7fd9fa7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9fc0b5dc0 0x7fd9fc15c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:10.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.503+0000 7fd9fa7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9fc0b5dc0 0x7fd9fc15c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43354/0 (socket says 172.21.15.40:43354) 2023-11-08T08:19:10.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.503+0000 7fd9fa7fc700 1 -- 172.21.15.40:0/3277854460 learned_addr learned my addr 172.21.15.40:0/3277854460 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:10.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.503+0000 7fd9faffd700 1 --2- 172.21.15.40:0/3277854460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9fc0b3f50 0x7fd9fc158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:10.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.503+0000 7fd9fbfff700 1 --2- 172.21.15.40:0/3277854460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9fc15a1a0 0x7fd9fc15cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:10.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.503+0000 7fd9fa7fc700 1 -- 172.21.15.40:0/3277854460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9fc15a1a0 msgr2=0x7fd9fc15cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:10.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.503+0000 7fd9fa7fc700 1 --2- 172.21.15.40:0/3277854460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9fc15a1a0 0x7fd9fc15cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.503+0000 7fd9fa7fc700 1 -- 172.21.15.40:0/3277854460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9fc0b3f50 msgr2=0x7fd9fc158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:10.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.503+0000 7fd9fa7fc700 1 --2- 172.21.15.40:0/3277854460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9fc0b3f50 0x7fd9fc158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.503+0000 7fd9fa7fc700 1 -- 172.21.15.40:0/3277854460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f4009b30 con 0x7fd9fc0b5dc0 2023-11-08T08:19:10.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.503+0000 7fd9fa7fc700 1 --2- 172.21.15.40:0/3277854460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9fc0b5dc0 0x7fd9fc15c740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000c820 tx=0x7fd9f000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:10.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.507+0000 7fd9e37fe700 1 -- 172.21.15.40:0/3277854460 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0011800 con 0x7fd9fc0b5dc0 2023-11-08T08:19:10.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.507+0000 7fd9e37fe700 1 -- 172.21.15.40:0/3277854460 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f0011960 con 0x7fd9fc0b5dc0 2023-11-08T08:19:10.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.507+0000 7fd9e37fe700 1 -- 172.21.15.40:0/3277854460 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0011d10 con 0x7fd9fc0b5dc0 2023-11-08T08:19:10.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.507+0000 7fda01301700 1 -- 172.21.15.40:0/3277854460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9fc15e5b0 con 0x7fd9fc0b5dc0 2023-11-08T08:19:10.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.507+0000 7fda01301700 1 -- 172.21.15.40:0/3277854460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9fc15e2d0 con 0x7fd9fc0b5dc0 2023-11-08T08:19:10.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.507+0000 7fda01301700 1 -- 172.21.15.40:0/3277854460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9fc069bf0 con 0x7fd9fc0b5dc0 2023-11-08T08:19:10.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.507+0000 7fd9e37fe700 1 -- 172.21.15.40:0/3277854460 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd9f0021070 con 0x7fd9fc0b5dc0 2023-11-08T08:19:10.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.511+0000 7fd9e37fe700 1 --2- 172.21.15.40:0/3277854460 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd9e8064ee0 0x7fd9e8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:10.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.511+0000 7fd9e37fe700 1 -- 172.21.15.40:0/3277854460 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd9f0081fa0 con 0x7fd9fc0b5dc0 2023-11-08T08:19:10.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.511+0000 7fd9faffd700 1 --2- 172.21.15.40:0/3277854460 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd9e8064ee0 0x7fd9e8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:10.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.511+0000 7fd9faffd700 1 --2- 172.21.15.40:0/3277854460 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd9e8064ee0 0x7fd9e8067390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00ab60 tx=0x7fd9ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:10.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.515+0000 7fd9e37fe700 1 -- 172.21.15.40:0/3277854460 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd9f004fcd0 con 0x7fd9fc0b5dc0 2023-11-08T08:19:10.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.723+0000 7fda01301700 1 -- 172.21.15.40:0/3277854460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9fc0726c0 con 0x7fd9fc0b5dc0 2023-11-08T08:19:10.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.723+0000 7fd9e37fe700 1 -- 172.21.15.40:0/3277854460 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4641 (secure 0 0 0) 0x7fd9f001f020 con 0x7fd9fc0b5dc0 2023-11-08T08:19:10.731 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:10.731 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/196211087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":196211087},{"type":"v1","addr":"172.21.15.40:6834","nonce":196211087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/67127106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":67127106},{"type":"v1","addr":"172.21.15.40:6835","nonce":67127106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1231240563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1231240563},{"type":"v1","addr":"172.21.15.169:6835","nonce":1231240563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:19:04.456508+0000","modified":"2023-11-08T08:19:08.980644+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6837/1989495204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1989495204},{"type":"v1","addr":"172.21.15.169:6837","nonce":1989495204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-08T08:19:10.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.727+0000 7fda01301700 1 -- 172.21.15.40:0/3277854460 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd9e8064ee0 msgr2=0x7fd9e8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:10.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.727+0000 7fda01301700 1 --2- 172.21.15.40:0/3277854460 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd9e8064ee0 0x7fd9e8067390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00ab60 tx=0x7fd9ec008040 comp rx=0 tx=0).stop 2023-11-08T08:19:10.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.727+0000 7fda01301700 1 -- 172.21.15.40:0/3277854460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9fc0b5dc0 msgr2=0x7fd9fc15c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:10.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.727+0000 7fda01301700 1 --2- 172.21.15.40:0/3277854460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9fc0b5dc0 0x7fd9fc15c740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000c820 tx=0x7fd9f000cb30 comp rx=0 tx=0).stop 2023-11-08T08:19:10.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.727+0000 7fda01301700 1 -- 172.21.15.40:0/3277854460 shutdown_connections 2023-11-08T08:19:10.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.727+0000 7fda01301700 1 --2- 172.21.15.40:0/3277854460 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd9e8064ee0 0x7fd9e8067390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.727+0000 7fda01301700 1 --2- 172.21.15.40:0/3277854460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9fc15a1a0 0x7fd9fc15cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.727+0000 7fda01301700 1 --2- 172.21.15.40:0/3277854460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9fc0b5dc0 0x7fd9fc15c740 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.727+0000 7fda01301700 1 --2- 172.21.15.40:0/3277854460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9fc0b3f50 0x7fd9fc158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:10.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.727+0000 7fda01301700 1 -- 172.21.15.40:0/3277854460 >> 172.21.15.40:0/3277854460 conn(0x7fd9fc0ccb30 msgr2=0x7fd9fc0cf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:10.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.727+0000 7fda01301700 1 -- 172.21.15.40:0/3277854460 shutdown_connections 2023-11-08T08:19:10.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:10.727+0000 7fda01301700 1 -- 172.21.15.40:0/3277854460 wait complete. 2023-11-08T08:19:10.737 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2023-11-08T08:19:11.156 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T08:19:04.456508+0000', 'modified': '2023-11-08T08:19:08.980644+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4291}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4291': {'gid': 4291, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6837/1989495204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6836', 'nonce': 1989495204}, {'type': 'v1', 'addr': '172.21.15.169:6837', 'nonce': 1989495204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-11-08T08:19:11.157 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T08:19:11.157 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-08T08:19:11.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.335+0000 7f1ec791e700 1 Processor -- start 2023-11-08T08:19:11.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.339+0000 7f1ec791e700 1 -- start start 2023-11-08T08:19:11.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.339+0000 7f1ec791e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00cd800 0x7f1ec0161980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.339+0000 7f1ec791e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ec00cdbd0 0x7f1ec00ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.339+0000 7f1ec791e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1ec00c38b0 0x7f1ec00c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.339+0000 7f1ec791e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec00c4290 con 0x7f1ec00cdbd0 2023-11-08T08:19:11.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.339+0000 7f1ec791e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec0164d10 con 0x7f1ec00c38b0 2023-11-08T08:19:11.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.339+0000 7f1ec791e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec006cd10 con 0x7f1ec00cd800 2023-11-08T08:19:11.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.339+0000 7f1ec4eb9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ec00cdbd0 0x7f1ec00ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.339+0000 7f1ec4eb9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ec00cdbd0 0x7f1ec00ce5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43368/0 (socket says 172.21.15.40:43368) 2023-11-08T08:19:11.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.339+0000 7f1ec4eb9700 1 -- 172.21.15.40:0/3625684404 learned_addr learned my addr 172.21.15.40:0/3625684404 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:11.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1ec56ba700 1 --2- 172.21.15.40:0/3625684404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00cd800 0x7f1ec0161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1ec5ebb700 1 --2- 172.21.15.40:0/3625684404 >> v2:172.21.15.169:3300/0 conn(0x7f1ec00c38b0 0x7f1ec00c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1ec4eb9700 1 -- 172.21.15.40:0/3625684404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00cd800 msgr2=0x7f1ec0161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:11.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1ec4eb9700 1 --2- 172.21.15.40:0/3625684404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00cd800 0x7f1ec0161980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1ec4eb9700 1 -- 172.21.15.40:0/3625684404 >> v2:172.21.15.169:3300/0 conn(0x7f1ec00c38b0 msgr2=0x7f1ec00c3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:11.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1ec4eb9700 1 --2- 172.21.15.40:0/3625684404 >> v2:172.21.15.169:3300/0 conn(0x7f1ec00c38b0 0x7f1ec00c3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1ec4eb9700 1 -- 172.21.15.40:0/3625684404 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec01643e0 con 0x7f1ec00cdbd0 2023-11-08T08:19:11.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1ec56ba700 1 --2- 172.21.15.40:0/3625684404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00cd800 0x7f1ec0161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:11.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1ec5ebb700 1 --2- 172.21.15.40:0/3625684404 >> v2:172.21.15.169:3300/0 conn(0x7f1ec00c38b0 0x7f1ec00c3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:11.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1ec4eb9700 1 --2- 172.21.15.40:0/3625684404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec00cdbd0 0x7f1ec00ce5e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0072260 tx=0x7f1eb0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d35de270246f52 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:11.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1eb7fff700 1 -- 172.21.15.40:0/3625684404 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000b8a0 con 0x7f1ec00cdbd0 2023-11-08T08:19:11.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1eb7fff700 1 -- 172.21.15.40:0/3625684404 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb000ba00 con 0x7f1ec00cdbd0 2023-11-08T08:19:11.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1eb7fff700 1 -- 172.21.15.40:0/3625684404 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000bd90 con 0x7f1ec00cdbd0 2023-11-08T08:19:11.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1ec791e700 1 -- 172.21.15.40:0/3625684404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec00cdbd0 msgr2=0x7f1ec00ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:11.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1ec791e700 1 --2- 172.21.15.40:0/3625684404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec00cdbd0 0x7f1ec00ce5e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0072260 tx=0x7f1eb0009bf0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1ec791e700 1 -- 172.21.15.40:0/3625684404 shutdown_connections 2023-11-08T08:19:11.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1ec791e700 1 --2- 172.21.15.40:0/3625684404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec00cdbd0 0x7f1ec00ce5e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1ec791e700 1 --2- 172.21.15.40:0/3625684404 >> v2:172.21.15.169:3300/0 conn(0x7f1ec00c38b0 0x7f1ec00c3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1ec791e700 1 --2- 172.21.15.40:0/3625684404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00cd800 0x7f1ec0161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1ec791e700 1 -- 172.21.15.40:0/3625684404 >> 172.21.15.40:0/3625684404 conn(0x7f1ec006e5d0 msgr2=0x7f1ec006e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:11.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1ec791e700 1 -- 172.21.15.40:0/3625684404 shutdown_connections 2023-11-08T08:19:11.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1ec791e700 1 -- 172.21.15.40:0/3625684404 wait complete. 2023-11-08T08:19:11.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.343+0000 7f1ec791e700 1 Processor -- start 2023-11-08T08:19:11.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1ec791e700 1 -- start start 2023-11-08T08:19:11.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1ec791e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ec00c38b0 0x7f1ec0157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1ec791e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec00cd800 0x7f1ec01ec0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1ec791e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00cdbd0 0x7f1ec0162360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1ec791e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec01ec5e0 con 0x7f1ec00cd800 2023-11-08T08:19:11.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1ec791e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec01eabb0 con 0x7f1ec00c38b0 2023-11-08T08:19:11.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1ec791e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec01eae90 con 0x7f1ec00cdbd0 2023-11-08T08:19:11.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1ec4eb9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec00cd800 0x7f1ec01ec0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1ec4eb9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec00cd800 0x7f1ec01ec0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43382/0 (socket says 172.21.15.40:43382) 2023-11-08T08:19:11.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1ec4eb9700 1 -- 172.21.15.40:0/369886585 learned_addr learned my addr 172.21.15.40:0/369886585 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:11.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1ec4eb9700 1 -- 172.21.15.40:0/369886585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00cdbd0 msgr2=0x7f1ec0162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:11.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1ec4eb9700 1 --2- 172.21.15.40:0/369886585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00cdbd0 0x7f1ec0162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1ec4eb9700 1 -- 172.21.15.40:0/369886585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ec00c38b0 msgr2=0x7f1ec0157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:11.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1ec4eb9700 1 --2- 172.21.15.40:0/369886585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ec00c38b0 0x7f1ec0157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1ec4eb9700 1 -- 172.21.15.40:0/369886585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb0009570 con 0x7f1ec00cd800 2023-11-08T08:19:11.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1ec4eb9700 1 --2- 172.21.15.40:0/369886585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec00cd800 0x7f1ec01ec0a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0000c00 tx=0x7f1eb0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:11.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1eb5ffb700 1 -- 172.21.15.40:0/369886585 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0003ad0 con 0x7f1ec00cd800 2023-11-08T08:19:11.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1eb5ffb700 1 -- 172.21.15.40:0/369886585 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb001e070 con 0x7f1ec00cd800 2023-11-08T08:19:11.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1eb5ffb700 1 -- 172.21.15.40:0/369886585 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0003e90 con 0x7f1ec00cd800 2023-11-08T08:19:11.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1ec791e700 1 -- 172.21.15.40:0/369886585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec01649a0 con 0x7f1ec00cd800 2023-11-08T08:19:11.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.347+0000 7f1ec791e700 1 -- 172.21.15.40:0/369886585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec01646c0 con 0x7f1ec00cd800 2023-11-08T08:19:11.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.351+0000 7f1ecaba5740 1 Processor -- start 2023-11-08T08:19:11.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.351+0000 7f1eb5ffb700 1 -- 172.21.15.40:0/369886585 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1eb000f050 con 0x7f1ec00cd800 2023-11-08T08:19:11.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ecaba5740 1 -- start start 2023-11-08T08:19:11.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1eb5ffb700 1 --2- 172.21.15.40:0/369886585 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1ea4064ee0 0x7f1ea4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1eb5ffb700 1 -- 172.21.15.40:0/369886585 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1eb0020070 con 0x7f1ec00cd800 2023-11-08T08:19:11.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ecaba5740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251eae0 0x27cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ecaba5740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x251e710 0x27cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ecaba5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2569790 0x27cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ecaba5740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x268dae0 con 0x251eae0 2023-11-08T08:19:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ecaba5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x26fdbd0 con 0x2569790 2023-11-08T08:19:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ecaba5740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x26f6ed0 con 0x251e710 2023-11-08T08:19:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ec56ba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251eae0 0x27cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ec5ebb700 1 --2- 172.21.15.40:0/369886585 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1ea4064ee0 0x7f1ea4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ec56ba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251eae0 0x27cee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43388/0 (socket says 172.21.15.40:43388) 2023-11-08T08:19:11.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ec56ba700 1 -- 172.21.15.40:0/1395887333 learned_addr learned my addr 172.21.15.40:0/1395887333 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:11.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ec4eb9700 1 --2- 172.21.15.40:0/1395887333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x251e710 0x27cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ec5ebb700 1 --2- 172.21.15.40:0/1395887333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2569790 0x27cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ec56ba700 1 -- 172.21.15.40:0/1395887333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x251e710 msgr2=0x27cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ec56ba700 1 --2- 172.21.15.40:0/1395887333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x251e710 0x27cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ec56ba700 1 -- 172.21.15.40:0/1395887333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2569790 msgr2=0x27cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ec56ba700 1 --2- 172.21.15.40:0/1395887333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2569790 0x27cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ec56ba700 1 -- 172.21.15.40:0/1395887333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x271ccc0 con 0x251eae0 2023-11-08T08:19:11.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ec56ba700 1 --2- 172.21.15.40:0/1395887333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251eae0 0x27cee00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00aa20 tx=0x7f1ebc00ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d450ecb9488ac465 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ec5ebb700 1 --2- 172.21.15.40:0/369886585 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1ea4064ee0 0x7f1ea4067390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009fd0 tx=0x7f1eb80094e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ea2ffd700 1 -- 172.21.15.40:0/1395887333 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc00ca50 con 0x251eae0 2023-11-08T08:19:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.355+0000 7f1ea2ffd700 1 -- 172.21.15.40:0/1395887333 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ebc00cbb0 con 0x251eae0 2023-11-08T08:19:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ea2ffd700 1 -- 172.21.15.40:0/1395887333 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0113f0 con 0x251eae0 2023-11-08T08:19:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ecaba5740 1 -- 172.21.15.40:0/1395887333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251eae0 msgr2=0x27cee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:11.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ecaba5740 1 --2- 172.21.15.40:0/1395887333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251eae0 0x27cee00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00aa20 tx=0x7f1ebc00ad30 comp rx=0 tx=0).stop 2023-11-08T08:19:11.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ecaba5740 1 -- 172.21.15.40:0/1395887333 shutdown_connections 2023-11-08T08:19:11.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ecaba5740 1 --2- 172.21.15.40:0/1395887333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2569790 0x27cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ecaba5740 1 --2- 172.21.15.40:0/1395887333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x251e710 0x27cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ecaba5740 1 --2- 172.21.15.40:0/1395887333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251eae0 0x27cee00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ecaba5740 1 -- 172.21.15.40:0/1395887333 >> 172.21.15.40:0/1395887333 conn(0x251eeb0 msgr2=0x27b7700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:11.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ecaba5740 1 -- 172.21.15.40:0/1395887333 shutdown_connections 2023-11-08T08:19:11.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ecaba5740 1 -- 172.21.15.40:0/1395887333 wait complete. 2023-11-08T08:19:11.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ecaba5740 1 -- start start 2023-11-08T08:19:11.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ecaba5740 1 Processor -- start 2023-11-08T08:19:11.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ecaba5740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251e710 0x27b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ecaba5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x251eae0 0x278d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ecaba5740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2569790 0x278d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ecaba5740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x25a3b20 con 0x251e710 2023-11-08T08:19:11.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ecaba5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x278ab50 con 0x251eae0 2023-11-08T08:19:11.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ecaba5740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x278adf0 con 0x2569790 2023-11-08T08:19:11.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ecaba5740 10 client.0 _ll_register_callbacks cb 0x27530d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:19:11.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ec56ba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251e710 0x27b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ec56ba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251e710 0x27b5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43392/0 (socket says 172.21.15.40:43392) 2023-11-08T08:19:11.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ec56ba700 1 -- 172.21.15.40:0/427312189 learned_addr learned my addr 172.21.15.40:0/427312189 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:11.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ec4eb9700 1 --2- 172.21.15.40:0/427312189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x251eae0 0x278d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.359+0000 7f1ec5ebb700 1 --2- 172.21.15.40:0/427312189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2569790 0x278d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1ec56ba700 1 -- 172.21.15.40:0/427312189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2569790 msgr2=0x278d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:11.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1ec56ba700 1 --2- 172.21.15.40:0/427312189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2569790 0x278d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1ec56ba700 1 -- 172.21.15.40:0/427312189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x251eae0 msgr2=0x278d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:11.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1ec56ba700 1 --2- 172.21.15.40:0/427312189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x251eae0 0x278d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1ec56ba700 1 -- 172.21.15.40:0/427312189 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ebc00a620 con 0x251e710 2023-11-08T08:19:11.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1ec56ba700 1 --2- 172.21.15.40:0/427312189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251e710 0x27b5c70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc006bc0 tx=0x7f1ebc006b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c7404de980dca4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:11.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1e9bfff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:19:11.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1e9bfff700 1 -- 172.21.15.40:0/427312189 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc020070 con 0x251e710 2023-11-08T08:19:11.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1ecaba5740 1 -- 172.21.15.40:0/427312189 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x26eff50 con 0x251e710 2023-11-08T08:19:11.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1e9bfff700 1 -- 172.21.15.40:0/427312189 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1ebc01a450 con 0x251e710 2023-11-08T08:19:11.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1e9bfff700 1 -- 172.21.15.40:0/427312189 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ebc01a610 con 0x251e710 2023-11-08T08:19:11.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1ecaba5740 10 client.4434 fetch_fsmap learned FSMap version 7 2023-11-08T08:19:11.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1ecaba5740 1 -- 172.21.15.40:0/427312189 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x256a9e0 con 0x251e710 2023-11-08T08:19:11.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1e9bfff700 1 -- 172.21.15.40:0/427312189 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x26eff50 con 0x251e710 2023-11-08T08:19:11.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1e9bfff700 1 -- 172.21.15.40:0/427312189 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1ebc01a770 con 0x251e710 2023-11-08T08:19:11.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1ecaba5740 10 client.4434 fetch_fsmap finished waiting for FSMap version 7 2023-11-08T08:19:11.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1ecaba5740 10 client.4434 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T08:19:11.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1ecaba5740 20 client.4434 populate_metadata read hostname 'smithi040' 2023-11-08T08:19:11.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1ecaba5740 1 --2- 172.21.15.40:0/427312189 >> [v2:172.21.15.169:6836/1989495204,v1:172.21.15.169:6837/1989495204] conn(0x278a370 0x278f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1ecaba5740 4 client.4434 mds_command: new command op to 4291 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-08T08:19:11.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.363+0000 7f1ecaba5740 1 -- 172.21.15.40:0/427312189 --> [v2:172.21.15.169:6836/1989495204,v1:172.21.15.169:6837/1989495204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x26efc70 con 0x278a370 2023-11-08T08:19:11.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.367+0000 7f1ec4eb9700 1 --2- 172.21.15.40:0/427312189 >> [v2:172.21.15.169:6836/1989495204,v1:172.21.15.169:6837/1989495204] conn(0x278a370 0x278f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.367+0000 7f1e9bfff700 10 client.4434 ms_handle_connect on v2:172.21.15.169:6836/1989495204 2023-11-08T08:19:11.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.367+0000 7f1ec4eb9700 1 --2- 172.21.15.40:0/427312189 >> [v2:172.21.15.169:6836/1989495204,v1:172.21.15.169:6837/1989495204] conn(0x278a370 0x278f9d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6b53175c8b31ad6 server_cookie=1793aafde12fc142 in_seq=0 out_seq=0 2023-11-08T08:19:11.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.367+0000 7f1e9bfff700 1 -- 172.21.15.40:0/427312189 <== mds.0 v2:172.21.15.169:6836/1989495204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x26efc70 con 0x278a370 2023-11-08T08:19:11.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.367+0000 7f1e9bfff700 10 client.4434 handle_command_reply: tid=0 2023-11-08T08:19:11.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.371+0000 7f1ecaba5740 1 client.4434 shutdown 2023-11-08T08:19:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.371+0000 7f1ecaba5740 1 -- 172.21.15.40:0/427312189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251e710 msgr2=0x27b5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.371+0000 7f1ecaba5740 1 --2- 172.21.15.40:0/427312189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251e710 0x27b5c70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc006bc0 tx=0x7f1ebc006b10 comp rx=0 tx=0).stop 2023-11-08T08:19:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.371+0000 7f1ecaba5740 1 -- 172.21.15.40:0/427312189 shutdown_connections 2023-11-08T08:19:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.371+0000 7f1ecaba5740 1 --2- 172.21.15.40:0/427312189 >> [v2:172.21.15.169:6836/1989495204,v1:172.21.15.169:6837/1989495204] conn(0x278a370 0x278f9d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.371+0000 7f1ecaba5740 1 --2- 172.21.15.40:0/427312189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2569790 0x278d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.371+0000 7f1ecaba5740 1 --2- 172.21.15.40:0/427312189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x251eae0 0x278d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.371+0000 7f1ecaba5740 1 --2- 172.21.15.40:0/427312189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251e710 0x27b5c70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.371+0000 7f1ecaba5740 1 -- 172.21.15.40:0/427312189 >> 172.21.15.40:0/427312189 conn(0x251eeb0 msgr2=0x27b7710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:11.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.371+0000 7f1e9bfff700 0 client.4434 ms_handle_reset on v2:172.21.15.169:6836/1989495204 2023-11-08T08:19:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.371+0000 7f1ecaba5740 1 -- 172.21.15.40:0/427312189 shutdown_connections 2023-11-08T08:19:11.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.371+0000 7f1ecaba5740 1 -- 172.21.15.40:0/427312189 wait complete. 2023-11-08T08:19:11.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.371+0000 7f1ecaba5740 20 client.4434 trim_cache size 0 max 16384 2023-11-08T08:19:11.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.395+0000 7f1ecaba5740 1 Processor -- start 2023-11-08T08:19:11.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.395+0000 7f1ecaba5740 1 -- start start 2023-11-08T08:19:11.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.395+0000 7f1ecaba5740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x251e710 0x27d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.395+0000 7f1ecaba5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x251eae0 0x27d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.395+0000 7f1ecaba5740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2569790 0x27b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.395+0000 7f1ecaba5740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x26c7bb0 con 0x2569790 2023-11-08T08:19:11.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.395+0000 7f1ecaba5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x269d750 con 0x251eae0 2023-11-08T08:19:11.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.395+0000 7f1ecaba5740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x26aca40 con 0x251e710 2023-11-08T08:19:11.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.395+0000 7f1ec5ebb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2569790 0x27b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.395+0000 7f1ec5ebb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2569790 0x27b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43398/0 (socket says 172.21.15.40:43398) 2023-11-08T08:19:11.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.395+0000 7f1ec5ebb700 1 -- 172.21.15.40:0/1436098394 learned_addr learned my addr 172.21.15.40:0/1436098394 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:11.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.395+0000 7f1ec4eb9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x251eae0 0x27d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.395+0000 7f1ec56ba700 1 --2- 172.21.15.40:0/1436098394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x251e710 0x27d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.395+0000 7f1ec5ebb700 1 -- 172.21.15.40:0/1436098394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x251e710 msgr2=0x27d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:11.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.395+0000 7f1ec5ebb700 1 --2- 172.21.15.40:0/1436098394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x251e710 0x27d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.395+0000 7f1ec5ebb700 1 -- 172.21.15.40:0/1436098394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x251eae0 msgr2=0x27d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:11.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.395+0000 7f1ec5ebb700 1 --2- 172.21.15.40:0/1436098394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x251eae0 0x27d0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.395+0000 7f1ec5ebb700 1 -- 172.21.15.40:0/1436098394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec0162c00 con 0x2569790 2023-11-08T08:19:11.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.395+0000 7f1ec4eb9700 1 --2- 172.21.15.40:0/1436098394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x251eae0 0x27d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:11.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.399+0000 7f1ec5ebb700 1 --2- 172.21.15.40:0/1436098394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2569790 0x27b1960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800e5b0 tx=0x7f1eb8010070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8244de62e5b70123 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:11.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.399+0000 7f1ea37fe700 1 -- 172.21.15.40:0/1436098394 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8026400 con 0x2569790 2023-11-08T08:19:11.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.399+0000 7f1ea37fe700 1 -- 172.21.15.40:0/1436098394 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb8026560 con 0x2569790 2023-11-08T08:19:11.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.399+0000 7f1ea37fe700 1 -- 172.21.15.40:0/1436098394 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb80268d0 con 0x2569790 2023-11-08T08:19:11.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.399+0000 7f1ecaba5740 1 -- 172.21.15.40:0/1436098394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2569790 msgr2=0x27b1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:11.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.399+0000 7f1ecaba5740 1 --2- 172.21.15.40:0/1436098394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2569790 0x27b1960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800e5b0 tx=0x7f1eb8010070 comp rx=0 tx=0).stop 2023-11-08T08:19:11.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.399+0000 7f1ecaba5740 1 -- 172.21.15.40:0/1436098394 shutdown_connections 2023-11-08T08:19:11.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.399+0000 7f1ecaba5740 1 --2- 172.21.15.40:0/1436098394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2569790 0x27b1960 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.399+0000 7f1ecaba5740 1 --2- 172.21.15.40:0/1436098394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x251eae0 0x27d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.399+0000 7f1ecaba5740 1 --2- 172.21.15.40:0/1436098394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x251e710 0x27d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.399+0000 7f1ecaba5740 1 -- 172.21.15.40:0/1436098394 >> 172.21.15.40:0/1436098394 conn(0x251eeb0 msgr2=0x27cfb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:11.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.399+0000 7f1ecaba5740 1 -- 172.21.15.40:0/1436098394 shutdown_connections 2023-11-08T08:19:11.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.399+0000 7f1ecaba5740 1 -- 172.21.15.40:0/1436098394 wait complete. 2023-11-08T08:19:11.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.399+0000 7f1ecaba5740 1 -- start start 2023-11-08T08:19:11.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.399+0000 7f1ecaba5740 1 Processor -- start 2023-11-08T08:19:11.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ecaba5740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251e710 0x27b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ecaba5740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x251eae0 0x27e7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ecaba5740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2569790 0x2832f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ecaba5740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x278b660 con 0x251e710 2023-11-08T08:19:11.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ecaba5740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x27b55d0 con 0x2569790 2023-11-08T08:19:11.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ecaba5740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x278b890 con 0x251eae0 2023-11-08T08:19:11.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ecaba5740 10 client.0 _ll_register_callbacks cb 0x26a32d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:19:11.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ec56ba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251e710 0x27b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ec56ba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251e710 0x27b4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43414/0 (socket says 172.21.15.40:43414) 2023-11-08T08:19:11.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ec56ba700 1 -- 172.21.15.40:0/2461202136 learned_addr learned my addr 172.21.15.40:0/2461202136 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:11.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ec4eb9700 1 --2- 172.21.15.40:0/2461202136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x251eae0 0x27e7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ec5ebb700 1 --2- 172.21.15.40:0/2461202136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2569790 0x2832f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ec56ba700 1 -- 172.21.15.40:0/2461202136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x251eae0 msgr2=0x27e7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:11.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ec56ba700 1 --2- 172.21.15.40:0/2461202136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x251eae0 0x27e7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ec56ba700 1 -- 172.21.15.40:0/2461202136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2569790 msgr2=0x2832f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:11.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ec56ba700 1 --2- 172.21.15.40:0/2461202136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2569790 0x2832f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ec56ba700 1 -- 172.21.15.40:0/2461202136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec01646c0 con 0x251e710 2023-11-08T08:19:11.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ec56ba700 1 --2- 172.21.15.40:0/2461202136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251e710 0x27b4b30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc02a7c0 tx=0x7f1ebc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39e9e2f16ce8c05 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:11.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1e9bfff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:19:11.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1e9bfff700 1 -- 172.21.15.40:0/2461202136 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc00aec0 con 0x251e710 2023-11-08T08:19:11.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ecaba5740 1 -- 172.21.15.40:0/2461202136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ebc00cbb0 con 0x251e710 2023-11-08T08:19:11.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1e9bfff700 1 -- 172.21.15.40:0/2461202136 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1ebc000e40 con 0x251e710 2023-11-08T08:19:11.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1e9bfff700 1 -- 172.21.15.40:0/2461202136 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ebc01ac30 con 0x251e710 2023-11-08T08:19:11.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ecaba5740 10 client.4440 fetch_fsmap learned FSMap version 7 2023-11-08T08:19:11.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ecaba5740 1 -- 172.21.15.40:0/2461202136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x7f1eb8016040 con 0x251e710 2023-11-08T08:19:11.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1e9bfff700 1 -- 172.21.15.40:0/2461202136 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ebc00cbb0 con 0x251e710 2023-11-08T08:19:11.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1e9bfff700 1 -- 172.21.15.40:0/2461202136 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1ebc01a450 con 0x251e710 2023-11-08T08:19:11.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ecaba5740 10 client.4440 fetch_fsmap finished waiting for FSMap version 7 2023-11-08T08:19:11.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ecaba5740 10 client.4440 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T08:19:11.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ecaba5740 20 client.4440 populate_metadata read hostname 'smithi040' 2023-11-08T08:19:11.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ecaba5740 1 --2- 172.21.15.40:0/2461202136 >> [v2:172.21.15.169:6836/1989495204,v1:172.21.15.169:6837/1989495204] conn(0x2833470 0x2833840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ecaba5740 4 client.4440 mds_command: new command op to 4291 tid=0[{"prefix": "status"}] 2023-11-08T08:19:11.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ecaba5740 1 -- 172.21.15.40:0/2461202136 --> [v2:172.21.15.169:6836/1989495204,v1:172.21.15.169:6837/1989495204] -- command(tid 0: {"prefix": "status"}) v1 -- 0x259b260 con 0x2833470 2023-11-08T08:19:11.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.403+0000 7f1ec4eb9700 1 --2- 172.21.15.40:0/2461202136 >> [v2:172.21.15.169:6836/1989495204,v1:172.21.15.169:6837/1989495204] conn(0x2833470 0x2833840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ec4eb9700 1 --2- 172.21.15.40:0/2461202136 >> [v2:172.21.15.169:6836/1989495204,v1:172.21.15.169:6837/1989495204] conn(0x2833470 0x2833840 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95968860c54f7406 server_cookie=30cd04ed04c968dd in_seq=0 out_seq=0 2023-11-08T08:19:11.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1e9bfff700 10 client.4440 ms_handle_connect on v2:172.21.15.169:6836/1989495204 2023-11-08T08:19:11.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1e9bfff700 1 -- 172.21.15.40:0/2461202136 <== mds.0 v2:172.21.15.169:6836/1989495204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+323 (crc 0 0 0) 0x259b260 con 0x2833470 2023-11-08T08:19:11.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1e9bfff700 10 client.4440 handle_command_reply: tid=0 2023-11-08T08:19:11.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ecaba5740 1 client.4440 shutdown 2023-11-08T08:19:11.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ecaba5740 1 -- 172.21.15.40:0/2461202136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251e710 msgr2=0x27b4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:11.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ecaba5740 1 --2- 172.21.15.40:0/2461202136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251e710 0x27b4b30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc02a7c0 tx=0x7f1ebc013000 comp rx=0 tx=0).stop 2023-11-08T08:19:11.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ecaba5740 1 -- 172.21.15.40:0/2461202136 shutdown_connections 2023-11-08T08:19:11.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ecaba5740 1 --2- 172.21.15.40:0/2461202136 >> [v2:172.21.15.169:6836/1989495204,v1:172.21.15.169:6837/1989495204] conn(0x2833470 0x2833840 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ecaba5740 1 --2- 172.21.15.40:0/2461202136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2569790 0x2832f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ecaba5740 1 --2- 172.21.15.40:0/2461202136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x251eae0 0x27e7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ecaba5740 1 --2- 172.21.15.40:0/2461202136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x251e710 0x27b4b30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ecaba5740 1 -- 172.21.15.40:0/2461202136 >> 172.21.15.40:0/2461202136 conn(0x251eeb0 msgr2=0x27d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1e9bfff700 0 client.4440 ms_handle_reset on v2:172.21.15.169:6836/1989495204 2023-11-08T08:19:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ecaba5740 1 -- 172.21.15.40:0/2461202136 shutdown_connections 2023-11-08T08:19:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ecaba5740 1 -- 172.21.15.40:0/2461202136 wait complete. 2023-11-08T08:19:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ecaba5740 20 client.4440 trim_cache size 0 max 16384 2023-11-08T08:19:11.424 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:19:11.424 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "54ed45f4-d5b0-41c3-8c3b-b5a18b844f25", 2023-11-08T08:19:11.424 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2023-11-08T08:19:11.424 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4291, 2023-11-08T08:19:11.425 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-11-08T08:19:11.425 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-11-08T08:19:11.425 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-11-08T08:19:11.425 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 6.9469601399999998, 2023-11-08T08:19:11.425 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 7, 2023-11-08T08:19:11.426 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 28, 2023-11-08T08:19:11.426 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 27, 2023-11-08T08:19:11.426 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 12.45043029 2023-11-08T08:19:11.426 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:19:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ec791e700 1 -- 172.21.15.40:0/369886585 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1ea4064ee0 msgr2=0x7f1ea4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ec791e700 1 --2- 172.21.15.40:0/369886585 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1ea4064ee0 0x7f1ea4067390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009fd0 tx=0x7f1eb80094e0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ec791e700 1 -- 172.21.15.40:0/369886585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec00cd800 msgr2=0x7f1ec01ec0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ec791e700 1 --2- 172.21.15.40:0/369886585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec00cd800 0x7f1ec01ec0a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0000c00 tx=0x7f1eb0004770 comp rx=0 tx=0).stop 2023-11-08T08:19:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ec791e700 1 -- 172.21.15.40:0/369886585 shutdown_connections 2023-11-08T08:19:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ec791e700 1 --2- 172.21.15.40:0/369886585 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1ea4064ee0 0x7f1ea4067390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ec791e700 1 --2- 172.21.15.40:0/369886585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00cdbd0 0x7f1ec0162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ec791e700 1 --2- 172.21.15.40:0/369886585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec00cd800 0x7f1ec01ec0a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ec791e700 1 --2- 172.21.15.40:0/369886585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ec00c38b0 0x7f1ec0157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ec791e700 1 -- 172.21.15.40:0/369886585 >> 172.21.15.40:0/369886585 conn(0x7f1ec006e5d0 msgr2=0x7f1ec00cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ec791e700 1 -- 172.21.15.40:0/369886585 shutdown_connections 2023-11-08T08:19:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.407+0000 7f1ec791e700 1 -- 172.21.15.40:0/369886585 wait complete. 2023-11-08T08:19:11.433 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-08T08:19:11.434 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-08T08:19:11.435 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T08:19:11.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.595+0000 7f8795145700 1 Processor -- start 2023-11-08T08:19:11.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.595+0000 7f8795145700 1 -- start start 2023-11-08T08:19:11.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.595+0000 7f8795145700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f879015c410 0x7f8790160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.595+0000 7f8795145700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87900ca740 con 0x7f87900cf2a0 2023-11-08T08:19:11.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.595+0000 7f8795145700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8790161020 con 0x7f87900b1240 2023-11-08T08:19:11.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.599+0000 7f8795145700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87900ca9e0 con 0x7f879015c410 2023-11-08T08:19:11.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.599+0000 7f878ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f879015c410 0x7f8790160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.599+0000 7f8787fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f87900cf2a0 0x7f8790069c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51456/0 (socket says 172.21.15.40:51456) 2023-11-08T08:19:11.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.599+0000 7f8787fff700 1 -- 172.21.15.40:0/507789645 learned_addr learned my addr 172.21.15.40:0/507789645 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:11.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.599+0000 7f878f7fe700 1 -- 172.21.15.40:0/507789645 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 459357208 0 0) 0x7f87900ca740 con 0x7f87900cf2a0 2023-11-08T08:19:11.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.599+0000 7f878f7fe700 1 -- 172.21.15.40:0/507789645 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f877c0035d0 con 0x7f87900cf2a0 2023-11-08T08:19:11.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.599+0000 7f878ffff700 1 -- 172.21.15.40:0/507789645 >> v1:172.21.15.169:6789/0 conn(0x7f87900b1240 legacy=0x7f87900ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:11.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.599+0000 7f878ffff700 1 -- 172.21.15.40:0/507789645 >> v1:172.21.15.40:6789/0 conn(0x7f87900cf2a0 legacy=0x7f8790069c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:11.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.599+0000 7f878ffff700 1 -- 172.21.15.40:0/507789645 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87900cacc0 con 0x7f879015c410 2023-11-08T08:19:11.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.599+0000 7f878ffff700 1 --2- 172.21.15.40:0/507789645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f879015c410 0x7f8790160ae0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8780009880 tx=0x7f8780009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd63929532cdaf2c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:11.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.599+0000 7f878f7fe700 1 -- 172.21.15.40:0/507789645 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8780011830 con 0x7f879015c410 2023-11-08T08:19:11.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.599+0000 7f878f7fe700 1 -- 172.21.15.40:0/507789645 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8780011990 con 0x7f879015c410 2023-11-08T08:19:11.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.599+0000 7f878f7fe700 1 -- 172.21.15.40:0/507789645 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8780011d20 con 0x7f879015c410 2023-11-08T08:19:11.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.599+0000 7f8795145700 1 -- 172.21.15.40:0/507789645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f879015c410 msgr2=0x7f8790160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:11.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.599+0000 7f8795145700 1 --2- 172.21.15.40:0/507789645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f879015c410 0x7f8790160ae0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8780009880 tx=0x7f8780009b90 comp rx=0 tx=0).stop 2023-11-08T08:19:11.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.599+0000 7f8795145700 1 -- 172.21.15.40:0/507789645 shutdown_connections 2023-11-08T08:19:11.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.599+0000 7f8795145700 1 --2- 172.21.15.40:0/507789645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f879015c410 0x7f8790160ae0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.599+0000 7f8795145700 1 -- 172.21.15.40:0/507789645 >> 172.21.15.40:0/507789645 conn(0x7f87900cc240 msgr2=0x7f87900cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:11.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.599+0000 7f8795145700 1 -- 172.21.15.40:0/507789645 shutdown_connections 2023-11-08T08:19:11.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.599+0000 7f8795145700 1 -- 172.21.15.40:0/507789645 wait complete. 2023-11-08T08:19:11.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.603+0000 7f8795145700 1 Processor -- start 2023-11-08T08:19:11.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.603+0000 7f8795145700 1 -- start start 2023-11-08T08:19:11.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.603+0000 7f8795145700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87900b1240 0x7f87901f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.603+0000 7f8795145700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87900cf2a0 0x7f879015a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.603+0000 7f8795145700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879015c410 0x7f879015a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.603+0000 7f8795145700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87901586a0 con 0x7f879015c410 2023-11-08T08:19:11.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.603+0000 7f8795145700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87901f3100 con 0x7f87900b1240 2023-11-08T08:19:11.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.603+0000 7f8795145700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87901588f0 con 0x7f87900cf2a0 2023-11-08T08:19:11.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.607+0000 7f8794944700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879015c410 0x7f879015a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.607+0000 7f8794944700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879015c410 0x7f879015a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43428/0 (socket says 172.21.15.40:43428) 2023-11-08T08:19:11.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.607+0000 7f8794944700 1 -- 172.21.15.40:0/2208234432 learned_addr learned my addr 172.21.15.40:0/2208234432 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:11.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.607+0000 7f8787fff700 1 --2- 172.21.15.40:0/2208234432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87900cf2a0 0x7f879015a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.607+0000 7f878ffff700 1 --2- 172.21.15.40:0/2208234432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87900b1240 0x7f87901f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.607+0000 7f8794944700 1 -- 172.21.15.40:0/2208234432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87900cf2a0 msgr2=0x7f879015a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:11.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.607+0000 7f8794944700 1 --2- 172.21.15.40:0/2208234432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87900cf2a0 0x7f879015a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.607+0000 7f8794944700 1 -- 172.21.15.40:0/2208234432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87900b1240 msgr2=0x7f87901f2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:11.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.607+0000 7f8794944700 1 --2- 172.21.15.40:0/2208234432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87900b1240 0x7f87901f2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.607+0000 7f8794944700 1 -- 172.21.15.40:0/2208234432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87800094b0 con 0x7f879015c410 2023-11-08T08:19:11.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.607+0000 7f8787fff700 1 --2- 172.21.15.40:0/2208234432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87900cf2a0 0x7f879015a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:11.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.607+0000 7f878ffff700 1 --2- 172.21.15.40:0/2208234432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87900b1240 0x7f87901f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:11.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.607+0000 7f8794944700 1 --2- 172.21.15.40:0/2208234432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879015c410 0x7f879015a780 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f878800cc80 tx=0x7f878800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:11.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.607+0000 7f878d7fa700 1 -- 172.21.15.40:0/2208234432 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788003c20 con 0x7f879015c410 2023-11-08T08:19:11.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.607+0000 7f878d7fa700 1 -- 172.21.15.40:0/2208234432 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8788003d80 con 0x7f879015c410 2023-11-08T08:19:11.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.607+0000 7f878d7fa700 1 -- 172.21.15.40:0/2208234432 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878800ab50 con 0x7f879015c410 2023-11-08T08:19:11.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.607+0000 7f8795145700 1 -- 172.21.15.40:0/2208234432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87900cde70 con 0x7f879015c410 2023-11-08T08:19:11.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.607+0000 7f8795145700 1 -- 172.21.15.40:0/2208234432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87900cdb90 con 0x7f879015c410 2023-11-08T08:19:11.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.607+0000 7f8795145700 1 -- 172.21.15.40:0/2208234432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f879006a370 con 0x7f879015c410 2023-11-08T08:19:11.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.611+0000 7f878d7fa700 1 -- 172.21.15.40:0/2208234432 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f8788010050 con 0x7f879015c410 2023-11-08T08:19:11.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.611+0000 7f878d7fa700 1 --2- 172.21.15.40:0/2208234432 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8770064ee0 0x7f8770067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:11.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.611+0000 7f878d7fa700 1 -- 172.21.15.40:0/2208234432 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8788082160 con 0x7f879015c410 2023-11-08T08:19:11.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.611+0000 7f878ffff700 1 --2- 172.21.15.40:0/2208234432 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8770064ee0 0x7f8770067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:11.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.615+0000 7f878ffff700 1 --2- 172.21.15.40:0/2208234432 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8770064ee0 0x7f8770067390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f87800090b0 tx=0x7f8780010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:11.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.615+0000 7f878d7fa700 1 -- 172.21.15.40:0/2208234432 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f878804fec0 con 0x7f879015c410 2023-11-08T08:19:11.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.827+0000 7f8795145700 1 -- 172.21.15.40:0/2208234432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f879006ed90 con 0x7f879015c410 2023-11-08T08:19:11.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.827+0000 7f878d7fa700 1 -- 172.21.15.40:0/2208234432 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f87880534e0 con 0x7f879015c410 2023-11-08T08:19:11.833 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-08T08:19:11.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.831+0000 7f8795145700 1 -- 172.21.15.40:0/2208234432 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8770064ee0 msgr2=0x7f8770067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:11.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.831+0000 7f8795145700 1 --2- 172.21.15.40:0/2208234432 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8770064ee0 0x7f8770067390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f87800090b0 tx=0x7f8780010000 comp rx=0 tx=0).stop 2023-11-08T08:19:11.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.831+0000 7f8795145700 1 -- 172.21.15.40:0/2208234432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879015c410 msgr2=0x7f879015a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:11.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.831+0000 7f8795145700 1 --2- 172.21.15.40:0/2208234432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879015c410 0x7f879015a780 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f878800cc80 tx=0x7f878800cf90 comp rx=0 tx=0).stop 2023-11-08T08:19:11.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.831+0000 7f8795145700 1 -- 172.21.15.40:0/2208234432 shutdown_connections 2023-11-08T08:19:11.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.831+0000 7f8795145700 1 --2- 172.21.15.40:0/2208234432 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8770064ee0 0x7f8770067390 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.831+0000 7f8795145700 1 --2- 172.21.15.40:0/2208234432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879015c410 0x7f879015a780 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.831+0000 7f8795145700 1 --2- 172.21.15.40:0/2208234432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87900cf2a0 0x7f879015a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.831+0000 7f8795145700 1 --2- 172.21.15.40:0/2208234432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87900b1240 0x7f87901f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:11.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.831+0000 7f8795145700 1 -- 172.21.15.40:0/2208234432 >> 172.21.15.40:0/2208234432 conn(0x7f87900cc240 msgr2=0x7f87900cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:11.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.831+0000 7f8795145700 1 -- 172.21.15.40:0/2208234432 shutdown_connections 2023-11-08T08:19:11.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:11.831+0000 7f8795145700 1 -- 172.21.15.40:0/2208234432 wait complete. 2023-11-08T08:19:11.861 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-08T08:19:11.861 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T08:19:11.862 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-11-08T08:19:11.862 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-11-08T08:19:11.862 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-08T08:19:11.863 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-08T08:19:11.863 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T08:19:11.863 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T08:19:11.864 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-11-08T08:19:11.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:11.864 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-11-08T08:19:11.886 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T08:19:11.887 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T08:19:11.887 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T08:19:11.887 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:11.887 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-11-08T08:19:11.887 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:11.888 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:19:11.888 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:11.888 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2023-11-08T08:19:11.888 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:19:11.888 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:11.889 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2023-11-08T08:19:11.889 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T08:19:11.889 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:11.889 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2023-11-08T08:19:11.889 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-08T08:19:11.889 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42369sec preferred_lft 42369sec 2023-11-08T08:19:11.890 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link 2023-11-08T08:19:11.890 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:11.890 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:19:11.890 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:11.891 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2023-11-08T08:19:11.891 INFO:teuthology.orchestra.run.smithi040.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-08T08:19:11.891 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 02:42:94:e4:3d:2e brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:11.891 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-08T08:19:11.891 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:11.893 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-08T08:19:11.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:11.894 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-11-08T08:19:11.894 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-11-08T08:19:11.894 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add name ceph-brx type bridge 2023-11-08T08:19:11.895 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr flush dev ceph-brx 2023-11-08T08:19:11.895 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set ceph-brx up 2023-11-08T08:19:11.895 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-08T08:19:11.896 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-11-08T08:19:12.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:12.049 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T08:19:12.077 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-11-08T08:19:12.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:12.081 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-11-08T08:19:12.708 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-11-08T08:19:12.708 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T08:19:12.709 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-08T08:19:12.709 INFO:teuthology.orchestra.run.smithi040.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-08T08:19:12.709 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-08T08:19:12.709 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T08:19:12.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:12.710 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-11-08T08:19:12.711 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-11-08T08:19:12.711 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-08T08:19:12.711 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-08T08:19:12.711 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-08T08:19:12.712 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-11-08T08:19:12.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:12.806 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-11-08T08:19:12.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:12.824 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2023-11-08T08:19:12.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:12.890 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-11-08T08:19:12.891 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-11-08T08:19:12.891 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-08T08:19:12.891 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-11-08T08:19:12.892 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-11-08T08:19:13.014 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-11-08T08:19:13.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:13.015 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-11-08T08:19:13.016 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-11-08T08:19:13.016 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-11-08T08:19:13.016 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-08T08:19:13.017 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-11-08T08:19:13.017 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-11-08T08:19:13.017 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-11-08T08:19:13.018 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-11-08T08:19:13.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:13.192 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-11-08T08:19:13.193 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-11-08T08:19:13.193 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2023-11-08T08:19:13.193 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.0 master ceph-brx 2023-11-08T08:19:13.194 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-11-08T08:19:13.265 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-11-08T08:19:13.265 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T08:19:13.266 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-11-08T08:19:13.280 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-11-08T08:19:13.280 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T08:19:13.281 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-11-08T08:19:13.340 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2023-11-08T08:19:13.419 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-08T08:19:13.420 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-08T08:19:13.500 INFO:teuthology.orchestra.run.smithi040.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-08T08:19:13.502 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-08T08:19:13.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:13.503 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2023-11-08T08:19:13.562 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-08T08:19:13.562 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_mountpoint=/ --client_fs=cephfs) 2023-11-08T08:19:13.610 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2023-11-08T08:19:13.648 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-08T08:19:13.648 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-08T08:19:13.721 INFO:teuthology.orchestra.run.smithi040.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-08T08:19:13.722 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-08T08:19:13.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:13.724 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2023-11-08T08:19:13.764 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi040.stderr:2023-11-08T08:19:13.759+0000 7f70ae83b640 -1 init, newargv = 0x5648f56d49d0 newargc=15 2023-11-08T08:19:13.764 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi040.stderr:ceph-fuse[41249]: starting ceph client 2023-11-08T08:19:13.782 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi040.stderr:ceph-fuse[41249]: starting fuse 2023-11-08T08:19:14.879 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2023-11-08T08:19:14.925 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-08T08:19:14.926 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-08T08:19:15.008 INFO:teuthology.orchestra.run.smithi040.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-08T08:19:15.011 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-08T08:19:15.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:15.012 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2023-11-08T08:19:15.082 INFO:teuthology.orchestra.run.smithi040.stdout:49 2023-11-08T08:19:15.082 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [49] 2023-11-08T08:19:15.083 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-08T08:19:15.083 DEBUG:teuthology.orchestra.run.smithi040:> import glob 2023-11-08T08:19:15.083 DEBUG:teuthology.orchestra.run.smithi040:> import re 2023-11-08T08:19:15.084 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-11-08T08:19:15.084 DEBUG:teuthology.orchestra.run.smithi040:> import subprocess 2023-11-08T08:19:15.084 DEBUG:teuthology.orchestra.run.smithi040:> 2023-11-08T08:19:15.085 DEBUG:teuthology.orchestra.run.smithi040:> def _find_admin_socket(client_name): 2023-11-08T08:19:15.085 DEBUG:teuthology.orchestra.run.smithi040:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-11-08T08:19:15.085 DEBUG:teuthology.orchestra.run.smithi040:> files = glob.glob(asok_path) 2023-11-08T08:19:15.086 DEBUG:teuthology.orchestra.run.smithi040:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-11-08T08:19:15.086 DEBUG:teuthology.orchestra.run.smithi040:> 2023-11-08T08:19:15.086 DEBUG:teuthology.orchestra.run.smithi040:> # Given a non-glob path, it better be there 2023-11-08T08:19:15.087 DEBUG:teuthology.orchestra.run.smithi040:> if "*" not in asok_path: 2023-11-08T08:19:15.087 DEBUG:teuthology.orchestra.run.smithi040:> assert(len(files) == 1) 2023-11-08T08:19:15.087 DEBUG:teuthology.orchestra.run.smithi040:> return files[0] 2023-11-08T08:19:15.087 DEBUG:teuthology.orchestra.run.smithi040:> 2023-11-08T08:19:15.088 DEBUG:teuthology.orchestra.run.smithi040:> for f in files: 2023-11-08T08:19:15.088 DEBUG:teuthology.orchestra.run.smithi040:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-08T08:19:15.088 DEBUG:teuthology.orchestra.run.smithi040:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-08T08:19:15.089 DEBUG:teuthology.orchestra.run.smithi040:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-08T08:19:15.089 DEBUG:teuthology.orchestra.run.smithi040:> contents = proc_f.read() 2023-11-08T08:19:15.089 DEBUG:teuthology.orchestra.run.smithi040:> if mountpoint in contents: 2023-11-08T08:19:15.090 DEBUG:teuthology.orchestra.run.smithi040:> return f 2023-11-08T08:19:15.090 DEBUG:teuthology.orchestra.run.smithi040:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-08T08:19:15.090 DEBUG:teuthology.orchestra.run.smithi040:> 2023-11-08T08:19:15.091 DEBUG:teuthology.orchestra.run.smithi040:> print(_find_admin_socket("client.admin")) 2023-11-08T08:19:15.091 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-11-08T08:19:15.260 INFO:teuthology.orchestra.run.smithi040.stdout:/var/run/ceph/ceph-client.admin.41249.asok 2023-11-08T08:19:15.440 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.41249.asok 2023-11-08T08:19:15.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:15.441 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.41249.asok status 2023-11-08T08:19:15.650 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:19:15.651 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": { 2023-11-08T08:19:15.651 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_sha1": "b820560f30d310ab2570761b9a294523225d6998", 2023-11-08T08:19:15.651 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_version": "ceph version 18.0.0-2698-gb820560f (b820560f30d310ab2570761b9a294523225d6998) reef (dev)", 2023-11-08T08:19:15.651 INFO:teuthology.orchestra.run.smithi040.stdout: "entity_id": "admin", 2023-11-08T08:19:15.652 INFO:teuthology.orchestra.run.smithi040.stdout: "hostname": "smithi040", 2023-11-08T08:19:15.652 INFO:teuthology.orchestra.run.smithi040.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-11-08T08:19:15.652 INFO:teuthology.orchestra.run.smithi040.stdout: "pid": "41249", 2023-11-08T08:19:15.652 INFO:teuthology.orchestra.run.smithi040.stdout: "root": "/" 2023-11-08T08:19:15.652 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:19:15.652 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_count": 0, 2023-11-08T08:19:15.653 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_pinned_count": 0, 2023-11-08T08:19:15.653 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4355, 2023-11-08T08:19:15.653 INFO:teuthology.orchestra.run.smithi040.stdout: "inst": { 2023-11-08T08:19:15.653 INFO:teuthology.orchestra.run.smithi040.stdout: "name": { 2023-11-08T08:19:15.653 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2023-11-08T08:19:15.654 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4355 2023-11-08T08:19:15.654 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:19:15.654 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2023-11-08T08:19:15.654 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-11-08T08:19:15.654 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "192.168.0.1:0", 2023-11-08T08:19:15.655 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2661489926 2023-11-08T08:19:15.655 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:19:15.655 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:19:15.655 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2023-11-08T08:19:15.655 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-11-08T08:19:15.656 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "192.168.0.1:0", 2023-11-08T08:19:15.656 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2661489926 2023-11-08T08:19:15.656 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:19:15.656 INFO:teuthology.orchestra.run.smithi040.stdout: "inst_str": "client.4355 192.168.0.1:0/2661489926", 2023-11-08T08:19:15.657 INFO:teuthology.orchestra.run.smithi040.stdout: "addr_str": "192.168.0.1:0/2661489926", 2023-11-08T08:19:15.657 INFO:teuthology.orchestra.run.smithi040.stdout: "inode_count": 1, 2023-11-08T08:19:15.657 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_epoch": 7, 2023-11-08T08:19:15.657 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch": 28, 2023-11-08T08:19:15.657 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch_barrier": 0, 2023-11-08T08:19:15.657 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklisted": false, 2023-11-08T08:19:15.658 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs" 2023-11-08T08:19:15.658 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:19:15.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:15.666 DEBUG:teuthology.orchestra.run.smithi040:> stat --file-system '--printf=%T 2023-11-08T08:19:15.666 DEBUG:teuthology.orchestra.run.smithi040:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-11-08T08:19:15.687 INFO:teuthology.orchestra.run.smithi040.stdout:fuseblk 2023-11-08T08:19:15.687 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-11-08T08:19:15.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:15.688 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-11-08T08:19:15.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:15.774 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-11-08T08:19:15.887 INFO:teuthology.orchestra.run.smithi040.stdout:# file: . 2023-11-08T08:19:15.887 INFO:teuthology.orchestra.run.smithi040.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-11-08T08:19:15.887 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:15.889 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:19:15.889 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:19:15.907 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:19:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.135+0000 7f311e271700 1 Processor -- start 2023-11-08T08:19:16.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.135+0000 7f311e271700 1 -- start start 2023-11-08T08:19:16.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.135+0000 7f311e271700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f311806d4c0 0x7f311816b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:16.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.135+0000 7f311e271700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3118164080 con 0x7f31180cdfb0 2023-11-08T08:19:16.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.135+0000 7f311e271700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3118069bf0 con 0x7f3118163cb0 2023-11-08T08:19:16.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.135+0000 7f311e271700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31180cc290 con 0x7f311806d4c0 2023-11-08T08:19:16.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.135+0000 7f31177fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f31180cdfb0 0x7f311806cdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51462/0 (socket says 172.21.15.40:51462) 2023-11-08T08:19:16.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.135+0000 7f31177fe700 1 -- 172.21.15.40:0/178561666 learned_addr learned my addr 172.21.15.40:0/178561666 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.135+0000 7f311c80e700 1 --2- 172.21.15.40:0/178561666 >> v2:172.21.15.169:3300/0 conn(0x7f311806d4c0 0x7f311816b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3116ffd700 1 -- 172.21.15.40:0/178561666 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 520558162 0 0) 0x7f3118164080 con 0x7f31180cdfb0 2023-11-08T08:19:16.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3116ffd700 1 -- 172.21.15.40:0/178561666 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31000035d0 con 0x7f31180cdfb0 2023-11-08T08:19:16.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3116ffd700 1 -- 172.21.15.40:0/178561666 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811102046 0 0) 0x7f3118069bf0 con 0x7f3118163cb0 2023-11-08T08:19:16.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3116ffd700 1 -- 172.21.15.40:0/178561666 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3118164080 con 0x7f3118163cb0 2023-11-08T08:19:16.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3116ffd700 1 -- 172.21.15.40:0/178561666 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 319620941 0 0) 0x7f31000035d0 con 0x7f31180cdfb0 2023-11-08T08:19:16.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3116ffd700 1 -- 172.21.15.40:0/178561666 >> v2:172.21.15.169:3300/0 conn(0x7f311806d4c0 msgr2=0x7f311816b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:16.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3116ffd700 1 --2- 172.21.15.40:0/178561666 >> v2:172.21.15.169:3300/0 conn(0x7f311806d4c0 0x7f311816b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3116ffd700 1 -- 172.21.15.40:0/178561666 >> v1:172.21.15.169:6789/0 conn(0x7f3118163cb0 legacy=0x7f3118164cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:16.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3116ffd700 1 -- 172.21.15.40:0/178561666 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31180cc4a0 con 0x7f31180cdfb0 2023-11-08T08:19:16.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3116ffd700 1 -- 172.21.15.40:0/178561666 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f310c001670 con 0x7f31180cdfb0 2023-11-08T08:19:16.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3116ffd700 1 -- 172.21.15.40:0/178561666 >> v1:172.21.15.40:6789/0 conn(0x7f31180cdfb0 legacy=0x7f311806cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:16.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3116ffd700 1 --2- 172.21.15.40:0/178561666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3100003bb0 0x7f3100003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:16.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3116ffd700 1 --2- 172.21.15.40:0/178561666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3100004870 0x7f3100004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:16.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3116ffd700 1 --2- 172.21.15.40:0/178561666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3100005220 0x7f310000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:16.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3116ffd700 1 -- 172.21.15.40:0/178561666 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31180cc290 con 0x7f3100003bb0 2023-11-08T08:19:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3116ffd700 1 -- 172.21.15.40:0/178561666 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310000c130 con 0x7f3100004870 2023-11-08T08:19:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3116ffd700 1 -- 172.21.15.40:0/178561666 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310000c2f0 con 0x7f3100005220 2023-11-08T08:19:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f311c80e700 1 --2- 172.21.15.40:0/178561666 >> v2:172.21.15.169:3300/0 conn(0x7f311806d4c0 0x7f311816b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3117fff700 1 --2- 172.21.15.40:0/178561666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3100003bb0 0x7f3100003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f31177fe700 1 --2- 172.21.15.40:0/178561666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3100004870 0x7f3100004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:16.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f311c80e700 1 --2- 172.21.15.40:0/178561666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3100005220 0x7f310000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:16.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3117fff700 1 -- 172.21.15.40:0/178561666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3100005220 msgr2=0x7f310000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:16.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3117fff700 1 --2- 172.21.15.40:0/178561666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3100005220 0x7f310000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3117fff700 1 -- 172.21.15.40:0/178561666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3100004870 msgr2=0x7f3100004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:16.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3117fff700 1 --2- 172.21.15.40:0/178561666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3100004870 0x7f3100004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3117fff700 1 -- 172.21.15.40:0/178561666 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31000035d0 con 0x7f3100003bb0 2023-11-08T08:19:16.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f311c80e700 1 --2- 172.21.15.40:0/178561666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3100005220 0x7f310000ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:16.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f311c80e700 1 -- 172.21.15.40:0/178561666 reap_dead start 2023-11-08T08:19:16.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3117fff700 1 --2- 172.21.15.40:0/178561666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3100003bb0 0x7f3100003fa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f310800c2f0 tx=0x7f31000058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837eb7825a8ae400 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:16.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3116ffd700 1 -- 172.21.15.40:0/178561666 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108001f20 con 0x7f3100003bb0 2023-11-08T08:19:16.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3116ffd700 1 -- 172.21.15.40:0/178561666 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310800cdf0 con 0x7f3100003bb0 2023-11-08T08:19:16.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.139+0000 7f3116ffd700 1 -- 172.21.15.40:0/178561666 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108003c60 con 0x7f3100003bb0 2023-11-08T08:19:16.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.143+0000 7f311e271700 1 -- 172.21.15.40:0/178561666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3100003bb0 msgr2=0x7f3100003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:16.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.143+0000 7f311e271700 1 --2- 172.21.15.40:0/178561666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3100003bb0 0x7f3100003fa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f310800c2f0 tx=0x7f31000058a0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.143+0000 7f311e271700 1 -- 172.21.15.40:0/178561666 shutdown_connections 2023-11-08T08:19:16.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.143+0000 7f311e271700 1 --2- 172.21.15.40:0/178561666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3100003bb0 0x7f3100003fa0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.143+0000 7f311e271700 1 -- 172.21.15.40:0/178561666 >> 172.21.15.40:0/178561666 conn(0x7f31180cf350 msgr2=0x7f31180b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:16.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.143+0000 7f311e271700 1 -- 172.21.15.40:0/178561666 shutdown_connections 2023-11-08T08:19:16.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.143+0000 7f311e271700 1 -- 172.21.15.40:0/178561666 wait complete. 2023-11-08T08:19:16.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.143+0000 7f311e271700 1 Processor -- start 2023-11-08T08:19:16.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.143+0000 7f311e271700 1 -- start start 2023-11-08T08:19:16.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f311e271700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3100003bb0 0x7f31180c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:16.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f311e271700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31180b14f0 0x7f31180c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:16.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f311e271700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31180ca260 0x7f31181f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:16.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f311e271700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31180ca6d0 con 0x7f31180b14f0 2023-11-08T08:19:16.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f311e271700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31180c2a20 con 0x7f31180ca260 2023-11-08T08:19:16.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f311e271700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31180c2d00 con 0x7f3100003bb0 2023-11-08T08:19:16.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f31177fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31180b14f0 0x7f31180c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:16.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f31177fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31180b14f0 0x7f31180c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43444/0 (socket says 172.21.15.40:43444) 2023-11-08T08:19:16.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f31177fe700 1 -- 172.21.15.40:0/375513481 learned_addr learned my addr 172.21.15.40:0/375513481 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:16.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f3117fff700 1 --2- 172.21.15.40:0/375513481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3100003bb0 0x7f31180c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:16.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f31177fe700 1 -- 172.21.15.40:0/375513481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3100003bb0 msgr2=0x7f31180c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:16.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f31177fe700 1 --2- 172.21.15.40:0/375513481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3100003bb0 0x7f31180c21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f31177fe700 1 -- 172.21.15.40:0/375513481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31180ca260 msgr2=0x7f31181f6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:16.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f31177fe700 1 --2- 172.21.15.40:0/375513481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31180ca260 0x7f31181f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f31177fe700 1 -- 172.21.15.40:0/375513481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3118069e40 con 0x7f31180b14f0 2023-11-08T08:19:16.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f3117fff700 1 --2- 172.21.15.40:0/375513481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3100003bb0 0x7f31180c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:16.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f31177fe700 1 --2- 172.21.15.40:0/375513481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31180b14f0 0x7f31180c9d20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f310c00e700 tx=0x7f310c00eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:16.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f3114ff9700 1 -- 172.21.15.40:0/375513481 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c012590 con 0x7f31180b14f0 2023-11-08T08:19:16.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f3114ff9700 1 -- 172.21.15.40:0/375513481 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c0126f0 con 0x7f31180b14f0 2023-11-08T08:19:16.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f311e271700 1 -- 172.21.15.40:0/375513481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31180cc780 con 0x7f31180b14f0 2023-11-08T08:19:16.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f3114ff9700 1 -- 172.21.15.40:0/375513481 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c012a80 con 0x7f31180b14f0 2023-11-08T08:19:16.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.147+0000 7f311e271700 1 -- 172.21.15.40:0/375513481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311806a120 con 0x7f31180b14f0 2023-11-08T08:19:16.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.151+0000 7f311e271700 1 -- 172.21.15.40:0/375513481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31180726c0 con 0x7f31180b14f0 2023-11-08T08:19:16.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.151+0000 7f3114ff9700 1 -- 172.21.15.40:0/375513481 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f310c012be0 con 0x7f31180b14f0 2023-11-08T08:19:16.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.155+0000 7f3114ff9700 1 --2- 172.21.15.40:0/375513481 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3104064ee0 0x7f3104067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:16.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.155+0000 7f3114ff9700 1 -- 172.21.15.40:0/375513481 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f310c0821c0 con 0x7f31180b14f0 2023-11-08T08:19:16.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.155+0000 7f3117fff700 1 --2- 172.21.15.40:0/375513481 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3104064ee0 0x7f3104067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:16.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.155+0000 7f3117fff700 1 --2- 172.21.15.40:0/375513481 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3104064ee0 0x7f3104067390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3108015040 tx=0x7f3108009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:16.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.159+0000 7f3114ff9700 1 -- 172.21.15.40:0/375513481 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f310c04fea0 con 0x7f31180b14f0 2023-11-08T08:19:16.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.335+0000 7f311e271700 1 -- 172.21.15.40:0/375513481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31180c1710 con 0x7f31180b14f0 2023-11-08T08:19:16.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.339+0000 7f3114ff9700 1 -- 172.21.15.40:0/375513481 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19652 (secure 0 0 0) 0x7f310c0534c0 con 0x7f31180b14f0 2023-11-08T08:19:16.345 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:16.346 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":28,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:08.980158+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:19:16.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.339+0000 7f311e271700 1 -- 172.21.15.40:0/375513481 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3104064ee0 msgr2=0x7f3104067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:16.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.339+0000 7f311e271700 1 --2- 172.21.15.40:0/375513481 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3104064ee0 0x7f3104067390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3108015040 tx=0x7f3108009030 comp rx=0 tx=0).stop 2023-11-08T08:19:16.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.339+0000 7f311e271700 1 -- 172.21.15.40:0/375513481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31180b14f0 msgr2=0x7f31180c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:16.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.339+0000 7f311e271700 1 --2- 172.21.15.40:0/375513481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31180b14f0 0x7f31180c9d20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f310c00e700 tx=0x7f310c00eac0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.339+0000 7f311e271700 1 -- 172.21.15.40:0/375513481 shutdown_connections 2023-11-08T08:19:16.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.339+0000 7f311e271700 1 --2- 172.21.15.40:0/375513481 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3104064ee0 0x7f3104067390 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.339+0000 7f311e271700 1 --2- 172.21.15.40:0/375513481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31180ca260 0x7f31181f6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.339+0000 7f311e271700 1 --2- 172.21.15.40:0/375513481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31180b14f0 0x7f31180c9d20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.339+0000 7f311e271700 1 --2- 172.21.15.40:0/375513481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3100003bb0 0x7f31180c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.339+0000 7f311e271700 1 -- 172.21.15.40:0/375513481 >> 172.21.15.40:0/375513481 conn(0x7f31180cf350 msgr2=0x7f3118162ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:16.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.343+0000 7f311e271700 1 -- 172.21.15.40:0/375513481 shutdown_connections 2023-11-08T08:19:16.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.343+0000 7f311e271700 1 -- 172.21.15.40:0/375513481 wait complete. 2023-11-08T08:19:16.371 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T08:19:02.627691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T08:19:03.473821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T08:19:06.539942+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T08:19:16.372 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:19:16.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.547+0000 7fb0a674f700 1 Processor -- start 2023-11-08T08:19:16.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.551+0000 7fb0a674f700 1 -- start start 2023-11-08T08:19:16.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.551+0000 7fb0a674f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb0a0070a90 0x7fb0a00c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:16.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.551+0000 7fb0a674f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a0070600 con 0x7fb0a015a9f0 2023-11-08T08:19:16.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.551+0000 7fb0a674f700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a015adf0 con 0x7fb0a0159f10 2023-11-08T08:19:16.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.551+0000 7fb0a674f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a00afcc0 con 0x7fb0a0070a90 2023-11-08T08:19:16.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.551+0000 7fb0a4f4c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb0a0070a90 0x7fb0a00c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:16.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.551+0000 7fb0a5f4e700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb0a015a9f0 0x7fb0a006be80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51478/0 (socket says 172.21.15.40:51478) 2023-11-08T08:19:16.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.551+0000 7fb0a5f4e700 1 -- 172.21.15.40:0/2247789299 learned_addr learned my addr 172.21.15.40:0/2247789299 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:16.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.551+0000 7fb0a574d700 1 --1- 172.21.15.40:0/2247789299 >> v1:172.21.15.169:6789/0 conn(0x7fb0a0159f10 0x7fb0a015a2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:58646/0 (socket says 172.21.15.40:58646) 2023-11-08T08:19:16.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.555+0000 7fb097fff700 1 -- 172.21.15.40:0/2247789299 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2046346930 0 0) 0x7fb0a015adf0 con 0x7fb0a0159f10 2023-11-08T08:19:16.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.555+0000 7fb0a4f4c700 1 -- 172.21.15.40:0/2247789299 >> v1:172.21.15.169:6789/0 conn(0x7fb0a0159f10 legacy=0x7fb0a015a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:16.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.555+0000 7fb0a4f4c700 1 -- 172.21.15.40:0/2247789299 >> v1:172.21.15.40:6789/0 conn(0x7fb0a015a9f0 legacy=0x7fb0a006be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:16.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.555+0000 7fb0a4f4c700 1 -- 172.21.15.40:0/2247789299 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a006ffb0 con 0x7fb0a0070a90 2023-11-08T08:19:16.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.555+0000 7fb0a4f4c700 1 --2- 172.21.15.40:0/2247789299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0a0070a90 0x7fb0a00c8c90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb090009850 tx=0x7fb090009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb3d0e7c5e3e8ca server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:16.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.555+0000 7fb097fff700 1 -- 172.21.15.40:0/2247789299 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090011830 con 0x7fb0a0070a90 2023-11-08T08:19:16.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.555+0000 7fb097fff700 1 -- 172.21.15.40:0/2247789299 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb090011990 con 0x7fb0a0070a90 2023-11-08T08:19:16.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.555+0000 7fb097fff700 1 -- 172.21.15.40:0/2247789299 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090011d20 con 0x7fb0a0070a90 2023-11-08T08:19:16.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.555+0000 7fb0a674f700 1 -- 172.21.15.40:0/2247789299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0a0070a90 msgr2=0x7fb0a00c8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:16.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.555+0000 7fb0a674f700 1 --2- 172.21.15.40:0/2247789299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0a0070a90 0x7fb0a00c8c90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb090009850 tx=0x7fb090009b60 comp rx=0 tx=0).stop 2023-11-08T08:19:16.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.555+0000 7fb0a674f700 1 -- 172.21.15.40:0/2247789299 shutdown_connections 2023-11-08T08:19:16.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.555+0000 7fb0a674f700 1 --2- 172.21.15.40:0/2247789299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0a0070a90 0x7fb0a00c8c90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.555+0000 7fb0a674f700 1 -- 172.21.15.40:0/2247789299 >> 172.21.15.40:0/2247789299 conn(0x7fb0a00cf280 msgr2=0x7fb0a00cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:16.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.555+0000 7fb0a674f700 1 -- 172.21.15.40:0/2247789299 shutdown_connections 2023-11-08T08:19:16.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.555+0000 7fb0a674f700 1 -- 172.21.15.40:0/2247789299 wait complete. 2023-11-08T08:19:16.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.559+0000 7fb0a674f700 1 Processor -- start 2023-11-08T08:19:16.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.559+0000 7fb0a674f700 1 -- start start 2023-11-08T08:19:16.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a674f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0a0070a90 0x7fb0a014e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:16.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a674f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0a0159f10 0x7fb0a00b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:16.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a674f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0a015a9f0 0x7fb0a01f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:16.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a674f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00b46c0 con 0x7fb0a0159f10 2023-11-08T08:19:16.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a674f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a014e6e0 con 0x7fb0a0070a90 2023-11-08T08:19:16.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a674f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00b3e20 con 0x7fb0a015a9f0 2023-11-08T08:19:16.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a4f4c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0a0159f10 0x7fb0a00b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:16.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a4f4c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0a0159f10 0x7fb0a00b1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43464/0 (socket says 172.21.15.40:43464) 2023-11-08T08:19:16.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a4f4c700 1 -- 172.21.15.40:0/4217251899 learned_addr learned my addr 172.21.15.40:0/4217251899 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:16.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a574d700 1 --2- 172.21.15.40:0/4217251899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0a0070a90 0x7fb0a014e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:16.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a5f4e700 1 --2- 172.21.15.40:0/4217251899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0a015a9f0 0x7fb0a01f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:16.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a4f4c700 1 -- 172.21.15.40:0/4217251899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0a015a9f0 msgr2=0x7fb0a01f4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:16.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a4f4c700 1 --2- 172.21.15.40:0/4217251899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0a015a9f0 0x7fb0a01f4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a4f4c700 1 -- 172.21.15.40:0/4217251899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0a0070a90 msgr2=0x7fb0a014e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:16.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a4f4c700 1 --2- 172.21.15.40:0/4217251899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0a0070a90 0x7fb0a014e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a4f4c700 1 -- 172.21.15.40:0/4217251899 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0900094b0 con 0x7fb0a0159f10 2023-11-08T08:19:16.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a5f4e700 1 --2- 172.21.15.40:0/4217251899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0a015a9f0 0x7fb0a01f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:16.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a574d700 1 --2- 172.21.15.40:0/4217251899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0a0070a90 0x7fb0a014e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:16.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a4f4c700 1 --2- 172.21.15.40:0/4217251899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0a0159f10 0x7fb0a00b1540 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00717e0 tx=0x7fb090004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:16.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb095ffb700 1 -- 172.21.15.40:0/4217251899 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09000bbd0 con 0x7fb0a0159f10 2023-11-08T08:19:16.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb095ffb700 1 -- 172.21.15.40:0/4217251899 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09001f070 con 0x7fb0a0159f10 2023-11-08T08:19:16.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a674f700 1 -- 172.21.15.40:0/4217251899 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a00affa0 con 0x7fb0a0159f10 2023-11-08T08:19:16.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb095ffb700 1 -- 172.21.15.40:0/4217251899 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09001a430 con 0x7fb0a0159f10 2023-11-08T08:19:16.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.563+0000 7fb0a674f700 1 -- 172.21.15.40:0/4217251899 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a0070290 con 0x7fb0a0159f10 2023-11-08T08:19:16.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.567+0000 7fb0a674f700 1 -- 172.21.15.40:0/4217251899 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a006b300 con 0x7fb0a0159f10 2023-11-08T08:19:16.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.567+0000 7fb095ffb700 1 -- 172.21.15.40:0/4217251899 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fb09000f050 con 0x7fb0a0159f10 2023-11-08T08:19:16.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.567+0000 7fb095ffb700 1 --2- 172.21.15.40:0/4217251899 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb084064ee0 0x7fb084067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:16.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.567+0000 7fb095ffb700 1 -- 172.21.15.40:0/4217251899 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb090021070 con 0x7fb0a0159f10 2023-11-08T08:19:16.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.567+0000 7fb0a574d700 1 --2- 172.21.15.40:0/4217251899 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb084064ee0 0x7fb084067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:16.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.571+0000 7fb0a574d700 1 --2- 172.21.15.40:0/4217251899 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb084064ee0 0x7fb084067390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0012e0 tx=0x7fb09c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.575+0000 7fb095ffb700 1 -- 172.21.15.40:0/4217251899 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fb0900518d0 con 0x7fb0a0159f10 2023-11-08T08:19:16.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.747+0000 7fb0a674f700 1 -- 172.21.15.40:0/4217251899 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0a006cf30 con 0x7fb0a0159f10 2023-11-08T08:19:16.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.747+0000 7fb095ffb700 1 -- 172.21.15.40:0/4217251899 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb090054ef0 con 0x7fb0a0159f10 2023-11-08T08:19:16.753 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:19:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.747+0000 7fb0a674f700 1 -- 172.21.15.40:0/4217251899 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb084064ee0 msgr2=0x7fb084067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.747+0000 7fb0a674f700 1 --2- 172.21.15.40:0/4217251899 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb084064ee0 0x7fb084067390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0012e0 tx=0x7fb09c009040 comp rx=0 tx=0).stop 2023-11-08T08:19:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.747+0000 7fb0a674f700 1 -- 172.21.15.40:0/4217251899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0a0159f10 msgr2=0x7fb0a00b1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:16.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.747+0000 7fb0a674f700 1 --2- 172.21.15.40:0/4217251899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0a0159f10 0x7fb0a00b1540 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00717e0 tx=0x7fb090004770 comp rx=0 tx=0).stop 2023-11-08T08:19:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.747+0000 7fb0a674f700 1 -- 172.21.15.40:0/4217251899 shutdown_connections 2023-11-08T08:19:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.747+0000 7fb0a674f700 1 --2- 172.21.15.40:0/4217251899 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb084064ee0 0x7fb084067390 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.747+0000 7fb0a674f700 1 --2- 172.21.15.40:0/4217251899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0a015a9f0 0x7fb0a01f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.747+0000 7fb0a674f700 1 --2- 172.21.15.40:0/4217251899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0a0159f10 0x7fb0a00b1540 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.747+0000 7fb0a674f700 1 --2- 172.21.15.40:0/4217251899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0a0070a90 0x7fb0a014e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.747+0000 7fb0a674f700 1 -- 172.21.15.40:0/4217251899 >> 172.21.15.40:0/4217251899 conn(0x7fb0a00cf280 msgr2=0x7fb0a00c8870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:16.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.751+0000 7fb0a674f700 1 -- 172.21.15.40:0/4217251899 shutdown_connections 2023-11-08T08:19:16.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.751+0000 7fb0a674f700 1 -- 172.21.15.40:0/4217251899 wait complete. 2023-11-08T08:19:16.778 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:19:16.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.963+0000 7fb5989b9700 1 Processor -- start 2023-11-08T08:19:16.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.963+0000 7fb5989b9700 1 -- start start 2023-11-08T08:19:16.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb5989b9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb594070620 0x7fb5940b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:16.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb5989b9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb594069bf0 0x7fb59406a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:16.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb5989b9700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59415c520 con 0x7fb59415c150 2023-11-08T08:19:16.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb5989b9700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5940b4240 con 0x7fb594070620 2023-11-08T08:19:16.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb5989b9700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59406cc50 con 0x7fb594069bf0 2023-11-08T08:19:16.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb592d9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb59415c150 0x7fb59406ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51494/0 (socket says 172.21.15.40:51494) 2023-11-08T08:19:16.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb592d9d700 1 -- 172.21.15.40:0/1182247922 learned_addr learned my addr 172.21.15.40:0/1182247922 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:16.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb59259c700 1 --2- 172.21.15.40:0/1182247922 >> v2:172.21.15.40:3300/0 conn(0x7fb594070620 0x7fb5940b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:16.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb59359e700 1 --2- 172.21.15.40:0/1182247922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb594069bf0 0x7fb59406a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:16.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb591d9b700 1 -- 172.21.15.40:0/1182247922 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2501834548 0 0) 0x7fb59415c520 con 0x7fb59415c150 2023-11-08T08:19:16.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb591d9b700 1 -- 172.21.15.40:0/1182247922 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb57c0035d0 con 0x7fb59415c150 2023-11-08T08:19:16.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb59259c700 1 -- 172.21.15.40:0/1182247922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb594069bf0 msgr2=0x7fb59406a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:16.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb59259c700 1 --2- 172.21.15.40:0/1182247922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb594069bf0 0x7fb59406a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb59259c700 1 -- 172.21.15.40:0/1182247922 >> v1:172.21.15.40:6789/0 conn(0x7fb59415c150 legacy=0x7fb59406ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:16.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb59259c700 1 -- 172.21.15.40:0/1182247922 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5940d2450 con 0x7fb594070620 2023-11-08T08:19:16.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb59259c700 1 --2- 172.21.15.40:0/1182247922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb594070620 0x7fb5940b3d00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb588009840 tx=0x7fb588009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f734f2288d3c88ea server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:16.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb591d9b700 1 -- 172.21.15.40:0/1182247922 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588005e50 con 0x7fb594070620 2023-11-08T08:19:16.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb591d9b700 1 -- 172.21.15.40:0/1182247922 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb588006030 con 0x7fb594070620 2023-11-08T08:19:16.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb591d9b700 1 -- 172.21.15.40:0/1182247922 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5880063c0 con 0x7fb594070620 2023-11-08T08:19:16.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb5989b9700 1 -- 172.21.15.40:0/1182247922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb594070620 msgr2=0x7fb5940b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:16.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.967+0000 7fb5989b9700 1 --2- 172.21.15.40:0/1182247922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb594070620 0x7fb5940b3d00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb588009840 tx=0x7fb588009b50 comp rx=0 tx=0).stop 2023-11-08T08:19:16.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.971+0000 7fb5989b9700 1 -- 172.21.15.40:0/1182247922 shutdown_connections 2023-11-08T08:19:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.971+0000 7fb5989b9700 1 --2- 172.21.15.40:0/1182247922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb594069bf0 0x7fb59406a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.971+0000 7fb5989b9700 1 --2- 172.21.15.40:0/1182247922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb594070620 0x7fb5940b3d00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.971+0000 7fb5989b9700 1 -- 172.21.15.40:0/1182247922 >> 172.21.15.40:0/1182247922 conn(0x7fb5940d3680 msgr2=0x7fb5940cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.971+0000 7fb5989b9700 1 -- 172.21.15.40:0/1182247922 shutdown_connections 2023-11-08T08:19:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.971+0000 7fb5989b9700 1 -- 172.21.15.40:0/1182247922 wait complete. 2023-11-08T08:19:16.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.971+0000 7fb5989b9700 1 Processor -- start 2023-11-08T08:19:16.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.971+0000 7fb5989b9700 1 -- start start 2023-11-08T08:19:16.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.971+0000 7fb5989b9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb594069bf0 0x7fb5940cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:16.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.971+0000 7fb5989b9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb594070620 0x7fb5941443d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:16.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.975+0000 7fb5989b9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb59415c150 0x7fb5940b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:16.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.975+0000 7fb5989b9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59414afe0 con 0x7fb59415c150 2023-11-08T08:19:16.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.975+0000 7fb5989b9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb594157fe0 con 0x7fb594069bf0 2023-11-08T08:19:16.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.975+0000 7fb5989b9700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5941582c0 con 0x7fb594070620 2023-11-08T08:19:16.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.975+0000 7fb59359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb59415c150 0x7fb5940b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:16.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.975+0000 7fb592d9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb594069bf0 0x7fb5940cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:16.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.975+0000 7fb59259c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb594070620 0x7fb5941443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:16.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.975+0000 7fb592d9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb594069bf0 0x7fb5940cf8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:53524/0 (socket says 172.21.15.40:53524) 2023-11-08T08:19:16.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.975+0000 7fb592d9d700 1 -- 172.21.15.40:0/308409353 learned_addr learned my addr 172.21.15.40:0/308409353 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:16.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.975+0000 7fb592d9d700 1 -- 172.21.15.40:0/308409353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb594070620 msgr2=0x7fb5941443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:16.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.975+0000 7fb592d9d700 1 --2- 172.21.15.40:0/308409353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb594070620 0x7fb5941443d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.975+0000 7fb592d9d700 1 -- 172.21.15.40:0/308409353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb59415c150 msgr2=0x7fb5940b6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.975+0000 7fb592d9d700 1 --2- 172.21.15.40:0/308409353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb59415c150 0x7fb5940b6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.975+0000 7fb592d9d700 1 -- 172.21.15.40:0/308409353 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb588009450 con 0x7fb594069bf0 2023-11-08T08:19:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.975+0000 7fb592d9d700 1 --2- 172.21.15.40:0/308409353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb594069bf0 0x7fb5940cf8f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb58400c4d0 tx=0x7fb58400c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:16.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.975+0000 7fb57b7fe700 1 -- 172.21.15.40:0/308409353 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb584005d90 con 0x7fb594069bf0 2023-11-08T08:19:16.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.975+0000 7fb57b7fe700 1 -- 172.21.15.40:0/308409353 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5840070d0 con 0x7fb594069bf0 2023-11-08T08:19:16.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.975+0000 7fb5989b9700 1 -- 172.21.15.40:0/308409353 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5940d05e0 con 0x7fb594069bf0 2023-11-08T08:19:16.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.975+0000 7fb57b7fe700 1 -- 172.21.15.40:0/308409353 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb584007480 con 0x7fb594069bf0 2023-11-08T08:19:16.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.975+0000 7fb5989b9700 1 -- 172.21.15.40:0/308409353 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5940d2730 con 0x7fb594069bf0 2023-11-08T08:19:16.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.979+0000 7fb5989b9700 1 -- 172.21.15.40:0/308409353 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5940d2140 con 0x7fb594069bf0 2023-11-08T08:19:16.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.979+0000 7fb57b7fe700 1 -- 172.21.15.40:0/308409353 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fb584019030 con 0x7fb594069bf0 2023-11-08T08:19:16.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.979+0000 7fb57b7fe700 1 --2- 172.21.15.40:0/308409353 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb580064ee0 0x7fb580067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:16.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.979+0000 7fb57b7fe700 1 -- 172.21.15.40:0/308409353 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb584081490 con 0x7fb594069bf0 2023-11-08T08:19:16.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.979+0000 7fb59259c700 1 --2- 172.21.15.40:0/308409353 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb580064ee0 0x7fb580067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:16.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.983+0000 7fb59259c700 1 --2- 172.21.15.40:0/308409353 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb580064ee0 0x7fb580067390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb594070a10 tx=0x7fb588008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:16.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:16.991+0000 7fb57b7fe700 1 -- 172.21.15.40:0/308409353 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fb58404f340 con 0x7fb594069bf0 2023-11-08T08:19:17.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.175+0000 7fb5989b9700 1 -- 172.21.15.40:0/308409353 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb594073d20 con 0x7fb594069bf0 2023-11-08T08:19:17.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.175+0000 7fb57b7fe700 1 -- 172.21.15.40:0/308409353 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb584052960 con 0x7fb594069bf0 2023-11-08T08:19:17.184 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:19:17.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.179+0000 7fb5989b9700 1 -- 172.21.15.40:0/308409353 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb580064ee0 msgr2=0x7fb580067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:17.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.179+0000 7fb5989b9700 1 --2- 172.21.15.40:0/308409353 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb580064ee0 0x7fb580067390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb594070a10 tx=0x7fb588008070 comp rx=0 tx=0).stop 2023-11-08T08:19:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.179+0000 7fb5989b9700 1 -- 172.21.15.40:0/308409353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb594069bf0 msgr2=0x7fb5940cf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.179+0000 7fb5989b9700 1 --2- 172.21.15.40:0/308409353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb594069bf0 0x7fb5940cf8f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb58400c4d0 tx=0x7fb58400c7e0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.179+0000 7fb5989b9700 1 -- 172.21.15.40:0/308409353 shutdown_connections 2023-11-08T08:19:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.179+0000 7fb5989b9700 1 --2- 172.21.15.40:0/308409353 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb580064ee0 0x7fb580067390 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.179+0000 7fb5989b9700 1 --2- 172.21.15.40:0/308409353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb59415c150 0x7fb5940b6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.179+0000 7fb5989b9700 1 --2- 172.21.15.40:0/308409353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb594070620 0x7fb5941443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.179+0000 7fb5989b9700 1 --2- 172.21.15.40:0/308409353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb594069bf0 0x7fb5940cf8f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.179+0000 7fb5989b9700 1 -- 172.21.15.40:0/308409353 >> 172.21.15.40:0/308409353 conn(0x7fb5940d3680 msgr2=0x7fb594157270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:17.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.179+0000 7fb5989b9700 1 -- 172.21.15.40:0/308409353 shutdown_connections 2023-11-08T08:19:17.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.179+0000 7fb5989b9700 1 -- 172.21.15.40:0/308409353 wait complete. 2023-11-08T08:19:17.208 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T08:19:17.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.399+0000 7ff196cd1700 1 Processor -- start 2023-11-08T08:19:17.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.399+0000 7ff196cd1700 1 -- start start 2023-11-08T08:19:17.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff196cd1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff19015c150 0x7ff1900b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:17.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff196cd1700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff19006ff10 0x7ff190070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:17.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff196cd1700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff19015fc30 0x7ff190160090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:17.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff196cd1700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1900b4240 con 0x7ff19015c150 2023-11-08T08:19:17.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff196cd1700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff19015c520 con 0x7ff19006ff10 2023-11-08T08:19:17.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff196cd1700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900708a0 con 0x7ff19015fc30 2023-11-08T08:19:17.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff195ccf700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff19015c150 0x7ff1900b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:17.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff1954ce700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff19006ff10 0x7ff190070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:17.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff195ccf700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff19015c150 0x7ff1900b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35940/0 (socket says 172.21.15.40:35940) 2023-11-08T08:19:17.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff195ccf700 1 -- 172.21.15.40:0/3308628185 learned_addr learned my addr 172.21.15.40:0/3308628185 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:17.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff1964d0700 1 --2- 172.21.15.40:0/3308628185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff19015fc30 0x7ff190160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:17.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff195ccf700 1 -- 172.21.15.40:0/3308628185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff19015fc30 msgr2=0x7ff190160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:17.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff195ccf700 1 --2- 172.21.15.40:0/3308628185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff19015fc30 0x7ff190160090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff195ccf700 1 -- 172.21.15.40:0/3308628185 >> v2:172.21.15.169:3300/0 conn(0x7ff19006ff10 msgr2=0x7ff190070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:17.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff195ccf700 1 --2- 172.21.15.40:0/3308628185 >> v2:172.21.15.169:3300/0 conn(0x7ff19006ff10 0x7ff190070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff195ccf700 1 -- 172.21.15.40:0/3308628185 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19006cc50 con 0x7ff19015c150 2023-11-08T08:19:17.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff1954ce700 1 --2- 172.21.15.40:0/3308628185 >> v2:172.21.15.169:3300/0 conn(0x7ff19006ff10 0x7ff190070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:17.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff195ccf700 1 --2- 172.21.15.40:0/3308628185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff19015c150 0x7ff1900b3d00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff190066d30 tx=0x7ff18c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9259b4c83572fb79 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:17.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff194ccd700 1 -- 172.21.15.40:0/3308628185 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c011830 con 0x7ff19015c150 2023-11-08T08:19:17.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff194ccd700 1 -- 172.21.15.40:0/3308628185 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18c011990 con 0x7ff19015c150 2023-11-08T08:19:17.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff194ccd700 1 -- 172.21.15.40:0/3308628185 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c011d40 con 0x7ff19015c150 2023-11-08T08:19:17.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff196cd1700 1 -- 172.21.15.40:0/3308628185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff19015c150 msgr2=0x7ff1900b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:17.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.403+0000 7ff196cd1700 1 --2- 172.21.15.40:0/3308628185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff19015c150 0x7ff1900b3d00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff190066d30 tx=0x7ff18c009c20 comp rx=0 tx=0).stop 2023-11-08T08:19:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.407+0000 7ff196cd1700 1 -- 172.21.15.40:0/3308628185 shutdown_connections 2023-11-08T08:19:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.407+0000 7ff196cd1700 1 --2- 172.21.15.40:0/3308628185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff19015fc30 0x7ff190160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.407+0000 7ff196cd1700 1 --2- 172.21.15.40:0/3308628185 >> v2:172.21.15.169:3300/0 conn(0x7ff19006ff10 0x7ff190070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.407+0000 7ff196cd1700 1 --2- 172.21.15.40:0/3308628185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff19015c150 0x7ff1900b3d00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.407+0000 7ff196cd1700 1 -- 172.21.15.40:0/3308628185 >> 172.21.15.40:0/3308628185 conn(0x7ff1900d3680 msgr2=0x7ff1900cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:17.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.407+0000 7ff196cd1700 1 -- 172.21.15.40:0/3308628185 shutdown_connections 2023-11-08T08:19:17.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.407+0000 7ff196cd1700 1 -- 172.21.15.40:0/3308628185 wait complete. 2023-11-08T08:19:17.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.407+0000 7ff196cd1700 1 Processor -- start 2023-11-08T08:19:17.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.407+0000 7ff196cd1700 1 -- start start 2023-11-08T08:19:17.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.407+0000 7ff196cd1700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff19006ff10 0x7ff19015d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:17.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.407+0000 7ff196cd1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19015c150 0x7ff19015e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:17.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.407+0000 7ff196cd1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff19015fc30 0x7ff190150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:17.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.407+0000 7ff196cd1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19015e560 con 0x7ff19015fc30 2023-11-08T08:19:17.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.407+0000 7ff196cd1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900ca410 con 0x7ff19015c150 2023-11-08T08:19:17.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.407+0000 7ff196cd1700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900c9ae0 con 0x7ff19006ff10 2023-11-08T08:19:17.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.411+0000 7ff1964d0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff19015fc30 0x7ff190150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:17.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.411+0000 7ff1964d0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff19015fc30 0x7ff190150360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35946/0 (socket says 172.21.15.40:35946) 2023-11-08T08:19:17.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.411+0000 7ff1964d0700 1 -- 172.21.15.40:0/318230957 learned_addr learned my addr 172.21.15.40:0/318230957 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:17.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.411+0000 7ff195ccf700 1 --2- 172.21.15.40:0/318230957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff19006ff10 0x7ff19015d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:17.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.411+0000 7ff1954ce700 1 --2- 172.21.15.40:0/318230957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19015c150 0x7ff19015e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:17.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.411+0000 7ff1964d0700 1 -- 172.21.15.40:0/318230957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff19006ff10 msgr2=0x7ff19015d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:17.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.411+0000 7ff1964d0700 1 --2- 172.21.15.40:0/318230957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff19006ff10 0x7ff19015d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.411+0000 7ff1964d0700 1 -- 172.21.15.40:0/318230957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19015c150 msgr2=0x7ff19015e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:17.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.411+0000 7ff1964d0700 1 --2- 172.21.15.40:0/318230957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19015c150 0x7ff19015e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.411+0000 7ff1964d0700 1 -- 172.21.15.40:0/318230957 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff180009450 con 0x7ff19015fc30 2023-11-08T08:19:17.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.411+0000 7ff1964d0700 1 --2- 172.21.15.40:0/318230957 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff19015fc30 0x7ff190150360 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff18800efc0 tx=0x7ff18800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:17.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.411+0000 7ff1867fc700 1 -- 172.21.15.40:0/318230957 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18800cad0 con 0x7ff19015fc30 2023-11-08T08:19:17.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.411+0000 7ff1867fc700 1 -- 172.21.15.40:0/318230957 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18800f040 con 0x7ff19015fc30 2023-11-08T08:19:17.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.411+0000 7ff1867fc700 1 -- 172.21.15.40:0/318230957 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18800ce70 con 0x7ff19015fc30 2023-11-08T08:19:17.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.411+0000 7ff196cd1700 1 -- 172.21.15.40:0/318230957 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18c009570 con 0x7ff19015fc30 2023-11-08T08:19:17.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.411+0000 7ff196cd1700 1 -- 172.21.15.40:0/318230957 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1900d2730 con 0x7ff19015fc30 2023-11-08T08:19:17.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.415+0000 7ff196cd1700 1 -- 172.21.15.40:0/318230957 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1900d2450 con 0x7ff19015fc30 2023-11-08T08:19:17.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.415+0000 7ff1867fc700 1 -- 172.21.15.40:0/318230957 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff188014480 con 0x7ff19015fc30 2023-11-08T08:19:17.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.419+0000 7ff1867fc700 1 --2- 172.21.15.40:0/318230957 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff174064ee0 0x7ff174067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:17.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.419+0000 7ff1867fc700 1 -- 172.21.15.40:0/318230957 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff188082b30 con 0x7ff19015fc30 2023-11-08T08:19:17.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.419+0000 7ff195ccf700 1 --2- 172.21.15.40:0/318230957 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff174064ee0 0x7ff174067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:17.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.419+0000 7ff195ccf700 1 --2- 172.21.15.40:0/318230957 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff174064ee0 0x7ff174067390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009860 tx=0x7ff18c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:17.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.423+0000 7ff1867fc700 1 -- 172.21.15.40:0/318230957 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ff188050890 con 0x7ff19015fc30 2023-11-08T08:19:17.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.607+0000 7ff196cd1700 1 -- 172.21.15.40:0/318230957 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1900d05e0 con 0x7ff19015fc30 2023-11-08T08:19:17.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.611+0000 7ff1867fc700 1 -- 172.21.15.40:0/318230957 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff188053eb0 con 0x7ff19015fc30 2023-11-08T08:19:17.616 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:19:17.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.615+0000 7ff196cd1700 1 -- 172.21.15.40:0/318230957 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff174064ee0 msgr2=0x7ff174067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:17.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.615+0000 7ff196cd1700 1 --2- 172.21.15.40:0/318230957 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff174064ee0 0x7ff174067390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009860 tx=0x7ff18c008030 comp rx=0 tx=0).stop 2023-11-08T08:19:17.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.615+0000 7ff196cd1700 1 -- 172.21.15.40:0/318230957 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff19015fc30 msgr2=0x7ff190150360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:17.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.615+0000 7ff196cd1700 1 --2- 172.21.15.40:0/318230957 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff19015fc30 0x7ff190150360 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff18800efc0 tx=0x7ff18800c0c0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.615+0000 7ff196cd1700 1 -- 172.21.15.40:0/318230957 shutdown_connections 2023-11-08T08:19:17.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.615+0000 7ff196cd1700 1 --2- 172.21.15.40:0/318230957 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff174064ee0 0x7ff174067390 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.615+0000 7ff196cd1700 1 --2- 172.21.15.40:0/318230957 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff19015fc30 0x7ff190150360 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.615+0000 7ff196cd1700 1 --2- 172.21.15.40:0/318230957 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19015c150 0x7ff19015e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.615+0000 7ff196cd1700 1 --2- 172.21.15.40:0/318230957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff19006ff10 0x7ff19015d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.615+0000 7ff196cd1700 1 -- 172.21.15.40:0/318230957 >> 172.21.15.40:0/318230957 conn(0x7ff1900d3680 msgr2=0x7ff190155cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:17.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.615+0000 7ff196cd1700 1 -- 172.21.15.40:0/318230957 shutdown_connections 2023-11-08T08:19:17.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.615+0000 7ff196cd1700 1 -- 172.21.15.40:0/318230957 wait complete. 2023-11-08T08:19:17.644 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T08:19:17.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.831+0000 7ff4019b4700 1 Processor -- start 2023-11-08T08:19:17.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.831+0000 7ff4019b4700 1 -- start start 2023-11-08T08:19:17.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.831+0000 7ff4019b4700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3fc070a90 0x7ff3fc0c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:17.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.831+0000 7ff4019b4700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff3fc15a9f0 0x7ff3fc06b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:17.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.831+0000 7ff4019b4700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3fc070600 con 0x7ff3fc159f10 2023-11-08T08:19:17.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.831+0000 7ff4019b4700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3fc070f00 con 0x7ff3fc15a9f0 2023-11-08T08:19:17.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.831+0000 7ff4019b4700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc06b840 con 0x7ff3fc070a90 2023-11-08T08:19:17.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.831+0000 7ff4009b2700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff3fc159f10 0x7ff3fc15a2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33204/0 (socket says 172.21.15.40:33204) 2023-11-08T08:19:17.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.831+0000 7ff4009b2700 1 -- 172.21.15.40:0/3247049532 learned_addr learned my addr 172.21.15.40:0/3247049532 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:17.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.831+0000 7ff3f3fff700 1 --2- 172.21.15.40:0/3247049532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3fc070a90 0x7ff3fc0c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:17.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.831+0000 7ff4011b3700 1 --2- 172.21.15.40:0/3247049532 >> v2:172.21.15.169:3300/0 conn(0x7ff3fc15a9f0 0x7ff3fc06b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:17.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.831+0000 7ff3f37fe700 1 -- 172.21.15.40:0/3247049532 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2790442796 0 0) 0x7ff3fc070600 con 0x7ff3fc159f10 2023-11-08T08:19:17.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.831+0000 7ff3f37fe700 1 -- 172.21.15.40:0/3247049532 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3e40035d0 con 0x7ff3fc159f10 2023-11-08T08:19:17.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.831+0000 7ff4011b3700 1 -- 172.21.15.40:0/3247049532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3fc070a90 msgr2=0x7ff3fc0c8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:17.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.831+0000 7ff4011b3700 1 --2- 172.21.15.40:0/3247049532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3fc070a90 0x7ff3fc0c8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.831+0000 7ff4011b3700 1 -- 172.21.15.40:0/3247049532 >> v1:172.21.15.40:6789/0 conn(0x7ff3fc159f10 legacy=0x7ff3fc15a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:17.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.831+0000 7ff4011b3700 1 -- 172.21.15.40:0/3247049532 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3fc15ae30 con 0x7ff3fc15a9f0 2023-11-08T08:19:17.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.835+0000 7ff4011b3700 1 --2- 172.21.15.40:0/3247049532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3fc15a9f0 0x7ff3fc06b300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009d90 tx=0x7ff3f400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bb5cc280a06e03b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:17.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.835+0000 7ff3f37fe700 1 -- 172.21.15.40:0/3247049532 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f400e070 con 0x7ff3fc15a9f0 2023-11-08T08:19:17.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.835+0000 7ff3f37fe700 1 -- 172.21.15.40:0/3247049532 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f400bb20 con 0x7ff3fc15a9f0 2023-11-08T08:19:17.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.835+0000 7ff3f37fe700 1 -- 172.21.15.40:0/3247049532 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4012430 con 0x7ff3fc15a9f0 2023-11-08T08:19:17.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.835+0000 7ff4019b4700 1 -- 172.21.15.40:0/3247049532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3fc15a9f0 msgr2=0x7ff3fc06b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:17.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.835+0000 7ff4019b4700 1 --2- 172.21.15.40:0/3247049532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3fc15a9f0 0x7ff3fc06b300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009d90 tx=0x7ff3f400b330 comp rx=0 tx=0).stop 2023-11-08T08:19:17.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.835+0000 7ff4019b4700 1 -- 172.21.15.40:0/3247049532 shutdown_connections 2023-11-08T08:19:17.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.835+0000 7ff4019b4700 1 --2- 172.21.15.40:0/3247049532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3fc15a9f0 0x7ff3fc06b300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.835+0000 7ff4019b4700 1 --2- 172.21.15.40:0/3247049532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3fc070a90 0x7ff3fc0c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.835+0000 7ff4019b4700 1 -- 172.21.15.40:0/3247049532 >> 172.21.15.40:0/3247049532 conn(0x7ff3fc0cf280 msgr2=0x7ff3fc0cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:17.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.835+0000 7ff4019b4700 1 -- 172.21.15.40:0/3247049532 shutdown_connections 2023-11-08T08:19:17.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.835+0000 7ff4019b4700 1 -- 172.21.15.40:0/3247049532 wait complete. 2023-11-08T08:19:17.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff4019b4700 1 Processor -- start 2023-11-08T08:19:17.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff4019b4700 1 -- start start 2023-11-08T08:19:17.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff4019b4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3fc070a90 0x7ff3fc0aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:17.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff4019b4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3fc159f10 0x7ff3fc155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:17.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff4019b4700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3fc15a9f0 0x7ff3fc156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:17.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff4019b4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc0c8440 con 0x7ff3fc159f10 2023-11-08T08:19:17.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff4019b4700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc0c7820 con 0x7ff3fc070a90 2023-11-08T08:19:17.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff4019b4700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc0af250 con 0x7ff3fc15a9f0 2023-11-08T08:19:17.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff3f3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3fc159f10 0x7ff3fc155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:17.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff3f3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3fc159f10 0x7ff3fc155b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35952/0 (socket says 172.21.15.40:35952) 2023-11-08T08:19:17.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff3f3fff700 1 -- 172.21.15.40:0/3217861030 learned_addr learned my addr 172.21.15.40:0/3217861030 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:17.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff4011b3700 1 --2- 172.21.15.40:0/3217861030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3fc15a9f0 0x7ff3fc156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:17.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff4009b2700 1 --2- 172.21.15.40:0/3217861030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3fc070a90 0x7ff3fc0aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:17.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff3f3fff700 1 -- 172.21.15.40:0/3217861030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3fc15a9f0 msgr2=0x7ff3fc156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:17.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff3f3fff700 1 --2- 172.21.15.40:0/3217861030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3fc15a9f0 0x7ff3fc156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff3f3fff700 1 -- 172.21.15.40:0/3217861030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3fc070a90 msgr2=0x7ff3fc0aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:17.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff3f3fff700 1 --2- 172.21.15.40:0/3217861030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3fc070a90 0x7ff3fc0aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:17.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff3f3fff700 1 -- 172.21.15.40:0/3217861030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f4009a70 con 0x7ff3fc159f10 2023-11-08T08:19:17.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff3f3fff700 1 --2- 172.21.15.40:0/3217861030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3fc159f10 0x7ff3fc155b30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00e9d0 tx=0x7ff3ec00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:17.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff3f17fa700 1 -- 172.21.15.40:0/3217861030 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec009720 con 0x7ff3fc159f10 2023-11-08T08:19:17.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff3f17fa700 1 -- 172.21.15.40:0/3217861030 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ec00c850 con 0x7ff3fc159f10 2023-11-08T08:19:17.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff4019b4700 1 -- 172.21.15.40:0/3217861030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3fc06ffb0 con 0x7ff3fc159f10 2023-11-08T08:19:17.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff3f17fa700 1 -- 172.21.15.40:0/3217861030 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec00cbc0 con 0x7ff3fc159f10 2023-11-08T08:19:17.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff4019b4700 1 -- 172.21.15.40:0/3217861030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3fc0afcc0 con 0x7ff3fc159f10 2023-11-08T08:19:17.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff4019b4700 1 -- 172.21.15.40:0/3217861030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3fc070290 con 0x7ff3fc159f10 2023-11-08T08:19:17.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.843+0000 7ff3f17fa700 1 -- 172.21.15.40:0/3217861030 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff3ec019030 con 0x7ff3fc159f10 2023-11-08T08:19:17.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.847+0000 7ff3f17fa700 1 --2- 172.21.15.40:0/3217861030 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff3e8064ee0 0x7ff3e8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:17.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.847+0000 7ff3f17fa700 1 -- 172.21.15.40:0/3217861030 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff3ec0822a0 con 0x7ff3fc159f10 2023-11-08T08:19:17.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.847+0000 7ff4009b2700 1 --2- 172.21.15.40:0/3217861030 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff3e8064ee0 0x7ff3e8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:17.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.847+0000 7ff4009b2700 1 --2- 172.21.15.40:0/3217861030 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff3e8064ee0 0x7ff3e8067390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80011e0 tx=0x7ff3f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:17.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:17.855+0000 7ff3f17fa700 1 -- 172.21.15.40:0/3217861030 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ff3ec050060 con 0x7ff3fc159f10 2023-11-08T08:19:18.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.027+0000 7ff4019b4700 1 -- 172.21.15.40:0/3217861030 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff3fc06cf30 con 0x7ff3fc159f10 2023-11-08T08:19:18.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.031+0000 7ff3f17fa700 1 -- 172.21.15.40:0/3217861030 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff3ec053680 con 0x7ff3fc159f10 2023-11-08T08:19:18.036 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-11-08T08:19:18.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.031+0000 7ff4019b4700 1 -- 172.21.15.40:0/3217861030 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff3e8064ee0 msgr2=0x7ff3e8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:18.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.031+0000 7ff4019b4700 1 --2- 172.21.15.40:0/3217861030 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff3e8064ee0 0x7ff3e8067390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80011e0 tx=0x7ff3f8009040 comp rx=0 tx=0).stop 2023-11-08T08:19:18.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.031+0000 7ff4019b4700 1 -- 172.21.15.40:0/3217861030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3fc159f10 msgr2=0x7ff3fc155b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:18.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.031+0000 7ff4019b4700 1 --2- 172.21.15.40:0/3217861030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3fc159f10 0x7ff3fc155b30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00e9d0 tx=0x7ff3ec00ed90 comp rx=0 tx=0).stop 2023-11-08T08:19:18.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.031+0000 7ff4019b4700 1 -- 172.21.15.40:0/3217861030 shutdown_connections 2023-11-08T08:19:18.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.031+0000 7ff4019b4700 1 --2- 172.21.15.40:0/3217861030 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff3e8064ee0 0x7ff3e8067390 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.031+0000 7ff4019b4700 1 --2- 172.21.15.40:0/3217861030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3fc15a9f0 0x7ff3fc156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.031+0000 7ff4019b4700 1 --2- 172.21.15.40:0/3217861030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3fc159f10 0x7ff3fc155b30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.031+0000 7ff4019b4700 1 --2- 172.21.15.40:0/3217861030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3fc070a90 0x7ff3fc0aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.031+0000 7ff4019b4700 1 -- 172.21.15.40:0/3217861030 >> 172.21.15.40:0/3217861030 conn(0x7ff3fc0cf280 msgr2=0x7ff3fc0c4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:18.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.031+0000 7ff4019b4700 1 -- 172.21.15.40:0/3217861030 shutdown_connections 2023-11-08T08:19:18.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.031+0000 7ff4019b4700 1 -- 172.21.15.40:0/3217861030 wait complete. 2023-11-08T08:19:18.061 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T08:19:18.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.243+0000 7f1de19ba700 1 Processor -- start 2023-11-08T08:19:18.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.243+0000 7f1de19ba700 1 -- start start 2023-11-08T08:19:18.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1de19ba700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1ddc15c150 0x7f1ddc0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:18.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1de19ba700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ddc070360 0x7f1ddc160340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:18.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1de19ba700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ddc15c520 con 0x7f1ddc06ff10 2023-11-08T08:19:18.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1de19ba700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ddc0b4240 con 0x7f1ddc15c150 2023-11-08T08:19:18.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1de19ba700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc06ac00 con 0x7f1ddc070360 2023-11-08T08:19:18.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1dda7fc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1ddc06ff10 0x7f1ddc15fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33206/0 (socket says 172.21.15.40:33206) 2023-11-08T08:19:18.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1dda7fc700 1 -- 172.21.15.40:0/2376305629 learned_addr learned my addr 172.21.15.40:0/2376305629 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:18.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1ddaffd700 1 --2- 172.21.15.40:0/2376305629 >> v2:172.21.15.169:3300/0 conn(0x7f1ddc15c150 0x7f1ddc0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:18.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1dd9ffb700 1 -- 172.21.15.40:0/2376305629 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2816874137 0 0) 0x7f1ddc15c520 con 0x7f1ddc06ff10 2023-11-08T08:19:18.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1dd9ffb700 1 -- 172.21.15.40:0/2376305629 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dc40035d0 con 0x7f1ddc06ff10 2023-11-08T08:19:18.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1ddb7fe700 1 --2- 172.21.15.40:0/2376305629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ddc070360 0x7f1ddc160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:18.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1dd9ffb700 1 -- 172.21.15.40:0/2376305629 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1234948565 0 0) 0x7f1dc40035d0 con 0x7f1ddc06ff10 2023-11-08T08:19:18.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1dd9ffb700 1 -- 172.21.15.40:0/2376305629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ddc070360 msgr2=0x7f1ddc160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:18.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1dd9ffb700 1 --2- 172.21.15.40:0/2376305629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ddc070360 0x7f1ddc160340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1dd9ffb700 1 -- 172.21.15.40:0/2376305629 >> v2:172.21.15.169:3300/0 conn(0x7f1ddc15c150 msgr2=0x7f1ddc0b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:18.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1dd9ffb700 1 --2- 172.21.15.40:0/2376305629 >> v2:172.21.15.169:3300/0 conn(0x7f1ddc15c150 0x7f1ddc0b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1dd9ffb700 1 -- 172.21.15.40:0/2376305629 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ddc06cc50 con 0x7f1ddc06ff10 2023-11-08T08:19:18.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1dd9ffb700 1 -- 172.21.15.40:0/2376305629 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f1dd00017f0 con 0x7f1ddc06ff10 2023-11-08T08:19:18.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1dd9ffb700 1 -- 172.21.15.40:0/2376305629 >> v1:172.21.15.40:6789/0 conn(0x7f1ddc06ff10 legacy=0x7f1ddc15fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:18.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1dd9ffb700 1 --2- 172.21.15.40:0/2376305629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc4003a20 0x7f1dc4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:18.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1dd9ffb700 1 --2- 172.21.15.40:0/2376305629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1dc4004450 0x7f1dc4004910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:18.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1dd9ffb700 1 --2- 172.21.15.40:0/2376305629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ddc070360 0x7f1dc4005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:18.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1dd9ffb700 1 -- 172.21.15.40:0/2376305629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc0b4240 con 0x7f1dc4003a20 2023-11-08T08:19:18.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1dd9ffb700 1 -- 172.21.15.40:0/2376305629 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc06ac00 con 0x7f1dc4004450 2023-11-08T08:19:18.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1dd9ffb700 1 -- 172.21.15.40:0/2376305629 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc400ba30 con 0x7f1ddc070360 2023-11-08T08:19:18.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1ddaffd700 1 --2- 172.21.15.40:0/2376305629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc4003a20 0x7f1dc4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:18.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1dda7fc700 1 --2- 172.21.15.40:0/2376305629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1dc4004450 0x7f1dc4004910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:18.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1ddb7fe700 1 --2- 172.21.15.40:0/2376305629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ddc070360 0x7f1dc4005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:18.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1ddaffd700 1 -- 172.21.15.40:0/2376305629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ddc070360 msgr2=0x7f1dc4005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:18.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1ddaffd700 1 --2- 172.21.15.40:0/2376305629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ddc070360 0x7f1dc4005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1ddaffd700 1 -- 172.21.15.40:0/2376305629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1dc4004450 msgr2=0x7f1dc4004910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:18.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1ddaffd700 1 --2- 172.21.15.40:0/2376305629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1dc4004450 0x7f1dc4004910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1ddaffd700 1 -- 172.21.15.40:0/2376305629 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc40035d0 con 0x7f1dc4003a20 2023-11-08T08:19:18.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.247+0000 7f1ddb7fe700 1 --2- 172.21.15.40:0/2376305629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ddc070360 0x7f1dc4005090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:18.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.251+0000 7f1ddaffd700 1 --2- 172.21.15.40:0/2376305629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc4003a20 0x7f1dc4003e30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc009b70 tx=0x7f1dc40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b35eeadb6bb4bb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:18.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.251+0000 7f1dd9ffb700 1 -- 172.21.15.40:0/2376305629 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc00e6c0 con 0x7f1dc4003a20 2023-11-08T08:19:18.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.251+0000 7f1dd9ffb700 1 -- 172.21.15.40:0/2376305629 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dcc00e820 con 0x7f1dc4003a20 2023-11-08T08:19:18.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.251+0000 7f1dd9ffb700 1 -- 172.21.15.40:0/2376305629 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc00eb90 con 0x7f1dc4003a20 2023-11-08T08:19:18.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.251+0000 7f1de19ba700 1 -- 172.21.15.40:0/2376305629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc4003a20 msgr2=0x7f1dc4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:18.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.251+0000 7f1de19ba700 1 --2- 172.21.15.40:0/2376305629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dc4003a20 0x7f1dc4003e30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc009b70 tx=0x7f1dc40058a0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.251+0000 7f1ddb7fe700 1 -- 172.21.15.40:0/2376305629 reap_dead start 2023-11-08T08:19:18.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.251+0000 7f1de19ba700 1 -- 172.21.15.40:0/2376305629 shutdown_connections 2023-11-08T08:19:18.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.251+0000 7f1de19ba700 1 -- 172.21.15.40:0/2376305629 >> 172.21.15.40:0/2376305629 conn(0x7f1ddc0d3680 msgr2=0x7f1ddc0cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:18.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.251+0000 7f1de19ba700 1 -- 172.21.15.40:0/2376305629 shutdown_connections 2023-11-08T08:19:18.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.251+0000 7f1de19ba700 1 -- 172.21.15.40:0/2376305629 wait complete. 2023-11-08T08:19:18.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.255+0000 7f1de19ba700 1 Processor -- start 2023-11-08T08:19:18.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.255+0000 7f1de19ba700 1 -- start start 2023-11-08T08:19:18.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.255+0000 7f1de19ba700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1dc4003a20 0x7f1ddc14cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:18.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.255+0000 7f1de19ba700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ddc1518f0 0x7f1ddc0b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.255+0000 7f1de19ba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ddc0b69e0 0x7f1ddc1534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.255+0000 7f1de19ba700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc0b6e80 con 0x7f1ddc0b69e0 2023-11-08T08:19:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.255+0000 7f1de19ba700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc0ca8a0 con 0x7f1ddc1518f0 2023-11-08T08:19:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.255+0000 7f1de19ba700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc0cbee0 con 0x7f1dc4003a20 2023-11-08T08:19:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.255+0000 7f1ddb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ddc0b69e0 0x7f1ddc1534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:18.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.255+0000 7f1ddb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ddc0b69e0 0x7f1ddc1534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35980/0 (socket says 172.21.15.40:35980) 2023-11-08T08:19:18.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.255+0000 7f1ddb7fe700 1 -- 172.21.15.40:0/3985604214 learned_addr learned my addr 172.21.15.40:0/3985604214 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:18.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.255+0000 7f1dda7fc700 1 --2- 172.21.15.40:0/3985604214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ddc1518f0 0x7f1ddc0b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:18.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.255+0000 7f1ddaffd700 1 --2- 172.21.15.40:0/3985604214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1dc4003a20 0x7f1ddc14cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:18.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.255+0000 7f1ddb7fe700 1 -- 172.21.15.40:0/3985604214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1dc4003a20 msgr2=0x7f1ddc14cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:18.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.255+0000 7f1ddb7fe700 1 --2- 172.21.15.40:0/3985604214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1dc4003a20 0x7f1ddc14cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.255+0000 7f1ddb7fe700 1 -- 172.21.15.40:0/3985604214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ddc1518f0 msgr2=0x7f1ddc0b64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:18.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.255+0000 7f1ddb7fe700 1 --2- 172.21.15.40:0/3985604214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ddc1518f0 0x7f1ddc0b64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.255+0000 7f1ddb7fe700 1 -- 172.21.15.40:0/3985604214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ddc0d2730 con 0x7f1ddc0b69e0 2023-11-08T08:19:18.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.255+0000 7f1dda7fc700 1 --2- 172.21.15.40:0/3985604214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ddc1518f0 0x7f1ddc0b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.255+0000 7f1ddb7fe700 1 --2- 172.21.15.40:0/3985604214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ddc0b69e0 0x7f1ddc1534e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc06afd0 tx=0x7f1dd40048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.259+0000 7f1dc3fff700 1 -- 172.21.15.40:0/3985604214 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd40078f0 con 0x7f1ddc0b69e0 2023-11-08T08:19:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.259+0000 7f1dc3fff700 1 -- 172.21.15.40:0/3985604214 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd400f050 con 0x7f1ddc0b69e0 2023-11-08T08:19:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.259+0000 7f1de19ba700 1 -- 172.21.15.40:0/3985604214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ddc0d2450 con 0x7f1ddc0b69e0 2023-11-08T08:19:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.259+0000 7f1dc3fff700 1 -- 172.21.15.40:0/3985604214 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4007cb0 con 0x7f1ddc0b69e0 2023-11-08T08:19:18.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.259+0000 7f1de19ba700 1 -- 172.21.15.40:0/3985604214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ddc0d05e0 con 0x7f1ddc0b69e0 2023-11-08T08:19:18.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.259+0000 7f1de19ba700 1 -- 172.21.15.40:0/3985604214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ddc073d20 con 0x7f1ddc0b69e0 2023-11-08T08:19:18.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.263+0000 7f1dc3fff700 1 -- 172.21.15.40:0/3985604214 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1dd401e460 con 0x7f1ddc0b69e0 2023-11-08T08:19:18.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.263+0000 7f1dc3fff700 1 --2- 172.21.15.40:0/3985604214 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1dc8064ee0 0x7f1dc8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:18.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.263+0000 7f1dc3fff700 1 -- 172.21.15.40:0/3985604214 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1dd40824e0 con 0x7f1ddc0b69e0 2023-11-08T08:19:18.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.263+0000 7f1ddaffd700 1 --2- 172.21.15.40:0/3985604214 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1dc8064ee0 0x7f1dc8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:18.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.263+0000 7f1ddaffd700 1 --2- 172.21.15.40:0/3985604214 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1dc8064ee0 0x7f1dc8067390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc0090b0 tx=0x7f1dcc01e040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:18.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.275+0000 7f1dc3fff700 1 -- 172.21.15.40:0/3985604214 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1dd40502d0 con 0x7f1ddc0b69e0 2023-11-08T08:19:18.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.451+0000 7f1de19ba700 1 -- 172.21.15.40:0/3985604214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1ddc0d0150 con 0x7f1ddc0b69e0 2023-11-08T08:19:18.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.451+0000 7f1dc3fff700 1 -- 172.21.15.40:0/3985604214 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1dd40538f0 con 0x7f1ddc0b69e0 2023-11-08T08:19:18.459 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:19:18.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.455+0000 7f1de19ba700 1 -- 172.21.15.40:0/3985604214 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1dc8064ee0 msgr2=0x7f1dc8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:18.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.455+0000 7f1de19ba700 1 --2- 172.21.15.40:0/3985604214 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1dc8064ee0 0x7f1dc8067390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc0090b0 tx=0x7f1dcc01e040 comp rx=0 tx=0).stop 2023-11-08T08:19:18.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.455+0000 7f1de19ba700 1 -- 172.21.15.40:0/3985604214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ddc0b69e0 msgr2=0x7f1ddc1534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:18.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.455+0000 7f1de19ba700 1 --2- 172.21.15.40:0/3985604214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ddc0b69e0 0x7f1ddc1534e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc06afd0 tx=0x7f1dd40048c0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.455+0000 7f1de19ba700 1 -- 172.21.15.40:0/3985604214 shutdown_connections 2023-11-08T08:19:18.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.455+0000 7f1de19ba700 1 --2- 172.21.15.40:0/3985604214 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1dc8064ee0 0x7f1dc8067390 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.455+0000 7f1de19ba700 1 --2- 172.21.15.40:0/3985604214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ddc0b69e0 0x7f1ddc1534e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.455+0000 7f1de19ba700 1 --2- 172.21.15.40:0/3985604214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ddc1518f0 0x7f1ddc0b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.455+0000 7f1de19ba700 1 --2- 172.21.15.40:0/3985604214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1dc4003a20 0x7f1ddc14cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.455+0000 7f1de19ba700 1 -- 172.21.15.40:0/3985604214 >> 172.21.15.40:0/3985604214 conn(0x7f1ddc0d3680 msgr2=0x7f1ddc1567f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:18.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.455+0000 7f1de19ba700 1 -- 172.21.15.40:0/3985604214 shutdown_connections 2023-11-08T08:19:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.455+0000 7f1de19ba700 1 -- 172.21.15.40:0/3985604214 wait complete. 2023-11-08T08:19:18.487 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T08:19:18.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.683+0000 7f17e2f90700 1 Processor -- start 2023-11-08T08:19:18.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.683+0000 7f17e2f90700 1 -- start start 2023-11-08T08:19:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.687+0000 7f17e2f90700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f17dc15c640 0x7f17dc0d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.687+0000 7f17e2f90700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17dc06c600 con 0x7f17dc071aa0 2023-11-08T08:19:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.687+0000 7f17e2f90700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17dc0722f0 con 0x7f17dc069e00 2023-11-08T08:19:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.687+0000 7f17e2f90700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17dc06cef0 con 0x7f17dc15c640 2023-11-08T08:19:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.687+0000 7f17e1f8e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f17dc15c640 0x7f17dc0d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.687+0000 7f17e178d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f17dc071aa0 0x7f17dc06d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33218/0 (socket says 172.21.15.40:33218) 2023-11-08T08:19:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.687+0000 7f17e178d700 1 -- 172.21.15.40:0/228165117 learned_addr learned my addr 172.21.15.40:0/228165117 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.687+0000 7f17e1f8e700 1 -- 172.21.15.40:0/228165117 >> v1:172.21.15.169:6789/0 conn(0x7f17dc069e00 legacy=0x7f17dc0d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.687+0000 7f17e1f8e700 1 -- 172.21.15.40:0/228165117 >> v1:172.21.15.40:6789/0 conn(0x7f17dc071aa0 legacy=0x7f17dc06d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:18.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.687+0000 7f17e1f8e700 1 -- 172.21.15.40:0/228165117 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17dc06da20 con 0x7f17dc15c640 2023-11-08T08:19:18.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.687+0000 7f17e1f8e700 1 --2- 172.21.15.40:0/228165117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17dc15c640 0x7f17dc0d31e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f17d8009850 tx=0x7f17d8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81614eafb166dd7c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:18.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.687+0000 7f17e0f8c700 1 -- 172.21.15.40:0/228165117 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d8011830 con 0x7f17dc15c640 2023-11-08T08:19:18.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.687+0000 7f17e0f8c700 1 -- 172.21.15.40:0/228165117 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d8011990 con 0x7f17dc15c640 2023-11-08T08:19:18.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.687+0000 7f17e0f8c700 1 -- 172.21.15.40:0/228165117 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d8011d20 con 0x7f17dc15c640 2023-11-08T08:19:18.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.687+0000 7f17e2f90700 1 -- 172.21.15.40:0/228165117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17dc15c640 msgr2=0x7f17dc0d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:18.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.687+0000 7f17e2f90700 1 --2- 172.21.15.40:0/228165117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17dc15c640 0x7f17dc0d31e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f17d8009850 tx=0x7f17d8009b60 comp rx=0 tx=0).stop 2023-11-08T08:19:18.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.691+0000 7f17e2f90700 1 -- 172.21.15.40:0/228165117 shutdown_connections 2023-11-08T08:19:18.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.691+0000 7f17e2f90700 1 --2- 172.21.15.40:0/228165117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17dc15c640 0x7f17dc0d31e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.691+0000 7f17e2f90700 1 -- 172.21.15.40:0/228165117 >> 172.21.15.40:0/228165117 conn(0x7f17dc0b6520 msgr2=0x7f17dc071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:18.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.691+0000 7f17e2f90700 1 -- 172.21.15.40:0/228165117 shutdown_connections 2023-11-08T08:19:18.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.691+0000 7f17e2f90700 1 -- 172.21.15.40:0/228165117 wait complete. 2023-11-08T08:19:18.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.691+0000 7f17e2f90700 1 Processor -- start 2023-11-08T08:19:18.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.691+0000 7f17e2f90700 1 -- start start 2023-11-08T08:19:18.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17e2f90700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17dc069e00 0x7f17dc0c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:18.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17e2f90700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17dc071aa0 0x7f17dc09be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:18.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17e2f90700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17dc15c640 0x7f17dc0c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:18.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17e2f90700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc0c7970 con 0x7f17dc071aa0 2023-11-08T08:19:18.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17e2f90700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc0c6a20 con 0x7f17dc069e00 2023-11-08T08:19:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17e2f90700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc0c6cf0 con 0x7f17dc15c640 2023-11-08T08:19:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17e178d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17dc071aa0 0x7f17dc09be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17e178d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17dc071aa0 0x7f17dc09be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35992/0 (socket says 172.21.15.40:35992) 2023-11-08T08:19:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17e178d700 1 -- 172.21.15.40:0/404584270 learned_addr learned my addr 172.21.15.40:0/404584270 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:18.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17e1f8e700 1 --2- 172.21.15.40:0/404584270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17dc069e00 0x7f17dc0c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:18.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17e278f700 1 --2- 172.21.15.40:0/404584270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17dc15c640 0x7f17dc0c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:18.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17e178d700 1 -- 172.21.15.40:0/404584270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17dc15c640 msgr2=0x7f17dc0c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:18.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17e178d700 1 --2- 172.21.15.40:0/404584270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17dc15c640 0x7f17dc0c64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17e178d700 1 -- 172.21.15.40:0/404584270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17dc069e00 msgr2=0x7f17dc0c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:18.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17e178d700 1 --2- 172.21.15.40:0/404584270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17dc069e00 0x7f17dc0c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17e178d700 1 -- 172.21.15.40:0/404584270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d80094b0 con 0x7f17dc071aa0 2023-11-08T08:19:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17e178d700 1 --2- 172.21.15.40:0/404584270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17dc071aa0 0x7f17dc09be40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f17cc001eb0 tx=0x7f17cc00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17d27fc700 1 -- 172.21.15.40:0/404584270 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc005d90 con 0x7f17dc071aa0 2023-11-08T08:19:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17d27fc700 1 -- 172.21.15.40:0/404584270 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17cc0070d0 con 0x7f17dc071aa0 2023-11-08T08:19:18.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17e2f90700 1 -- 172.21.15.40:0/404584270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17dc072010 con 0x7f17dc071aa0 2023-11-08T08:19:18.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.695+0000 7f17d27fc700 1 -- 172.21.15.40:0/404584270 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc007440 con 0x7f17dc071aa0 2023-11-08T08:19:18.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.699+0000 7f17e2f90700 1 -- 172.21.15.40:0/404584270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17dc15ca10 con 0x7f17dc071aa0 2023-11-08T08:19:18.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.699+0000 7f17e2f90700 1 -- 172.21.15.40:0/404584270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17dc06c830 con 0x7f17dc071aa0 2023-11-08T08:19:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.699+0000 7f17d27fc700 1 -- 172.21.15.40:0/404584270 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f17cc019030 con 0x7f17dc071aa0 2023-11-08T08:19:18.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.703+0000 7f17d27fc700 1 --2- 172.21.15.40:0/404584270 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f17c0064f10 0x7f17c00673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:18.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.703+0000 7f17d27fc700 1 -- 172.21.15.40:0/404584270 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f17cc081350 con 0x7f17dc071aa0 2023-11-08T08:19:18.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.703+0000 7f17e1f8e700 1 --2- 172.21.15.40:0/404584270 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f17c0064f10 0x7f17c00673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:18.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.707+0000 7f17e1f8e700 1 --2- 172.21.15.40:0/404584270 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f17c0064f10 0x7f17c00673c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f17d8009fd0 tx=0x7f17d8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:18.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.711+0000 7f17d27fc700 1 -- 172.21.15.40:0/404584270 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f17cc04a400 con 0x7f17dc071aa0 2023-11-08T08:19:18.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.915+0000 7f17e2f90700 1 -- 172.21.15.40:0/404584270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f17dc073cb0 con 0x7f17dc071aa0 2023-11-08T08:19:18.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.915+0000 7f17d27fc700 1 -- 172.21.15.40:0/404584270 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f17cc017080 con 0x7f17dc071aa0 2023-11-08T08:19:18.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.919+0000 7f17e2f90700 1 -- 172.21.15.40:0/404584270 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f17c0064f10 msgr2=0x7f17c00673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:18.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.919+0000 7f17e2f90700 1 --2- 172.21.15.40:0/404584270 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f17c0064f10 0x7f17c00673c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f17d8009fd0 tx=0x7f17d8008030 comp rx=0 tx=0).stop 2023-11-08T08:19:18.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.919+0000 7f17e2f90700 1 -- 172.21.15.40:0/404584270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17dc071aa0 msgr2=0x7f17dc09be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:18.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.919+0000 7f17e2f90700 1 --2- 172.21.15.40:0/404584270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17dc071aa0 0x7f17dc09be40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f17cc001eb0 tx=0x7f17cc00c7b0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.919+0000 7f17e2f90700 1 -- 172.21.15.40:0/404584270 shutdown_connections 2023-11-08T08:19:18.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.919+0000 7f17e2f90700 1 --2- 172.21.15.40:0/404584270 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f17c0064f10 0x7f17c00673c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.919+0000 7f17e2f90700 1 --2- 172.21.15.40:0/404584270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17dc15c640 0x7f17dc0c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.919+0000 7f17e2f90700 1 --2- 172.21.15.40:0/404584270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17dc071aa0 0x7f17dc09be40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.919+0000 7f17e2f90700 1 --2- 172.21.15.40:0/404584270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17dc069e00 0x7f17dc0c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:18.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.919+0000 7f17e2f90700 1 -- 172.21.15.40:0/404584270 >> 172.21.15.40:0/404584270 conn(0x7f17dc0b6520 msgr2=0x7f17dc0b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:18.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.919+0000 7f17e2f90700 1 -- 172.21.15.40:0/404584270 shutdown_connections 2023-11-08T08:19:18.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:18.919+0000 7f17e2f90700 1 -- 172.21.15.40:0/404584270 wait complete. 2023-11-08T08:19:18.925 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-11-08T08:19:18.946 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:19:18.947 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:19:18.966 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:19:19.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.195+0000 7f9558aed700 1 Processor -- start 2023-11-08T08:19:19.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.195+0000 7f9558aed700 1 -- start start 2023-11-08T08:19:19.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9558aed700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9554071aa0 0x7f955406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:19.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9558aed700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9554069e00 0x7f955406a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:19.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9558aed700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955406c600 con 0x7f955415c640 2023-11-08T08:19:19.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f95537fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f955415c640 0x7f95540d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33224/0 (socket says 172.21.15.40:33224) 2023-11-08T08:19:19.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9558aed700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f955406cef0 con 0x7f9554071aa0 2023-11-08T08:19:19.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f95537fe700 1 -- 172.21.15.40:0/2493036741 learned_addr learned my addr 172.21.15.40:0/2493036741 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:19.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9558aed700 1 -- 172.21.15.40:0/2493036741 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95540d38f0 con 0x7f9554069e00 2023-11-08T08:19:19.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9553fff700 1 --2- 172.21.15.40:0/2493036741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9554069e00 0x7f955406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:19.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9552ffd700 1 -- 172.21.15.40:0/2493036741 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414097126 0 0) 0x7f955406c600 con 0x7f955415c640 2023-11-08T08:19:19.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f954bfff700 1 --2- 172.21.15.40:0/2493036741 >> v2:172.21.15.169:3300/0 conn(0x7f9554071aa0 0x7f955406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:19.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9552ffd700 1 -- 172.21.15.40:0/2493036741 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95400035d0 con 0x7f955415c640 2023-11-08T08:19:19.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9552ffd700 1 -- 172.21.15.40:0/2493036741 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 593781389 0 0) 0x7f95400035d0 con 0x7f955415c640 2023-11-08T08:19:19.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9552ffd700 1 -- 172.21.15.40:0/2493036741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9554069e00 msgr2=0x7f955406a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:19.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9552ffd700 1 --2- 172.21.15.40:0/2493036741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9554069e00 0x7f955406a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9552ffd700 1 -- 172.21.15.40:0/2493036741 >> v2:172.21.15.169:3300/0 conn(0x7f9554071aa0 msgr2=0x7f955406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:19.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9552ffd700 1 --2- 172.21.15.40:0/2493036741 >> v2:172.21.15.169:3300/0 conn(0x7f9554071aa0 0x7f955406d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9552ffd700 1 -- 172.21.15.40:0/2493036741 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95540722f0 con 0x7f955415c640 2023-11-08T08:19:19.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9552ffd700 1 -- 172.21.15.40:0/2493036741 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f9544001670 con 0x7f955415c640 2023-11-08T08:19:19.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9552ffd700 1 -- 172.21.15.40:0/2493036741 >> v1:172.21.15.40:6789/0 conn(0x7f955415c640 legacy=0x7f95540d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:19.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9552ffd700 1 --2- 172.21.15.40:0/2493036741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9540003a20 0x7f9540003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:19.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9552ffd700 1 --2- 172.21.15.40:0/2493036741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9554069e00 0x7f9540004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:19.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9552ffd700 1 --2- 172.21.15.40:0/2493036741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9540004db0 0x7f9540005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:19.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9552ffd700 1 -- 172.21.15.40:0/2493036741 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955406cef0 con 0x7f9540004db0 2023-11-08T08:19:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9552ffd700 1 -- 172.21.15.40:0/2493036741 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95540d38f0 con 0x7f9540003a20 2023-11-08T08:19:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9552ffd700 1 -- 172.21.15.40:0/2493036741 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954000ba30 con 0x7f9554069e00 2023-11-08T08:19:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9553fff700 1 --2- 172.21.15.40:0/2493036741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9540004db0 0x7f9540005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f954bfff700 1 --2- 172.21.15.40:0/2493036741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9554069e00 0x7f9540004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f95537fe700 1 --2- 172.21.15.40:0/2493036741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9540003a20 0x7f9540003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9553fff700 1 -- 172.21.15.40:0/2493036741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9554069e00 msgr2=0x7f9540004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9553fff700 1 --2- 172.21.15.40:0/2493036741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9554069e00 0x7f9540004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9553fff700 1 -- 172.21.15.40:0/2493036741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9540003a20 msgr2=0x7f9540003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9553fff700 1 --2- 172.21.15.40:0/2493036741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9540003a20 0x7f9540003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f9553fff700 1 -- 172.21.15.40:0/2493036741 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95400035d0 con 0x7f9540004db0 2023-11-08T08:19:19.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.199+0000 7f954bfff700 1 --2- 172.21.15.40:0/2493036741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9554069e00 0x7f9540004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:19.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.203+0000 7f9553fff700 1 --2- 172.21.15.40:0/2493036741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9540004db0 0x7f9540005220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f954c009fc0 tx=0x7f95400058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b556a6b5425dd82a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:19.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.203+0000 7f9552ffd700 1 -- 172.21.15.40:0/2493036741 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c00ec30 con 0x7f9540004db0 2023-11-08T08:19:19.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.203+0000 7f9552ffd700 1 -- 172.21.15.40:0/2493036741 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c00f040 con 0x7f9540004db0 2023-11-08T08:19:19.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.203+0000 7f9552ffd700 1 -- 172.21.15.40:0/2493036741 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c014480 con 0x7f9540004db0 2023-11-08T08:19:19.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.203+0000 7f9558aed700 1 -- 172.21.15.40:0/2493036741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9540004db0 msgr2=0x7f9540005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:19.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.203+0000 7f9558aed700 1 --2- 172.21.15.40:0/2493036741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9540004db0 0x7f9540005220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f954c009fc0 tx=0x7f95400058a0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.203+0000 7f9553fff700 1 -- 172.21.15.40:0/2493036741 reap_dead start 2023-11-08T08:19:19.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.203+0000 7f9558aed700 1 -- 172.21.15.40:0/2493036741 shutdown_connections 2023-11-08T08:19:19.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.203+0000 7f9558aed700 1 -- 172.21.15.40:0/2493036741 >> 172.21.15.40:0/2493036741 conn(0x7f95540b6520 msgr2=0x7f9554071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:19.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.203+0000 7f9558aed700 1 -- 172.21.15.40:0/2493036741 shutdown_connections 2023-11-08T08:19:19.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.203+0000 7f9558aed700 1 -- 172.21.15.40:0/2493036741 wait complete. 2023-11-08T08:19:19.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.207+0000 7f9558aed700 1 Processor -- start 2023-11-08T08:19:19.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.207+0000 7f9558aed700 1 -- start start 2023-11-08T08:19:19.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f9558aed700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9540004db0 0x7f95540c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:19.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f9558aed700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95540d1120 0x7f9554159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:19.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f9558aed700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9554159dc0 0x7f95540ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:19.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f9558aed700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955415a230 con 0x7f95540d1120 2023-11-08T08:19:19.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f9558aed700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95540b2df0 con 0x7f9554159dc0 2023-11-08T08:19:19.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f9558aed700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95540c9bb0 con 0x7f9540004db0 2023-11-08T08:19:19.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f954bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95540d1120 0x7f9554159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:19.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f954bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95540d1120 0x7f9554159880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36016/0 (socket says 172.21.15.40:36016) 2023-11-08T08:19:19.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f954bfff700 1 -- 172.21.15.40:0/3565992467 learned_addr learned my addr 172.21.15.40:0/3565992467 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:19.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f95537fe700 1 --2- 172.21.15.40:0/3565992467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9540004db0 0x7f95540c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:19.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f9553fff700 1 --2- 172.21.15.40:0/3565992467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9554159dc0 0x7f95540ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:19.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f954bfff700 1 -- 172.21.15.40:0/3565992467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9540004db0 msgr2=0x7f95540c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:19.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f954bfff700 1 --2- 172.21.15.40:0/3565992467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9540004db0 0x7f95540c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f954bfff700 1 -- 172.21.15.40:0/3565992467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9554159dc0 msgr2=0x7f95540ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:19.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f954bfff700 1 --2- 172.21.15.40:0/3565992467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9554159dc0 0x7f95540ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f954bfff700 1 -- 172.21.15.40:0/3565992467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9554072010 con 0x7f95540d1120 2023-11-08T08:19:19.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f954bfff700 1 --2- 172.21.15.40:0/3565992467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95540d1120 0x7f9554159880 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f953c010a00 tx=0x7f953c010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:19.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f9550ff9700 1 -- 172.21.15.40:0/3565992467 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c010e40 con 0x7f95540d1120 2023-11-08T08:19:19.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f9550ff9700 1 -- 172.21.15.40:0/3565992467 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c004020 con 0x7f95540d1120 2023-11-08T08:19:19.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f9550ff9700 1 -- 172.21.15.40:0/3565992467 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c0043b0 con 0x7f95540d1120 2023-11-08T08:19:19.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f9558aed700 1 -- 172.21.15.40:0/3565992467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f955415ca10 con 0x7f95540d1120 2023-11-08T08:19:19.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.211+0000 7f9558aed700 1 -- 172.21.15.40:0/3565992467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955406c830 con 0x7f95540d1120 2023-11-08T08:19:19.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.215+0000 7f9558aed700 1 -- 172.21.15.40:0/3565992467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9554073cb0 con 0x7f95540d1120 2023-11-08T08:19:19.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.215+0000 7f9550ff9700 1 -- 172.21.15.40:0/3565992467 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f953c025070 con 0x7f95540d1120 2023-11-08T08:19:19.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.219+0000 7f9550ff9700 1 --2- 172.21.15.40:0/3565992467 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9534064ee0 0x7f9534067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:19.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.219+0000 7f9550ff9700 1 -- 172.21.15.40:0/3565992467 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f953c082430 con 0x7f95540d1120 2023-11-08T08:19:19.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.219+0000 7f95537fe700 1 --2- 172.21.15.40:0/3565992467 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9534064ee0 0x7f9534067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:19.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.219+0000 7f95537fe700 1 --2- 172.21.15.40:0/3565992467 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9534064ee0 0x7f9534067390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9544001eb0 tx=0x7f9544007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:19.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.223+0000 7f9550ff9700 1 -- 172.21.15.40:0/3565992467 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f953c0501f0 con 0x7f95540d1120 2023-11-08T08:19:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.399+0000 7f9558aed700 1 -- 172.21.15.40:0/3565992467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95540d2950 con 0x7f95540d1120 2023-11-08T08:19:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.399+0000 7f9550ff9700 1 -- 172.21.15.40:0/3565992467 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19652 (secure 0 0 0) 0x7f953c053810 con 0x7f95540d1120 2023-11-08T08:19:19.404 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:19.404 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":28,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:08.980158+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:19:19.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.403+0000 7f9558aed700 1 -- 172.21.15.40:0/3565992467 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9534064ee0 msgr2=0x7f9534067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:19.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.403+0000 7f9558aed700 1 --2- 172.21.15.40:0/3565992467 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9534064ee0 0x7f9534067390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9544001eb0 tx=0x7f9544007040 comp rx=0 tx=0).stop 2023-11-08T08:19:19.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.403+0000 7f9558aed700 1 -- 172.21.15.40:0/3565992467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95540d1120 msgr2=0x7f9554159880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:19.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.403+0000 7f9558aed700 1 --2- 172.21.15.40:0/3565992467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95540d1120 0x7f9554159880 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f953c010a00 tx=0x7f953c010d10 comp rx=0 tx=0).stop 2023-11-08T08:19:19.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.403+0000 7f9558aed700 1 -- 172.21.15.40:0/3565992467 shutdown_connections 2023-11-08T08:19:19.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.403+0000 7f9558aed700 1 --2- 172.21.15.40:0/3565992467 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9534064ee0 0x7f9534067390 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.403+0000 7f9558aed700 1 --2- 172.21.15.40:0/3565992467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9554159dc0 0x7f95540ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.403+0000 7f9558aed700 1 --2- 172.21.15.40:0/3565992467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95540d1120 0x7f9554159880 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.403+0000 7f9558aed700 1 --2- 172.21.15.40:0/3565992467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9540004db0 0x7f95540c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.403+0000 7f9558aed700 1 -- 172.21.15.40:0/3565992467 >> 172.21.15.40:0/3565992467 conn(0x7f95540b6520 msgr2=0x7f955415b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:19.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.403+0000 7f9558aed700 1 -- 172.21.15.40:0/3565992467 shutdown_connections 2023-11-08T08:19:19.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.403+0000 7f9558aed700 1 -- 172.21.15.40:0/3565992467 wait complete. 2023-11-08T08:19:19.430 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T08:19:02.627691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T08:19:03.473821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T08:19:06.539942+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T08:19:19.435 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:19:19.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.635+0000 7f98cb59e700 1 Processor -- start 2023-11-08T08:19:19.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.635+0000 7f98cb59e700 1 -- start start 2023-11-08T08:19:19.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.639+0000 7f98cb59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98cc070620 0x7f98cc0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:19.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.639+0000 7f98cb59e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f98cc069bf0 0x7f98cc069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:19.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.639+0000 7f98cb59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98cc15c520 con 0x7f98cc15c150 2023-11-08T08:19:19.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.639+0000 7f98cb59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98cc0b4240 con 0x7f98cc070620 2023-11-08T08:19:19.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.639+0000 7f98cb59e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98cc06cc50 con 0x7f98cc069bf0 2023-11-08T08:19:19.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.639+0000 7f98c9d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98cc070620 0x7f98cc0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:19.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.639+0000 7f98ca59c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f98cc15c150 0x7f98cc06ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33240/0 (socket says 172.21.15.40:33240) 2023-11-08T08:19:19.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.639+0000 7f98ca59c700 1 -- 172.21.15.40:0/2419606581 learned_addr learned my addr 172.21.15.40:0/2419606581 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:19.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.639+0000 7f98cad9d700 1 --2- 172.21.15.40:0/2419606581 >> v2:172.21.15.169:3300/0 conn(0x7f98cc069bf0 0x7f98cc069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:19.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.639+0000 7f98c9d9b700 1 -- 172.21.15.40:0/2419606581 >> v2:172.21.15.169:3300/0 conn(0x7f98cc069bf0 msgr2=0x7f98cc069ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:19.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.639+0000 7f98c9d9b700 1 --2- 172.21.15.40:0/2419606581 >> v2:172.21.15.169:3300/0 conn(0x7f98cc069bf0 0x7f98cc069ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.639+0000 7f98c9d9b700 1 -- 172.21.15.40:0/2419606581 >> v1:172.21.15.40:6789/0 conn(0x7f98cc15c150 legacy=0x7f98cc06ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:19.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.639+0000 7f98c9d9b700 1 -- 172.21.15.40:0/2419606581 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc0d2450 con 0x7f98cc070620 2023-11-08T08:19:19.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.639+0000 7f98c9d9b700 1 --2- 172.21.15.40:0/2419606581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc070620 0x7f98cc0b3d00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f98cc071460 tx=0x7f98c0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6cb620a2a2fc28 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:19.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.643+0000 7f98c959a700 1 -- 172.21.15.40:0/2419606581 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0005e50 con 0x7f98cc070620 2023-11-08T08:19:19.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.643+0000 7f98c959a700 1 -- 172.21.15.40:0/2419606581 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c0006030 con 0x7f98cc070620 2023-11-08T08:19:19.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.643+0000 7f98c959a700 1 -- 172.21.15.40:0/2419606581 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c00063c0 con 0x7f98cc070620 2023-11-08T08:19:19.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.643+0000 7f98cb59e700 1 -- 172.21.15.40:0/2419606581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc070620 msgr2=0x7f98cc0b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:19.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.643+0000 7f98cb59e700 1 --2- 172.21.15.40:0/2419606581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc070620 0x7f98cc0b3d00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f98cc071460 tx=0x7f98c0009b20 comp rx=0 tx=0).stop 2023-11-08T08:19:19.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.643+0000 7f98cb59e700 1 -- 172.21.15.40:0/2419606581 shutdown_connections 2023-11-08T08:19:19.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.643+0000 7f98cb59e700 1 --2- 172.21.15.40:0/2419606581 >> v2:172.21.15.169:3300/0 conn(0x7f98cc069bf0 0x7f98cc069ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.643+0000 7f98cb59e700 1 --2- 172.21.15.40:0/2419606581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc070620 0x7f98cc0b3d00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.643+0000 7f98cb59e700 1 -- 172.21.15.40:0/2419606581 >> 172.21.15.40:0/2419606581 conn(0x7f98cc0d3680 msgr2=0x7f98cc0cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:19.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.643+0000 7f98cb59e700 1 -- 172.21.15.40:0/2419606581 shutdown_connections 2023-11-08T08:19:19.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.643+0000 7f98cb59e700 1 -- 172.21.15.40:0/2419606581 wait complete. 2023-11-08T08:19:19.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98cb59e700 1 Processor -- start 2023-11-08T08:19:19.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98cb59e700 1 -- start start 2023-11-08T08:19:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98cb59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc069bf0 0x7f98cc14f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98cb59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98cc070620 0x7f98cc0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98cb59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98cc15c150 0x7f98cc0b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:19.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98cb59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0b7690 con 0x7f98cc069bf0 2023-11-08T08:19:19.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98cb59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc1e5c10 con 0x7f98cc15c150 2023-11-08T08:19:19.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98cb59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc1e5ef0 con 0x7f98cc070620 2023-11-08T08:19:19.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98ca59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc069bf0 0x7f98cc14f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:19.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98ca59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc069bf0 0x7f98cc14f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36036/0 (socket says 172.21.15.40:36036) 2023-11-08T08:19:19.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98ca59c700 1 -- 172.21.15.40:0/2050657468 learned_addr learned my addr 172.21.15.40:0/2050657468 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:19.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98c9d9b700 1 --2- 172.21.15.40:0/2050657468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98cc070620 0x7f98cc0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:19.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98cad9d700 1 --2- 172.21.15.40:0/2050657468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98cc15c150 0x7f98cc0b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:19.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98ca59c700 1 -- 172.21.15.40:0/2050657468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98cc070620 msgr2=0x7f98cc0b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:19.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98ca59c700 1 --2- 172.21.15.40:0/2050657468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98cc070620 0x7f98cc0b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98ca59c700 1 -- 172.21.15.40:0/2050657468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98cc15c150 msgr2=0x7f98cc0b7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:19.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98ca59c700 1 --2- 172.21.15.40:0/2050657468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98cc15c150 0x7f98cc0b7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98ca59c700 1 -- 172.21.15.40:0/2050657468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c0009450 con 0x7f98cc069bf0 2023-11-08T08:19:19.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98ca59c700 1 --2- 172.21.15.40:0/2050657468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc069bf0 0x7f98cc14f710 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f98bc001eb0 tx=0x7f98bc00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:19.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98baffd700 1 -- 172.21.15.40:0/2050657468 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc005d90 con 0x7f98cc069bf0 2023-11-08T08:19:19.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98baffd700 1 -- 172.21.15.40:0/2050657468 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98bc0070d0 con 0x7f98cc069bf0 2023-11-08T08:19:19.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98cb59e700 1 -- 172.21.15.40:0/2050657468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98cc0d05e0 con 0x7f98cc069bf0 2023-11-08T08:19:19.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.647+0000 7f98baffd700 1 -- 172.21.15.40:0/2050657468 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc007440 con 0x7f98cc069bf0 2023-11-08T08:19:19.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.651+0000 7f98cb59e700 1 -- 172.21.15.40:0/2050657468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc0d2730 con 0x7f98cc069bf0 2023-11-08T08:19:19.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.651+0000 7f98cb59e700 1 -- 172.21.15.40:0/2050657468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98cc0d2140 con 0x7f98cc069bf0 2023-11-08T08:19:19.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.651+0000 7f98baffd700 1 -- 172.21.15.40:0/2050657468 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f98bc019030 con 0x7f98cc069bf0 2023-11-08T08:19:19.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.655+0000 7f98baffd700 1 --2- 172.21.15.40:0/2050657468 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98ac064ee0 0x7f98ac067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:19.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.655+0000 7f98baffd700 1 -- 172.21.15.40:0/2050657468 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f98bc081340 con 0x7f98cc069bf0 2023-11-08T08:19:19.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.655+0000 7f98c9d9b700 1 --2- 172.21.15.40:0/2050657468 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98ac064ee0 0x7f98ac067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:19.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.655+0000 7f98c9d9b700 1 --2- 172.21.15.40:0/2050657468 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98ac064ee0 0x7f98ac067390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f98cc070a10 tx=0x7f98c0008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:19.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.659+0000 7f98baffd700 1 -- 172.21.15.40:0/2050657468 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f98bc04a400 con 0x7f98cc069bf0 2023-11-08T08:19:19.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.831+0000 7f98cb59e700 1 -- 172.21.15.40:0/2050657468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f98cc073d20 con 0x7f98cc069bf0 2023-11-08T08:19:19.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.835+0000 7f98baffd700 1 -- 172.21.15.40:0/2050657468 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f98bc04f1b0 con 0x7f98cc069bf0 2023-11-08T08:19:19.836 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:19:19.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.835+0000 7f98cb59e700 1 -- 172.21.15.40:0/2050657468 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98ac064ee0 msgr2=0x7f98ac067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:19.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.835+0000 7f98cb59e700 1 --2- 172.21.15.40:0/2050657468 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98ac064ee0 0x7f98ac067390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f98cc070a10 tx=0x7f98c0008070 comp rx=0 tx=0).stop 2023-11-08T08:19:19.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.835+0000 7f98cb59e700 1 -- 172.21.15.40:0/2050657468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc069bf0 msgr2=0x7f98cc14f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:19.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.835+0000 7f98cb59e700 1 --2- 172.21.15.40:0/2050657468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc069bf0 0x7f98cc14f710 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f98bc001eb0 tx=0x7f98bc00c7b0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.835+0000 7f98cb59e700 1 -- 172.21.15.40:0/2050657468 shutdown_connections 2023-11-08T08:19:19.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.835+0000 7f98cb59e700 1 --2- 172.21.15.40:0/2050657468 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98ac064ee0 0x7f98ac067390 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.835+0000 7f98cb59e700 1 --2- 172.21.15.40:0/2050657468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98cc15c150 0x7f98cc0b7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.835+0000 7f98cb59e700 1 --2- 172.21.15.40:0/2050657468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98cc070620 0x7f98cc0b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.835+0000 7f98cb59e700 1 --2- 172.21.15.40:0/2050657468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc069bf0 0x7f98cc14f710 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:19.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.835+0000 7f98cb59e700 1 -- 172.21.15.40:0/2050657468 >> 172.21.15.40:0/2050657468 conn(0x7f98cc0d3680 msgr2=0x7f98cc159740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:19.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.835+0000 7f98cb59e700 1 -- 172.21.15.40:0/2050657468 shutdown_connections 2023-11-08T08:19:19.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:19.835+0000 7f98cb59e700 1 -- 172.21.15.40:0/2050657468 wait complete. 2023-11-08T08:19:19.859 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:19:20.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.043+0000 7f8a012ab700 1 Processor -- start 2023-11-08T08:19:20.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.047+0000 7f8a012ab700 1 -- start start 2023-11-08T08:19:20.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.047+0000 7f8a012ab700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f89fc15c150 0x7f89fc0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:20.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.047+0000 7f8a012ab700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89fc070360 0x7f89fc160340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:20.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.047+0000 7f8a012ab700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89fc15c520 con 0x7f89fc06ff10 2023-11-08T08:19:20.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.047+0000 7f8a012ab700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89fc0b4240 con 0x7f89fc15c150 2023-11-08T08:19:20.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.047+0000 7f8a012ab700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc06ac00 con 0x7f89fc070360 2023-11-08T08:19:20.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f89fbfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f89fc15c150 0x7f89fc0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:20.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f89fb7fe700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f89fc06ff10 0x7f89fc15fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:59058/0 (socket says 172.21.15.40:59058) 2023-11-08T08:19:20.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f89fb7fe700 1 -- 172.21.15.40:0/1532848389 learned_addr learned my addr 172.21.15.40:0/1532848389 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:20.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f8a00aaa700 1 --2- 172.21.15.40:0/1532848389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89fc070360 0x7f89fc160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:20.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f89fbfff700 1 -- 172.21.15.40:0/1532848389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89fc070360 msgr2=0x7f89fc160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:20.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f89fbfff700 1 --2- 172.21.15.40:0/1532848389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89fc070360 0x7f89fc160340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f89fbfff700 1 -- 172.21.15.40:0/1532848389 >> v1:172.21.15.169:6789/0 conn(0x7f89fc06ff10 legacy=0x7f89fc15fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:20.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f89fbfff700 1 -- 172.21.15.40:0/1532848389 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc06cc50 con 0x7f89fc15c150 2023-11-08T08:19:20.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f89faffd700 1 -- 172.21.15.40:0/1532848389 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615741118 0 0) 0x7f89fc15c520 con 0x7f89fc06ff10 2023-11-08T08:19:20.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f89fbfff700 1 --2- 172.21.15.40:0/1532848389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc15c150 0x7f89fc0b3d00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f89ec009850 tx=0x7f89ec009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b2be0f90ed9b69 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:20.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f89faffd700 1 -- 172.21.15.40:0/1532848389 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec011830 con 0x7f89fc15c150 2023-11-08T08:19:20.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f89faffd700 1 -- 172.21.15.40:0/1532848389 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec011990 con 0x7f89fc15c150 2023-11-08T08:19:20.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f89faffd700 1 -- 172.21.15.40:0/1532848389 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec011d40 con 0x7f89fc15c150 2023-11-08T08:19:20.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f8a012ab700 1 -- 172.21.15.40:0/1532848389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc15c150 msgr2=0x7f89fc0b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f8a012ab700 1 --2- 172.21.15.40:0/1532848389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc15c150 0x7f89fc0b3d00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f89ec009850 tx=0x7f89ec009b60 comp rx=0 tx=0).stop 2023-11-08T08:19:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f8a012ab700 1 -- 172.21.15.40:0/1532848389 shutdown_connections 2023-11-08T08:19:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f8a012ab700 1 --2- 172.21.15.40:0/1532848389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89fc070360 0x7f89fc160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f8a012ab700 1 --2- 172.21.15.40:0/1532848389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc15c150 0x7f89fc0b3d00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f8a012ab700 1 -- 172.21.15.40:0/1532848389 >> 172.21.15.40:0/1532848389 conn(0x7f89fc0d3680 msgr2=0x7f89fc0cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f8a012ab700 1 -- 172.21.15.40:0/1532848389 shutdown_connections 2023-11-08T08:19:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.051+0000 7f8a012ab700 1 -- 172.21.15.40:0/1532848389 wait complete. 2023-11-08T08:19:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.055+0000 7f8a012ab700 1 Processor -- start 2023-11-08T08:19:20.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.055+0000 7f8a012ab700 1 -- start start 2023-11-08T08:19:20.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f8a012ab700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89fc06ff10 0x7f89fc1f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f8a012ab700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89fc070360 0x7f89fc15fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f8a012ab700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc15c150 0x7f89fc1f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f8a012ab700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0cb710 con 0x7f89fc15c150 2023-11-08T08:19:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f8a012ab700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc15f140 con 0x7f89fc070360 2023-11-08T08:19:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f8a012ab700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0cae20 con 0x7f89fc06ff10 2023-11-08T08:19:20.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f8a00aaa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc15c150 0x7f89fc1f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:20.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f8a00aaa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc15c150 0x7f89fc1f4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36058/0 (socket says 172.21.15.40:36058) 2023-11-08T08:19:20.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f8a00aaa700 1 -- 172.21.15.40:0/147008179 learned_addr learned my addr 172.21.15.40:0/147008179 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:20.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f89fbfff700 1 --2- 172.21.15.40:0/147008179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89fc06ff10 0x7f89fc1f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f89fb7fe700 1 --2- 172.21.15.40:0/147008179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89fc070360 0x7f89fc15fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f8a00aaa700 1 -- 172.21.15.40:0/147008179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89fc06ff10 msgr2=0x7f89fc1f17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f8a00aaa700 1 --2- 172.21.15.40:0/147008179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89fc06ff10 0x7f89fc1f17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f8a00aaa700 1 -- 172.21.15.40:0/147008179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89fc070360 msgr2=0x7f89fc15fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f8a00aaa700 1 --2- 172.21.15.40:0/147008179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89fc070360 0x7f89fc15fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f8a00aaa700 1 -- 172.21.15.40:0/147008179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ec0094b0 con 0x7f89fc15c150 2023-11-08T08:19:20.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f8a00aaa700 1 --2- 172.21.15.40:0/147008179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc15c150 0x7f89fc1f4210 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f89f400efc0 tx=0x7f89f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:20.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f89f8ff9700 1 -- 172.21.15.40:0/147008179 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f4009ea0 con 0x7f89fc15c150 2023-11-08T08:19:20.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f89f8ff9700 1 -- 172.21.15.40:0/147008179 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f400f040 con 0x7f89fc15c150 2023-11-08T08:19:20.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f89f8ff9700 1 -- 172.21.15.40:0/147008179 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f400ced0 con 0x7f89fc15c150 2023-11-08T08:19:20.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f8a012ab700 1 -- 172.21.15.40:0/147008179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89fc0d2730 con 0x7f89fc15c150 2023-11-08T08:19:20.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.059+0000 7f8a012ab700 1 -- 172.21.15.40:0/147008179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89fc0d2450 con 0x7f89fc15c150 2023-11-08T08:19:20.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.063+0000 7f8a012ab700 1 -- 172.21.15.40:0/147008179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89fc0d05e0 con 0x7f89fc15c150 2023-11-08T08:19:20.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.063+0000 7f89f8ff9700 1 -- 172.21.15.40:0/147008179 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f89f4014480 con 0x7f89fc15c150 2023-11-08T08:19:20.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.063+0000 7f89f8ff9700 1 --2- 172.21.15.40:0/147008179 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f89e8064ee0 0x7f89e8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:20.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.063+0000 7f89f8ff9700 1 -- 172.21.15.40:0/147008179 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f89f4082bf0 con 0x7f89fc15c150 2023-11-08T08:19:20.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.067+0000 7f89fbfff700 1 --2- 172.21.15.40:0/147008179 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f89e8064ee0 0x7f89e8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:20.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.067+0000 7f89fbfff700 1 --2- 172.21.15.40:0/147008179 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f89e8064ee0 0x7f89e8067390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0090b0 tx=0x7f89ec01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:20.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.071+0000 7f89f8ff9700 1 -- 172.21.15.40:0/147008179 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f89f4050950 con 0x7f89fc15c150 2023-11-08T08:19:20.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.247+0000 7f8a012ab700 1 -- 172.21.15.40:0/147008179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f89fc073d20 con 0x7f89fc15c150 2023-11-08T08:19:20.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.247+0000 7f89f8ff9700 1 -- 172.21.15.40:0/147008179 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7f89f4053f70 con 0x7f89fc15c150 2023-11-08T08:19:20.249 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:19:20.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.247+0000 7f8a012ab700 1 -- 172.21.15.40:0/147008179 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f89e8064ee0 msgr2=0x7f89e8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:20.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.247+0000 7f8a012ab700 1 --2- 172.21.15.40:0/147008179 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f89e8064ee0 0x7f89e8067390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0090b0 tx=0x7f89ec01c040 comp rx=0 tx=0).stop 2023-11-08T08:19:20.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.247+0000 7f8a012ab700 1 -- 172.21.15.40:0/147008179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc15c150 msgr2=0x7f89fc1f4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:20.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.247+0000 7f8a012ab700 1 --2- 172.21.15.40:0/147008179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc15c150 0x7f89fc1f4210 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f89f400efc0 tx=0x7f89f400c3a0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.251+0000 7f8a012ab700 1 -- 172.21.15.40:0/147008179 shutdown_connections 2023-11-08T08:19:20.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.251+0000 7f8a012ab700 1 --2- 172.21.15.40:0/147008179 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f89e8064ee0 0x7f89e8067390 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.251+0000 7f8a012ab700 1 --2- 172.21.15.40:0/147008179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc15c150 0x7f89fc1f4210 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.251+0000 7f8a012ab700 1 --2- 172.21.15.40:0/147008179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89fc070360 0x7f89fc15fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.251+0000 7f8a012ab700 1 --2- 172.21.15.40:0/147008179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89fc06ff10 0x7f89fc1f17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.251+0000 7f8a012ab700 1 -- 172.21.15.40:0/147008179 >> 172.21.15.40:0/147008179 conn(0x7f89fc0d3680 msgr2=0x7f89fc0b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:20.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.251+0000 7f8a012ab700 1 -- 172.21.15.40:0/147008179 shutdown_connections 2023-11-08T08:19:20.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.251+0000 7f8a012ab700 1 -- 172.21.15.40:0/147008179 wait complete. 2023-11-08T08:19:20.276 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T08:19:20.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.455+0000 7f2590cdd700 1 Processor -- start 2023-11-08T08:19:20.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.459+0000 7f2590cdd700 1 -- start start 2023-11-08T08:19:20.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.459+0000 7f2590cdd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f258c159320 0x7f258c0b1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:20.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.459+0000 7f2590cdd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f258c0cdaa0 0x7f258c06b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:20.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.463+0000 7f2590cdd700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258c15cde0 con 0x7f258c0b1780 2023-11-08T08:19:20.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.463+0000 7f2590cdd700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258c0b1c20 con 0x7f258c159320 2023-11-08T08:19:20.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.463+0000 7f2590cdd700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258c0af040 con 0x7f258c0cdaa0 2023-11-08T08:19:20.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.463+0000 7f258affd700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f258c0b1780 0x7f258c0cd390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:59064/0 (socket says 172.21.15.40:59064) 2023-11-08T08:19:20.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.463+0000 7f258affd700 1 -- 172.21.15.40:0/1769953380 learned_addr learned my addr 172.21.15.40:0/1769953380 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:20.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.463+0000 7f258b7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f258c159320 0x7f258c0b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:20.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.463+0000 7f258bfff700 1 --2- 172.21.15.40:0/1769953380 >> v2:172.21.15.169:3300/0 conn(0x7f258c0cdaa0 0x7f258c06b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:20.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.463+0000 7f258a7fc700 1 -- 172.21.15.40:0/1769953380 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1029434947 0 0) 0x7f258c15cde0 con 0x7f258c0b1780 2023-11-08T08:19:20.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.463+0000 7f258a7fc700 1 -- 172.21.15.40:0/1769953380 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25740035d0 con 0x7f258c0b1780 2023-11-08T08:19:20.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.463+0000 7f258b7fe700 1 -- 172.21.15.40:0/1769953380 >> v2:172.21.15.169:3300/0 conn(0x7f258c0cdaa0 msgr2=0x7f258c06b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:20.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.463+0000 7f258b7fe700 1 --2- 172.21.15.40:0/1769953380 >> v2:172.21.15.169:3300/0 conn(0x7f258c0cdaa0 0x7f258c06b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.463+0000 7f258b7fe700 1 -- 172.21.15.40:0/1769953380 >> v1:172.21.15.169:6789/0 conn(0x7f258c0b1780 legacy=0x7f258c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:20.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.463+0000 7f258b7fe700 1 -- 172.21.15.40:0/1769953380 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258c0ca7f0 con 0x7f258c159320 2023-11-08T08:19:20.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.463+0000 7f258b7fe700 1 --2- 172.21.15.40:0/1769953380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f258c159320 0x7f258c0b1240 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f257c009850 tx=0x7f257c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d66ac8709f85a6d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:20.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.463+0000 7f258a7fc700 1 -- 172.21.15.40:0/1769953380 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c011830 con 0x7f258c159320 2023-11-08T08:19:20.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.463+0000 7f258a7fc700 1 -- 172.21.15.40:0/1769953380 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257c011990 con 0x7f258c159320 2023-11-08T08:19:20.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.463+0000 7f258a7fc700 1 -- 172.21.15.40:0/1769953380 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c011d00 con 0x7f258c159320 2023-11-08T08:19:20.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.467+0000 7f2590cdd700 1 -- 172.21.15.40:0/1769953380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f258c159320 msgr2=0x7f258c0b1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:20.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.467+0000 7f2590cdd700 1 --2- 172.21.15.40:0/1769953380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f258c159320 0x7f258c0b1240 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f257c009850 tx=0x7f257c009b60 comp rx=0 tx=0).stop 2023-11-08T08:19:20.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.467+0000 7f2590cdd700 1 -- 172.21.15.40:0/1769953380 shutdown_connections 2023-11-08T08:19:20.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.467+0000 7f2590cdd700 1 --2- 172.21.15.40:0/1769953380 >> v2:172.21.15.169:3300/0 conn(0x7f258c0cdaa0 0x7f258c06b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.467+0000 7f2590cdd700 1 --2- 172.21.15.40:0/1769953380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f258c159320 0x7f258c0b1240 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.467+0000 7f2590cdd700 1 -- 172.21.15.40:0/1769953380 >> 172.21.15.40:0/1769953380 conn(0x7f258c0cc240 msgr2=0x7f258c0cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:20.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.467+0000 7f2590cdd700 1 -- 172.21.15.40:0/1769953380 shutdown_connections 2023-11-08T08:19:20.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.467+0000 7f2590cdd700 1 -- 172.21.15.40:0/1769953380 wait complete. 2023-11-08T08:19:20.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.471+0000 7f2590cdd700 1 Processor -- start 2023-11-08T08:19:20.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.471+0000 7f2590cdd700 1 -- start start 2023-11-08T08:19:20.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.471+0000 7f2590cdd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f258c0b1780 0x7f258c165180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:20.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.471+0000 7f2590cdd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f258c0cdaa0 0x7f258c166ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:20.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.471+0000 7f2590cdd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f258c159320 0x7f258c15a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:20.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.471+0000 7f2590cdd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c1666c0 con 0x7f258c159320 2023-11-08T08:19:20.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.471+0000 7f2590cdd700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c165a40 con 0x7f258c0cdaa0 2023-11-08T08:19:20.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.471+0000 7f2590cdd700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258c165d20 con 0x7f258c0b1780 2023-11-08T08:19:20.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.471+0000 7f258bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f258c159320 0x7f258c15a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:20.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.471+0000 7f258bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f258c159320 0x7f258c15a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36074/0 (socket says 172.21.15.40:36074) 2023-11-08T08:19:20.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.471+0000 7f258bfff700 1 -- 172.21.15.40:0/3074028678 learned_addr learned my addr 172.21.15.40:0/3074028678 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:20.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.471+0000 7f258b7fe700 1 --2- 172.21.15.40:0/3074028678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f258c0b1780 0x7f258c165180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:20.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.471+0000 7f258affd700 1 --2- 172.21.15.40:0/3074028678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f258c0cdaa0 0x7f258c166ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:20.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.475+0000 7f258bfff700 1 -- 172.21.15.40:0/3074028678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f258c0b1780 msgr2=0x7f258c165180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:20.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.475+0000 7f258bfff700 1 --2- 172.21.15.40:0/3074028678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f258c0b1780 0x7f258c165180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.475+0000 7f258bfff700 1 -- 172.21.15.40:0/3074028678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f258c0cdaa0 msgr2=0x7f258c166ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:20.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.475+0000 7f258bfff700 1 --2- 172.21.15.40:0/3074028678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f258c0cdaa0 0x7f258c166ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.475+0000 7f258bfff700 1 -- 172.21.15.40:0/3074028678 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c0094b0 con 0x7f258c159320 2023-11-08T08:19:20.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.475+0000 7f258bfff700 1 --2- 172.21.15.40:0/3074028678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f258c159320 0x7f258c15a540 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f258400ef90 tx=0x7f258400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:20.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.475+0000 7f2573fff700 1 -- 172.21.15.40:0/3074028678 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258400cc90 con 0x7f258c159320 2023-11-08T08:19:20.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.475+0000 7f2573fff700 1 -- 172.21.15.40:0/3074028678 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258400f040 con 0x7f258c159320 2023-11-08T08:19:20.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.475+0000 7f2573fff700 1 -- 172.21.15.40:0/3074028678 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584014480 con 0x7f258c159320 2023-11-08T08:19:20.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.475+0000 7f2590cdd700 1 -- 172.21.15.40:0/3074028678 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258c15d030 con 0x7f258c159320 2023-11-08T08:19:20.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.475+0000 7f2590cdd700 1 -- 172.21.15.40:0/3074028678 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258c1596f0 con 0x7f258c159320 2023-11-08T08:19:20.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.479+0000 7f2590cdd700 1 -- 172.21.15.40:0/3074028678 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f258c15d310 con 0x7f258c159320 2023-11-08T08:19:20.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.479+0000 7f2573fff700 1 -- 172.21.15.40:0/3074028678 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f25840145e0 con 0x7f258c159320 2023-11-08T08:19:20.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.479+0000 7f2573fff700 1 --2- 172.21.15.40:0/3074028678 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2578064ee0 0x7f2578067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:20.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.479+0000 7f2573fff700 1 -- 172.21.15.40:0/3074028678 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2584082ed0 con 0x7f258c159320 2023-11-08T08:19:20.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.483+0000 7f258b7fe700 1 --2- 172.21.15.40:0/3074028678 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2578064ee0 0x7f2578067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:20.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.483+0000 7f258b7fe700 1 --2- 172.21.15.40:0/3074028678 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2578064ee0 0x7f2578067390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f257c0090b0 tx=0x7f257c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:20.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.487+0000 7f2573fff700 1 -- 172.21.15.40:0/3074028678 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f2584050d00 con 0x7f258c159320 2023-11-08T08:19:20.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.659+0000 7f2590cdd700 1 -- 172.21.15.40:0/3074028678 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f258c158b60 con 0x7f258c159320 2023-11-08T08:19:20.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.663+0000 7f2573fff700 1 -- 172.21.15.40:0/3074028678 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2584018020 con 0x7f258c159320 2023-11-08T08:19:20.664 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:19:20.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.663+0000 7f2590cdd700 1 -- 172.21.15.40:0/3074028678 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2578064ee0 msgr2=0x7f2578067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:20.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.663+0000 7f2590cdd700 1 --2- 172.21.15.40:0/3074028678 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2578064ee0 0x7f2578067390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f257c0090b0 tx=0x7f257c008030 comp rx=0 tx=0).stop 2023-11-08T08:19:20.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.663+0000 7f2590cdd700 1 -- 172.21.15.40:0/3074028678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f258c159320 msgr2=0x7f258c15a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:20.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.663+0000 7f2590cdd700 1 --2- 172.21.15.40:0/3074028678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f258c159320 0x7f258c15a540 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f258400ef90 tx=0x7f258400c3a0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.663+0000 7f2590cdd700 1 -- 172.21.15.40:0/3074028678 shutdown_connections 2023-11-08T08:19:20.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.663+0000 7f2590cdd700 1 --2- 172.21.15.40:0/3074028678 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2578064ee0 0x7f2578067390 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.663+0000 7f2590cdd700 1 --2- 172.21.15.40:0/3074028678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f258c159320 0x7f258c15a540 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.663+0000 7f2590cdd700 1 --2- 172.21.15.40:0/3074028678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f258c0cdaa0 0x7f258c166ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.663+0000 7f2590cdd700 1 --2- 172.21.15.40:0/3074028678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f258c0b1780 0x7f258c165180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.663+0000 7f2590cdd700 1 -- 172.21.15.40:0/3074028678 >> 172.21.15.40:0/3074028678 conn(0x7f258c0cc240 msgr2=0x7f258c14e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:20.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.663+0000 7f2590cdd700 1 -- 172.21.15.40:0/3074028678 shutdown_connections 2023-11-08T08:19:20.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.663+0000 7f2590cdd700 1 -- 172.21.15.40:0/3074028678 wait complete. 2023-11-08T08:19:20.692 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T08:19:20.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.883+0000 7fd69b0bc700 1 Processor -- start 2023-11-08T08:19:20.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.883+0000 7fd69b0bc700 1 -- start start 2023-11-08T08:19:20.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.887+0000 7fd69b0bc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd69415c150 0x7fd6940b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:20.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.887+0000 7fd69b0bc700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd694070360 0x7fd694160340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:20.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.887+0000 7fd69b0bc700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69415c520 con 0x7fd69406ff10 2023-11-08T08:19:20.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.887+0000 7fd69b0bc700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6940b4240 con 0x7fd694070360 2023-11-08T08:19:20.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.887+0000 7fd69b0bc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69406ac00 con 0x7fd69415c150 2023-11-08T08:19:20.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.887+0000 7fd6998b9700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd69406ff10 0x7fd69415fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33256/0 (socket says 172.21.15.40:33256) 2023-11-08T08:19:20.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.887+0000 7fd69a8bb700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd694070360 0x7fd694160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:20.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.887+0000 7fd6998b9700 1 -- 172.21.15.40:0/3124954602 learned_addr learned my addr 172.21.15.40:0/3124954602 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:20.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.887+0000 7fd69a0ba700 1 --2- 172.21.15.40:0/3124954602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd69415c150 0x7fd6940b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.891+0000 7fd6990b8700 1 -- 172.21.15.40:0/3124954602 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907684745 0 0) 0x7fd69415c520 con 0x7fd69406ff10 2023-11-08T08:19:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.891+0000 7fd6990b8700 1 -- 172.21.15.40:0/3124954602 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6800035d0 con 0x7fd69406ff10 2023-11-08T08:19:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.891+0000 7fd69a8bb700 1 -- 172.21.15.40:0/3124954602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd69415c150 msgr2=0x7fd6940b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.891+0000 7fd69a8bb700 1 --2- 172.21.15.40:0/3124954602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd69415c150 0x7fd6940b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.891+0000 7fd69a8bb700 1 -- 172.21.15.40:0/3124954602 >> v1:172.21.15.40:6789/0 conn(0x7fd69406ff10 legacy=0x7fd69415fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.891+0000 7fd69a8bb700 1 -- 172.21.15.40:0/3124954602 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69406cc50 con 0x7fd694070360 2023-11-08T08:19:20.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.891+0000 7fd69a0ba700 1 --2- 172.21.15.40:0/3124954602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd69415c150 0x7fd6940b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:20.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.891+0000 7fd69a8bb700 1 --2- 172.21.15.40:0/3124954602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd694070360 0x7fd694160340 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009d90 tx=0x7fd68c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8089e4cab7f2116d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:20.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.891+0000 7fd6990b8700 1 -- 172.21.15.40:0/3124954602 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c00e070 con 0x7fd694070360 2023-11-08T08:19:20.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.891+0000 7fd6990b8700 1 -- 172.21.15.40:0/3124954602 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c00bb20 con 0x7fd694070360 2023-11-08T08:19:20.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.891+0000 7fd6990b8700 1 -- 172.21.15.40:0/3124954602 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c012430 con 0x7fd694070360 2023-11-08T08:19:20.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.891+0000 7fd69b0bc700 1 -- 172.21.15.40:0/3124954602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd694070360 msgr2=0x7fd694160340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:20.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.891+0000 7fd69b0bc700 1 --2- 172.21.15.40:0/3124954602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd694070360 0x7fd694160340 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009d90 tx=0x7fd68c00b330 comp rx=0 tx=0).stop 2023-11-08T08:19:20.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.891+0000 7fd69b0bc700 1 -- 172.21.15.40:0/3124954602 shutdown_connections 2023-11-08T08:19:20.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.891+0000 7fd69b0bc700 1 --2- 172.21.15.40:0/3124954602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd694070360 0x7fd694160340 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.891+0000 7fd69b0bc700 1 --2- 172.21.15.40:0/3124954602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd69415c150 0x7fd6940b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.891+0000 7fd69b0bc700 1 -- 172.21.15.40:0/3124954602 >> 172.21.15.40:0/3124954602 conn(0x7fd6940d3680 msgr2=0x7fd6940cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.891+0000 7fd69b0bc700 1 -- 172.21.15.40:0/3124954602 shutdown_connections 2023-11-08T08:19:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.891+0000 7fd69b0bc700 1 -- 172.21.15.40:0/3124954602 wait complete. 2023-11-08T08:19:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.895+0000 7fd69b0bc700 1 Processor -- start 2023-11-08T08:19:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.895+0000 7fd69b0bc700 1 -- start start 2023-11-08T08:19:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.895+0000 7fd69b0bc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd69406ff10 0x7fd69413c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.895+0000 7fd69b0bc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd694070360 0x7fd6940cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.895+0000 7fd69b0bc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69415c150 0x7fd6940ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.895+0000 7fd69b0bc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940b6ad0 con 0x7fd69415c150 2023-11-08T08:19:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.895+0000 7fd69b0bc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940b5e50 con 0x7fd69406ff10 2023-11-08T08:19:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.895+0000 7fd69b0bc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940b6130 con 0x7fd694070360 2023-11-08T08:19:20.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.895+0000 7fd69a8bb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69415c150 0x7fd6940ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:20.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.895+0000 7fd69a8bb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69415c150 0x7fd6940ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36076/0 (socket says 172.21.15.40:36076) 2023-11-08T08:19:20.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.895+0000 7fd69a8bb700 1 -- 172.21.15.40:0/3190183965 learned_addr learned my addr 172.21.15.40:0/3190183965 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:20.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.895+0000 7fd69a0ba700 1 --2- 172.21.15.40:0/3190183965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd69406ff10 0x7fd69413c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:20.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.895+0000 7fd6998b9700 1 --2- 172.21.15.40:0/3190183965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd694070360 0x7fd6940cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:20.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.895+0000 7fd69a8bb700 1 -- 172.21.15.40:0/3190183965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd694070360 msgr2=0x7fd6940cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:20.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.895+0000 7fd69a8bb700 1 --2- 172.21.15.40:0/3190183965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd694070360 0x7fd6940cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.895+0000 7fd69a8bb700 1 -- 172.21.15.40:0/3190183965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd69406ff10 msgr2=0x7fd69413c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:20.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.895+0000 7fd69a8bb700 1 --2- 172.21.15.40:0/3190183965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd69406ff10 0x7fd69413c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.899+0000 7fd69a8bb700 1 -- 172.21.15.40:0/3190183965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6900094b0 con 0x7fd69415c150 2023-11-08T08:19:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.899+0000 7fd69a8bb700 1 --2- 172.21.15.40:0/3190183965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69415c150 0x7fd6940ceb00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd68c014040 tx=0x7fd68c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.899+0000 7fd68affd700 1 -- 172.21.15.40:0/3190183965 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c0048b0 con 0x7fd69415c150 2023-11-08T08:19:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.899+0000 7fd68affd700 1 -- 172.21.15.40:0/3190183965 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c020070 con 0x7fd69415c150 2023-11-08T08:19:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.899+0000 7fd68affd700 1 -- 172.21.15.40:0/3190183965 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c0276c0 con 0x7fd69415c150 2023-11-08T08:19:20.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.899+0000 7fd69b0bc700 1 -- 172.21.15.40:0/3190183965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd68c009a70 con 0x7fd69415c150 2023-11-08T08:19:20.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.899+0000 7fd69b0bc700 1 -- 172.21.15.40:0/3190183965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6940d2730 con 0x7fd69415c150 2023-11-08T08:19:20.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.899+0000 7fd69b0bc700 1 -- 172.21.15.40:0/3190183965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6940d2450 con 0x7fd69415c150 2023-11-08T08:19:20.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.899+0000 7fd68affd700 1 -- 172.21.15.40:0/3190183965 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd68c02e070 con 0x7fd69415c150 2023-11-08T08:19:20.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.903+0000 7fd68affd700 1 --2- 172.21.15.40:0/3190183965 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd678064ee0 0x7fd678067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:20.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.903+0000 7fd68affd700 1 -- 172.21.15.40:0/3190183965 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd68c022070 con 0x7fd69415c150 2023-11-08T08:19:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.903+0000 7fd69a0ba700 1 --2- 172.21.15.40:0/3190183965 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd678064ee0 0x7fd678067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.903+0000 7fd69a0ba700 1 --2- 172.21.15.40:0/3190183965 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd678064ee0 0x7fd678067390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd690009ce0 tx=0x7fd690009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:20.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:20.911+0000 7fd68affd700 1 -- 172.21.15.40:0/3190183965 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd68c051ba0 con 0x7fd69415c150 2023-11-08T08:19:21.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.099+0000 7fd69b0bc700 1 -- 172.21.15.40:0/3190183965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd6940d05e0 con 0x7fd69415c150 2023-11-08T08:19:21.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.099+0000 7fd68affd700 1 -- 172.21.15.40:0/3190183965 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd68c02c0a0 con 0x7fd69415c150 2023-11-08T08:19:21.101 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-11-08T08:19:21.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.099+0000 7fd69b0bc700 1 -- 172.21.15.40:0/3190183965 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd678064ee0 msgr2=0x7fd678067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:21.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.099+0000 7fd69b0bc700 1 --2- 172.21.15.40:0/3190183965 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd678064ee0 0x7fd678067390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd690009ce0 tx=0x7fd690009240 comp rx=0 tx=0).stop 2023-11-08T08:19:21.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.099+0000 7fd69b0bc700 1 -- 172.21.15.40:0/3190183965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69415c150 msgr2=0x7fd6940ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:21.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.099+0000 7fd69b0bc700 1 --2- 172.21.15.40:0/3190183965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69415c150 0x7fd6940ceb00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd68c014040 tx=0x7fd68c004af0 comp rx=0 tx=0).stop 2023-11-08T08:19:21.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.099+0000 7fd69b0bc700 1 -- 172.21.15.40:0/3190183965 shutdown_connections 2023-11-08T08:19:21.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.099+0000 7fd69b0bc700 1 --2- 172.21.15.40:0/3190183965 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd678064ee0 0x7fd678067390 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:21.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.099+0000 7fd69b0bc700 1 --2- 172.21.15.40:0/3190183965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69415c150 0x7fd6940ceb00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:21.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.099+0000 7fd69b0bc700 1 --2- 172.21.15.40:0/3190183965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd694070360 0x7fd6940cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:21.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.099+0000 7fd69b0bc700 1 --2- 172.21.15.40:0/3190183965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd69406ff10 0x7fd69413c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:21.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.099+0000 7fd69b0bc700 1 -- 172.21.15.40:0/3190183965 >> 172.21.15.40:0/3190183965 conn(0x7fd6940d3680 msgr2=0x7fd6940b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:21.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.103+0000 7fd69b0bc700 1 -- 172.21.15.40:0/3190183965 shutdown_connections 2023-11-08T08:19:21.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.103+0000 7fd69b0bc700 1 -- 172.21.15.40:0/3190183965 wait complete. 2023-11-08T08:19:21.126 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T08:19:21.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.307+0000 7f100b59e700 1 Processor -- start 2023-11-08T08:19:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.307+0000 7f100b59e700 1 -- start start 2023-11-08T08:19:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.307+0000 7f100b59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f100c15c840 0x7f100c15cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.307+0000 7f100b59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f100c15d610 con 0x7f100c070080 2023-11-08T08:19:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.307+0000 7f100b59e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f100c06aae0 con 0x7f100c15c470 2023-11-08T08:19:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.307+0000 7f100b59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f100c069bf0 con 0x7f100c15c840 2023-11-08T08:19:21.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.307+0000 7f1009d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f100c15c840 0x7f100c15cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:21.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.307+0000 7f100ad9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f100c070080 0x7f100c0704c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33260/0 (socket says 172.21.15.40:33260) 2023-11-08T08:19:21.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.307+0000 7f100ad9d700 1 -- 172.21.15.40:0/2772328444 learned_addr learned my addr 172.21.15.40:0/2772328444 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:21.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.311+0000 7f1009d9b700 1 -- 172.21.15.40:0/2772328444 >> v1:172.21.15.169:6789/0 conn(0x7f100c15c470 legacy=0x7f100c0c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:21.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.311+0000 7f1009d9b700 1 -- 172.21.15.40:0/2772328444 >> v1:172.21.15.40:6789/0 conn(0x7f100c070080 legacy=0x7f100c0704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:21.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.311+0000 7f1009d9b700 1 -- 172.21.15.40:0/2772328444 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c069ed0 con 0x7f100c15c840 2023-11-08T08:19:21.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.311+0000 7f1009d9b700 1 --2- 172.21.15.40:0/2772328444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c15c840 0x7f100c15cca0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1000009850 tx=0x7f1000009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9305fcad1009f35c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.311+0000 7f100959a700 1 -- 172.21.15.40:0/2772328444 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1000011830 con 0x7f100c15c840 2023-11-08T08:19:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.311+0000 7f100959a700 1 -- 172.21.15.40:0/2772328444 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1000011990 con 0x7f100c15c840 2023-11-08T08:19:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.311+0000 7f100959a700 1 -- 172.21.15.40:0/2772328444 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1000011d20 con 0x7f100c15c840 2023-11-08T08:19:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.311+0000 7f100b59e700 1 -- 172.21.15.40:0/2772328444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c15c840 msgr2=0x7f100c15cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.311+0000 7f100b59e700 1 --2- 172.21.15.40:0/2772328444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c15c840 0x7f100c15cca0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1000009850 tx=0x7f1000009b60 comp rx=0 tx=0).stop 2023-11-08T08:19:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.311+0000 7f100b59e700 1 -- 172.21.15.40:0/2772328444 shutdown_connections 2023-11-08T08:19:21.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.311+0000 7f100b59e700 1 --2- 172.21.15.40:0/2772328444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c15c840 0x7f100c15cca0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:21.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.311+0000 7f100b59e700 1 -- 172.21.15.40:0/2772328444 >> 172.21.15.40:0/2772328444 conn(0x7f100c0b53a0 msgr2=0x7f100c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:21.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.311+0000 7f100b59e700 1 -- 172.21.15.40:0/2772328444 shutdown_connections 2023-11-08T08:19:21.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.311+0000 7f100b59e700 1 -- 172.21.15.40:0/2772328444 wait complete. 2023-11-08T08:19:21.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.315+0000 7f100b59e700 1 Processor -- start 2023-11-08T08:19:21.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.315+0000 7f100b59e700 1 -- start start 2023-11-08T08:19:21.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.315+0000 7f100b59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c070080 0x7f100c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:21.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.315+0000 7f100b59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f100c15c470 0x7f100c15a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.315+0000 7f100b59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f100c15c840 0x7f100c15a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.315+0000 7f100b59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c0cb570 con 0x7f100c070080 2023-11-08T08:19:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.315+0000 7f100b59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c0cba50 con 0x7f100c15c840 2023-11-08T08:19:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.315+0000 7f100b59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100c0cbd30 con 0x7f100c15c470 2023-11-08T08:19:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.315+0000 7f100a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c070080 0x7f100c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.315+0000 7f100a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c070080 0x7f100c0cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36088/0 (socket says 172.21.15.40:36088) 2023-11-08T08:19:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.315+0000 7f100a59c700 1 -- 172.21.15.40:0/1246830344 learned_addr learned my addr 172.21.15.40:0/1246830344 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.315+0000 7f100ad9d700 1 --2- 172.21.15.40:0/1246830344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f100c15c840 0x7f100c15a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.315+0000 7f1009d9b700 1 --2- 172.21.15.40:0/1246830344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f100c15c470 0x7f100c15a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.315+0000 7f100a59c700 1 -- 172.21.15.40:0/1246830344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f100c15c470 msgr2=0x7f100c15a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.315+0000 7f100a59c700 1 --2- 172.21.15.40:0/1246830344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f100c15c470 0x7f100c15a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.315+0000 7f100a59c700 1 -- 172.21.15.40:0/1246830344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f100c15c840 msgr2=0x7f100c15a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.315+0000 7f100a59c700 1 --2- 172.21.15.40:0/1246830344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f100c15c840 0x7f100c15a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.315+0000 7f100a59c700 1 -- 172.21.15.40:0/1246830344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10000094b0 con 0x7f100c070080 2023-11-08T08:19:21.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.319+0000 7f100a59c700 1 --2- 172.21.15.40:0/1246830344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c070080 0x7f100c0cc810 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00c6b0 tx=0x7f0ffc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:21.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.319+0000 7f0ffaffd700 1 -- 172.21.15.40:0/1246830344 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc0052d0 con 0x7f100c070080 2023-11-08T08:19:21.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.319+0000 7f0ffaffd700 1 -- 172.21.15.40:0/1246830344 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ffc005450 con 0x7f100c070080 2023-11-08T08:19:21.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.319+0000 7f100b59e700 1 -- 172.21.15.40:0/1246830344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100c06a490 con 0x7f100c070080 2023-11-08T08:19:21.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.319+0000 7f0ffaffd700 1 -- 172.21.15.40:0/1246830344 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc0038e0 con 0x7f100c070080 2023-11-08T08:19:21.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.319+0000 7f100b59e700 1 -- 172.21.15.40:0/1246830344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f100c06a1b0 con 0x7f100c070080 2023-11-08T08:19:21.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.319+0000 7f100b59e700 1 -- 172.21.15.40:0/1246830344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100c06a770 con 0x7f100c070080 2023-11-08T08:19:21.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.323+0000 7f0ffaffd700 1 -- 172.21.15.40:0/1246830344 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f0ffc019030 con 0x7f100c070080 2023-11-08T08:19:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.323+0000 7f0ffaffd700 1 --2- 172.21.15.40:0/1246830344 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0fec064ee0 0x7f0fec067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.323+0000 7f0ffaffd700 1 -- 172.21.15.40:0/1246830344 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0ffc081510 con 0x7f100c070080 2023-11-08T08:19:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.323+0000 7f1009d9b700 1 --2- 172.21.15.40:0/1246830344 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0fec064ee0 0x7f0fec067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:21.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.323+0000 7f1009d9b700 1 --2- 172.21.15.40:0/1246830344 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0fec064ee0 0x7f0fec067390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1000014040 tx=0x7f100001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.327+0000 7f0ffaffd700 1 -- 172.21.15.40:0/1246830344 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f0ffc04f2d0 con 0x7f100c070080 2023-11-08T08:19:21.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.503+0000 7f100b59e700 1 -- 172.21.15.40:0/1246830344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f100c073d20 con 0x7f100c070080 2023-11-08T08:19:21.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.503+0000 7f0ffaffd700 1 -- 172.21.15.40:0/1246830344 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0ffc0528f0 con 0x7f100c070080 2023-11-08T08:19:21.507 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:19:21.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.507+0000 7f100b59e700 1 -- 172.21.15.40:0/1246830344 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0fec064ee0 msgr2=0x7f0fec067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:21.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.507+0000 7f100b59e700 1 --2- 172.21.15.40:0/1246830344 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0fec064ee0 0x7f0fec067390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1000014040 tx=0x7f100001c040 comp rx=0 tx=0).stop 2023-11-08T08:19:21.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.507+0000 7f100b59e700 1 -- 172.21.15.40:0/1246830344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c070080 msgr2=0x7f100c0cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:21.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.507+0000 7f100b59e700 1 --2- 172.21.15.40:0/1246830344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c070080 0x7f100c0cc810 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00c6b0 tx=0x7f0ffc00c9c0 comp rx=0 tx=0).stop 2023-11-08T08:19:21.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.507+0000 7f100b59e700 1 -- 172.21.15.40:0/1246830344 shutdown_connections 2023-11-08T08:19:21.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.507+0000 7f100b59e700 1 --2- 172.21.15.40:0/1246830344 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0fec064ee0 0x7f0fec067390 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:21.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.507+0000 7f100b59e700 1 --2- 172.21.15.40:0/1246830344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f100c15c840 0x7f100c15a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:21.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.507+0000 7f100b59e700 1 --2- 172.21.15.40:0/1246830344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f100c15c470 0x7f100c15a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.507+0000 7f100b59e700 1 --2- 172.21.15.40:0/1246830344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c070080 0x7f100c0cc810 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.507+0000 7f100b59e700 1 -- 172.21.15.40:0/1246830344 >> 172.21.15.40:0/1246830344 conn(0x7f100c0b53a0 msgr2=0x7f100c0cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.507+0000 7f100b59e700 1 -- 172.21.15.40:0/1246830344 shutdown_connections 2023-11-08T08:19:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.507+0000 7f100b59e700 1 -- 172.21.15.40:0/1246830344 wait complete. 2023-11-08T08:19:21.531 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T08:19:21.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.707+0000 7f7e6ce4c700 1 Processor -- start 2023-11-08T08:19:21.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.707+0000 7f7e6ce4c700 1 -- start start 2023-11-08T08:19:21.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.711+0000 7f7e6ce4c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7e68069e00 0x7f7e6806a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:21.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.711+0000 7f7e6ce4c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6806c600 con 0x7f7e6815c640 2023-11-08T08:19:21.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.711+0000 7f7e6ce4c700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e680722f0 con 0x7f7e68071aa0 2023-11-08T08:19:21.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.711+0000 7f7e6ce4c700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e6806cef0 con 0x7f7e68069e00 2023-11-08T08:19:21.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.711+0000 7f7e677fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7e6815c640 0x7f7e680d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33274/0 (socket says 172.21.15.40:33274) 2023-11-08T08:19:21.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.711+0000 7f7e677fe700 1 -- 172.21.15.40:0/3380226869 learned_addr learned my addr 172.21.15.40:0/3380226869 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:21.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.711+0000 7f7e67fff700 1 --2- 172.21.15.40:0/3380226869 >> v2:172.21.15.169:3300/0 conn(0x7f7e68069e00 0x7f7e6806a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:21.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.711+0000 7f7e667fc700 1 -- 172.21.15.40:0/3380226869 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1491396710 0 0) 0x7f7e6806c600 con 0x7f7e6815c640 2023-11-08T08:19:21.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.711+0000 7f7e667fc700 1 -- 172.21.15.40:0/3380226869 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e500035d0 con 0x7f7e6815c640 2023-11-08T08:19:21.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.711+0000 7f7e667fc700 1 -- 172.21.15.40:0/3380226869 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056194605 0 0) 0x7f7e680722f0 con 0x7f7e68071aa0 2023-11-08T08:19:21.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.711+0000 7f7e667fc700 1 -- 172.21.15.40:0/3380226869 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e6806c600 con 0x7f7e68071aa0 2023-11-08T08:19:21.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.711+0000 7f7e67fff700 1 -- 172.21.15.40:0/3380226869 >> v1:172.21.15.169:6789/0 conn(0x7f7e68071aa0 legacy=0x7f7e6806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:21.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.711+0000 7f7e67fff700 1 -- 172.21.15.40:0/3380226869 >> v1:172.21.15.40:6789/0 conn(0x7f7e6815c640 legacy=0x7f7e680d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:21.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.711+0000 7f7e67fff700 1 -- 172.21.15.40:0/3380226869 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6806da20 con 0x7f7e68069e00 2023-11-08T08:19:21.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.711+0000 7f7e67fff700 1 --2- 172.21.15.40:0/3380226869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e68069e00 0x7f7e6806a280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009d90 tx=0x7f7e6000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9bd503810c8a3e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:21.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.711+0000 7f7e667fc700 1 -- 172.21.15.40:0/3380226869 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6000e070 con 0x7f7e68069e00 2023-11-08T08:19:21.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.711+0000 7f7e667fc700 1 -- 172.21.15.40:0/3380226869 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6000bb20 con 0x7f7e68069e00 2023-11-08T08:19:21.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.711+0000 7f7e667fc700 1 -- 172.21.15.40:0/3380226869 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e60012430 con 0x7f7e68069e00 2023-11-08T08:19:21.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.715+0000 7f7e6ce4c700 1 -- 172.21.15.40:0/3380226869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e68069e00 msgr2=0x7f7e6806a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:21.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.715+0000 7f7e6ce4c700 1 --2- 172.21.15.40:0/3380226869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e68069e00 0x7f7e6806a280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009d90 tx=0x7f7e6000b330 comp rx=0 tx=0).stop 2023-11-08T08:19:21.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.715+0000 7f7e6ce4c700 1 -- 172.21.15.40:0/3380226869 shutdown_connections 2023-11-08T08:19:21.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.715+0000 7f7e6ce4c700 1 --2- 172.21.15.40:0/3380226869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e68069e00 0x7f7e6806a280 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:21.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.715+0000 7f7e6ce4c700 1 -- 172.21.15.40:0/3380226869 >> 172.21.15.40:0/3380226869 conn(0x7f7e680b6520 msgr2=0x7f7e68071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:21.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.715+0000 7f7e6ce4c700 1 -- 172.21.15.40:0/3380226869 shutdown_connections 2023-11-08T08:19:21.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.715+0000 7f7e6ce4c700 1 -- 172.21.15.40:0/3380226869 wait complete. 2023-11-08T08:19:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.715+0000 7f7e6ce4c700 1 Processor -- start 2023-11-08T08:19:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.715+0000 7f7e6ce4c700 1 -- start start 2023-11-08T08:19:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e6ce4c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e68069e00 0x7f7e681f1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e6ce4c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e68071aa0 0x7f7e680c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e6ce4c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e6815c640 0x7f7e680c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e6ce4c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e680d38f0 con 0x7f7e68071aa0 2023-11-08T08:19:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e6ce4c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e680c6920 con 0x7f7e6815c640 2023-11-08T08:19:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e6ce4c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e680cb280 con 0x7f7e68069e00 2023-11-08T08:19:21.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e66ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e68071aa0 0x7f7e680c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:21.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e66ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e68071aa0 0x7f7e680c7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36090/0 (socket says 172.21.15.40:36090) 2023-11-08T08:19:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e66ffd700 1 -- 172.21.15.40:0/378466958 learned_addr learned my addr 172.21.15.40:0/378466958 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e677fe700 1 --2- 172.21.15.40:0/378466958 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e68069e00 0x7f7e681f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e67fff700 1 --2- 172.21.15.40:0/378466958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e6815c640 0x7f7e680c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e66ffd700 1 -- 172.21.15.40:0/378466958 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e68069e00 msgr2=0x7f7e681f1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e66ffd700 1 --2- 172.21.15.40:0/378466958 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e68069e00 0x7f7e681f1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e66ffd700 1 -- 172.21.15.40:0/378466958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e6815c640 msgr2=0x7f7e680c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e66ffd700 1 --2- 172.21.15.40:0/378466958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e6815c640 0x7f7e680c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e66ffd700 1 -- 172.21.15.40:0/378466958 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e60009a70 con 0x7f7e68071aa0 2023-11-08T08:19:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e677fe700 1 --2- 172.21.15.40:0/378466958 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e68069e00 0x7f7e681f1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e66ffd700 1 --2- 172.21.15.40:0/378466958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e68071aa0 0x7f7e680c7450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00c490 tx=0x7f7e5c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:21.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e4ffff700 1 -- 172.21.15.40:0/378466958 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0052d0 con 0x7f7e68071aa0 2023-11-08T08:19:21.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e4ffff700 1 -- 172.21.15.40:0/378466958 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c005450 con 0x7f7e68071aa0 2023-11-08T08:19:21.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e4ffff700 1 -- 172.21.15.40:0/378466958 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0038e0 con 0x7f7e68071aa0 2023-11-08T08:19:21.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e6ce4c700 1 -- 172.21.15.40:0/378466958 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e68072010 con 0x7f7e68071aa0 2023-11-08T08:19:21.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.719+0000 7f7e6ce4c700 1 -- 172.21.15.40:0/378466958 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6815ca10 con 0x7f7e68071aa0 2023-11-08T08:19:21.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.723+0000 7f7e6ce4c700 1 -- 172.21.15.40:0/378466958 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e6806c830 con 0x7f7e68071aa0 2023-11-08T08:19:21.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.723+0000 7f7e4ffff700 1 -- 172.21.15.40:0/378466958 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7e5c019030 con 0x7f7e68071aa0 2023-11-08T08:19:21.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.723+0000 7f7e4ffff700 1 --2- 172.21.15.40:0/378466958 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e54064ee0 0x7f7e54067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:21.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.723+0000 7f7e4ffff700 1 -- 172.21.15.40:0/378466958 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7e5c081560 con 0x7f7e68071aa0 2023-11-08T08:19:21.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.727+0000 7f7e677fe700 1 --2- 172.21.15.40:0/378466958 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e54064ee0 0x7f7e54067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.727+0000 7f7e677fe700 1 --2- 172.21.15.40:0/378466958 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e54064ee0 0x7f7e54067390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7e580011e0 tx=0x7f7e58009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:21.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.731+0000 7f7e4ffff700 1 -- 172.21.15.40:0/378466958 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7e5c04f410 con 0x7f7e68071aa0 2023-11-08T08:19:21.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.923+0000 7f7e6ce4c700 1 -- 172.21.15.40:0/378466958 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7e68073cb0 con 0x7f7e68071aa0 2023-11-08T08:19:21.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.923+0000 7f7e4ffff700 1 -- 172.21.15.40:0/378466958 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7e5c052a30 con 0x7f7e68071aa0 2023-11-08T08:19:21.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.927+0000 7f7e6ce4c700 1 -- 172.21.15.40:0/378466958 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e54064ee0 msgr2=0x7f7e54067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:21.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.927+0000 7f7e6ce4c700 1 --2- 172.21.15.40:0/378466958 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e54064ee0 0x7f7e54067390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7e580011e0 tx=0x7f7e58009040 comp rx=0 tx=0).stop 2023-11-08T08:19:21.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.927+0000 7f7e6ce4c700 1 -- 172.21.15.40:0/378466958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e68071aa0 msgr2=0x7f7e680c7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.927+0000 7f7e6ce4c700 1 --2- 172.21.15.40:0/378466958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e68071aa0 0x7f7e680c7450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00c490 tx=0x7f7e5c00c850 comp rx=0 tx=0).stop 2023-11-08T08:19:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.927+0000 7f7e6ce4c700 1 -- 172.21.15.40:0/378466958 shutdown_connections 2023-11-08T08:19:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.927+0000 7f7e6ce4c700 1 --2- 172.21.15.40:0/378466958 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e54064ee0 0x7f7e54067390 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.927+0000 7f7e6ce4c700 1 --2- 172.21.15.40:0/378466958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e6815c640 0x7f7e680c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:21.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.927+0000 7f7e6ce4c700 1 --2- 172.21.15.40:0/378466958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e68071aa0 0x7f7e680c7450 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:21.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.927+0000 7f7e6ce4c700 1 --2- 172.21.15.40:0/378466958 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e68069e00 0x7f7e681f1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:21.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.927+0000 7f7e6ce4c700 1 -- 172.21.15.40:0/378466958 >> 172.21.15.40:0/378466958 conn(0x7f7e680b6520 msgr2=0x7f7e680b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:21.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.927+0000 7f7e6ce4c700 1 -- 172.21.15.40:0/378466958 shutdown_connections 2023-11-08T08:19:21.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:21.927+0000 7f7e6ce4c700 1 -- 172.21.15.40:0/378466958 wait complete. 2023-11-08T08:19:21.933 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-11-08T08:19:21.952 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi040.front.sepia.ceph.com... 2023-11-08T08:19:21.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:21.953 DEBUG:teuthology.orchestra.run.smithi040:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-11-08T08:19:21.993 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi040.stderr:ceph-fuse[41249]: fuse finished with error 0 and tester_r 0 2023-11-08T08:19:21.998 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:19:26.232 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:19:26.241 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:19:28.100 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-11-08T08:19:28.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:28.101 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-11-08T08:19:28.132 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:19:28.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.359+0000 7f470184f700 1 Processor -- start 2023-11-08T08:19:28.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.359+0000 7f470184f700 1 -- start start 2023-11-08T08:19:28.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.359+0000 7f470184f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46fc15c640 0x7f46fc0d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:28.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.359+0000 7f470184f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f46fc069e00 0x7f46fc06a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:28.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.359+0000 7f470184f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46fc06c600 con 0x7f46fc071aa0 2023-11-08T08:19:28.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.359+0000 7f470184f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46fc06cef0 con 0x7f46fc069e00 2023-11-08T08:19:28.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.359+0000 7f470184f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc06a7c0 con 0x7f46fc15c640 2023-11-08T08:19:28.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.359+0000 7f46fbfff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f46fc071aa0 0x7f46fc06d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49664/0 (socket says 172.21.15.40:49664) 2023-11-08T08:19:28.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.359+0000 7f46fbfff700 1 -- 172.21.15.40:0/3544729758 learned_addr learned my addr 172.21.15.40:0/3544729758 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:28.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f470084d700 1 --2- 172.21.15.40:0/3544729758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46fc15c640 0x7f46fc0d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:28.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f470104e700 1 --2- 172.21.15.40:0/3544729758 >> v2:172.21.15.40:3300/0 conn(0x7f46fc069e00 0x7f46fc06a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:28.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f46fb7fe700 1 -- 172.21.15.40:0/3544729758 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3065246353 0 0) 0x7f46fc06c600 con 0x7f46fc071aa0 2023-11-08T08:19:28.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f46fb7fe700 1 -- 172.21.15.40:0/3544729758 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46ec0035d0 con 0x7f46fc071aa0 2023-11-08T08:19:28.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f470104e700 1 -- 172.21.15.40:0/3544729758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46fc15c640 msgr2=0x7f46fc0d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:28.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f470104e700 1 --2- 172.21.15.40:0/3544729758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46fc15c640 0x7f46fc0d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:28.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f470104e700 1 -- 172.21.15.40:0/3544729758 >> v1:172.21.15.40:6789/0 conn(0x7f46fc071aa0 legacy=0x7f46fc06d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:28.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f46fb7fe700 1 -- 172.21.15.40:0/3544729758 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4274538666 0 0) 0x7f46ec0035d0 con 0x7f46fc071aa0 2023-11-08T08:19:28.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f470104e700 1 -- 172.21.15.40:0/3544729758 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46fc0722f0 con 0x7f46fc069e00 2023-11-08T08:19:28.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f470084d700 1 --2- 172.21.15.40:0/3544729758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46fc15c640 0x7f46fc0d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:28.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f470104e700 1 --2- 172.21.15.40:0/3544729758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46fc069e00 0x7f46fc06a280 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009d90 tx=0x7f46f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553778a404c6296a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:28.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f46fb7fe700 1 -- 172.21.15.40:0/3544729758 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f400e070 con 0x7f46fc069e00 2023-11-08T08:19:28.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f46fb7fe700 1 -- 172.21.15.40:0/3544729758 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f4003a40 con 0x7f46fc069e00 2023-11-08T08:19:28.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f46fb7fe700 1 -- 172.21.15.40:0/3544729758 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e8001670 con 0x7f46fc069e00 2023-11-08T08:19:28.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f470184f700 1 -- 172.21.15.40:0/3544729758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46fc069e00 msgr2=0x7f46fc06a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:28.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f470184f700 1 --2- 172.21.15.40:0/3544729758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46fc069e00 0x7f46fc06a280 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009d90 tx=0x7f46f400b330 comp rx=0 tx=0).stop 2023-11-08T08:19:28.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f470184f700 1 -- 172.21.15.40:0/3544729758 shutdown_connections 2023-11-08T08:19:28.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f470184f700 1 --2- 172.21.15.40:0/3544729758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46fc069e00 0x7f46fc06a280 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:28.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f470184f700 1 --2- 172.21.15.40:0/3544729758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46fc15c640 0x7f46fc0d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:28.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f470184f700 1 -- 172.21.15.40:0/3544729758 >> 172.21.15.40:0/3544729758 conn(0x7f46fc0b6520 msgr2=0x7f46fc071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:28.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f470184f700 1 -- 172.21.15.40:0/3544729758 shutdown_connections 2023-11-08T08:19:28.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.363+0000 7f470184f700 1 -- 172.21.15.40:0/3544729758 wait complete. 2023-11-08T08:19:28.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.367+0000 7f470184f700 1 Processor -- start 2023-11-08T08:19:28.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.367+0000 7f470184f700 1 -- start start 2023-11-08T08:19:28.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f470184f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46fc069e00 0x7f46fc0d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:28.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f470184f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46fc071aa0 0x7f46fc15a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:28.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f470184f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46fc15c640 0x7f46fc0cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:28.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f470184f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0cdb40 con 0x7f46fc071aa0 2023-11-08T08:19:28.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f470184f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0cd7f0 con 0x7f46fc069e00 2023-11-08T08:19:28.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f470184f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc15ad90 con 0x7f46fc15c640 2023-11-08T08:19:28.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f46fbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46fc071aa0 0x7f46fc15a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:28.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f46fbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46fc071aa0 0x7f46fc15a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52076/0 (socket says 172.21.15.40:52076) 2023-11-08T08:19:28.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f46fbfff700 1 -- 172.21.15.40:0/3611911891 learned_addr learned my addr 172.21.15.40:0/3611911891 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:28.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f470104e700 1 --2- 172.21.15.40:0/3611911891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46fc15c640 0x7f46fc0cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:28.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f470084d700 1 --2- 172.21.15.40:0/3611911891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46fc069e00 0x7f46fc0d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:28.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f46fbfff700 1 -- 172.21.15.40:0/3611911891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46fc15c640 msgr2=0x7f46fc0cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:28.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f46fbfff700 1 --2- 172.21.15.40:0/3611911891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46fc15c640 0x7f46fc0cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:28.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f46fbfff700 1 -- 172.21.15.40:0/3611911891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46fc069e00 msgr2=0x7f46fc0d1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:28.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f46fbfff700 1 --2- 172.21.15.40:0/3611911891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46fc069e00 0x7f46fc0d1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:28.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f46fbfff700 1 -- 172.21.15.40:0/3611911891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f00094b0 con 0x7f46fc071aa0 2023-11-08T08:19:28.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f470084d700 1 --2- 172.21.15.40:0/3611911891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46fc069e00 0x7f46fc0d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:28.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f46fbfff700 1 --2- 172.21.15.40:0/3611911891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46fc071aa0 0x7f46fc15a850 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f46e800c520 tx=0x7f46e800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:28.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f46f97fa700 1 -- 172.21.15.40:0/3611911891 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e8004ca0 con 0x7f46fc071aa0 2023-11-08T08:19:28.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f46f97fa700 1 -- 172.21.15.40:0/3611911891 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e8004e20 con 0x7f46fc071aa0 2023-11-08T08:19:28.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f470184f700 1 -- 172.21.15.40:0/3611911891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f4009a70 con 0x7f46fc071aa0 2023-11-08T08:19:28.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f46f97fa700 1 -- 172.21.15.40:0/3611911891 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e8007310 con 0x7f46fc071aa0 2023-11-08T08:19:28.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.371+0000 7f470184f700 1 -- 172.21.15.40:0/3611911891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46fc15ca10 con 0x7f46fc071aa0 2023-11-08T08:19:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.375+0000 7f470184f700 1 -- 172.21.15.40:0/3611911891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46fc06da20 con 0x7f46fc071aa0 2023-11-08T08:19:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.375+0000 7f46f97fa700 1 -- 172.21.15.40:0/3611911891 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f46e801a020 con 0x7f46fc071aa0 2023-11-08T08:19:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.375+0000 7f46f97fa700 1 --2- 172.21.15.40:0/3611911891 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f46e0064f30 0x7f46e00673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:28.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.375+0000 7f46f97fa700 1 -- 172.21.15.40:0/3611911891 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f46e8081df0 con 0x7f46fc071aa0 2023-11-08T08:19:28.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.379+0000 7f470084d700 1 --2- 172.21.15.40:0/3611911891 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f46e0064f30 0x7f46e00673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:28.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.379+0000 7f470084d700 1 --2- 172.21.15.40:0/3611911891 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f46e0064f30 0x7f46e00673e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009d30 tx=0x7f46f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:28.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.383+0000 7f46f97fa700 1 -- 172.21.15.40:0/3611911891 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f46e804fbd0 con 0x7f46fc071aa0 2023-11-08T08:19:28.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.567+0000 7f470184f700 1 -- 172.21.15.40:0/3611911891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46fc072010 con 0x7f46fc071aa0 2023-11-08T08:19:28.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.571+0000 7f46f97fa700 1 -- 172.21.15.40:0/3611911891 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19652 (secure 0 0 0) 0x7f46e80531f0 con 0x7f46fc071aa0 2023-11-08T08:19:28.573 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:28.573 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":28,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:08.980158+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:19:28.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.571+0000 7f470184f700 1 -- 172.21.15.40:0/3611911891 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f46e0064f30 msgr2=0x7f46e00673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:28.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.571+0000 7f470184f700 1 --2- 172.21.15.40:0/3611911891 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f46e0064f30 0x7f46e00673e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009d30 tx=0x7f46f0008040 comp rx=0 tx=0).stop 2023-11-08T08:19:28.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.571+0000 7f470184f700 1 -- 172.21.15.40:0/3611911891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46fc071aa0 msgr2=0x7f46fc15a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:28.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.571+0000 7f470184f700 1 --2- 172.21.15.40:0/3611911891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46fc071aa0 0x7f46fc15a850 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f46e800c520 tx=0x7f46e800c830 comp rx=0 tx=0).stop 2023-11-08T08:19:28.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.571+0000 7f470184f700 1 -- 172.21.15.40:0/3611911891 shutdown_connections 2023-11-08T08:19:28.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.571+0000 7f470184f700 1 --2- 172.21.15.40:0/3611911891 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f46e0064f30 0x7f46e00673e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:28.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.571+0000 7f470184f700 1 --2- 172.21.15.40:0/3611911891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46fc15c640 0x7f46fc0cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:28.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.571+0000 7f470184f700 1 --2- 172.21.15.40:0/3611911891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46fc071aa0 0x7f46fc15a850 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:28.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.571+0000 7f470184f700 1 --2- 172.21.15.40:0/3611911891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46fc069e00 0x7f46fc0d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:28.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.571+0000 7f470184f700 1 -- 172.21.15.40:0/3611911891 >> 172.21.15.40:0/3611911891 conn(0x7f46fc0b6520 msgr2=0x7f46fc15ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:28.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.571+0000 7f470184f700 1 -- 172.21.15.40:0/3611911891 shutdown_connections 2023-11-08T08:19:28.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.571+0000 7f470184f700 1 -- 172.21.15.40:0/3611911891 wait complete. 2023-11-08T08:19:28.598 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:19:28.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.779+0000 7fa49359a700 1 Processor -- start 2023-11-08T08:19:28.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.779+0000 7fa49359a700 1 -- start start 2023-11-08T08:19:28.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.783+0000 7fa49359a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa48c0b43a0 0x7fa48c15e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:28.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.783+0000 7fa49359a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa48c0b1640 0x7fa48c070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:28.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.783+0000 7fa49359a700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa48c15a550 con 0x7fa48c0c9040 2023-11-08T08:19:28.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.783+0000 7fa49359a700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa48c06b0a0 con 0x7fa48c0b1640 2023-11-08T08:19:28.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.783+0000 7fa49359a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c0caac0 con 0x7fa48c0b43a0 2023-11-08T08:19:28.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.783+0000 7fa491d97700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa48c0b1640 0x7fa48c070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:28.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.783+0000 7fa491d97700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa48c0b1640 0x7fa48c070080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33324/0 (socket says 172.21.15.40:33324) 2023-11-08T08:19:28.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.783+0000 7fa491d97700 1 -- 172.21.15.40:0/2972428522 learned_addr learned my addr 172.21.15.40:0/2972428522 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:28.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.783+0000 7fa492598700 1 --2- 172.21.15.40:0/2972428522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa48c0b43a0 0x7fa48c15e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:28.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.783+0000 7fa491d97700 1 -- 172.21.15.40:0/2972428522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa48c0b43a0 msgr2=0x7fa48c15e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:28.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.783+0000 7fa491d97700 1 --2- 172.21.15.40:0/2972428522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa48c0b43a0 0x7fa48c15e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:28.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.783+0000 7fa491d97700 1 -- 172.21.15.40:0/2972428522 >> v1:172.21.15.169:6789/0 conn(0x7fa48c0c9040 legacy=0x7fa48c069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:28.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.783+0000 7fa491d97700 1 -- 172.21.15.40:0/2972428522 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c0cada0 con 0x7fa48c0b1640 2023-11-08T08:19:28.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.783+0000 7fa491d97700 1 --2- 172.21.15.40:0/2972428522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa48c0b1640 0x7fa48c070080 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009910 tx=0x7fa47c009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83b5e38fa60888a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:28.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.787+0000 7fa491596700 1 -- 172.21.15.40:0/2972428522 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c011830 con 0x7fa48c0b1640 2023-11-08T08:19:28.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.787+0000 7fa491596700 1 -- 172.21.15.40:0/2972428522 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47c011990 con 0x7fa48c0b1640 2023-11-08T08:19:28.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.787+0000 7fa491596700 1 -- 172.21.15.40:0/2972428522 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c011d40 con 0x7fa48c0b1640 2023-11-08T08:19:28.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.787+0000 7fa49359a700 1 -- 172.21.15.40:0/2972428522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa48c0b1640 msgr2=0x7fa48c070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:28.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.787+0000 7fa49359a700 1 --2- 172.21.15.40:0/2972428522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa48c0b1640 0x7fa48c070080 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009910 tx=0x7fa47c009c20 comp rx=0 tx=0).stop 2023-11-08T08:19:28.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.787+0000 7fa49359a700 1 -- 172.21.15.40:0/2972428522 shutdown_connections 2023-11-08T08:19:28.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.787+0000 7fa49359a700 1 --2- 172.21.15.40:0/2972428522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa48c0b1640 0x7fa48c070080 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:28.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.787+0000 7fa49359a700 1 --2- 172.21.15.40:0/2972428522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa48c0b43a0 0x7fa48c15e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:28.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.787+0000 7fa49359a700 1 -- 172.21.15.40:0/2972428522 >> 172.21.15.40:0/2972428522 conn(0x7fa48c0b0f80 msgr2=0x7fa48c0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:28.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.787+0000 7fa49359a700 1 -- 172.21.15.40:0/2972428522 shutdown_connections 2023-11-08T08:19:28.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.787+0000 7fa49359a700 1 -- 172.21.15.40:0/2972428522 wait complete. 2023-11-08T08:19:28.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.791+0000 7fa49359a700 1 Processor -- start 2023-11-08T08:19:28.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.791+0000 7fa49359a700 1 -- start start 2023-11-08T08:19:28.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.791+0000 7fa49359a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa48c0b1640 0x7fa48c151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:28.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.791+0000 7fa49359a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa48c0b43a0 0x7fa48c1533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:28.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.791+0000 7fa49359a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa48c0c9040 0x7fa48c150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:28.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.791+0000 7fa49359a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c1586b0 con 0x7fa48c0b43a0 2023-11-08T08:19:28.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.791+0000 7fa49359a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c157750 con 0x7fa48c0b1640 2023-11-08T08:19:28.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.791+0000 7fa49359a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c157a30 con 0x7fa48c0c9040 2023-11-08T08:19:28.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.795+0000 7fa491d97700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa48c0b43a0 0x7fa48c1533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:28.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.795+0000 7fa491d97700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa48c0b43a0 0x7fa48c1533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52092/0 (socket says 172.21.15.40:52092) 2023-11-08T08:19:28.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.795+0000 7fa491d97700 1 -- 172.21.15.40:0/3245842484 learned_addr learned my addr 172.21.15.40:0/3245842484 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:28.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.795+0000 7fa492598700 1 --2- 172.21.15.40:0/3245842484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa48c0b1640 0x7fa48c151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:28.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.795+0000 7fa492d99700 1 --2- 172.21.15.40:0/3245842484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa48c0c9040 0x7fa48c150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:28.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.795+0000 7fa491d97700 1 -- 172.21.15.40:0/3245842484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa48c0c9040 msgr2=0x7fa48c150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:28.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.795+0000 7fa491d97700 1 --2- 172.21.15.40:0/3245842484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa48c0c9040 0x7fa48c150d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:28.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.795+0000 7fa491d97700 1 -- 172.21.15.40:0/3245842484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa48c0b1640 msgr2=0x7fa48c151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:28.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.795+0000 7fa491d97700 1 --2- 172.21.15.40:0/3245842484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa48c0b1640 0x7fa48c151630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:28.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.795+0000 7fa491d97700 1 -- 172.21.15.40:0/3245842484 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c009570 con 0x7fa48c0b43a0 2023-11-08T08:19:28.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.795+0000 7fa492598700 1 --2- 172.21.15.40:0/3245842484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa48c0b1640 0x7fa48c151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:28.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.795+0000 7fa491d97700 1 --2- 172.21.15.40:0/3245842484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa48c0b43a0 0x7fa48c1533d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa48c072eb0 tx=0x7fa47c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:28.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.795+0000 7fa482ffd700 1 -- 172.21.15.40:0/3245842484 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c004590 con 0x7fa48c0b43a0 2023-11-08T08:19:28.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.795+0000 7fa482ffd700 1 -- 172.21.15.40:0/3245842484 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47c01e070 con 0x7fa48c0b43a0 2023-11-08T08:19:28.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.795+0000 7fa49359a700 1 -- 172.21.15.40:0/3245842484 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48c06b5d0 con 0x7fa48c0b43a0 2023-11-08T08:19:28.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.795+0000 7fa482ffd700 1 -- 172.21.15.40:0/3245842484 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c003900 con 0x7fa48c0b43a0 2023-11-08T08:19:28.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.795+0000 7fa49359a700 1 -- 172.21.15.40:0/3245842484 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48c06b2f0 con 0x7fa48c0b43a0 2023-11-08T08:19:28.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.795+0000 7fa49359a700 1 -- 172.21.15.40:0/3245842484 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa48c15eb10 con 0x7fa48c0b43a0 2023-11-08T08:19:28.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.799+0000 7fa482ffd700 1 -- 172.21.15.40:0/3245842484 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa47c00f050 con 0x7fa48c0b43a0 2023-11-08T08:19:28.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.799+0000 7fa482ffd700 1 --2- 172.21.15.40:0/3245842484 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa470064ee0 0x7fa470067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:28.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.799+0000 7fa482ffd700 1 -- 172.21.15.40:0/3245842484 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa47c083250 con 0x7fa48c0b43a0 2023-11-08T08:19:28.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.799+0000 7fa492598700 1 --2- 172.21.15.40:0/3245842484 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa470064ee0 0x7fa470067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:28.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.803+0000 7fa492598700 1 --2- 172.21.15.40:0/3245842484 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa470064ee0 0x7fa470067390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0b3af0 tx=0x7fa488008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:28.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.807+0000 7fa482ffd700 1 -- 172.21.15.40:0/3245842484 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa47c051010 con 0x7fa48c0b43a0 2023-11-08T08:19:28.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.979+0000 7fa49359a700 1 -- 172.21.15.40:0/3245842484 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa48c073cb0 con 0x7fa48c0b43a0 2023-11-08T08:19:28.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.979+0000 7fa482ffd700 1 -- 172.21.15.40:0/3245842484 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19652 (secure 0 0 0) 0x7fa47c054630 con 0x7fa48c0b43a0 2023-11-08T08:19:28.983 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:28.983 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":28,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:08.980158+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:19:28.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.983+0000 7fa49359a700 1 -- 172.21.15.40:0/3245842484 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa470064ee0 msgr2=0x7fa470067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:28.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.983+0000 7fa49359a700 1 --2- 172.21.15.40:0/3245842484 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa470064ee0 0x7fa470067390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0b3af0 tx=0x7fa488008040 comp rx=0 tx=0).stop 2023-11-08T08:19:28.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.983+0000 7fa49359a700 1 -- 172.21.15.40:0/3245842484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa48c0b43a0 msgr2=0x7fa48c1533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:28.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.983+0000 7fa49359a700 1 --2- 172.21.15.40:0/3245842484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa48c0b43a0 0x7fa48c1533d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa48c072eb0 tx=0x7fa47c004770 comp rx=0 tx=0).stop 2023-11-08T08:19:28.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.983+0000 7fa49359a700 1 -- 172.21.15.40:0/3245842484 shutdown_connections 2023-11-08T08:19:28.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.983+0000 7fa49359a700 1 --2- 172.21.15.40:0/3245842484 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa470064ee0 0x7fa470067390 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:28.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.983+0000 7fa49359a700 1 --2- 172.21.15.40:0/3245842484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa48c0c9040 0x7fa48c150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:28.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.983+0000 7fa49359a700 1 --2- 172.21.15.40:0/3245842484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa48c0b43a0 0x7fa48c1533d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:28.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.983+0000 7fa49359a700 1 --2- 172.21.15.40:0/3245842484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa48c0b1640 0x7fa48c151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:28.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.983+0000 7fa49359a700 1 -- 172.21.15.40:0/3245842484 >> 172.21.15.40:0/3245842484 conn(0x7fa48c0b0f80 msgr2=0x7fa48c0ae290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:28.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.983+0000 7fa49359a700 1 -- 172.21.15.40:0/3245842484 shutdown_connections 2023-11-08T08:19:28.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:28.983+0000 7fa49359a700 1 -- 172.21.15.40:0/3245842484 wait complete. 2023-11-08T08:19:29.009 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T08:19:29.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.187+0000 7fe8ab59e700 1 Processor -- start 2023-11-08T08:19:29.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.187+0000 7fe8ab59e700 1 -- start start 2023-11-08T08:19:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.187+0000 7fe8ab59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe8ac159cd0 0x7fe8ac15d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.187+0000 7fe8ab59e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe8ac06ff10 0x7fe8ac070390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:29.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.187+0000 7fe8ab59e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ac15d650 con 0x7fe8ac0cc1a0 2023-11-08T08:19:29.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.187+0000 7fe8ab59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ac06be80 con 0x7fe8ac159cd0 2023-11-08T08:19:29.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.187+0000 7fe8ab59e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ac0708d0 con 0x7fe8ac06ff10 2023-11-08T08:19:29.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.191+0000 7fe8aa59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe8ac159cd0 0x7fe8ac15d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.191+0000 7fe8aa59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe8ac159cd0 0x7fe8ac15d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52098/0 (socket says 172.21.15.40:52098) 2023-11-08T08:19:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.191+0000 7fe8aa59c700 1 -- 172.21.15.40:0/545795802 learned_addr learned my addr 172.21.15.40:0/545795802 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.191+0000 7fe8aad9d700 1 --2- 172.21.15.40:0/545795802 >> v2:172.21.15.169:3300/0 conn(0x7fe8ac06ff10 0x7fe8ac070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.191+0000 7fe8a959a700 1 -- 172.21.15.40:0/545795802 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1447636396 0 0) 0x7fe8ac15d650 con 0x7fe8ac0cc1a0 2023-11-08T08:19:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.191+0000 7fe8a959a700 1 -- 172.21.15.40:0/545795802 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8940035d0 con 0x7fe8ac0cc1a0 2023-11-08T08:19:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.191+0000 7fe8aa59c700 1 -- 172.21.15.40:0/545795802 >> v2:172.21.15.169:3300/0 conn(0x7fe8ac06ff10 msgr2=0x7fe8ac070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.191+0000 7fe8aa59c700 1 --2- 172.21.15.40:0/545795802 >> v2:172.21.15.169:3300/0 conn(0x7fe8ac06ff10 0x7fe8ac070390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.191+0000 7fe8aa59c700 1 -- 172.21.15.40:0/545795802 >> v1:172.21.15.169:6789/0 conn(0x7fe8ac0cc1a0 legacy=0x7fe8ac06c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.191+0000 7fe8aa59c700 1 -- 172.21.15.40:0/545795802 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac15a0a0 con 0x7fe8ac159cd0 2023-11-08T08:19:29.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.191+0000 7fe8aad9d700 1 --2- 172.21.15.40:0/545795802 >> v2:172.21.15.169:3300/0 conn(0x7fe8ac06ff10 0x7fe8ac070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:29.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.191+0000 7fe8aa59c700 1 --2- 172.21.15.40:0/545795802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8ac159cd0 0x7fe8ac15d110 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac15c6e0 tx=0x7fe89c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f5340f9a86afe9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:29.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.191+0000 7fe8a959a700 1 -- 172.21.15.40:0/545795802 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c011830 con 0x7fe8ac159cd0 2023-11-08T08:19:29.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.191+0000 7fe8a959a700 1 -- 172.21.15.40:0/545795802 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c011990 con 0x7fe8ac159cd0 2023-11-08T08:19:29.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.191+0000 7fe8a959a700 1 -- 172.21.15.40:0/545795802 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c011d20 con 0x7fe8ac159cd0 2023-11-08T08:19:29.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.191+0000 7fe8ab59e700 1 -- 172.21.15.40:0/545795802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8ac159cd0 msgr2=0x7fe8ac15d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:29.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.191+0000 7fe8ab59e700 1 --2- 172.21.15.40:0/545795802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8ac159cd0 0x7fe8ac15d110 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac15c6e0 tx=0x7fe89c009bf0 comp rx=0 tx=0).stop 2023-11-08T08:19:29.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.195+0000 7fe8ab59e700 1 -- 172.21.15.40:0/545795802 shutdown_connections 2023-11-08T08:19:29.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.195+0000 7fe8ab59e700 1 --2- 172.21.15.40:0/545795802 >> v2:172.21.15.169:3300/0 conn(0x7fe8ac06ff10 0x7fe8ac070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:29.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.195+0000 7fe8ab59e700 1 --2- 172.21.15.40:0/545795802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8ac159cd0 0x7fe8ac15d110 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:29.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.195+0000 7fe8ab59e700 1 -- 172.21.15.40:0/545795802 >> 172.21.15.40:0/545795802 conn(0x7fe8ac0cf280 msgr2=0x7fe8ac0cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:29.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.195+0000 7fe8ab59e700 1 -- 172.21.15.40:0/545795802 shutdown_connections 2023-11-08T08:19:29.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.195+0000 7fe8ab59e700 1 -- 172.21.15.40:0/545795802 wait complete. 2023-11-08T08:19:29.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.195+0000 7fe8ab59e700 1 Processor -- start 2023-11-08T08:19:29.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.195+0000 7fe8ab59e700 1 -- start start 2023-11-08T08:19:29.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe8ab59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8ac06ff10 0x7fe8ac143fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:29.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe8ab59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8ac0cc1a0 0x7fe8ac0b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:29.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe8ab59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8ac159cd0 0x7fe8ac15a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:29.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe8ab59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0b2fa0 con 0x7fe8ac0cc1a0 2023-11-08T08:19:29.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe8ab59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0b1f00 con 0x7fe8ac159cd0 2023-11-08T08:19:29.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe8ab59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0b21e0 con 0x7fe8ac06ff10 2023-11-08T08:19:29.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe8a9d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8ac0cc1a0 0x7fe8ac0b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe8a9d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8ac0cc1a0 0x7fe8ac0b1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52106/0 (socket says 172.21.15.40:52106) 2023-11-08T08:19:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe8a9d9b700 1 -- 172.21.15.40:0/2199877083 learned_addr learned my addr 172.21.15.40:0/2199877083 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe8aa59c700 1 --2- 172.21.15.40:0/2199877083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8ac06ff10 0x7fe8ac143fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe8aad9d700 1 --2- 172.21.15.40:0/2199877083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8ac159cd0 0x7fe8ac15a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe8a9d9b700 1 -- 172.21.15.40:0/2199877083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8ac06ff10 msgr2=0x7fe8ac143fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe8a9d9b700 1 --2- 172.21.15.40:0/2199877083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8ac06ff10 0x7fe8ac143fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe8a9d9b700 1 -- 172.21.15.40:0/2199877083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8ac159cd0 msgr2=0x7fe8ac15a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe8a9d9b700 1 --2- 172.21.15.40:0/2199877083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8ac159cd0 0x7fe8ac15a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe8a9d9b700 1 -- 172.21.15.40:0/2199877083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a4009a70 con 0x7fe8ac0cc1a0 2023-11-08T08:19:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe8aad9d700 1 --2- 172.21.15.40:0/2199877083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8ac159cd0 0x7fe8ac15a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe8a9d9b700 1 --2- 172.21.15.40:0/2199877083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8ac0cc1a0 0x7fe8ac0b1540 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000c730 tx=0x7fe8a000ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe89affd700 1 -- 172.21.15.40:0/2199877083 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a00052d0 con 0x7fe8ac0cc1a0 2023-11-08T08:19:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe89affd700 1 -- 172.21.15.40:0/2199877083 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a0005450 con 0x7fe8ac0cc1a0 2023-11-08T08:19:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe89affd700 1 -- 172.21.15.40:0/2199877083 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a00038e0 con 0x7fe8ac0cc1a0 2023-11-08T08:19:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe8ab59e700 1 -- 172.21.15.40:0/2199877083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89c009570 con 0x7fe8ac0cc1a0 2023-11-08T08:19:29.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.199+0000 7fe8ab59e700 1 -- 172.21.15.40:0/2199877083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac0ce260 con 0x7fe8ac0cc1a0 2023-11-08T08:19:29.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.203+0000 7fe8ab59e700 1 -- 172.21.15.40:0/2199877083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ac0cdf80 con 0x7fe8ac0cc1a0 2023-11-08T08:19:29.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.203+0000 7fe89affd700 1 -- 172.21.15.40:0/2199877083 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fe8a0019030 con 0x7fe8ac0cc1a0 2023-11-08T08:19:29.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.203+0000 7fe89affd700 1 --2- 172.21.15.40:0/2199877083 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe88c064ee0 0x7fe88c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:29.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.203+0000 7fe89affd700 1 -- 172.21.15.40:0/2199877083 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe8a00816b0 con 0x7fe8ac0cc1a0 2023-11-08T08:19:29.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.207+0000 7fe8aa59c700 1 --2- 172.21.15.40:0/2199877083 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe88c064ee0 0x7fe88c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:29.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.207+0000 7fe8aa59c700 1 --2- 172.21.15.40:0/2199877083 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe88c064ee0 0x7fe88c067390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe89c000c00 tx=0x7fe89c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:29.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.211+0000 7fe89affd700 1 -- 172.21.15.40:0/2199877083 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fe8a004f3f0 con 0x7fe8ac0cc1a0 2023-11-08T08:19:29.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:29.419+0000 7fe8ab59e700 1 -- 172.21.15.40:0/2199877083 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe8ac0cca80 con 0x7fe8ac0cc1a0 2023-11-08T08:19:30.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.023+0000 7fe89affd700 1 -- 172.21.15.40:0/2199877083 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe8a0052a10 con 0x7fe8ac0cc1a0 2023-11-08T08:19:30.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.027+0000 7fe8ab59e700 1 -- 172.21.15.40:0/2199877083 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe88c064ee0 msgr2=0x7fe88c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:30.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.027+0000 7fe8ab59e700 1 --2- 172.21.15.40:0/2199877083 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe88c064ee0 0x7fe88c067390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe89c000c00 tx=0x7fe89c008030 comp rx=0 tx=0).stop 2023-11-08T08:19:30.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.027+0000 7fe8ab59e700 1 -- 172.21.15.40:0/2199877083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8ac0cc1a0 msgr2=0x7fe8ac0b1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.027+0000 7fe8ab59e700 1 --2- 172.21.15.40:0/2199877083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8ac0cc1a0 0x7fe8ac0b1540 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000c730 tx=0x7fe8a000ca40 comp rx=0 tx=0).stop 2023-11-08T08:19:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.027+0000 7fe8ab59e700 1 -- 172.21.15.40:0/2199877083 shutdown_connections 2023-11-08T08:19:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.027+0000 7fe8ab59e700 1 --2- 172.21.15.40:0/2199877083 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe88c064ee0 0x7fe88c067390 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.027+0000 7fe8ab59e700 1 --2- 172.21.15.40:0/2199877083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8ac159cd0 0x7fe8ac15a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:30.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.027+0000 7fe8ab59e700 1 --2- 172.21.15.40:0/2199877083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8ac0cc1a0 0x7fe8ac0b1540 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:30.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.027+0000 7fe8ab59e700 1 --2- 172.21.15.40:0/2199877083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8ac06ff10 0x7fe8ac143fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:30.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.027+0000 7fe8ab59e700 1 -- 172.21.15.40:0/2199877083 >> 172.21.15.40:0/2199877083 conn(0x7fe8ac0cf280 msgr2=0x7fe8ac155330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:30.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.027+0000 7fe8ab59e700 1 -- 172.21.15.40:0/2199877083 shutdown_connections 2023-11-08T08:19:30.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.027+0000 7fe8ab59e700 1 -- 172.21.15.40:0/2199877083 wait complete. 2023-11-08T08:19:30.056 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-11-08T08:19:30.057 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T08:19:30.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.243+0000 7f390359e700 1 Processor -- start 2023-11-08T08:19:30.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.247+0000 7f390359e700 1 -- start start 2023-11-08T08:19:30.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.247+0000 7f390359e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f390415c150 0x7f39040b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:30.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.247+0000 7f390359e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f390406ff10 0x7f3904070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:30.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.247+0000 7f390359e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390415c520 con 0x7f390415fc30 2023-11-08T08:19:30.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.247+0000 7f390359e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39040b4240 con 0x7f390406ff10 2023-11-08T08:19:30.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.247+0000 7f390359e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39040708a0 con 0x7f390415c150 2023-11-08T08:19:30.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.251+0000 7f3901d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f390406ff10 0x7f3904070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:30.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.251+0000 7f390259c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f390415c150 0x7f39040b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:30.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.251+0000 7f3901d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f390406ff10 0x7f3904070360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52122/0 (socket says 172.21.15.40:52122) 2023-11-08T08:19:30.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.251+0000 7f3901d9b700 1 -- 172.21.15.40:0/2217100492 learned_addr learned my addr 172.21.15.40:0/2217100492 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:30.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.251+0000 7f390259c700 1 -- 172.21.15.40:0/2217100492 >> v2:172.21.15.40:3300/0 conn(0x7f390406ff10 msgr2=0x7f3904070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:30.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.251+0000 7f390259c700 1 --2- 172.21.15.40:0/2217100492 >> v2:172.21.15.40:3300/0 conn(0x7f390406ff10 0x7f3904070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:30.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.251+0000 7f390259c700 1 -- 172.21.15.40:0/2217100492 >> v1:172.21.15.169:6789/0 conn(0x7f390415fc30 legacy=0x7f3904160050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:30.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.251+0000 7f390259c700 1 -- 172.21.15.40:0/2217100492 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390406ac00 con 0x7f390415c150 2023-11-08T08:19:30.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.251+0000 7f390159a700 1 -- 172.21.15.40:0/2217100492 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 21826768 0 0) 0x7f390415c520 con 0x7f390415fc30 2023-11-08T08:19:30.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.251+0000 7f3901d9b700 1 --2- 172.21.15.40:0/2217100492 >> v2:172.21.15.40:3300/0 conn(0x7f390406ff10 0x7f3904070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:30.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.251+0000 7f390259c700 1 --2- 172.21.15.40:0/2217100492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f390415c150 0x7f39040b3d00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3904071900 tx=0x7f38f4009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb7030fe9783742b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:30.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.251+0000 7f390159a700 1 -- 172.21.15.40:0/2217100492 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4011830 con 0x7f390415c150 2023-11-08T08:19:30.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.251+0000 7f390159a700 1 -- 172.21.15.40:0/2217100492 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f4011990 con 0x7f390415c150 2023-11-08T08:19:30.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.251+0000 7f390159a700 1 -- 172.21.15.40:0/2217100492 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4011d20 con 0x7f390415c150 2023-11-08T08:19:30.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.255+0000 7f390359e700 1 -- 172.21.15.40:0/2217100492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f390415c150 msgr2=0x7f39040b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:30.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.255+0000 7f390359e700 1 --2- 172.21.15.40:0/2217100492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f390415c150 0x7f39040b3d00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3904071900 tx=0x7f38f4009b30 comp rx=0 tx=0).stop 2023-11-08T08:19:30.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.255+0000 7f390359e700 1 -- 172.21.15.40:0/2217100492 shutdown_connections 2023-11-08T08:19:30.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.255+0000 7f390359e700 1 --2- 172.21.15.40:0/2217100492 >> v2:172.21.15.40:3300/0 conn(0x7f390406ff10 0x7f3904070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:30.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.255+0000 7f390359e700 1 --2- 172.21.15.40:0/2217100492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f390415c150 0x7f39040b3d00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:30.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.255+0000 7f390359e700 1 -- 172.21.15.40:0/2217100492 >> 172.21.15.40:0/2217100492 conn(0x7f39040d3680 msgr2=0x7f39040cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:30.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.255+0000 7f390359e700 1 -- 172.21.15.40:0/2217100492 shutdown_connections 2023-11-08T08:19:30.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.255+0000 7f390359e700 1 -- 172.21.15.40:0/2217100492 wait complete. 2023-11-08T08:19:30.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.259+0000 7f390359e700 1 Processor -- start 2023-11-08T08:19:30.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.259+0000 7f390359e700 1 -- start start 2023-11-08T08:19:30.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.259+0000 7f390359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f390406ff10 0x7f39040ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.259+0000 7f390359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f390415c150 0x7f390414a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.263+0000 7f390359e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f390415fc30 0x7f390414a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.263+0000 7f390359e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390414aef0 con 0x7f390406ff10 2023-11-08T08:19:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.263+0000 7f390359e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390415b190 con 0x7f390415fc30 2023-11-08T08:19:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.263+0000 7f390359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390414b140 con 0x7f390415c150 2023-11-08T08:19:30.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.263+0000 7f390259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f390406ff10 0x7f39040ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:30.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.263+0000 7f390259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f390406ff10 0x7f39040ce260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52132/0 (socket says 172.21.15.40:52132) 2023-11-08T08:19:30.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.263+0000 7f390259c700 1 -- 172.21.15.40:0/86701368 learned_addr learned my addr 172.21.15.40:0/86701368 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:30.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.263+0000 7f3901d9b700 1 --2- 172.21.15.40:0/86701368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f390415c150 0x7f390414a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:30.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.263+0000 7f3902d9d700 1 --2- 172.21.15.40:0/86701368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f390415fc30 0x7f390414a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:30.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.263+0000 7f390259c700 1 -- 172.21.15.40:0/86701368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f390415c150 msgr2=0x7f390414a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:30.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.263+0000 7f390259c700 1 --2- 172.21.15.40:0/86701368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f390415c150 0x7f390414a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:30.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.263+0000 7f390259c700 1 -- 172.21.15.40:0/86701368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f390415fc30 msgr2=0x7f390414a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:30.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.263+0000 7f390259c700 1 --2- 172.21.15.40:0/86701368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f390415fc30 0x7f390414a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:30.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.263+0000 7f390259c700 1 -- 172.21.15.40:0/86701368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f8009520 con 0x7f390406ff10 2023-11-08T08:19:30.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.263+0000 7f390259c700 1 --2- 172.21.15.40:0/86701368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f390406ff10 0x7f39040ce260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009fd0 tx=0x7f38f4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:30.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.263+0000 7f38f2ffd700 1 -- 172.21.15.40:0/86701368 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4004480 con 0x7f390406ff10 2023-11-08T08:19:30.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.263+0000 7f38f2ffd700 1 -- 172.21.15.40:0/86701368 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f401f070 con 0x7f390406ff10 2023-11-08T08:19:30.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.263+0000 7f390359e700 1 -- 172.21.15.40:0/86701368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f40094b0 con 0x7f390406ff10 2023-11-08T08:19:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.263+0000 7f38f2ffd700 1 -- 172.21.15.40:0/86701368 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4003900 con 0x7f390406ff10 2023-11-08T08:19:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.263+0000 7f390359e700 1 -- 172.21.15.40:0/86701368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39040d2450 con 0x7f390406ff10 2023-11-08T08:19:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.267+0000 7f390359e700 1 -- 172.21.15.40:0/86701368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390406cc50 con 0x7f390406ff10 2023-11-08T08:19:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.267+0000 7f38f2ffd700 1 -- 172.21.15.40:0/86701368 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f38f402a020 con 0x7f390406ff10 2023-11-08T08:19:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.267+0000 7f38f2ffd700 1 --2- 172.21.15.40:0/86701368 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f38e4064ee0 0x7f38e4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.267+0000 7f38f2ffd700 1 -- 172.21.15.40:0/86701368 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f38f4083600 con 0x7f390406ff10 2023-11-08T08:19:30.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.271+0000 7f3901d9b700 1 --2- 172.21.15.40:0/86701368 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f38e4064ee0 0x7f38e4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:30.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.271+0000 7f3901d9b700 1 --2- 172.21.15.40:0/86701368 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f38e4064ee0 0x7f38e4067390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f390406d3a0 tx=0x7f38f8009290 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:30.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.275+0000 7f38f2ffd700 1 -- 172.21.15.40:0/86701368 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f38f4051430 con 0x7f390406ff10 2023-11-08T08:19:30.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:30.495+0000 7f390359e700 1 -- 172.21.15.40:0/86701368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f39040d2730 con 0x7f390406ff10 2023-11-08T08:19:31.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.027+0000 7f38f2ffd700 1 -- 172.21.15.40:0/86701368 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f38f4054a50 con 0x7f390406ff10 2023-11-08T08:19:31.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.031+0000 7f390359e700 1 -- 172.21.15.40:0/86701368 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f38e4064ee0 msgr2=0x7f38e4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:31.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.031+0000 7f390359e700 1 --2- 172.21.15.40:0/86701368 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f38e4064ee0 0x7f38e4067390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f390406d3a0 tx=0x7f38f8009290 comp rx=0 tx=0).stop 2023-11-08T08:19:31.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.031+0000 7f390359e700 1 -- 172.21.15.40:0/86701368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f390406ff10 msgr2=0x7f39040ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:31.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.031+0000 7f390359e700 1 --2- 172.21.15.40:0/86701368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f390406ff10 0x7f39040ce260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009fd0 tx=0x7f38f4004770 comp rx=0 tx=0).stop 2023-11-08T08:19:31.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.031+0000 7f390359e700 1 -- 172.21.15.40:0/86701368 shutdown_connections 2023-11-08T08:19:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.031+0000 7f390359e700 1 --2- 172.21.15.40:0/86701368 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f38e4064ee0 0x7f38e4067390 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.031+0000 7f390359e700 1 --2- 172.21.15.40:0/86701368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f390415fc30 0x7f390414a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.031+0000 7f390359e700 1 --2- 172.21.15.40:0/86701368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f390415c150 0x7f390414a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.031+0000 7f390359e700 1 --2- 172.21.15.40:0/86701368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f390406ff10 0x7f39040ce260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.031+0000 7f390359e700 1 -- 172.21.15.40:0/86701368 >> 172.21.15.40:0/86701368 conn(0x7f39040d3680 msgr2=0x7f3904160030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:31.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.031+0000 7f390359e700 1 -- 172.21.15.40:0/86701368 shutdown_connections 2023-11-08T08:19:31.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.031+0000 7f390359e700 1 -- 172.21.15.40:0/86701368 wait complete. 2023-11-08T08:19:31.060 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T08:19:31.061 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:19:31.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.239+0000 7f4aeb1d2700 1 Processor -- start 2023-11-08T08:19:31.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.239+0000 7f4aeb1d2700 1 -- start start 2023-11-08T08:19:31.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.239+0000 7f4aeb1d2700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ae415c1a0 0x7f4ae406c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:31.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.239+0000 7f4aeb1d2700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ae4070790 0x7f4ae406a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:31.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.239+0000 7f4aeb1d2700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ae40adf90 con 0x7f4ae40cdeb0 2023-11-08T08:19:31.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.239+0000 7f4aeb1d2700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae4069bf0 con 0x7f4ae4070790 2023-11-08T08:19:31.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.239+0000 7f4aeb1d2700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae40ce370 con 0x7f4ae415c1a0 2023-11-08T08:19:31.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4aea9d1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ae4070790 0x7f4ae406a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:31.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4aea9d1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ae4070790 0x7f4ae406a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52144/0 (socket says 172.21.15.40:52144) 2023-11-08T08:19:31.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4aea9d1700 1 -- 172.21.15.40:0/4160798427 learned_addr learned my addr 172.21.15.40:0/4160798427 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:31.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4aea1d0700 1 --2- 172.21.15.40:0/4160798427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ae415c1a0 0x7f4ae406c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:31.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4aea9d1700 1 -- 172.21.15.40:0/4160798427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ae415c1a0 msgr2=0x7f4ae406c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:31.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4aea9d1700 1 --2- 172.21.15.40:0/4160798427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ae415c1a0 0x7f4ae406c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4aea9d1700 1 -- 172.21.15.40:0/4160798427 >> v1:172.21.15.169:6789/0 conn(0x7f4ae40cdeb0 legacy=0x7f4ae4070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:31.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4aea9d1700 1 -- 172.21.15.40:0/4160798427 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae40cc4a0 con 0x7f4ae4070790 2023-11-08T08:19:31.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4aea1d0700 1 --2- 172.21.15.40:0/4160798427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ae415c1a0 0x7f4ae406c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:31.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4aea9d1700 1 --2- 172.21.15.40:0/4160798427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ae4070790 0x7f4ae406a010 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009e50 tx=0x7f4adc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c202d52987ef17f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:31.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4ae91ce700 1 -- 172.21.15.40:0/4160798427 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc00e070 con 0x7f4ae4070790 2023-11-08T08:19:31.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4ae91ce700 1 -- 172.21.15.40:0/4160798427 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4adc00b9c0 con 0x7f4ae4070790 2023-11-08T08:19:31.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4ae91ce700 1 -- 172.21.15.40:0/4160798427 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc00bd70 con 0x7f4ae4070790 2023-11-08T08:19:31.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4aeb1d2700 1 -- 172.21.15.40:0/4160798427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ae4070790 msgr2=0x7f4ae406a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:31.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4aeb1d2700 1 --2- 172.21.15.40:0/4160798427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ae4070790 0x7f4ae406a010 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009e50 tx=0x7f4adc00b330 comp rx=0 tx=0).stop 2023-11-08T08:19:31.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4aeb1d2700 1 -- 172.21.15.40:0/4160798427 shutdown_connections 2023-11-08T08:19:31.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4aeb1d2700 1 --2- 172.21.15.40:0/4160798427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ae4070790 0x7f4ae406a010 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4aeb1d2700 1 --2- 172.21.15.40:0/4160798427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ae415c1a0 0x7f4ae406c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4aeb1d2700 1 -- 172.21.15.40:0/4160798427 >> 172.21.15.40:0/4160798427 conn(0x7f4ae40b0f80 msgr2=0x7f4ae40ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:31.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4aeb1d2700 1 -- 172.21.15.40:0/4160798427 shutdown_connections 2023-11-08T08:19:31.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.243+0000 7f4aeb1d2700 1 -- 172.21.15.40:0/4160798427 wait complete. 2023-11-08T08:19:31.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.247+0000 7f4aeb1d2700 1 Processor -- start 2023-11-08T08:19:31.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.247+0000 7f4aeb1d2700 1 -- start start 2023-11-08T08:19:31.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4aeb1d2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ae4070790 0x7f4ae414aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:31.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4aeb1d2700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ae40cdeb0 0x7f4ae40c1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:31.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4aeb1d2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ae415c1a0 0x7f4ae40c23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:31.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4aeb1d2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae40c4e10 con 0x7f4ae4070790 2023-11-08T08:19:31.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4aeb1d2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae40c50f0 con 0x7f4ae415c1a0 2023-11-08T08:19:31.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4aeb1d2700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae40c53d0 con 0x7f4ae40cdeb0 2023-11-08T08:19:31.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4aea1d0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ae4070790 0x7f4ae414aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:31.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4aea1d0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ae4070790 0x7f4ae414aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52158/0 (socket says 172.21.15.40:52158) 2023-11-08T08:19:31.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4aea1d0700 1 -- 172.21.15.40:0/2001964329 learned_addr learned my addr 172.21.15.40:0/2001964329 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:31.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4aea9d1700 1 --2- 172.21.15.40:0/2001964329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ae415c1a0 0x7f4ae40c23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:31.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4ae99cf700 1 --2- 172.21.15.40:0/2001964329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ae40cdeb0 0x7f4ae40c1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:31.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4aea1d0700 1 -- 172.21.15.40:0/2001964329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ae40cdeb0 msgr2=0x7f4ae40c1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:31.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4aea1d0700 1 --2- 172.21.15.40:0/2001964329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ae40cdeb0 0x7f4ae40c1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4aea1d0700 1 -- 172.21.15.40:0/2001964329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ae415c1a0 msgr2=0x7f4ae40c23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:31.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4aea1d0700 1 --2- 172.21.15.40:0/2001964329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ae415c1a0 0x7f4ae40c23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4aea1d0700 1 -- 172.21.15.40:0/2001964329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc009b30 con 0x7f4ae4070790 2023-11-08T08:19:31.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4aea9d1700 1 --2- 172.21.15.40:0/2001964329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ae415c1a0 0x7f4ae40c23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:31.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4aea1d0700 1 --2- 172.21.15.40:0/2001964329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ae4070790 0x7f4ae414aa80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000e810 tx=0x7f4ae000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:31.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4adaffd700 1 -- 172.21.15.40:0/2001964329 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0004230 con 0x7f4ae4070790 2023-11-08T08:19:31.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4adaffd700 1 -- 172.21.15.40:0/2001964329 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae0004390 con 0x7f4ae4070790 2023-11-08T08:19:31.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4aeb1d2700 1 -- 172.21.15.40:0/2001964329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae406cdf0 con 0x7f4ae4070790 2023-11-08T08:19:31.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4adaffd700 1 -- 172.21.15.40:0/2001964329 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae00036c0 con 0x7f4ae4070790 2023-11-08T08:19:31.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.251+0000 7f4aeb1d2700 1 -- 172.21.15.40:0/2001964329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae40cc780 con 0x7f4ae4070790 2023-11-08T08:19:31.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.255+0000 7f4aeb1d2700 1 -- 172.21.15.40:0/2001964329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae415c570 con 0x7f4ae4070790 2023-11-08T08:19:31.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.255+0000 7f4adaffd700 1 -- 172.21.15.40:0/2001964329 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4ae0022070 con 0x7f4ae4070790 2023-11-08T08:19:31.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.259+0000 7f4adaffd700 1 --2- 172.21.15.40:0/2001964329 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4ac8064ee0 0x7f4ac8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:31.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.259+0000 7f4adaffd700 1 -- 172.21.15.40:0/2001964329 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4ae0081bd0 con 0x7f4ae4070790 2023-11-08T08:19:31.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.259+0000 7f4ae99cf700 1 --2- 172.21.15.40:0/2001964329 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4ac8064ee0 0x7f4ac8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:31.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.259+0000 7f4ae99cf700 1 --2- 172.21.15.40:0/2001964329 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4ac8064ee0 0x7f4ac8067390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4ae415f4b0 tx=0x7f4ad4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:31.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.263+0000 7f4adaffd700 1 -- 172.21.15.40:0/2001964329 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4ae004fa20 con 0x7f4ae4070790 2023-11-08T08:19:31.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.483+0000 7f4aeb1d2700 1 -- 172.21.15.40:0/2001964329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ae415d8d0 con 0x7f4ae4070790 2023-11-08T08:19:31.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.483+0000 7f4adaffd700 1 -- 172.21.15.40:0/2001964329 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4641 (secure 0 0 0) 0x7f4ae415d8d0 con 0x7f4ae4070790 2023-11-08T08:19:31.487 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:31.487 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/196211087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":196211087},{"type":"v1","addr":"172.21.15.40:6834","nonce":196211087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/67127106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":67127106},{"type":"v1","addr":"172.21.15.40:6835","nonce":67127106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1231240563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1231240563},{"type":"v1","addr":"172.21.15.169:6835","nonce":1231240563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:19:04.456508+0000","modified":"2023-11-08T08:19:31.023792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6837/1989495204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1989495204},{"type":"v1","addr":"172.21.15.169:6837","nonce":1989495204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T08:19:31.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.487+0000 7f4aeb1d2700 1 -- 172.21.15.40:0/2001964329 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4ac8064ee0 msgr2=0x7f4ac8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:31.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.487+0000 7f4aeb1d2700 1 --2- 172.21.15.40:0/2001964329 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4ac8064ee0 0x7f4ac8067390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4ae415f4b0 tx=0x7f4ad4009040 comp rx=0 tx=0).stop 2023-11-08T08:19:31.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.487+0000 7f4aeb1d2700 1 -- 172.21.15.40:0/2001964329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ae4070790 msgr2=0x7f4ae414aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:31.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.487+0000 7f4aeb1d2700 1 --2- 172.21.15.40:0/2001964329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ae4070790 0x7f4ae414aa80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000e810 tx=0x7f4ae000ebd0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.487+0000 7f4aeb1d2700 1 -- 172.21.15.40:0/2001964329 shutdown_connections 2023-11-08T08:19:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.487+0000 7f4aeb1d2700 1 --2- 172.21.15.40:0/2001964329 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4ac8064ee0 0x7f4ac8067390 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.487+0000 7f4aeb1d2700 1 --2- 172.21.15.40:0/2001964329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ae415c1a0 0x7f4ae40c23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.487+0000 7f4aeb1d2700 1 --2- 172.21.15.40:0/2001964329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ae40cdeb0 0x7f4ae40c1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.487+0000 7f4aeb1d2700 1 --2- 172.21.15.40:0/2001964329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ae4070790 0x7f4ae414aa80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.487+0000 7f4aeb1d2700 1 -- 172.21.15.40:0/2001964329 >> 172.21.15.40:0/2001964329 conn(0x7f4ae40b0f80 msgr2=0x7f4ae40aee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:31.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.487+0000 7f4aeb1d2700 1 -- 172.21.15.40:0/2001964329 shutdown_connections 2023-11-08T08:19:31.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.487+0000 7f4aeb1d2700 1 -- 172.21.15.40:0/2001964329 wait complete. 2023-11-08T08:19:31.493 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 9 2023-11-08T08:19:31.516 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:19:31.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.687+0000 7f108359e700 1 Processor -- start 2023-11-08T08:19:31.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.687+0000 7f108359e700 1 -- start start 2023-11-08T08:19:31.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f108359e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f108415c640 0x7f10840d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:31.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f108359e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1084071aa0 0x7f108406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:31.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f108359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1084069e00 0x7f108406a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:31.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f108359e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108406cef0 con 0x7f1084071aa0 2023-11-08T08:19:31.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f108359e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108406a7c0 con 0x7f108415c640 2023-11-08T08:19:31.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f108359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108406c600 con 0x7f1084069e00 2023-11-08T08:19:31.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f1079d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1084071aa0 0x7f108406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:31.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f1079d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1084071aa0 0x7f108406d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52164/0 (socket says 172.21.15.40:52164) 2023-11-08T08:19:31.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f1079d9b700 1 -- 172.21.15.40:0/3895244523 learned_addr learned my addr 172.21.15.40:0/3895244523 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:31.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f108259c700 1 --2- 172.21.15.40:0/3895244523 >> v2:172.21.15.169:3300/0 conn(0x7f108415c640 0x7f10840d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:31.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f1082d9d700 1 --2- 172.21.15.40:0/3895244523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1084069e00 0x7f108406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:31.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f1079d9b700 1 -- 172.21.15.40:0/3895244523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1084069e00 msgr2=0x7f108406a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:31.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f1079d9b700 1 --2- 172.21.15.40:0/3895244523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1084069e00 0x7f108406a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f1079d9b700 1 -- 172.21.15.40:0/3895244523 >> v2:172.21.15.169:3300/0 conn(0x7f108415c640 msgr2=0x7f10840d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:31.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f1079d9b700 1 --2- 172.21.15.40:0/3895244523 >> v2:172.21.15.169:3300/0 conn(0x7f108415c640 0x7f10840d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f1079d9b700 1 -- 172.21.15.40:0/3895244523 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10840722f0 con 0x7f1084071aa0 2023-11-08T08:19:31.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f108259c700 1 --2- 172.21.15.40:0/3895244523 >> v2:172.21.15.169:3300/0 conn(0x7f108415c640 0x7f10840d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:31.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f1079d9b700 1 --2- 172.21.15.40:0/3895244523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1084071aa0 0x7f108406d310 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f108406e290 tx=0x7f106c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e70a8ca3319b1e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:31.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f1081d9b700 1 -- 172.21.15.40:0/3895244523 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c00b8a0 con 0x7f1084071aa0 2023-11-08T08:19:31.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f1081d9b700 1 -- 172.21.15.40:0/3895244523 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106c00ba00 con 0x7f1084071aa0 2023-11-08T08:19:31.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.691+0000 7f1081d9b700 1 -- 172.21.15.40:0/3895244523 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c00bd90 con 0x7f1084071aa0 2023-11-08T08:19:31.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.695+0000 7f108359e700 1 -- 172.21.15.40:0/3895244523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1084071aa0 msgr2=0x7f108406d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:31.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.695+0000 7f108359e700 1 --2- 172.21.15.40:0/3895244523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1084071aa0 0x7f108406d310 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f108406e290 tx=0x7f106c009bf0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.695+0000 7f108359e700 1 -- 172.21.15.40:0/3895244523 shutdown_connections 2023-11-08T08:19:31.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.695+0000 7f108359e700 1 --2- 172.21.15.40:0/3895244523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1084071aa0 0x7f108406d310 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.695+0000 7f108359e700 1 --2- 172.21.15.40:0/3895244523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1084069e00 0x7f108406a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.695+0000 7f108359e700 1 --2- 172.21.15.40:0/3895244523 >> v2:172.21.15.169:3300/0 conn(0x7f108415c640 0x7f10840d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.695+0000 7f108359e700 1 -- 172.21.15.40:0/3895244523 >> 172.21.15.40:0/3895244523 conn(0x7f10840b6520 msgr2=0x7f1084071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:31.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.695+0000 7f108359e700 1 -- 172.21.15.40:0/3895244523 shutdown_connections 2023-11-08T08:19:31.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.695+0000 7f108359e700 1 -- 172.21.15.40:0/3895244523 wait complete. 2023-11-08T08:19:31.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.699+0000 7f108359e700 1 Processor -- start 2023-11-08T08:19:31.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.699+0000 7f108359e700 1 -- start start 2023-11-08T08:19:31.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.699+0000 7f108359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1084069e00 0x7f10840d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:31.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.699+0000 7f108359e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1084071aa0 0x7f10841e3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:31.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.699+0000 7f108359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f108415c640 0x7f108415b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:31.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.699+0000 7f108359e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10840b3210 con 0x7f1084069e00 2023-11-08T08:19:31.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.699+0000 7f108359e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10840b2590 con 0x7f1084071aa0 2023-11-08T08:19:31.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.699+0000 7f108359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10840b2870 con 0x7f108415c640 2023-11-08T08:19:31.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.699+0000 7f108259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1084069e00 0x7f10840d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:31.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.699+0000 7f108259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1084069e00 0x7f10840d19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52166/0 (socket says 172.21.15.40:52166) 2023-11-08T08:19:31.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.699+0000 7f108259c700 1 -- 172.21.15.40:0/1064665888 learned_addr learned my addr 172.21.15.40:0/1064665888 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:31.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.699+0000 7f1082d9d700 1 --2- 172.21.15.40:0/1064665888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f108415c640 0x7f108415b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:31.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.699+0000 7f1079d9b700 1 --2- 172.21.15.40:0/1064665888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1084071aa0 0x7f10841e3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:31.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.699+0000 7f108259c700 1 -- 172.21.15.40:0/1064665888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f108415c640 msgr2=0x7f108415b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:31.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.699+0000 7f108259c700 1 --2- 172.21.15.40:0/1064665888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f108415c640 0x7f108415b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.699+0000 7f108259c700 1 -- 172.21.15.40:0/1064665888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1084071aa0 msgr2=0x7f10841e3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:31.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.699+0000 7f108259c700 1 --2- 172.21.15.40:0/1064665888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1084071aa0 0x7f10841e3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.699+0000 7f108259c700 1 -- 172.21.15.40:0/1064665888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c009570 con 0x7f1084069e00 2023-11-08T08:19:31.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.699+0000 7f108259c700 1 --2- 172.21.15.40:0/1064665888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1084069e00 0x7f10840d19b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f107400e760 tx=0x7f107400ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:31.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.703+0000 7f107b7fe700 1 -- 172.21.15.40:0/1064665888 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107400c070 con 0x7f1084069e00 2023-11-08T08:19:31.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.703+0000 7f107b7fe700 1 -- 172.21.15.40:0/1064665888 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1074004250 con 0x7f1084069e00 2023-11-08T08:19:31.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.703+0000 7f108359e700 1 -- 172.21.15.40:0/1064665888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1084072010 con 0x7f1084069e00 2023-11-08T08:19:31.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.703+0000 7f107b7fe700 1 -- 172.21.15.40:0/1064665888 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10740036c0 con 0x7f1084069e00 2023-11-08T08:19:31.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.703+0000 7f108359e700 1 -- 172.21.15.40:0/1064665888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f108415ca10 con 0x7f1084069e00 2023-11-08T08:19:31.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.703+0000 7f108359e700 1 -- 172.21.15.40:0/1064665888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f108406c830 con 0x7f1084069e00 2023-11-08T08:19:31.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.703+0000 7f107b7fe700 1 -- 172.21.15.40:0/1064665888 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1074019020 con 0x7f1084069e00 2023-11-08T08:19:31.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.707+0000 7f107b7fe700 1 --2- 172.21.15.40:0/1064665888 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1064064ee0 0x7f1064067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:31.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.707+0000 7f107b7fe700 1 -- 172.21.15.40:0/1064665888 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1074081e90 con 0x7f1084069e00 2023-11-08T08:19:31.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.707+0000 7f1079d9b700 1 --2- 172.21.15.40:0/1064665888 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1064064ee0 0x7f1064067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:31.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.707+0000 7f1079d9b700 1 --2- 172.21.15.40:0/1064665888 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1064064ee0 0x7f1064067390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f106c009860 tx=0x7f106c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:31.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.711+0000 7f107b7fe700 1 -- 172.21.15.40:0/1064665888 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f107404fd40 con 0x7f1084069e00 2023-11-08T08:19:31.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.891+0000 7f108359e700 1 -- 172.21.15.40:0/1064665888 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1084073cb0 con 0x7f1084069e00 2023-11-08T08:19:31.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.895+0000 7f107b7fe700 1 -- 172.21.15.40:0/1064665888 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19652 (secure 0 0 0) 0x7f10740094e0 con 0x7f1084069e00 2023-11-08T08:19:31.898 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:31.899 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":28,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:08.980158+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:19:31.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.899+0000 7f108359e700 1 -- 172.21.15.40:0/1064665888 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1064064ee0 msgr2=0x7f1064067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:31.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.899+0000 7f108359e700 1 --2- 172.21.15.40:0/1064665888 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1064064ee0 0x7f1064067390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f106c009860 tx=0x7f106c008030 comp rx=0 tx=0).stop 2023-11-08T08:19:31.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.899+0000 7f108359e700 1 -- 172.21.15.40:0/1064665888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1084069e00 msgr2=0x7f10840d19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:31.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.899+0000 7f108359e700 1 --2- 172.21.15.40:0/1064665888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1084069e00 0x7f10840d19b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f107400e760 tx=0x7f107400ea70 comp rx=0 tx=0).stop 2023-11-08T08:19:31.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.899+0000 7f108359e700 1 -- 172.21.15.40:0/1064665888 shutdown_connections 2023-11-08T08:19:31.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.899+0000 7f108359e700 1 --2- 172.21.15.40:0/1064665888 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1064064ee0 0x7f1064067390 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.899+0000 7f108359e700 1 --2- 172.21.15.40:0/1064665888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f108415c640 0x7f108415b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.899+0000 7f108359e700 1 --2- 172.21.15.40:0/1064665888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1084071aa0 0x7f10841e3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.899+0000 7f108359e700 1 --2- 172.21.15.40:0/1064665888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1084069e00 0x7f10840d19b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:31.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.899+0000 7f108359e700 1 -- 172.21.15.40:0/1064665888 >> 172.21.15.40:0/1064665888 conn(0x7f10840b6520 msgr2=0x7f10841e3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:31.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.899+0000 7f108359e700 1 -- 172.21.15.40:0/1064665888 shutdown_connections 2023-11-08T08:19:31.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:31.899+0000 7f108359e700 1 -- 172.21.15.40:0/1064665888 wait complete. 2023-11-08T08:19:31.923 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T08:19:31.924 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T08:19:32.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.103+0000 7fcf25a9f700 1 Processor -- start 2023-11-08T08:19:32.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.103+0000 7fcf25a9f700 1 -- start start 2023-11-08T08:19:32.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.103+0000 7fcf25a9f700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcf200b1640 0x7fcf20070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:32.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.103+0000 7fcf25a9f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcf200c9040 0x7fcf200705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:32.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.103+0000 7fcf25a9f700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf2015a550 con 0x7fcf200b43a0 2023-11-08T08:19:32.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.103+0000 7fcf25a9f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf2006b0a0 con 0x7fcf200c9040 2023-11-08T08:19:32.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.103+0000 7fcf25a9f700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf200caac0 con 0x7fcf200b1640 2023-11-08T08:19:32.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.103+0000 7fcf2529e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcf200c9040 0x7fcf200705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:32.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf2529e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcf200c9040 0x7fcf200705c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52178/0 (socket says 172.21.15.40:52178) 2023-11-08T08:19:32.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf2529e700 1 -- 172.21.15.40:0/1019921312 learned_addr learned my addr 172.21.15.40:0/1019921312 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:32.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf1ffff700 1 --2- 172.21.15.40:0/1019921312 >> v2:172.21.15.169:3300/0 conn(0x7fcf200b1640 0x7fcf20070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:32.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf1f7fe700 1 -- 172.21.15.40:0/1019921312 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001983154 0 0) 0x7fcf14001490 con 0x7fcf200b43a0 2023-11-08T08:19:32.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf1f7fe700 1 -- 172.21.15.40:0/1019921312 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf100035d0 con 0x7fcf200b43a0 2023-11-08T08:19:32.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf2529e700 1 -- 172.21.15.40:0/1019921312 >> v2:172.21.15.169:3300/0 conn(0x7fcf200b1640 msgr2=0x7fcf20070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:32.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf2529e700 1 --2- 172.21.15.40:0/1019921312 >> v2:172.21.15.169:3300/0 conn(0x7fcf200b1640 0x7fcf20070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:32.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf2529e700 1 -- 172.21.15.40:0/1019921312 >> v1:172.21.15.169:6789/0 conn(0x7fcf200b43a0 legacy=0x7fcf2015e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:32.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf2529e700 1 -- 172.21.15.40:0/1019921312 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf200cada0 con 0x7fcf200c9040 2023-11-08T08:19:32.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf1ffff700 1 --2- 172.21.15.40:0/1019921312 >> v2:172.21.15.169:3300/0 conn(0x7fcf200b1640 0x7fcf20070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:32.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf2529e700 1 --2- 172.21.15.40:0/1019921312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf200c9040 0x7fcf200705c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcf20072d40 tx=0x7fcf18009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ce3c69de8906f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:32.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf1f7fe700 1 -- 172.21.15.40:0/1019921312 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf18011830 con 0x7fcf200c9040 2023-11-08T08:19:32.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf1f7fe700 1 -- 172.21.15.40:0/1019921312 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf18011990 con 0x7fcf200c9040 2023-11-08T08:19:32.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf1f7fe700 1 -- 172.21.15.40:0/1019921312 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf18011d20 con 0x7fcf200c9040 2023-11-08T08:19:32.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf25a9f700 1 -- 172.21.15.40:0/1019921312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf200c9040 msgr2=0x7fcf200705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:32.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf25a9f700 1 --2- 172.21.15.40:0/1019921312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf200c9040 0x7fcf200705c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcf20072d40 tx=0x7fcf18009bf0 comp rx=0 tx=0).stop 2023-11-08T08:19:32.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf25a9f700 1 -- 172.21.15.40:0/1019921312 shutdown_connections 2023-11-08T08:19:32.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf25a9f700 1 --2- 172.21.15.40:0/1019921312 >> v2:172.21.15.169:3300/0 conn(0x7fcf200b1640 0x7fcf20070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:32.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf25a9f700 1 --2- 172.21.15.40:0/1019921312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf200c9040 0x7fcf200705c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:32.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf25a9f700 1 -- 172.21.15.40:0/1019921312 >> 172.21.15.40:0/1019921312 conn(0x7fcf200b0f80 msgr2=0x7fcf200ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:32.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf25a9f700 1 -- 172.21.15.40:0/1019921312 shutdown_connections 2023-11-08T08:19:32.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.107+0000 7fcf25a9f700 1 -- 172.21.15.40:0/1019921312 wait complete. 2023-11-08T08:19:32.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.111+0000 7fcf25a9f700 1 Processor -- start 2023-11-08T08:19:32.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.111+0000 7fcf25a9f700 1 -- start start 2023-11-08T08:19:32.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.111+0000 7fcf25a9f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf200b1640 0x7fcf200c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:32.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.111+0000 7fcf25a9f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf200b43a0 0x7fcf2015ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:32.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.111+0000 7fcf25a9f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf200c9040 0x7fcf2015ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:32.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.111+0000 7fcf25a9f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf200c50d0 con 0x7fcf200b1640 2023-11-08T08:19:32.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.111+0000 7fcf25a9f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf200b3680 con 0x7fcf200c9040 2023-11-08T08:19:32.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.111+0000 7fcf25a9f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf200b2a70 con 0x7fcf200b43a0 2023-11-08T08:19:32.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.115+0000 7fcf24a9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf200b1640 0x7fcf200c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:32.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.115+0000 7fcf24a9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf200b1640 0x7fcf200c4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52184/0 (socket says 172.21.15.40:52184) 2023-11-08T08:19:32.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.115+0000 7fcf24a9d700 1 -- 172.21.15.40:0/2792856263 learned_addr learned my addr 172.21.15.40:0/2792856263 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:32.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.115+0000 7fcf2529e700 1 --2- 172.21.15.40:0/2792856263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf200c9040 0x7fcf2015ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:32.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.115+0000 7fcf1ffff700 1 --2- 172.21.15.40:0/2792856263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf200b43a0 0x7fcf2015ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:32.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.115+0000 7fcf24a9d700 1 -- 172.21.15.40:0/2792856263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf200b43a0 msgr2=0x7fcf2015ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:32.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.115+0000 7fcf24a9d700 1 --2- 172.21.15.40:0/2792856263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf200b43a0 0x7fcf2015ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:32.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.115+0000 7fcf24a9d700 1 -- 172.21.15.40:0/2792856263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf200c9040 msgr2=0x7fcf2015ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:32.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.115+0000 7fcf24a9d700 1 --2- 172.21.15.40:0/2792856263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf200c9040 0x7fcf2015ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:32.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.115+0000 7fcf24a9d700 1 -- 172.21.15.40:0/2792856263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf18009570 con 0x7fcf200b1640 2023-11-08T08:19:32.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.115+0000 7fcf2529e700 1 --2- 172.21.15.40:0/2792856263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf200c9040 0x7fcf2015ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:32.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.115+0000 7fcf24a9d700 1 --2- 172.21.15.40:0/2792856263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf200b1640 0x7fcf200c4b90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400cc40 tx=0x7fcf1400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:32.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.115+0000 7fcf1d7fa700 1 -- 172.21.15.40:0/2792856263 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf14003c20 con 0x7fcf200b1640 2023-11-08T08:19:32.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.115+0000 7fcf1d7fa700 1 -- 172.21.15.40:0/2792856263 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf14003da0 con 0x7fcf200b1640 2023-11-08T08:19:32.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.115+0000 7fcf25a9f700 1 -- 172.21.15.40:0/2792856263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2006b5d0 con 0x7fcf200b1640 2023-11-08T08:19:32.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.115+0000 7fcf1d7fa700 1 -- 172.21.15.40:0/2792856263 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1400ac20 con 0x7fcf200b1640 2023-11-08T08:19:32.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.115+0000 7fcf25a9f700 1 -- 172.21.15.40:0/2792856263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2006b2f0 con 0x7fcf200b1640 2023-11-08T08:19:32.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.119+0000 7fcf25a9f700 1 -- 172.21.15.40:0/2792856263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf200ca7b0 con 0x7fcf200b1640 2023-11-08T08:19:32.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.119+0000 7fcf1d7fa700 1 -- 172.21.15.40:0/2792856263 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fcf14010050 con 0x7fcf200b1640 2023-11-08T08:19:32.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.119+0000 7fcf1d7fa700 1 --2- 172.21.15.40:0/2792856263 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcf04064ee0 0x7fcf04067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:32.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.119+0000 7fcf1d7fa700 1 -- 172.21.15.40:0/2792856263 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fcf140820a0 con 0x7fcf200b1640 2023-11-08T08:19:32.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.123+0000 7fcf1ffff700 1 --2- 172.21.15.40:0/2792856263 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcf04064ee0 0x7fcf04067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:32.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.123+0000 7fcf1ffff700 1 --2- 172.21.15.40:0/2792856263 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcf04064ee0 0x7fcf04067390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcf200aba80 tx=0x7fcf0c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:32.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.127+0000 7fcf1d7fa700 1 -- 172.21.15.40:0/2792856263 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fcf1404fe60 con 0x7fcf200b1640 2023-11-08T08:19:32.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.299+0000 7fcf25a9f700 1 -- 172.21.15.40:0/2792856263 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf200b37c0 con 0x7fcf04064ee0 2023-11-08T08:19:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.323+0000 7fcf1d7fa700 1 -- 172.21.15.40:0/2792856263 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+366970 (secure 0 0 0) 0x7fcf0c00c5b0 con 0x7fcf04064ee0 2023-11-08T08:19:32.334 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.331+0000 7fcf25a9f700 1 -- 172.21.15.40:0/2792856263 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcf04064ee0 msgr2=0x7fcf04067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.331+0000 7fcf25a9f700 1 --2- 172.21.15.40:0/2792856263 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcf04064ee0 0x7fcf04067390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcf200aba80 tx=0x7fcf0c008040 comp rx=0 tx=0).stop 2023-11-08T08:19:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.331+0000 7fcf25a9f700 1 -- 172.21.15.40:0/2792856263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf200b1640 msgr2=0x7fcf200c4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:32.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.331+0000 7fcf25a9f700 1 --2- 172.21.15.40:0/2792856263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf200b1640 0x7fcf200c4b90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400cc40 tx=0x7fcf1400a060 comp rx=0 tx=0).stop 2023-11-08T08:19:32.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.331+0000 7fcf25a9f700 1 -- 172.21.15.40:0/2792856263 shutdown_connections 2023-11-08T08:19:32.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.331+0000 7fcf25a9f700 1 --2- 172.21.15.40:0/2792856263 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcf04064ee0 0x7fcf04067390 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:32.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.331+0000 7fcf25a9f700 1 --2- 172.21.15.40:0/2792856263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf200c9040 0x7fcf2015ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:32.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.331+0000 7fcf25a9f700 1 --2- 172.21.15.40:0/2792856263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf200b43a0 0x7fcf2015ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:32.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.331+0000 7fcf25a9f700 1 --2- 172.21.15.40:0/2792856263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf200b1640 0x7fcf200c4b90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:32.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.331+0000 7fcf25a9f700 1 -- 172.21.15.40:0/2792856263 >> 172.21.15.40:0/2792856263 conn(0x7fcf200b0f80 msgr2=0x7fcf200ce580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:32.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.331+0000 7fcf25a9f700 1 -- 172.21.15.40:0/2792856263 shutdown_connections 2023-11-08T08:19:32.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.331+0000 7fcf25a9f700 1 -- 172.21.15.40:0/2792856263 wait complete. 2023-11-08T08:19:32.339 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-11-08T08:19:32.357 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2023-11-08T08:19:31.889739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594895,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67784,"kb_used_data":3592,"kb_used_omap":0,"kb_used_meta":64128,"kb_avail":754906936,"statfs":{"total":773094113280,"available":773024702464,"internally_reserved":0,"allocated":3678208,"data_stored":2186366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65667072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004841"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.370829+0000","last_change":"2023-11-08T08:19:08.519864+0000","last_active":"2023-11-08T08:19:09.370829+0000","last_peered":"2023-11-08T08:19:09.370829+0000","last_clean":"2023-11-08T08:19:09.370829+0000","last_became_active":"2023-11-08T08:19:08.518845+0000","last_became_peered":"2023-11-08T08:19:08.518845+0000","last_unstale":"2023-11-08T08:19:09.370829+0000","last_undegraded":"2023-11-08T08:19:09.370829+0000","last_fullsized":"2023-11-08T08:19:09.370829+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:47:58.348898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.496570+0000","last_change":"2023-11-08T08:19:04.015635+0000","last_active":"2023-11-08T08:19:08.496570+0000","last_peered":"2023-11-08T08:19:08.496570+0000","last_clean":"2023-11-08T08:19:08.496570+0000","last_became_active":"2023-11-08T08:19:04.014748+0000","last_became_peered":"2023-11-08T08:19:04.014748+0000","last_unstale":"2023-11-08T08:19:08.496570+0000","last_undegraded":"2023-11-08T08:19:08.496570+0000","last_fullsized":"2023-11-08T08:19:08.496570+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:01:41.280977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513402+0000","last_change":"2023-11-08T08:19:04.505994+0000","last_active":"2023-11-08T08:19:08.513402+0000","last_peered":"2023-11-08T08:19:08.513402+0000","last_clean":"2023-11-08T08:19:08.513402+0000","last_became_active":"2023-11-08T08:19:04.505596+0000","last_became_peered":"2023-11-08T08:19:04.505596+0000","last_unstale":"2023-11-08T08:19:08.513402+0000","last_undegraded":"2023-11-08T08:19:08.513402+0000","last_fullsized":"2023-11-08T08:19:08.513402+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:06:47.395577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.992842+0000","last_change":"2023-11-08T08:19:08.526866+0000","last_active":"2023-11-08T08:19:08.992842+0000","last_peered":"2023-11-08T08:19:08.992842+0000","last_clean":"2023-11-08T08:19:08.992842+0000","last_became_active":"2023-11-08T08:19:08.526405+0000","last_became_peered":"2023-11-08T08:19:08.526405+0000","last_unstale":"2023-11-08T08:19:08.992842+0000","last_undegraded":"2023-11-08T08:19:08.992842+0000","last_fullsized":"2023-11-08T08:19:08.992842+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:54:12.699998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519574+0000","last_change":"2023-11-08T08:19:04.021634+0000","last_active":"2023-11-08T08:19:08.519574+0000","last_peered":"2023-11-08T08:19:08.519574+0000","last_clean":"2023-11-08T08:19:08.519574+0000","last_became_active":"2023-11-08T08:19:04.017916+0000","last_became_peered":"2023-11-08T08:19:04.017916+0000","last_unstale":"2023-11-08T08:19:08.519574+0000","last_undegraded":"2023-11-08T08:19:08.519574+0000","last_fullsized":"2023-11-08T08:19:08.519574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:58:24.706350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518078+0000","last_change":"2023-11-08T08:19:04.481276+0000","last_active":"2023-11-08T08:19:08.518078+0000","last_peered":"2023-11-08T08:19:08.518078+0000","last_clean":"2023-11-08T08:19:08.518078+0000","last_became_active":"2023-11-08T08:19:04.480340+0000","last_became_peered":"2023-11-08T08:19:04.480340+0000","last_unstale":"2023-11-08T08:19:08.518078+0000","last_undegraded":"2023-11-08T08:19:08.518078+0000","last_fullsized":"2023-11-08T08:19:08.518078+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:04:55.302575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372099+0000","last_change":"2023-11-08T08:19:08.528726+0000","last_active":"2023-11-08T08:19:09.372099+0000","last_peered":"2023-11-08T08:19:09.372099+0000","last_clean":"2023-11-08T08:19:09.372099+0000","last_became_active":"2023-11-08T08:19:08.527853+0000","last_became_peered":"2023-11-08T08:19:08.527853+0000","last_unstale":"2023-11-08T08:19:09.372099+0000","last_undegraded":"2023-11-08T08:19:09.372099+0000","last_fullsized":"2023-11-08T08:19:09.372099+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:16:14.032597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519819+0000","last_change":"2023-11-08T08:19:04.025513+0000","last_active":"2023-11-08T08:19:08.519819+0000","last_peered":"2023-11-08T08:19:08.519819+0000","last_clean":"2023-11-08T08:19:08.519819+0000","last_became_active":"2023-11-08T08:19:04.025220+0000","last_became_peered":"2023-11-08T08:19:04.025220+0000","last_unstale":"2023-11-08T08:19:08.519819+0000","last_undegraded":"2023-11-08T08:19:08.519819+0000","last_fullsized":"2023-11-08T08:19:08.519819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:55:07.041721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512027+0000","last_change":"2023-11-08T08:19:04.512364+0000","last_active":"2023-11-08T08:19:08.512027+0000","last_peered":"2023-11-08T08:19:08.512027+0000","last_clean":"2023-11-08T08:19:08.512027+0000","last_became_active":"2023-11-08T08:19:04.511958+0000","last_became_peered":"2023-11-08T08:19:04.511958+0000","last_unstale":"2023-11-08T08:19:08.512027+0000","last_undegraded":"2023-11-08T08:19:08.512027+0000","last_fullsized":"2023-11-08T08:19:08.512027+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:32:57.245760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.994907+0000","last_change":"2023-11-08T08:19:08.527583+0000","last_active":"2023-11-08T08:19:08.994907+0000","last_peered":"2023-11-08T08:19:08.994907+0000","last_clean":"2023-11-08T08:19:08.994907+0000","last_became_active":"2023-11-08T08:19:08.527054+0000","last_became_peered":"2023-11-08T08:19:08.527054+0000","last_unstale":"2023-11-08T08:19:08.994907+0000","last_undegraded":"2023-11-08T08:19:08.994907+0000","last_fullsized":"2023-11-08T08:19:08.994907+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:08.797102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519426+0000","last_change":"2023-11-08T08:19:04.025416+0000","last_active":"2023-11-08T08:19:08.519426+0000","last_peered":"2023-11-08T08:19:08.519426+0000","last_clean":"2023-11-08T08:19:08.519426+0000","last_became_active":"2023-11-08T08:19:04.025091+0000","last_became_peered":"2023-11-08T08:19:04.025091+0000","last_unstale":"2023-11-08T08:19:08.519426+0000","last_undegraded":"2023-11-08T08:19:08.519426+0000","last_fullsized":"2023-11-08T08:19:08.519426+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:07:35.132845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.511963+0000","last_change":"2023-11-08T08:19:04.512794+0000","last_active":"2023-11-08T08:19:08.511963+0000","last_peered":"2023-11-08T08:19:08.511963+0000","last_clean":"2023-11-08T08:19:08.511963+0000","last_became_active":"2023-11-08T08:19:04.512499+0000","last_became_peered":"2023-11-08T08:19:04.512499+0000","last_unstale":"2023-11-08T08:19:08.511963+0000","last_undegraded":"2023-11-08T08:19:08.511963+0000","last_fullsized":"2023-11-08T08:19:08.511963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:05:16.516877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372287+0000","last_change":"2023-11-08T08:19:08.539289+0000","last_active":"2023-11-08T08:19:09.372287+0000","last_peered":"2023-11-08T08:19:09.372287+0000","last_clean":"2023-11-08T08:19:09.372287+0000","last_became_active":"2023-11-08T08:19:08.532011+0000","last_became_peered":"2023-11-08T08:19:08.532011+0000","last_unstale":"2023-11-08T08:19:09.372287+0000","last_undegraded":"2023-11-08T08:19:09.372287+0000","last_fullsized":"2023-11-08T08:19:09.372287+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:56:38.412909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.495705+0000","last_change":"2023-11-08T08:19:04.024246+0000","last_active":"2023-11-08T08:19:08.495705+0000","last_peered":"2023-11-08T08:19:08.495705+0000","last_clean":"2023-11-08T08:19:08.495705+0000","last_became_active":"2023-11-08T08:19:04.023681+0000","last_became_peered":"2023-11-08T08:19:04.023681+0000","last_unstale":"2023-11-08T08:19:08.495705+0000","last_undegraded":"2023-11-08T08:19:08.495705+0000","last_fullsized":"2023-11-08T08:19:08.495705+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:50:41.342112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.509382+0000","last_change":"2023-11-08T08:19:04.494535+0000","last_active":"2023-11-08T08:19:08.509382+0000","last_peered":"2023-11-08T08:19:08.509382+0000","last_clean":"2023-11-08T08:19:08.509382+0000","last_became_active":"2023-11-08T08:19:04.494226+0000","last_became_peered":"2023-11-08T08:19:04.494226+0000","last_unstale":"2023-11-08T08:19:08.509382+0000","last_undegraded":"2023-11-08T08:19:08.509382+0000","last_fullsized":"2023-11-08T08:19:08.509382+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:18:15.282329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.993497+0000","last_change":"2023-11-08T08:19:08.532723+0000","last_active":"2023-11-08T08:19:08.993497+0000","last_peered":"2023-11-08T08:19:08.993497+0000","last_clean":"2023-11-08T08:19:08.993497+0000","last_became_active":"2023-11-08T08:19:08.532320+0000","last_became_peered":"2023-11-08T08:19:08.532320+0000","last_unstale":"2023-11-08T08:19:08.993497+0000","last_undegraded":"2023-11-08T08:19:08.993497+0000","last_fullsized":"2023-11-08T08:19:08.993497+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:10:25.517705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512234+0000","last_change":"2023-11-08T08:19:04.020285+0000","last_active":"2023-11-08T08:19:08.512234+0000","last_peered":"2023-11-08T08:19:08.512234+0000","last_clean":"2023-11-08T08:19:08.512234+0000","last_became_active":"2023-11-08T08:19:04.019791+0000","last_became_peered":"2023-11-08T08:19:04.019791+0000","last_unstale":"2023-11-08T08:19:08.512234+0000","last_undegraded":"2023-11-08T08:19:08.512234+0000","last_fullsized":"2023-11-08T08:19:08.512234+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:21:09.218618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521337+0000","last_change":"2023-11-08T08:19:04.497059+0000","last_active":"2023-11-08T08:19:08.521337+0000","last_peered":"2023-11-08T08:19:08.521337+0000","last_clean":"2023-11-08T08:19:08.521337+0000","last_became_active":"2023-11-08T08:19:04.495199+0000","last_became_peered":"2023-11-08T08:19:04.495199+0000","last_unstale":"2023-11-08T08:19:08.521337+0000","last_undegraded":"2023-11-08T08:19:08.521337+0000","last_fullsized":"2023-11-08T08:19:08.521337+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:56:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.374791+0000","last_change":"2023-11-08T08:19:08.530445+0000","last_active":"2023-11-08T08:19:09.374791+0000","last_peered":"2023-11-08T08:19:09.374791+0000","last_clean":"2023-11-08T08:19:09.374791+0000","last_became_active":"2023-11-08T08:19:08.530020+0000","last_became_peered":"2023-11-08T08:19:08.530020+0000","last_unstale":"2023-11-08T08:19:09.374791+0000","last_undegraded":"2023-11-08T08:19:09.374791+0000","last_fullsized":"2023-11-08T08:19:09.374791+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:24:13.984326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.502373+0000","last_change":"2023-11-08T08:19:04.018882+0000","last_active":"2023-11-08T08:19:08.502373+0000","last_peered":"2023-11-08T08:19:08.502373+0000","last_clean":"2023-11-08T08:19:08.502373+0000","last_became_active":"2023-11-08T08:19:04.018537+0000","last_became_peered":"2023-11-08T08:19:04.018537+0000","last_unstale":"2023-11-08T08:19:08.502373+0000","last_undegraded":"2023-11-08T08:19:08.502373+0000","last_fullsized":"2023-11-08T08:19:08.502373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:51:22.228427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.496833+0000","last_change":"2023-11-08T08:19:04.503358+0000","last_active":"2023-11-08T08:19:08.496833+0000","last_peered":"2023-11-08T08:19:08.496833+0000","last_clean":"2023-11-08T08:19:08.496833+0000","last_became_active":"2023-11-08T08:19:04.502856+0000","last_became_peered":"2023-11-08T08:19:04.502856+0000","last_unstale":"2023-11-08T08:19:08.496833+0000","last_undegraded":"2023-11-08T08:19:08.496833+0000","last_fullsized":"2023-11-08T08:19:08.496833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.989197+0000","last_change":"2023-11-08T08:19:08.531064+0000","last_active":"2023-11-08T08:19:08.989197+0000","last_peered":"2023-11-08T08:19:08.989197+0000","last_clean":"2023-11-08T08:19:08.989197+0000","last_became_active":"2023-11-08T08:19:08.530517+0000","last_became_peered":"2023-11-08T08:19:08.530517+0000","last_unstale":"2023-11-08T08:19:08.989197+0000","last_undegraded":"2023-11-08T08:19:08.989197+0000","last_fullsized":"2023-11-08T08:19:08.989197+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:51:11.743430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517381+0000","last_change":"2023-11-08T08:19:04.022640+0000","last_active":"2023-11-08T08:19:08.517381+0000","last_peered":"2023-11-08T08:19:08.517381+0000","last_clean":"2023-11-08T08:19:08.517381+0000","last_became_active":"2023-11-08T08:19:04.022244+0000","last_became_peered":"2023-11-08T08:19:04.022244+0000","last_unstale":"2023-11-08T08:19:08.517381+0000","last_undegraded":"2023-11-08T08:19:08.517381+0000","last_fullsized":"2023-11-08T08:19:08.517381+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:50:41.342112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521762+0000","last_change":"2023-11-08T08:19:04.497516+0000","last_active":"2023-11-08T08:19:08.521762+0000","last_peered":"2023-11-08T08:19:08.521762+0000","last_clean":"2023-11-08T08:19:08.521762+0000","last_became_active":"2023-11-08T08:19:04.493572+0000","last_became_peered":"2023-11-08T08:19:04.493572+0000","last_unstale":"2023-11-08T08:19:08.521762+0000","last_undegraded":"2023-11-08T08:19:08.521762+0000","last_fullsized":"2023-11-08T08:19:08.521762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:57:06.487072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517281+0000","last_change":"2023-11-08T08:19:04.495610+0000","last_active":"2023-11-08T08:19:08.517281+0000","last_peered":"2023-11-08T08:19:08.517281+0000","last_clean":"2023-11-08T08:19:08.517281+0000","last_became_active":"2023-11-08T08:19:04.495307+0000","last_became_peered":"2023-11-08T08:19:04.495307+0000","last_unstale":"2023-11-08T08:19:08.517281+0000","last_undegraded":"2023-11-08T08:19:08.517281+0000","last_fullsized":"2023-11-08T08:19:08.517281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.496569+0000","last_change":"2023-11-08T08:19:04.015753+0000","last_active":"2023-11-08T08:19:08.496569+0000","last_peered":"2023-11-08T08:19:08.496569+0000","last_clean":"2023-11-08T08:19:08.496569+0000","last_became_active":"2023-11-08T08:19:04.015173+0000","last_became_peered":"2023-11-08T08:19:04.015173+0000","last_unstale":"2023-11-08T08:19:08.496569+0000","last_undegraded":"2023-11-08T08:19:08.496569+0000","last_fullsized":"2023-11-08T08:19:08.496569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:58:24.706350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.373895+0000","last_change":"2023-11-08T08:19:08.525297+0000","last_active":"2023-11-08T08:19:09.373895+0000","last_peered":"2023-11-08T08:19:09.373895+0000","last_clean":"2023-11-08T08:19:09.373895+0000","last_became_active":"2023-11-08T08:19:08.524571+0000","last_became_peered":"2023-11-08T08:19:08.524571+0000","last_unstale":"2023-11-08T08:19:09.373895+0000","last_undegraded":"2023-11-08T08:19:09.373895+0000","last_fullsized":"2023-11-08T08:19:09.373895+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:33:00.754113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517964+0000","last_change":"2023-11-08T08:19:04.500363+0000","last_active":"2023-11-08T08:19:08.517964+0000","last_peered":"2023-11-08T08:19:08.517964+0000","last_clean":"2023-11-08T08:19:08.517964+0000","last_became_active":"2023-11-08T08:19:04.500049+0000","last_became_peered":"2023-11-08T08:19:04.500049+0000","last_unstale":"2023-11-08T08:19:08.517964+0000","last_undegraded":"2023-11-08T08:19:08.517964+0000","last_fullsized":"2023-11-08T08:19:08.517964+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:06:47.395577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521546+0000","last_change":"2023-11-08T08:19:04.020555+0000","last_active":"2023-11-08T08:19:08.521546+0000","last_peered":"2023-11-08T08:19:08.521546+0000","last_clean":"2023-11-08T08:19:08.521546+0000","last_became_active":"2023-11-08T08:19:04.020053+0000","last_became_peered":"2023-11-08T08:19:04.020053+0000","last_unstale":"2023-11-08T08:19:08.521546+0000","last_undegraded":"2023-11-08T08:19:08.521546+0000","last_fullsized":"2023-11-08T08:19:08.521546+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:01:41.280977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.994150+0000","last_change":"2023-11-08T08:19:08.526638+0000","last_active":"2023-11-08T08:19:08.994150+0000","last_peered":"2023-11-08T08:19:08.994150+0000","last_clean":"2023-11-08T08:19:08.994150+0000","last_became_active":"2023-11-08T08:19:08.526219+0000","last_became_peered":"2023-11-08T08:19:08.526219+0000","last_unstale":"2023-11-08T08:19:08.994150+0000","last_undegraded":"2023-11-08T08:19:08.994150+0000","last_fullsized":"2023-11-08T08:19:08.994150+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:56:38.412909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.500591+0000","last_change":"2023-11-08T08:19:04.022271+0000","last_active":"2023-11-08T08:19:08.500591+0000","last_peered":"2023-11-08T08:19:08.500591+0000","last_clean":"2023-11-08T08:19:08.500591+0000","last_became_active":"2023-11-08T08:19:04.021306+0000","last_became_peered":"2023-11-08T08:19:04.021306+0000","last_unstale":"2023-11-08T08:19:08.500591+0000","last_undegraded":"2023-11-08T08:19:08.500591+0000","last_fullsized":"2023-11-08T08:19:08.500591+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:26:26.874402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.502798+0000","last_change":"2023-11-08T08:19:04.494816+0000","last_active":"2023-11-08T08:19:08.502798+0000","last_peered":"2023-11-08T08:19:08.502798+0000","last_clean":"2023-11-08T08:19:08.502798+0000","last_became_active":"2023-11-08T08:19:04.494321+0000","last_became_peered":"2023-11-08T08:19:04.494321+0000","last_unstale":"2023-11-08T08:19:08.502798+0000","last_undegraded":"2023-11-08T08:19:08.502798+0000","last_fullsized":"2023-11-08T08:19:08.502798+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:04:55.302575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372527+0000","last_change":"2023-11-08T08:19:08.542779+0000","last_active":"2023-11-08T08:19:09.372527+0000","last_peered":"2023-11-08T08:19:09.372527+0000","last_clean":"2023-11-08T08:19:09.372527+0000","last_became_active":"2023-11-08T08:19:08.542399+0000","last_became_peered":"2023-11-08T08:19:08.542399+0000","last_unstale":"2023-11-08T08:19:09.372527+0000","last_undegraded":"2023-11-08T08:19:09.372527+0000","last_fullsized":"2023-11-08T08:19:09.372527+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:16:14.032597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:21.997708+0000","last_change":"2023-11-08T08:19:04.018152+0000","last_active":"2023-11-08T08:19:21.997708+0000","last_peered":"2023-11-08T08:19:21.997708+0000","last_clean":"2023-11-08T08:19:21.997708+0000","last_became_active":"2023-11-08T08:19:04.017695+0000","last_became_peered":"2023-11-08T08:19:04.017695+0000","last_unstale":"2023-11-08T08:19:21.997708+0000","last_undegraded":"2023-11-08T08:19:21.997708+0000","last_fullsized":"2023-11-08T08:19:21.997708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:55:20.782396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3745,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512046+0000","last_change":"2023-11-08T08:19:04.501576+0000","last_active":"2023-11-08T08:19:08.512046+0000","last_peered":"2023-11-08T08:19:08.512046+0000","last_clean":"2023-11-08T08:19:08.512046+0000","last_became_active":"2023-11-08T08:19:04.500829+0000","last_became_peered":"2023-11-08T08:19:04.500829+0000","last_unstale":"2023-11-08T08:19:08.512046+0000","last_undegraded":"2023-11-08T08:19:08.512046+0000","last_fullsized":"2023-11-08T08:19:08.512046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372324+0000","last_change":"2023-11-08T08:19:08.536704+0000","last_active":"2023-11-08T08:19:09.372324+0000","last_peered":"2023-11-08T08:19:09.372324+0000","last_clean":"2023-11-08T08:19:09.372324+0000","last_became_active":"2023-11-08T08:19:08.536339+0000","last_became_peered":"2023-11-08T08:19:08.536339+0000","last_unstale":"2023-11-08T08:19:09.372324+0000","last_undegraded":"2023-11-08T08:19:09.372324+0000","last_fullsized":"2023-11-08T08:19:09.372324+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:51:11.743430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513610+0000","last_change":"2023-11-08T08:19:04.496877+0000","last_active":"2023-11-08T08:19:08.513610+0000","last_peered":"2023-11-08T08:19:08.513610+0000","last_clean":"2023-11-08T08:19:08.513610+0000","last_became_active":"2023-11-08T08:19:04.496505+0000","last_became_peered":"2023-11-08T08:19:04.496505+0000","last_unstale":"2023-11-08T08:19:08.513610+0000","last_undegraded":"2023-11-08T08:19:08.513610+0000","last_fullsized":"2023-11-08T08:19:08.513610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:59:35.582106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521817+0000","last_change":"2023-11-08T08:18:58.771310+0000","last_active":"2023-11-08T08:19:08.521817+0000","last_peered":"2023-11-08T08:19:08.521817+0000","last_clean":"2023-11-08T08:19:08.521817+0000","last_became_active":"2023-11-08T08:18:58.770891+0000","last_became_peered":"2023-11-08T08:18:58.770891+0000","last_unstale":"2023-11-08T08:19:08.521817+0000","last_undegraded":"2023-11-08T08:19:08.521817+0000","last_fullsized":"2023-11-08T08:19:08.521817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"27'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.969187+0000","last_change":"2023-11-08T08:19:04.012926+0000","last_active":"2023-11-08T08:19:08.969187+0000","last_peered":"2023-11-08T08:19:08.969187+0000","last_clean":"2023-11-08T08:19:08.969187+0000","last_became_active":"2023-11-08T08:19:04.012634+0000","last_became_peered":"2023-11-08T08:19:04.012634+0000","last_unstale":"2023-11-08T08:19:08.969187+0000","last_undegraded":"2023-11-08T08:19:08.969187+0000","last_fullsized":"2023-11-08T08:19:08.969187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:07:35.132845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.371641+0000","last_change":"2023-11-08T08:19:08.532617+0000","last_active":"2023-11-08T08:19:09.371641+0000","last_peered":"2023-11-08T08:19:09.371641+0000","last_clean":"2023-11-08T08:19:09.371641+0000","last_became_active":"2023-11-08T08:19:08.532177+0000","last_became_peered":"2023-11-08T08:19:08.532177+0000","last_unstale":"2023-11-08T08:19:09.371641+0000","last_undegraded":"2023-11-08T08:19:09.371641+0000","last_fullsized":"2023-11-08T08:19:09.371641+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:08.797102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517321+0000","last_change":"2023-11-08T08:19:04.504132+0000","last_active":"2023-11-08T08:19:08.517321+0000","last_peered":"2023-11-08T08:19:08.517321+0000","last_clean":"2023-11-08T08:19:08.517321+0000","last_became_active":"2023-11-08T08:19:04.503747+0000","last_became_peered":"2023-11-08T08:19:04.503747+0000","last_unstale":"2023-11-08T08:19:08.517321+0000","last_undegraded":"2023-11-08T08:19:08.517321+0000","last_fullsized":"2023-11-08T08:19:08.517321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:27:40.872420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503547+0000","last_change":"2023-11-08T08:18:58.770483+0000","last_active":"2023-11-08T08:19:08.503547+0000","last_peered":"2023-11-08T08:19:08.503547+0000","last_clean":"2023-11-08T08:19:08.503547+0000","last_became_active":"2023-11-08T08:18:58.770040+0000","last_became_peered":"2023-11-08T08:18:58.770040+0000","last_unstale":"2023-11-08T08:19:08.503547+0000","last_undegraded":"2023-11-08T08:19:08.503547+0000","last_fullsized":"2023-11-08T08:19:08.503547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.520152+0000","last_change":"2023-11-08T08:19:04.022667+0000","last_active":"2023-11-08T08:19:08.520152+0000","last_peered":"2023-11-08T08:19:08.520152+0000","last_clean":"2023-11-08T08:19:08.520152+0000","last_became_active":"2023-11-08T08:19:04.022430+0000","last_became_peered":"2023-11-08T08:19:04.022430+0000","last_unstale":"2023-11-08T08:19:08.520152+0000","last_undegraded":"2023-11-08T08:19:08.520152+0000","last_fullsized":"2023-11-08T08:19:08.520152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:15:32.560862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.992931+0000","last_change":"2023-11-08T08:19:08.531983+0000","last_active":"2023-11-08T08:19:08.992931+0000","last_peered":"2023-11-08T08:19:08.992931+0000","last_clean":"2023-11-08T08:19:08.992931+0000","last_became_active":"2023-11-08T08:19:08.531584+0000","last_became_peered":"2023-11-08T08:19:08.531584+0000","last_unstale":"2023-11-08T08:19:08.992931+0000","last_undegraded":"2023-11-08T08:19:08.992931+0000","last_fullsized":"2023-11-08T08:19:08.992931+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:56:34.397672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.495387+0000","last_change":"2023-11-08T08:19:04.481283+0000","last_active":"2023-11-08T08:19:08.495387+0000","last_peered":"2023-11-08T08:19:08.495387+0000","last_clean":"2023-11-08T08:19:08.495387+0000","last_became_active":"2023-11-08T08:19:04.480992+0000","last_became_peered":"2023-11-08T08:19:04.480992+0000","last_unstale":"2023-11-08T08:19:08.495387+0000","last_undegraded":"2023-11-08T08:19:08.495387+0000","last_fullsized":"2023-11-08T08:19:08.495387+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.500946+0000","last_change":"2023-11-08T08:18:58.766848+0000","last_active":"2023-11-08T08:19:08.500946+0000","last_peered":"2023-11-08T08:19:08.500946+0000","last_clean":"2023-11-08T08:19:08.500946+0000","last_became_active":"2023-11-08T08:18:58.766378+0000","last_became_peered":"2023-11-08T08:18:58.766378+0000","last_unstale":"2023-11-08T08:19:08.500946+0000","last_undegraded":"2023-11-08T08:19:08.500946+0000","last_fullsized":"2023-11-08T08:19:08.500946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503466+0000","last_change":"2023-11-08T08:19:04.018717+0000","last_active":"2023-11-08T08:19:08.503466+0000","last_peered":"2023-11-08T08:19:08.503466+0000","last_clean":"2023-11-08T08:19:08.503466+0000","last_became_active":"2023-11-08T08:19:04.018355+0000","last_became_peered":"2023-11-08T08:19:04.018355+0000","last_unstale":"2023-11-08T08:19:08.503466+0000","last_undegraded":"2023-11-08T08:19:08.503466+0000","last_fullsized":"2023-11-08T08:19:08.503466+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:21:09.218618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.374050+0000","last_change":"2023-11-08T08:19:08.539068+0000","last_active":"2023-11-08T08:19:09.374050+0000","last_peered":"2023-11-08T08:19:09.374050+0000","last_clean":"2023-11-08T08:19:09.374050+0000","last_became_active":"2023-11-08T08:19:08.538763+0000","last_became_peered":"2023-11-08T08:19:08.538763+0000","last_unstale":"2023-11-08T08:19:09.374050+0000","last_undegraded":"2023-11-08T08:19:09.374050+0000","last_fullsized":"2023-11-08T08:19:09.374050+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:27:44.380774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.502722+0000","last_change":"2023-11-08T08:19:04.508515+0000","last_active":"2023-11-08T08:19:08.502722+0000","last_peered":"2023-11-08T08:19:08.502722+0000","last_clean":"2023-11-08T08:19:08.502722+0000","last_became_active":"2023-11-08T08:19:04.508227+0000","last_became_peered":"2023-11-08T08:19:04.508227+0000","last_unstale":"2023-11-08T08:19:08.502722+0000","last_undegraded":"2023-11-08T08:19:08.502722+0000","last_fullsized":"2023-11-08T08:19:08.502722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:48:06.180083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503646+0000","last_change":"2023-11-08T08:18:58.769764+0000","last_active":"2023-11-08T08:19:08.503646+0000","last_peered":"2023-11-08T08:19:08.503646+0000","last_clean":"2023-11-08T08:19:08.503646+0000","last_became_active":"2023-11-08T08:18:58.769357+0000","last_became_peered":"2023-11-08T08:18:58.769357+0000","last_unstale":"2023-11-08T08:19:08.503646+0000","last_undegraded":"2023-11-08T08:19:08.503646+0000","last_fullsized":"2023-11-08T08:19:08.503646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:13.748094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519826+0000","last_change":"2023-11-08T08:19:04.019534+0000","last_active":"2023-11-08T08:19:08.519826+0000","last_peered":"2023-11-08T08:19:08.519826+0000","last_clean":"2023-11-08T08:19:08.519826+0000","last_became_active":"2023-11-08T08:19:04.019269+0000","last_became_peered":"2023-11-08T08:19:04.019269+0000","last_unstale":"2023-11-08T08:19:08.519826+0000","last_undegraded":"2023-11-08T08:19:08.519826+0000","last_fullsized":"2023-11-08T08:19:08.519826+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:18:14.269118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.989010+0000","last_change":"2023-11-08T08:19:08.534164+0000","last_active":"2023-11-08T08:19:08.989010+0000","last_peered":"2023-11-08T08:19:08.989010+0000","last_clean":"2023-11-08T08:19:08.989010+0000","last_became_active":"2023-11-08T08:19:08.533749+0000","last_became_peered":"2023-11-08T08:19:08.533749+0000","last_unstale":"2023-11-08T08:19:08.989010+0000","last_undegraded":"2023-11-08T08:19:08.989010+0000","last_fullsized":"2023-11-08T08:19:08.989010+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:44:30.267694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503356+0000","last_change":"2023-11-08T08:19:04.492859+0000","last_active":"2023-11-08T08:19:08.503356+0000","last_peered":"2023-11-08T08:19:08.503356+0000","last_clean":"2023-11-08T08:19:08.503356+0000","last_became_active":"2023-11-08T08:19:04.492564+0000","last_became_peered":"2023-11-08T08:19:04.492564+0000","last_unstale":"2023-11-08T08:19:08.503356+0000","last_undegraded":"2023-11-08T08:19:08.503356+0000","last_fullsized":"2023-11-08T08:19:08.503356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512115+0000","last_change":"2023-11-08T08:19:04.024444+0000","last_active":"2023-11-08T08:19:08.512115+0000","last_peered":"2023-11-08T08:19:08.512115+0000","last_clean":"2023-11-08T08:19:08.512115+0000","last_became_active":"2023-11-08T08:19:04.024046+0000","last_became_peered":"2023-11-08T08:19:04.024046+0000","last_unstale":"2023-11-08T08:19:08.512115+0000","last_undegraded":"2023-11-08T08:19:08.512115+0000","last_fullsized":"2023-11-08T08:19:08.512115+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:15:32.560862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.514051+0000","last_change":"2023-11-08T08:18:58.771460+0000","last_active":"2023-11-08T08:19:08.514051+0000","last_peered":"2023-11-08T08:19:08.514051+0000","last_clean":"2023-11-08T08:19:08.514051+0000","last_became_active":"2023-11-08T08:18:58.770648+0000","last_became_peered":"2023-11-08T08:18:58.770648+0000","last_unstale":"2023-11-08T08:19:08.514051+0000","last_undegraded":"2023-11-08T08:19:08.514051+0000","last_fullsized":"2023-11-08T08:19:08.514051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.988677+0000","last_change":"2023-11-08T08:19:08.525326+0000","last_active":"2023-11-08T08:19:08.988677+0000","last_peered":"2023-11-08T08:19:08.988677+0000","last_clean":"2023-11-08T08:19:08.988677+0000","last_became_active":"2023-11-08T08:19:08.524957+0000","last_became_peered":"2023-11-08T08:19:08.524957+0000","last_unstale":"2023-11-08T08:19:08.988677+0000","last_undegraded":"2023-11-08T08:19:08.988677+0000","last_fullsized":"2023-11-08T08:19:08.988677+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:08.797102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503646+0000","last_change":"2023-11-08T08:19:04.503414+0000","last_active":"2023-11-08T08:19:08.503646+0000","last_peered":"2023-11-08T08:19:08.503646+0000","last_clean":"2023-11-08T08:19:08.503646+0000","last_became_active":"2023-11-08T08:19:04.503122+0000","last_became_peered":"2023-11-08T08:19:04.503122+0000","last_unstale":"2023-11-08T08:19:08.503646+0000","last_undegraded":"2023-11-08T08:19:08.503646+0000","last_fullsized":"2023-11-08T08:19:08.503646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501677+0000","last_change":"2023-11-08T08:19:04.020937+0000","last_active":"2023-11-08T08:19:08.501677+0000","last_peered":"2023-11-08T08:19:08.501677+0000","last_clean":"2023-11-08T08:19:08.501677+0000","last_became_active":"2023-11-08T08:19:04.020517+0000","last_became_peered":"2023-11-08T08:19:04.020517+0000","last_unstale":"2023-11-08T08:19:08.501677+0000","last_undegraded":"2023-11-08T08:19:08.501677+0000","last_fullsized":"2023-11-08T08:19:08.501677+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:50:41.342112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.509869+0000","last_change":"2023-11-08T08:18:58.973732+0000","last_active":"2023-11-08T08:19:08.509869+0000","last_peered":"2023-11-08T08:19:08.509869+0000","last_clean":"2023-11-08T08:19:08.509869+0000","last_became_active":"2023-11-08T08:18:58.973007+0000","last_became_peered":"2023-11-08T08:18:58.973007+0000","last_unstale":"2023-11-08T08:19:08.509869+0000","last_undegraded":"2023-11-08T08:19:08.509869+0000","last_fullsized":"2023-11-08T08:19:08.509869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.375576+0000","last_change":"2023-11-08T08:19:08.542391+0000","last_active":"2023-11-08T08:19:09.375576+0000","last_peered":"2023-11-08T08:19:09.375576+0000","last_clean":"2023-11-08T08:19:09.375576+0000","last_became_active":"2023-11-08T08:19:08.538962+0000","last_became_peered":"2023-11-08T08:19:08.538962+0000","last_unstale":"2023-11-08T08:19:09.375576+0000","last_undegraded":"2023-11-08T08:19:09.375576+0000","last_fullsized":"2023-11-08T08:19:09.375576+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:05:20.025231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.496994+0000","last_change":"2023-11-08T08:19:04.495234+0000","last_active":"2023-11-08T08:19:08.496994+0000","last_peered":"2023-11-08T08:19:08.496994+0000","last_clean":"2023-11-08T08:19:08.496994+0000","last_became_active":"2023-11-08T08:19:04.494991+0000","last_became_peered":"2023-11-08T08:19:04.494991+0000","last_unstale":"2023-11-08T08:19:08.496994+0000","last_undegraded":"2023-11-08T08:19:08.496994+0000","last_fullsized":"2023-11-08T08:19:08.496994+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:55:43.865351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.494564+0000","last_change":"2023-11-08T08:19:04.025597+0000","last_active":"2023-11-08T08:19:08.494564+0000","last_peered":"2023-11-08T08:19:08.494564+0000","last_clean":"2023-11-08T08:19:08.494564+0000","last_became_active":"2023-11-08T08:19:04.025292+0000","last_became_peered":"2023-11-08T08:19:04.025292+0000","last_unstale":"2023-11-08T08:19:08.494564+0000","last_undegraded":"2023-11-08T08:19:08.494564+0000","last_fullsized":"2023-11-08T08:19:08.494564+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:48:40.803835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.499836+0000","last_change":"2023-11-08T08:18:58.770136+0000","last_active":"2023-11-08T08:19:08.499836+0000","last_peered":"2023-11-08T08:19:08.499836+0000","last_clean":"2023-11-08T08:19:08.499836+0000","last_became_active":"2023-11-08T08:18:58.769797+0000","last_became_peered":"2023-11-08T08:18:58.769797+0000","last_unstale":"2023-11-08T08:19:08.499836+0000","last_undegraded":"2023-11-08T08:19:08.499836+0000","last_fullsized":"2023-11-08T08:19:08.499836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:29:12.107128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372601+0000","last_change":"2023-11-08T08:19:08.532719+0000","last_active":"2023-11-08T08:19:09.372601+0000","last_peered":"2023-11-08T08:19:09.372601+0000","last_clean":"2023-11-08T08:19:09.372601+0000","last_became_active":"2023-11-08T08:19:08.532282+0000","last_became_peered":"2023-11-08T08:19:08.532282+0000","last_unstale":"2023-11-08T08:19:09.372601+0000","last_undegraded":"2023-11-08T08:19:09.372601+0000","last_fullsized":"2023-11-08T08:19:09.372601+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:28:49.369446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.520613+0000","last_change":"2023-11-08T08:19:04.506360+0000","last_active":"2023-11-08T08:19:08.520613+0000","last_peered":"2023-11-08T08:19:08.520613+0000","last_clean":"2023-11-08T08:19:08.520613+0000","last_became_active":"2023-11-08T08:19:04.506117+0000","last_became_peered":"2023-11-08T08:19:04.506117+0000","last_unstale":"2023-11-08T08:19:08.520613+0000","last_undegraded":"2023-11-08T08:19:08.520613+0000","last_fullsized":"2023-11-08T08:19:08.520613+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:17:34.354695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517669+0000","last_change":"2023-11-08T08:19:04.020276+0000","last_active":"2023-11-08T08:19:08.517669+0000","last_peered":"2023-11-08T08:19:08.517669+0000","last_clean":"2023-11-08T08:19:08.517669+0000","last_became_active":"2023-11-08T08:19:04.019739+0000","last_became_peered":"2023-11-08T08:19:04.019739+0000","last_unstale":"2023-11-08T08:19:08.517669+0000","last_undegraded":"2023-11-08T08:19:08.517669+0000","last_fullsized":"2023-11-08T08:19:08.517669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:48:40.803835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519500+0000","last_change":"2023-11-08T08:18:58.777140+0000","last_active":"2023-11-08T08:19:08.519500+0000","last_peered":"2023-11-08T08:19:08.519500+0000","last_clean":"2023-11-08T08:19:08.519500+0000","last_became_active":"2023-11-08T08:18:58.776834+0000","last_became_peered":"2023-11-08T08:18:58.776834+0000","last_unstale":"2023-11-08T08:19:08.519500+0000","last_undegraded":"2023-11-08T08:19:08.519500+0000","last_fullsized":"2023-11-08T08:19:08.519500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.499953+0000","last_change":"2023-11-08T08:18:53.780089+0000","last_active":"2023-11-08T08:19:08.499953+0000","last_peered":"2023-11-08T08:19:08.499953+0000","last_clean":"2023-11-08T08:19:08.499953+0000","last_became_active":"2023-11-08T08:18:53.779533+0000","last_became_peered":"2023-11-08T08:18:53.779533+0000","last_unstale":"2023-11-08T08:19:08.499953+0000","last_undegraded":"2023-11-08T08:19:08.499953+0000","last_fullsized":"2023-11-08T08:19:08.499953+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_clean_scrub_stamp":"2023-11-08T08:18:52.744693+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:58:24.877070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.994953+0000","last_change":"2023-11-08T08:19:08.535027+0000","last_active":"2023-11-08T08:19:08.994953+0000","last_peered":"2023-11-08T08:19:08.994953+0000","last_clean":"2023-11-08T08:19:08.994953+0000","last_became_active":"2023-11-08T08:19:08.534620+0000","last_became_peered":"2023-11-08T08:19:08.534620+0000","last_unstale":"2023-11-08T08:19:08.994953+0000","last_undegraded":"2023-11-08T08:19:08.994953+0000","last_fullsized":"2023-11-08T08:19:08.994953+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:51:11.743430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503552+0000","last_change":"2023-11-08T08:19:04.503360+0000","last_active":"2023-11-08T08:19:08.503552+0000","last_peered":"2023-11-08T08:19:08.503552+0000","last_clean":"2023-11-08T08:19:08.503552+0000","last_became_active":"2023-11-08T08:19:04.502756+0000","last_became_peered":"2023-11-08T08:19:04.502756+0000","last_unstale":"2023-11-08T08:19:08.503552+0000","last_undegraded":"2023-11-08T08:19:08.503552+0000","last_fullsized":"2023-11-08T08:19:08.503552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:57:06.487072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517506+0000","last_change":"2023-11-08T08:19:04.025872+0000","last_active":"2023-11-08T08:19:08.517506+0000","last_peered":"2023-11-08T08:19:08.517506+0000","last_clean":"2023-11-08T08:19:08.517506+0000","last_became_active":"2023-11-08T08:19:04.025567+0000","last_became_peered":"2023-11-08T08:19:04.025567+0000","last_unstale":"2023-11-08T08:19:08.517506+0000","last_undegraded":"2023-11-08T08:19:08.517506+0000","last_fullsized":"2023-11-08T08:19:08.517506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:52:24.620964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.994736+0000","last_change":"2023-11-08T08:19:08.523353+0000","last_active":"2023-11-08T08:19:08.994736+0000","last_peered":"2023-11-08T08:19:08.994736+0000","last_clean":"2023-11-08T08:19:08.994736+0000","last_became_active":"2023-11-08T08:19:08.522741+0000","last_became_peered":"2023-11-08T08:19:08.522741+0000","last_unstale":"2023-11-08T08:19:08.994736+0000","last_undegraded":"2023-11-08T08:19:08.994736+0000","last_fullsized":"2023-11-08T08:19:08.994736+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:01:48.615826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512369+0000","last_change":"2023-11-08T08:19:04.512228+0000","last_active":"2023-11-08T08:19:08.512369+0000","last_peered":"2023-11-08T08:19:08.512369+0000","last_clean":"2023-11-08T08:19:08.512369+0000","last_became_active":"2023-11-08T08:19:04.501319+0000","last_became_peered":"2023-11-08T08:19:04.501319+0000","last_unstale":"2023-11-08T08:19:08.512369+0000","last_undegraded":"2023-11-08T08:19:08.512369+0000","last_fullsized":"2023-11-08T08:19:08.512369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:27:36.453040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513674+0000","last_change":"2023-11-08T08:19:04.012809+0000","last_active":"2023-11-08T08:19:08.513674+0000","last_peered":"2023-11-08T08:19:08.513674+0000","last_clean":"2023-11-08T08:19:08.513674+0000","last_became_active":"2023-11-08T08:19:04.012289+0000","last_became_peered":"2023-11-08T08:19:04.012289+0000","last_unstale":"2023-11-08T08:19:08.513674+0000","last_undegraded":"2023-11-08T08:19:08.513674+0000","last_fullsized":"2023-11-08T08:19:08.513674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:52:24.620964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372268+0000","last_change":"2023-11-08T08:19:08.538675+0000","last_active":"2023-11-08T08:19:09.372268+0000","last_peered":"2023-11-08T08:19:09.372268+0000","last_clean":"2023-11-08T08:19:09.372268+0000","last_became_active":"2023-11-08T08:19:08.537804+0000","last_became_peered":"2023-11-08T08:19:08.537804+0000","last_unstale":"2023-11-08T08:19:09.372268+0000","last_undegraded":"2023-11-08T08:19:09.372268+0000","last_fullsized":"2023-11-08T08:19:09.372268+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:32:44.139418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.516288+0000","last_change":"2023-11-08T08:19:04.498523+0000","last_active":"2023-11-08T08:19:08.516288+0000","last_peered":"2023-11-08T08:19:08.516288+0000","last_clean":"2023-11-08T08:19:08.516288+0000","last_became_active":"2023-11-08T08:19:04.497819+0000","last_became_peered":"2023-11-08T08:19:04.497819+0000","last_unstale":"2023-11-08T08:19:08.516288+0000","last_undegraded":"2023-11-08T08:19:08.516288+0000","last_fullsized":"2023-11-08T08:19:08.516288+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:55:43.865351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518299+0000","last_change":"2023-11-08T08:19:04.024906+0000","last_active":"2023-11-08T08:19:08.518299+0000","last_peered":"2023-11-08T08:19:08.518299+0000","last_clean":"2023-11-08T08:19:08.518299+0000","last_became_active":"2023-11-08T08:19:04.024603+0000","last_became_peered":"2023-11-08T08:19:04.024603+0000","last_unstale":"2023-11-08T08:19:08.518299+0000","last_undegraded":"2023-11-08T08:19:08.518299+0000","last_fullsized":"2023-11-08T08:19:08.518299+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:21:09.218618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372046+0000","last_change":"2023-11-08T08:19:08.525624+0000","last_active":"2023-11-08T08:19:09.372046+0000","last_peered":"2023-11-08T08:19:09.372046+0000","last_clean":"2023-11-08T08:19:09.372046+0000","last_became_active":"2023-11-08T08:19:08.524202+0000","last_became_peered":"2023-11-08T08:19:08.524202+0000","last_unstale":"2023-11-08T08:19:09.372046+0000","last_undegraded":"2023-11-08T08:19:09.372046+0000","last_fullsized":"2023-11-08T08:19:09.372046+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:08.797102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513312+0000","last_change":"2023-11-08T08:19:04.511393+0000","last_active":"2023-11-08T08:19:08.513312+0000","last_peered":"2023-11-08T08:19:08.513312+0000","last_clean":"2023-11-08T08:19:08.513312+0000","last_became_active":"2023-11-08T08:19:04.510749+0000","last_became_peered":"2023-11-08T08:19:04.510749+0000","last_unstale":"2023-11-08T08:19:08.513312+0000","last_undegraded":"2023-11-08T08:19:08.513312+0000","last_fullsized":"2023-11-08T08:19:08.513312+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:09:30.686343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518295+0000","last_change":"2023-11-08T08:19:04.022458+0000","last_active":"2023-11-08T08:19:08.518295+0000","last_peered":"2023-11-08T08:19:08.518295+0000","last_clean":"2023-11-08T08:19:08.518295+0000","last_became_active":"2023-11-08T08:19:04.022003+0000","last_became_peered":"2023-11-08T08:19:04.022003+0000","last_unstale":"2023-11-08T08:19:08.518295+0000","last_undegraded":"2023-11-08T08:19:08.518295+0000","last_fullsized":"2023-11-08T08:19:08.518295+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:21:09.218618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.375400+0000","last_change":"2023-11-08T08:19:08.536502+0000","last_active":"2023-11-08T08:19:09.375400+0000","last_peered":"2023-11-08T08:19:09.375400+0000","last_clean":"2023-11-08T08:19:09.375400+0000","last_became_active":"2023-11-08T08:19:08.536112+0000","last_became_peered":"2023-11-08T08:19:08.536112+0000","last_unstale":"2023-11-08T08:19:09.375400+0000","last_undegraded":"2023-11-08T08:19:09.375400+0000","last_fullsized":"2023-11-08T08:19:09.375400+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:26:20.245634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.495202+0000","last_change":"2023-11-08T08:19:04.493387+0000","last_active":"2023-11-08T08:19:08.495202+0000","last_peered":"2023-11-08T08:19:08.495202+0000","last_clean":"2023-11-08T08:19:08.495202+0000","last_became_active":"2023-11-08T08:19:04.493131+0000","last_became_peered":"2023-11-08T08:19:04.493131+0000","last_unstale":"2023-11-08T08:19:08.495202+0000","last_undegraded":"2023-11-08T08:19:08.495202+0000","last_fullsized":"2023-11-08T08:19:08.495202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:56:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519525+0000","last_change":"2023-11-08T08:19:04.021637+0000","last_active":"2023-11-08T08:19:08.519525+0000","last_peered":"2023-11-08T08:19:08.519525+0000","last_clean":"2023-11-08T08:19:08.519525+0000","last_became_active":"2023-11-08T08:19:04.018995+0000","last_became_peered":"2023-11-08T08:19:04.018995+0000","last_unstale":"2023-11-08T08:19:08.519525+0000","last_undegraded":"2023-11-08T08:19:08.519525+0000","last_fullsized":"2023-11-08T08:19:08.519525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:44.433066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.375204+0000","last_change":"2023-11-08T08:19:08.535800+0000","last_active":"2023-11-08T08:19:09.375204+0000","last_peered":"2023-11-08T08:19:09.375204+0000","last_clean":"2023-11-08T08:19:09.375204+0000","last_became_active":"2023-11-08T08:19:08.529039+0000","last_became_peered":"2023-11-08T08:19:08.529039+0000","last_unstale":"2023-11-08T08:19:09.375204+0000","last_undegraded":"2023-11-08T08:19:09.375204+0000","last_fullsized":"2023-11-08T08:19:09.375204+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:55:58.092159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518136+0000","last_change":"2023-11-08T08:19:04.497093+0000","last_active":"2023-11-08T08:19:08.518136+0000","last_peered":"2023-11-08T08:19:08.518136+0000","last_clean":"2023-11-08T08:19:08.518136+0000","last_became_active":"2023-11-08T08:19:04.496619+0000","last_became_peered":"2023-11-08T08:19:04.496619+0000","last_unstale":"2023-11-08T08:19:08.518136+0000","last_undegraded":"2023-11-08T08:19:08.518136+0000","last_fullsized":"2023-11-08T08:19:08.518136+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:27:39.386956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.500739+0000","last_change":"2023-11-08T08:19:04.015072+0000","last_active":"2023-11-08T08:19:08.500739+0000","last_peered":"2023-11-08T08:19:08.500739+0000","last_clean":"2023-11-08T08:19:08.500739+0000","last_became_active":"2023-11-08T08:19:04.014727+0000","last_became_peered":"2023-11-08T08:19:04.014727+0000","last_unstale":"2023-11-08T08:19:08.500739+0000","last_undegraded":"2023-11-08T08:19:08.500739+0000","last_fullsized":"2023-11-08T08:19:08.500739+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:17:33.341483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.375246+0000","last_change":"2023-11-08T08:19:08.534568+0000","last_active":"2023-11-08T08:19:09.375246+0000","last_peered":"2023-11-08T08:19:09.375246+0000","last_clean":"2023-11-08T08:19:09.375246+0000","last_became_active":"2023-11-08T08:19:08.534139+0000","last_became_peered":"2023-11-08T08:19:08.534139+0000","last_unstale":"2023-11-08T08:19:09.375246+0000","last_undegraded":"2023-11-08T08:19:09.375246+0000","last_fullsized":"2023-11-08T08:19:09.375246+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:59:39.090459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513527+0000","last_change":"2023-11-08T08:19:04.512845+0000","last_active":"2023-11-08T08:19:08.513527+0000","last_peered":"2023-11-08T08:19:08.513527+0000","last_clean":"2023-11-08T08:19:08.513527+0000","last_became_active":"2023-11-08T08:19:04.512499+0000","last_became_peered":"2023-11-08T08:19:04.512499+0000","last_unstale":"2023-11-08T08:19:08.513527+0000","last_undegraded":"2023-11-08T08:19:08.513527+0000","last_fullsized":"2023-11-08T08:19:08.513527+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:24:10.475972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501411+0000","last_change":"2023-11-08T08:19:04.018033+0000","last_active":"2023-11-08T08:19:08.501411+0000","last_peered":"2023-11-08T08:19:08.501411+0000","last_clean":"2023-11-08T08:19:08.501411+0000","last_became_active":"2023-11-08T08:19:04.017059+0000","last_became_peered":"2023-11-08T08:19:04.017059+0000","last_unstale":"2023-11-08T08:19:08.501411+0000","last_undegraded":"2023-11-08T08:19:08.501411+0000","last_fullsized":"2023-11-08T08:19:08.501411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:44.433066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.369706+0000","last_change":"2023-11-08T08:19:08.539326+0000","last_active":"2023-11-08T08:19:09.369706+0000","last_peered":"2023-11-08T08:19:09.369706+0000","last_clean":"2023-11-08T08:19:09.369706+0000","last_became_active":"2023-11-08T08:19:08.538846+0000","last_became_peered":"2023-11-08T08:19:08.538846+0000","last_unstale":"2023-11-08T08:19:09.369706+0000","last_undegraded":"2023-11-08T08:19:09.369706+0000","last_fullsized":"2023-11-08T08:19:09.369706+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:33:00.754113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513023+0000","last_change":"2023-11-08T08:19:04.503981+0000","last_active":"2023-11-08T08:19:08.513023+0000","last_peered":"2023-11-08T08:19:08.513023+0000","last_clean":"2023-11-08T08:19:08.513023+0000","last_became_active":"2023-11-08T08:19:04.503651+0000","last_became_peered":"2023-11-08T08:19:04.503651+0000","last_unstale":"2023-11-08T08:19:08.513023+0000","last_undegraded":"2023-11-08T08:19:08.513023+0000","last_fullsized":"2023-11-08T08:19:08.513023+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:55:43.865351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.500289+0000","last_change":"2023-11-08T08:19:04.022166+0000","last_active":"2023-11-08T08:19:08.500289+0000","last_peered":"2023-11-08T08:19:08.500289+0000","last_clean":"2023-11-08T08:19:08.500289+0000","last_became_active":"2023-11-08T08:19:04.017289+0000","last_became_peered":"2023-11-08T08:19:04.017289+0000","last_unstale":"2023-11-08T08:19:08.500289+0000","last_undegraded":"2023-11-08T08:19:08.500289+0000","last_fullsized":"2023-11-08T08:19:08.500289+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:51:22.228427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.995287+0000","last_change":"2023-11-08T08:19:08.534923+0000","last_active":"2023-11-08T08:19:08.995287+0000","last_peered":"2023-11-08T08:19:08.995287+0000","last_clean":"2023-11-08T08:19:08.995287+0000","last_became_active":"2023-11-08T08:19:08.530869+0000","last_became_peered":"2023-11-08T08:19:08.530869+0000","last_unstale":"2023-11-08T08:19:08.995287+0000","last_undegraded":"2023-11-08T08:19:08.995287+0000","last_fullsized":"2023-11-08T08:19:08.995287+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:47:09.660706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372246+0000","last_change":"2023-11-08T08:19:08.541714+0000","last_active":"2023-11-08T08:19:09.372246+0000","last_peered":"2023-11-08T08:19:09.372246+0000","last_clean":"2023-11-08T08:19:09.372246+0000","last_became_active":"2023-11-08T08:19:08.537859+0000","last_became_peered":"2023-11-08T08:19:08.537859+0000","last_unstale":"2023-11-08T08:19:09.372246+0000","last_undegraded":"2023-11-08T08:19:09.372246+0000","last_fullsized":"2023-11-08T08:19:09.372246+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:33:11.313632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518613+0000","last_change":"2023-11-08T08:19:04.008921+0000","last_active":"2023-11-08T08:19:08.518613+0000","last_peered":"2023-11-08T08:19:08.518613+0000","last_clean":"2023-11-08T08:19:08.518613+0000","last_became_active":"2023-11-08T08:19:04.008616+0000","last_became_peered":"2023-11-08T08:19:04.008616+0000","last_unstale":"2023-11-08T08:19:08.518613+0000","last_undegraded":"2023-11-08T08:19:08.518613+0000","last_fullsized":"2023-11-08T08:19:08.518613+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:50:41.342112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.520011+0000","last_change":"2023-11-08T08:19:04.501127+0000","last_active":"2023-11-08T08:19:08.520011+0000","last_peered":"2023-11-08T08:19:08.520011+0000","last_clean":"2023-11-08T08:19:08.520011+0000","last_became_active":"2023-11-08T08:19:04.500705+0000","last_became_peered":"2023-11-08T08:19:04.500705+0000","last_unstale":"2023-11-08T08:19:08.520011+0000","last_undegraded":"2023-11-08T08:19:08.520011+0000","last_fullsized":"2023-11-08T08:19:08.520011+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:25.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372698+0000","last_change":"2023-11-08T08:19:08.533064+0000","last_active":"2023-11-08T08:19:09.372698+0000","last_peered":"2023-11-08T08:19:09.372698+0000","last_clean":"2023-11-08T08:19:09.372698+0000","last_became_active":"2023-11-08T08:19:08.532631+0000","last_became_peered":"2023-11-08T08:19:08.532631+0000","last_unstale":"2023-11-08T08:19:09.372698+0000","last_undegraded":"2023-11-08T08:19:09.372698+0000","last_fullsized":"2023-11-08T08:19:09.372698+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:32:44.139418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501706+0000","last_change":"2023-11-08T08:19:04.017775+0000","last_active":"2023-11-08T08:19:08.501706+0000","last_peered":"2023-11-08T08:19:08.501706+0000","last_clean":"2023-11-08T08:19:08.501706+0000","last_became_active":"2023-11-08T08:19:04.017572+0000","last_became_peered":"2023-11-08T08:19:04.017572+0000","last_unstale":"2023-11-08T08:19:08.501706+0000","last_undegraded":"2023-11-08T08:19:08.501706+0000","last_fullsized":"2023-11-08T08:19:08.501706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:55:07.041721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.497148+0000","last_change":"2023-11-08T08:19:04.488153+0000","last_active":"2023-11-08T08:19:08.497148+0000","last_peered":"2023-11-08T08:19:08.497148+0000","last_clean":"2023-11-08T08:19:08.497148+0000","last_became_active":"2023-11-08T08:19:04.487871+0000","last_became_peered":"2023-11-08T08:19:04.487871+0000","last_unstale":"2023-11-08T08:19:08.497148+0000","last_undegraded":"2023-11-08T08:19:08.497148+0000","last_fullsized":"2023-11-08T08:19:08.497148+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:06:47.395577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.494806+0000","last_change":"2023-11-08T08:19:04.011650+0000","last_active":"2023-11-08T08:19:08.494806+0000","last_peered":"2023-11-08T08:19:08.494806+0000","last_clean":"2023-11-08T08:19:08.494806+0000","last_became_active":"2023-11-08T08:19:04.011071+0000","last_became_peered":"2023-11-08T08:19:04.011071+0000","last_unstale":"2023-11-08T08:19:08.494806+0000","last_undegraded":"2023-11-08T08:19:08.494806+0000","last_fullsized":"2023-11-08T08:19:08.494806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:51:22.228427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519773+0000","last_change":"2023-11-08T08:19:04.505594+0000","last_active":"2023-11-08T08:19:08.519773+0000","last_peered":"2023-11-08T08:19:08.519773+0000","last_clean":"2023-11-08T08:19:08.519773+0000","last_became_active":"2023-11-08T08:19:04.505258+0000","last_became_peered":"2023-11-08T08:19:04.505258+0000","last_unstale":"2023-11-08T08:19:08.519773+0000","last_undegraded":"2023-11-08T08:19:08.519773+0000","last_fullsized":"2023-11-08T08:19:08.519773+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:56:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.373555+0000","last_change":"2023-11-08T08:19:08.529766+0000","last_active":"2023-11-08T08:19:09.373555+0000","last_peered":"2023-11-08T08:19:09.373555+0000","last_clean":"2023-11-08T08:19:09.373555+0000","last_became_active":"2023-11-08T08:19:08.529334+0000","last_became_peered":"2023-11-08T08:19:08.529334+0000","last_unstale":"2023-11-08T08:19:09.373555+0000","last_undegraded":"2023-11-08T08:19:09.373555+0000","last_fullsized":"2023-11-08T08:19:09.373555+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:28:49.369446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501076+0000","last_change":"2023-11-08T08:19:04.020027+0000","last_active":"2023-11-08T08:19:08.501076+0000","last_peered":"2023-11-08T08:19:08.501076+0000","last_clean":"2023-11-08T08:19:08.501076+0000","last_became_active":"2023-11-08T08:19:04.019727+0000","last_became_peered":"2023-11-08T08:19:04.019727+0000","last_unstale":"2023-11-08T08:19:08.501076+0000","last_undegraded":"2023-11-08T08:19:08.501076+0000","last_fullsized":"2023-11-08T08:19:08.501076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:18:14.269118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519487+0000","last_change":"2023-11-08T08:19:04.495283+0000","last_active":"2023-11-08T08:19:08.519487+0000","last_peered":"2023-11-08T08:19:08.519487+0000","last_clean":"2023-11-08T08:19:08.519487+0000","last_became_active":"2023-11-08T08:19:04.494955+0000","last_became_peered":"2023-11-08T08:19:04.494955+0000","last_unstale":"2023-11-08T08:19:08.519487+0000","last_undegraded":"2023-11-08T08:19:08.519487+0000","last_fullsized":"2023-11-08T08:19:08.519487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372061+0000","last_change":"2023-11-08T08:19:08.532374+0000","last_active":"2023-11-08T08:19:09.372061+0000","last_peered":"2023-11-08T08:19:09.372061+0000","last_clean":"2023-11-08T08:19:09.372061+0000","last_became_active":"2023-11-08T08:19:08.531953+0000","last_became_peered":"2023-11-08T08:19:08.531953+0000","last_unstale":"2023-11-08T08:19:09.372061+0000","last_undegraded":"2023-11-08T08:19:09.372061+0000","last_fullsized":"2023-11-08T08:19:09.372061+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:54:33.577798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518515+0000","last_change":"2023-11-08T08:19:04.491042+0000","last_active":"2023-11-08T08:19:08.518515+0000","last_peered":"2023-11-08T08:19:08.518515+0000","last_clean":"2023-11-08T08:19:08.518515+0000","last_became_active":"2023-11-08T08:19:04.490734+0000","last_became_peered":"2023-11-08T08:19:04.490734+0000","last_unstale":"2023-11-08T08:19:08.518515+0000","last_undegraded":"2023-11-08T08:19:08.518515+0000","last_fullsized":"2023-11-08T08:19:08.518515+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:26:27.887614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517935+0000","last_change":"2023-11-08T08:19:04.507871+0000","last_active":"2023-11-08T08:19:08.517935+0000","last_peered":"2023-11-08T08:19:08.517935+0000","last_clean":"2023-11-08T08:19:08.517935+0000","last_became_active":"2023-11-08T08:19:04.507600+0000","last_became_peered":"2023-11-08T08:19:04.507600+0000","last_unstale":"2023-11-08T08:19:08.517935+0000","last_undegraded":"2023-11-08T08:19:08.517935+0000","last_fullsized":"2023-11-08T08:19:08.517935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:55:43.865351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512136+0000","last_change":"2023-11-08T08:19:04.505023+0000","last_active":"2023-11-08T08:19:08.512136+0000","last_peered":"2023-11-08T08:19:08.512136+0000","last_clean":"2023-11-08T08:19:08.512136+0000","last_became_active":"2023-11-08T08:19:04.504750+0000","last_became_peered":"2023-11-08T08:19:04.504750+0000","last_unstale":"2023-11-08T08:19:08.512136+0000","last_undegraded":"2023-11-08T08:19:08.512136+0000","last_fullsized":"2023-11-08T08:19:08.512136+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:25.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518063+0000","last_change":"2023-11-08T08:19:04.504115+0000","last_active":"2023-11-08T08:19:08.518063+0000","last_peered":"2023-11-08T08:19:08.518063+0000","last_clean":"2023-11-08T08:19:08.518063+0000","last_became_active":"2023-11-08T08:19:04.503841+0000","last_became_peered":"2023-11-08T08:19:04.503841+0000","last_unstale":"2023-11-08T08:19:08.518063+0000","last_undegraded":"2023-11-08T08:19:08.518063+0000","last_fullsized":"2023-11-08T08:19:08.518063+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:33:12.861277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503562+0000","last_change":"2023-11-08T08:19:04.504534+0000","last_active":"2023-11-08T08:19:08.503562+0000","last_peered":"2023-11-08T08:19:08.503562+0000","last_clean":"2023-11-08T08:19:08.503562+0000","last_became_active":"2023-11-08T08:19:04.504283+0000","last_became_peered":"2023-11-08T08:19:04.504283+0000","last_unstale":"2023-11-08T08:19:08.503562+0000","last_undegraded":"2023-11-08T08:19:08.503562+0000","last_fullsized":"2023-11-08T08:19:08.503562+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:56:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519721+0000","last_change":"2023-11-08T08:19:04.503978+0000","last_active":"2023-11-08T08:19:08.519721+0000","last_peered":"2023-11-08T08:19:08.519721+0000","last_clean":"2023-11-08T08:19:08.519721+0000","last_became_active":"2023-11-08T08:19:04.503653+0000","last_became_peered":"2023-11-08T08:19:04.503653+0000","last_unstale":"2023-11-08T08:19:08.519721+0000","last_undegraded":"2023-11-08T08:19:08.519721+0000","last_fullsized":"2023-11-08T08:19:08.519721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503754+0000","last_change":"2023-11-08T08:19:04.491309+0000","last_active":"2023-11-08T08:19:08.503754+0000","last_peered":"2023-11-08T08:19:08.503754+0000","last_clean":"2023-11-08T08:19:08.503754+0000","last_became_active":"2023-11-08T08:19:04.491035+0000","last_became_peered":"2023-11-08T08:19:04.491035+0000","last_unstale":"2023-11-08T08:19:08.503754+0000","last_undegraded":"2023-11-08T08:19:08.503754+0000","last_fullsized":"2023-11-08T08:19:08.503754+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:18:36.796288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503455+0000","last_change":"2023-11-08T08:19:04.503092+0000","last_active":"2023-11-08T08:19:08.503455+0000","last_peered":"2023-11-08T08:19:08.503455+0000","last_clean":"2023-11-08T08:19:08.503455+0000","last_became_active":"2023-11-08T08:19:04.502751+0000","last_became_peered":"2023-11-08T08:19:04.502751+0000","last_unstale":"2023-11-08T08:19:08.503455+0000","last_undegraded":"2023-11-08T08:19:08.503455+0000","last_fullsized":"2023-11-08T08:19:08.503455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:40:44.425321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.499633+0000","last_change":"2023-11-08T08:19:04.509538+0000","last_active":"2023-11-08T08:19:08.499633+0000","last_peered":"2023-11-08T08:19:08.499633+0000","last_clean":"2023-11-08T08:19:08.499633+0000","last_became_active":"2023-11-08T08:19:04.509190+0000","last_became_peered":"2023-11-08T08:19:04.509190+0000","last_unstale":"2023-11-08T08:19:08.499633+0000","last_undegraded":"2023-11-08T08:19:08.499633+0000","last_fullsized":"2023-11-08T08:19:08.499633+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:04:55.302575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517870+0000","last_change":"2023-11-08T08:19:04.489238+0000","last_active":"2023-11-08T08:19:08.517870+0000","last_peered":"2023-11-08T08:19:08.517870+0000","last_clean":"2023-11-08T08:19:08.517870+0000","last_became_active":"2023-11-08T08:19:04.488998+0000","last_became_peered":"2023-11-08T08:19:04.488998+0000","last_unstale":"2023-11-08T08:19:08.517870+0000","last_undegraded":"2023-11-08T08:19:08.517870+0000","last_fullsized":"2023-11-08T08:19:08.517870+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:27:36.453040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.494671+0000","last_change":"2023-11-08T08:19:04.499236+0000","last_active":"2023-11-08T08:19:08.494671+0000","last_peered":"2023-11-08T08:19:08.494671+0000","last_clean":"2023-11-08T08:19:08.494671+0000","last_became_active":"2023-11-08T08:19:04.498978+0000","last_became_peered":"2023-11-08T08:19:04.498978+0000","last_unstale":"2023-11-08T08:19:08.494671+0000","last_undegraded":"2023-11-08T08:19:08.494671+0000","last_fullsized":"2023-11-08T08:19:08.494671+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:04:55.302575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.500995+0000","last_change":"2023-11-08T08:19:04.513191+0000","last_active":"2023-11-08T08:19:08.500995+0000","last_peered":"2023-11-08T08:19:08.500995+0000","last_clean":"2023-11-08T08:19:08.500995+0000","last_became_active":"2023-11-08T08:19:04.512519+0000","last_became_peered":"2023-11-08T08:19:04.512519+0000","last_unstale":"2023-11-08T08:19:08.500995+0000","last_undegraded":"2023-11-08T08:19:08.500995+0000","last_fullsized":"2023-11-08T08:19:08.500995+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:56:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513262+0000","last_change":"2023-11-08T08:19:04.510593+0000","last_active":"2023-11-08T08:19:08.513262+0000","last_peered":"2023-11-08T08:19:08.513262+0000","last_clean":"2023-11-08T08:19:08.513262+0000","last_became_active":"2023-11-08T08:19:04.501361+0000","last_became_peered":"2023-11-08T08:19:04.501361+0000","last_unstale":"2023-11-08T08:19:08.513262+0000","last_undegraded":"2023-11-08T08:19:08.513262+0000","last_fullsized":"2023-11-08T08:19:08.513262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.509488+0000","last_change":"2023-11-08T08:19:04.497954+0000","last_active":"2023-11-08T08:19:08.509488+0000","last_peered":"2023-11-08T08:19:08.509488+0000","last_clean":"2023-11-08T08:19:08.509488+0000","last_became_active":"2023-11-08T08:19:04.497675+0000","last_became_peered":"2023-11-08T08:19:04.497675+0000","last_unstale":"2023-11-08T08:19:08.509488+0000","last_undegraded":"2023-11-08T08:19:08.509488+0000","last_fullsized":"2023-11-08T08:19:08.509488+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:07:36.146057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512026+0000","last_change":"2023-11-08T08:19:04.494551+0000","last_active":"2023-11-08T08:19:08.512026+0000","last_peered":"2023-11-08T08:19:08.512026+0000","last_clean":"2023-11-08T08:19:08.512026+0000","last_became_active":"2023-11-08T08:19:04.494261+0000","last_became_peered":"2023-11-08T08:19:04.494261+0000","last_unstale":"2023-11-08T08:19:08.512026+0000","last_undegraded":"2023-11-08T08:19:08.512026+0000","last_fullsized":"2023-11-08T08:19:08.512026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:27:40.872420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.496353+0000","last_change":"2023-11-08T08:19:04.486999+0000","last_active":"2023-11-08T08:19:08.496353+0000","last_peered":"2023-11-08T08:19:08.496353+0000","last_clean":"2023-11-08T08:19:08.496353+0000","last_became_active":"2023-11-08T08:19:04.486746+0000","last_became_peered":"2023-11-08T08:19:04.486746+0000","last_unstale":"2023-11-08T08:19:08.496353+0000","last_undegraded":"2023-11-08T08:19:08.496353+0000","last_fullsized":"2023-11-08T08:19:08.496353+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:27:36.453040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513306+0000","last_change":"2023-11-08T08:19:04.506263+0000","last_active":"2023-11-08T08:19:08.513306+0000","last_peered":"2023-11-08T08:19:08.513306+0000","last_clean":"2023-11-08T08:19:08.513306+0000","last_became_active":"2023-11-08T08:19:04.505930+0000","last_became_peered":"2023-11-08T08:19:04.505930+0000","last_unstale":"2023-11-08T08:19:08.513306+0000","last_undegraded":"2023-11-08T08:19:08.513306+0000","last_fullsized":"2023-11-08T08:19:08.513306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:41:09.071125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501581+0000","last_change":"2023-11-08T08:19:04.514677+0000","last_active":"2023-11-08T08:19:08.501581+0000","last_peered":"2023-11-08T08:19:08.501581+0000","last_clean":"2023-11-08T08:19:08.501581+0000","last_became_active":"2023-11-08T08:19:04.511465+0000","last_became_peered":"2023-11-08T08:19:04.511465+0000","last_unstale":"2023-11-08T08:19:08.501581+0000","last_undegraded":"2023-11-08T08:19:08.501581+0000","last_fullsized":"2023-11-08T08:19:08.501581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513976+0000","last_change":"2023-11-08T08:19:04.490773+0000","last_active":"2023-11-08T08:19:08.513976+0000","last_peered":"2023-11-08T08:19:08.513976+0000","last_clean":"2023-11-08T08:19:08.513976+0000","last_became_active":"2023-11-08T08:19:04.490474+0000","last_became_peered":"2023-11-08T08:19:04.490474+0000","last_unstale":"2023-11-08T08:19:08.513976+0000","last_undegraded":"2023-11-08T08:19:08.513976+0000","last_fullsized":"2023-11-08T08:19:08.513976+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512799+0000","last_change":"2023-11-08T08:19:04.487159+0000","last_active":"2023-11-08T08:19:08.512799+0000","last_peered":"2023-11-08T08:19:08.512799+0000","last_clean":"2023-11-08T08:19:08.512799+0000","last_became_active":"2023-11-08T08:19:04.486861+0000","last_became_peered":"2023-11-08T08:19:04.486861+0000","last_unstale":"2023-11-08T08:19:08.512799+0000","last_undegraded":"2023-11-08T08:19:08.512799+0000","last_fullsized":"2023-11-08T08:19:08.512799+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:45.446278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.509301+0000","last_change":"2023-11-08T08:19:04.491757+0000","last_active":"2023-11-08T08:19:08.509301+0000","last_peered":"2023-11-08T08:19:08.509301+0000","last_clean":"2023-11-08T08:19:08.509301+0000","last_became_active":"2023-11-08T08:19:04.491359+0000","last_became_peered":"2023-11-08T08:19:04.491359+0000","last_unstale":"2023-11-08T08:19:08.509301+0000","last_undegraded":"2023-11-08T08:19:08.509301+0000","last_fullsized":"2023-11-08T08:19:08.509301+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:01:42.294189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.516256+0000","last_change":"2023-11-08T08:19:04.498748+0000","last_active":"2023-11-08T08:19:08.516256+0000","last_peered":"2023-11-08T08:19:08.516256+0000","last_clean":"2023-11-08T08:19:08.516256+0000","last_became_active":"2023-11-08T08:19:04.498208+0000","last_became_peered":"2023-11-08T08:19:04.498208+0000","last_unstale":"2023-11-08T08:19:08.516256+0000","last_undegraded":"2023-11-08T08:19:08.516256+0000","last_fullsized":"2023-11-08T08:19:08.516256+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:33:12.861277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521468+0000","last_change":"2023-11-08T08:19:04.490052+0000","last_active":"2023-11-08T08:19:08.521468+0000","last_peered":"2023-11-08T08:19:08.521468+0000","last_clean":"2023-11-08T08:19:08.521468+0000","last_became_active":"2023-11-08T08:19:04.489758+0000","last_became_peered":"2023-11-08T08:19:04.489758+0000","last_unstale":"2023-11-08T08:19:08.521468+0000","last_undegraded":"2023-11-08T08:19:08.521468+0000","last_fullsized":"2023-11-08T08:19:08.521468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:25.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.495092+0000","last_change":"2023-11-08T08:19:04.489823+0000","last_active":"2023-11-08T08:19:08.495092+0000","last_peered":"2023-11-08T08:19:08.495092+0000","last_clean":"2023-11-08T08:19:08.495092+0000","last_became_active":"2023-11-08T08:19:04.489559+0000","last_became_peered":"2023-11-08T08:19:04.489559+0000","last_unstale":"2023-11-08T08:19:08.495092+0000","last_undegraded":"2023-11-08T08:19:08.495092+0000","last_fullsized":"2023-11-08T08:19:08.495092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:25.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512940+0000","last_change":"2023-11-08T08:19:04.509192+0000","last_active":"2023-11-08T08:19:08.512940+0000","last_peered":"2023-11-08T08:19:08.512940+0000","last_clean":"2023-11-08T08:19:08.512940+0000","last_became_active":"2023-11-08T08:19:04.508856+0000","last_became_peered":"2023-11-08T08:19:04.508856+0000","last_unstale":"2023-11-08T08:19:08.512940+0000","last_undegraded":"2023-11-08T08:19:08.512940+0000","last_fullsized":"2023-11-08T08:19:08.512940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:27:36.453040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501455+0000","last_change":"2023-11-08T08:19:04.510274+0000","last_active":"2023-11-08T08:19:08.501455+0000","last_peered":"2023-11-08T08:19:08.501455+0000","last_clean":"2023-11-08T08:19:08.501455+0000","last_became_active":"2023-11-08T08:19:04.510052+0000","last_became_peered":"2023-11-08T08:19:04.510052+0000","last_unstale":"2023-11-08T08:19:08.501455+0000","last_undegraded":"2023-11-08T08:19:08.501455+0000","last_fullsized":"2023-11-08T08:19:08.501455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.511920+0000","last_change":"2023-11-08T08:19:04.507388+0000","last_active":"2023-11-08T08:19:08.511920+0000","last_peered":"2023-11-08T08:19:08.511920+0000","last_clean":"2023-11-08T08:19:08.511920+0000","last_became_active":"2023-11-08T08:19:04.506655+0000","last_became_peered":"2023-11-08T08:19:04.506655+0000","last_unstale":"2023-11-08T08:19:08.511920+0000","last_undegraded":"2023-11-08T08:19:08.511920+0000","last_fullsized":"2023-11-08T08:19:08.511920+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:26:16.737280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.495536+0000","last_change":"2023-11-08T08:19:04.484402+0000","last_active":"2023-11-08T08:19:08.495536+0000","last_peered":"2023-11-08T08:19:08.495536+0000","last_clean":"2023-11-08T08:19:08.495536+0000","last_became_active":"2023-11-08T08:19:04.480030+0000","last_became_peered":"2023-11-08T08:19:04.480030+0000","last_unstale":"2023-11-08T08:19:08.495536+0000","last_undegraded":"2023-11-08T08:19:08.495536+0000","last_fullsized":"2023-11-08T08:19:08.495536+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:27:40.872420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517912+0000","last_change":"2023-11-08T08:19:04.500239+0000","last_active":"2023-11-08T08:19:08.517912+0000","last_peered":"2023-11-08T08:19:08.517912+0000","last_clean":"2023-11-08T08:19:08.517912+0000","last_became_active":"2023-11-08T08:19:04.499933+0000","last_became_peered":"2023-11-08T08:19:04.499933+0000","last_unstale":"2023-11-08T08:19:08.517912+0000","last_undegraded":"2023-11-08T08:19:08.517912+0000","last_fullsized":"2023-11-08T08:19:08.517912+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:45.446278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.516154+0000","last_change":"2023-11-08T08:19:04.507285+0000","last_active":"2023-11-08T08:19:08.516154+0000","last_peered":"2023-11-08T08:19:08.516154+0000","last_clean":"2023-11-08T08:19:08.516154+0000","last_became_active":"2023-11-08T08:19:04.506999+0000","last_became_peered":"2023-11-08T08:19:04.506999+0000","last_unstale":"2023-11-08T08:19:08.516154+0000","last_undegraded":"2023-11-08T08:19:08.516154+0000","last_fullsized":"2023-11-08T08:19:08.516154+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:55:54.583805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521418+0000","last_change":"2023-11-08T08:19:04.496585+0000","last_active":"2023-11-08T08:19:08.521418+0000","last_peered":"2023-11-08T08:19:08.521418+0000","last_clean":"2023-11-08T08:19:08.521418+0000","last_became_active":"2023-11-08T08:19:04.496307+0000","last_became_peered":"2023-11-08T08:19:04.496307+0000","last_unstale":"2023-11-08T08:19:08.521418+0000","last_undegraded":"2023-11-08T08:19:08.521418+0000","last_fullsized":"2023-11-08T08:19:08.521418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:06:47.395577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.509596+0000","last_change":"2023-11-08T08:19:04.487836+0000","last_active":"2023-11-08T08:19:08.509596+0000","last_peered":"2023-11-08T08:19:08.509596+0000","last_clean":"2023-11-08T08:19:08.509596+0000","last_became_active":"2023-11-08T08:19:04.487548+0000","last_became_peered":"2023-11-08T08:19:04.487548+0000","last_unstale":"2023-11-08T08:19:08.509596+0000","last_undegraded":"2023-11-08T08:19:08.509596+0000","last_fullsized":"2023-11-08T08:19:08.509596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:55:08.054932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4527,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9296,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362544,"statfs":{"total":96636764160,"available":96627245056,"internally_reserved":0,"allocated":925696,"data_stored":718901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5029999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.282}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5409999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.581}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8176,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363664,"statfs":{"total":96636764160,"available":96628391936,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.194}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.456}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8184,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363656,"statfs":{"total":96636764160,"available":96628383744,"internally_reserved":0,"allocated":311296,"data_stored":125208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5089999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.569}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.425}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.952}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8260,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363580,"statfs":{"total":96636764160,"available":96628305920,"internally_reserved":0,"allocated":323584,"data_stored":128373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.522}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.496}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8244,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363596,"statfs":{"total":96636764160,"available":96628322304,"internally_reserved":0,"allocated":307200,"data_stored":124662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.498}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8112,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363728,"statfs":{"total":96636764160,"available":96628457472,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.194}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.456}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8240,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363600,"statfs":{"total":96636764160,"available":96628326400,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.659}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8089999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.615}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2320000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9272,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362568,"statfs":{"total":96636764160,"available":96627269632,"internally_reserved":0,"allocated":901120,"data_stored":715506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T08:19:32.365 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T08:19:32.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.575+0000 7f5b7a4cf700 1 Processor -- start 2023-11-08T08:19:32.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.575+0000 7f5b7a4cf700 1 -- start start 2023-11-08T08:19:32.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.579+0000 7f5b7a4cf700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b7415a480 0x7f5b740cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:32.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.579+0000 7f5b7a4cf700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5b740c8d10 0x7f5b740cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:32.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.579+0000 7f5b7a4cf700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7415a850 con 0x7f5b740ccd10 2023-11-08T08:19:32.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.579+0000 7f5b7a4cf700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b740ce060 con 0x7f5b740c8d10 2023-11-08T08:19:32.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.579+0000 7f5b7a4cf700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b740ce300 con 0x7f5b7415a480 2023-11-08T08:19:32.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.579+0000 7f5b73fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b7415a480 0x7f5b740cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:32.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.579+0000 7f5b73fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b7415a480 0x7f5b740cc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:53250/0 (socket says 172.21.15.40:53250) 2023-11-08T08:19:32.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.579+0000 7f5b737fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5b740c8d10 0x7f5b740cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:32.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.579+0000 7f5b73fff700 1 -- 172.21.15.40:0/3368979723 learned_addr learned my addr 172.21.15.40:0/3368979723 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:32.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.579+0000 7f5b78a6c700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f5b740ccd10 0x7f5b7415dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:38138/0 (socket says 172.21.15.40:38138) 2023-11-08T08:19:32.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.579+0000 7f5b73fff700 1 -- 172.21.15.40:0/3368979723 >> v2:172.21.15.169:3300/0 conn(0x7f5b740c8d10 msgr2=0x7f5b740cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:32.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.579+0000 7f5b73fff700 1 --2- 172.21.15.40:0/3368979723 >> v2:172.21.15.169:3300/0 conn(0x7f5b740c8d10 0x7f5b740cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:32.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.579+0000 7f5b73fff700 1 -- 172.21.15.40:0/3368979723 >> v1:172.21.15.169:6789/0 conn(0x7f5b740ccd10 legacy=0x7f5b7415dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:32.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.579+0000 7f5b73fff700 1 -- 172.21.15.40:0/3368979723 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b740cada0 con 0x7f5b7415a480 2023-11-08T08:19:32.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.583+0000 7f5b73fff700 1 --2- 172.21.15.40:0/3368979723 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b7415a480 0x7f5b740cc290 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5b64009e50 tx=0x7f5b6400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2979dc8d59179c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:32.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.583+0000 7f5b72ffd700 1 -- 172.21.15.40:0/3368979723 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6400e070 con 0x7f5b7415a480 2023-11-08T08:19:32.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.583+0000 7f5b72ffd700 1 -- 172.21.15.40:0/3368979723 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6400b910 con 0x7f5b7415a480 2023-11-08T08:19:32.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.583+0000 7f5b72ffd700 1 -- 172.21.15.40:0/3368979723 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6400bcc0 con 0x7f5b7415a480 2023-11-08T08:19:32.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.583+0000 7f5b7a4cf700 1 -- 172.21.15.40:0/3368979723 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b7415a480 msgr2=0x7f5b740cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.583+0000 7f5b7a4cf700 1 --2- 172.21.15.40:0/3368979723 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b7415a480 0x7f5b740cc290 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5b64009e50 tx=0x7f5b6400b330 comp rx=0 tx=0).stop 2023-11-08T08:19:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.583+0000 7f5b7a4cf700 1 -- 172.21.15.40:0/3368979723 shutdown_connections 2023-11-08T08:19:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.583+0000 7f5b7a4cf700 1 --2- 172.21.15.40:0/3368979723 >> v2:172.21.15.169:3300/0 conn(0x7f5b740c8d10 0x7f5b740cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.583+0000 7f5b7a4cf700 1 --2- 172.21.15.40:0/3368979723 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b7415a480 0x7f5b740cc290 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:32.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.583+0000 7f5b7a4cf700 1 -- 172.21.15.40:0/3368979723 >> 172.21.15.40:0/3368979723 conn(0x7f5b740cf350 msgr2=0x7f5b740b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:32.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.583+0000 7f5b7a4cf700 1 -- 172.21.15.40:0/3368979723 shutdown_connections 2023-11-08T08:19:32.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.583+0000 7f5b7a4cf700 1 -- 172.21.15.40:0/3368979723 wait complete. 2023-11-08T08:19:32.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.587+0000 7f5b7a4cf700 1 Processor -- start 2023-11-08T08:19:32.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.587+0000 7f5b7a4cf700 1 -- start start 2023-11-08T08:19:32.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.587+0000 7f5b7a4cf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b740c8d10 0x7f5b741f9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:32.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.587+0000 7f5b7a4cf700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b740ccd10 0x7f5b7406c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:32.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.587+0000 7f5b7a4cf700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b7415a480 0x7f5b741f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:32.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.587+0000 7f5b7a4cf700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b741f9b70 con 0x7f5b740c8d10 2023-11-08T08:19:32.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.587+0000 7f5b7a4cf700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7406c950 con 0x7f5b7415a480 2023-11-08T08:19:32.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.587+0000 7f5b7a4cf700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b741f6730 con 0x7f5b740ccd10 2023-11-08T08:19:32.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.587+0000 7f5b73fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b740c8d10 0x7f5b741f9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:32.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.587+0000 7f5b73fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b740c8d10 0x7f5b741f9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52194/0 (socket says 172.21.15.40:52194) 2023-11-08T08:19:32.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.587+0000 7f5b73fff700 1 -- 172.21.15.40:0/2192808908 learned_addr learned my addr 172.21.15.40:0/2192808908 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:32.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.587+0000 7f5b78a6c700 1 --2- 172.21.15.40:0/2192808908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b7415a480 0x7f5b741f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:32.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.587+0000 7f5b737fe700 1 --2- 172.21.15.40:0/2192808908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b740ccd10 0x7f5b7406c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:32.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.587+0000 7f5b73fff700 1 -- 172.21.15.40:0/2192808908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b740ccd10 msgr2=0x7f5b7406c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:32.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.587+0000 7f5b73fff700 1 --2- 172.21.15.40:0/2192808908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b740ccd10 0x7f5b7406c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:32.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.587+0000 7f5b73fff700 1 -- 172.21.15.40:0/2192808908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b7415a480 msgr2=0x7f5b741f78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:32.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.587+0000 7f5b73fff700 1 --2- 172.21.15.40:0/2192808908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b7415a480 0x7f5b741f78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:32.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.587+0000 7f5b73fff700 1 -- 172.21.15.40:0/2192808908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b64009b30 con 0x7f5b740c8d10 2023-11-08T08:19:32.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.591+0000 7f5b73fff700 1 --2- 172.21.15.40:0/2192808908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b740c8d10 0x7f5b741f9630 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5b64000f80 tx=0x7f5b64004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:32.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.591+0000 7f5b70ff9700 1 -- 172.21.15.40:0/2192808908 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64004880 con 0x7f5b740c8d10 2023-11-08T08:19:32.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.591+0000 7f5b70ff9700 1 -- 172.21.15.40:0/2192808908 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b640099b0 con 0x7f5b740c8d10 2023-11-08T08:19:32.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.591+0000 7f5b70ff9700 1 -- 172.21.15.40:0/2192808908 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6401b670 con 0x7f5b740c8d10 2023-11-08T08:19:32.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.591+0000 7f5b7a4cf700 1 -- 172.21.15.40:0/2192808908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b740add50 con 0x7f5b740c8d10 2023-11-08T08:19:32.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.591+0000 7f5b7a4cf700 1 -- 172.21.15.40:0/2192808908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b740caa30 con 0x7f5b740c8d10 2023-11-08T08:19:32.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.591+0000 7f5b7a4cf700 1 -- 172.21.15.40:0/2192808908 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b740ca6e0 con 0x7f5b740c8d10 2023-11-08T08:19:32.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.595+0000 7f5b70ff9700 1 -- 172.21.15.40:0/2192808908 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5b64021030 con 0x7f5b740c8d10 2023-11-08T08:19:32.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.595+0000 7f5b70ff9700 1 --2- 172.21.15.40:0/2192808908 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5b60064f30 0x7f5b600673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:32.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.595+0000 7f5b70ff9700 1 -- 172.21.15.40:0/2192808908 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5b640835c0 con 0x7f5b740c8d10 2023-11-08T08:19:32.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.599+0000 7f5b737fe700 1 --2- 172.21.15.40:0/2192808908 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5b60064f30 0x7f5b600673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:32.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.599+0000 7f5b737fe700 1 --2- 172.21.15.40:0/2192808908 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5b60064f30 0x7f5b600673e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5b7406dce0 tx=0x7f5b68009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:32.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.603+0000 7f5b70ff9700 1 -- 172.21.15.40:0/2192808908 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5b64051400 con 0x7f5b740c8d10 2023-11-08T08:19:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.779+0000 7f5b7a4cf700 1 -- 172.21.15.40:0/2192808908 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5b741f68f0 con 0x7f5b60064f30 2023-11-08T08:19:32.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.803+0000 7f5b70ff9700 1 -- 172.21.15.40:0/2192808908 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+366970 (secure 0 0 0) 0x7f5b741f68f0 con 0x7f5b60064f30 2023-11-08T08:19:32.808 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:32.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.811+0000 7f5b7a4cf700 1 -- 172.21.15.40:0/2192808908 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5b60064f30 msgr2=0x7f5b600673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:32.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.811+0000 7f5b7a4cf700 1 --2- 172.21.15.40:0/2192808908 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5b60064f30 0x7f5b600673e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5b7406dce0 tx=0x7f5b68009240 comp rx=0 tx=0).stop 2023-11-08T08:19:32.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.811+0000 7f5b7a4cf700 1 -- 172.21.15.40:0/2192808908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b740c8d10 msgr2=0x7f5b741f9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:32.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.811+0000 7f5b7a4cf700 1 --2- 172.21.15.40:0/2192808908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b740c8d10 0x7f5b741f9630 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5b64000f80 tx=0x7f5b64004af0 comp rx=0 tx=0).stop 2023-11-08T08:19:32.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.811+0000 7f5b7a4cf700 1 -- 172.21.15.40:0/2192808908 shutdown_connections 2023-11-08T08:19:32.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.811+0000 7f5b7a4cf700 1 --2- 172.21.15.40:0/2192808908 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5b60064f30 0x7f5b600673e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.811+0000 7f5b7a4cf700 1 --2- 172.21.15.40:0/2192808908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b7415a480 0x7f5b741f78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.811+0000 7f5b7a4cf700 1 --2- 172.21.15.40:0/2192808908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b740ccd10 0x7f5b7406c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.811+0000 7f5b7a4cf700 1 --2- 172.21.15.40:0/2192808908 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b740c8d10 0x7f5b741f9630 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.811+0000 7f5b7a4cf700 1 -- 172.21.15.40:0/2192808908 >> 172.21.15.40:0/2192808908 conn(0x7f5b740cf350 msgr2=0x7f5b740adee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.811+0000 7f5b7a4cf700 1 -- 172.21.15.40:0/2192808908 shutdown_connections 2023-11-08T08:19:32.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:32.811+0000 7f5b7a4cf700 1 -- 172.21.15.40:0/2192808908 wait complete. 2023-11-08T08:19:32.817 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-11-08T08:19:32.835 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2023-11-08T08:19:31.889739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594895,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67784,"kb_used_data":3592,"kb_used_omap":0,"kb_used_meta":64128,"kb_avail":754906936,"statfs":{"total":773094113280,"available":773024702464,"internally_reserved":0,"allocated":3678208,"data_stored":2186366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65667072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004841"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.370829+0000","last_change":"2023-11-08T08:19:08.519864+0000","last_active":"2023-11-08T08:19:09.370829+0000","last_peered":"2023-11-08T08:19:09.370829+0000","last_clean":"2023-11-08T08:19:09.370829+0000","last_became_active":"2023-11-08T08:19:08.518845+0000","last_became_peered":"2023-11-08T08:19:08.518845+0000","last_unstale":"2023-11-08T08:19:09.370829+0000","last_undegraded":"2023-11-08T08:19:09.370829+0000","last_fullsized":"2023-11-08T08:19:09.370829+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:47:58.348898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.496570+0000","last_change":"2023-11-08T08:19:04.015635+0000","last_active":"2023-11-08T08:19:08.496570+0000","last_peered":"2023-11-08T08:19:08.496570+0000","last_clean":"2023-11-08T08:19:08.496570+0000","last_became_active":"2023-11-08T08:19:04.014748+0000","last_became_peered":"2023-11-08T08:19:04.014748+0000","last_unstale":"2023-11-08T08:19:08.496570+0000","last_undegraded":"2023-11-08T08:19:08.496570+0000","last_fullsized":"2023-11-08T08:19:08.496570+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:01:41.280977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513402+0000","last_change":"2023-11-08T08:19:04.505994+0000","last_active":"2023-11-08T08:19:08.513402+0000","last_peered":"2023-11-08T08:19:08.513402+0000","last_clean":"2023-11-08T08:19:08.513402+0000","last_became_active":"2023-11-08T08:19:04.505596+0000","last_became_peered":"2023-11-08T08:19:04.505596+0000","last_unstale":"2023-11-08T08:19:08.513402+0000","last_undegraded":"2023-11-08T08:19:08.513402+0000","last_fullsized":"2023-11-08T08:19:08.513402+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:06:47.395577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.992842+0000","last_change":"2023-11-08T08:19:08.526866+0000","last_active":"2023-11-08T08:19:08.992842+0000","last_peered":"2023-11-08T08:19:08.992842+0000","last_clean":"2023-11-08T08:19:08.992842+0000","last_became_active":"2023-11-08T08:19:08.526405+0000","last_became_peered":"2023-11-08T08:19:08.526405+0000","last_unstale":"2023-11-08T08:19:08.992842+0000","last_undegraded":"2023-11-08T08:19:08.992842+0000","last_fullsized":"2023-11-08T08:19:08.992842+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:54:12.699998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519574+0000","last_change":"2023-11-08T08:19:04.021634+0000","last_active":"2023-11-08T08:19:08.519574+0000","last_peered":"2023-11-08T08:19:08.519574+0000","last_clean":"2023-11-08T08:19:08.519574+0000","last_became_active":"2023-11-08T08:19:04.017916+0000","last_became_peered":"2023-11-08T08:19:04.017916+0000","last_unstale":"2023-11-08T08:19:08.519574+0000","last_undegraded":"2023-11-08T08:19:08.519574+0000","last_fullsized":"2023-11-08T08:19:08.519574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:58:24.706350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518078+0000","last_change":"2023-11-08T08:19:04.481276+0000","last_active":"2023-11-08T08:19:08.518078+0000","last_peered":"2023-11-08T08:19:08.518078+0000","last_clean":"2023-11-08T08:19:08.518078+0000","last_became_active":"2023-11-08T08:19:04.480340+0000","last_became_peered":"2023-11-08T08:19:04.480340+0000","last_unstale":"2023-11-08T08:19:08.518078+0000","last_undegraded":"2023-11-08T08:19:08.518078+0000","last_fullsized":"2023-11-08T08:19:08.518078+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:04:55.302575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372099+0000","last_change":"2023-11-08T08:19:08.528726+0000","last_active":"2023-11-08T08:19:09.372099+0000","last_peered":"2023-11-08T08:19:09.372099+0000","last_clean":"2023-11-08T08:19:09.372099+0000","last_became_active":"2023-11-08T08:19:08.527853+0000","last_became_peered":"2023-11-08T08:19:08.527853+0000","last_unstale":"2023-11-08T08:19:09.372099+0000","last_undegraded":"2023-11-08T08:19:09.372099+0000","last_fullsized":"2023-11-08T08:19:09.372099+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:16:14.032597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519819+0000","last_change":"2023-11-08T08:19:04.025513+0000","last_active":"2023-11-08T08:19:08.519819+0000","last_peered":"2023-11-08T08:19:08.519819+0000","last_clean":"2023-11-08T08:19:08.519819+0000","last_became_active":"2023-11-08T08:19:04.025220+0000","last_became_peered":"2023-11-08T08:19:04.025220+0000","last_unstale":"2023-11-08T08:19:08.519819+0000","last_undegraded":"2023-11-08T08:19:08.519819+0000","last_fullsized":"2023-11-08T08:19:08.519819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:55:07.041721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512027+0000","last_change":"2023-11-08T08:19:04.512364+0000","last_active":"2023-11-08T08:19:08.512027+0000","last_peered":"2023-11-08T08:19:08.512027+0000","last_clean":"2023-11-08T08:19:08.512027+0000","last_became_active":"2023-11-08T08:19:04.511958+0000","last_became_peered":"2023-11-08T08:19:04.511958+0000","last_unstale":"2023-11-08T08:19:08.512027+0000","last_undegraded":"2023-11-08T08:19:08.512027+0000","last_fullsized":"2023-11-08T08:19:08.512027+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:32:57.245760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.994907+0000","last_change":"2023-11-08T08:19:08.527583+0000","last_active":"2023-11-08T08:19:08.994907+0000","last_peered":"2023-11-08T08:19:08.994907+0000","last_clean":"2023-11-08T08:19:08.994907+0000","last_became_active":"2023-11-08T08:19:08.527054+0000","last_became_peered":"2023-11-08T08:19:08.527054+0000","last_unstale":"2023-11-08T08:19:08.994907+0000","last_undegraded":"2023-11-08T08:19:08.994907+0000","last_fullsized":"2023-11-08T08:19:08.994907+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:08.797102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519426+0000","last_change":"2023-11-08T08:19:04.025416+0000","last_active":"2023-11-08T08:19:08.519426+0000","last_peered":"2023-11-08T08:19:08.519426+0000","last_clean":"2023-11-08T08:19:08.519426+0000","last_became_active":"2023-11-08T08:19:04.025091+0000","last_became_peered":"2023-11-08T08:19:04.025091+0000","last_unstale":"2023-11-08T08:19:08.519426+0000","last_undegraded":"2023-11-08T08:19:08.519426+0000","last_fullsized":"2023-11-08T08:19:08.519426+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:07:35.132845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.511963+0000","last_change":"2023-11-08T08:19:04.512794+0000","last_active":"2023-11-08T08:19:08.511963+0000","last_peered":"2023-11-08T08:19:08.511963+0000","last_clean":"2023-11-08T08:19:08.511963+0000","last_became_active":"2023-11-08T08:19:04.512499+0000","last_became_peered":"2023-11-08T08:19:04.512499+0000","last_unstale":"2023-11-08T08:19:08.511963+0000","last_undegraded":"2023-11-08T08:19:08.511963+0000","last_fullsized":"2023-11-08T08:19:08.511963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:05:16.516877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372287+0000","last_change":"2023-11-08T08:19:08.539289+0000","last_active":"2023-11-08T08:19:09.372287+0000","last_peered":"2023-11-08T08:19:09.372287+0000","last_clean":"2023-11-08T08:19:09.372287+0000","last_became_active":"2023-11-08T08:19:08.532011+0000","last_became_peered":"2023-11-08T08:19:08.532011+0000","last_unstale":"2023-11-08T08:19:09.372287+0000","last_undegraded":"2023-11-08T08:19:09.372287+0000","last_fullsized":"2023-11-08T08:19:09.372287+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:56:38.412909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.495705+0000","last_change":"2023-11-08T08:19:04.024246+0000","last_active":"2023-11-08T08:19:08.495705+0000","last_peered":"2023-11-08T08:19:08.495705+0000","last_clean":"2023-11-08T08:19:08.495705+0000","last_became_active":"2023-11-08T08:19:04.023681+0000","last_became_peered":"2023-11-08T08:19:04.023681+0000","last_unstale":"2023-11-08T08:19:08.495705+0000","last_undegraded":"2023-11-08T08:19:08.495705+0000","last_fullsized":"2023-11-08T08:19:08.495705+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:50:41.342112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.509382+0000","last_change":"2023-11-08T08:19:04.494535+0000","last_active":"2023-11-08T08:19:08.509382+0000","last_peered":"2023-11-08T08:19:08.509382+0000","last_clean":"2023-11-08T08:19:08.509382+0000","last_became_active":"2023-11-08T08:19:04.494226+0000","last_became_peered":"2023-11-08T08:19:04.494226+0000","last_unstale":"2023-11-08T08:19:08.509382+0000","last_undegraded":"2023-11-08T08:19:08.509382+0000","last_fullsized":"2023-11-08T08:19:08.509382+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:18:15.282329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.993497+0000","last_change":"2023-11-08T08:19:08.532723+0000","last_active":"2023-11-08T08:19:08.993497+0000","last_peered":"2023-11-08T08:19:08.993497+0000","last_clean":"2023-11-08T08:19:08.993497+0000","last_became_active":"2023-11-08T08:19:08.532320+0000","last_became_peered":"2023-11-08T08:19:08.532320+0000","last_unstale":"2023-11-08T08:19:08.993497+0000","last_undegraded":"2023-11-08T08:19:08.993497+0000","last_fullsized":"2023-11-08T08:19:08.993497+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:10:25.517705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512234+0000","last_change":"2023-11-08T08:19:04.020285+0000","last_active":"2023-11-08T08:19:08.512234+0000","last_peered":"2023-11-08T08:19:08.512234+0000","last_clean":"2023-11-08T08:19:08.512234+0000","last_became_active":"2023-11-08T08:19:04.019791+0000","last_became_peered":"2023-11-08T08:19:04.019791+0000","last_unstale":"2023-11-08T08:19:08.512234+0000","last_undegraded":"2023-11-08T08:19:08.512234+0000","last_fullsized":"2023-11-08T08:19:08.512234+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:21:09.218618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521337+0000","last_change":"2023-11-08T08:19:04.497059+0000","last_active":"2023-11-08T08:19:08.521337+0000","last_peered":"2023-11-08T08:19:08.521337+0000","last_clean":"2023-11-08T08:19:08.521337+0000","last_became_active":"2023-11-08T08:19:04.495199+0000","last_became_peered":"2023-11-08T08:19:04.495199+0000","last_unstale":"2023-11-08T08:19:08.521337+0000","last_undegraded":"2023-11-08T08:19:08.521337+0000","last_fullsized":"2023-11-08T08:19:08.521337+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:56:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.374791+0000","last_change":"2023-11-08T08:19:08.530445+0000","last_active":"2023-11-08T08:19:09.374791+0000","last_peered":"2023-11-08T08:19:09.374791+0000","last_clean":"2023-11-08T08:19:09.374791+0000","last_became_active":"2023-11-08T08:19:08.530020+0000","last_became_peered":"2023-11-08T08:19:08.530020+0000","last_unstale":"2023-11-08T08:19:09.374791+0000","last_undegraded":"2023-11-08T08:19:09.374791+0000","last_fullsized":"2023-11-08T08:19:09.374791+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:24:13.984326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.502373+0000","last_change":"2023-11-08T08:19:04.018882+0000","last_active":"2023-11-08T08:19:08.502373+0000","last_peered":"2023-11-08T08:19:08.502373+0000","last_clean":"2023-11-08T08:19:08.502373+0000","last_became_active":"2023-11-08T08:19:04.018537+0000","last_became_peered":"2023-11-08T08:19:04.018537+0000","last_unstale":"2023-11-08T08:19:08.502373+0000","last_undegraded":"2023-11-08T08:19:08.502373+0000","last_fullsized":"2023-11-08T08:19:08.502373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:51:22.228427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.496833+0000","last_change":"2023-11-08T08:19:04.503358+0000","last_active":"2023-11-08T08:19:08.496833+0000","last_peered":"2023-11-08T08:19:08.496833+0000","last_clean":"2023-11-08T08:19:08.496833+0000","last_became_active":"2023-11-08T08:19:04.502856+0000","last_became_peered":"2023-11-08T08:19:04.502856+0000","last_unstale":"2023-11-08T08:19:08.496833+0000","last_undegraded":"2023-11-08T08:19:08.496833+0000","last_fullsized":"2023-11-08T08:19:08.496833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.989197+0000","last_change":"2023-11-08T08:19:08.531064+0000","last_active":"2023-11-08T08:19:08.989197+0000","last_peered":"2023-11-08T08:19:08.989197+0000","last_clean":"2023-11-08T08:19:08.989197+0000","last_became_active":"2023-11-08T08:19:08.530517+0000","last_became_peered":"2023-11-08T08:19:08.530517+0000","last_unstale":"2023-11-08T08:19:08.989197+0000","last_undegraded":"2023-11-08T08:19:08.989197+0000","last_fullsized":"2023-11-08T08:19:08.989197+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:51:11.743430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517381+0000","last_change":"2023-11-08T08:19:04.022640+0000","last_active":"2023-11-08T08:19:08.517381+0000","last_peered":"2023-11-08T08:19:08.517381+0000","last_clean":"2023-11-08T08:19:08.517381+0000","last_became_active":"2023-11-08T08:19:04.022244+0000","last_became_peered":"2023-11-08T08:19:04.022244+0000","last_unstale":"2023-11-08T08:19:08.517381+0000","last_undegraded":"2023-11-08T08:19:08.517381+0000","last_fullsized":"2023-11-08T08:19:08.517381+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:50:41.342112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521762+0000","last_change":"2023-11-08T08:19:04.497516+0000","last_active":"2023-11-08T08:19:08.521762+0000","last_peered":"2023-11-08T08:19:08.521762+0000","last_clean":"2023-11-08T08:19:08.521762+0000","last_became_active":"2023-11-08T08:19:04.493572+0000","last_became_peered":"2023-11-08T08:19:04.493572+0000","last_unstale":"2023-11-08T08:19:08.521762+0000","last_undegraded":"2023-11-08T08:19:08.521762+0000","last_fullsized":"2023-11-08T08:19:08.521762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:57:06.487072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517281+0000","last_change":"2023-11-08T08:19:04.495610+0000","last_active":"2023-11-08T08:19:08.517281+0000","last_peered":"2023-11-08T08:19:08.517281+0000","last_clean":"2023-11-08T08:19:08.517281+0000","last_became_active":"2023-11-08T08:19:04.495307+0000","last_became_peered":"2023-11-08T08:19:04.495307+0000","last_unstale":"2023-11-08T08:19:08.517281+0000","last_undegraded":"2023-11-08T08:19:08.517281+0000","last_fullsized":"2023-11-08T08:19:08.517281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.496569+0000","last_change":"2023-11-08T08:19:04.015753+0000","last_active":"2023-11-08T08:19:08.496569+0000","last_peered":"2023-11-08T08:19:08.496569+0000","last_clean":"2023-11-08T08:19:08.496569+0000","last_became_active":"2023-11-08T08:19:04.015173+0000","last_became_peered":"2023-11-08T08:19:04.015173+0000","last_unstale":"2023-11-08T08:19:08.496569+0000","last_undegraded":"2023-11-08T08:19:08.496569+0000","last_fullsized":"2023-11-08T08:19:08.496569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:58:24.706350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.373895+0000","last_change":"2023-11-08T08:19:08.525297+0000","last_active":"2023-11-08T08:19:09.373895+0000","last_peered":"2023-11-08T08:19:09.373895+0000","last_clean":"2023-11-08T08:19:09.373895+0000","last_became_active":"2023-11-08T08:19:08.524571+0000","last_became_peered":"2023-11-08T08:19:08.524571+0000","last_unstale":"2023-11-08T08:19:09.373895+0000","last_undegraded":"2023-11-08T08:19:09.373895+0000","last_fullsized":"2023-11-08T08:19:09.373895+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:33:00.754113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517964+0000","last_change":"2023-11-08T08:19:04.500363+0000","last_active":"2023-11-08T08:19:08.517964+0000","last_peered":"2023-11-08T08:19:08.517964+0000","last_clean":"2023-11-08T08:19:08.517964+0000","last_became_active":"2023-11-08T08:19:04.500049+0000","last_became_peered":"2023-11-08T08:19:04.500049+0000","last_unstale":"2023-11-08T08:19:08.517964+0000","last_undegraded":"2023-11-08T08:19:08.517964+0000","last_fullsized":"2023-11-08T08:19:08.517964+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:06:47.395577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521546+0000","last_change":"2023-11-08T08:19:04.020555+0000","last_active":"2023-11-08T08:19:08.521546+0000","last_peered":"2023-11-08T08:19:08.521546+0000","last_clean":"2023-11-08T08:19:08.521546+0000","last_became_active":"2023-11-08T08:19:04.020053+0000","last_became_peered":"2023-11-08T08:19:04.020053+0000","last_unstale":"2023-11-08T08:19:08.521546+0000","last_undegraded":"2023-11-08T08:19:08.521546+0000","last_fullsized":"2023-11-08T08:19:08.521546+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:01:41.280977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.994150+0000","last_change":"2023-11-08T08:19:08.526638+0000","last_active":"2023-11-08T08:19:08.994150+0000","last_peered":"2023-11-08T08:19:08.994150+0000","last_clean":"2023-11-08T08:19:08.994150+0000","last_became_active":"2023-11-08T08:19:08.526219+0000","last_became_peered":"2023-11-08T08:19:08.526219+0000","last_unstale":"2023-11-08T08:19:08.994150+0000","last_undegraded":"2023-11-08T08:19:08.994150+0000","last_fullsized":"2023-11-08T08:19:08.994150+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:56:38.412909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.500591+0000","last_change":"2023-11-08T08:19:04.022271+0000","last_active":"2023-11-08T08:19:08.500591+0000","last_peered":"2023-11-08T08:19:08.500591+0000","last_clean":"2023-11-08T08:19:08.500591+0000","last_became_active":"2023-11-08T08:19:04.021306+0000","last_became_peered":"2023-11-08T08:19:04.021306+0000","last_unstale":"2023-11-08T08:19:08.500591+0000","last_undegraded":"2023-11-08T08:19:08.500591+0000","last_fullsized":"2023-11-08T08:19:08.500591+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:26:26.874402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.502798+0000","last_change":"2023-11-08T08:19:04.494816+0000","last_active":"2023-11-08T08:19:08.502798+0000","last_peered":"2023-11-08T08:19:08.502798+0000","last_clean":"2023-11-08T08:19:08.502798+0000","last_became_active":"2023-11-08T08:19:04.494321+0000","last_became_peered":"2023-11-08T08:19:04.494321+0000","last_unstale":"2023-11-08T08:19:08.502798+0000","last_undegraded":"2023-11-08T08:19:08.502798+0000","last_fullsized":"2023-11-08T08:19:08.502798+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:04:55.302575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372527+0000","last_change":"2023-11-08T08:19:08.542779+0000","last_active":"2023-11-08T08:19:09.372527+0000","last_peered":"2023-11-08T08:19:09.372527+0000","last_clean":"2023-11-08T08:19:09.372527+0000","last_became_active":"2023-11-08T08:19:08.542399+0000","last_became_peered":"2023-11-08T08:19:08.542399+0000","last_unstale":"2023-11-08T08:19:09.372527+0000","last_undegraded":"2023-11-08T08:19:09.372527+0000","last_fullsized":"2023-11-08T08:19:09.372527+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:16:14.032597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:21.997708+0000","last_change":"2023-11-08T08:19:04.018152+0000","last_active":"2023-11-08T08:19:21.997708+0000","last_peered":"2023-11-08T08:19:21.997708+0000","last_clean":"2023-11-08T08:19:21.997708+0000","last_became_active":"2023-11-08T08:19:04.017695+0000","last_became_peered":"2023-11-08T08:19:04.017695+0000","last_unstale":"2023-11-08T08:19:21.997708+0000","last_undegraded":"2023-11-08T08:19:21.997708+0000","last_fullsized":"2023-11-08T08:19:21.997708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:55:20.782396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3745,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512046+0000","last_change":"2023-11-08T08:19:04.501576+0000","last_active":"2023-11-08T08:19:08.512046+0000","last_peered":"2023-11-08T08:19:08.512046+0000","last_clean":"2023-11-08T08:19:08.512046+0000","last_became_active":"2023-11-08T08:19:04.500829+0000","last_became_peered":"2023-11-08T08:19:04.500829+0000","last_unstale":"2023-11-08T08:19:08.512046+0000","last_undegraded":"2023-11-08T08:19:08.512046+0000","last_fullsized":"2023-11-08T08:19:08.512046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372324+0000","last_change":"2023-11-08T08:19:08.536704+0000","last_active":"2023-11-08T08:19:09.372324+0000","last_peered":"2023-11-08T08:19:09.372324+0000","last_clean":"2023-11-08T08:19:09.372324+0000","last_became_active":"2023-11-08T08:19:08.536339+0000","last_became_peered":"2023-11-08T08:19:08.536339+0000","last_unstale":"2023-11-08T08:19:09.372324+0000","last_undegraded":"2023-11-08T08:19:09.372324+0000","last_fullsized":"2023-11-08T08:19:09.372324+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:51:11.743430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513610+0000","last_change":"2023-11-08T08:19:04.496877+0000","last_active":"2023-11-08T08:19:08.513610+0000","last_peered":"2023-11-08T08:19:08.513610+0000","last_clean":"2023-11-08T08:19:08.513610+0000","last_became_active":"2023-11-08T08:19:04.496505+0000","last_became_peered":"2023-11-08T08:19:04.496505+0000","last_unstale":"2023-11-08T08:19:08.513610+0000","last_undegraded":"2023-11-08T08:19:08.513610+0000","last_fullsized":"2023-11-08T08:19:08.513610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:59:35.582106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521817+0000","last_change":"2023-11-08T08:18:58.771310+0000","last_active":"2023-11-08T08:19:08.521817+0000","last_peered":"2023-11-08T08:19:08.521817+0000","last_clean":"2023-11-08T08:19:08.521817+0000","last_became_active":"2023-11-08T08:18:58.770891+0000","last_became_peered":"2023-11-08T08:18:58.770891+0000","last_unstale":"2023-11-08T08:19:08.521817+0000","last_undegraded":"2023-11-08T08:19:08.521817+0000","last_fullsized":"2023-11-08T08:19:08.521817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"27'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.969187+0000","last_change":"2023-11-08T08:19:04.012926+0000","last_active":"2023-11-08T08:19:08.969187+0000","last_peered":"2023-11-08T08:19:08.969187+0000","last_clean":"2023-11-08T08:19:08.969187+0000","last_became_active":"2023-11-08T08:19:04.012634+0000","last_became_peered":"2023-11-08T08:19:04.012634+0000","last_unstale":"2023-11-08T08:19:08.969187+0000","last_undegraded":"2023-11-08T08:19:08.969187+0000","last_fullsized":"2023-11-08T08:19:08.969187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:07:35.132845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.371641+0000","last_change":"2023-11-08T08:19:08.532617+0000","last_active":"2023-11-08T08:19:09.371641+0000","last_peered":"2023-11-08T08:19:09.371641+0000","last_clean":"2023-11-08T08:19:09.371641+0000","last_became_active":"2023-11-08T08:19:08.532177+0000","last_became_peered":"2023-11-08T08:19:08.532177+0000","last_unstale":"2023-11-08T08:19:09.371641+0000","last_undegraded":"2023-11-08T08:19:09.371641+0000","last_fullsized":"2023-11-08T08:19:09.371641+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:08.797102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517321+0000","last_change":"2023-11-08T08:19:04.504132+0000","last_active":"2023-11-08T08:19:08.517321+0000","last_peered":"2023-11-08T08:19:08.517321+0000","last_clean":"2023-11-08T08:19:08.517321+0000","last_became_active":"2023-11-08T08:19:04.503747+0000","last_became_peered":"2023-11-08T08:19:04.503747+0000","last_unstale":"2023-11-08T08:19:08.517321+0000","last_undegraded":"2023-11-08T08:19:08.517321+0000","last_fullsized":"2023-11-08T08:19:08.517321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:27:40.872420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503547+0000","last_change":"2023-11-08T08:18:58.770483+0000","last_active":"2023-11-08T08:19:08.503547+0000","last_peered":"2023-11-08T08:19:08.503547+0000","last_clean":"2023-11-08T08:19:08.503547+0000","last_became_active":"2023-11-08T08:18:58.770040+0000","last_became_peered":"2023-11-08T08:18:58.770040+0000","last_unstale":"2023-11-08T08:19:08.503547+0000","last_undegraded":"2023-11-08T08:19:08.503547+0000","last_fullsized":"2023-11-08T08:19:08.503547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.520152+0000","last_change":"2023-11-08T08:19:04.022667+0000","last_active":"2023-11-08T08:19:08.520152+0000","last_peered":"2023-11-08T08:19:08.520152+0000","last_clean":"2023-11-08T08:19:08.520152+0000","last_became_active":"2023-11-08T08:19:04.022430+0000","last_became_peered":"2023-11-08T08:19:04.022430+0000","last_unstale":"2023-11-08T08:19:08.520152+0000","last_undegraded":"2023-11-08T08:19:08.520152+0000","last_fullsized":"2023-11-08T08:19:08.520152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:15:32.560862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.992931+0000","last_change":"2023-11-08T08:19:08.531983+0000","last_active":"2023-11-08T08:19:08.992931+0000","last_peered":"2023-11-08T08:19:08.992931+0000","last_clean":"2023-11-08T08:19:08.992931+0000","last_became_active":"2023-11-08T08:19:08.531584+0000","last_became_peered":"2023-11-08T08:19:08.531584+0000","last_unstale":"2023-11-08T08:19:08.992931+0000","last_undegraded":"2023-11-08T08:19:08.992931+0000","last_fullsized":"2023-11-08T08:19:08.992931+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:56:34.397672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.495387+0000","last_change":"2023-11-08T08:19:04.481283+0000","last_active":"2023-11-08T08:19:08.495387+0000","last_peered":"2023-11-08T08:19:08.495387+0000","last_clean":"2023-11-08T08:19:08.495387+0000","last_became_active":"2023-11-08T08:19:04.480992+0000","last_became_peered":"2023-11-08T08:19:04.480992+0000","last_unstale":"2023-11-08T08:19:08.495387+0000","last_undegraded":"2023-11-08T08:19:08.495387+0000","last_fullsized":"2023-11-08T08:19:08.495387+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.500946+0000","last_change":"2023-11-08T08:18:58.766848+0000","last_active":"2023-11-08T08:19:08.500946+0000","last_peered":"2023-11-08T08:19:08.500946+0000","last_clean":"2023-11-08T08:19:08.500946+0000","last_became_active":"2023-11-08T08:18:58.766378+0000","last_became_peered":"2023-11-08T08:18:58.766378+0000","last_unstale":"2023-11-08T08:19:08.500946+0000","last_undegraded":"2023-11-08T08:19:08.500946+0000","last_fullsized":"2023-11-08T08:19:08.500946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503466+0000","last_change":"2023-11-08T08:19:04.018717+0000","last_active":"2023-11-08T08:19:08.503466+0000","last_peered":"2023-11-08T08:19:08.503466+0000","last_clean":"2023-11-08T08:19:08.503466+0000","last_became_active":"2023-11-08T08:19:04.018355+0000","last_became_peered":"2023-11-08T08:19:04.018355+0000","last_unstale":"2023-11-08T08:19:08.503466+0000","last_undegraded":"2023-11-08T08:19:08.503466+0000","last_fullsized":"2023-11-08T08:19:08.503466+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:21:09.218618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.374050+0000","last_change":"2023-11-08T08:19:08.539068+0000","last_active":"2023-11-08T08:19:09.374050+0000","last_peered":"2023-11-08T08:19:09.374050+0000","last_clean":"2023-11-08T08:19:09.374050+0000","last_became_active":"2023-11-08T08:19:08.538763+0000","last_became_peered":"2023-11-08T08:19:08.538763+0000","last_unstale":"2023-11-08T08:19:09.374050+0000","last_undegraded":"2023-11-08T08:19:09.374050+0000","last_fullsized":"2023-11-08T08:19:09.374050+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:27:44.380774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.502722+0000","last_change":"2023-11-08T08:19:04.508515+0000","last_active":"2023-11-08T08:19:08.502722+0000","last_peered":"2023-11-08T08:19:08.502722+0000","last_clean":"2023-11-08T08:19:08.502722+0000","last_became_active":"2023-11-08T08:19:04.508227+0000","last_became_peered":"2023-11-08T08:19:04.508227+0000","last_unstale":"2023-11-08T08:19:08.502722+0000","last_undegraded":"2023-11-08T08:19:08.502722+0000","last_fullsized":"2023-11-08T08:19:08.502722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:48:06.180083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503646+0000","last_change":"2023-11-08T08:18:58.769764+0000","last_active":"2023-11-08T08:19:08.503646+0000","last_peered":"2023-11-08T08:19:08.503646+0000","last_clean":"2023-11-08T08:19:08.503646+0000","last_became_active":"2023-11-08T08:18:58.769357+0000","last_became_peered":"2023-11-08T08:18:58.769357+0000","last_unstale":"2023-11-08T08:19:08.503646+0000","last_undegraded":"2023-11-08T08:19:08.503646+0000","last_fullsized":"2023-11-08T08:19:08.503646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:13.748094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519826+0000","last_change":"2023-11-08T08:19:04.019534+0000","last_active":"2023-11-08T08:19:08.519826+0000","last_peered":"2023-11-08T08:19:08.519826+0000","last_clean":"2023-11-08T08:19:08.519826+0000","last_became_active":"2023-11-08T08:19:04.019269+0000","last_became_peered":"2023-11-08T08:19:04.019269+0000","last_unstale":"2023-11-08T08:19:08.519826+0000","last_undegraded":"2023-11-08T08:19:08.519826+0000","last_fullsized":"2023-11-08T08:19:08.519826+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:18:14.269118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.989010+0000","last_change":"2023-11-08T08:19:08.534164+0000","last_active":"2023-11-08T08:19:08.989010+0000","last_peered":"2023-11-08T08:19:08.989010+0000","last_clean":"2023-11-08T08:19:08.989010+0000","last_became_active":"2023-11-08T08:19:08.533749+0000","last_became_peered":"2023-11-08T08:19:08.533749+0000","last_unstale":"2023-11-08T08:19:08.989010+0000","last_undegraded":"2023-11-08T08:19:08.989010+0000","last_fullsized":"2023-11-08T08:19:08.989010+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:44:30.267694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503356+0000","last_change":"2023-11-08T08:19:04.492859+0000","last_active":"2023-11-08T08:19:08.503356+0000","last_peered":"2023-11-08T08:19:08.503356+0000","last_clean":"2023-11-08T08:19:08.503356+0000","last_became_active":"2023-11-08T08:19:04.492564+0000","last_became_peered":"2023-11-08T08:19:04.492564+0000","last_unstale":"2023-11-08T08:19:08.503356+0000","last_undegraded":"2023-11-08T08:19:08.503356+0000","last_fullsized":"2023-11-08T08:19:08.503356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512115+0000","last_change":"2023-11-08T08:19:04.024444+0000","last_active":"2023-11-08T08:19:08.512115+0000","last_peered":"2023-11-08T08:19:08.512115+0000","last_clean":"2023-11-08T08:19:08.512115+0000","last_became_active":"2023-11-08T08:19:04.024046+0000","last_became_peered":"2023-11-08T08:19:04.024046+0000","last_unstale":"2023-11-08T08:19:08.512115+0000","last_undegraded":"2023-11-08T08:19:08.512115+0000","last_fullsized":"2023-11-08T08:19:08.512115+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:15:32.560862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.514051+0000","last_change":"2023-11-08T08:18:58.771460+0000","last_active":"2023-11-08T08:19:08.514051+0000","last_peered":"2023-11-08T08:19:08.514051+0000","last_clean":"2023-11-08T08:19:08.514051+0000","last_became_active":"2023-11-08T08:18:58.770648+0000","last_became_peered":"2023-11-08T08:18:58.770648+0000","last_unstale":"2023-11-08T08:19:08.514051+0000","last_undegraded":"2023-11-08T08:19:08.514051+0000","last_fullsized":"2023-11-08T08:19:08.514051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.988677+0000","last_change":"2023-11-08T08:19:08.525326+0000","last_active":"2023-11-08T08:19:08.988677+0000","last_peered":"2023-11-08T08:19:08.988677+0000","last_clean":"2023-11-08T08:19:08.988677+0000","last_became_active":"2023-11-08T08:19:08.524957+0000","last_became_peered":"2023-11-08T08:19:08.524957+0000","last_unstale":"2023-11-08T08:19:08.988677+0000","last_undegraded":"2023-11-08T08:19:08.988677+0000","last_fullsized":"2023-11-08T08:19:08.988677+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:08.797102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503646+0000","last_change":"2023-11-08T08:19:04.503414+0000","last_active":"2023-11-08T08:19:08.503646+0000","last_peered":"2023-11-08T08:19:08.503646+0000","last_clean":"2023-11-08T08:19:08.503646+0000","last_became_active":"2023-11-08T08:19:04.503122+0000","last_became_peered":"2023-11-08T08:19:04.503122+0000","last_unstale":"2023-11-08T08:19:08.503646+0000","last_undegraded":"2023-11-08T08:19:08.503646+0000","last_fullsized":"2023-11-08T08:19:08.503646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501677+0000","last_change":"2023-11-08T08:19:04.020937+0000","last_active":"2023-11-08T08:19:08.501677+0000","last_peered":"2023-11-08T08:19:08.501677+0000","last_clean":"2023-11-08T08:19:08.501677+0000","last_became_active":"2023-11-08T08:19:04.020517+0000","last_became_peered":"2023-11-08T08:19:04.020517+0000","last_unstale":"2023-11-08T08:19:08.501677+0000","last_undegraded":"2023-11-08T08:19:08.501677+0000","last_fullsized":"2023-11-08T08:19:08.501677+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:50:41.342112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.509869+0000","last_change":"2023-11-08T08:18:58.973732+0000","last_active":"2023-11-08T08:19:08.509869+0000","last_peered":"2023-11-08T08:19:08.509869+0000","last_clean":"2023-11-08T08:19:08.509869+0000","last_became_active":"2023-11-08T08:18:58.973007+0000","last_became_peered":"2023-11-08T08:18:58.973007+0000","last_unstale":"2023-11-08T08:19:08.509869+0000","last_undegraded":"2023-11-08T08:19:08.509869+0000","last_fullsized":"2023-11-08T08:19:08.509869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.375576+0000","last_change":"2023-11-08T08:19:08.542391+0000","last_active":"2023-11-08T08:19:09.375576+0000","last_peered":"2023-11-08T08:19:09.375576+0000","last_clean":"2023-11-08T08:19:09.375576+0000","last_became_active":"2023-11-08T08:19:08.538962+0000","last_became_peered":"2023-11-08T08:19:08.538962+0000","last_unstale":"2023-11-08T08:19:09.375576+0000","last_undegraded":"2023-11-08T08:19:09.375576+0000","last_fullsized":"2023-11-08T08:19:09.375576+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:05:20.025231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.496994+0000","last_change":"2023-11-08T08:19:04.495234+0000","last_active":"2023-11-08T08:19:08.496994+0000","last_peered":"2023-11-08T08:19:08.496994+0000","last_clean":"2023-11-08T08:19:08.496994+0000","last_became_active":"2023-11-08T08:19:04.494991+0000","last_became_peered":"2023-11-08T08:19:04.494991+0000","last_unstale":"2023-11-08T08:19:08.496994+0000","last_undegraded":"2023-11-08T08:19:08.496994+0000","last_fullsized":"2023-11-08T08:19:08.496994+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:55:43.865351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.494564+0000","last_change":"2023-11-08T08:19:04.025597+0000","last_active":"2023-11-08T08:19:08.494564+0000","last_peered":"2023-11-08T08:19:08.494564+0000","last_clean":"2023-11-08T08:19:08.494564+0000","last_became_active":"2023-11-08T08:19:04.025292+0000","last_became_peered":"2023-11-08T08:19:04.025292+0000","last_unstale":"2023-11-08T08:19:08.494564+0000","last_undegraded":"2023-11-08T08:19:08.494564+0000","last_fullsized":"2023-11-08T08:19:08.494564+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:48:40.803835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.499836+0000","last_change":"2023-11-08T08:18:58.770136+0000","last_active":"2023-11-08T08:19:08.499836+0000","last_peered":"2023-11-08T08:19:08.499836+0000","last_clean":"2023-11-08T08:19:08.499836+0000","last_became_active":"2023-11-08T08:18:58.769797+0000","last_became_peered":"2023-11-08T08:18:58.769797+0000","last_unstale":"2023-11-08T08:19:08.499836+0000","last_undegraded":"2023-11-08T08:19:08.499836+0000","last_fullsized":"2023-11-08T08:19:08.499836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:29:12.107128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372601+0000","last_change":"2023-11-08T08:19:08.532719+0000","last_active":"2023-11-08T08:19:09.372601+0000","last_peered":"2023-11-08T08:19:09.372601+0000","last_clean":"2023-11-08T08:19:09.372601+0000","last_became_active":"2023-11-08T08:19:08.532282+0000","last_became_peered":"2023-11-08T08:19:08.532282+0000","last_unstale":"2023-11-08T08:19:09.372601+0000","last_undegraded":"2023-11-08T08:19:09.372601+0000","last_fullsized":"2023-11-08T08:19:09.372601+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:28:49.369446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.520613+0000","last_change":"2023-11-08T08:19:04.506360+0000","last_active":"2023-11-08T08:19:08.520613+0000","last_peered":"2023-11-08T08:19:08.520613+0000","last_clean":"2023-11-08T08:19:08.520613+0000","last_became_active":"2023-11-08T08:19:04.506117+0000","last_became_peered":"2023-11-08T08:19:04.506117+0000","last_unstale":"2023-11-08T08:19:08.520613+0000","last_undegraded":"2023-11-08T08:19:08.520613+0000","last_fullsized":"2023-11-08T08:19:08.520613+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:17:34.354695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517669+0000","last_change":"2023-11-08T08:19:04.020276+0000","last_active":"2023-11-08T08:19:08.517669+0000","last_peered":"2023-11-08T08:19:08.517669+0000","last_clean":"2023-11-08T08:19:08.517669+0000","last_became_active":"2023-11-08T08:19:04.019739+0000","last_became_peered":"2023-11-08T08:19:04.019739+0000","last_unstale":"2023-11-08T08:19:08.517669+0000","last_undegraded":"2023-11-08T08:19:08.517669+0000","last_fullsized":"2023-11-08T08:19:08.517669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:48:40.803835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519500+0000","last_change":"2023-11-08T08:18:58.777140+0000","last_active":"2023-11-08T08:19:08.519500+0000","last_peered":"2023-11-08T08:19:08.519500+0000","last_clean":"2023-11-08T08:19:08.519500+0000","last_became_active":"2023-11-08T08:18:58.776834+0000","last_became_peered":"2023-11-08T08:18:58.776834+0000","last_unstale":"2023-11-08T08:19:08.519500+0000","last_undegraded":"2023-11-08T08:19:08.519500+0000","last_fullsized":"2023-11-08T08:19:08.519500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.499953+0000","last_change":"2023-11-08T08:18:53.780089+0000","last_active":"2023-11-08T08:19:08.499953+0000","last_peered":"2023-11-08T08:19:08.499953+0000","last_clean":"2023-11-08T08:19:08.499953+0000","last_became_active":"2023-11-08T08:18:53.779533+0000","last_became_peered":"2023-11-08T08:18:53.779533+0000","last_unstale":"2023-11-08T08:19:08.499953+0000","last_undegraded":"2023-11-08T08:19:08.499953+0000","last_fullsized":"2023-11-08T08:19:08.499953+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_clean_scrub_stamp":"2023-11-08T08:18:52.744693+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:58:24.877070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.994953+0000","last_change":"2023-11-08T08:19:08.535027+0000","last_active":"2023-11-08T08:19:08.994953+0000","last_peered":"2023-11-08T08:19:08.994953+0000","last_clean":"2023-11-08T08:19:08.994953+0000","last_became_active":"2023-11-08T08:19:08.534620+0000","last_became_peered":"2023-11-08T08:19:08.534620+0000","last_unstale":"2023-11-08T08:19:08.994953+0000","last_undegraded":"2023-11-08T08:19:08.994953+0000","last_fullsized":"2023-11-08T08:19:08.994953+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:51:11.743430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503552+0000","last_change":"2023-11-08T08:19:04.503360+0000","last_active":"2023-11-08T08:19:08.503552+0000","last_peered":"2023-11-08T08:19:08.503552+0000","last_clean":"2023-11-08T08:19:08.503552+0000","last_became_active":"2023-11-08T08:19:04.502756+0000","last_became_peered":"2023-11-08T08:19:04.502756+0000","last_unstale":"2023-11-08T08:19:08.503552+0000","last_undegraded":"2023-11-08T08:19:08.503552+0000","last_fullsized":"2023-11-08T08:19:08.503552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:57:06.487072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517506+0000","last_change":"2023-11-08T08:19:04.025872+0000","last_active":"2023-11-08T08:19:08.517506+0000","last_peered":"2023-11-08T08:19:08.517506+0000","last_clean":"2023-11-08T08:19:08.517506+0000","last_became_active":"2023-11-08T08:19:04.025567+0000","last_became_peered":"2023-11-08T08:19:04.025567+0000","last_unstale":"2023-11-08T08:19:08.517506+0000","last_undegraded":"2023-11-08T08:19:08.517506+0000","last_fullsized":"2023-11-08T08:19:08.517506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:52:24.620964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.994736+0000","last_change":"2023-11-08T08:19:08.523353+0000","last_active":"2023-11-08T08:19:08.994736+0000","last_peered":"2023-11-08T08:19:08.994736+0000","last_clean":"2023-11-08T08:19:08.994736+0000","last_became_active":"2023-11-08T08:19:08.522741+0000","last_became_peered":"2023-11-08T08:19:08.522741+0000","last_unstale":"2023-11-08T08:19:08.994736+0000","last_undegraded":"2023-11-08T08:19:08.994736+0000","last_fullsized":"2023-11-08T08:19:08.994736+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:01:48.615826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512369+0000","last_change":"2023-11-08T08:19:04.512228+0000","last_active":"2023-11-08T08:19:08.512369+0000","last_peered":"2023-11-08T08:19:08.512369+0000","last_clean":"2023-11-08T08:19:08.512369+0000","last_became_active":"2023-11-08T08:19:04.501319+0000","last_became_peered":"2023-11-08T08:19:04.501319+0000","last_unstale":"2023-11-08T08:19:08.512369+0000","last_undegraded":"2023-11-08T08:19:08.512369+0000","last_fullsized":"2023-11-08T08:19:08.512369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:27:36.453040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513674+0000","last_change":"2023-11-08T08:19:04.012809+0000","last_active":"2023-11-08T08:19:08.513674+0000","last_peered":"2023-11-08T08:19:08.513674+0000","last_clean":"2023-11-08T08:19:08.513674+0000","last_became_active":"2023-11-08T08:19:04.012289+0000","last_became_peered":"2023-11-08T08:19:04.012289+0000","last_unstale":"2023-11-08T08:19:08.513674+0000","last_undegraded":"2023-11-08T08:19:08.513674+0000","last_fullsized":"2023-11-08T08:19:08.513674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:52:24.620964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372268+0000","last_change":"2023-11-08T08:19:08.538675+0000","last_active":"2023-11-08T08:19:09.372268+0000","last_peered":"2023-11-08T08:19:09.372268+0000","last_clean":"2023-11-08T08:19:09.372268+0000","last_became_active":"2023-11-08T08:19:08.537804+0000","last_became_peered":"2023-11-08T08:19:08.537804+0000","last_unstale":"2023-11-08T08:19:09.372268+0000","last_undegraded":"2023-11-08T08:19:09.372268+0000","last_fullsized":"2023-11-08T08:19:09.372268+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:32:44.139418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.516288+0000","last_change":"2023-11-08T08:19:04.498523+0000","last_active":"2023-11-08T08:19:08.516288+0000","last_peered":"2023-11-08T08:19:08.516288+0000","last_clean":"2023-11-08T08:19:08.516288+0000","last_became_active":"2023-11-08T08:19:04.497819+0000","last_became_peered":"2023-11-08T08:19:04.497819+0000","last_unstale":"2023-11-08T08:19:08.516288+0000","last_undegraded":"2023-11-08T08:19:08.516288+0000","last_fullsized":"2023-11-08T08:19:08.516288+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:55:43.865351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518299+0000","last_change":"2023-11-08T08:19:04.024906+0000","last_active":"2023-11-08T08:19:08.518299+0000","last_peered":"2023-11-08T08:19:08.518299+0000","last_clean":"2023-11-08T08:19:08.518299+0000","last_became_active":"2023-11-08T08:19:04.024603+0000","last_became_peered":"2023-11-08T08:19:04.024603+0000","last_unstale":"2023-11-08T08:19:08.518299+0000","last_undegraded":"2023-11-08T08:19:08.518299+0000","last_fullsized":"2023-11-08T08:19:08.518299+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:21:09.218618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372046+0000","last_change":"2023-11-08T08:19:08.525624+0000","last_active":"2023-11-08T08:19:09.372046+0000","last_peered":"2023-11-08T08:19:09.372046+0000","last_clean":"2023-11-08T08:19:09.372046+0000","last_became_active":"2023-11-08T08:19:08.524202+0000","last_became_peered":"2023-11-08T08:19:08.524202+0000","last_unstale":"2023-11-08T08:19:09.372046+0000","last_undegraded":"2023-11-08T08:19:09.372046+0000","last_fullsized":"2023-11-08T08:19:09.372046+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:08.797102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513312+0000","last_change":"2023-11-08T08:19:04.511393+0000","last_active":"2023-11-08T08:19:08.513312+0000","last_peered":"2023-11-08T08:19:08.513312+0000","last_clean":"2023-11-08T08:19:08.513312+0000","last_became_active":"2023-11-08T08:19:04.510749+0000","last_became_peered":"2023-11-08T08:19:04.510749+0000","last_unstale":"2023-11-08T08:19:08.513312+0000","last_undegraded":"2023-11-08T08:19:08.513312+0000","last_fullsized":"2023-11-08T08:19:08.513312+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:09:30.686343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518295+0000","last_change":"2023-11-08T08:19:04.022458+0000","last_active":"2023-11-08T08:19:08.518295+0000","last_peered":"2023-11-08T08:19:08.518295+0000","last_clean":"2023-11-08T08:19:08.518295+0000","last_became_active":"2023-11-08T08:19:04.022003+0000","last_became_peered":"2023-11-08T08:19:04.022003+0000","last_unstale":"2023-11-08T08:19:08.518295+0000","last_undegraded":"2023-11-08T08:19:08.518295+0000","last_fullsized":"2023-11-08T08:19:08.518295+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:21:09.218618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.375400+0000","last_change":"2023-11-08T08:19:08.536502+0000","last_active":"2023-11-08T08:19:09.375400+0000","last_peered":"2023-11-08T08:19:09.375400+0000","last_clean":"2023-11-08T08:19:09.375400+0000","last_became_active":"2023-11-08T08:19:08.536112+0000","last_became_peered":"2023-11-08T08:19:08.536112+0000","last_unstale":"2023-11-08T08:19:09.375400+0000","last_undegraded":"2023-11-08T08:19:09.375400+0000","last_fullsized":"2023-11-08T08:19:09.375400+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:26:20.245634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.495202+0000","last_change":"2023-11-08T08:19:04.493387+0000","last_active":"2023-11-08T08:19:08.495202+0000","last_peered":"2023-11-08T08:19:08.495202+0000","last_clean":"2023-11-08T08:19:08.495202+0000","last_became_active":"2023-11-08T08:19:04.493131+0000","last_became_peered":"2023-11-08T08:19:04.493131+0000","last_unstale":"2023-11-08T08:19:08.495202+0000","last_undegraded":"2023-11-08T08:19:08.495202+0000","last_fullsized":"2023-11-08T08:19:08.495202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:56:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519525+0000","last_change":"2023-11-08T08:19:04.021637+0000","last_active":"2023-11-08T08:19:08.519525+0000","last_peered":"2023-11-08T08:19:08.519525+0000","last_clean":"2023-11-08T08:19:08.519525+0000","last_became_active":"2023-11-08T08:19:04.018995+0000","last_became_peered":"2023-11-08T08:19:04.018995+0000","last_unstale":"2023-11-08T08:19:08.519525+0000","last_undegraded":"2023-11-08T08:19:08.519525+0000","last_fullsized":"2023-11-08T08:19:08.519525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:44.433066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.375204+0000","last_change":"2023-11-08T08:19:08.535800+0000","last_active":"2023-11-08T08:19:09.375204+0000","last_peered":"2023-11-08T08:19:09.375204+0000","last_clean":"2023-11-08T08:19:09.375204+0000","last_became_active":"2023-11-08T08:19:08.529039+0000","last_became_peered":"2023-11-08T08:19:08.529039+0000","last_unstale":"2023-11-08T08:19:09.375204+0000","last_undegraded":"2023-11-08T08:19:09.375204+0000","last_fullsized":"2023-11-08T08:19:09.375204+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:55:58.092159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518136+0000","last_change":"2023-11-08T08:19:04.497093+0000","last_active":"2023-11-08T08:19:08.518136+0000","last_peered":"2023-11-08T08:19:08.518136+0000","last_clean":"2023-11-08T08:19:08.518136+0000","last_became_active":"2023-11-08T08:19:04.496619+0000","last_became_peered":"2023-11-08T08:19:04.496619+0000","last_unstale":"2023-11-08T08:19:08.518136+0000","last_undegraded":"2023-11-08T08:19:08.518136+0000","last_fullsized":"2023-11-08T08:19:08.518136+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:27:39.386956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.500739+0000","last_change":"2023-11-08T08:19:04.015072+0000","last_active":"2023-11-08T08:19:08.500739+0000","last_peered":"2023-11-08T08:19:08.500739+0000","last_clean":"2023-11-08T08:19:08.500739+0000","last_became_active":"2023-11-08T08:19:04.014727+0000","last_became_peered":"2023-11-08T08:19:04.014727+0000","last_unstale":"2023-11-08T08:19:08.500739+0000","last_undegraded":"2023-11-08T08:19:08.500739+0000","last_fullsized":"2023-11-08T08:19:08.500739+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:17:33.341483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.375246+0000","last_change":"2023-11-08T08:19:08.534568+0000","last_active":"2023-11-08T08:19:09.375246+0000","last_peered":"2023-11-08T08:19:09.375246+0000","last_clean":"2023-11-08T08:19:09.375246+0000","last_became_active":"2023-11-08T08:19:08.534139+0000","last_became_peered":"2023-11-08T08:19:08.534139+0000","last_unstale":"2023-11-08T08:19:09.375246+0000","last_undegraded":"2023-11-08T08:19:09.375246+0000","last_fullsized":"2023-11-08T08:19:09.375246+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:59:39.090459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513527+0000","last_change":"2023-11-08T08:19:04.512845+0000","last_active":"2023-11-08T08:19:08.513527+0000","last_peered":"2023-11-08T08:19:08.513527+0000","last_clean":"2023-11-08T08:19:08.513527+0000","last_became_active":"2023-11-08T08:19:04.512499+0000","last_became_peered":"2023-11-08T08:19:04.512499+0000","last_unstale":"2023-11-08T08:19:08.513527+0000","last_undegraded":"2023-11-08T08:19:08.513527+0000","last_fullsized":"2023-11-08T08:19:08.513527+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:24:10.475972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501411+0000","last_change":"2023-11-08T08:19:04.018033+0000","last_active":"2023-11-08T08:19:08.501411+0000","last_peered":"2023-11-08T08:19:08.501411+0000","last_clean":"2023-11-08T08:19:08.501411+0000","last_became_active":"2023-11-08T08:19:04.017059+0000","last_became_peered":"2023-11-08T08:19:04.017059+0000","last_unstale":"2023-11-08T08:19:08.501411+0000","last_undegraded":"2023-11-08T08:19:08.501411+0000","last_fullsized":"2023-11-08T08:19:08.501411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:44.433066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.369706+0000","last_change":"2023-11-08T08:19:08.539326+0000","last_active":"2023-11-08T08:19:09.369706+0000","last_peered":"2023-11-08T08:19:09.369706+0000","last_clean":"2023-11-08T08:19:09.369706+0000","last_became_active":"2023-11-08T08:19:08.538846+0000","last_became_peered":"2023-11-08T08:19:08.538846+0000","last_unstale":"2023-11-08T08:19:09.369706+0000","last_undegraded":"2023-11-08T08:19:09.369706+0000","last_fullsized":"2023-11-08T08:19:09.369706+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:33:00.754113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513023+0000","last_change":"2023-11-08T08:19:04.503981+0000","last_active":"2023-11-08T08:19:08.513023+0000","last_peered":"2023-11-08T08:19:08.513023+0000","last_clean":"2023-11-08T08:19:08.513023+0000","last_became_active":"2023-11-08T08:19:04.503651+0000","last_became_peered":"2023-11-08T08:19:04.503651+0000","last_unstale":"2023-11-08T08:19:08.513023+0000","last_undegraded":"2023-11-08T08:19:08.513023+0000","last_fullsized":"2023-11-08T08:19:08.513023+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:55:43.865351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.500289+0000","last_change":"2023-11-08T08:19:04.022166+0000","last_active":"2023-11-08T08:19:08.500289+0000","last_peered":"2023-11-08T08:19:08.500289+0000","last_clean":"2023-11-08T08:19:08.500289+0000","last_became_active":"2023-11-08T08:19:04.017289+0000","last_became_peered":"2023-11-08T08:19:04.017289+0000","last_unstale":"2023-11-08T08:19:08.500289+0000","last_undegraded":"2023-11-08T08:19:08.500289+0000","last_fullsized":"2023-11-08T08:19:08.500289+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:51:22.228427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.995287+0000","last_change":"2023-11-08T08:19:08.534923+0000","last_active":"2023-11-08T08:19:08.995287+0000","last_peered":"2023-11-08T08:19:08.995287+0000","last_clean":"2023-11-08T08:19:08.995287+0000","last_became_active":"2023-11-08T08:19:08.530869+0000","last_became_peered":"2023-11-08T08:19:08.530869+0000","last_unstale":"2023-11-08T08:19:08.995287+0000","last_undegraded":"2023-11-08T08:19:08.995287+0000","last_fullsized":"2023-11-08T08:19:08.995287+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:47:09.660706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372246+0000","last_change":"2023-11-08T08:19:08.541714+0000","last_active":"2023-11-08T08:19:09.372246+0000","last_peered":"2023-11-08T08:19:09.372246+0000","last_clean":"2023-11-08T08:19:09.372246+0000","last_became_active":"2023-11-08T08:19:08.537859+0000","last_became_peered":"2023-11-08T08:19:08.537859+0000","last_unstale":"2023-11-08T08:19:09.372246+0000","last_undegraded":"2023-11-08T08:19:09.372246+0000","last_fullsized":"2023-11-08T08:19:09.372246+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:33:11.313632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518613+0000","last_change":"2023-11-08T08:19:04.008921+0000","last_active":"2023-11-08T08:19:08.518613+0000","last_peered":"2023-11-08T08:19:08.518613+0000","last_clean":"2023-11-08T08:19:08.518613+0000","last_became_active":"2023-11-08T08:19:04.008616+0000","last_became_peered":"2023-11-08T08:19:04.008616+0000","last_unstale":"2023-11-08T08:19:08.518613+0000","last_undegraded":"2023-11-08T08:19:08.518613+0000","last_fullsized":"2023-11-08T08:19:08.518613+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:50:41.342112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.520011+0000","last_change":"2023-11-08T08:19:04.501127+0000","last_active":"2023-11-08T08:19:08.520011+0000","last_peered":"2023-11-08T08:19:08.520011+0000","last_clean":"2023-11-08T08:19:08.520011+0000","last_became_active":"2023-11-08T08:19:04.500705+0000","last_became_peered":"2023-11-08T08:19:04.500705+0000","last_unstale":"2023-11-08T08:19:08.520011+0000","last_undegraded":"2023-11-08T08:19:08.520011+0000","last_fullsized":"2023-11-08T08:19:08.520011+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:25.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372698+0000","last_change":"2023-11-08T08:19:08.533064+0000","last_active":"2023-11-08T08:19:09.372698+0000","last_peered":"2023-11-08T08:19:09.372698+0000","last_clean":"2023-11-08T08:19:09.372698+0000","last_became_active":"2023-11-08T08:19:08.532631+0000","last_became_peered":"2023-11-08T08:19:08.532631+0000","last_unstale":"2023-11-08T08:19:09.372698+0000","last_undegraded":"2023-11-08T08:19:09.372698+0000","last_fullsized":"2023-11-08T08:19:09.372698+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:32:44.139418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501706+0000","last_change":"2023-11-08T08:19:04.017775+0000","last_active":"2023-11-08T08:19:08.501706+0000","last_peered":"2023-11-08T08:19:08.501706+0000","last_clean":"2023-11-08T08:19:08.501706+0000","last_became_active":"2023-11-08T08:19:04.017572+0000","last_became_peered":"2023-11-08T08:19:04.017572+0000","last_unstale":"2023-11-08T08:19:08.501706+0000","last_undegraded":"2023-11-08T08:19:08.501706+0000","last_fullsized":"2023-11-08T08:19:08.501706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:55:07.041721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.497148+0000","last_change":"2023-11-08T08:19:04.488153+0000","last_active":"2023-11-08T08:19:08.497148+0000","last_peered":"2023-11-08T08:19:08.497148+0000","last_clean":"2023-11-08T08:19:08.497148+0000","last_became_active":"2023-11-08T08:19:04.487871+0000","last_became_peered":"2023-11-08T08:19:04.487871+0000","last_unstale":"2023-11-08T08:19:08.497148+0000","last_undegraded":"2023-11-08T08:19:08.497148+0000","last_fullsized":"2023-11-08T08:19:08.497148+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:06:47.395577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.494806+0000","last_change":"2023-11-08T08:19:04.011650+0000","last_active":"2023-11-08T08:19:08.494806+0000","last_peered":"2023-11-08T08:19:08.494806+0000","last_clean":"2023-11-08T08:19:08.494806+0000","last_became_active":"2023-11-08T08:19:04.011071+0000","last_became_peered":"2023-11-08T08:19:04.011071+0000","last_unstale":"2023-11-08T08:19:08.494806+0000","last_undegraded":"2023-11-08T08:19:08.494806+0000","last_fullsized":"2023-11-08T08:19:08.494806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:51:22.228427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519773+0000","last_change":"2023-11-08T08:19:04.505594+0000","last_active":"2023-11-08T08:19:08.519773+0000","last_peered":"2023-11-08T08:19:08.519773+0000","last_clean":"2023-11-08T08:19:08.519773+0000","last_became_active":"2023-11-08T08:19:04.505258+0000","last_became_peered":"2023-11-08T08:19:04.505258+0000","last_unstale":"2023-11-08T08:19:08.519773+0000","last_undegraded":"2023-11-08T08:19:08.519773+0000","last_fullsized":"2023-11-08T08:19:08.519773+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:56:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.373555+0000","last_change":"2023-11-08T08:19:08.529766+0000","last_active":"2023-11-08T08:19:09.373555+0000","last_peered":"2023-11-08T08:19:09.373555+0000","last_clean":"2023-11-08T08:19:09.373555+0000","last_became_active":"2023-11-08T08:19:08.529334+0000","last_became_peered":"2023-11-08T08:19:08.529334+0000","last_unstale":"2023-11-08T08:19:09.373555+0000","last_undegraded":"2023-11-08T08:19:09.373555+0000","last_fullsized":"2023-11-08T08:19:09.373555+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:28:49.369446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501076+0000","last_change":"2023-11-08T08:19:04.020027+0000","last_active":"2023-11-08T08:19:08.501076+0000","last_peered":"2023-11-08T08:19:08.501076+0000","last_clean":"2023-11-08T08:19:08.501076+0000","last_became_active":"2023-11-08T08:19:04.019727+0000","last_became_peered":"2023-11-08T08:19:04.019727+0000","last_unstale":"2023-11-08T08:19:08.501076+0000","last_undegraded":"2023-11-08T08:19:08.501076+0000","last_fullsized":"2023-11-08T08:19:08.501076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:18:14.269118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519487+0000","last_change":"2023-11-08T08:19:04.495283+0000","last_active":"2023-11-08T08:19:08.519487+0000","last_peered":"2023-11-08T08:19:08.519487+0000","last_clean":"2023-11-08T08:19:08.519487+0000","last_became_active":"2023-11-08T08:19:04.494955+0000","last_became_peered":"2023-11-08T08:19:04.494955+0000","last_unstale":"2023-11-08T08:19:08.519487+0000","last_undegraded":"2023-11-08T08:19:08.519487+0000","last_fullsized":"2023-11-08T08:19:08.519487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372061+0000","last_change":"2023-11-08T08:19:08.532374+0000","last_active":"2023-11-08T08:19:09.372061+0000","last_peered":"2023-11-08T08:19:09.372061+0000","last_clean":"2023-11-08T08:19:09.372061+0000","last_became_active":"2023-11-08T08:19:08.531953+0000","last_became_peered":"2023-11-08T08:19:08.531953+0000","last_unstale":"2023-11-08T08:19:09.372061+0000","last_undegraded":"2023-11-08T08:19:09.372061+0000","last_fullsized":"2023-11-08T08:19:09.372061+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:54:33.577798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518515+0000","last_change":"2023-11-08T08:19:04.491042+0000","last_active":"2023-11-08T08:19:08.518515+0000","last_peered":"2023-11-08T08:19:08.518515+0000","last_clean":"2023-11-08T08:19:08.518515+0000","last_became_active":"2023-11-08T08:19:04.490734+0000","last_became_peered":"2023-11-08T08:19:04.490734+0000","last_unstale":"2023-11-08T08:19:08.518515+0000","last_undegraded":"2023-11-08T08:19:08.518515+0000","last_fullsized":"2023-11-08T08:19:08.518515+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:26:27.887614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517935+0000","last_change":"2023-11-08T08:19:04.507871+0000","last_active":"2023-11-08T08:19:08.517935+0000","last_peered":"2023-11-08T08:19:08.517935+0000","last_clean":"2023-11-08T08:19:08.517935+0000","last_became_active":"2023-11-08T08:19:04.507600+0000","last_became_peered":"2023-11-08T08:19:04.507600+0000","last_unstale":"2023-11-08T08:19:08.517935+0000","last_undegraded":"2023-11-08T08:19:08.517935+0000","last_fullsized":"2023-11-08T08:19:08.517935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:55:43.865351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512136+0000","last_change":"2023-11-08T08:19:04.505023+0000","last_active":"2023-11-08T08:19:08.512136+0000","last_peered":"2023-11-08T08:19:08.512136+0000","last_clean":"2023-11-08T08:19:08.512136+0000","last_became_active":"2023-11-08T08:19:04.504750+0000","last_became_peered":"2023-11-08T08:19:04.504750+0000","last_unstale":"2023-11-08T08:19:08.512136+0000","last_undegraded":"2023-11-08T08:19:08.512136+0000","last_fullsized":"2023-11-08T08:19:08.512136+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:25.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518063+0000","last_change":"2023-11-08T08:19:04.504115+0000","last_active":"2023-11-08T08:19:08.518063+0000","last_peered":"2023-11-08T08:19:08.518063+0000","last_clean":"2023-11-08T08:19:08.518063+0000","last_became_active":"2023-11-08T08:19:04.503841+0000","last_became_peered":"2023-11-08T08:19:04.503841+0000","last_unstale":"2023-11-08T08:19:08.518063+0000","last_undegraded":"2023-11-08T08:19:08.518063+0000","last_fullsized":"2023-11-08T08:19:08.518063+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:33:12.861277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503562+0000","last_change":"2023-11-08T08:19:04.504534+0000","last_active":"2023-11-08T08:19:08.503562+0000","last_peered":"2023-11-08T08:19:08.503562+0000","last_clean":"2023-11-08T08:19:08.503562+0000","last_became_active":"2023-11-08T08:19:04.504283+0000","last_became_peered":"2023-11-08T08:19:04.504283+0000","last_unstale":"2023-11-08T08:19:08.503562+0000","last_undegraded":"2023-11-08T08:19:08.503562+0000","last_fullsized":"2023-11-08T08:19:08.503562+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:56:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519721+0000","last_change":"2023-11-08T08:19:04.503978+0000","last_active":"2023-11-08T08:19:08.519721+0000","last_peered":"2023-11-08T08:19:08.519721+0000","last_clean":"2023-11-08T08:19:08.519721+0000","last_became_active":"2023-11-08T08:19:04.503653+0000","last_became_peered":"2023-11-08T08:19:04.503653+0000","last_unstale":"2023-11-08T08:19:08.519721+0000","last_undegraded":"2023-11-08T08:19:08.519721+0000","last_fullsized":"2023-11-08T08:19:08.519721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503754+0000","last_change":"2023-11-08T08:19:04.491309+0000","last_active":"2023-11-08T08:19:08.503754+0000","last_peered":"2023-11-08T08:19:08.503754+0000","last_clean":"2023-11-08T08:19:08.503754+0000","last_became_active":"2023-11-08T08:19:04.491035+0000","last_became_peered":"2023-11-08T08:19:04.491035+0000","last_unstale":"2023-11-08T08:19:08.503754+0000","last_undegraded":"2023-11-08T08:19:08.503754+0000","last_fullsized":"2023-11-08T08:19:08.503754+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:18:36.796288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503455+0000","last_change":"2023-11-08T08:19:04.503092+0000","last_active":"2023-11-08T08:19:08.503455+0000","last_peered":"2023-11-08T08:19:08.503455+0000","last_clean":"2023-11-08T08:19:08.503455+0000","last_became_active":"2023-11-08T08:19:04.502751+0000","last_became_peered":"2023-11-08T08:19:04.502751+0000","last_unstale":"2023-11-08T08:19:08.503455+0000","last_undegraded":"2023-11-08T08:19:08.503455+0000","last_fullsized":"2023-11-08T08:19:08.503455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:40:44.425321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.499633+0000","last_change":"2023-11-08T08:19:04.509538+0000","last_active":"2023-11-08T08:19:08.499633+0000","last_peered":"2023-11-08T08:19:08.499633+0000","last_clean":"2023-11-08T08:19:08.499633+0000","last_became_active":"2023-11-08T08:19:04.509190+0000","last_became_peered":"2023-11-08T08:19:04.509190+0000","last_unstale":"2023-11-08T08:19:08.499633+0000","last_undegraded":"2023-11-08T08:19:08.499633+0000","last_fullsized":"2023-11-08T08:19:08.499633+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:04:55.302575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517870+0000","last_change":"2023-11-08T08:19:04.489238+0000","last_active":"2023-11-08T08:19:08.517870+0000","last_peered":"2023-11-08T08:19:08.517870+0000","last_clean":"2023-11-08T08:19:08.517870+0000","last_became_active":"2023-11-08T08:19:04.488998+0000","last_became_peered":"2023-11-08T08:19:04.488998+0000","last_unstale":"2023-11-08T08:19:08.517870+0000","last_undegraded":"2023-11-08T08:19:08.517870+0000","last_fullsized":"2023-11-08T08:19:08.517870+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:27:36.453040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.494671+0000","last_change":"2023-11-08T08:19:04.499236+0000","last_active":"2023-11-08T08:19:08.494671+0000","last_peered":"2023-11-08T08:19:08.494671+0000","last_clean":"2023-11-08T08:19:08.494671+0000","last_became_active":"2023-11-08T08:19:04.498978+0000","last_became_peered":"2023-11-08T08:19:04.498978+0000","last_unstale":"2023-11-08T08:19:08.494671+0000","last_undegraded":"2023-11-08T08:19:08.494671+0000","last_fullsized":"2023-11-08T08:19:08.494671+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:04:55.302575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.500995+0000","last_change":"2023-11-08T08:19:04.513191+0000","last_active":"2023-11-08T08:19:08.500995+0000","last_peered":"2023-11-08T08:19:08.500995+0000","last_clean":"2023-11-08T08:19:08.500995+0000","last_became_active":"2023-11-08T08:19:04.512519+0000","last_became_peered":"2023-11-08T08:19:04.512519+0000","last_unstale":"2023-11-08T08:19:08.500995+0000","last_undegraded":"2023-11-08T08:19:08.500995+0000","last_fullsized":"2023-11-08T08:19:08.500995+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:56:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513262+0000","last_change":"2023-11-08T08:19:04.510593+0000","last_active":"2023-11-08T08:19:08.513262+0000","last_peered":"2023-11-08T08:19:08.513262+0000","last_clean":"2023-11-08T08:19:08.513262+0000","last_became_active":"2023-11-08T08:19:04.501361+0000","last_became_peered":"2023-11-08T08:19:04.501361+0000","last_unstale":"2023-11-08T08:19:08.513262+0000","last_undegraded":"2023-11-08T08:19:08.513262+0000","last_fullsized":"2023-11-08T08:19:08.513262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.509488+0000","last_change":"2023-11-08T08:19:04.497954+0000","last_active":"2023-11-08T08:19:08.509488+0000","last_peered":"2023-11-08T08:19:08.509488+0000","last_clean":"2023-11-08T08:19:08.509488+0000","last_became_active":"2023-11-08T08:19:04.497675+0000","last_became_peered":"2023-11-08T08:19:04.497675+0000","last_unstale":"2023-11-08T08:19:08.509488+0000","last_undegraded":"2023-11-08T08:19:08.509488+0000","last_fullsized":"2023-11-08T08:19:08.509488+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:07:36.146057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512026+0000","last_change":"2023-11-08T08:19:04.494551+0000","last_active":"2023-11-08T08:19:08.512026+0000","last_peered":"2023-11-08T08:19:08.512026+0000","last_clean":"2023-11-08T08:19:08.512026+0000","last_became_active":"2023-11-08T08:19:04.494261+0000","last_became_peered":"2023-11-08T08:19:04.494261+0000","last_unstale":"2023-11-08T08:19:08.512026+0000","last_undegraded":"2023-11-08T08:19:08.512026+0000","last_fullsized":"2023-11-08T08:19:08.512026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:27:40.872420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.496353+0000","last_change":"2023-11-08T08:19:04.486999+0000","last_active":"2023-11-08T08:19:08.496353+0000","last_peered":"2023-11-08T08:19:08.496353+0000","last_clean":"2023-11-08T08:19:08.496353+0000","last_became_active":"2023-11-08T08:19:04.486746+0000","last_became_peered":"2023-11-08T08:19:04.486746+0000","last_unstale":"2023-11-08T08:19:08.496353+0000","last_undegraded":"2023-11-08T08:19:08.496353+0000","last_fullsized":"2023-11-08T08:19:08.496353+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:27:36.453040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513306+0000","last_change":"2023-11-08T08:19:04.506263+0000","last_active":"2023-11-08T08:19:08.513306+0000","last_peered":"2023-11-08T08:19:08.513306+0000","last_clean":"2023-11-08T08:19:08.513306+0000","last_became_active":"2023-11-08T08:19:04.505930+0000","last_became_peered":"2023-11-08T08:19:04.505930+0000","last_unstale":"2023-11-08T08:19:08.513306+0000","last_undegraded":"2023-11-08T08:19:08.513306+0000","last_fullsized":"2023-11-08T08:19:08.513306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:41:09.071125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501581+0000","last_change":"2023-11-08T08:19:04.514677+0000","last_active":"2023-11-08T08:19:08.501581+0000","last_peered":"2023-11-08T08:19:08.501581+0000","last_clean":"2023-11-08T08:19:08.501581+0000","last_became_active":"2023-11-08T08:19:04.511465+0000","last_became_peered":"2023-11-08T08:19:04.511465+0000","last_unstale":"2023-11-08T08:19:08.501581+0000","last_undegraded":"2023-11-08T08:19:08.501581+0000","last_fullsized":"2023-11-08T08:19:08.501581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513976+0000","last_change":"2023-11-08T08:19:04.490773+0000","last_active":"2023-11-08T08:19:08.513976+0000","last_peered":"2023-11-08T08:19:08.513976+0000","last_clean":"2023-11-08T08:19:08.513976+0000","last_became_active":"2023-11-08T08:19:04.490474+0000","last_became_peered":"2023-11-08T08:19:04.490474+0000","last_unstale":"2023-11-08T08:19:08.513976+0000","last_undegraded":"2023-11-08T08:19:08.513976+0000","last_fullsized":"2023-11-08T08:19:08.513976+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512799+0000","last_change":"2023-11-08T08:19:04.487159+0000","last_active":"2023-11-08T08:19:08.512799+0000","last_peered":"2023-11-08T08:19:08.512799+0000","last_clean":"2023-11-08T08:19:08.512799+0000","last_became_active":"2023-11-08T08:19:04.486861+0000","last_became_peered":"2023-11-08T08:19:04.486861+0000","last_unstale":"2023-11-08T08:19:08.512799+0000","last_undegraded":"2023-11-08T08:19:08.512799+0000","last_fullsized":"2023-11-08T08:19:08.512799+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:45.446278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.509301+0000","last_change":"2023-11-08T08:19:04.491757+0000","last_active":"2023-11-08T08:19:08.509301+0000","last_peered":"2023-11-08T08:19:08.509301+0000","last_clean":"2023-11-08T08:19:08.509301+0000","last_became_active":"2023-11-08T08:19:04.491359+0000","last_became_peered":"2023-11-08T08:19:04.491359+0000","last_unstale":"2023-11-08T08:19:08.509301+0000","last_undegraded":"2023-11-08T08:19:08.509301+0000","last_fullsized":"2023-11-08T08:19:08.509301+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:01:42.294189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.516256+0000","last_change":"2023-11-08T08:19:04.498748+0000","last_active":"2023-11-08T08:19:08.516256+0000","last_peered":"2023-11-08T08:19:08.516256+0000","last_clean":"2023-11-08T08:19:08.516256+0000","last_became_active":"2023-11-08T08:19:04.498208+0000","last_became_peered":"2023-11-08T08:19:04.498208+0000","last_unstale":"2023-11-08T08:19:08.516256+0000","last_undegraded":"2023-11-08T08:19:08.516256+0000","last_fullsized":"2023-11-08T08:19:08.516256+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:33:12.861277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521468+0000","last_change":"2023-11-08T08:19:04.490052+0000","last_active":"2023-11-08T08:19:08.521468+0000","last_peered":"2023-11-08T08:19:08.521468+0000","last_clean":"2023-11-08T08:19:08.521468+0000","last_became_active":"2023-11-08T08:19:04.489758+0000","last_became_peered":"2023-11-08T08:19:04.489758+0000","last_unstale":"2023-11-08T08:19:08.521468+0000","last_undegraded":"2023-11-08T08:19:08.521468+0000","last_fullsized":"2023-11-08T08:19:08.521468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:25.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.495092+0000","last_change":"2023-11-08T08:19:04.489823+0000","last_active":"2023-11-08T08:19:08.495092+0000","last_peered":"2023-11-08T08:19:08.495092+0000","last_clean":"2023-11-08T08:19:08.495092+0000","last_became_active":"2023-11-08T08:19:04.489559+0000","last_became_peered":"2023-11-08T08:19:04.489559+0000","last_unstale":"2023-11-08T08:19:08.495092+0000","last_undegraded":"2023-11-08T08:19:08.495092+0000","last_fullsized":"2023-11-08T08:19:08.495092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:25.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512940+0000","last_change":"2023-11-08T08:19:04.509192+0000","last_active":"2023-11-08T08:19:08.512940+0000","last_peered":"2023-11-08T08:19:08.512940+0000","last_clean":"2023-11-08T08:19:08.512940+0000","last_became_active":"2023-11-08T08:19:04.508856+0000","last_became_peered":"2023-11-08T08:19:04.508856+0000","last_unstale":"2023-11-08T08:19:08.512940+0000","last_undegraded":"2023-11-08T08:19:08.512940+0000","last_fullsized":"2023-11-08T08:19:08.512940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:27:36.453040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501455+0000","last_change":"2023-11-08T08:19:04.510274+0000","last_active":"2023-11-08T08:19:08.501455+0000","last_peered":"2023-11-08T08:19:08.501455+0000","last_clean":"2023-11-08T08:19:08.501455+0000","last_became_active":"2023-11-08T08:19:04.510052+0000","last_became_peered":"2023-11-08T08:19:04.510052+0000","last_unstale":"2023-11-08T08:19:08.501455+0000","last_undegraded":"2023-11-08T08:19:08.501455+0000","last_fullsized":"2023-11-08T08:19:08.501455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.511920+0000","last_change":"2023-11-08T08:19:04.507388+0000","last_active":"2023-11-08T08:19:08.511920+0000","last_peered":"2023-11-08T08:19:08.511920+0000","last_clean":"2023-11-08T08:19:08.511920+0000","last_became_active":"2023-11-08T08:19:04.506655+0000","last_became_peered":"2023-11-08T08:19:04.506655+0000","last_unstale":"2023-11-08T08:19:08.511920+0000","last_undegraded":"2023-11-08T08:19:08.511920+0000","last_fullsized":"2023-11-08T08:19:08.511920+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:26:16.737280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.495536+0000","last_change":"2023-11-08T08:19:04.484402+0000","last_active":"2023-11-08T08:19:08.495536+0000","last_peered":"2023-11-08T08:19:08.495536+0000","last_clean":"2023-11-08T08:19:08.495536+0000","last_became_active":"2023-11-08T08:19:04.480030+0000","last_became_peered":"2023-11-08T08:19:04.480030+0000","last_unstale":"2023-11-08T08:19:08.495536+0000","last_undegraded":"2023-11-08T08:19:08.495536+0000","last_fullsized":"2023-11-08T08:19:08.495536+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:27:40.872420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517912+0000","last_change":"2023-11-08T08:19:04.500239+0000","last_active":"2023-11-08T08:19:08.517912+0000","last_peered":"2023-11-08T08:19:08.517912+0000","last_clean":"2023-11-08T08:19:08.517912+0000","last_became_active":"2023-11-08T08:19:04.499933+0000","last_became_peered":"2023-11-08T08:19:04.499933+0000","last_unstale":"2023-11-08T08:19:08.517912+0000","last_undegraded":"2023-11-08T08:19:08.517912+0000","last_fullsized":"2023-11-08T08:19:08.517912+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:45.446278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.516154+0000","last_change":"2023-11-08T08:19:04.507285+0000","last_active":"2023-11-08T08:19:08.516154+0000","last_peered":"2023-11-08T08:19:08.516154+0000","last_clean":"2023-11-08T08:19:08.516154+0000","last_became_active":"2023-11-08T08:19:04.506999+0000","last_became_peered":"2023-11-08T08:19:04.506999+0000","last_unstale":"2023-11-08T08:19:08.516154+0000","last_undegraded":"2023-11-08T08:19:08.516154+0000","last_fullsized":"2023-11-08T08:19:08.516154+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:55:54.583805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521418+0000","last_change":"2023-11-08T08:19:04.496585+0000","last_active":"2023-11-08T08:19:08.521418+0000","last_peered":"2023-11-08T08:19:08.521418+0000","last_clean":"2023-11-08T08:19:08.521418+0000","last_became_active":"2023-11-08T08:19:04.496307+0000","last_became_peered":"2023-11-08T08:19:04.496307+0000","last_unstale":"2023-11-08T08:19:08.521418+0000","last_undegraded":"2023-11-08T08:19:08.521418+0000","last_fullsized":"2023-11-08T08:19:08.521418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:06:47.395577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.509596+0000","last_change":"2023-11-08T08:19:04.487836+0000","last_active":"2023-11-08T08:19:08.509596+0000","last_peered":"2023-11-08T08:19:08.509596+0000","last_clean":"2023-11-08T08:19:08.509596+0000","last_became_active":"2023-11-08T08:19:04.487548+0000","last_became_peered":"2023-11-08T08:19:04.487548+0000","last_unstale":"2023-11-08T08:19:08.509596+0000","last_undegraded":"2023-11-08T08:19:08.509596+0000","last_fullsized":"2023-11-08T08:19:08.509596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:55:08.054932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4527,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9296,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362544,"statfs":{"total":96636764160,"available":96627245056,"internally_reserved":0,"allocated":925696,"data_stored":718901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5029999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.282}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5409999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.581}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8176,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363664,"statfs":{"total":96636764160,"available":96628391936,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.194}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.456}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8184,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363656,"statfs":{"total":96636764160,"available":96628383744,"internally_reserved":0,"allocated":311296,"data_stored":125208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5089999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.569}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.425}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.952}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8260,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363580,"statfs":{"total":96636764160,"available":96628305920,"internally_reserved":0,"allocated":323584,"data_stored":128373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.522}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.496}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8244,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363596,"statfs":{"total":96636764160,"available":96628322304,"internally_reserved":0,"allocated":307200,"data_stored":124662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.498}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8112,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363728,"statfs":{"total":96636764160,"available":96628457472,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.194}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.456}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8240,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363600,"statfs":{"total":96636764160,"available":96628326400,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.659}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8089999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.615}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2320000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9272,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362568,"statfs":{"total":96636764160,"available":96627269632,"internally_reserved":0,"allocated":901120,"data_stored":715506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T08:19:32.843 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T08:19:32.844 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-08T08:19:32.844 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-08T08:19:32.844 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-08T08:19:32.845 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-08T08:19:33.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.066+0000 7f6955cdd700 1 Processor -- start 2023-11-08T08:19:33.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.066+0000 7f6955cdd700 1 -- start start 2023-11-08T08:19:33.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.066+0000 7f6955cdd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6950069e00 0x7f695006a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:33.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.066+0000 7f6955cdd700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f695006c600 con 0x7f6950071aa0 2023-11-08T08:19:33.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.066+0000 7f6955cdd700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69500722f0 con 0x7f695015c640 2023-11-08T08:19:33.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.066+0000 7f6955cdd700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f695006cef0 con 0x7f6950069e00 2023-11-08T08:19:33.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.066+0000 7f694ffff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6950071aa0 0x7f695006d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49668/0 (socket says 172.21.15.40:49668) 2023-11-08T08:19:33.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.066+0000 7f694ffff700 1 -- 172.21.15.40:0/2065073225 learned_addr learned my addr 172.21.15.40:0/2065073225 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:33.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.066+0000 7f69554dc700 1 --2- 172.21.15.40:0/2065073225 >> v2:172.21.15.40:3300/0 conn(0x7f6950069e00 0x7f695006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:33.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.066+0000 7f694f7fe700 1 -- 172.21.15.40:0/2065073225 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592814644 0 0) 0x7f69500722f0 con 0x7f695015c640 2023-11-08T08:19:33.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.066+0000 7f694f7fe700 1 -- 172.21.15.40:0/2065073225 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69400035d0 con 0x7f695015c640 2023-11-08T08:19:33.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.066+0000 7f694f7fe700 1 -- 172.21.15.40:0/2065073225 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801076275 0 0) 0x7f695006c600 con 0x7f6950071aa0 2023-11-08T08:19:33.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.066+0000 7f694f7fe700 1 -- 172.21.15.40:0/2065073225 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69500722f0 con 0x7f6950071aa0 2023-11-08T08:19:33.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.066+0000 7f69554dc700 1 -- 172.21.15.40:0/2065073225 >> v1:172.21.15.169:6789/0 conn(0x7f695015c640 legacy=0x7f69500d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:33.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.066+0000 7f69554dc700 1 -- 172.21.15.40:0/2065073225 >> v1:172.21.15.40:6789/0 conn(0x7f6950071aa0 legacy=0x7f695006d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:33.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.066+0000 7f69554dc700 1 -- 172.21.15.40:0/2065073225 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695006da20 con 0x7f6950069e00 2023-11-08T08:19:33.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.070+0000 7f69554dc700 1 --2- 172.21.15.40:0/2065073225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6950069e00 0x7f695006a280 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6948009d90 tx=0x7f694800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25191b5d92436ae1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:33.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.070+0000 7f694f7fe700 1 -- 172.21.15.40:0/2065073225 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694800e070 con 0x7f6950069e00 2023-11-08T08:19:33.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.070+0000 7f694f7fe700 1 -- 172.21.15.40:0/2065073225 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694800bb20 con 0x7f6950069e00 2023-11-08T08:19:33.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.070+0000 7f694f7fe700 1 -- 172.21.15.40:0/2065073225 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6948012430 con 0x7f6950069e00 2023-11-08T08:19:33.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.070+0000 7f6955cdd700 1 -- 172.21.15.40:0/2065073225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6950069e00 msgr2=0x7f695006a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:33.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.070+0000 7f6955cdd700 1 --2- 172.21.15.40:0/2065073225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6950069e00 0x7f695006a280 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6948009d90 tx=0x7f694800b330 comp rx=0 tx=0).stop 2023-11-08T08:19:33.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.070+0000 7f6955cdd700 1 -- 172.21.15.40:0/2065073225 shutdown_connections 2023-11-08T08:19:33.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.070+0000 7f6955cdd700 1 --2- 172.21.15.40:0/2065073225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6950069e00 0x7f695006a280 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:33.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.070+0000 7f6955cdd700 1 -- 172.21.15.40:0/2065073225 >> 172.21.15.40:0/2065073225 conn(0x7f69500b6520 msgr2=0x7f6950071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:33.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.070+0000 7f6955cdd700 1 -- 172.21.15.40:0/2065073225 shutdown_connections 2023-11-08T08:19:33.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.070+0000 7f6955cdd700 1 -- 172.21.15.40:0/2065073225 wait complete. 2023-11-08T08:19:33.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.074+0000 7f6955cdd700 1 Processor -- start 2023-11-08T08:19:33.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.074+0000 7f6955cdd700 1 -- start start 2023-11-08T08:19:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.074+0000 7f6955cdd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6950069e00 0x7f695009bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.074+0000 7f6955cdd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6950071aa0 0x7f69501f1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.074+0000 7f6955cdd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f695015c640 0x7f69500c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.074+0000 7f6955cdd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69500d38f0 con 0x7f6950069e00 2023-11-08T08:19:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.074+0000 7f6955cdd700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69500ceb70 con 0x7f6950071aa0 2023-11-08T08:19:33.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.074+0000 7f6955cdd700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69501f1eb0 con 0x7f695015c640 2023-11-08T08:19:33.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.074+0000 7f6954cdb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6950069e00 0x7f695009bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:33.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.074+0000 7f6954cdb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6950069e00 0x7f695009bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52210/0 (socket says 172.21.15.40:52210) 2023-11-08T08:19:33.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.074+0000 7f6954cdb700 1 -- 172.21.15.40:0/312631503 learned_addr learned my addr 172.21.15.40:0/312631503 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:33.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.074+0000 7f694ffff700 1 --2- 172.21.15.40:0/312631503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6950071aa0 0x7f69501f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:33.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.074+0000 7f69554dc700 1 --2- 172.21.15.40:0/312631503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f695015c640 0x7f69500c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:33.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.074+0000 7f6954cdb700 1 -- 172.21.15.40:0/312631503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f695015c640 msgr2=0x7f69500c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:33.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.074+0000 7f6954cdb700 1 --2- 172.21.15.40:0/312631503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f695015c640 0x7f69500c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:33.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.074+0000 7f6954cdb700 1 -- 172.21.15.40:0/312631503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6950071aa0 msgr2=0x7f69501f1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:33.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.074+0000 7f6954cdb700 1 --2- 172.21.15.40:0/312631503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6950071aa0 0x7f69501f1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:33.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.074+0000 7f6954cdb700 1 -- 172.21.15.40:0/312631503 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6948009a70 con 0x7f6950069e00 2023-11-08T08:19:33.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.078+0000 7f6954cdb700 1 --2- 172.21.15.40:0/312631503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6950069e00 0x7f695009bb30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f69440010e0 tx=0x7f694400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:33.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.078+0000 7f694d7fa700 1 -- 172.21.15.40:0/312631503 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69440052d0 con 0x7f6950069e00 2023-11-08T08:19:33.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.078+0000 7f694d7fa700 1 -- 172.21.15.40:0/312631503 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6944005430 con 0x7f6950069e00 2023-11-08T08:19:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.078+0000 7f6955cdd700 1 -- 172.21.15.40:0/312631503 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6950072010 con 0x7f6950069e00 2023-11-08T08:19:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.078+0000 7f694d7fa700 1 -- 172.21.15.40:0/312631503 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69440055a0 con 0x7f6950069e00 2023-11-08T08:19:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.078+0000 7f6955cdd700 1 -- 172.21.15.40:0/312631503 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f695015ca10 con 0x7f6950069e00 2023-11-08T08:19:33.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.078+0000 7f6955cdd700 1 -- 172.21.15.40:0/312631503 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f695006c830 con 0x7f6950069e00 2023-11-08T08:19:33.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.082+0000 7f694d7fa700 1 -- 172.21.15.40:0/312631503 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f6944019030 con 0x7f6950069e00 2023-11-08T08:19:33.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.082+0000 7f694d7fa700 1 --2- 172.21.15.40:0/312631503 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6934064f10 0x7f69340673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:33.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.082+0000 7f694d7fa700 1 -- 172.21.15.40:0/312631503 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6944081630 con 0x7f6950069e00 2023-11-08T08:19:33.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.082+0000 7f694ffff700 1 --2- 172.21.15.40:0/312631503 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6934064f10 0x7f69340673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:33.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.082+0000 7f694ffff700 1 --2- 172.21.15.40:0/312631503 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6934064f10 0x7f69340673c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f69500b5100 tx=0x7f693c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:33.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.086+0000 7f694d7fa700 1 -- 172.21.15.40:0/312631503 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f694404f3f0 con 0x7f6950069e00 2023-11-08T08:19:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.298+0000 7f6955cdd700 1 -- 172.21.15.40:0/312631503 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f6950073cb0 con 0x7f6950069e00 2023-11-08T08:19:33.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.306+0000 7f694d7fa700 1 -- 172.21.15.40:0/312631503 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+161636 (secure 0 0 0) 0x7f6944017090 con 0x7f6950069e00 2023-11-08T08:19:33.311 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:33.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.310+0000 7f6955cdd700 1 -- 172.21.15.40:0/312631503 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6934064f10 msgr2=0x7f69340673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:33.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.310+0000 7f6955cdd700 1 --2- 172.21.15.40:0/312631503 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6934064f10 0x7f69340673c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f69500b5100 tx=0x7f693c009040 comp rx=0 tx=0).stop 2023-11-08T08:19:33.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.310+0000 7f6955cdd700 1 -- 172.21.15.40:0/312631503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6950069e00 msgr2=0x7f695009bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:33.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.310+0000 7f6955cdd700 1 --2- 172.21.15.40:0/312631503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6950069e00 0x7f695009bb30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f69440010e0 tx=0x7f694400ca90 comp rx=0 tx=0).stop 2023-11-08T08:19:33.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.310+0000 7f6955cdd700 1 -- 172.21.15.40:0/312631503 shutdown_connections 2023-11-08T08:19:33.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.310+0000 7f6955cdd700 1 --2- 172.21.15.40:0/312631503 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6934064f10 0x7f69340673c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:33.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.310+0000 7f6955cdd700 1 --2- 172.21.15.40:0/312631503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f695015c640 0x7f69500c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:33.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.310+0000 7f6955cdd700 1 --2- 172.21.15.40:0/312631503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6950071aa0 0x7f69501f1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:33.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.310+0000 7f6955cdd700 1 --2- 172.21.15.40:0/312631503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6950069e00 0x7f695009bb30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:33.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.310+0000 7f6955cdd700 1 -- 172.21.15.40:0/312631503 >> 172.21.15.40:0/312631503 conn(0x7f69500b6520 msgr2=0x7f69500b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:33.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.310+0000 7f6955cdd700 1 -- 172.21.15.40:0/312631503 shutdown_connections 2023-11-08T08:19:33.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.310+0000 7f6955cdd700 1 -- 172.21.15.40:0/312631503 wait complete. 2023-11-08T08:19:33.337 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":5,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":35452},{"type":"v1","addr":"172.21.15.169:6833","nonce":35452}]},"active_addr":"172.21.15.169:6833/35452","active_change":"2023-11-08T08:18:43.856667+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.169:0","nonce":1897689952}]},{"addrvec":[{"type":"v2","addr":"172.21.15.169:0","nonce":1933920566}]},{"addrvec":[{"type":"v2","addr":"172.21.15.169:0","nonce":2460949473}]},{"addrvec":[{"type":"v2","addr":"172.21.15.169:0","nonce":383947712}]}]} 2023-11-08T08:19:33.341 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-08T08:19:33.341 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-08T08:19:33.341 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:19:33.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.518+0000 7f130e66f700 1 Processor -- start 2023-11-08T08:19:33.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.522+0000 7f130e66f700 1 -- start start 2023-11-08T08:19:33.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.522+0000 7f130e66f700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1308070080 0x7f13080704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:33.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.522+0000 7f130e66f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1308164250 0x7f13081646d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:33.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.522+0000 7f130e66f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13080ad6c0 con 0x7f1308160710 2023-11-08T08:19:33.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.522+0000 7f130e66f700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f130806b1c0 con 0x7f1308070080 2023-11-08T08:19:33.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.522+0000 7f130e66f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13080adc60 con 0x7f1308164250 2023-11-08T08:19:33.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.522+0000 7f130ce6c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1308070080 0x7f13080704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:33.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.522+0000 7f130d66d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1308160710 0x7f13080cc320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49672/0 (socket says 172.21.15.40:49672) 2023-11-08T08:19:33.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.522+0000 7f130d66d700 1 -- 172.21.15.40:0/3780280136 learned_addr learned my addr 172.21.15.40:0/3780280136 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:33.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.522+0000 7f130de6e700 1 --2- 172.21.15.40:0/3780280136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1308164250 0x7f13081646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:33.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.522+0000 7f12fffff700 1 -- 172.21.15.40:0/3780280136 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 716962453 0 0) 0x7f13080ad6c0 con 0x7f1308160710 2023-11-08T08:19:33.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.522+0000 7f12fffff700 1 -- 172.21.15.40:0/3780280136 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12f40035d0 con 0x7f1308160710 2023-11-08T08:19:33.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.526+0000 7f130ce6c700 1 -- 172.21.15.40:0/3780280136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1308164250 msgr2=0x7f13081646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:33.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.526+0000 7f130ce6c700 1 --2- 172.21.15.40:0/3780280136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1308164250 0x7f13081646d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:33.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.526+0000 7f130ce6c700 1 -- 172.21.15.40:0/3780280136 >> v1:172.21.15.40:6789/0 conn(0x7f1308160710 legacy=0x7f13080cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:33.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.526+0000 7f130ce6c700 1 -- 172.21.15.40:0/3780280136 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13080709f0 con 0x7f1308070080 2023-11-08T08:19:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.526+0000 7f130ce6c700 1 --2- 172.21.15.40:0/3780280136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1308070080 0x7f13080704b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009850 tx=0x7f12f8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad9ecfeaf4501a36 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.526+0000 7f12fffff700 1 -- 172.21.15.40:0/3780280136 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8011830 con 0x7f1308070080 2023-11-08T08:19:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.526+0000 7f12fffff700 1 -- 172.21.15.40:0/3780280136 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f8011990 con 0x7f1308070080 2023-11-08T08:19:33.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.526+0000 7f12fffff700 1 -- 172.21.15.40:0/3780280136 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8011d40 con 0x7f1308070080 2023-11-08T08:19:33.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.526+0000 7f130e66f700 1 -- 172.21.15.40:0/3780280136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1308070080 msgr2=0x7f13080704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:33.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.526+0000 7f130e66f700 1 --2- 172.21.15.40:0/3780280136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1308070080 0x7f13080704b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009850 tx=0x7f12f8009b60 comp rx=0 tx=0).stop 2023-11-08T08:19:33.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.526+0000 7f130e66f700 1 -- 172.21.15.40:0/3780280136 shutdown_connections 2023-11-08T08:19:33.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.526+0000 7f130e66f700 1 --2- 172.21.15.40:0/3780280136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1308164250 0x7f13081646d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.526+0000 7f130e66f700 1 --2- 172.21.15.40:0/3780280136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1308070080 0x7f13080704b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.526+0000 7f130e66f700 1 -- 172.21.15.40:0/3780280136 >> 172.21.15.40:0/3780280136 conn(0x7f13080b0f80 msgr2=0x7f13080ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.526+0000 7f130e66f700 1 -- 172.21.15.40:0/3780280136 shutdown_connections 2023-11-08T08:19:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.526+0000 7f130e66f700 1 -- 172.21.15.40:0/3780280136 wait complete. 2023-11-08T08:19:33.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.530+0000 7f130e66f700 1 Processor -- start 2023-11-08T08:19:33.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.530+0000 7f130e66f700 1 -- start start 2023-11-08T08:19:33.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.530+0000 7f130e66f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1308070080 0x7f130815b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:33.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.530+0000 7f130e66f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1308160710 0x7f13080c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:33.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.530+0000 7f130e66f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1308164250 0x7f13081613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:33.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.530+0000 7f130e66f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130815b9d0 con 0x7f1308070080 2023-11-08T08:19:33.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.530+0000 7f130e66f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13080c0820 con 0x7f1308160710 2023-11-08T08:19:33.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.530+0000 7f130e66f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13080c0b00 con 0x7f1308164250 2023-11-08T08:19:33.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.530+0000 7f130d66d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1308070080 0x7f130815b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:33.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.530+0000 7f130d66d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1308070080 0x7f130815b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52212/0 (socket says 172.21.15.40:52212) 2023-11-08T08:19:33.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.530+0000 7f130d66d700 1 -- 172.21.15.40:0/177167751 learned_addr learned my addr 172.21.15.40:0/177167751 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:33.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.530+0000 7f130ce6c700 1 --2- 172.21.15.40:0/177167751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1308160710 0x7f13080c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:33.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.530+0000 7f130de6e700 1 --2- 172.21.15.40:0/177167751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1308164250 0x7f13081613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:33.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.530+0000 7f130d66d700 1 -- 172.21.15.40:0/177167751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1308164250 msgr2=0x7f13081613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:33.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.530+0000 7f130d66d700 1 --2- 172.21.15.40:0/177167751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1308164250 0x7f13081613c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:33.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.530+0000 7f130d66d700 1 -- 172.21.15.40:0/177167751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1308160710 msgr2=0x7f13080c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:33.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.530+0000 7f130d66d700 1 --2- 172.21.15.40:0/177167751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1308160710 0x7f13080c2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:33.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.530+0000 7f130d66d700 1 -- 172.21.15.40:0/177167751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f80094b0 con 0x7f1308070080 2023-11-08T08:19:33.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.530+0000 7f130ce6c700 1 --2- 172.21.15.40:0/177167751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1308160710 0x7f13080c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:33.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.530+0000 7f130d66d700 1 --2- 172.21.15.40:0/177167751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1308070080 0x7f130815b490 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f130400c700 tx=0x7f130400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:33.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.534+0000 7f12fdffb700 1 -- 172.21.15.40:0/177167751 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1304005d90 con 0x7f1308070080 2023-11-08T08:19:33.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.534+0000 7f12fdffb700 1 -- 172.21.15.40:0/177167751 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13040070d0 con 0x7f1308070080 2023-11-08T08:19:33.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.534+0000 7f130e66f700 1 -- 172.21.15.40:0/177167751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130806b3d0 con 0x7f1308070080 2023-11-08T08:19:33.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.534+0000 7f12fdffb700 1 -- 172.21.15.40:0/177167751 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1304007480 con 0x7f1308070080 2023-11-08T08:19:33.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.534+0000 7f130e66f700 1 -- 172.21.15.40:0/177167751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13080cca30 con 0x7f1308070080 2023-11-08T08:19:33.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.534+0000 7f130e66f700 1 -- 172.21.15.40:0/177167751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f130806b6b0 con 0x7f1308070080 2023-11-08T08:19:33.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.534+0000 7f12fdffb700 1 -- 172.21.15.40:0/177167751 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1304019030 con 0x7f1308070080 2023-11-08T08:19:33.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.538+0000 7f12fdffb700 1 --2- 172.21.15.40:0/177167751 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f12ec064ee0 0x7f12ec067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:33.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.538+0000 7f12fdffb700 1 -- 172.21.15.40:0/177167751 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1304081570 con 0x7f1308070080 2023-11-08T08:19:33.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.538+0000 7f130ce6c700 1 --2- 172.21.15.40:0/177167751 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f12ec064ee0 0x7f12ec067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:33.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.538+0000 7f130ce6c700 1 --2- 172.21.15.40:0/177167751 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f12ec064ee0 0x7f12ec067390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f12f80090b0 tx=0x7f12f8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:33.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.546+0000 7f12fdffb700 1 -- 172.21.15.40:0/177167751 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f130404f330 con 0x7f1308070080 2023-11-08T08:19:33.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.718+0000 7f130e66f700 1 -- 172.21.15.40:0/177167751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1308073cb0 con 0x7f1308070080 2023-11-08T08:19:33.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.722+0000 7f12fdffb700 1 -- 172.21.15.40:0/177167751 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19652 (secure 0 0 0) 0x7f1304017090 con 0x7f1308070080 2023-11-08T08:19:33.727 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:33.728 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":28,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:08.980158+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:19:33.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.726+0000 7f130e66f700 1 -- 172.21.15.40:0/177167751 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f12ec064ee0 msgr2=0x7f12ec067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:33.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.726+0000 7f130e66f700 1 --2- 172.21.15.40:0/177167751 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f12ec064ee0 0x7f12ec067390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f12f80090b0 tx=0x7f12f8008030 comp rx=0 tx=0).stop 2023-11-08T08:19:33.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.726+0000 7f130e66f700 1 -- 172.21.15.40:0/177167751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1308070080 msgr2=0x7f130815b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:33.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.726+0000 7f130e66f700 1 --2- 172.21.15.40:0/177167751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1308070080 0x7f130815b490 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f130400c700 tx=0x7f130400ca10 comp rx=0 tx=0).stop 2023-11-08T08:19:33.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.726+0000 7f130e66f700 1 -- 172.21.15.40:0/177167751 shutdown_connections 2023-11-08T08:19:33.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.726+0000 7f130e66f700 1 --2- 172.21.15.40:0/177167751 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f12ec064ee0 0x7f12ec067390 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:33.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.726+0000 7f130e66f700 1 --2- 172.21.15.40:0/177167751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1308164250 0x7f13081613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:33.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.726+0000 7f130e66f700 1 --2- 172.21.15.40:0/177167751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1308160710 0x7f13080c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:33.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.726+0000 7f130e66f700 1 --2- 172.21.15.40:0/177167751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1308070080 0x7f130815b490 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:33.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.726+0000 7f130e66f700 1 -- 172.21.15.40:0/177167751 >> 172.21.15.40:0/177167751 conn(0x7f13080b0f80 msgr2=0x7f13080ae0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:33.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.726+0000 7f130e66f700 1 -- 172.21.15.40:0/177167751 shutdown_connections 2023-11-08T08:19:33.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.726+0000 7f130e66f700 1 -- 172.21.15.40:0/177167751 wait complete. 2023-11-08T08:19:33.751 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-08T08:19:33.751 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:19:33.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.926+0000 7f0ac6a53700 1 Processor -- start 2023-11-08T08:19:33.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.930+0000 7f0ac6a53700 1 -- start start 2023-11-08T08:19:33.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.930+0000 7f0ac6a53700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0ac00ce370 0x7f0ac00ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:33.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.930+0000 7f0ac6a53700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac00cb9c0 con 0x7f0ac00b1b30 2023-11-08T08:19:33.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.930+0000 7f0ac6a53700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac00cadf0 con 0x7f0ac00cca20 2023-11-08T08:19:33.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.930+0000 7f0ac6a53700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac00ced30 con 0x7f0ac00ce370 2023-11-08T08:19:33.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.930+0000 7f0ac5a51700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0ac00b1b30 0x7f0ac00cd950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49686/0 (socket says 172.21.15.40:49686) 2023-11-08T08:19:33.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.930+0000 7f0ac5a51700 1 -- 172.21.15.40:0/1808529071 learned_addr learned my addr 172.21.15.40:0/1808529071 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:33.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.930+0000 7f0ac6252700 1 --2- 172.21.15.40:0/1808529071 >> v2:172.21.15.169:3300/0 conn(0x7f0ac00ce370 0x7f0ac00ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:33.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.930+0000 7f0ac4a4f700 1 -- 172.21.15.40:0/1808529071 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119285290 0 0) 0x7f0ac00cb9c0 con 0x7f0ac00b1b30 2023-11-08T08:19:33.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.930+0000 7f0ac4a4f700 1 -- 172.21.15.40:0/1808529071 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aac0035d0 con 0x7f0ac00b1b30 2023-11-08T08:19:33.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.930+0000 7f0ac4a4f700 1 -- 172.21.15.40:0/1808529071 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934981118 0 0) 0x7f0ac00cadf0 con 0x7f0ac00cca20 2023-11-08T08:19:33.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.930+0000 7f0ac4a4f700 1 -- 172.21.15.40:0/1808529071 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ac00cb9c0 con 0x7f0ac00cca20 2023-11-08T08:19:33.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.930+0000 7f0ac6252700 1 -- 172.21.15.40:0/1808529071 >> v1:172.21.15.169:6789/0 conn(0x7f0ac00cca20 legacy=0x7f0ac00b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:33.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.930+0000 7f0ac6252700 1 -- 172.21.15.40:0/1808529071 >> v1:172.21.15.40:6789/0 conn(0x7f0ac00b1b30 legacy=0x7f0ac00cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:33.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.930+0000 7f0ac6252700 1 -- 172.21.15.40:0/1808529071 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac00cb090 con 0x7f0ac00ce370 2023-11-08T08:19:33.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.934+0000 7f0ac6252700 1 --2- 172.21.15.40:0/1808529071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac00ce370 0x7f0ac00ce7f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009d90 tx=0x7f0ab800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6020fbc52a6040cb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:33.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.934+0000 7f0ac4a4f700 1 -- 172.21.15.40:0/1808529071 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab800e070 con 0x7f0ac00ce370 2023-11-08T08:19:33.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.934+0000 7f0ac4a4f700 1 -- 172.21.15.40:0/1808529071 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab800bb20 con 0x7f0ac00ce370 2023-11-08T08:19:33.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.934+0000 7f0ac4a4f700 1 -- 172.21.15.40:0/1808529071 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8012430 con 0x7f0ac00ce370 2023-11-08T08:19:33.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.934+0000 7f0ac6a53700 1 -- 172.21.15.40:0/1808529071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac00ce370 msgr2=0x7f0ac00ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:33.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.934+0000 7f0ac6a53700 1 --2- 172.21.15.40:0/1808529071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac00ce370 0x7f0ac00ce7f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009d90 tx=0x7f0ab800b330 comp rx=0 tx=0).stop 2023-11-08T08:19:33.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.934+0000 7f0ac6a53700 1 -- 172.21.15.40:0/1808529071 shutdown_connections 2023-11-08T08:19:33.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.934+0000 7f0ac6a53700 1 --2- 172.21.15.40:0/1808529071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac00ce370 0x7f0ac00ce7f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:33.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.934+0000 7f0ac6a53700 1 -- 172.21.15.40:0/1808529071 >> 172.21.15.40:0/1808529071 conn(0x7f0ac0099af0 msgr2=0x7f0ac00d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:33.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.934+0000 7f0ac6a53700 1 -- 172.21.15.40:0/1808529071 shutdown_connections 2023-11-08T08:19:33.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.934+0000 7f0ac6a53700 1 -- 172.21.15.40:0/1808529071 wait complete. 2023-11-08T08:19:33.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.938+0000 7f0ac6a53700 1 Processor -- start 2023-11-08T08:19:33.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.938+0000 7f0ac6a53700 1 -- start start 2023-11-08T08:19:33.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.938+0000 7f0ac6a53700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac00b1b30 0x7f0ac00c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:33.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.938+0000 7f0ac6a53700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ac00cca20 0x7f0ac00c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:33.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.938+0000 7f0ac6a53700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac00ce370 0x7f0ac00b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:33.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.938+0000 7f0ac6a53700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00b65d0 con 0x7f0ac00b1b30 2023-11-08T08:19:33.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.938+0000 7f0ac6a53700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00c7db0 con 0x7f0ac00ce370 2023-11-08T08:19:33.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.938+0000 7f0ac6a53700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00c7fe0 con 0x7f0ac00cca20 2023-11-08T08:19:33.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.938+0000 7f0ac5a51700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac00b1b30 0x7f0ac00c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:33.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.938+0000 7f0ac5a51700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac00b1b30 0x7f0ac00c7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52222/0 (socket says 172.21.15.40:52222) 2023-11-08T08:19:33.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.938+0000 7f0ac5a51700 1 -- 172.21.15.40:0/3514214829 learned_addr learned my addr 172.21.15.40:0/3514214829 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:33.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.938+0000 7f0ac5250700 1 --2- 172.21.15.40:0/3514214829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ac00cca20 0x7f0ac00c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:33.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.938+0000 7f0ac6252700 1 --2- 172.21.15.40:0/3514214829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac00ce370 0x7f0ac00b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:33.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.938+0000 7f0ac5a51700 1 -- 172.21.15.40:0/3514214829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ac00cca20 msgr2=0x7f0ac00c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:33.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.938+0000 7f0ac5a51700 1 --2- 172.21.15.40:0/3514214829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ac00cca20 0x7f0ac00c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:33.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.938+0000 7f0ac5a51700 1 -- 172.21.15.40:0/3514214829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac00ce370 msgr2=0x7f0ac00b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:33.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.938+0000 7f0ac5a51700 1 --2- 172.21.15.40:0/3514214829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac00ce370 0x7f0ac00b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:33.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.938+0000 7f0ac5a51700 1 -- 172.21.15.40:0/3514214829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab8009a70 con 0x7f0ac00b1b30 2023-11-08T08:19:33.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.938+0000 7f0ac6252700 1 --2- 172.21.15.40:0/3514214829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac00ce370 0x7f0ac00b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:33.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.938+0000 7f0ac5a51700 1 --2- 172.21.15.40:0/3514214829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac00b1b30 0x7f0ac00c7870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0abc001eb0 tx=0x7f0abc00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:33.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.942+0000 7f0ab67fc700 1 -- 172.21.15.40:0/3514214829 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc0074e0 con 0x7f0ac00b1b30 2023-11-08T08:19:33.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.942+0000 7f0ab67fc700 1 -- 172.21.15.40:0/3514214829 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0abc007640 con 0x7f0ac00b1b30 2023-11-08T08:19:33.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.942+0000 7f0ab67fc700 1 -- 172.21.15.40:0/3514214829 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc005a50 con 0x7f0ac00b1b30 2023-11-08T08:19:33.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.942+0000 7f0ac6a53700 1 -- 172.21.15.40:0/3514214829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac00cb650 con 0x7f0ac00b1b30 2023-11-08T08:19:33.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.942+0000 7f0ac6a53700 1 -- 172.21.15.40:0/3514214829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac00cb370 con 0x7f0ac00b1b30 2023-11-08T08:19:33.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.942+0000 7f0ac6a53700 1 -- 172.21.15.40:0/3514214829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac00b3d00 con 0x7f0ac00b1b30 2023-11-08T08:19:33.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.942+0000 7f0ab67fc700 1 -- 172.21.15.40:0/3514214829 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f0abc019030 con 0x7f0ac00b1b30 2023-11-08T08:19:33.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.946+0000 7f0ab67fc700 1 --2- 172.21.15.40:0/3514214829 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0aa4064ee0 0x7f0aa4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:33.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.946+0000 7f0ab67fc700 1 -- 172.21.15.40:0/3514214829 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0abc081380 con 0x7f0ac00b1b30 2023-11-08T08:19:33.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.946+0000 7f0ac5250700 1 --2- 172.21.15.40:0/3514214829 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0aa4064ee0 0x7f0aa4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:33.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.946+0000 7f0ac5250700 1 --2- 172.21.15.40:0/3514214829 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0aa4064ee0 0x7f0aa4067390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0ac00b6230 tx=0x7f0ab0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:33.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:33.950+0000 7f0ab67fc700 1 -- 172.21.15.40:0/3514214829 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f0abc04a400 con 0x7f0ac00b1b30 2023-11-08T08:19:34.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.126+0000 7f0ac6a53700 1 -- 172.21.15.40:0/3514214829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ac00b6720 con 0x7f0ac00b1b30 2023-11-08T08:19:34.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.130+0000 7f0ab67fc700 1 -- 172.21.15.40:0/3514214829 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19652 (secure 0 0 0) 0x7f0abc04f2e0 con 0x7f0ac00b1b30 2023-11-08T08:19:34.133 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:34.134 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":28,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:08.980158+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:19:34.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.130+0000 7f0ac6a53700 1 -- 172.21.15.40:0/3514214829 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0aa4064ee0 msgr2=0x7f0aa4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.130+0000 7f0ac6a53700 1 --2- 172.21.15.40:0/3514214829 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0aa4064ee0 0x7f0aa4067390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0ac00b6230 tx=0x7f0ab0009040 comp rx=0 tx=0).stop 2023-11-08T08:19:34.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.134+0000 7f0ac6a53700 1 -- 172.21.15.40:0/3514214829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac00b1b30 msgr2=0x7f0ac00c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.134+0000 7f0ac6a53700 1 --2- 172.21.15.40:0/3514214829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac00b1b30 0x7f0ac00c7870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0abc001eb0 tx=0x7f0abc00c7b0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.134+0000 7f0ac6a53700 1 -- 172.21.15.40:0/3514214829 shutdown_connections 2023-11-08T08:19:34.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.134+0000 7f0ac6a53700 1 --2- 172.21.15.40:0/3514214829 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0aa4064ee0 0x7f0aa4067390 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.134+0000 7f0ac6a53700 1 --2- 172.21.15.40:0/3514214829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac00ce370 0x7f0ac00b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.134+0000 7f0ac6a53700 1 --2- 172.21.15.40:0/3514214829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ac00cca20 0x7f0ac00c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.134+0000 7f0ac6a53700 1 --2- 172.21.15.40:0/3514214829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac00b1b30 0x7f0ac00c7870 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.134+0000 7f0ac6a53700 1 -- 172.21.15.40:0/3514214829 >> 172.21.15.40:0/3514214829 conn(0x7f0ac0099af0 msgr2=0x7f0ac0159c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.134+0000 7f0ac6a53700 1 -- 172.21.15.40:0/3514214829 shutdown_connections 2023-11-08T08:19:34.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.134+0000 7f0ac6a53700 1 -- 172.21.15.40:0/3514214829 wait complete. 2023-11-08T08:19:34.157 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-08T08:19:34.158 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-08T08:19:34.159 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-08T08:19:34.160 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-08T08:19:34.160 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-08T08:19:34.161 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-08T08:19:34.162 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-08T08:19:34.162 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-08T08:19:34.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.406+0000 7fbe1640b700 1 Processor -- start 2023-11-08T08:19:34.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.406+0000 7fbe1640b700 1 -- start start 2023-11-08T08:19:34.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.406+0000 7fa187fe6700 1 Processor -- start 2023-11-08T08:19:34.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.406+0000 7f4b48674700 1 Processor -- start 2023-11-08T08:19:34.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.406+0000 7fbe1640b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe08038560 0x7fbe0803ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.406+0000 7fbe1640b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbe0803b160 0x7fbe08057210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.406+0000 7fbe1640b700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe08007360 con 0x7fbe080e29f0 2023-11-08T08:19:34.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.406+0000 7fbe1640b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe0803b600 con 0x7fbe0803b160 2023-11-08T08:19:34.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7fbe1640b700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe080e2dc0 con 0x7fbe08038560 2023-11-08T08:19:34.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.406+0000 7f4b48674700 1 -- start start 2023-11-08T08:19:34.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.406+0000 7fa187fe6700 1 -- start start 2023-11-08T08:19:34.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7fa187fe6700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa180070130 0x7fa180070500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7fa187fe6700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1800cc290 0x7fa1800cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7fa187fe6700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18015bca0 con 0x7fa18015beb0 2023-11-08T08:19:34.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7fa187fe6700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18006a5c0 con 0x7fa1800cc290 2023-11-08T08:19:34.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7fa187fe6700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa180070a40 con 0x7fa180070130 2023-11-08T08:19:34.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7fbe14c08700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe08038560 0x7fbe0803ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7fbe15409700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fbe080e29f0 0x7fbe08055490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:38176/0 (socket says 172.21.15.40:38176) 2023-11-08T08:19:34.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7fbe15409700 1 -- 172.21.15.40:0/3663883546 learned_addr learned my addr 172.21.15.40:0/3663883546 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:34.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7fa185581700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1800cc290 0x7fa1800cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7fbe15c0a700 1 --2- 172.21.15.40:0/3663883546 >> v2:172.21.15.40:3300/0 conn(0x7fbe0803b160 0x7fbe08057210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7f4b48674700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b4015d510 0x7f4b400b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7f4b48674700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4b400b6030 0x7f4b40070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7f4b48674700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4015d080 con 0x7f4b401534a0 2023-11-08T08:19:34.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7f4b48674700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4015d940 con 0x7f4b400b6030 2023-11-08T08:19:34.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7f4b48674700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b40153870 con 0x7f4b4015d510 2023-11-08T08:19:34.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7fa185d82700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa180070130 0x7fa180070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7fa185d82700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa180070130 0x7fa180070500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33472/0 (socket says 172.21.15.40:33472) 2023-11-08T08:19:34.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7fa185d82700 1 -- 172.21.15.40:0/1613463128 learned_addr learned my addr 172.21.15.40:0/1613463128 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:34.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7f8a65fa1700 1 Processor -- start 2023-11-08T08:19:34.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7fbe14c08700 1 -- 172.21.15.40:0/3663883546 >> v2:172.21.15.40:3300/0 conn(0x7fbe0803b160 msgr2=0x7fbe08057210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:34.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7fbe14c08700 1 --2- 172.21.15.40:0/3663883546 >> v2:172.21.15.40:3300/0 conn(0x7fbe0803b160 0x7fbe08057210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7fbe14c08700 1 -- 172.21.15.40:0/3663883546 >> v1:172.21.15.169:6789/0 conn(0x7fbe080e29f0 legacy=0x7fbe08055490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7fbe07fff700 1 -- 172.21.15.40:0/3663883546 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3396033432 0 0) 0x7fbe08007360 con 0x7fbe080e29f0 2023-11-08T08:19:34.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7f4b46410700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f4b401534a0 0x7f4b40160920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:38182/0 (socket says 172.21.15.40:38182) 2023-11-08T08:19:34.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.410+0000 7f4b46410700 1 -- 172.21.15.40:0/186841116 learned_addr learned my addr 172.21.15.40:0/186841116 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:34.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7f4b46c11700 1 --2- 172.21.15.40:0/186841116 >> v2:172.21.15.169:3300/0 conn(0x7f4b400b6030 0x7f4b40070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7f4b45c0f700 1 --2- 172.21.15.40:0/186841116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b4015d510 0x7f4b400b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7fbe14c08700 1 -- 172.21.15.40:0/3663883546 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe08053f70 con 0x7fbe08038560 2023-11-08T08:19:34.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7fa185d82700 1 -- 172.21.15.40:0/1613463128 >> v2:172.21.15.40:3300/0 conn(0x7fa1800cc290 msgr2=0x7fa1800cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:34.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7fa185d82700 1 --2- 172.21.15.40:0/1613463128 >> v2:172.21.15.40:3300/0 conn(0x7fa1800cc290 0x7fa1800cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7fa185d82700 1 -- 172.21.15.40:0/1613463128 >> v1:172.21.15.169:6789/0 conn(0x7fa18015beb0 legacy=0x7fa18015c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7fa185d82700 1 -- 172.21.15.40:0/1613463128 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa180069c90 con 0x7fa180070130 2023-11-08T08:19:34.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7fa185581700 1 --2- 172.21.15.40:0/1613463128 >> v2:172.21.15.40:3300/0 conn(0x7fa1800cc290 0x7fa1800cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:34.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7f4b46c11700 1 -- 172.21.15.40:0/186841116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b4015d510 msgr2=0x7f4b400b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:34.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7fbe14c08700 1 --2- 172.21.15.40:0/3663883546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe08038560 0x7fbe0803ac20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009850 tx=0x7fbe00009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20d33f4e96761d29 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7f4b46c11700 1 --2- 172.21.15.40:0/186841116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b4015d510 0x7f4b400b5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7f4b46c11700 1 -- 172.21.15.40:0/186841116 >> v1:172.21.15.169:6789/0 conn(0x7f4b401534a0 legacy=0x7f4b40160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7f4b46c11700 1 -- 172.21.15.40:0/186841116 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b400b6450 con 0x7f4b400b6030 2023-11-08T08:19:34.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7f8a65fa1700 1 -- start start 2023-11-08T08:19:34.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7fbe07fff700 1 -- 172.21.15.40:0/3663883546 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe00011830 con 0x7fbe08038560 2023-11-08T08:19:34.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7fbe07fff700 1 -- 172.21.15.40:0/3663883546 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe00011990 con 0x7fbe08038560 2023-11-08T08:19:34.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7fbe07fff700 1 -- 172.21.15.40:0/3663883546 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe00011830 con 0x7fbe08038560 2023-11-08T08:19:34.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7fca189f1700 1 Processor -- start 2023-11-08T08:19:34.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7f4b46c11700 1 --2- 172.21.15.40:0/186841116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b400b6030 0x7f4b40070630 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009d90 tx=0x7f4b3800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37d5be1e84134ce3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7fa185d82700 1 --2- 172.21.15.40:0/1613463128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa180070130 0x7fa180070500 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa180071830 tx=0x7fa170009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6bc3b6fa4d70bf4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7ff185ba7700 1 Processor -- start 2023-11-08T08:19:34.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7fbe1640b700 1 -- 172.21.15.40:0/3663883546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe08038560 msgr2=0x7fbe0803ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7f8a65fa1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a60070080 0x7f8a60070450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7fbe1640b700 1 --2- 172.21.15.40:0/3663883546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe08038560 0x7fbe0803ac20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009850 tx=0x7fbe00009b60 comp rx=0 tx=0).stop 2023-11-08T08:19:34.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7fa184d80700 1 -- 172.21.15.40:0/1613463128 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170011830 con 0x7fa180070130 2023-11-08T08:19:34.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7fa184d80700 1 -- 172.21.15.40:0/1613463128 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa170011990 con 0x7fa180070130 2023-11-08T08:19:34.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7fa184d80700 1 -- 172.21.15.40:0/1613463128 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170011d20 con 0x7fa180070130 2023-11-08T08:19:34.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7ff185ba7700 1 -- start start 2023-11-08T08:19:34.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7efe9f2cb700 1 Processor -- start 2023-11-08T08:19:34.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7f8a5ffff700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f8a6006cb00 0x7f8a600cc320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:38188/0 (socket says 172.21.15.40:38188) 2023-11-08T08:19:34.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7f8a5ffff700 1 -- 172.21.15.40:0/3271195673 learned_addr learned my addr 172.21.15.40:0/3271195673 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:34.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7fca189f1700 1 -- start start 2023-11-08T08:19:34.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7fca189f1700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fca1415c6d0 0x7fca140b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fca189f1700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca1406a300 0x7fca1406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7f8a65fa1700 1 -- 172.21.15.40:0/3271195673 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a6006a450 con 0x7f8a6006a8e0 2023-11-08T08:19:34.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fca189f1700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca14072270 con 0x7fca140cfc40 2023-11-08T08:19:34.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fca189f1700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca140d0400 con 0x7fca1415c6d0 2023-11-08T08:19:34.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fca189f1700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca140d0120 con 0x7fca1406a300 2023-11-08T08:19:34.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7f4b4540e700 1 -- 172.21.15.40:0/186841116 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3800e070 con 0x7f4b400b6030 2023-11-08T08:19:34.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7f4b4540e700 1 -- 172.21.15.40:0/186841116 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3800bb20 con 0x7f4b400b6030 2023-11-08T08:19:34.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7f4b4540e700 1 -- 172.21.15.40:0/186841116 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b38012430 con 0x7f4b400b6030 2023-11-08T08:19:34.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fa187fe6700 1 -- 172.21.15.40:0/1613463128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa180070130 msgr2=0x7fa180070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fa187fe6700 1 --2- 172.21.15.40:0/1613463128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa180070130 0x7fa180070500 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa180071830 tx=0x7fa170009bf0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.414+0000 7efe9f2cb700 1 -- start start 2023-11-08T08:19:34.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fbe1640b700 1 -- 172.21.15.40:0/3663883546 shutdown_connections 2023-11-08T08:19:34.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fbe1640b700 1 --2- 172.21.15.40:0/3663883546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe08038560 0x7fbe0803ac20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fbe1640b700 1 --2- 172.21.15.40:0/3663883546 >> v2:172.21.15.40:3300/0 conn(0x7fbe0803b160 0x7fbe08057210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fbe1640b700 1 -- 172.21.15.40:0/3663883546 >> 172.21.15.40:0/3663883546 conn(0x7fbe08055c20 msgr2=0x7fbe08056020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7f8a65fa1700 1 -- 172.21.15.40:0/3271195673 --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a60070990 con 0x7f8a6006cb00 2023-11-08T08:19:34.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7f8a65fa1700 1 -- 172.21.15.40:0/3271195673 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a60158760 con 0x7f8a60070080 2023-11-08T08:19:34.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7f8a64f9f700 1 --2- 172.21.15.40:0/3271195673 >> v2:172.21.15.40:3300/0 conn(0x7f8a60070080 0x7f8a60070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fbe1640b700 1 -- 172.21.15.40:0/3663883546 shutdown_connections 2023-11-08T08:19:34.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fca12d9d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fca1415c6d0 0x7fca140b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fca1359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca1406a300 0x7fca1406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fca12d9d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fca1415c6d0 0x7fca140b17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33490/0 (socket says 172.21.15.40:33490) 2023-11-08T08:19:34.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fca12d9d700 1 -- 172.21.15.40:0/3534072668 learned_addr learned my addr 172.21.15.40:0/3534072668 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:34.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7ff185ba7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff180163cb0 0x7ff180164cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7ff185ba7700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff1800cdfb0 0x7ff18006cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7ff185ba7700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff180164080 con 0x7ff18006d2f0 2023-11-08T08:19:34.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7ff185ba7700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1800cc290 con 0x7ff1800cdfb0 2023-11-08T08:19:34.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7ff185ba7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18006d770 con 0x7ff180163cb0 2023-11-08T08:19:34.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7f8a5f7fe700 1 -- 172.21.15.40:0/3271195673 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212762011 0 0) 0x7f8a60070990 con 0x7f8a6006cb00 2023-11-08T08:19:34.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7f8a5f7fe700 1 -- 172.21.15.40:0/3271195673 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a500035d0 con 0x7f8a6006cb00 2023-11-08T08:19:34.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7f8a64f9f700 1 -- 172.21.15.40:0/3271195673 >> v1:172.21.15.169:6789/0 conn(0x7f8a6006cb00 legacy=0x7f8a600cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7f8a64f9f700 1 -- 172.21.15.40:0/3271195673 >> v1:172.21.15.40:6789/0 conn(0x7f8a6006a8e0 legacy=0x7f8a6006ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:34.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7f4b48674700 1 -- 172.21.15.40:0/186841116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b400b6030 msgr2=0x7f4b40070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7f4b48674700 1 --2- 172.21.15.40:0/186841116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b400b6030 0x7f4b40070630 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009d90 tx=0x7f4b3800b330 comp rx=0 tx=0).stop 2023-11-08T08:19:34.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fca12d9d700 1 -- 172.21.15.40:0/3534072668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca1406a300 msgr2=0x7fca1406c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:34.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fca12d9d700 1 --2- 172.21.15.40:0/3534072668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca1406a300 0x7fca1406c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fca12d9d700 1 -- 172.21.15.40:0/3534072668 >> v1:172.21.15.169:6789/0 conn(0x7fca140cfc40 legacy=0x7fca14069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:34.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fca12d9d700 1 -- 172.21.15.40:0/3534072668 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca14071c20 con 0x7fca1415c6d0 2023-11-08T08:19:34.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7efe9f2cb700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe98160710 0x7efe980cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7efe9f2cb700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efe980cc860 0x7efe98070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7efe9f2cb700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe980ad6c0 con 0x7efe980705c0 2023-11-08T08:19:34.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7efe9f2cb700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe98070a60 con 0x7efe980cc860 2023-11-08T08:19:34.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7efe9f2cb700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9806b1c0 con 0x7efe98160710 2023-11-08T08:19:34.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7f4b48674700 1 -- 172.21.15.40:0/186841116 shutdown_connections 2023-11-08T08:19:34.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7f4b48674700 1 --2- 172.21.15.40:0/186841116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b400b6030 0x7f4b40070630 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7f4b48674700 1 --2- 172.21.15.40:0/186841116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b4015d510 0x7f4b400b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7f8a64f9f700 1 -- 172.21.15.40:0/3271195673 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a600ccc20 con 0x7f8a60070080 2023-11-08T08:19:34.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7ff17f7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff180163cb0 0x7ff180164cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7ff17effd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff1800cdfb0 0x7ff18006cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7f4b48674700 1 -- 172.21.15.40:0/186841116 >> 172.21.15.40:0/186841116 conn(0x7f4b4006bfc0 msgr2=0x7f4b4006c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7ff17f7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff180163cb0 0x7ff180164cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:53338/0 (socket says 172.21.15.40:53338) 2023-11-08T08:19:34.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7ff17ffff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff18006d2f0 0x7ff18006bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49704/0 (socket says 172.21.15.40:49704) 2023-11-08T08:19:34.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fca12d9d700 1 --2- 172.21.15.40:0/3534072668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca1415c6d0 0x7fca140b17a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009910 tx=0x7fca0c009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8811eabe807ac2a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fa187fe6700 1 -- 172.21.15.40:0/1613463128 shutdown_connections 2023-11-08T08:19:34.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fa187fe6700 1 --2- 172.21.15.40:0/1613463128 >> v2:172.21.15.40:3300/0 conn(0x7fa1800cc290 0x7fa1800cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fa187fe6700 1 --2- 172.21.15.40:0/1613463128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa180070130 0x7fa180070500 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fa187fe6700 1 -- 172.21.15.40:0/1613463128 >> 172.21.15.40:0/1613463128 conn(0x7fa1800ce6c0 msgr2=0x7fa1800cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7ff17ffff700 1 -- 172.21.15.40:0/2385567621 learned_addr learned my addr 172.21.15.40:0/2385567621 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:34.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7efe9e2c9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe98160710 0x7efe980cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7efe9e2c9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe98160710 0x7efe980cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:53346/0 (socket says 172.21.15.40:53346) 2023-11-08T08:19:34.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7efe9e2c9700 1 -- 172.21.15.40:0/2889757249 learned_addr learned my addr 172.21.15.40:0/2889757249 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:34.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7efe9dac8700 1 --2- 172.21.15.40:0/2889757249 >> v2:172.21.15.169:3300/0 conn(0x7efe980cc860 0x7efe98070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fa187fe6700 1 -- 172.21.15.40:0/1613463128 shutdown_connections 2023-11-08T08:19:34.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fa187fe6700 1 -- 172.21.15.40:0/1613463128 wait complete. 2023-11-08T08:19:34.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7f8a64f9f700 1 --2- 172.21.15.40:0/3271195673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a60070080 0x7f8a60070450 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009260 tx=0x7f8a54009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913989926c0a1c5b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7ff17effd700 1 -- 172.21.15.40:0/2385567621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff180163cb0 msgr2=0x7ff180164cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:34.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7ff17effd700 1 --2- 172.21.15.40:0/2385567621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff180163cb0 0x7ff180164cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7ff17effd700 1 -- 172.21.15.40:0/2385567621 >> v1:172.21.15.40:6789/0 conn(0x7ff18006d2f0 legacy=0x7ff18006bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7ff17effd700 1 -- 172.21.15.40:0/2385567621 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff180069bf0 con 0x7ff1800cdfb0 2023-11-08T08:19:34.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fbe1640b700 1 -- 172.21.15.40:0/3663883546 wait complete. 2023-11-08T08:19:34.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.418+0000 7fbe1640b700 1 Processor -- start 2023-11-08T08:19:34.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fca11d9b700 1 -- 172.21.15.40:0/3534072668 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c004230 con 0x7fca1415c6d0 2023-11-08T08:19:34.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fca11d9b700 1 -- 172.21.15.40:0/3534072668 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0c004390 con 0x7fca1415c6d0 2023-11-08T08:19:34.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fca11d9b700 1 -- 172.21.15.40:0/3534072668 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c009e60 con 0x7fca1415c6d0 2023-11-08T08:19:34.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7f4b48674700 1 -- 172.21.15.40:0/186841116 shutdown_connections 2023-11-08T08:19:34.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7f8a5f7fe700 1 -- 172.21.15.40:0/3271195673 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5400c880 con 0x7f8a60070080 2023-11-08T08:19:34.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7ff17effd700 1 --2- 172.21.15.40:0/2385567621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1800cdfb0 0x7ff18006cdb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff170009850 tx=0x7ff170009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aef7815e3ac13fd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7efe9dac8700 1 -- 172.21.15.40:0/2889757249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe98160710 msgr2=0x7efe980cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:34.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7efe9dac8700 1 --2- 172.21.15.40:0/2889757249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe98160710 0x7efe980cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7efe9dac8700 1 -- 172.21.15.40:0/2889757249 >> v1:172.21.15.169:6789/0 conn(0x7efe980705c0 legacy=0x7efe98069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:34.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fca189f1700 1 -- 172.21.15.40:0/3534072668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca1415c6d0 msgr2=0x7fca140b17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fca189f1700 1 --2- 172.21.15.40:0/3534072668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca1415c6d0 0x7fca140b17a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009910 tx=0x7fca0c009c20 comp rx=0 tx=0).stop 2023-11-08T08:19:34.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7f8a5f7fe700 1 -- 172.21.15.40:0/3271195673 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5400c9e0 con 0x7f8a60070080 2023-11-08T08:19:34.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7f8a5f7fe700 1 -- 172.21.15.40:0/3271195673 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5400cd90 con 0x7f8a60070080 2023-11-08T08:19:34.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7ff17e7fc700 1 -- 172.21.15.40:0/2385567621 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff170011830 con 0x7ff1800cdfb0 2023-11-08T08:19:34.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7ff17e7fc700 1 -- 172.21.15.40:0/2385567621 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff170011990 con 0x7ff1800cdfb0 2023-11-08T08:19:34.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7ff17e7fc700 1 -- 172.21.15.40:0/2385567621 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff170011d40 con 0x7ff1800cdfb0 2023-11-08T08:19:34.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fcb1d0fa700 1 Processor -- start 2023-11-08T08:19:34.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fcb1d0fa700 1 -- start start 2023-11-08T08:19:34.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7efe9dac8700 1 -- 172.21.15.40:0/2889757249 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe980adc60 con 0x7efe980cc860 2023-11-08T08:19:34.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7efe9e2c9700 1 --2- 172.21.15.40:0/2889757249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe98160710 0x7efe980cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:34.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7efe9dac8700 1 --2- 172.21.15.40:0/2889757249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe980cc860 0x7efe98070080 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efe88009850 tx=0x7efe88009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d74bacac753f9cb1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fbe1640b700 1 -- start start 2023-11-08T08:19:34.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7f4b48674700 1 -- 172.21.15.40:0/186841116 wait complete. 2023-11-08T08:19:34.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7f8a65fa1700 1 -- 172.21.15.40:0/3271195673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a60070080 msgr2=0x7f8a60070450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7f8a65fa1700 1 --2- 172.21.15.40:0/3271195673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a60070080 0x7f8a60070450 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009260 tx=0x7f8a54009570 comp rx=0 tx=0).stop 2023-11-08T08:19:34.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fbe1640b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe08038560 0x7fbe080dd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7efe9d2c7700 1 -- 172.21.15.40:0/2889757249 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe88011830 con 0x7efe980cc860 2023-11-08T08:19:34.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fbe1640b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe0803b160 0x7fbe080e03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7efe9d2c7700 1 -- 172.21.15.40:0/2889757249 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe88011990 con 0x7efe980cc860 2023-11-08T08:19:34.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7efe9d2c7700 1 -- 172.21.15.40:0/2889757249 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe88011d40 con 0x7efe980cc860 2023-11-08T08:19:34.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fa187fe6700 1 Processor -- start 2023-11-08T08:19:34.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fbe1640b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe080e29f0 0x7fbe0804f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fbe1640b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe080e1540 con 0x7fbe080e29f0 2023-11-08T08:19:34.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fbe1640b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0804db70 con 0x7fbe0803b160 2023-11-08T08:19:34.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fbe1640b700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0804de50 con 0x7fbe08038560 2023-11-08T08:19:34.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fcb1d0fa700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcb181534a0 0x7fcb1815d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fcb1d0fa700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb1815d080 con 0x7fcb180b6200 2023-11-08T08:19:34.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fcb1d0fa700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb18153870 con 0x7fcb18160920 2023-11-08T08:19:34.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fcb1d0fa700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb180b53c0 con 0x7fcb181534a0 2023-11-08T08:19:34.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fa187fe6700 1 -- start start 2023-11-08T08:19:34.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fbe15c0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe080e29f0 0x7fbe0804f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7f8a65fa1700 1 -- 172.21.15.40:0/3271195673 shutdown_connections 2023-11-08T08:19:34.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fcb0e59c700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fcb18160920 0x7fcb180b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:38218/0 (socket says 172.21.15.40:38218) 2023-11-08T08:19:34.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fcb0e59c700 1 -- 172.21.15.40:0/1580170314 learned_addr learned my addr 172.21.15.40:0/1580170314 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:34.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7ff185ba7700 1 -- 172.21.15.40:0/2385567621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1800cdfb0 msgr2=0x7ff18006cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7ff185ba7700 1 --2- 172.21.15.40:0/2385567621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1800cdfb0 0x7ff18006cdb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff170009850 tx=0x7ff170009b60 comp rx=0 tx=0).stop 2023-11-08T08:19:34.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fbe15c0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe080e29f0 0x7fbe0804f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52262/0 (socket says 172.21.15.40:52262) 2023-11-08T08:19:34.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7ff185ba7700 1 -- 172.21.15.40:0/2385567621 shutdown_connections 2023-11-08T08:19:34.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7ff185ba7700 1 --2- 172.21.15.40:0/2385567621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1800cdfb0 0x7ff18006cdb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7ff185ba7700 1 --2- 172.21.15.40:0/2385567621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff180163cb0 0x7ff180164cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7ff185ba7700 1 -- 172.21.15.40:0/2385567621 >> 172.21.15.40:0/2385567621 conn(0x7ff1800cf350 msgr2=0x7ff1800b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fbe15c0a700 1 -- 172.21.15.40:0/4018486245 learned_addr learned my addr 172.21.15.40:0/4018486245 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:34.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fbe15409700 1 --2- 172.21.15.40:0/4018486245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe08038560 0x7fbe080dd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7f8a65fa1700 1 --2- 172.21.15.40:0/3271195673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a60070080 0x7f8a60070450 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7f8a65fa1700 1 -- 172.21.15.40:0/3271195673 >> 172.21.15.40:0/3271195673 conn(0x7f8a600b0f90 msgr2=0x7f8a600ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.422+0000 7fcb1659c700 1 -- 172.21.15.40:0/1580170314 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3584386567 0 0) 0x7fcb18153870 con 0x7fcb18160920 2023-11-08T08:19:34.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1659c700 1 -- 172.21.15.40:0/1580170314 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb040035d0 con 0x7fcb18160920 2023-11-08T08:19:34.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7efe9f2cb700 1 -- 172.21.15.40:0/2889757249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe980cc860 msgr2=0x7efe98070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7efe9f2cb700 1 --2- 172.21.15.40:0/2889757249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe980cc860 0x7efe98070080 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efe88009850 tx=0x7efe88009b60 comp rx=0 tx=0).stop 2023-11-08T08:19:34.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7f4b48674700 1 Processor -- start 2023-11-08T08:19:34.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fca189f1700 1 -- 172.21.15.40:0/3534072668 shutdown_connections 2023-11-08T08:19:34.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fbe14c08700 1 --2- 172.21.15.40:0/4018486245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe0803b160 0x7fbe080e03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fbe15c0a700 1 -- 172.21.15.40:0/4018486245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe08038560 msgr2=0x7fbe080dd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fca189f1700 1 --2- 172.21.15.40:0/3534072668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca1406a300 0x7fca1406c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fca189f1700 1 --2- 172.21.15.40:0/3534072668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca1415c6d0 0x7fca140b17a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fca189f1700 1 -- 172.21.15.40:0/3534072668 >> 172.21.15.40:0/3534072668 conn(0x7fca140b6520 msgr2=0x7fca14071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7ff185ba7700 1 -- 172.21.15.40:0/2385567621 shutdown_connections 2023-11-08T08:19:34.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fca189f1700 1 -- 172.21.15.40:0/3534072668 shutdown_connections 2023-11-08T08:19:34.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7ff185ba7700 1 -- 172.21.15.40:0/2385567621 wait complete. 2023-11-08T08:19:34.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb16d9d700 1 --2- 172.21.15.40:0/1580170314 >> v2:172.21.15.169:3300/0 conn(0x7fcb181534a0 0x7fcb1815d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa187fe6700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa180070130 0x7fa1800c4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7f8a65fa1700 1 -- 172.21.15.40:0/3271195673 shutdown_connections 2023-11-08T08:19:34.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa187fe6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1800cc290 0x7fa1800b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa187fe6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa18015beb0 0x7fa1800c26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa187fe6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1800ae230 con 0x7fa18015beb0 2023-11-08T08:19:34.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa187fe6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1800c4a60 con 0x7fa1800cc290 2023-11-08T08:19:34.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa187fe6700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1800c4d40 con 0x7fa180070130 2023-11-08T08:19:34.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1659c700 1 -- 172.21.15.40:0/1580170314 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2922245404 0 0) 0x7fcb040035d0 con 0x7fcb18160920 2023-11-08T08:19:34.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1659c700 1 -- 172.21.15.40:0/1580170314 >> v2:172.21.15.169:3300/0 conn(0x7fcb181534a0 msgr2=0x7fcb1815d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:34.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1659c700 1 --2- 172.21.15.40:0/1580170314 >> v2:172.21.15.169:3300/0 conn(0x7fcb181534a0 0x7fcb1815d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1659c700 1 -- 172.21.15.40:0/1580170314 >> v1:172.21.15.40:6789/0 conn(0x7fcb180b6200 legacy=0x7fcb18070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1659c700 1 -- 172.21.15.40:0/1580170314 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb180b56a0 con 0x7fcb18160920 2023-11-08T08:19:34.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1659c700 1 -- 172.21.15.40:0/1580170314 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7fcb00001670 con 0x7fcb18160920 2023-11-08T08:19:34.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1659c700 1 -- 172.21.15.40:0/1580170314 >> v1:172.21.15.169:6789/0 conn(0x7fcb18160920 legacy=0x7fcb180b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1659c700 1 --2- 172.21.15.40:0/1580170314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb04003a20 0x7fcb04003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1659c700 1 --2- 172.21.15.40:0/1580170314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb04004870 0x7fcb04004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1659c700 1 --2- 172.21.15.40:0/1580170314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb040052f0 0x7fcb04014240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1659c700 1 -- 172.21.15.40:0/1580170314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb180b53c0 con 0x7fcb04004870 2023-11-08T08:19:34.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1659c700 1 -- 172.21.15.40:0/1580170314 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb04014a30 con 0x7fcb04003a20 2023-11-08T08:19:34.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1659c700 1 -- 172.21.15.40:0/1580170314 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb04014bf0 con 0x7fcb040052f0 2023-11-08T08:19:34.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fca189f1700 1 -- 172.21.15.40:0/3534072668 wait complete. 2023-11-08T08:19:34.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa185d82700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa180070130 0x7fa1800c4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa185d82700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa180070130 0x7fa1800c4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:53358/0 (socket says 172.21.15.40:53358) 2023-11-08T08:19:34.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa185d82700 1 -- 172.21.15.40:0/747681450 learned_addr learned my addr 172.21.15.40:0/747681450 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:34.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fbe15c0a700 1 --2- 172.21.15.40:0/4018486245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe08038560 0x7fbe080dd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fbe15c0a700 1 -- 172.21.15.40:0/4018486245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe0803b160 msgr2=0x7fbe080e03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fbe15c0a700 1 --2- 172.21.15.40:0/4018486245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe0803b160 0x7fbe080e03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fbe15c0a700 1 -- 172.21.15.40:0/4018486245 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe000094b0 con 0x7fbe080e29f0 2023-11-08T08:19:34.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fbe15409700 1 --2- 172.21.15.40:0/4018486245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbe08038560 0x7fbe080dd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:34.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fbe14c08700 1 --2- 172.21.15.40:0/4018486245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe0803b160 0x7fbe080e03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:34.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa185581700 1 --2- 172.21.15.40:0/747681450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1800cc290 0x7fa1800b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa186583700 1 --2- 172.21.15.40:0/747681450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa18015beb0 0x7fa1800c26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1759e700 1 --2- 172.21.15.40:0/1580170314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb040052f0 0x7fcb04014240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fbe15c0a700 1 --2- 172.21.15.40:0/4018486245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe080e29f0 0x7fbe0804f250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe10063810 tx=0x7fbe10072ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa186583700 1 -- 172.21.15.40:0/747681450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa180070130 msgr2=0x7fa1800c4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa186583700 1 --2- 172.21.15.40:0/747681450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa180070130 0x7fa1800c4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa186583700 1 -- 172.21.15.40:0/747681450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1800cc290 msgr2=0x7fa1800b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa186583700 1 --2- 172.21.15.40:0/747681450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1800cc290 0x7fa1800b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa186583700 1 -- 172.21.15.40:0/747681450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c0094b0 con 0x7fa18015beb0 2023-11-08T08:19:34.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7f4b48674700 1 -- start start 2023-11-08T08:19:34.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa185581700 1 --2- 172.21.15.40:0/747681450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1800cc290 0x7fa1800b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:34.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa186583700 1 --2- 172.21.15.40:0/747681450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa18015beb0 0x7fa1800c26c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa17800cb90 tx=0x7fa17800cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fbe05ffb700 1 -- 172.21.15.40:0/4018486245 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1008e070 con 0x7fbe080e29f0 2023-11-08T08:19:34.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fbe1640b700 1 -- 172.21.15.40:0/4018486245 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe08007080 con 0x7fbe080e29f0 2023-11-08T08:19:34.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7f4b48674700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b400b6030 0x7f4b4008d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa1767fc700 1 -- 172.21.15.40:0/747681450 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa178003c20 con 0x7fa18015beb0 2023-11-08T08:19:34.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa187fe6700 1 -- 172.21.15.40:0/747681450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa170009570 con 0x7fa18015beb0 2023-11-08T08:19:34.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fa187fe6700 1 -- 172.21.15.40:0/747681450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18006a250 con 0x7fa18015beb0 2023-11-08T08:19:34.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7ff185ba7700 1 Processor -- start 2023-11-08T08:19:34.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb16d9d700 1 --2- 172.21.15.40:0/1580170314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb04003a20 0x7fcb04003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb0e59c700 1 --2- 172.21.15.40:0/1580170314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb04004870 0x7fcb04004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1759e700 1 -- 172.21.15.40:0/1580170314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb04003a20 msgr2=0x7fcb04003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1759e700 1 --2- 172.21.15.40:0/1580170314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb04003a20 0x7fcb04003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1759e700 1 -- 172.21.15.40:0/1580170314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb04004870 msgr2=0x7fcb04004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1759e700 1 --2- 172.21.15.40:0/1580170314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb04004870 0x7fcb04004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1759e700 1 -- 172.21.15.40:0/1580170314 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb180b56a0 con 0x7fcb040052f0 2023-11-08T08:19:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb16d9d700 1 --2- 172.21.15.40:0/1580170314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb04003a20 0x7fcb04003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb0e59c700 1 --2- 172.21.15.40:0/1580170314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb04004870 0x7fcb04004ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:34.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fcb1759e700 1 -- 172.21.15.40:0/1580170314 reap_dead start 2023-11-08T08:19:34.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fbe05ffb700 1 -- 172.21.15.40:0/4018486245 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe10073070 con 0x7fbe080e29f0 2023-11-08T08:19:34.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fbe05ffb700 1 -- 172.21.15.40:0/4018486245 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe100733e0 con 0x7fbe080e29f0 2023-11-08T08:19:34.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fbe1640b700 1 -- 172.21.15.40:0/4018486245 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe08006dc0 con 0x7fbe080e29f0 2023-11-08T08:19:34.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7fca189f1700 1 Processor -- start 2023-11-08T08:19:34.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7fca189f1700 1 -- start start 2023-11-08T08:19:34.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7f4b48674700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b401534a0 0x7f4b40155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f4b48674700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b4015d510 0x7f4b40157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f4b48674700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4008b600 con 0x7f4b401534a0 2023-11-08T08:19:34.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f4b48674700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4008b8e0 con 0x7f4b400b6030 2023-11-08T08:19:34.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f4b48674700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4008bbc0 con 0x7f4b4015d510 2023-11-08T08:19:34.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f4b46410700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b400b6030 0x7f4b4008d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f4b46410700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b400b6030 0x7f4b4008d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33556/0 (socket says 172.21.15.40:33556) 2023-11-08T08:19:34.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f4b46410700 1 -- 172.21.15.40:0/1930274824 learned_addr learned my addr 172.21.15.40:0/1930274824 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:34.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.426+0000 7f8a65fa1700 1 -- 172.21.15.40:0/3271195673 wait complete. 2023-11-08T08:19:34.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f8a65fa1700 1 Processor -- start 2023-11-08T08:19:34.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7ff185ba7700 1 -- start start 2023-11-08T08:19:34.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7fcb1759e700 1 --2- 172.21.15.40:0/1580170314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb040052f0 0x7fcb04014240 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000c190 tx=0x7fcb040058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f47c157e2ae17d56 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7efe9f2cb700 1 -- 172.21.15.40:0/2889757249 shutdown_connections 2023-11-08T08:19:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7efe9f2cb700 1 --2- 172.21.15.40:0/2889757249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe980cc860 0x7efe98070080 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7efe9f2cb700 1 --2- 172.21.15.40:0/2889757249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe98160710 0x7efe980cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7efe9f2cb700 1 -- 172.21.15.40:0/2889757249 >> 172.21.15.40:0/2889757249 conn(0x7efe980b0f80 msgr2=0x7efe980ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7fbe1640b700 1 -- 172.21.15.40:0/4018486245 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbdf0000fc0 con 0x7fbe080e29f0 2023-11-08T08:19:34.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7fa187fe6700 1 -- 172.21.15.40:0/747681450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa180069f70 con 0x7fa18015beb0 2023-11-08T08:19:34.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7fa1767fc700 1 -- 172.21.15.40:0/747681450 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa178003d80 con 0x7fa18015beb0 2023-11-08T08:19:34.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7fa1767fc700 1 -- 172.21.15.40:0/747681450 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17800aa80 con 0x7fa18015beb0 2023-11-08T08:19:34.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f8a65fa1700 1 -- start start 2023-11-08T08:19:34.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7efe9f2cb700 1 -- 172.21.15.40:0/2889757249 shutdown_connections 2023-11-08T08:19:34.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7fcb1659c700 1 -- 172.21.15.40:0/1580170314 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10001db0 con 0x7fcb040052f0 2023-11-08T08:19:34.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f4b46410700 1 -- 172.21.15.40:0/1930274824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b4015d510 msgr2=0x7f4b40157840 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-08T08:19:34.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f4b46410700 1 --2- 172.21.15.40:0/1930274824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b4015d510 0x7f4b40157840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f4b46410700 1 -- 172.21.15.40:0/1930274824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b401534a0 msgr2=0x7f4b40155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f4b46410700 1 --2- 172.21.15.40:0/1930274824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b401534a0 0x7f4b40155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f4b46410700 1 -- 172.21.15.40:0/1930274824 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b38009a70 con 0x7f4b400b6030 2023-11-08T08:19:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7efe9f2cb700 1 -- 172.21.15.40:0/2889757249 wait complete. 2023-11-08T08:19:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7fca189f1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca1406a300 0x7fca14159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7fca189f1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca140cfc40 0x7fca140d1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7fca189f1700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca1415c6d0 0x7fca1415f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7fca12d9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca1406a300 0x7fca14159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7fca12d9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca1406a300 0x7fca14159bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33564/0 (socket says 172.21.15.40:33564) 2023-11-08T08:19:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f4b46410700 1 --2- 172.21.15.40:0/1930274824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b400b6030 0x7f4b4008d650 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00c500 tx=0x7f4b3c00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7fca12d9d700 1 -- 172.21.15.40:0/829629169 learned_addr learned my addr 172.21.15.40:0/829629169 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:34.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7fca1359e700 1 --2- 172.21.15.40:0/829629169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca1415c6d0 0x7fca1415f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7fca189f1700 1 -- 172.21.15.40:0/829629169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1415f0c0 con 0x7fca140cfc40 2023-11-08T08:19:34.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f8a65fa1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a6006a8e0 0x7f8a601ff580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f8a65fa1700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a6006cb00 0x7f8a600c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f8a65fa1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a60070080 0x7f8a601548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7fbe05ffb700 1 -- 172.21.15.40:0/4018486245 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fbe100a2070 con 0x7fbe080e29f0 2023-11-08T08:19:34.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f8a65fa1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a601ffc80 con 0x7f8a60070080 2023-11-08T08:19:34.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f8a65fa1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a601fff60 con 0x7f8a6006a8e0 2023-11-08T08:19:34.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f8a65fa1700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a60200240 con 0x7f8a6006cb00 2023-11-08T08:19:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7ff185ba7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18006d2f0 0x7ff1801f6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7ff185ba7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1800cdfb0 0x7ff1800c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f4b36ffd700 1 -- 172.21.15.40:0/1930274824 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c005d90 con 0x7f4b400b6030 2023-11-08T08:19:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7fca189f1700 1 -- 172.21.15.40:0/829629169 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1415e210 con 0x7fca1406a300 2023-11-08T08:19:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f4b36ffd700 1 -- 172.21.15.40:0/1930274824 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3c0070d0 con 0x7f4b400b6030 2023-11-08T08:19:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f4b48674700 1 -- 172.21.15.40:0/1930274824 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b400b56a0 con 0x7f4b400b6030 2023-11-08T08:19:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7fca189f1700 1 -- 172.21.15.40:0/829629169 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1415e4b0 con 0x7fca1415c6d0 2023-11-08T08:19:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7ff185ba7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff180163cb0 0x7ff18015eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7ff185ba7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1800c4910 con 0x7ff18006d2f0 2023-11-08T08:19:34.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7ff185ba7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18015f1e0 con 0x7ff180163cb0 2023-11-08T08:19:34.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7ff185ba7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18015f480 con 0x7ff1800cdfb0 2023-11-08T08:19:34.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f8a5ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a6006cb00 0x7f8a600c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f4b36ffd700 1 -- 172.21.15.40:0/1930274824 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c007440 con 0x7f4b400b6030 2023-11-08T08:19:34.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f8a5ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a6006cb00 0x7f8a600c5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:53404/0 (socket says 172.21.15.40:53404) 2023-11-08T08:19:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.430+0000 7f8a5ffff700 1 -- 172.21.15.40:0/843745657 learned_addr learned my addr 172.21.15.40:0/843745657 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7f8a64f9f700 1 --2- 172.21.15.40:0/843745657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a6006a8e0 0x7f8a601ff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fcb1659c700 1 -- 172.21.15.40:0/1580170314 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1000ccf0 con 0x7fcb040052f0 2023-11-08T08:19:34.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fcb1659c700 1 -- 172.21.15.40:0/1580170314 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10003c60 con 0x7fcb040052f0 2023-11-08T08:19:34.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fbe05ffb700 1 --2- 172.21.15.40:0/4018486245 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbdf4064ee0 0x7fbdf4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fbe05ffb700 1 -- 172.21.15.40:0/4018486245 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbe10072040 con 0x7fbe080e29f0 2023-11-08T08:19:34.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fbe05ffb700 1 --2- 172.21.15.40:0/4018486245 >> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] conn(0x7fbdf406ea30 0x7fbdf4070e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fbe05ffb700 1 -- 172.21.15.40:0/4018486245 --> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbdf4071530 con 0x7fbdf406ea30 2023-11-08T08:19:34.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fca1359e700 1 -- 172.21.15.40:0/829629169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca1406a300 msgr2=0x7fca14159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fca1359e700 1 --2- 172.21.15.40:0/829629169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca1406a300 0x7fca14159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fca1359e700 1 -- 172.21.15.40:0/829629169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca140cfc40 msgr2=0x7fca140d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fca1359e700 1 --2- 172.21.15.40:0/829629169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca140cfc40 0x7fca140d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fca1359e700 1 -- 172.21.15.40:0/829629169 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0c009570 con 0x7fca1415c6d0 2023-11-08T08:19:34.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fca1359e700 1 --2- 172.21.15.40:0/829629169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca1415c6d0 0x7fca1415f550 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fca04007d90 tx=0x7fca0400e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7f4b48674700 1 -- 172.21.15.40:0/1930274824 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b400b53c0 con 0x7f4b400b6030 2023-11-08T08:19:34.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7ff17effd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1800cdfb0 0x7ff1800c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7ff17effd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1800cdfb0 0x7ff1800c43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:53420/0 (socket says 172.21.15.40:53420) 2023-11-08T08:19:34.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7ff17effd700 1 -- 172.21.15.40:0/655561061 learned_addr learned my addr 172.21.15.40:0/655561061 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:34.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7ff17effd700 1 -- 172.21.15.40:0/655561061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff180163cb0 msgr2=0x7ff18015eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7ff17effd700 1 --2- 172.21.15.40:0/655561061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff180163cb0 0x7ff18015eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7ff17effd700 1 -- 172.21.15.40:0/655561061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18006d2f0 msgr2=0x7ff1801f6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7ff17effd700 1 --2- 172.21.15.40:0/655561061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18006d2f0 0x7ff1801f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7ff17effd700 1 -- 172.21.15.40:0/655561061 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1700094b0 con 0x7ff1800cdfb0 2023-11-08T08:19:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7efe9f2cb700 1 Processor -- start 2023-11-08T08:19:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7ff17effd700 1 --2- 172.21.15.40:0/655561061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1800cdfb0 0x7ff1800c43d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff170000c00 tx=0x7ff170004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fc9fb7fe700 1 -- 172.21.15.40:0/829629169 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca04018070 con 0x7fca1415c6d0 2023-11-08T08:19:34.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7efe9f2cb700 1 -- start start 2023-11-08T08:19:34.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fca189f1700 1 -- 172.21.15.40:0/829629169 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca1415caa0 con 0x7fca1415c6d0 2023-11-08T08:19:34.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fc9fb7fe700 1 -- 172.21.15.40:0/829629169 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0400ebe0 con 0x7fca1415c6d0 2023-11-08T08:19:34.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fca189f1700 1 -- 172.21.15.40:0/829629169 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca14071f00 con 0x7fca1415c6d0 2023-11-08T08:19:34.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fbe14c08700 1 --2- 172.21.15.40:0/4018486245 >> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] conn(0x7fbdf406ea30 0x7fbdf4070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fbe15409700 1 --2- 172.21.15.40:0/4018486245 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbdf4064ee0 0x7fbdf4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7f8a64f9f700 1 -- 172.21.15.40:0/843745657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a6006cb00 msgr2=0x7f8a600c5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7f8a64f9f700 1 --2- 172.21.15.40:0/843745657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a6006cb00 0x7f8a600c5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7f8a64f9f700 1 -- 172.21.15.40:0/843745657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a60070080 msgr2=0x7f8a601548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7f8a64f9f700 1 --2- 172.21.15.40:0/843745657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a60070080 0x7f8a601548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7f8a64f9f700 1 -- 172.21.15.40:0/843745657 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a54009d40 con 0x7f8a6006a8e0 2023-11-08T08:19:34.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7ff167fff700 1 -- 172.21.15.40:0/655561061 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff170004610 con 0x7ff1800cdfb0 2023-11-08T08:19:34.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7ff167fff700 1 -- 172.21.15.40:0/655561061 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17001f070 con 0x7ff1800cdfb0 2023-11-08T08:19:34.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7ff167fff700 1 -- 172.21.15.40:0/655561061 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1700038c0 con 0x7ff1800cdfb0 2023-11-08T08:19:34.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fcb1d0fa700 1 -- 172.21.15.40:0/1580170314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb040052f0 msgr2=0x7fcb04014240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fcb1d0fa700 1 --2- 172.21.15.40:0/1580170314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb040052f0 0x7fcb04014240 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000c190 tx=0x7fcb040058a0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7ff185ba7700 1 -- 172.21.15.40:0/655561061 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1800cc4a0 con 0x7ff1800cdfb0 2023-11-08T08:19:34.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fcb1d0fa700 1 -- 172.21.15.40:0/1580170314 shutdown_connections 2023-11-08T08:19:34.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7f4b48674700 1 -- 172.21.15.40:0/1930274824 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4b4015ca30 con 0x7f4b400b6030 2023-11-08T08:19:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fcb1d0fa700 1 --2- 172.21.15.40:0/1580170314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb040052f0 0x7fcb04014240 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7ff185ba7700 1 -- 172.21.15.40:0/655561061 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1801651f0 con 0x7ff1800cdfb0 2023-11-08T08:19:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7f8a64f9f700 1 --2- 172.21.15.40:0/843745657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a6006a8e0 0x7f8a601ff580 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8a54017000 tx=0x7f8a54004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7efe9f2cb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe980705c0 0x7efe9815b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7efe9f2cb700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe980cc860 0x7efe981614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7efe9f2cb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe98160710 0x7efe981633d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7efe9f2cb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe98162fb0 con 0x7efe98160710 2023-11-08T08:19:34.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7efe9f2cb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9815f9c0 con 0x7efe980705c0 2023-11-08T08:19:34.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7efe9f2cb700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe98162330 con 0x7efe980cc860 2023-11-08T08:19:34.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fcb1d0fa700 1 -- 172.21.15.40:0/1580170314 >> 172.21.15.40:0/1580170314 conn(0x7fcb1806bfc0 msgr2=0x7fcb1806c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fcb1d0fa700 1 -- 172.21.15.40:0/1580170314 shutdown_connections 2023-11-08T08:19:34.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fca189f1700 1 -- 172.21.15.40:0/829629169 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fca14073cb0 con 0x7fca1415c6d0 2023-11-08T08:19:34.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fc9fb7fe700 1 -- 172.21.15.40:0/829629169 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca04018070 con 0x7fca1415c6d0 2023-11-08T08:19:34.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7efe9e2c9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe980705c0 0x7efe9815b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7efe9e2c9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe980705c0 0x7efe9815b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33594/0 (socket says 172.21.15.40:33594) 2023-11-08T08:19:34.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7efe9e2c9700 1 -- 172.21.15.40:0/1326293102 learned_addr learned my addr 172.21.15.40:0/1326293102 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:34.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7efe9dac8700 1 --2- 172.21.15.40:0/1326293102 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe980cc860 0x7efe981614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7efe9e2c9700 1 -- 172.21.15.40:0/1326293102 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe980cc860 msgr2=0x7efe981614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7efe9e2c9700 1 --2- 172.21.15.40:0/1326293102 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe980cc860 0x7efe981614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7ff185ba7700 1 -- 172.21.15.40:0/655561061 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff1800cc780 con 0x7ff1800cdfb0 2023-11-08T08:19:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7efe9e2c9700 1 -- 172.21.15.40:0/1326293102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe98160710 msgr2=0x7efe981633d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-08T08:19:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7efe9e2c9700 1 --2- 172.21.15.40:0/1326293102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe98160710 0x7efe981633d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7efe9e2c9700 1 -- 172.21.15.40:0/1326293102 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe94009570 con 0x7efe980705c0 2023-11-08T08:19:34.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fbe05ffb700 1 -- 172.21.15.40:0/4018486245 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe100c9dc0 con 0x7fbe080e29f0 2023-11-08T08:19:34.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7efe9e2c9700 1 --2- 172.21.15.40:0/1326293102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe980705c0 0x7efe9815b0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efe9400e9d0 tx=0x7efe9400ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.434+0000 7fcb1d0fa700 1 -- 172.21.15.40:0/1580170314 wait complete. 2023-11-08T08:19:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7f8a5d7fa700 1 -- 172.21.15.40:0/843745657 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5401c040 con 0x7f8a6006a8e0 2023-11-08T08:19:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7f8a5d7fa700 1 -- 172.21.15.40:0/843745657 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a540038c0 con 0x7f8a6006a8e0 2023-11-08T08:19:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7f8a65fa1700 1 -- 172.21.15.40:0/843745657 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6006a0e0 con 0x7f8a6006a8e0 2023-11-08T08:19:34.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7f8a5d7fa700 1 -- 172.21.15.40:0/843745657 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a54003c30 con 0x7f8a6006a8e0 2023-11-08T08:19:34.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7f8a65fa1700 1 -- 172.21.15.40:0/843745657 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a60069e00 con 0x7f8a6006a8e0 2023-11-08T08:19:34.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7efe8effd700 1 -- 172.21.15.40:0/1326293102 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9400c850 con 0x7efe980705c0 2023-11-08T08:19:34.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7efe8effd700 1 -- 172.21.15.40:0/1326293102 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9400c9b0 con 0x7efe980705c0 2023-11-08T08:19:34.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7efe8effd700 1 -- 172.21.15.40:0/1326293102 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9400cd20 con 0x7efe980705c0 2023-11-08T08:19:34.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7efe9f2cb700 1 -- 172.21.15.40:0/1326293102 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe880094b0 con 0x7efe980705c0 2023-11-08T08:19:34.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7efe9f2cb700 1 -- 172.21.15.40:0/1326293102 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9806b6b0 con 0x7efe980705c0 2023-11-08T08:19:34.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7fbe15409700 1 --2- 172.21.15.40:0/4018486245 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbdf4064ee0 0x7fbdf4067390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c000f60 tx=0x7fbe0c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7fa1767fc700 1 -- 172.21.15.40:0/747681450 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa178010050 con 0x7fa18015beb0 2023-11-08T08:19:34.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7f8a65fa1700 1 -- 172.21.15.40:0/843745657 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8a600c8da0 con 0x7f8a6006a8e0 2023-11-08T08:19:34.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7ff167fff700 1 -- 172.21.15.40:0/655561061 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff17000f050 con 0x7ff1800cdfb0 2023-11-08T08:19:34.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7efe9f2cb700 1 -- 172.21.15.40:0/1326293102 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efe80000fc0 con 0x7efe980705c0 2023-11-08T08:19:34.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7f4b36ffd700 1 -- 172.21.15.40:0/1930274824 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4b3c019030 con 0x7f4b400b6030 2023-11-08T08:19:34.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7f4b36ffd700 1 --2- 172.21.15.40:0/1930274824 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4b24064fb0 0x7f4b24067460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7f4b36ffd700 1 -- 172.21.15.40:0/1930274824 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4b3c081430 con 0x7f4b400b6030 2023-11-08T08:19:34.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7f4b36ffd700 1 --2- 172.21.15.40:0/1930274824 >> [v2:172.21.15.40:6824/35486,v1:172.21.15.40:6825/35486] conn(0x7f4b2406ec80 0x7f4b240710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7f4b36ffd700 1 -- 172.21.15.40:0/1930274824 --> [v2:172.21.15.40:6824/35486,v1:172.21.15.40:6825/35486] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b24071780 con 0x7f4b2406ec80 2023-11-08T08:19:34.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7f4b36ffd700 1 -- 172.21.15.40:0/1930274824 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b3c083a50 con 0x7f4b400b6030 2023-11-08T08:19:34.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.438+0000 7fbe14c08700 1 --2- 172.21.15.40:0/4018486245 >> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] conn(0x7fbdf406ea30 0x7fbdf4070e60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7ff167fff700 1 --2- 172.21.15.40:0/655561061 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff16c064f30 0x7ff16c0673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7fcb1d0fa700 1 Processor -- start 2023-11-08T08:19:34.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7ff167fff700 1 -- 172.21.15.40:0/655561061 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff170021070 con 0x7ff1800cdfb0 2023-11-08T08:19:34.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7f4b45c0f700 1 --2- 172.21.15.40:0/1930274824 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4b24064fb0 0x7f4b24067460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7f4b46c11700 1 --2- 172.21.15.40:0/1930274824 >> [v2:172.21.15.40:6824/35486,v1:172.21.15.40:6825/35486] conn(0x7f4b2406ec80 0x7f4b240710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7ff167fff700 1 --2- 172.21.15.40:0/655561061 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x7ff16c06ec00 0x7ff16c071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7fcb1d0fa700 1 -- start start 2023-11-08T08:19:34.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7ff167fff700 1 -- 172.21.15.40:0/655561061 --> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff16c071700 con 0x7ff16c06ec00 2023-11-08T08:19:34.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7ff167fff700 1 -- 172.21.15.40:0/655561061 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff170085e10 con 0x7ff1800cdfb0 2023-11-08T08:19:34.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7fc9fb7fe700 1 -- 172.21.15.40:0/829629169 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fca04012050 con 0x7fca1415c6d0 2023-11-08T08:19:34.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7fa1767fc700 1 --2- 172.21.15.40:0/747681450 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa164064f10 0x7fa1640673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7fa1767fc700 1 -- 172.21.15.40:0/747681450 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa1780828e0 con 0x7fa18015beb0 2023-11-08T08:19:34.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7fa1767fc700 1 --2- 172.21.15.40:0/747681450 >> [v2:172.21.15.40:6816/35484,v1:172.21.15.40:6817/35484] conn(0x7fa16406ebe0 0x7fa164071010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7fbe05ffb700 1 -- 172.21.15.40:0/4018486245 <== osd.0 v2:172.21.15.40:6808/35482 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbdf4071530 con 0x7fbdf406ea30 2023-11-08T08:19:34.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7fa1767fc700 1 -- 172.21.15.40:0/747681450 --> [v2:172.21.15.40:6816/35484,v1:172.21.15.40:6817/35484] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa1640716e0 con 0x7fa16406ebe0 2023-11-08T08:19:34.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7fa1767fc700 1 -- 172.21.15.40:0/747681450 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa178082c90 con 0x7fa18015beb0 2023-11-08T08:19:34.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7fa185d82700 1 --2- 172.21.15.40:0/747681450 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa164064f10 0x7fa1640673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7efe8effd700 1 -- 172.21.15.40:0/1326293102 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7efe94023070 con 0x7efe980705c0 2023-11-08T08:19:34.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7f4b45c0f700 1 --2- 172.21.15.40:0/1930274824 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4b24064fb0 0x7f4b24067460 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4b30006b40 tx=0x7f4b30006ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7f4b46c11700 1 --2- 172.21.15.40:0/1930274824 >> [v2:172.21.15.40:6824/35486,v1:172.21.15.40:6825/35486] conn(0x7f4b2406ec80 0x7f4b240710b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7f8a5d7fa700 1 -- 172.21.15.40:0/843745657 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f8a5400e040 con 0x7f8a6006a8e0 2023-11-08T08:19:34.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7fc9fb7fe700 1 --2- 172.21.15.40:0/829629169 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fca00064f40 0x7fca000673f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7fc9fb7fe700 1 -- 172.21.15.40:0/829629169 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fca04082c60 con 0x7fca1415c6d0 2023-11-08T08:19:34.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7fc9fb7fe700 1 --2- 172.21.15.40:0/829629169 >> [v2:172.21.15.40:6800/35480,v1:172.21.15.40:6801/35480] conn(0x7fca0006eb10 0x7fca00070f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7fc9fb7fe700 1 -- 172.21.15.40:0/829629169 --> [v2:172.21.15.40:6800/35480,v1:172.21.15.40:6801/35480] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca00071610 con 0x7fca0006eb10 2023-11-08T08:19:34.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7fc9fb7fe700 1 -- 172.21.15.40:0/829629169 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca040830c0 con 0x7fca1415c6d0 2023-11-08T08:19:34.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7fca12d9d700 1 --2- 172.21.15.40:0/829629169 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fca00064f40 0x7fca000673f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7ff17ffff700 1 --2- 172.21.15.40:0/655561061 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x7ff16c06ec00 0x7ff16c071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7ff17f7fe700 1 --2- 172.21.15.40:0/655561061 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff16c064f30 0x7ff16c0673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7efe8effd700 1 --2- 172.21.15.40:0/1326293102 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efe7c064fb0 0x7efe7c067460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fca1259c700 1 --2- 172.21.15.40:0/829629169 >> [v2:172.21.15.40:6800/35480,v1:172.21.15.40:6801/35480] conn(0x7fca0006eb10 0x7fca00070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.442+0000 7fa185581700 1 --2- 172.21.15.40:0/747681450 >> [v2:172.21.15.40:6816/35484,v1:172.21.15.40:6817/35484] conn(0x7fa16406ebe0 0x7fa164071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fa185d82700 1 --2- 172.21.15.40:0/747681450 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa164064f10 0x7fa1640673c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa17000a010 tx=0x7fa170008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7ff17ffff700 1 --2- 172.21.15.40:0/655561061 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x7ff16c06ec00 0x7ff16c071030 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fcb1d0fa700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb040052f0 0x7fcb1808c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fcb1d0fa700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb1808c8f0 0x7fcb1814fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fcb1d0fa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb1808cd20 0x7fcb181503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fcb16d9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb040052f0 0x7fcb1808c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fcb16d9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb040052f0 0x7fcb1808c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33602/0 (socket says 172.21.15.40:33602) 2023-11-08T08:19:34.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7efe8effd700 1 -- 172.21.15.40:0/1326293102 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7efe940824b0 con 0x7efe980705c0 2023-11-08T08:19:34.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7efe9dac8700 1 --2- 172.21.15.40:0/1326293102 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efe7c064fb0 0x7efe7c067460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7efe8effd700 1 --2- 172.21.15.40:0/1326293102 >> [v2:172.21.15.169:6808/35792,v1:172.21.15.169:6810/35792] conn(0x7efe7c06ec00 0x7efe7c071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7efe8effd700 1 -- 172.21.15.40:0/1326293102 --> [v2:172.21.15.169:6808/35792,v1:172.21.15.169:6810/35792] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efe7c071700 con 0x7efe7c06ec00 2023-11-08T08:19:34.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7efe8effd700 1 -- 172.21.15.40:0/1326293102 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe9400ce80 con 0x7efe980705c0 2023-11-08T08:19:34.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fcb16d9d700 1 -- 172.21.15.40:0/1930409545 learned_addr learned my addr 172.21.15.40:0/1930409545 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:34.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fcb0e59c700 1 --2- 172.21.15.40:0/1930409545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb1808c8f0 0x7fcb1814fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fcb1d0fa700 1 -- 172.21.15.40:0/1930409545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1808db00 con 0x7fcb1808cd20 2023-11-08T08:19:34.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fcb1d0fa700 1 -- 172.21.15.40:0/1930409545 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1808d260 con 0x7fcb040052f0 2023-11-08T08:19:34.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fcb1d0fa700 1 -- 172.21.15.40:0/1930409545 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1808d540 con 0x7fcb1808c8f0 2023-11-08T08:19:34.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7efe9eaca700 1 --2- 172.21.15.40:0/1326293102 >> [v2:172.21.15.169:6808/35792,v1:172.21.15.169:6810/35792] conn(0x7efe7c06ec00 0x7efe7c071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fa185581700 1 --2- 172.21.15.40:0/747681450 >> [v2:172.21.15.40:6816/35484,v1:172.21.15.40:6817/35484] conn(0x7fa16406ebe0 0x7fa164071010 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fcb1759e700 1 --2- 172.21.15.40:0/1930409545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb1808cd20 0x7fcb181503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fca12d9d700 1 --2- 172.21.15.40:0/829629169 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fca00064f40 0x7fca000673f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009c60 tx=0x7fca0c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7f4b36ffd700 1 -- 172.21.15.40:0/1930274824 <== osd.2 v2:172.21.15.40:6824/35486 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4b24071780 con 0x7f4b2406ec80 2023-11-08T08:19:34.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7f8a5d7fa700 1 --2- 172.21.15.40:0/843745657 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8a44064fb0 0x7f8a44067460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7f8a5d7fa700 1 -- 172.21.15.40:0/843745657 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8a54083ce0 con 0x7f8a6006a8e0 2023-11-08T08:19:34.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7f8a5d7fa700 1 --2- 172.21.15.40:0/843745657 >> [v2:172.21.15.169:6811/35790,v1:172.21.15.169:6814/35790] conn(0x7f8a4406ec80 0x7f8a440710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7f8a5d7fa700 1 -- 172.21.15.40:0/843745657 --> [v2:172.21.15.169:6811/35790,v1:172.21.15.169:6814/35790] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8a44071780 con 0x7f8a4406ec80 2023-11-08T08:19:34.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7f8a5d7fa700 1 -- 172.21.15.40:0/843745657 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a54086510 con 0x7f8a6006a8e0 2023-11-08T08:19:34.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7f8a5ffff700 1 --2- 172.21.15.40:0/843745657 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8a44064fb0 0x7f8a44067460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7ff17f7fe700 1 --2- 172.21.15.40:0/655561061 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff16c064f30 0x7ff16c0673e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff1800c8c20 tx=0x7ff17800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7efe9dac8700 1 --2- 172.21.15.40:0/1326293102 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efe7c064fb0 0x7efe7c067460 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efe880090b0 tx=0x7efe88008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7ff167fff700 1 -- 172.21.15.40:0/655561061 <== osd.5 v2:172.21.15.169:6809/35793 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff16c071700 con 0x7ff16c06ec00 2023-11-08T08:19:34.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fcb1759e700 1 -- 172.21.15.40:0/1930409545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb1808c8f0 msgr2=0x7fcb1814fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fcb1759e700 1 --2- 172.21.15.40:0/1930409545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb1808c8f0 0x7fcb1814fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fcb1759e700 1 -- 172.21.15.40:0/1930409545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb040052f0 msgr2=0x7fcb1808c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fcb1759e700 1 --2- 172.21.15.40:0/1930409545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb040052f0 0x7fcb1808c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7efe9eaca700 1 --2- 172.21.15.40:0/1326293102 >> [v2:172.21.15.169:6808/35792,v1:172.21.15.169:6810/35792] conn(0x7efe7c06ec00 0x7efe7c071030 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fcb1759e700 1 -- 172.21.15.40:0/1930409545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1815cd10 con 0x7fcb1808cd20 2023-11-08T08:19:34.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fca1259c700 1 --2- 172.21.15.40:0/829629169 >> [v2:172.21.15.40:6800/35480,v1:172.21.15.40:6801/35480] conn(0x7fca0006eb10 0x7fca00070f40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fcb1759e700 1 --2- 172.21.15.40:0/1930409545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb1808cd20 0x7fcb181503c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcb18160cf0 tx=0x7fcb100014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7f8a657a0700 1 --2- 172.21.15.40:0/843745657 >> [v2:172.21.15.169:6811/35790,v1:172.21.15.169:6814/35790] conn(0x7f8a4406ec80 0x7f8a440710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7f8a5ffff700 1 --2- 172.21.15.40:0/843745657 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8a44064fb0 0x7f8a44067460 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8a600c8d40 tx=0x7f8a4c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fa1767fc700 1 -- 172.21.15.40:0/747681450 <== osd.3 v2:172.21.15.40:6816/35484 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa1640716e0 con 0x7fa16406ebe0 2023-11-08T08:19:34.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fcb0ffff700 1 -- 172.21.15.40:0/1930409545 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10022070 con 0x7fcb1808cd20 2023-11-08T08:19:34.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.446+0000 7fcb0ffff700 1 -- 172.21.15.40:0/1930409545 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb10003cb0 con 0x7fcb1808cd20 2023-11-08T08:19:34.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.450+0000 7fcb0ffff700 1 -- 172.21.15.40:0/1930409545 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10005270 con 0x7fcb1808cd20 2023-11-08T08:19:34.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.450+0000 7fcb1d0fa700 1 -- 172.21.15.40:0/1930409545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb1815ca30 con 0x7fcb1808cd20 2023-11-08T08:19:34.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.450+0000 7fcb1d0fa700 1 -- 172.21.15.40:0/1930409545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb180b7f10 con 0x7fcb1808cd20 2023-11-08T08:19:34.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.450+0000 7fcb1d0fa700 1 -- 172.21.15.40:0/1930409545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcb180726c0 con 0x7fcb1808cd20 2023-11-08T08:19:34.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.450+0000 7efe8effd700 1 -- 172.21.15.40:0/1326293102 <== osd.7 v2:172.21.15.169:6808/35792 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7efe7c071700 con 0x7efe7c06ec00 2023-11-08T08:19:34.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.450+0000 7f8a657a0700 1 --2- 172.21.15.40:0/843745657 >> [v2:172.21.15.169:6811/35790,v1:172.21.15.169:6814/35790] conn(0x7f8a4406ec80 0x7f8a440710b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.450+0000 7fcb0ffff700 1 -- 172.21.15.40:0/1930409545 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fcb10010050 con 0x7fcb1808cd20 2023-11-08T08:19:34.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.454+0000 7fc9fb7fe700 1 -- 172.21.15.40:0/829629169 <== osd.1 v2:172.21.15.40:6800/35480 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fca00071610 con 0x7fca0006eb10 2023-11-08T08:19:34.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.454+0000 7fcb0ffff700 1 --2- 172.21.15.40:0/1930409545 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcaf8064f10 0x7fcaf80673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.454+0000 7fcb0ffff700 1 -- 172.21.15.40:0/1930409545 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fcb10083590 con 0x7fcb1808cd20 2023-11-08T08:19:34.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.454+0000 7f8a5d7fa700 1 -- 172.21.15.40:0/843745657 <== osd.4 v2:172.21.15.169:6811/35790 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8a44071780 con 0x7f8a4406ec80 2023-11-08T08:19:34.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.454+0000 7fcb0ffff700 1 --2- 172.21.15.40:0/1930409545 >> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] conn(0x7fcaf806eb60 0x7fcaf8070f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.454+0000 7fcb0ffff700 1 -- 172.21.15.40:0/1930409545 --> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcaf8071660 con 0x7fcaf806eb60 2023-11-08T08:19:34.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.454+0000 7fcb0ffff700 1 -- 172.21.15.40:0/1930409545 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb10083900 con 0x7fcb1808cd20 2023-11-08T08:19:34.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.454+0000 7fcb0e59c700 1 --2- 172.21.15.40:0/1930409545 >> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] conn(0x7fcaf806eb60 0x7fcaf8070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.454+0000 7fcb16d9d700 1 --2- 172.21.15.40:0/1930409545 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcaf8064f10 0x7fcaf80673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.454+0000 7fcb16d9d700 1 --2- 172.21.15.40:0/1930409545 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcaf8064f10 0x7fcaf80673c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800a600 tx=0x7fcb08008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.458+0000 7fcb0e59c700 1 --2- 172.21.15.40:0/1930409545 >> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] conn(0x7fcaf806eb60 0x7fcaf8070f90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.458+0000 7fcb0ffff700 1 -- 172.21.15.40:0/1930409545 <== osd.6 v2:172.21.15.169:6800/35791 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fcaf8071660 con 0x7fcaf806eb60 2023-11-08T08:19:34.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.470+0000 7fbe1640b700 1 -- 172.21.15.40:0/4018486245 --> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbdf0002d60 con 0x7fbdf406ea30 2023-11-08T08:19:34.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.474+0000 7fbe05ffb700 1 -- 172.21.15.40:0/4018486245 <== osd.0 v2:172.21.15.40:6808/35482 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbdf0002d60 con 0x7fbdf406ea30 2023-11-08T08:19:34.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7fbe1640b700 1 -- 172.21.15.40:0/4018486245 >> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] conn(0x7fbdf406ea30 msgr2=0x7fbdf4070e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7fbe1640b700 1 --2- 172.21.15.40:0/4018486245 >> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] conn(0x7fbdf406ea30 0x7fbdf4070e60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7fbe1640b700 1 -- 172.21.15.40:0/4018486245 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbdf4064ee0 msgr2=0x7fbdf4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7fbe1640b700 1 --2- 172.21.15.40:0/4018486245 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbdf4064ee0 0x7fbdf4067390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c000f60 tx=0x7fbe0c009040 comp rx=0 tx=0).stop 2023-11-08T08:19:34.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7fbe1640b700 1 -- 172.21.15.40:0/4018486245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe080e29f0 msgr2=0x7fbe0804f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7fbe1640b700 1 --2- 172.21.15.40:0/4018486245 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe080e29f0 0x7fbe0804f250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe10063810 tx=0x7fbe10072ce0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7fbe15c0a700 1 -- 172.21.15.40:0/4018486245 reap_dead start 2023-11-08T08:19:34.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7fbe1640b700 1 -- 172.21.15.40:0/4018486245 shutdown_connections 2023-11-08T08:19:34.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7fbe1640b700 1 -- 172.21.15.40:0/4018486245 >> 172.21.15.40:0/4018486245 conn(0x7fbe08055c20 msgr2=0x7fbe0803c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7fa187fe6700 1 -- 172.21.15.40:0/747681450 --> [v2:172.21.15.40:6816/35484,v1:172.21.15.40:6817/35484] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa18014f720 con 0x7fa16406ebe0 2023-11-08T08:19:34.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7fbe1640b700 1 -- 172.21.15.40:0/4018486245 shutdown_connections 2023-11-08T08:19:34.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7fbe1640b700 1 -- 172.21.15.40:0/4018486245 wait complete. 2023-11-08T08:19:34.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7efe9f2cb700 1 -- 172.21.15.40:0/1326293102 --> [v2:172.21.15.169:6808/35792,v1:172.21.15.169:6810/35792] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efe80002d60 con 0x7efe7c06ec00 2023-11-08T08:19:34.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7f8a65fa1700 1 -- 172.21.15.40:0/843745657 --> [v2:172.21.15.169:6811/35790,v1:172.21.15.169:6814/35790] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8a60073cb0 con 0x7f8a4406ec80 2023-11-08T08:19:34.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7ff185ba7700 1 -- 172.21.15.40:0/655561061 --> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff1800726c0 con 0x7ff16c06ec00 2023-11-08T08:19:34.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7fa1767fc700 1 -- 172.21.15.40:0/747681450 <== osd.3 v2:172.21.15.40:6816/35484 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa18014f720 con 0x7fa16406ebe0 2023-11-08T08:19:34.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7efe8effd700 1 -- 172.21.15.40:0/1326293102 <== osd.7 v2:172.21.15.169:6808/35792 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efe80002d60 con 0x7efe7c06ec00 2023-11-08T08:19:34.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7f4b48674700 1 -- 172.21.15.40:0/1930274824 --> [v2:172.21.15.40:6824/35486,v1:172.21.15.40:6825/35486] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4b400726c0 con 0x7f4b2406ec80 2023-11-08T08:19:34.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7fa163fff700 1 -- 172.21.15.40:0/747681450 >> [v2:172.21.15.40:6816/35484,v1:172.21.15.40:6817/35484] conn(0x7fa16406ebe0 msgr2=0x7fa164071010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7fa163fff700 1 --2- 172.21.15.40:0/747681450 >> [v2:172.21.15.40:6816/35484,v1:172.21.15.40:6817/35484] conn(0x7fa16406ebe0 0x7fa164071010 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.478+0000 7fa163fff700 1 -- 172.21.15.40:0/747681450 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa164064f10 msgr2=0x7fa1640673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7efe9f2cb700 1 -- 172.21.15.40:0/1326293102 >> [v2:172.21.15.169:6808/35792,v1:172.21.15.169:6810/35792] conn(0x7efe7c06ec00 msgr2=0x7efe7c071030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7efe9f2cb700 1 --2- 172.21.15.40:0/1326293102 >> [v2:172.21.15.169:6808/35792,v1:172.21.15.169:6810/35792] conn(0x7efe7c06ec00 0x7efe7c071030 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7f4b36ffd700 1 -- 172.21.15.40:0/1930274824 <== osd.2 v2:172.21.15.40:6824/35486 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4b400726c0 con 0x7f4b2406ec80 2023-11-08T08:19:34.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7fa163fff700 1 --2- 172.21.15.40:0/747681450 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa164064f10 0x7fa1640673c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa17000a010 tx=0x7fa170008030 comp rx=0 tx=0).stop 2023-11-08T08:19:34.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7fa163fff700 1 -- 172.21.15.40:0/747681450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa18015beb0 msgr2=0x7fa1800c26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7fa163fff700 1 --2- 172.21.15.40:0/747681450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa18015beb0 0x7fa1800c26c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa17800cb90 tx=0x7fa17800cea0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7fa186583700 1 -- 172.21.15.40:0/747681450 reap_dead start 2023-11-08T08:19:34.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7efe9f2cb700 1 -- 172.21.15.40:0/1326293102 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efe7c064fb0 msgr2=0x7efe7c067460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7efe9f2cb700 1 --2- 172.21.15.40:0/1326293102 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efe7c064fb0 0x7efe7c067460 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efe880090b0 tx=0x7efe88008030 comp rx=0 tx=0).stop 2023-11-08T08:19:34.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7f8a5d7fa700 1 -- 172.21.15.40:0/843745657 <== osd.4 v2:172.21.15.169:6811/35790 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8a60073cb0 con 0x7f8a4406ec80 2023-11-08T08:19:34.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7efe9f2cb700 1 -- 172.21.15.40:0/1326293102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe980705c0 msgr2=0x7efe9815b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7efe9f2cb700 1 --2- 172.21.15.40:0/1326293102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe980705c0 0x7efe9815b0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efe9400e9d0 tx=0x7efe9400ece0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7efe9eaca700 1 -- 172.21.15.40:0/1326293102 reap_dead start 2023-11-08T08:19:34.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7fa163fff700 1 -- 172.21.15.40:0/747681450 shutdown_connections 2023-11-08T08:19:34.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7fa163fff700 1 -- 172.21.15.40:0/747681450 >> 172.21.15.40:0/747681450 conn(0x7fa1800ce6c0 msgr2=0x7fa180156710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7efe9f2cb700 1 -- 172.21.15.40:0/1326293102 shutdown_connections 2023-11-08T08:19:34.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7efe9f2cb700 1 -- 172.21.15.40:0/1326293102 >> 172.21.15.40:0/1326293102 conn(0x7efe980b0f80 msgr2=0x7efe9815bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7efe9f2cb700 1 -- 172.21.15.40:0/1326293102 shutdown_connections 2023-11-08T08:19:34.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7efe9f2cb700 1 -- 172.21.15.40:0/1326293102 wait complete. 2023-11-08T08:19:34.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7fa163fff700 1 -- 172.21.15.40:0/747681450 shutdown_connections 2023-11-08T08:19:34.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7f4b48674700 1 -- 172.21.15.40:0/1930274824 >> [v2:172.21.15.40:6824/35486,v1:172.21.15.40:6825/35486] conn(0x7f4b2406ec80 msgr2=0x7f4b240710b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7ff167fff700 1 -- 172.21.15.40:0/655561061 <== osd.5 v2:172.21.15.169:6809/35793 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff1800726c0 con 0x7ff16c06ec00 2023-11-08T08:19:34.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7f4b48674700 1 --2- 172.21.15.40:0/1930274824 >> [v2:172.21.15.40:6824/35486,v1:172.21.15.40:6825/35486] conn(0x7f4b2406ec80 0x7f4b240710b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7fa163fff700 1 -- 172.21.15.40:0/747681450 wait complete. 2023-11-08T08:19:34.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7f4b48674700 1 -- 172.21.15.40:0/1930274824 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4b24064fb0 msgr2=0x7f4b24067460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7f4b48674700 1 --2- 172.21.15.40:0/1930274824 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4b24064fb0 0x7f4b24067460 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4b30006b40 tx=0x7f4b30006ad0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7f4b48674700 1 -- 172.21.15.40:0/1930274824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b400b6030 msgr2=0x7f4b4008d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7f4b48674700 1 --2- 172.21.15.40:0/1930274824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b400b6030 0x7f4b4008d650 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00c500 tx=0x7f4b3c00c8c0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7f4b46c11700 1 -- 172.21.15.40:0/1930274824 reap_dead start 2023-11-08T08:19:34.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7ff185ba7700 1 -- 172.21.15.40:0/655561061 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x7ff16c06ec00 msgr2=0x7ff16c071030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7f4b48674700 1 -- 172.21.15.40:0/1930274824 shutdown_connections 2023-11-08T08:19:34.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7f4b48674700 1 -- 172.21.15.40:0/1930274824 >> 172.21.15.40:0/1930274824 conn(0x7f4b4006bfc0 msgr2=0x7f4b4006a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7ff185ba7700 1 --2- 172.21.15.40:0/655561061 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x7ff16c06ec00 0x7ff16c071030 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7ff185ba7700 1 -- 172.21.15.40:0/655561061 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff16c064f30 msgr2=0x7ff16c0673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7ff185ba7700 1 --2- 172.21.15.40:0/655561061 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff16c064f30 0x7ff16c0673e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff1800c8c20 tx=0x7ff17800b040 comp rx=0 tx=0).stop 2023-11-08T08:19:34.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7ff185ba7700 1 -- 172.21.15.40:0/655561061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1800cdfb0 msgr2=0x7ff1800c43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7ff185ba7700 1 --2- 172.21.15.40:0/655561061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1800cdfb0 0x7ff1800c43d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff170000c00 tx=0x7ff170004770 comp rx=0 tx=0).stop 2023-11-08T08:19:34.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7f8a65fa1700 1 -- 172.21.15.40:0/843745657 >> [v2:172.21.15.169:6811/35790,v1:172.21.15.169:6814/35790] conn(0x7f8a4406ec80 msgr2=0x7f8a440710b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7ff17ffff700 1 -- 172.21.15.40:0/655561061 reap_dead start 2023-11-08T08:19:34.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7ff185ba7700 1 -- 172.21.15.40:0/655561061 shutdown_connections 2023-11-08T08:19:34.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7ff185ba7700 1 -- 172.21.15.40:0/655561061 >> 172.21.15.40:0/655561061 conn(0x7ff1800cf350 msgr2=0x7ff180164210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7fca189f1700 1 -- 172.21.15.40:0/829629169 --> [v2:172.21.15.40:6800/35480,v1:172.21.15.40:6801/35480] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fca140731c0 con 0x7fca0006eb10 2023-11-08T08:19:34.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7f4b48674700 1 -- 172.21.15.40:0/1930274824 shutdown_connections 2023-11-08T08:19:34.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7f4b48674700 1 -- 172.21.15.40:0/1930274824 wait complete. 2023-11-08T08:19:34.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7f8a65fa1700 1 --2- 172.21.15.40:0/843745657 >> [v2:172.21.15.169:6811/35790,v1:172.21.15.169:6814/35790] conn(0x7f8a4406ec80 0x7f8a440710b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7f8a65fa1700 1 -- 172.21.15.40:0/843745657 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8a44064fb0 msgr2=0x7f8a44067460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7f8a65fa1700 1 --2- 172.21.15.40:0/843745657 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8a44064fb0 0x7f8a44067460 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8a600c8d40 tx=0x7f8a4c009040 comp rx=0 tx=0).stop 2023-11-08T08:19:34.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7f8a65fa1700 1 -- 172.21.15.40:0/843745657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a6006a8e0 msgr2=0x7f8a601ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.482+0000 7f8a65fa1700 1 --2- 172.21.15.40:0/843745657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a6006a8e0 0x7f8a601ff580 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8a54017000 tx=0x7f8a54004770 comp rx=0 tx=0).stop 2023-11-08T08:19:34.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.486+0000 7f8a657a0700 1 -- 172.21.15.40:0/843745657 reap_dead start 2023-11-08T08:19:34.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.486+0000 7f8a65fa1700 1 -- 172.21.15.40:0/843745657 shutdown_connections 2023-11-08T08:19:34.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.486+0000 7f8a65fa1700 1 -- 172.21.15.40:0/843745657 >> 172.21.15.40:0/843745657 conn(0x7f8a600b0f90 msgr2=0x7f8a600ae7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.486+0000 7fc9fb7fe700 1 -- 172.21.15.40:0/829629169 <== osd.1 v2:172.21.15.40:6800/35480 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fca140731c0 con 0x7fca0006eb10 2023-11-08T08:19:34.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.486+0000 7f8a65fa1700 1 -- 172.21.15.40:0/843745657 shutdown_connections 2023-11-08T08:19:34.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.486+0000 7f8a65fa1700 1 -- 172.21.15.40:0/843745657 wait complete. 2023-11-08T08:19:34.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.486+0000 7fcb1d0fa700 1 -- 172.21.15.40:0/1930409545 --> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcb1806b480 con 0x7fcaf806eb60 2023-11-08T08:19:34.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.486+0000 7ff185ba7700 1 -- 172.21.15.40:0/655561061 shutdown_connections 2023-11-08T08:19:34.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.486+0000 7fca189f1700 1 -- 172.21.15.40:0/829629169 >> [v2:172.21.15.40:6800/35480,v1:172.21.15.40:6801/35480] conn(0x7fca0006eb10 msgr2=0x7fca00070f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.486+0000 7fca189f1700 1 --2- 172.21.15.40:0/829629169 >> [v2:172.21.15.40:6800/35480,v1:172.21.15.40:6801/35480] conn(0x7fca0006eb10 0x7fca00070f40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.486+0000 7ff185ba7700 1 -- 172.21.15.40:0/655561061 wait complete. 2023-11-08T08:19:34.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.486+0000 7fcb0ffff700 1 -- 172.21.15.40:0/1930409545 <== osd.6 v2:172.21.15.169:6800/35791 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcb1806b480 con 0x7fcaf806eb60 2023-11-08T08:19:34.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.486+0000 7fca189f1700 1 -- 172.21.15.40:0/829629169 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fca00064f40 msgr2=0x7fca000673f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.486+0000 7fca189f1700 1 --2- 172.21.15.40:0/829629169 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fca00064f40 0x7fca000673f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009c60 tx=0x7fca0c008030 comp rx=0 tx=0).stop 2023-11-08T08:19:34.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.486+0000 7fca189f1700 1 -- 172.21.15.40:0/829629169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca1415c6d0 msgr2=0x7fca1415f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.486+0000 7fca189f1700 1 --2- 172.21.15.40:0/829629169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca1415c6d0 0x7fca1415f550 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fca04007d90 tx=0x7fca0400e3c0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.486+0000 7fca1359e700 1 -- 172.21.15.40:0/829629169 reap_dead start 2023-11-08T08:19:34.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.486+0000 7fcb0d59a700 1 -- 172.21.15.40:0/1930409545 >> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] conn(0x7fcaf806eb60 msgr2=0x7fcaf8070f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.486+0000 7fcb0d59a700 1 --2- 172.21.15.40:0/1930409545 >> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] conn(0x7fcaf806eb60 0x7fcaf8070f90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.490+0000 7fcb0d59a700 1 -- 172.21.15.40:0/1930409545 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcaf8064f10 msgr2=0x7fcaf80673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.490+0000 7fcb0d59a700 1 --2- 172.21.15.40:0/1930409545 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcaf8064f10 0x7fcaf80673c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800a600 tx=0x7fcb08008040 comp rx=0 tx=0).stop 2023-11-08T08:19:34.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.490+0000 7fcb0d59a700 1 -- 172.21.15.40:0/1930409545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb1808cd20 msgr2=0x7fcb181503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.490+0000 7fcb0d59a700 1 --2- 172.21.15.40:0/1930409545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb1808cd20 0x7fcb181503c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcb18160cf0 tx=0x7fcb100014a0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.490+0000 7fcb1759e700 1 -- 172.21.15.40:0/1930409545 reap_dead start 2023-11-08T08:19:34.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.490+0000 7fcb0d59a700 1 -- 172.21.15.40:0/1930409545 shutdown_connections 2023-11-08T08:19:34.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.490+0000 7fcb0d59a700 1 -- 172.21.15.40:0/1930409545 >> 172.21.15.40:0/1930409545 conn(0x7fcb1806bfc0 msgr2=0x7fcb180d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.490+0000 7fca189f1700 1 -- 172.21.15.40:0/829629169 shutdown_connections 2023-11-08T08:19:34.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.490+0000 7fca189f1700 1 -- 172.21.15.40:0/829629169 >> 172.21.15.40:0/829629169 conn(0x7fca140b6520 msgr2=0x7fca140b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.490+0000 7fcb0d59a700 1 -- 172.21.15.40:0/1930409545 shutdown_connections 2023-11-08T08:19:34.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.490+0000 7fcb0d59a700 1 -- 172.21.15.40:0/1930409545 wait complete. 2023-11-08T08:19:34.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.490+0000 7fca189f1700 1 -- 172.21.15.40:0/829629169 shutdown_connections 2023-11-08T08:19:34.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.490+0000 7fca189f1700 1 -- 172.21.15.40:0/829629169 wait complete. 2023-11-08T08:19:34.632 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2023-11-08T08:19:34.632 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-08T08:19:34.633 INFO:teuthology.orchestra.run.smithi040.stdout:55834574860 2023-11-08T08:19:34.634 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-08T08:19:34.635 INFO:teuthology.orchestra.run.smithi040.stdout:55834574860 2023-11-08T08:19:34.635 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-08T08:19:34.636 INFO:teuthology.orchestra.run.smithi040.stdout:55834574858 2023-11-08T08:19:34.636 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-08T08:19:34.637 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2023-11-08T08:19:34.638 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-08T08:19:34.639 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2023-11-08T08:19:34.639 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-08T08:19:34.640 INFO:teuthology.orchestra.run.smithi040.stdout:55834574860 2023-11-08T08:19:34.640 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-08T08:19:34.641 INFO:teuthology.orchestra.run.smithi040.stdout:55834574858 2023-11-08T08:19:34.642 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-08T08:19:34.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.914+0000 7fd50e9b7700 1 Processor -- start 2023-11-08T08:19:34.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.914+0000 7fd50e9b7700 1 -- start start 2023-11-08T08:19:34.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.918+0000 7f4aaaf62700 1 Processor -- start 2023-11-08T08:19:34.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.918+0000 7fd50e9b7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000e6d60 0x7fd500006d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.918+0000 7fd50e9b7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd500007870 0x7fd500054cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.918+0000 7fd50e9b7700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd500007cc0 con 0x7fd50003ac20 2023-11-08T08:19:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.918+0000 7fd50e9b7700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd500055ab0 con 0x7fd500007870 2023-11-08T08:19:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.918+0000 7fd50e9b7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000554f0 con 0x7fd5000e6d60 2023-11-08T08:19:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.918+0000 7fd50d1b4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd500007870 0x7fd500054cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.918+0000 7fd50d1b4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd500007870 0x7fd500054cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52320/0 (socket says 172.21.15.40:52320) 2023-11-08T08:19:34.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.918+0000 7fd50d1b4700 1 -- 172.21.15.40:0/3556164324 learned_addr learned my addr 172.21.15.40:0/3556164324 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:34.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.918+0000 7fd50d9b5700 1 --2- 172.21.15.40:0/3556164324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000e6d60 0x7fd500006d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.918+0000 7fd50d1b4700 1 -- 172.21.15.40:0/3556164324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000e6d60 msgr2=0x7fd500006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:34.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.918+0000 7fd50d1b4700 1 --2- 172.21.15.40:0/3556164324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000e6d60 0x7fd500006d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.918+0000 7fd50d1b4700 1 -- 172.21.15.40:0/3556164324 >> v1:172.21.15.169:6789/0 conn(0x7fd50003ac20 legacy=0x7fd5000ea9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:34.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.918+0000 7fd50d1b4700 1 -- 172.21.15.40:0/3556164324 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5000557d0 con 0x7fd500007870 2023-11-08T08:19:34.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.918+0000 7f4aaaf62700 1 -- start start 2023-11-08T08:19:34.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7fd50d1b4700 1 --2- 172.21.15.40:0/3556164324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd500007870 0x7fd500054cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd504009910 tx=0x7fd504009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac9dc1129fa12e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7fd50c9b3700 1 -- 172.21.15.40:0/3556164324 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd504004230 con 0x7fd500007870 2023-11-08T08:19:34.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7fd50c9b3700 1 -- 172.21.15.40:0/3556164324 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd504004390 con 0x7fd500007870 2023-11-08T08:19:34.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7f4aaaf62700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a9c050420 0x7f4a9c0e30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7fd50c9b3700 1 -- 172.21.15.40:0/3556164324 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd504009e60 con 0x7fd500007870 2023-11-08T08:19:34.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7f4aaaf62700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4a9c0340a0 0x7f4a9c036810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7f4aaaf62700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4a9c036d50 0x7f4a9c0dfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7f4aaaf62700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a9c0507f0 con 0x7f4a9c036d50 2023-11-08T08:19:34.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7f4aaaf62700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a9c0371d0 con 0x7f4a9c0340a0 2023-11-08T08:19:34.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7f4aaaf62700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c0e3fd0 con 0x7f4a9c050420 2023-11-08T08:19:34.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7f4aa8cfe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4a9c036d50 0x7f4a9c0dfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7f4aa8cfe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4a9c036d50 0x7f4a9c0dfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52328/0 (socket says 172.21.15.40:52328) 2023-11-08T08:19:34.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7fd50e9b7700 1 -- 172.21.15.40:0/3556164324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd500007870 msgr2=0x7fd500054cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7fd50e9b7700 1 --2- 172.21.15.40:0/3556164324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd500007870 0x7fd500054cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd504009910 tx=0x7fd504009c20 comp rx=0 tx=0).stop 2023-11-08T08:19:34.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7f4aa8cfe700 1 -- 172.21.15.40:0/3129338762 learned_addr learned my addr 172.21.15.40:0/3129338762 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:34.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7fd50e9b7700 1 -- 172.21.15.40:0/3556164324 shutdown_connections 2023-11-08T08:19:34.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7fd50e9b7700 1 --2- 172.21.15.40:0/3556164324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd500007870 0x7fd500054cc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7fd50e9b7700 1 --2- 172.21.15.40:0/3556164324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000e6d60 0x7fd500006d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7fd50e9b7700 1 -- 172.21.15.40:0/3556164324 >> 172.21.15.40:0/3556164324 conn(0x7fd500055c20 msgr2=0x7fd500056020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7f98b7ea7700 1 Processor -- start 2023-11-08T08:19:34.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7f4aa9d00700 1 --2- 172.21.15.40:0/3129338762 >> v2:172.21.15.169:3300/0 conn(0x7f4a9c0340a0 0x7f4a9c036810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7fd50e9b7700 1 -- 172.21.15.40:0/3556164324 shutdown_connections 2023-11-08T08:19:34.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7f4aa3fff700 1 --2- 172.21.15.40:0/3129338762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a9c050420 0x7f4a9c0e30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7f4aa8cfe700 1 -- 172.21.15.40:0/3129338762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a9c050420 msgr2=0x7f4a9c0e30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:34.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.926+0000 7f98b7ea7700 1 -- start start 2023-11-08T08:19:34.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7f4aa8cfe700 1 --2- 172.21.15.40:0/3129338762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a9c050420 0x7f4a9c0e30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7f4aa8cfe700 1 -- 172.21.15.40:0/3129338762 >> v2:172.21.15.169:3300/0 conn(0x7f4a9c0340a0 msgr2=0x7f4a9c036810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:34.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7f4aa8cfe700 1 --2- 172.21.15.40:0/3129338762 >> v2:172.21.15.169:3300/0 conn(0x7f4a9c0340a0 0x7f4a9c036810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7f4aa8cfe700 1 -- 172.21.15.40:0/3129338762 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9c0e3730 con 0x7f4a9c036d50 2023-11-08T08:19:34.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.926+0000 7f4aa8cfe700 1 --2- 172.21.15.40:0/3129338762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a9c036d50 0x7f4a9c0dfa80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c037980 tx=0x7f4a90009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1805e23cf478f6ed server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.926+0000 7f4aa37fe700 1 -- 172.21.15.40:0/3129338762 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90014e90 con 0x7f4a9c036d50 2023-11-08T08:19:34.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.922+0000 7fd50e9b7700 1 -- 172.21.15.40:0/3556164324 wait complete. 2023-11-08T08:19:34.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.926+0000 7f4aa37fe700 1 -- 172.21.15.40:0/3129338762 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9001c440 con 0x7f4a9c036d50 2023-11-08T08:19:34.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.926+0000 7f98b7ea7700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f98a803b560 0x7f98a8052050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.926+0000 7f4aa37fe700 1 -- 172.21.15.40:0/3129338762 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9001c7d0 con 0x7f4a9c036d50 2023-11-08T08:19:34.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.926+0000 7fd50e9b7700 1 Processor -- start 2023-11-08T08:19:34.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.926+0000 7f98b7ea7700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a80e3b40 con 0x7f98a8037f90 2023-11-08T08:19:34.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.926+0000 7f98b7ea7700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a80e4070 con 0x7f98a80558a0 2023-11-08T08:19:34.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.926+0000 7f98b7ea7700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a8035690 con 0x7f98a803b560 2023-11-08T08:19:34.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f98b5442700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f98a80558a0 0x7f98a8037880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:38242/0 (socket says 172.21.15.40:38242) 2023-11-08T08:19:34.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f98b5442700 1 -- 172.21.15.40:0/3771114931 learned_addr learned my addr 172.21.15.40:0/3771114931 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:34.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.926+0000 7f98b5c43700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f98a803b560 0x7f98a8052050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.926+0000 7f4aaaf62700 1 -- 172.21.15.40:0/3129338762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a9c036d50 msgr2=0x7f4a9c0dfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.926+0000 7f4aaaf62700 1 --2- 172.21.15.40:0/3129338762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a9c036d50 0x7f4a9c0dfa80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c037980 tx=0x7f4a90009bf0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f4aaaf62700 1 -- 172.21.15.40:0/3129338762 shutdown_connections 2023-11-08T08:19:34.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f4aaaf62700 1 --2- 172.21.15.40:0/3129338762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a9c036d50 0x7f4a9c0dfa80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f4aaaf62700 1 --2- 172.21.15.40:0/3129338762 >> v2:172.21.15.169:3300/0 conn(0x7f4a9c0340a0 0x7f4a9c036810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f4aaaf62700 1 --2- 172.21.15.40:0/3129338762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a9c050420 0x7f4a9c0e30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f4aaaf62700 1 -- 172.21.15.40:0/3129338762 >> 172.21.15.40:0/3129338762 conn(0x7f4a9c051810 msgr2=0x7f4a9c051be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f4aaaf62700 1 -- 172.21.15.40:0/3129338762 shutdown_connections 2023-11-08T08:19:34.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f4aaaf62700 1 -- 172.21.15.40:0/3129338762 wait complete. 2023-11-08T08:19:34.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.926+0000 7fd50e9b7700 1 -- start start 2023-11-08T08:19:34.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7fd50e9b7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd500007870 0x7fd50003a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7fd50e9b7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd50003ac20 0x7fd500184da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7fd50e9b7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5000e6d60 0x7fd500051d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7fd50e9b7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50003d150 con 0x7fd5000e6d60 2023-11-08T08:19:34.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f98b4c41700 1 -- 172.21.15.40:0/3771114931 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2519288838 0 0) 0x7f98a80e4070 con 0x7f98a80558a0 2023-11-08T08:19:34.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7fd50e9b7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50003a6f0 con 0x7fd500007870 2023-11-08T08:19:34.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7fd50e9b7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd500050d90 con 0x7fd50003ac20 2023-11-08T08:19:34.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f98b4c41700 1 -- 172.21.15.40:0/3771114931 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f989c0035d0 con 0x7f98a80558a0 2023-11-08T08:19:34.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f13d759e700 1 Processor -- start 2023-11-08T08:19:34.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f13d759e700 1 -- start start 2023-11-08T08:19:34.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7fd50e1b6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5000e6d60 0x7fd500051d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7fd50e1b6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5000e6d60 0x7fd500051d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52338/0 (socket says 172.21.15.40:52338) 2023-11-08T08:19:34.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f98b5c43700 1 -- 172.21.15.40:0/3771114931 >> v1:172.21.15.169:6789/0 conn(0x7f98a80558a0 legacy=0x7f98a8037880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f98b5c43700 1 -- 172.21.15.40:0/3771114931 >> v1:172.21.15.40:6789/0 conn(0x7f98a8037f90 legacy=0x7f98a8053d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f98b5c43700 1 -- 172.21.15.40:0/3771114931 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a8050bd0 con 0x7f98a803b560 2023-11-08T08:19:34.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7fd50e1b6700 1 -- 172.21.15.40:0/555015670 learned_addr learned my addr 172.21.15.40:0/555015670 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:34.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7fd50d1b4700 1 --2- 172.21.15.40:0/555015670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd50003ac20 0x7fd500184da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7fd50d9b5700 1 --2- 172.21.15.40:0/555015670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd500007870 0x7fd50003a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f0758f90700 1 Processor -- start 2023-11-08T08:19:34.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f98b5c43700 1 --2- 172.21.15.40:0/3771114931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98a803b560 0x7f98a8052050 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009850 tx=0x7f98ac009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64e5ffc25c5de3d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7fb0476ee700 1 Processor -- start 2023-11-08T08:19:34.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f0758f90700 1 -- start start 2023-11-08T08:19:34.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f4aaaf62700 1 Processor -- start 2023-11-08T08:19:34.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7fd50d1b4700 1 -- 172.21.15.40:0/555015670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd500007870 msgr2=0x7fd50003a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7fd50d1b4700 1 --2- 172.21.15.40:0/555015670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd500007870 0x7fd50003a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7fd50d1b4700 1 -- 172.21.15.40:0/555015670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5000e6d60 msgr2=0x7fd500051d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7fd50d1b4700 1 --2- 172.21.15.40:0/555015670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5000e6d60 0x7fd500051d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7fd50d1b4700 1 -- 172.21.15.40:0/555015670 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd504009570 con 0x7fd50003ac20 2023-11-08T08:19:34.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f98b4c41700 1 -- 172.21.15.40:0/3771114931 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac004230 con 0x7f98a803b560 2023-11-08T08:19:34.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f98b4c41700 1 -- 172.21.15.40:0/3771114931 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ac004390 con 0x7f98a803b560 2023-11-08T08:19:34.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f98b4c41700 1 -- 172.21.15.40:0/3771114931 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0036c0 con 0x7f98a803b560 2023-11-08T08:19:34.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7fd50d1b4700 1 --2- 172.21.15.40:0/555015670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd50003ac20 0x7fd500184da0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd504015000 tx=0x7fd504004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f98b7ea7700 1 -- 172.21.15.40:0/3771114931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98a803b560 msgr2=0x7f98a8052050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f98b7ea7700 1 --2- 172.21.15.40:0/3771114931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98a803b560 0x7f98a8052050 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009850 tx=0x7f98ac009b60 comp rx=0 tx=0).stop 2023-11-08T08:19:34.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f4aaaf62700 1 -- start start 2023-11-08T08:19:34.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f0758f90700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f074c051170 0x7f074c036630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f0758f90700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f074c036b70 0x7f074c052ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f0758f90700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f074c036fb0 con 0x7f074c0e6d20 2023-11-08T08:19:34.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f0758f90700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f074c048570 con 0x7f074c036b70 2023-11-08T08:19:34.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f0758f90700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f074c04f910 con 0x7f074c051170 2023-11-08T08:19:34.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7fd4fe7fc700 1 -- 172.21.15.40:0/555015670 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50400be60 con 0x7fd50003ac20 2023-11-08T08:19:34.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7fd4fe7fc700 1 -- 172.21.15.40:0/555015670 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5040036c0 con 0x7fd50003ac20 2023-11-08T08:19:34.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7fd4fe7fc700 1 -- 172.21.15.40:0/555015670 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd504003a70 con 0x7fd50003ac20 2023-11-08T08:19:34.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7fd50e9b7700 1 -- 172.21.15.40:0/555015670 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd500006830 con 0x7fd50003ac20 2023-11-08T08:19:34.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7fd50e9b7700 1 -- 172.21.15.40:0/555015670 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd500006550 con 0x7fd50003ac20 2023-11-08T08:19:34.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f4aaaf62700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a9c0340a0 0x7f4a9c173790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f4aaaf62700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a9c036d50 0x7f4a9c048760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f4aaaf62700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a9c050420 0x7f4a9c048ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f4aaaf62700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c0dc130 con 0x7f4a9c0340a0 2023-11-08T08:19:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f4aaaf62700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c0db4b0 con 0x7f4a9c036d50 2023-11-08T08:19:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f4aaaf62700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c0db790 con 0x7f4a9c050420 2023-11-08T08:19:34.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7fb0476ee700 1 -- start start 2023-11-08T08:19:34.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7fb0476ee700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb04015d510 0x7fb0400b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7fb0476ee700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0400b6030 0x7fb040070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f07537fe700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f074c0e6d20 0x7f074c04f200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:38268/0 (socket says 172.21.15.40:38268) 2023-11-08T08:19:34.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f07537fe700 1 -- 172.21.15.40:0/4188259301 learned_addr learned my addr 172.21.15.40:0/4188259301 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:34.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f0752ffd700 1 --2- 172.21.15.40:0/4188259301 >> v2:172.21.15.169:3300/0 conn(0x7f074c051170 0x7f074c036630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f98b7ea7700 1 -- 172.21.15.40:0/3771114931 shutdown_connections 2023-11-08T08:19:34.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f98b7ea7700 1 --2- 172.21.15.40:0/3771114931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98a803b560 0x7f98a8052050 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f98b7ea7700 1 -- 172.21.15.40:0/3771114931 >> 172.21.15.40:0/3771114931 conn(0x7f98a8052880 msgr2=0x7f98a8052c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7fb0476ee700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04015d080 con 0x7fb0401534a0 2023-11-08T08:19:34.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7fb0476ee700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04015d970 con 0x7fb04015d510 2023-11-08T08:19:34.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7fb0476ee700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb040153870 con 0x7fb0400b6030 2023-11-08T08:19:34.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f13d759e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f13d8160420 0x7f13d81607f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f13d759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13d80c8da0 0x7f13d8070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f13d759e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d8069c90 con 0x7f13d80cdf20 2023-11-08T08:19:34.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f13d759e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d80cf430 con 0x7f13d8160420 2023-11-08T08:19:34.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.930+0000 7f13d759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d80ce4b0 con 0x7f13d80c8da0 2023-11-08T08:19:34.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d5d9b700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f13d80cdf20 0x7f13d8164c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:38254/0 (socket says 172.21.15.40:38254) 2023-11-08T08:19:34.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d5d9b700 1 -- 172.21.15.40:0/2790505197 learned_addr learned my addr 172.21.15.40:0/2790505197 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:34.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d559a700 1 -- 172.21.15.40:0/2790505197 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323216687 0 0) 0x7f13d8069c90 con 0x7f13d80cdf20 2023-11-08T08:19:34.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d559a700 1 -- 172.21.15.40:0/2790505197 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13c00035d0 con 0x7f13d80cdf20 2023-11-08T08:19:34.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d559a700 1 -- 172.21.15.40:0/2790505197 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2907340266 0 0) 0x7f13c00035d0 con 0x7f13d80cdf20 2023-11-08T08:19:34.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d559a700 1 -- 172.21.15.40:0/2790505197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13d80c8da0 msgr2=0x7f13d8070080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-08T08:19:34.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d559a700 1 --2- 172.21.15.40:0/2790505197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13d80c8da0 0x7f13d8070080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d559a700 1 -- 172.21.15.40:0/2790505197 >> v2:172.21.15.40:3300/0 conn(0x7f13d8160420 msgr2=0x7f13d81607f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-08T08:19:34.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d559a700 1 --2- 172.21.15.40:0/2790505197 >> v2:172.21.15.40:3300/0 conn(0x7f13d8160420 0x7f13d81607f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d559a700 1 -- 172.21.15.40:0/2790505197 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d80cc3c0 con 0x7f13d80cdf20 2023-11-08T08:19:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d559a700 1 -- 172.21.15.40:0/2790505197 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f13d00019f0 con 0x7f13d80cdf20 2023-11-08T08:19:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d559a700 1 -- 172.21.15.40:0/2790505197 >> v1:172.21.15.169:6789/0 conn(0x7f13d80cdf20 legacy=0x7f13d8164c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d559a700 1 --2- 172.21.15.40:0/2790505197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13c0003a20 0x7f13c0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d559a700 1 --2- 172.21.15.40:0/2790505197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13c0004870 0x7f13c0004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d559a700 1 --2- 172.21.15.40:0/2790505197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13c0005340 0x7f13c000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d559a700 1 -- 172.21.15.40:0/2790505197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d80cf430 con 0x7f13c0003a20 2023-11-08T08:19:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d559a700 1 -- 172.21.15.40:0/2790505197 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d80ce4b0 con 0x7f13c0004870 2023-11-08T08:19:34.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d559a700 1 -- 172.21.15.40:0/2790505197 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c000c1e0 con 0x7f13c0005340 2023-11-08T08:19:34.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d5d9b700 1 --2- 172.21.15.40:0/2790505197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13c0004870 0x7f13c0004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7fb044c89700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb04015d510 0x7fb0400b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d6d9d700 1 --2- 172.21.15.40:0/2790505197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13c0005340 0x7f13c000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d5d9b700 1 -- 172.21.15.40:0/2790505197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13c0005340 msgr2=0x7f13c000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:34.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d5d9b700 1 --2- 172.21.15.40:0/2790505197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13c0005340 0x7f13c000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d5d9b700 1 -- 172.21.15.40:0/2790505197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13c0003a20 msgr2=0x7f13c0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:34.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d5d9b700 1 --2- 172.21.15.40:0/2790505197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13c0003a20 0x7f13c0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f13d5d9b700 1 -- 172.21.15.40:0/2790505197 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d80cc3c0 con 0x7f13c0004870 2023-11-08T08:19:34.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7fb044c89700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb04015d510 0x7fb0400b5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52376/0 (socket says 172.21.15.40:52376) 2023-11-08T08:19:34.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7fb044c89700 1 -- 172.21.15.40:0/3123835985 learned_addr learned my addr 172.21.15.40:0/3123835985 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:34.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f0753fff700 1 --2- 172.21.15.40:0/4188259301 >> v2:172.21.15.40:3300/0 conn(0x7f074c036b70 0x7f074c052ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f4aa9d00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a9c036d50 0x7f4a9c048760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f4aa9d00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a9c036d50 0x7f4a9c048760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33646/0 (socket says 172.21.15.40:33646) 2023-11-08T08:19:34.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f4aa9d00700 1 -- 172.21.15.40:0/2575977621 learned_addr learned my addr 172.21.15.40:0/2575977621 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:34.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f4aa9d00700 1 -- 172.21.15.40:0/2575977621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a9c050420 msgr2=0x7f4a9c048ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f4aa9d00700 1 --2- 172.21.15.40:0/2575977621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a9c050420 0x7f4a9c048ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f4aa9d00700 1 -- 172.21.15.40:0/2575977621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a9c0340a0 msgr2=0x7f4a9c173790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f4aa9d00700 1 --2- 172.21.15.40:0/2575977621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a9c0340a0 0x7f4a9c173790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f4aa9d00700 1 -- 172.21.15.40:0/2575977621 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a90009570 con 0x7f4a9c036d50 2023-11-08T08:19:34.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f13d6d9d700 1 --2- 172.21.15.40:0/2790505197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13c0005340 0x7f13c000ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:34.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f0752ffd700 1 -- 172.21.15.40:0/4188259301 >> v2:172.21.15.40:3300/0 conn(0x7f074c036b70 msgr2=0x7f074c052ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:34.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7f0752ffd700 1 --2- 172.21.15.40:0/4188259301 >> v2:172.21.15.40:3300/0 conn(0x7f074c036b70 0x7f074c052ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f0752ffd700 1 -- 172.21.15.40:0/4188259301 >> v1:172.21.15.169:6789/0 conn(0x7f074c0e6d20 legacy=0x7f074c04f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f0752ffd700 1 -- 172.21.15.40:0/4188259301 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074c033940 con 0x7f074c051170 2023-11-08T08:19:34.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7fb044c89700 1 -- 172.21.15.40:0/3123835985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0400b6030 msgr2=0x7fb040070630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-08T08:19:34.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7fb044c89700 1 --2- 172.21.15.40:0/3123835985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0400b6030 0x7fb040070630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.934+0000 7fb044c89700 1 -- 172.21.15.40:0/3123835985 >> v1:172.21.15.40:6789/0 conn(0x7fb0401534a0 legacy=0x7fb040160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:34.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7fb044c89700 1 -- 172.21.15.40:0/3123835985 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0400b6430 con 0x7fb04015d510 2023-11-08T08:19:34.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f4aa9d00700 1 --2- 172.21.15.40:0/2575977621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a9c036d50 0x7f4a9c048760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800fde0 tx=0x7f4a9800d630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7fb044c89700 1 --2- 172.21.15.40:0/3123835985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb04015d510 0x7fb0400b5af0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009910 tx=0x7fb03c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa54fbffd9c25dd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f13d5d9b700 1 --2- 172.21.15.40:0/2790505197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13c0004870 0x7f13c0004ce0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f13d000a940 tx=0x7f13c00058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43a8989454d122c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f98b7ea7700 1 -- 172.21.15.40:0/3771114931 shutdown_connections 2023-11-08T08:19:34.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f0752ffd700 1 --2- 172.21.15.40:0/4188259301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f074c051170 0x7f074c036630 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f074c00bbd0 tx=0x7f0740009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a529576165ac9ba4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f4aa17fa700 1 -- 172.21.15.40:0/2575977621 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9800b7d0 con 0x7f4a9c036d50 2023-11-08T08:19:34.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f4aa17fa700 1 -- 172.21.15.40:0/2575977621 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a98018640 con 0x7f4a9c036d50 2023-11-08T08:19:34.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f4aa17fa700 1 -- 172.21.15.40:0/2575977621 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a980189b0 con 0x7f4a9c036d50 2023-11-08T08:19:34.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f4aaaf62700 1 -- 172.21.15.40:0/2575977621 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9c0e3cf0 con 0x7f4a9c036d50 2023-11-08T08:19:34.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f4aaaf62700 1 -- 172.21.15.40:0/2575977621 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9c0e3a10 con 0x7f4a9c036d50 2023-11-08T08:19:34.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f13d559a700 1 -- 172.21.15.40:0/2790505197 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d00072d0 con 0x7f13c0004870 2023-11-08T08:19:34.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f13d559a700 1 -- 172.21.15.40:0/2790505197 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d0007450 con 0x7f13c0004870 2023-11-08T08:19:34.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f07527fc700 1 -- 172.21.15.40:0/4188259301 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0740004230 con 0x7f074c051170 2023-11-08T08:19:34.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f07527fc700 1 -- 172.21.15.40:0/4188259301 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0740004390 con 0x7f074c051170 2023-11-08T08:19:34.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f07527fc700 1 -- 172.21.15.40:0/4188259301 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07400036c0 con 0x7f074c051170 2023-11-08T08:19:34.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f13d559a700 1 -- 172.21.15.40:0/2790505197 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d0007800 con 0x7f13c0004870 2023-11-08T08:19:34.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7fb037fff700 1 -- 172.21.15.40:0/3123835985 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c004230 con 0x7fb04015d510 2023-11-08T08:19:34.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7fb037fff700 1 -- 172.21.15.40:0/3123835985 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c004390 con 0x7fb04015d510 2023-11-08T08:19:34.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7fb037fff700 1 -- 172.21.15.40:0/3123835985 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c009e60 con 0x7fb04015d510 2023-11-08T08:19:34.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f4aaaf62700 1 -- 172.21.15.40:0/2575977621 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a9c0e0da0 con 0x7f4a9c036d50 2023-11-08T08:19:34.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f13d759e700 1 -- 172.21.15.40:0/2790505197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13c0004870 msgr2=0x7f13c0004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f13d759e700 1 --2- 172.21.15.40:0/2790505197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13c0004870 0x7f13c0004ce0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f13d000a940 tx=0x7f13c00058a0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f13d6d9d700 1 -- 172.21.15.40:0/2790505197 reap_dead start 2023-11-08T08:19:34.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f0758f90700 1 -- 172.21.15.40:0/4188259301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f074c051170 msgr2=0x7f074c036630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f0758f90700 1 --2- 172.21.15.40:0/4188259301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f074c051170 0x7f074c036630 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f074c00bbd0 tx=0x7f0740009b30 comp rx=0 tx=0).stop 2023-11-08T08:19:34.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7fd4fe7fc700 1 -- 172.21.15.40:0/555015670 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd504017020 con 0x7fd50003ac20 2023-11-08T08:19:34.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7fd50e9b7700 1 -- 172.21.15.40:0/555015670 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50003d2c0 con 0x7fd50003ac20 2023-11-08T08:19:34.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7fd4fe7fc700 1 --2- 172.21.15.40:0/555015670 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd4ec064f30 0x7fd4ec0673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7fd4fe7fc700 1 -- 172.21.15.40:0/555015670 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd504083850 con 0x7fd50003ac20 2023-11-08T08:19:34.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7fe5ca984700 1 Processor -- start 2023-11-08T08:19:34.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f13d759e700 1 -- 172.21.15.40:0/2790505197 shutdown_connections 2023-11-08T08:19:34.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f13d759e700 1 -- 172.21.15.40:0/2790505197 >> 172.21.15.40:0/2790505197 conn(0x7f13d80b0f90 msgr2=0x7f13d80ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f0960471700 1 Processor -- start 2023-11-08T08:19:34.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f13d759e700 1 -- 172.21.15.40:0/2790505197 shutdown_connections 2023-11-08T08:19:34.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f13d759e700 1 -- 172.21.15.40:0/2790505197 wait complete. 2023-11-08T08:19:34.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7fd50d9b5700 1 --2- 172.21.15.40:0/555015670 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd4ec064f30 0x7fd4ec0673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:34.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7fb0476ee700 1 -- 172.21.15.40:0/3123835985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb04015d510 msgr2=0x7fb0400b5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:34.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7fb0476ee700 1 --2- 172.21.15.40:0/3123835985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb04015d510 0x7fb0400b5af0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009910 tx=0x7fb03c009c20 comp rx=0 tx=0).stop 2023-11-08T08:19:34.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f0960471700 1 -- start start 2023-11-08T08:19:34.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.938+0000 7f98b7ea7700 1 -- 172.21.15.40:0/3771114931 wait complete. 2023-11-08T08:19:34.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f4aa17fa700 1 -- 172.21.15.40:0/2575977621 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4a9801f030 con 0x7f4a9c036d50 2023-11-08T08:19:34.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f0758f90700 1 -- 172.21.15.40:0/4188259301 shutdown_connections 2023-11-08T08:19:34.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f0758f90700 1 --2- 172.21.15.40:0/4188259301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f074c051170 0x7f074c036630 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f0758f90700 1 --2- 172.21.15.40:0/4188259301 >> v2:172.21.15.40:3300/0 conn(0x7f074c036b70 0x7f074c052ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f0758f90700 1 -- 172.21.15.40:0/4188259301 >> 172.21.15.40:0/4188259301 conn(0x7f074c051630 msgr2=0x7f074c051a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7fb0476ee700 1 -- 172.21.15.40:0/3123835985 shutdown_connections 2023-11-08T08:19:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7fb0476ee700 1 --2- 172.21.15.40:0/3123835985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0400b6030 0x7fb040070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7fe5ca984700 1 -- start start 2023-11-08T08:19:34.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7fb0476ee700 1 --2- 172.21.15.40:0/3123835985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb04015d510 0x7fb0400b5af0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:34.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7fb0476ee700 1 -- 172.21.15.40:0/3123835985 >> 172.21.15.40:0/3123835985 conn(0x7fb04006bfc0 msgr2=0x7fb04006c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:34.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f0758f90700 1 -- 172.21.15.40:0/4188259301 shutdown_connections 2023-11-08T08:19:34.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f98b7ea7700 1 Processor -- start 2023-11-08T08:19:34.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f98b7ea7700 1 -- start start 2023-11-08T08:19:34.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f0758f90700 1 -- 172.21.15.40:0/4188259301 wait complete. 2023-11-08T08:19:34.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7fb0476ee700 1 -- 172.21.15.40:0/3123835985 shutdown_connections 2023-11-08T08:19:34.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7fb0476ee700 1 -- 172.21.15.40:0/3123835985 wait complete. 2023-11-08T08:19:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f0758f90700 1 Processor -- start 2023-11-08T08:19:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7fe5ca984700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe5c415c640 0x7fe5c40d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7fd50d9b5700 1 --2- 172.21.15.40:0/555015670 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd4ec064f30 0x7fd4ec0673e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009a00 tx=0x7fd4f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f0960471700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0950036f20 0x7f0950054730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f0960471700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0950051550 con 0x7f09500e2970 2023-11-08T08:19:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f0960471700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f095004f7a0 con 0x7f095002f780 2023-11-08T08:19:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f0960471700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095004bcc0 con 0x7f0950036f20 2023-11-08T08:19:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f4aa17fa700 1 --2- 172.21.15.40:0/2575977621 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4a88064f10 0x7f4a880673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f4aa17fa700 1 -- 172.21.15.40:0/2575977621 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4a98082710 con 0x7f4a9c036d50 2023-11-08T08:19:35.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7f13d759e700 1 Processor -- start 2023-11-08T08:19:35.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.942+0000 7fe5ca984700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c4069e00 0x7fe5c406a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f98b7ea7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98a8037f90 0x7f98a80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f13d759e700 1 -- start start 2023-11-08T08:19:35.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fb0476ee700 1 Processor -- start 2023-11-08T08:19:35.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fe5c9982700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe5c415c640 0x7fe5c40d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fe5c9982700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe5c415c640 0x7fe5c40d31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33662/0 (socket says 172.21.15.40:33662) 2023-11-08T08:19:35.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fb0476ee700 1 -- start start 2023-11-08T08:19:35.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f98b7ea7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98a803b560 0x7f98a80e17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fe5ca984700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c406c600 con 0x7fe5c4071aa0 2023-11-08T08:19:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fe5c9181700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fe5c4071aa0 0x7fe5c406d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:38280/0 (socket says 172.21.15.40:38280) 2023-11-08T08:19:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fe5c9982700 1 -- 172.21.15.40:0/2635954231 learned_addr learned my addr 172.21.15.40:0/2635954231 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:35.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f4aa3fff700 1 --2- 172.21.15.40:0/2575977621 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4a88064f10 0x7f4a880673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f98b5c43700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98a8037f90 0x7f98a80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f98b5c43700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98a8037f90 0x7f98a80d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:53484/0 (socket says 172.21.15.40:53484) 2023-11-08T08:19:35.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fe5ca183700 1 --2- 172.21.15.40:0/2635954231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c4069e00 0x7fe5c406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fe5ca984700 1 -- 172.21.15.40:0/2635954231 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c406cef0 con 0x7fe5c415c640 2023-11-08T08:19:35.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fe5ca984700 1 -- 172.21.15.40:0/2635954231 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c406a7c0 con 0x7fe5c4069e00 2023-11-08T08:19:35.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f98b7ea7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98a80558a0 0x7f98a80e1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f98b7ea7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a80e2220 con 0x7f98a803b560 2023-11-08T08:19:35.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f095f20f700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f095002f780 0x7f0950006160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:38284/0 (socket says 172.21.15.40:38284) 2023-11-08T08:19:35.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f095f20f700 1 -- 172.21.15.40:0/779606473 learned_addr learned my addr 172.21.15.40:0/779606473 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:35.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f98b5c43700 1 -- 172.21.15.40:0/870786651 learned_addr learned my addr 172.21.15.40:0/870786651 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:35.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f13d759e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13c0004870 0x7f13d8155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f13d759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13d80c61f0 0x7f13d80c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f13d759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13d80b3380 0x7f13d80b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f98b5442700 1 --2- 172.21.15.40:0/870786651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98a803b560 0x7f98a80e17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f13d759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d81f62b0 con 0x7f13d80b3380 2023-11-08T08:19:35.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f13d759e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d80b3d60 con 0x7f13c0004870 2023-11-08T08:19:35.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f13d759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d80c6c80 con 0x7f13d80c61f0 2023-11-08T08:19:35.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f0758f90700 1 -- start start 2023-11-08T08:19:35.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f98b6444700 1 --2- 172.21.15.40:0/870786651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98a80558a0 0x7f98a80e1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f98b7ea7700 1 -- 172.21.15.40:0/870786651 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a804f830 con 0x7f98a80558a0 2023-11-08T08:19:35.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f98b7ea7700 1 -- 172.21.15.40:0/870786651 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a804fb10 con 0x7f98a8037f90 2023-11-08T08:19:35.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f13d659c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13c0004870 0x7f13d8155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f4aa3fff700 1 --2- 172.21.15.40:0/2575977621 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4a88064f10 0x7f4a880673c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4aa404ed50 tx=0x7f4aa406d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:35.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f13d659c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13c0004870 0x7f13d8155dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33678/0 (socket says 172.21.15.40:33678) 2023-11-08T08:19:35.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f13d659c700 1 -- 172.21.15.40:0/1906160524 learned_addr learned my addr 172.21.15.40:0/1906160524 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:35.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fb0476ee700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0400b6030 0x7fb040159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fb0476ee700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0401534a0 0x7fb0401fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fb0476ee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb04015d510 0x7fb0401fbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fb0476ee700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0401ff950 con 0x7fb04015d510 2023-11-08T08:19:35.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fb0476ee700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04008c2e0 con 0x7fb0401534a0 2023-11-08T08:19:35.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fb0476ee700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0401fed40 con 0x7fb0400b6030 2023-11-08T08:19:35.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fe5c9982700 1 -- 172.21.15.40:0/2635954231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c4069e00 msgr2=0x7fe5c406a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:35.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fe5c9982700 1 --2- 172.21.15.40:0/2635954231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c4069e00 0x7fe5c406a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fe5c9982700 1 -- 172.21.15.40:0/2635954231 >> v1:172.21.15.169:6789/0 conn(0x7fe5c4071aa0 legacy=0x7fe5c406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f13d5d9b700 1 --2- 172.21.15.40:0/1906160524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13d80c61f0 0x7f13d80c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fe5c9982700 1 -- 172.21.15.40:0/2635954231 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c40722f0 con 0x7fe5c415c640 2023-11-08T08:19:35.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fb044c89700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0401534a0 0x7fb0401fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f13d659c700 1 -- 172.21.15.40:0/1906160524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13d80c61f0 msgr2=0x7f13d80c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f13d659c700 1 --2- 172.21.15.40:0/1906160524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13d80c61f0 0x7f13d80c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f13d659c700 1 -- 172.21.15.40:0/1906160524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13d80b3380 msgr2=0x7f13d80b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f13d659c700 1 --2- 172.21.15.40:0/1906160524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13d80b3380 0x7f13d80b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f13d659c700 1 -- 172.21.15.40:0/1906160524 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d80cc980 con 0x7f13c0004870 2023-11-08T08:19:35.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fb044c89700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0401534a0 0x7fb0401fb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33684/0 (socket says 172.21.15.40:33684) 2023-11-08T08:19:35.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fb044c89700 1 -- 172.21.15.40:0/1328659828 learned_addr learned my addr 172.21.15.40:0/1328659828 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:35.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fb04548a700 1 --2- 172.21.15.40:0/1328659828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0400b6030 0x7fb040159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fe5ca183700 1 --2- 172.21.15.40:0/2635954231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c4069e00 0x7fe5c406a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:35.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f98b6444700 1 -- 172.21.15.40:0/870786651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98a8037f90 msgr2=0x7f98a80d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f98b6444700 1 --2- 172.21.15.40:0/870786651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98a8037f90 0x7f98a80d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f98b6444700 1 -- 172.21.15.40:0/870786651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98a803b560 msgr2=0x7f98a80e17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f98b6444700 1 --2- 172.21.15.40:0/870786651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98a803b560 0x7f98a80e17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f98b6444700 1 -- 172.21.15.40:0/870786651 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac0094b0 con 0x7f98a80558a0 2023-11-08T08:19:35.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f13d659c700 1 --2- 172.21.15.40:0/1906160524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13c0004870 0x7f13d8155dd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00b580 tx=0x7f13cc00b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:35.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f0758f90700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f074c036b70 0x7f074c0e4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f0758f90700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f074c051170 0x7f074c04a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f0758f90700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f074c0e6d20 0x7f074c04adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f0758f90700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c04ca90 con 0x7f074c051170 2023-11-08T08:19:35.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f0758f90700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c184bf0 con 0x7f074c036b70 2023-11-08T08:19:35.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f0758f90700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f074c04c1f0 con 0x7f074c0e6d20 2023-11-08T08:19:35.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f095d20b700 1 -- 172.21.15.40:0/779606473 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986483966 0 0) 0x7f0950051550 con 0x7f09500e2970 2023-11-08T08:19:35.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f095d20b700 1 -- 172.21.15.40:0/779606473 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09440035d0 con 0x7f09500e2970 2023-11-08T08:19:35.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f095da0c700 1 --2- 172.21.15.40:0/779606473 >> v2:172.21.15.40:3300/0 conn(0x7f0950036f20 0x7f0950054730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fb044c89700 1 -- 172.21.15.40:0/1328659828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0400b6030 msgr2=0x7fb040159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fb044c89700 1 --2- 172.21.15.40:0/1328659828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0400b6030 0x7fb040159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fb044c89700 1 -- 172.21.15.40:0/1328659828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb04015d510 msgr2=0x7fb0401fbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fb044c89700 1 --2- 172.21.15.40:0/1328659828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb04015d510 0x7fb0401fbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fe5c9982700 1 --2- 172.21.15.40:0/2635954231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c415c640 0x7fe5c40d31e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009910 tx=0x7fe5c0009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32387cc76c2edefb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:35.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7fb044c89700 1 -- 172.21.15.40:0/1328659828 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c009570 con 0x7fb0401534a0 2023-11-08T08:19:35.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f095d20b700 1 -- 172.21.15.40:0/779606473 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3192416348 0 0) 0x7f09440035d0 con 0x7f09500e2970 2023-11-08T08:19:35.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f095d20b700 1 -- 172.21.15.40:0/779606473 >> v2:172.21.15.40:3300/0 conn(0x7f0950036f20 msgr2=0x7f0950054730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:35.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f095d20b700 1 --2- 172.21.15.40:0/779606473 >> v2:172.21.15.40:3300/0 conn(0x7f0950036f20 0x7f0950054730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f095d20b700 1 -- 172.21.15.40:0/779606473 >> v1:172.21.15.169:6789/0 conn(0x7f095002f780 legacy=0x7f0950006160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f095d20b700 1 -- 172.21.15.40:0/779606473 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09500e3b20 con 0x7f09500e2970 2023-11-08T08:19:35.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f98b6444700 1 --2- 172.21.15.40:0/870786651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98a80558a0 0x7f98a80e1ce0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f98b004f8c0 tx=0x7f98b004e880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:35.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f095d20b700 1 -- 172.21.15.40:0/779606473 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f0954001670 con 0x7f09500e2970 2023-11-08T08:19:35.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.946+0000 7f07537fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f074c036b70 0x7f074c0e4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f0753fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f074c0e6d20 0x7f074c04adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f095d20b700 1 -- 172.21.15.40:0/779606473 >> v1:172.21.15.40:6789/0 conn(0x7f09500e2970 legacy=0x7f0950036810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f0753fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f074c0e6d20 0x7f074c04adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:53514/0 (socket says 172.21.15.40:53514) 2023-11-08T08:19:35.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f07537fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f074c036b70 0x7f074c0e4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33686/0 (socket says 172.21.15.40:33686) 2023-11-08T08:19:35.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f0753fff700 1 -- 172.21.15.40:0/3223771923 learned_addr learned my addr 172.21.15.40:0/3223771923 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:35.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f095d20b700 1 --2- 172.21.15.40:0/779606473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0944003a20 0x7f0944003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7fb044c89700 1 --2- 172.21.15.40:0/1328659828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0401534a0 0x7fb0401fb990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009c60 tx=0x7fb03c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:35.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f095d20b700 1 --2- 172.21.15.40:0/779606473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0944004870 0x7f0944004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f095d20b700 1 --2- 172.21.15.40:0/779606473 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0944005220 0x7f094400b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f095d20b700 1 -- 172.21.15.40:0/779606473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095004bcc0 con 0x7f0944003a20 2023-11-08T08:19:35.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f095d20b700 1 -- 172.21.15.40:0/779606473 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094400c100 con 0x7f0944004870 2023-11-08T08:19:35.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f095d20b700 1 -- 172.21.15.40:0/779606473 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f094400c2c0 con 0x7f0944005220 2023-11-08T08:19:35.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f0752ffd700 1 --2- 172.21.15.40:0/3223771923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f074c051170 0x7f074c04a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f0753fff700 1 -- 172.21.15.40:0/3223771923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f074c036b70 msgr2=0x7f074c0e4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f0753fff700 1 --2- 172.21.15.40:0/3223771923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f074c036b70 0x7f074c0e4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f0753fff700 1 -- 172.21.15.40:0/3223771923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f074c051170 msgr2=0x7f074c04a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f0753fff700 1 --2- 172.21.15.40:0/3223771923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f074c051170 0x7f074c04a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f0753fff700 1 -- 172.21.15.40:0/3223771923 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07400094b0 con 0x7f074c0e6d20 2023-11-08T08:19:35.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f0752ffd700 1 --2- 172.21.15.40:0/3223771923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f074c051170 0x7f074c04a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:35.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f095f20f700 1 --2- 172.21.15.40:0/779606473 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0944005220 0x7f094400b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f095da0c700 1 --2- 172.21.15.40:0/779606473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0944004870 0x7f0944004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f095e20d700 1 --2- 172.21.15.40:0/779606473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0944003a20 0x7f0944003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f0753fff700 1 --2- 172.21.15.40:0/3223771923 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f074c0e6d20 0x7f074c04adf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0754065690 tx=0x7f07540728f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:35.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f095f20f700 1 -- 172.21.15.40:0/779606473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0944004870 msgr2=0x7f0944004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:35.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f095f20f700 1 --2- 172.21.15.40:0/779606473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0944004870 0x7f0944004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f095f20f700 1 -- 172.21.15.40:0/779606473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0944003a20 msgr2=0x7f0944003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:35.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f095f20f700 1 --2- 172.21.15.40:0/779606473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0944003a20 0x7f0944003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f095f20f700 1 -- 172.21.15.40:0/779606473 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09440035d0 con 0x7f0944005220 2023-11-08T08:19:35.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f095f20f700 1 -- 172.21.15.40:0/779606473 reap_dead start 2023-11-08T08:19:35.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.950+0000 7f095f20f700 1 --2- 172.21.15.40:0/779606473 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0944005220 0x7f094400b9a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f095804ed50 tx=0x7f09440058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=370f4b5abdfc1515 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:35.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.954+0000 7f073ffff700 1 -- 172.21.15.40:0/3223771923 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075408e070 con 0x7f074c0e6d20 2023-11-08T08:19:35.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.954+0000 7f095d20b700 1 -- 172.21.15.40:0/779606473 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095808d030 con 0x7f0944005220 2023-11-08T08:19:35.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.954+0000 7f073ffff700 1 -- 172.21.15.40:0/3223771923 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0754072cd0 con 0x7f074c0e6d20 2023-11-08T08:19:35.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.954+0000 7f073ffff700 1 -- 172.21.15.40:0/3223771923 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0754073270 con 0x7f074c0e6d20 2023-11-08T08:19:35.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.954+0000 7f0758f90700 1 -- 172.21.15.40:0/3223771923 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f074c033f00 con 0x7f074c0e6d20 2023-11-08T08:19:35.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.954+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1906160524 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc011830 con 0x7f13c0004870 2023-11-08T08:19:35.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.954+0000 7f13d759e700 1 -- 172.21.15.40:0/1906160524 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d80cc6a0 con 0x7f13c0004870 2023-11-08T08:19:35.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.954+0000 7fb035ffb700 1 -- 172.21.15.40:0/1328659828 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0038c0 con 0x7fb0401534a0 2023-11-08T08:19:35.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.954+0000 7f095d20b700 1 -- 172.21.15.40:0/779606473 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0958066d40 con 0x7f0944005220 2023-11-08T08:19:35.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.954+0000 7f095d20b700 1 -- 172.21.15.40:0/779606473 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0958072a00 con 0x7f0944005220 2023-11-08T08:19:35.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.954+0000 7f0758f90700 1 -- 172.21.15.40:0/3223771923 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074c033c20 con 0x7f074c0e6d20 2023-11-08T08:19:35.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7f13d759e700 1 -- 172.21.15.40:0/1906160524 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d80ccb10 con 0x7f13c0004870 2023-11-08T08:19:35.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7fd4fe7fc700 1 -- 172.21.15.40:0/555015670 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd504051690 con 0x7fd50003ac20 2023-11-08T08:19:35.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7fe5c8980700 1 -- 172.21.15.40:0/2635954231 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0011830 con 0x7fe5c415c640 2023-11-08T08:19:35.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7fe5c8980700 1 -- 172.21.15.40:0/2635954231 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c0011990 con 0x7fe5c415c640 2023-11-08T08:19:35.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7fe5c8980700 1 -- 172.21.15.40:0/2635954231 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0011d40 con 0x7fe5c415c640 2023-11-08T08:19:35.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7f98a67fc700 1 -- 172.21.15.40:0/870786651 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0061020 con 0x7f98a80558a0 2023-11-08T08:19:35.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7f98a67fc700 1 -- 172.21.15.40:0/870786651 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b00611c0 con 0x7f98a80558a0 2023-11-08T08:19:35.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7f98a67fc700 1 -- 172.21.15.40:0/870786651 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0073070 con 0x7f98a80558a0 2023-11-08T08:19:35.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7f4aa17fa700 1 -- 172.21.15.40:0/2575977621 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4a98050670 con 0x7f4a9c036d50 2023-11-08T08:19:35.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7f98b7ea7700 1 -- 172.21.15.40:0/870786651 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a8050580 con 0x7f98a80558a0 2023-11-08T08:19:35.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7f98b7ea7700 1 -- 172.21.15.40:0/870786651 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a80e3d90 con 0x7f98a80558a0 2023-11-08T08:19:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1906160524 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc011990 con 0x7f13c0004870 2023-11-08T08:19:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1906160524 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc011d40 con 0x7f13c0004870 2023-11-08T08:19:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7f0960471700 1 -- 172.21.15.40:0/779606473 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0944005220 msgr2=0x7f094400b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7f0960471700 1 --2- 172.21.15.40:0/779606473 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0944005220 0x7f094400b9a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f095804ed50 tx=0x7f09440058a0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7f0758f90700 1 -- 172.21.15.40:0/3223771923 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f074c0341e0 con 0x7f074c0e6d20 2023-11-08T08:19:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7f13d759e700 1 -- 172.21.15.40:0/1906160524 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13bc005310 con 0x7f13c0004870 2023-11-08T08:19:35.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7fb0476ee700 1 -- 172.21.15.40:0/1328659828 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0400b56a0 con 0x7fb0401534a0 2023-11-08T08:19:35.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7fb035ffb700 1 -- 172.21.15.40:0/1328659828 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c003a20 con 0x7fb0401534a0 2023-11-08T08:19:35.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7fb035ffb700 1 -- 172.21.15.40:0/1328659828 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c003dd0 con 0x7fb0401534a0 2023-11-08T08:19:35.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7fb0476ee700 1 -- 172.21.15.40:0/1328659828 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0400b53c0 con 0x7fb0401534a0 2023-11-08T08:19:35.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7f0960471700 1 -- 172.21.15.40:0/779606473 shutdown_connections 2023-11-08T08:19:35.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7f0960471700 1 --2- 172.21.15.40:0/779606473 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0944005220 0x7f094400b9a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.958+0000 7f0960471700 1 -- 172.21.15.40:0/779606473 >> 172.21.15.40:0/779606473 conn(0x7f0950051810 msgr2=0x7f0950051c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:35.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.962+0000 7f98b7ea7700 1 -- 172.21.15.40:0/870786651 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9898005310 con 0x7f98a80558a0 2023-11-08T08:19:35.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.962+0000 7fe5ca984700 1 -- 172.21.15.40:0/2635954231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c415c640 msgr2=0x7fe5c40d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.962+0000 7fe5ca984700 1 --2- 172.21.15.40:0/2635954231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c415c640 0x7fe5c40d31e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009910 tx=0x7fe5c0009c20 comp rx=0 tx=0).stop 2023-11-08T08:19:35.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.962+0000 7f98a67fc700 1 -- 172.21.15.40:0/870786651 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f98b00aba60 con 0x7f98a80558a0 2023-11-08T08:19:35.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.962+0000 7f0960471700 1 -- 172.21.15.40:0/779606473 shutdown_connections 2023-11-08T08:19:35.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.962+0000 7f0960471700 1 -- 172.21.15.40:0/779606473 wait complete. 2023-11-08T08:19:35.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.962+0000 7fe5ca984700 1 -- 172.21.15.40:0/2635954231 shutdown_connections 2023-11-08T08:19:35.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.962+0000 7fe5ca984700 1 --2- 172.21.15.40:0/2635954231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c4069e00 0x7fe5c406a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.962+0000 7fe5ca984700 1 --2- 172.21.15.40:0/2635954231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c415c640 0x7fe5c40d31e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.962+0000 7fe5ca984700 1 -- 172.21.15.40:0/2635954231 >> 172.21.15.40:0/2635954231 conn(0x7fe5c40b6520 msgr2=0x7fe5c4071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:35.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.962+0000 7fe5ca984700 1 -- 172.21.15.40:0/2635954231 shutdown_connections 2023-11-08T08:19:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.962+0000 7f073ffff700 1 -- 172.21.15.40:0/3223771923 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f07540733d0 con 0x7f074c0e6d20 2023-11-08T08:19:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.962+0000 7f073ffff700 1 --2- 172.21.15.40:0/3223771923 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0738064ee0 0x7f0738067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.962+0000 7f073ffff700 1 -- 172.21.15.40:0/3223771923 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0754072040 con 0x7f074c0e6d20 2023-11-08T08:19:35.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.962+0000 7fb02b7fe700 1 -- 172.21.15.40:0/1328659828 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb018005310 con 0x7fb0401534a0 2023-11-08T08:19:35.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7fb035ffb700 1 -- 172.21.15.40:0/1328659828 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fb03c017020 con 0x7fb0401534a0 2023-11-08T08:19:35.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1906160524 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f13cc021070 con 0x7f13c0004870 2023-11-08T08:19:35.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f07537fe700 1 --2- 172.21.15.40:0/3223771923 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0738064ee0 0x7f0738067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f0960471700 1 Processor -- start 2023-11-08T08:19:35.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f98a67fc700 1 --2- 172.21.15.40:0/870786651 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9894064fb0 0x7f9894067460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f0960471700 1 -- start start 2023-11-08T08:19:35.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f98a67fc700 1 -- 172.21.15.40:0/870786651 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f98b0072040 con 0x7f98a80558a0 2023-11-08T08:19:35.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f13c6ffd700 1 --2- 172.21.15.40:0/1906160524 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f13b8064ee0 0x7f13b8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1906160524 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f13cc0820b0 con 0x7f13c0004870 2023-11-08T08:19:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f07537fe700 1 --2- 172.21.15.40:0/3223771923 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0738064ee0 0x7f0738067390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f074c04ed90 tx=0x7f0748004a70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:35.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7fb035ffb700 1 --2- 172.21.15.40:0/1328659828 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb024064f30 0x7fb0240673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7fb035ffb700 1 -- 172.21.15.40:0/1328659828 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb03c084020 con 0x7fb0401534a0 2023-11-08T08:19:35.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.962+0000 7fe5ca984700 1 -- 172.21.15.40:0/2635954231 wait complete. 2023-11-08T08:19:35.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f98b5c43700 1 --2- 172.21.15.40:0/870786651 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9894064fb0 0x7f9894067460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7fb04548a700 1 --2- 172.21.15.40:0/1328659828 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb024064f30 0x7fb0240673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7fe5ca984700 1 Processor -- start 2023-11-08T08:19:35.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f13d5d9b700 1 --2- 172.21.15.40:0/1906160524 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f13b8064ee0 0x7f13b8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f0960471700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0944005220 0x7f09500d9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f0960471700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09500ddb60 0x7f095004c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7fe5ca984700 1 -- start start 2023-11-08T08:19:35.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f98b5c43700 1 --2- 172.21.15.40:0/870786651 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9894064fb0 0x7f9894067460 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009ba0 tx=0x7f98ac01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:35.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f0960471700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f095004ccd0 0x7f095004e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f0960471700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09500de0d0 con 0x7f095004ccd0 2023-11-08T08:19:35.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f0960471700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09500de3b0 con 0x7f09500ddb60 2023-11-08T08:19:35.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f0960471700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09500de690 con 0x7f0944005220 2023-11-08T08:19:35.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f095e20d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0944005220 0x7f09500d9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f095f20f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f095004ccd0 0x7f095004e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7fb04548a700 1 --2- 172.21.15.40:0/1328659828 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb024064f30 0x7fb0240673e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb04015af30 tx=0x7fb038009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:35.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f095e20d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0944005220 0x7f09500d9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:53538/0 (socket says 172.21.15.40:53538) 2023-11-08T08:19:35.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f095e20d700 1 -- 172.21.15.40:0/1370146026 learned_addr learned my addr 172.21.15.40:0/1370146026 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:35.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.966+0000 7f095da0c700 1 --2- 172.21.15.40:0/1370146026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09500ddb60 0x7f095004c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7f13d5d9b700 1 --2- 172.21.15.40:0/1906160524 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f13b8064ee0 0x7f13b8067390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f13d80b4080 tx=0x7f13d0009070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:35.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7f095e20d700 1 -- 172.21.15.40:0/1370146026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09500ddb60 msgr2=0x7f095004c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7f095e20d700 1 --2- 172.21.15.40:0/1370146026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09500ddb60 0x7f095004c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7f095e20d700 1 -- 172.21.15.40:0/1370146026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f095004ccd0 msgr2=0x7f095004e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7f095e20d700 1 --2- 172.21.15.40:0/1370146026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f095004ccd0 0x7f095004e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7f095e20d700 1 -- 172.21.15.40:0/1370146026 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09500e3810 con 0x7f0944005220 2023-11-08T08:19:35.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7f095da0c700 1 --2- 172.21.15.40:0/1370146026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09500ddb60 0x7f095004c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:35.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7f095f20f700 1 --2- 172.21.15.40:0/1370146026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f095004ccd0 0x7f095004e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:35.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7f095e20d700 1 --2- 172.21.15.40:0/1370146026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0944005220 0x7f09500d9e30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f095400fae0 tx=0x7f095400fdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:35.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7f98a67fc700 1 -- 172.21.15.40:0/870786651 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f98b00c8df0 con 0x7f98a80558a0 2023-11-08T08:19:35.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7fe5ca984700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c4069e00 0x7fe5c40c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7fe5ca984700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c4071aa0 0x7fe5c40cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7fe5c9982700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c4069e00 0x7fe5c40c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7fe5c9982700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c4069e00 0x7fe5c40c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33710/0 (socket says 172.21.15.40:33710) 2023-11-08T08:19:35.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7fe5c9982700 1 -- 172.21.15.40:0/3590670420 learned_addr learned my addr 172.21.15.40:0/3590670420 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:35.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7fe5c9181700 1 --2- 172.21.15.40:0/3590670420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c4071aa0 0x7fe5c40cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7fe5ca984700 1 --2- 172.21.15.40:0/3590670420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5c415c640 0x7fe5c41fe010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7fe5ca183700 1 --2- 172.21.15.40:0/3590670420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5c415c640 0x7fe5c41fe010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7fe5ca984700 1 -- 172.21.15.40:0/3590670420 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c40c9a80 con 0x7fe5c415c640 2023-11-08T08:19:35.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7fe5ca984700 1 -- 172.21.15.40:0/3590670420 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c40ca040 con 0x7fe5c4069e00 2023-11-08T08:19:35.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.970+0000 7fe5ca984700 1 -- 172.21.15.40:0/3590670420 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c40cc9b0 con 0x7fe5c4071aa0 2023-11-08T08:19:35.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.974+0000 7f094effd700 1 -- 172.21.15.40:0/1370146026 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095400fec0 con 0x7f0944005220 2023-11-08T08:19:35.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.974+0000 7f094effd700 1 -- 172.21.15.40:0/1370146026 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0954001690 con 0x7f0944005220 2023-11-08T08:19:35.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.974+0000 7f094effd700 1 -- 172.21.15.40:0/1370146026 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0954005500 con 0x7f0944005220 2023-11-08T08:19:35.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.974+0000 7f0960471700 1 -- 172.21.15.40:0/1370146026 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0950051030 con 0x7f0944005220 2023-11-08T08:19:35.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.974+0000 7f073ffff700 1 -- 172.21.15.40:0/3223771923 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f07540c9530 con 0x7f074c0e6d20 2023-11-08T08:19:35.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.974+0000 7f0960471700 1 -- 172.21.15.40:0/1370146026 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095004f3e0 con 0x7f0944005220 2023-11-08T08:19:35.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.974+0000 7fe5c9181700 1 -- 172.21.15.40:0/3590670420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c4069e00 msgr2=0x7fe5c40c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.974+0000 7fe5c9181700 1 --2- 172.21.15.40:0/3590670420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c4069e00 0x7fe5c40c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.974+0000 7fe5c9181700 1 -- 172.21.15.40:0/3590670420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5c415c640 msgr2=0x7fe5c41fe010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.974+0000 7fe5c9181700 1 --2- 172.21.15.40:0/3590670420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5c415c640 0x7fe5c41fe010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.974+0000 7fe5c9181700 1 -- 172.21.15.40:0/3590670420 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5bc009a70 con 0x7fe5c4071aa0 2023-11-08T08:19:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.974+0000 7fe5ca183700 1 --2- 172.21.15.40:0/3590670420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5c415c640 0x7fe5c41fe010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.974+0000 7fe5c9181700 1 --2- 172.21.15.40:0/3590670420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c4071aa0 0x7fe5c40cc470 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400c500 tx=0x7fe5b400c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:35.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.974+0000 7f0960471700 1 -- 172.21.15.40:0/1370146026 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09500055d0 con 0x7f0944005220 2023-11-08T08:19:35.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.974+0000 7fe5ba7fc700 1 -- 172.21.15.40:0/3590670420 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b400cea0 con 0x7fe5c4071aa0 2023-11-08T08:19:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.974+0000 7fe5ba7fc700 1 -- 172.21.15.40:0/3590670420 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b4005cd0 con 0x7fe5c4071aa0 2023-11-08T08:19:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.974+0000 7fe5ba7fc700 1 -- 172.21.15.40:0/3590670420 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b40072d0 con 0x7fe5c4071aa0 2023-11-08T08:19:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.974+0000 7fe5ca984700 1 -- 172.21.15.40:0/3590670420 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c0009570 con 0x7fe5c4071aa0 2023-11-08T08:19:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.974+0000 7fe5ca984700 1 -- 172.21.15.40:0/3590670420 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c415ca10 con 0x7fe5c4071aa0 2023-11-08T08:19:35.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.974+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1906160524 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f13cc04ff60 con 0x7f13c0004870 2023-11-08T08:19:35.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.978+0000 7fe5ca984700 1 -- 172.21.15.40:0/3590670420 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c406da20 con 0x7fe5c4071aa0 2023-11-08T08:19:35.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.978+0000 7fb035ffb700 1 -- 172.21.15.40:0/1328659828 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fb03c052040 con 0x7fb0401534a0 2023-11-08T08:19:35.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.978+0000 7fe5ba7fc700 1 -- 172.21.15.40:0/3590670420 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fe5b4019030 con 0x7fe5c4071aa0 2023-11-08T08:19:35.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.978+0000 7f094effd700 1 -- 172.21.15.40:0/1370146026 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f09540036a0 con 0x7f0944005220 2023-11-08T08:19:35.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.978+0000 7f094effd700 1 --2- 172.21.15.40:0/1370146026 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f093c064fe0 0x7f093c067490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.978+0000 7f094effd700 1 -- 172.21.15.40:0/1370146026 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0954082260 con 0x7f0944005220 2023-11-08T08:19:35.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.978+0000 7fe5ba7fc700 1 --2- 172.21.15.40:0/3590670420 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe5a8064f30 0x7fe5a80673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:35.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.978+0000 7fe5ba7fc700 1 -- 172.21.15.40:0/3590670420 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe5b4081260 con 0x7fe5c4071aa0 2023-11-08T08:19:35.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.978+0000 7f095da0c700 1 --2- 172.21.15.40:0/1370146026 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f093c064fe0 0x7f093c067490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.978+0000 7fe5c9982700 1 --2- 172.21.15.40:0/3590670420 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe5a8064f30 0x7fe5a80673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:35.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.982+0000 7f095da0c700 1 --2- 172.21.15.40:0/1370146026 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f093c064fe0 0x7f093c067490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f095002de60 tx=0x7f0948008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:35.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.982+0000 7fe5c9982700 1 --2- 172.21.15.40:0/3590670420 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe5a8064f30 0x7fe5a80673e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009c60 tx=0x7fe5c0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:35.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.986+0000 7f094effd700 1 -- 172.21.15.40:0/1370146026 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f0954053ad0 con 0x7f0944005220 2023-11-08T08:19:35.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:34.994+0000 7fe5ba7fc700 1 -- 172.21.15.40:0/3590670420 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fe5b404a400 con 0x7fe5c4071aa0 2023-11-08T08:19:35.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.298+0000 7fd50e9b7700 1 -- 172.21.15.40:0/555015670 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd500050ed0 con 0x7fd50003ac20 2023-11-08T08:19:35.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.302+0000 7fd4fe7fc700 1 -- 172.21.15.40:0/555015670 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd500050ed0 con 0x7fd50003ac20 2023-11-08T08:19:35.309 INFO:teuthology.orchestra.run.smithi040.stdout:55834574857 2023-11-08T08:19:35.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.306+0000 7f4aaaf62700 1 -- 172.21.15.40:0/2575977621 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4a9c0055d0 con 0x7f4a9c036d50 2023-11-08T08:19:35.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.306+0000 7f4aa17fa700 1 -- 172.21.15.40:0/2575977621 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4a9801d090 con 0x7f4a9c036d50 2023-11-08T08:19:35.311 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2023-11-08T08:19:35.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7f98b7ea7700 1 -- 172.21.15.40:0/870786651 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f98980055c0 con 0x7f98a80558a0 2023-11-08T08:19:35.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7f98a67fc700 1 -- 172.21.15.40:0/870786651 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f98b0096090 con 0x7f98a80558a0 2023-11-08T08:19:35.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7fd50e9b7700 1 -- 172.21.15.40:0/555015670 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd4ec064f30 msgr2=0x7fd4ec0673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7fd50e9b7700 1 --2- 172.21.15.40:0/555015670 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd4ec064f30 0x7fd4ec0673e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009a00 tx=0x7fd4f8008040 comp rx=0 tx=0).stop 2023-11-08T08:19:35.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7fd50e9b7700 1 -- 172.21.15.40:0/555015670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd50003ac20 msgr2=0x7fd500184da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7fd50e9b7700 1 --2- 172.21.15.40:0/555015670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd50003ac20 0x7fd500184da0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd504015000 tx=0x7fd504004770 comp rx=0 tx=0).stop 2023-11-08T08:19:35.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7fd50e9b7700 1 -- 172.21.15.40:0/555015670 shutdown_connections 2023-11-08T08:19:35.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7fd50e9b7700 1 --2- 172.21.15.40:0/555015670 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd4ec064f30 0x7fd4ec0673e0 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009a00 tx=0x7fd4f8008040 comp rx=0 tx=0).stop 2023-11-08T08:19:35.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7fd50e9b7700 1 --2- 172.21.15.40:0/555015670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5000e6d60 0x7fd500051d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7fd50e9b7700 1 --2- 172.21.15.40:0/555015670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd50003ac20 0x7fd500184da0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7fd50e9b7700 1 --2- 172.21.15.40:0/555015670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd500007870 0x7fd50003a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7fd50e9b7700 1 -- 172.21.15.40:0/555015670 >> 172.21.15.40:0/555015670 conn(0x7fd500055c20 msgr2=0x7fd5000eb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:35.316 INFO:teuthology.orchestra.run.smithi040.stdout:55834574858 2023-11-08T08:19:35.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7fd50e9b7700 1 -- 172.21.15.40:0/555015670 shutdown_connections 2023-11-08T08:19:35.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7fd50e9b7700 1 -- 172.21.15.40:0/555015670 wait complete. 2023-11-08T08:19:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7f4aaaf62700 1 -- 172.21.15.40:0/2575977621 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4a88064f10 msgr2=0x7f4a880673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7f4aaaf62700 1 --2- 172.21.15.40:0/2575977621 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4a88064f10 0x7f4a880673c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4aa404ed50 tx=0x7f4aa406d040 comp rx=0 tx=0).stop 2023-11-08T08:19:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7f4aaaf62700 1 -- 172.21.15.40:0/2575977621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a9c036d50 msgr2=0x7f4a9c048760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7f4aaaf62700 1 --2- 172.21.15.40:0/2575977621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a9c036d50 0x7f4a9c048760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800fde0 tx=0x7f4a9800d630 comp rx=0 tx=0).stop 2023-11-08T08:19:35.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7f4aaaf62700 1 -- 172.21.15.40:0/2575977621 shutdown_connections 2023-11-08T08:19:35.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7f4aaaf62700 1 --2- 172.21.15.40:0/2575977621 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4a88064f10 0x7f4a880673c0 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4aa404ed50 tx=0x7f4aa406d040 comp rx=0 tx=0).stop 2023-11-08T08:19:35.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7f4aaaf62700 1 --2- 172.21.15.40:0/2575977621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a9c050420 0x7f4a9c048ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7f4aaaf62700 1 --2- 172.21.15.40:0/2575977621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a9c036d50 0x7f4a9c048760 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7f4aaaf62700 1 --2- 172.21.15.40:0/2575977621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a9c0340a0 0x7f4a9c173790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.310+0000 7f4aaaf62700 1 -- 172.21.15.40:0/2575977621 >> 172.21.15.40:0/2575977621 conn(0x7f4a9c051810 msgr2=0x7f4a9c036620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:35.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.314+0000 7f98b7ea7700 1 -- 172.21.15.40:0/870786651 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9894064fb0 msgr2=0x7f9894067460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.314+0000 7f98b7ea7700 1 --2- 172.21.15.40:0/870786651 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9894064fb0 0x7f9894067460 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009ba0 tx=0x7f98ac01c040 comp rx=0 tx=0).stop 2023-11-08T08:19:35.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.314+0000 7f98b7ea7700 1 -- 172.21.15.40:0/870786651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98a80558a0 msgr2=0x7f98a80e1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.314+0000 7f98b7ea7700 1 --2- 172.21.15.40:0/870786651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98a80558a0 0x7f98a80e1ce0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f98b004f8c0 tx=0x7f98b004e880 comp rx=0 tx=0).stop 2023-11-08T08:19:35.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.314+0000 7f98b7ea7700 1 -- 172.21.15.40:0/870786651 shutdown_connections 2023-11-08T08:19:35.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.314+0000 7f98b7ea7700 1 --2- 172.21.15.40:0/870786651 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9894064fb0 0x7f9894067460 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009ba0 tx=0x7f98ac01c040 comp rx=0 tx=0).stop 2023-11-08T08:19:35.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.314+0000 7f98b7ea7700 1 --2- 172.21.15.40:0/870786651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98a80558a0 0x7f98a80e1ce0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.314+0000 7f98b7ea7700 1 --2- 172.21.15.40:0/870786651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98a803b560 0x7f98a80e17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.314+0000 7f98b7ea7700 1 --2- 172.21.15.40:0/870786651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98a8037f90 0x7f98a80d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.314+0000 7f98b7ea7700 1 -- 172.21.15.40:0/870786651 >> 172.21.15.40:0/870786651 conn(0x7f98a8052880 msgr2=0x7f98a803a030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:35.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.314+0000 7f4aaaf62700 1 -- 172.21.15.40:0/2575977621 shutdown_connections 2023-11-08T08:19:35.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.314+0000 7f4aaaf62700 1 -- 172.21.15.40:0/2575977621 wait complete. 2023-11-08T08:19:35.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.314+0000 7f98b7ea7700 1 -- 172.21.15.40:0/870786651 shutdown_connections 2023-11-08T08:19:35.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.314+0000 7f98b7ea7700 1 -- 172.21.15.40:0/870786651 wait complete. 2023-11-08T08:19:35.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.318+0000 7f0758f90700 1 -- 172.21.15.40:0/3223771923 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f074c0053f0 con 0x7f074c0e6d20 2023-11-08T08:19:35.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.322+0000 7f0960471700 1 -- 172.21.15.40:0/1370146026 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0950007390 con 0x7f0944005220 2023-11-08T08:19:35.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.322+0000 7f073ffff700 1 -- 172.21.15.40:0/3223771923 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0754097020 con 0x7f074c0e6d20 2023-11-08T08:19:35.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.322+0000 7fb02b7fe700 1 -- 172.21.15.40:0/1328659828 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb0180055c0 con 0x7fb0401534a0 2023-11-08T08:19:35.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.322+0000 7f094effd700 1 -- 172.21.15.40:0/1370146026 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f09540536c0 con 0x7f0944005220 2023-11-08T08:19:35.326 INFO:teuthology.orchestra.run.smithi040.stdout:55834574858 2023-11-08T08:19:35.328 INFO:teuthology.orchestra.run.smithi040.stdout:55834574858 2023-11-08T08:19:35.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.322+0000 7fb035ffb700 1 -- 172.21.15.40:0/1328659828 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb03c055660 con 0x7fb0401534a0 2023-11-08T08:19:35.329 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2023-11-08T08:19:35.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0960471700 1 -- 172.21.15.40:0/1370146026 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f093c064fe0 msgr2=0x7f093c067490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0960471700 1 --2- 172.21.15.40:0/1370146026 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f093c064fe0 0x7f093c067490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f095002de60 tx=0x7f0948008040 comp rx=0 tx=0).stop 2023-11-08T08:19:35.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0758f90700 1 -- 172.21.15.40:0/3223771923 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0738064ee0 msgr2=0x7f0738067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0758f90700 1 --2- 172.21.15.40:0/3223771923 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0738064ee0 0x7f0738067390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f074c04ed90 tx=0x7f0748004a70 comp rx=0 tx=0).stop 2023-11-08T08:19:35.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0758f90700 1 -- 172.21.15.40:0/3223771923 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f074c0e6d20 msgr2=0x7f074c04adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0758f90700 1 --2- 172.21.15.40:0/3223771923 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f074c0e6d20 0x7f074c04adf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0754065690 tx=0x7f07540728f0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0960471700 1 -- 172.21.15.40:0/1370146026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0944005220 msgr2=0x7f09500d9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0960471700 1 --2- 172.21.15.40:0/1370146026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0944005220 0x7f09500d9e30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f095400fae0 tx=0x7f095400fdf0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7fb02b7fe700 1 -- 172.21.15.40:0/1328659828 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb024064f30 msgr2=0x7fb0240673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7fb02b7fe700 1 --2- 172.21.15.40:0/1328659828 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb024064f30 0x7fb0240673e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb04015af30 tx=0x7fb038009040 comp rx=0 tx=0).stop 2023-11-08T08:19:35.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0758f90700 1 -- 172.21.15.40:0/3223771923 shutdown_connections 2023-11-08T08:19:35.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0758f90700 1 --2- 172.21.15.40:0/3223771923 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0738064ee0 0x7f0738067390 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0758f90700 1 --2- 172.21.15.40:0/3223771923 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f074c0e6d20 0x7f074c04adf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0758f90700 1 --2- 172.21.15.40:0/3223771923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f074c051170 0x7f074c04a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0758f90700 1 --2- 172.21.15.40:0/3223771923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f074c036b70 0x7f074c0e4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0758f90700 1 -- 172.21.15.40:0/3223771923 >> 172.21.15.40:0/3223771923 conn(0x7f074c051630 msgr2=0x7f074c035860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:35.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7fb02b7fe700 1 -- 172.21.15.40:0/1328659828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0401534a0 msgr2=0x7fb0401fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7fb02b7fe700 1 --2- 172.21.15.40:0/1328659828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0401534a0 0x7fb0401fb990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009c60 tx=0x7fb03c004770 comp rx=0 tx=0).stop 2023-11-08T08:19:35.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0960471700 1 -- 172.21.15.40:0/1370146026 shutdown_connections 2023-11-08T08:19:35.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0960471700 1 --2- 172.21.15.40:0/1370146026 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f093c064fe0 0x7f093c067490 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0960471700 1 --2- 172.21.15.40:0/1370146026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f095004ccd0 0x7f095004e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0960471700 1 --2- 172.21.15.40:0/1370146026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09500ddb60 0x7f095004c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0960471700 1 --2- 172.21.15.40:0/1370146026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0944005220 0x7f09500d9e30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0960471700 1 -- 172.21.15.40:0/1370146026 >> 172.21.15.40:0/1370146026 conn(0x7f0950051810 msgr2=0x7f0950006a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:35.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0758f90700 1 -- 172.21.15.40:0/3223771923 shutdown_connections 2023-11-08T08:19:35.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0758f90700 1 -- 172.21.15.40:0/3223771923 wait complete. 2023-11-08T08:19:35.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f13d759e700 1 -- 172.21.15.40:0/1906160524 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f13bc0055c0 con 0x7f13c0004870 2023-11-08T08:19:35.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7fb02b7fe700 1 -- 172.21.15.40:0/1328659828 shutdown_connections 2023-11-08T08:19:35.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0960471700 1 -- 172.21.15.40:0/1370146026 shutdown_connections 2023-11-08T08:19:35.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7fb02b7fe700 1 --2- 172.21.15.40:0/1328659828 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb024064f30 0x7fb0240673e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7fb02b7fe700 1 --2- 172.21.15.40:0/1328659828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb04015d510 0x7fb0401fbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7fb02b7fe700 1 --2- 172.21.15.40:0/1328659828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0401534a0 0x7fb0401fb990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7fb02b7fe700 1 --2- 172.21.15.40:0/1328659828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0400b6030 0x7fb040159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f0960471700 1 -- 172.21.15.40:0/1370146026 wait complete. 2023-11-08T08:19:35.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7fb02b7fe700 1 -- 172.21.15.40:0/1328659828 >> 172.21.15.40:0/1328659828 conn(0x7fb04006bfc0 msgr2=0x7fb0400d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:35.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7fb02b7fe700 1 -- 172.21.15.40:0/1328659828 shutdown_connections 2023-11-08T08:19:35.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7fb02b7fe700 1 -- 172.21.15.40:0/1328659828 wait complete. 2023-11-08T08:19:35.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.326+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1906160524 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f13cc053580 con 0x7f13c0004870 2023-11-08T08:19:35.345 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2023-11-08T08:19:35.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.334+0000 7f13d759e700 1 -- 172.21.15.40:0/1906160524 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f13b8064ee0 msgr2=0x7f13b8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.334+0000 7f13d759e700 1 --2- 172.21.15.40:0/1906160524 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f13b8064ee0 0x7f13b8067390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f13d80b4080 tx=0x7f13d0009070 comp rx=0 tx=0).stop 2023-11-08T08:19:35.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.334+0000 7f13d759e700 1 -- 172.21.15.40:0/1906160524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13c0004870 msgr2=0x7f13d8155dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.334+0000 7f13d759e700 1 --2- 172.21.15.40:0/1906160524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13c0004870 0x7f13d8155dd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00b580 tx=0x7f13cc00b940 comp rx=0 tx=0).stop 2023-11-08T08:19:35.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.334+0000 7f13d759e700 1 -- 172.21.15.40:0/1906160524 shutdown_connections 2023-11-08T08:19:35.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.334+0000 7f13d759e700 1 --2- 172.21.15.40:0/1906160524 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f13b8064ee0 0x7f13b8067390 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f13d80b4080 tx=0x7f13d0009070 comp rx=0 tx=0).stop 2023-11-08T08:19:35.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.334+0000 7f13d759e700 1 --2- 172.21.15.40:0/1906160524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13d80b3380 0x7f13d80b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.334+0000 7f13d759e700 1 --2- 172.21.15.40:0/1906160524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13d80c61f0 0x7f13d80c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.334+0000 7f13d759e700 1 --2- 172.21.15.40:0/1906160524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13c0004870 0x7f13d8155dd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.334+0000 7f13d759e700 1 -- 172.21.15.40:0/1906160524 >> 172.21.15.40:0/1906160524 conn(0x7f13d80b0f90 msgr2=0x7f13d806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:35.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.338+0000 7f13d759e700 1 -- 172.21.15.40:0/1906160524 shutdown_connections 2023-11-08T08:19:35.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.338+0000 7f13d759e700 1 -- 172.21.15.40:0/1906160524 wait complete. 2023-11-08T08:19:35.350 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.2 2023-11-08T08:19:35.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.338+0000 7fe5ca984700 1 -- 172.21.15.40:0/3590670420 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe5c4072010 con 0x7fe5c4071aa0 2023-11-08T08:19:35.351 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.3 2023-11-08T08:19:35.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.342+0000 7fe5ba7fc700 1 -- 172.21.15.40:0/3590670420 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe5b4017020 con 0x7fe5c4071aa0 2023-11-08T08:19:35.352 INFO:teuthology.orchestra.run.smithi040.stdout:55834574857 2023-11-08T08:19:35.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.342+0000 7fe5ca984700 1 -- 172.21.15.40:0/3590670420 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe5a8064f30 msgr2=0x7fe5a80673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.342+0000 7fe5ca984700 1 --2- 172.21.15.40:0/3590670420 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe5a8064f30 0x7fe5a80673e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009c60 tx=0x7fe5c0008030 comp rx=0 tx=0).stop 2023-11-08T08:19:35.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.342+0000 7fe5ca984700 1 -- 172.21.15.40:0/3590670420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c4071aa0 msgr2=0x7fe5c40cc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:35.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.342+0000 7fe5ca984700 1 --2- 172.21.15.40:0/3590670420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c4071aa0 0x7fe5c40cc470 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400c500 tx=0x7fe5b400c810 comp rx=0 tx=0).stop 2023-11-08T08:19:35.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.342+0000 7fe5ca984700 1 -- 172.21.15.40:0/3590670420 shutdown_connections 2023-11-08T08:19:35.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.342+0000 7fe5ca984700 1 --2- 172.21.15.40:0/3590670420 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe5a8064f30 0x7fe5a80673e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.342+0000 7fe5ca984700 1 --2- 172.21.15.40:0/3590670420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5c415c640 0x7fe5c41fe010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.342+0000 7fe5ca984700 1 --2- 172.21.15.40:0/3590670420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5c4071aa0 0x7fe5c40cc470 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.342+0000 7fe5ca984700 1 --2- 172.21.15.40:0/3590670420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c4069e00 0x7fe5c40c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:35.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.342+0000 7fe5ca984700 1 -- 172.21.15.40:0/3590670420 >> 172.21.15.40:0/3590670420 conn(0x7fe5c40b6520 msgr2=0x7fe5c406d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:35.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.346+0000 7fe5ca984700 1 -- 172.21.15.40:0/3590670420 shutdown_connections 2023-11-08T08:19:35.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:35.346+0000 7fe5ca984700 1 -- 172.21.15.40:0/3590670420 wait complete. 2023-11-08T08:19:35.359 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.0 2023-11-08T08:19:35.360 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.6 2023-11-08T08:19:35.361 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.4 2023-11-08T08:19:35.363 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.7 2023-11-08T08:19:35.363 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.5 2023-11-08T08:19:35.366 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.1 2023-11-08T08:19:36.351 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-08T08:19:36.352 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-08T08:19:36.360 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-08T08:19:36.361 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-08T08:19:36.362 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-08T08:19:36.365 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-08T08:19:36.365 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-08T08:19:36.366 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-08T08:19:36.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.578+0000 7f848f59e700 1 Processor -- start 2023-11-08T08:19:36.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.578+0000 7f848f59e700 1 -- start start 2023-11-08T08:19:36.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.582+0000 7f848e59c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f84880e29f0 0x7f8488055490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49744/0 (socket says 172.21.15.40:49744) 2023-11-08T08:19:36.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.582+0000 7f848e59c700 1 -- 172.21.15.40:0/2058910687 learned_addr learned my addr 172.21.15.40:0/2058910687 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:36.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.582+0000 7f848f59e700 1 --2- 172.21.15.40:0/2058910687 >> v2:172.21.15.169:3300/0 conn(0x7f8488038560 0x7f848803ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.582+0000 7f848f59e700 1 -- 172.21.15.40:0/2058910687 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8488007360 con 0x7f84880e29f0 2023-11-08T08:19:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.582+0000 7f848f59e700 1 -- 172.21.15.40:0/2058910687 --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8488053f70 con 0x7f848803b160 2023-11-08T08:19:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.582+0000 7f848dd9b700 1 --2- 172.21.15.40:0/2058910687 >> v2:172.21.15.169:3300/0 conn(0x7f8488038560 0x7f848803ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.582+0000 7f848f59e700 1 -- 172.21.15.40:0/2058910687 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84880e2dc0 con 0x7f8488038560 2023-11-08T08:19:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.582+0000 7f54e7630700 1 Processor -- start 2023-11-08T08:19:36.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848d59a700 1 -- 172.21.15.40:0/2058910687 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 118495321 0 0) 0x7f8488053f70 con 0x7f848803b160 2023-11-08T08:19:36.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848d59a700 1 -- 172.21.15.40:0/2058910687 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84780035d0 con 0x7f848803b160 2023-11-08T08:19:36.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f54e7630700 1 -- start start 2023-11-08T08:19:36.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848d59a700 1 -- 172.21.15.40:0/2058910687 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400990283 0 0) 0x7f8488007360 con 0x7f84880e29f0 2023-11-08T08:19:36.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848d59a700 1 -- 172.21.15.40:0/2058910687 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8488053f70 con 0x7f84880e29f0 2023-11-08T08:19:36.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848d59a700 1 -- 172.21.15.40:0/2058910687 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1633375750 0 0) 0x7f8488053f70 con 0x7f84880e29f0 2023-11-08T08:19:36.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848d59a700 1 -- 172.21.15.40:0/2058910687 >> v2:172.21.15.169:3300/0 conn(0x7f8488038560 msgr2=0x7f848803ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:36.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848d59a700 1 --2- 172.21.15.40:0/2058910687 >> v2:172.21.15.169:3300/0 conn(0x7f8488038560 0x7f848803ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848d59a700 1 -- 172.21.15.40:0/2058910687 >> v1:172.21.15.169:6789/0 conn(0x7f848803b160 legacy=0x7f8488057210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848d59a700 1 -- 172.21.15.40:0/2058910687 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848803b5a0 con 0x7f84880e29f0 2023-11-08T08:19:36.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848d59a700 1 -- 172.21.15.40:0/2058910687 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f8480001670 con 0x7f84880e29f0 2023-11-08T08:19:36.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f54e7630700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54e00710d0 0x7f54e015cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f54e7630700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e015d1e0 con 0x7f54e01587b0 2023-11-08T08:19:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f54e7630700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e0158b80 con 0x7f54e00ccc60 2023-11-08T08:19:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f54e7630700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e006db00 con 0x7f54e00710d0 2023-11-08T08:19:36.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848d59a700 1 -- 172.21.15.40:0/2058910687 >> v1:172.21.15.40:6789/0 conn(0x7f84880e29f0 legacy=0x7f8488055490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848d59a700 1 --2- 172.21.15.40:0/2058910687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8478003bb0 0x7f8478003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848d59a700 1 --2- 172.21.15.40:0/2058910687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8478004870 0x7f8478004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848d59a700 1 --2- 172.21.15.40:0/2058910687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8478005220 0x7f84780141b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848d59a700 1 -- 172.21.15.40:0/2058910687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84880e2dc0 con 0x7f8478004870 2023-11-08T08:19:36.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848d59a700 1 -- 172.21.15.40:0/2058910687 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8478014720 con 0x7f8478003bb0 2023-11-08T08:19:36.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848d59a700 1 -- 172.21.15.40:0/2058910687 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84780148e0 con 0x7f8478005220 2023-11-08T08:19:36.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848ed9d700 1 --2- 172.21.15.40:0/2058910687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8478005220 0x7f84780141b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848e59c700 1 --2- 172.21.15.40:0/2058910687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8478003bb0 0x7f8478003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848dd9b700 1 --2- 172.21.15.40:0/2058910687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8478004870 0x7f8478004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f54e53cc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f54e01587b0 0x7f54e006d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49752/0 (socket says 172.21.15.40:49752) 2023-11-08T08:19:36.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f54e53cc700 1 -- 172.21.15.40:0/2207038516 learned_addr learned my addr 172.21.15.40:0/2207038516 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:36.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f54e5bcd700 1 --2- 172.21.15.40:0/2207038516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54e00710d0 0x7f54e015cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848ed9d700 1 -- 172.21.15.40:0/2058910687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8478003bb0 msgr2=0x7f8478003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:36.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848ed9d700 1 --2- 172.21.15.40:0/2058910687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8478003bb0 0x7f8478003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.586+0000 7f848ed9d700 1 -- 172.21.15.40:0/2058910687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8478004870 msgr2=0x7f8478004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:36.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f848ed9d700 1 --2- 172.21.15.40:0/2058910687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8478004870 0x7f8478004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f848ed9d700 1 -- 172.21.15.40:0/2058910687 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848803b5a0 con 0x7f8478005220 2023-11-08T08:19:36.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f848ed9d700 1 -- 172.21.15.40:0/2058910687 reap_dead start 2023-11-08T08:19:36.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f848dd9b700 1 --2- 172.21.15.40:0/2058910687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8478004870 0x7f8478004ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:36.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f54e5bcd700 1 -- 172.21.15.40:0/2207038516 >> v1:172.21.15.169:6789/0 conn(0x7f54e00ccc60 legacy=0x7f54e00709c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:36.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f54e5bcd700 1 -- 172.21.15.40:0/2207038516 >> v1:172.21.15.40:6789/0 conn(0x7f54e01587b0 legacy=0x7f54e006d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f54e5bcd700 1 -- 172.21.15.40:0/2207038516 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e006c070 con 0x7f54e00710d0 2023-11-08T08:19:36.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f848ed9d700 1 --2- 172.21.15.40:0/2058910687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8478005220 0x7f84780141b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f849004f8c0 tx=0x7f84780058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8e6d558b1e953a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f848d59a700 1 -- 172.21.15.40:0/2058910687 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8490072cc0 con 0x7f8478005220 2023-11-08T08:19:36.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f848d59a700 1 -- 172.21.15.40:0/2058910687 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8490066b40 con 0x7f8478005220 2023-11-08T08:19:36.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f848d59a700 1 -- 172.21.15.40:0/2058910687 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8490066ef0 con 0x7f8478005220 2023-11-08T08:19:36.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f54e5bcd700 1 --2- 172.21.15.40:0/2207038516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54e00710d0 0x7f54e015cca0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f54d000aa20 tx=0x7f54d000ad30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c6468f496bde03 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f848f59e700 1 -- 172.21.15.40:0/2058910687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8478005220 msgr2=0x7f84780141b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f848f59e700 1 --2- 172.21.15.40:0/2058910687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8478005220 0x7f84780141b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f849004f8c0 tx=0x7f84780058a0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f848f59e700 1 -- 172.21.15.40:0/2058910687 shutdown_connections 2023-11-08T08:19:36.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f848f59e700 1 --2- 172.21.15.40:0/2058910687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8478005220 0x7f84780141b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f848f59e700 1 -- 172.21.15.40:0/2058910687 >> 172.21.15.40:0/2058910687 conn(0x7f8488055c20 msgr2=0x7f8488056020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:36.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f848f59e700 1 -- 172.21.15.40:0/2058910687 shutdown_connections 2023-11-08T08:19:36.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f848f59e700 1 -- 172.21.15.40:0/2058910687 wait complete. 2023-11-08T08:19:36.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f54dffff700 1 -- 172.21.15.40:0/2207038516 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d0004230 con 0x7f54e00710d0 2023-11-08T08:19:36.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f54dffff700 1 -- 172.21.15.40:0/2207038516 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d0004390 con 0x7f54e00710d0 2023-11-08T08:19:36.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.590+0000 7f54dffff700 1 -- 172.21.15.40:0/2207038516 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d000c870 con 0x7f54e00710d0 2023-11-08T08:19:36.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.594+0000 7f54e7630700 1 -- 172.21.15.40:0/2207038516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54e00710d0 msgr2=0x7f54e015cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.594+0000 7f54e7630700 1 --2- 172.21.15.40:0/2207038516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54e00710d0 0x7f54e015cca0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f54d000aa20 tx=0x7f54d000ad30 comp rx=0 tx=0).stop 2023-11-08T08:19:36.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.594+0000 7f54e7630700 1 -- 172.21.15.40:0/2207038516 shutdown_connections 2023-11-08T08:19:36.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.594+0000 7f54e7630700 1 --2- 172.21.15.40:0/2207038516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54e00710d0 0x7f54e015cca0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.594+0000 7f54e7630700 1 -- 172.21.15.40:0/2207038516 >> 172.21.15.40:0/2207038516 conn(0x7f54e00ce6c0 msgr2=0x7f54e00cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:36.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.594+0000 7f54e7630700 1 -- 172.21.15.40:0/2207038516 shutdown_connections 2023-11-08T08:19:36.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.594+0000 7f54e7630700 1 -- 172.21.15.40:0/2207038516 wait complete. 2023-11-08T08:19:36.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.594+0000 7f848f59e700 1 Processor -- start 2023-11-08T08:19:36.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.594+0000 7fafe6f2d700 1 Processor -- start 2023-11-08T08:19:36.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.594+0000 7fafe6f2d700 1 -- start start 2023-11-08T08:19:36.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.594+0000 7f848f59e700 1 -- start start 2023-11-08T08:19:36.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f848f59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8478005220 0x7f84880d1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7fafe6f2d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd8057460 0x7fafd80362e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7fafe6f2d700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd80e81f0 con 0x7fafd8035c00 2023-11-08T08:19:36.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7fafe6f2d700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd80e7430 con 0x7fafd80e6d60 2023-11-08T08:19:36.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f848f59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84880d6530 0x7f84880d1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7fafe6f2d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd8036100 con 0x7fafd8057460 2023-11-08T08:19:36.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f54e7630700 1 Processor -- start 2023-11-08T08:19:36.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f54e7630700 1 -- start start 2023-11-08T08:19:36.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f848f59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8488050970 0x7f84880d6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f848f59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84880da6a0 con 0x7f8488050970 2023-11-08T08:19:36.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f848f59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84880d9780 con 0x7f8478005220 2023-11-08T08:19:36.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f848f59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84880d9a20 con 0x7f84880d6530 2023-11-08T08:19:36.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7fafdffff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fafd8035c00 0x7fafd80534b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49760/0 (socket says 172.21.15.40:49760) 2023-11-08T08:19:36.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7fafdffff700 1 -- 172.21.15.40:0/2659043363 learned_addr learned my addr 172.21.15.40:0/2659043363 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:36.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f848e59c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8478005220 0x7f84880d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f848dd9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84880d6530 0x7f84880d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f848e59c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8478005220 0x7f84880d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33740/0 (socket says 172.21.15.40:33740) 2023-11-08T08:19:36.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f848dd9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84880d6530 0x7f84880d1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:53588/0 (socket says 172.21.15.40:53588) 2023-11-08T08:19:36.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7fafe54ca700 1 --2- 172.21.15.40:0/2659043363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd8057460 0x7fafd80362e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f848e59c700 1 -- 172.21.15.40:0/3623561651 learned_addr learned my addr 172.21.15.40:0/3623561651 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:36.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f848ed9d700 1 --2- 172.21.15.40:0/3623561651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8488050970 0x7f84880d6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f54e7630700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54e00710d0 0x7f54e0151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f54e7630700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54e00ccc60 0x7f54e015b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f54e7630700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54e01587b0 0x7f54e015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f54e7630700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e00c3c30 con 0x7f54e00ccc60 2023-11-08T08:19:36.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f54e7630700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e00c44d0 con 0x7f54e01587b0 2023-11-08T08:19:36.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f54e7630700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e00b41c0 con 0x7f54e00710d0 2023-11-08T08:19:36.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f54e53cc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54e00710d0 0x7f54e0151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f54e53cc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54e00710d0 0x7f54e0151510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:53618/0 (socket says 172.21.15.40:53618) 2023-11-08T08:19:36.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f54e53cc700 1 -- 172.21.15.40:0/956480771 learned_addr learned my addr 172.21.15.40:0/956480771 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:36.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f54e5bcd700 1 --2- 172.21.15.40:0/956480771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54e01587b0 0x7f54e015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f848dd9b700 1 -- 172.21.15.40:0/3623561651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8478005220 msgr2=0x7f84880d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f848dd9b700 1 --2- 172.21.15.40:0/3623561651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8478005220 0x7f84880d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f848dd9b700 1 -- 172.21.15.40:0/3623561651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8488050970 msgr2=0x7f84880d6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f848dd9b700 1 --2- 172.21.15.40:0/3623561651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8488050970 0x7f84880d6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f848dd9b700 1 -- 172.21.15.40:0/3623561651 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8488007080 con 0x7f84880d6530 2023-11-08T08:19:36.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f848e59c700 1 --2- 172.21.15.40:0/3623561651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8478005220 0x7f84880d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:36.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f848ed9d700 1 --2- 172.21.15.40:0/3623561651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8488050970 0x7f84880d6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:36.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f33b550d700 1 Processor -- start 2023-11-08T08:19:36.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f33b550d700 1 -- start start 2023-11-08T08:19:36.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7fafe54ca700 1 -- 172.21.15.40:0/2659043363 >> v1:172.21.15.169:6789/0 conn(0x7fafd80e6d60 legacy=0x7fafd8056d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7fafe54ca700 1 -- 172.21.15.40:0/2659043363 >> v1:172.21.15.40:6789/0 conn(0x7fafd8035c00 legacy=0x7fafd80534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7fafe54ca700 1 -- 172.21.15.40:0/2659043363 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd80e7710 con 0x7fafd8057460 2023-11-08T08:19:36.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f54e4bcb700 1 --2- 172.21.15.40:0/956480771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54e00ccc60 0x7f54e015b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.598+0000 7f848dd9b700 1 --2- 172.21.15.40:0/3623561651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84880d6530 0x7f84880d1540 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8488004320 tx=0x7f8484011eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f54e53cc700 1 -- 172.21.15.40:0/956480771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54e01587b0 msgr2=0x7f54e015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7fafe54ca700 1 --2- 172.21.15.40:0/2659043363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd8057460 0x7fafd80362e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fafe0000b60 tx=0x7fafe00618d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=675fd6a76c8d7533 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f54e53cc700 1 --2- 172.21.15.40:0/956480771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54e01587b0 0x7f54e015b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f54e53cc700 1 -- 172.21.15.40:0/956480771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54e00ccc60 msgr2=0x7f54e015b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f54e53cc700 1 --2- 172.21.15.40:0/956480771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54e00ccc60 0x7f54e015b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f54e53cc700 1 -- 172.21.15.40:0/956480771 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d000a650 con 0x7f54e00710d0 2023-11-08T08:19:36.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f54e5bcd700 1 --2- 172.21.15.40:0/956480771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54e01587b0 0x7f54e015b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:36.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f54e4bcb700 1 --2- 172.21.15.40:0/956480771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54e00ccc60 0x7f54e015b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:36.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f847effd700 1 -- 172.21.15.40:0/3623561651 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8484004020 con 0x7f84880d6530 2023-11-08T08:19:36.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7fafdf7fe700 1 -- 172.21.15.40:0/2659043363 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe0066b40 con 0x7fafd8057460 2023-11-08T08:19:36.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7fafdf7fe700 1 -- 172.21.15.40:0/2659043363 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe0066ca0 con 0x7fafd8057460 2023-11-08T08:19:36.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f847effd700 1 -- 172.21.15.40:0/3623561651 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8484004180 con 0x7f84880d6530 2023-11-08T08:19:36.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f847effd700 1 -- 172.21.15.40:0/3623561651 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8484004510 con 0x7f84880d6530 2023-11-08T08:19:36.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f848f59e700 1 -- 172.21.15.40:0/3623561651 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8488006dc0 con 0x7f84880d6530 2023-11-08T08:19:36.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f848f59e700 1 -- 172.21.15.40:0/3623561651 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8488053920 con 0x7f84880d6530 2023-11-08T08:19:36.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f54e53cc700 1 --2- 172.21.15.40:0/956480771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54e00710d0 0x7f54e0151510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f54d800c730 tx=0x7f54d800ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f54ddffb700 1 -- 172.21.15.40:0/956480771 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d8004ca0 con 0x7f54e00710d0 2023-11-08T08:19:36.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f33b550d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f33b015c640 0x7f33b00d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f33b550d700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b006c600 con 0x7f33b0071aa0 2023-11-08T08:19:36.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f33b550d700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b00722f0 con 0x7f33b0069e00 2023-11-08T08:19:36.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f33b550d700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b006cef0 con 0x7f33b015c640 2023-11-08T08:19:36.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7fafe6f2d700 1 -- 172.21.15.40:0/2659043363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd8057460 msgr2=0x7fafd80362e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7fafdf7fe700 1 -- 172.21.15.40:0/2659043363 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe0066b40 con 0x7fafd8057460 2023-11-08T08:19:36.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7fafe6f2d700 1 --2- 172.21.15.40:0/2659043363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd8057460 0x7fafd80362e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fafe0000b60 tx=0x7fafe00618d0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f54ddffb700 1 -- 172.21.15.40:0/956480771 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d8004e00 con 0x7f54e00710d0 2023-11-08T08:19:36.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f54ddffb700 1 -- 172.21.15.40:0/956480771 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d8007310 con 0x7f54e00710d0 2023-11-08T08:19:36.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f54e7630700 1 -- 172.21.15.40:0/956480771 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e006c630 con 0x7f54e00710d0 2023-11-08T08:19:36.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f54e7630700 1 -- 172.21.15.40:0/956480771 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e006c350 con 0x7f54e00710d0 2023-11-08T08:19:36.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f33aeffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f33b015c640 0x7f33b00d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f33aeffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f33b015c640 0x7f33b00d31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52500/0 (socket says 172.21.15.40:52500) 2023-11-08T08:19:36.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7f33aeffd700 1 -- 172.21.15.40:0/1643496990 learned_addr learned my addr 172.21.15.40:0/1643496990 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:36.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7fafe6f2d700 1 -- 172.21.15.40:0/2659043363 shutdown_connections 2023-11-08T08:19:36.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7fafe6f2d700 1 --2- 172.21.15.40:0/2659043363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd8057460 0x7fafd80362e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.602+0000 7fafe6f2d700 1 -- 172.21.15.40:0/2659043363 >> 172.21.15.40:0/2659043363 conn(0x7fafd8009f40 msgr2=0x7fafd800a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:36.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.606+0000 7f33aeffd700 1 -- 172.21.15.40:0/1643496990 >> v1:172.21.15.169:6789/0 conn(0x7f33b0069e00 legacy=0x7f33b00d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:36.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.606+0000 7f33aeffd700 1 -- 172.21.15.40:0/1643496990 >> v1:172.21.15.40:6789/0 conn(0x7f33b0071aa0 legacy=0x7f33b006d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.606+0000 7f33aeffd700 1 -- 172.21.15.40:0/1643496990 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b006da20 con 0x7f33b015c640 2023-11-08T08:19:36.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.606+0000 7fafe6f2d700 1 -- 172.21.15.40:0/2659043363 shutdown_connections 2023-11-08T08:19:36.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.606+0000 7f54e7630700 1 -- 172.21.15.40:0/956480771 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e006c910 con 0x7f54e00710d0 2023-11-08T08:19:36.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.606+0000 7f848f59e700 1 -- 172.21.15.40:0/3623561651 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8474005310 con 0x7f84880d6530 2023-11-08T08:19:36.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.606+0000 7fafe6f2d700 1 -- 172.21.15.40:0/2659043363 wait complete. 2023-11-08T08:19:36.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.606+0000 7f49d6b5e700 1 Processor -- start 2023-11-08T08:19:36.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.606+0000 7f49d6b5e700 1 -- start start 2023-11-08T08:19:36.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.606+0000 7f33aeffd700 1 --2- 172.21.15.40:0/1643496990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b015c640 0x7f33b00d31e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009910 tx=0x7f33a4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0ced92c352bb32 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.606+0000 7f54ddffb700 1 -- 172.21.15.40:0/956480771 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f54d801a030 con 0x7f54e00710d0 2023-11-08T08:19:36.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.606+0000 7f847effd700 1 -- 172.21.15.40:0/3623561651 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f8484024070 con 0x7f84880d6530 2023-11-08T08:19:36.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.606+0000 7f33adffb700 1 -- 172.21.15.40:0/1643496990 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a400ba60 con 0x7f33b015c640 2023-11-08T08:19:36.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.606+0000 7f33adffb700 1 -- 172.21.15.40:0/1643496990 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a400bbc0 con 0x7f33b015c640 2023-11-08T08:19:36.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.606+0000 7f33adffb700 1 -- 172.21.15.40:0/1643496990 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4009e60 con 0x7f33b015c640 2023-11-08T08:19:36.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.606+0000 7f49d6b5e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d00cdfb0 0x7f49d006cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.606+0000 7f49d6b5e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49d006d2f0 0x7f49d006a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.606+0000 7f49d6b5e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d0164080 con 0x7f49d0163cb0 2023-11-08T08:19:36.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.606+0000 7f49d6b5e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d00cc290 con 0x7f49d006d2f0 2023-11-08T08:19:36.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.606+0000 7f49d6b5e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d006d770 con 0x7f49d00cdfb0 2023-11-08T08:19:36.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f33b550d700 1 -- 172.21.15.40:0/1643496990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b015c640 msgr2=0x7f33b00d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f33b550d700 1 --2- 172.21.15.40:0/1643496990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b015c640 0x7f33b00d31e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009910 tx=0x7f33a4009c20 comp rx=0 tx=0).stop 2023-11-08T08:19:36.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f54ddffb700 1 --2- 172.21.15.40:0/956480771 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f54cc064f30 0x7f54cc0673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f54ddffb700 1 -- 172.21.15.40:0/956480771 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f54d80815a0 con 0x7f54e00710d0 2023-11-08T08:19:36.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f49cffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d00cdfb0 0x7f49d006cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f33b550d700 1 -- 172.21.15.40:0/1643496990 shutdown_connections 2023-11-08T08:19:36.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f49d48fa700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f49d0163cb0 0x7f49d0164cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49784/0 (socket says 172.21.15.40:49784) 2023-11-08T08:19:36.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f49d48fa700 1 -- 172.21.15.40:0/2705860089 learned_addr learned my addr 172.21.15.40:0/2705860089 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:36.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f33b550d700 1 --2- 172.21.15.40:0/1643496990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b015c640 0x7f33b00d31e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f33b550d700 1 -- 172.21.15.40:0/1643496990 >> 172.21.15.40:0/1643496990 conn(0x7f33b00b6520 msgr2=0x7f33b0071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:36.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f49d50fb700 1 --2- 172.21.15.40:0/2705860089 >> v2:172.21.15.40:3300/0 conn(0x7f49d006d2f0 0x7f49d006a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f33b550d700 1 -- 172.21.15.40:0/1643496990 shutdown_connections 2023-11-08T08:19:36.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f54e4bcb700 1 --2- 172.21.15.40:0/956480771 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f54cc064f30 0x7f54cc0673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f33b550d700 1 -- 172.21.15.40:0/1643496990 wait complete. 2023-11-08T08:19:36.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f49cffff700 1 -- 172.21.15.40:0/2705860089 >> v2:172.21.15.40:3300/0 conn(0x7f49d006d2f0 msgr2=0x7f49d006a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:36.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f49cffff700 1 --2- 172.21.15.40:0/2705860089 >> v2:172.21.15.40:3300/0 conn(0x7f49d006d2f0 0x7f49d006a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f49cffff700 1 -- 172.21.15.40:0/2705860089 >> v1:172.21.15.40:6789/0 conn(0x7f49d0163cb0 legacy=0x7f49d0164cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f49cffff700 1 -- 172.21.15.40:0/2705860089 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d0069bf0 con 0x7f49d00cdfb0 2023-11-08T08:19:36.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f49cffff700 1 --2- 172.21.15.40:0/2705860089 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d00cdfb0 0x7f49d006cdb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009880 tx=0x7f49c0009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de4ddbf11a119cd4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7fafe6f2d700 1 Processor -- start 2023-11-08T08:19:36.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7fafe6f2d700 1 -- start start 2023-11-08T08:19:36.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7fafe6f2d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd8035c00 0x7fafd8050320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f54e4bcb700 1 --2- 172.21.15.40:0/956480771 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f54cc064f30 0x7f54cc0673e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f54e00b3f50 tx=0x7f54d40042e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7fafe6f2d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd8057460 0x7fafd804c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7fafe6f2d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd80e6d60 0x7fafd8051470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7fafe6f2d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd804df10 con 0x7fafd80e6d60 2023-11-08T08:19:36.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7fafe6f2d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd804ce60 con 0x7fafd8035c00 2023-11-08T08:19:36.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7fafe6f2d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd804d140 con 0x7fafd8057460 2023-11-08T08:19:36.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f847effd700 1 --2- 172.21.15.40:0/3623561651 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8470064fb0 0x7f8470067460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f847effd700 1 -- 172.21.15.40:0/3623561651 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8484082220 con 0x7f84880d6530 2023-11-08T08:19:36.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7fafe4cc9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd8035c00 0x7fafd8050320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7fafe4cc9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd8035c00 0x7fafd8050320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33762/0 (socket says 172.21.15.40:33762) 2023-11-08T08:19:36.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7fafe4cc9700 1 -- 172.21.15.40:0/970029501 learned_addr learned my addr 172.21.15.40:0/970029501 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:36.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7fafdffff700 1 --2- 172.21.15.40:0/970029501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd8057460 0x7fafd804c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7fafe54ca700 1 --2- 172.21.15.40:0/970029501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd80e6d60 0x7fafd8051470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.610+0000 7f848e59c700 1 --2- 172.21.15.40:0/3623561651 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8470064fb0 0x7f8470067460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7fafe4cc9700 1 -- 172.21.15.40:0/970029501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd8057460 msgr2=0x7fafd804c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7fafe4cc9700 1 --2- 172.21.15.40:0/970029501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd8057460 0x7fafd804c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7fafe4cc9700 1 -- 172.21.15.40:0/970029501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd80e6d60 msgr2=0x7fafd8051470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7fafe4cc9700 1 --2- 172.21.15.40:0/970029501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd80e6d60 0x7fafd8051470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7fafe4cc9700 1 -- 172.21.15.40:0/970029501 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd80e7cd0 con 0x7fafd8035c00 2023-11-08T08:19:36.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7fafe4cc9700 1 --2- 172.21.15.40:0/970029501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd8035c00 0x7fafd8050320 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fafd400c500 tx=0x7fafd400c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7f33b550d700 1 Processor -- start 2023-11-08T08:19:36.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7fafdd7fa700 1 -- 172.21.15.40:0/970029501 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd40052d0 con 0x7fafd8035c00 2023-11-08T08:19:36.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7fafe6f2d700 1 -- 172.21.15.40:0/970029501 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd80e79f0 con 0x7fafd8035c00 2023-11-08T08:19:36.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7fafdd7fa700 1 -- 172.21.15.40:0/970029501 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd4005430 con 0x7fafd8035c00 2023-11-08T08:19:36.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7fafdd7fa700 1 -- 172.21.15.40:0/970029501 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd40055a0 con 0x7fafd8035c00 2023-11-08T08:19:36.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7fafe6f2d700 1 -- 172.21.15.40:0/970029501 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd80e7130 con 0x7fafd8035c00 2023-11-08T08:19:36.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7f33b550d700 1 -- start start 2023-11-08T08:19:36.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7f848e59c700 1 --2- 172.21.15.40:0/3623561651 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8470064fb0 0x7f8470067460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8480001a30 tx=0x7f84800079a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7fafe6f2d700 1 -- 172.21.15.40:0/970029501 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd804d310 con 0x7fafd8035c00 2023-11-08T08:19:36.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7f33b550d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0069e00 0x7f33b00ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7f33b550d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33b0071aa0 0x7f33b00b4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7f33b550d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33b015c640 0x7f33b00c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7f33b550d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b00b4eb0 con 0x7f33b0069e00 2023-11-08T08:19:36.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7f33b550d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b009c2b0 con 0x7f33b015c640 2023-11-08T08:19:36.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7f33b550d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b00cbdd0 con 0x7f33b0071aa0 2023-11-08T08:19:36.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7f847effd700 1 -- 172.21.15.40:0/3623561651 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f8484050060 con 0x7f84880d6530 2023-11-08T08:19:36.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7f33ae7fc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33b0071aa0 0x7f33b00b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7f33aeffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0069e00 0x7f33b00ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7f33af7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33b015c640 0x7f33b00c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7f33aeffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0069e00 0x7f33b00ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52526/0 (socket says 172.21.15.40:52526) 2023-11-08T08:19:36.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.614+0000 7f33aeffd700 1 -- 172.21.15.40:0/834729771 learned_addr learned my addr 172.21.15.40:0/834729771 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:36.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.618+0000 7f33aeffd700 1 -- 172.21.15.40:0/834729771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33b0071aa0 msgr2=0x7f33b00b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.618+0000 7f33aeffd700 1 --2- 172.21.15.40:0/834729771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33b0071aa0 0x7f33b00b4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.618+0000 7f33aeffd700 1 -- 172.21.15.40:0/834729771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33b015c640 msgr2=0x7f33b00c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.618+0000 7f33aeffd700 1 --2- 172.21.15.40:0/834729771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33b015c640 0x7f33b00c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.618+0000 7f33aeffd700 1 -- 172.21.15.40:0/834729771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a4009570 con 0x7f33b0069e00 2023-11-08T08:19:36.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.618+0000 7f33aeffd700 1 --2- 172.21.15.40:0/834729771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0069e00 0x7f33b00ccf70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f33a4000c00 tx=0x7f33a4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.618+0000 7f33ae7fc700 1 --2- 172.21.15.40:0/834729771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33b0071aa0 0x7f33b00b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:36.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.618+0000 7f33977fe700 1 -- 172.21.15.40:0/834729771 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a40036c0 con 0x7f33b0069e00 2023-11-08T08:19:36.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.618+0000 7fafdd7fa700 1 -- 172.21.15.40:0/970029501 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fafd4019030 con 0x7fafd8035c00 2023-11-08T08:19:36.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.618+0000 7f33977fe700 1 -- 172.21.15.40:0/834729771 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a401f070 con 0x7f33b0069e00 2023-11-08T08:19:36.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.618+0000 7f33b550d700 1 -- 172.21.15.40:0/834729771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b0072010 con 0x7f33b0069e00 2023-11-08T08:19:36.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.618+0000 7fafdd7fa700 1 --2- 172.21.15.40:0/970029501 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fafc4064ee0 0x7fafc4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.618+0000 7fafdd7fa700 1 -- 172.21.15.40:0/970029501 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fafd4081440 con 0x7fafd8035c00 2023-11-08T08:19:36.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.618+0000 7f33977fe700 1 -- 172.21.15.40:0/834729771 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4003a20 con 0x7f33b0069e00 2023-11-08T08:19:36.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.618+0000 7f33b550d700 1 -- 172.21.15.40:0/834729771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b015ca10 con 0x7f33b0069e00 2023-11-08T08:19:36.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.618+0000 7f33b550d700 1 -- 172.21.15.40:0/834729771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b006c830 con 0x7f33b0069e00 2023-11-08T08:19:36.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.618+0000 7f33977fe700 1 -- 172.21.15.40:0/834729771 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f33a400f050 con 0x7f33b0069e00 2023-11-08T08:19:36.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.622+0000 7fafdffff700 1 --2- 172.21.15.40:0/970029501 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fafc4064ee0 0x7fafc4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.622+0000 7f33977fe700 1 --2- 172.21.15.40:0/834729771 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f339c064f30 0x7f339c0673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.622+0000 7f33977fe700 1 -- 172.21.15.40:0/834729771 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f33a4021070 con 0x7f33b0069e00 2023-11-08T08:19:36.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.622+0000 7f33ae7fc700 1 --2- 172.21.15.40:0/834729771 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f339c064f30 0x7f339c0673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.622+0000 7fafdffff700 1 --2- 172.21.15.40:0/970029501 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fafc4064ee0 0x7fafc4067390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fafd80384c0 tx=0x7fafcc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.622+0000 7f33ae7fc700 1 --2- 172.21.15.40:0/834729771 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f339c064f30 0x7f339c0673e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f33b015a390 tx=0x7f33a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.622+0000 7f54ddffb700 1 -- 172.21.15.40:0/956480771 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f54d804f470 con 0x7f54e00710d0 2023-11-08T08:19:36.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.622+0000 7f49cf7fe700 1 -- 172.21.15.40:0/2705860089 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0011830 con 0x7f49d00cdfb0 2023-11-08T08:19:36.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.622+0000 7f49cf7fe700 1 -- 172.21.15.40:0/2705860089 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c0011990 con 0x7f49d00cdfb0 2023-11-08T08:19:36.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.622+0000 7f49cf7fe700 1 -- 172.21.15.40:0/2705860089 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0011d20 con 0x7f49d00cdfb0 2023-11-08T08:19:36.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.626+0000 7f49d6b5e700 1 -- 172.21.15.40:0/2705860089 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d00cdfb0 msgr2=0x7f49d006cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.626+0000 7f49d6b5e700 1 --2- 172.21.15.40:0/2705860089 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d00cdfb0 0x7f49d006cdb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009880 tx=0x7f49c0009b90 comp rx=0 tx=0).stop 2023-11-08T08:19:36.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.626+0000 7f49d6b5e700 1 -- 172.21.15.40:0/2705860089 shutdown_connections 2023-11-08T08:19:36.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.626+0000 7f49d6b5e700 1 --2- 172.21.15.40:0/2705860089 >> v2:172.21.15.40:3300/0 conn(0x7f49d006d2f0 0x7f49d006a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.626+0000 7f49d6b5e700 1 --2- 172.21.15.40:0/2705860089 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d00cdfb0 0x7f49d006cdb0 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009880 tx=0x7f49c0009b90 comp rx=0 tx=0).stop 2023-11-08T08:19:36.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.626+0000 7f49d6b5e700 1 -- 172.21.15.40:0/2705860089 >> 172.21.15.40:0/2705860089 conn(0x7f49d00cf350 msgr2=0x7f49d00b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:36.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.626+0000 7f49d6b5e700 1 -- 172.21.15.40:0/2705860089 shutdown_connections 2023-11-08T08:19:36.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.626+0000 7fafdd7fa700 1 -- 172.21.15.40:0/970029501 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fafd404f200 con 0x7fafd8035c00 2023-11-08T08:19:36.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.626+0000 7f49d6b5e700 1 -- 172.21.15.40:0/2705860089 wait complete. 2023-11-08T08:19:36.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.630+0000 7f49d6b5e700 1 Processor -- start 2023-11-08T08:19:36.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.630+0000 7f33977fe700 1 -- 172.21.15.40:0/834729771 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f33a4051b20 con 0x7f33b0069e00 2023-11-08T08:19:36.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.630+0000 7f49d6b5e700 1 -- start start 2023-11-08T08:19:36.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.634+0000 7f49d6b5e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d006d2f0 0x7f49d00b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.634+0000 7f49d6b5e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d0163cb0 0x7f49d01f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.634+0000 7f49d6b5e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49d0153890 0x7f49d00c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.634+0000 7f49d6b5e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d00c31c0 con 0x7f49d006d2f0 2023-11-08T08:19:36.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.634+0000 7f49d6b5e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d00c21b0 con 0x7f49d0153890 2023-11-08T08:19:36.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.634+0000 7f49d6b5e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d00b4380 con 0x7f49d0163cb0 2023-11-08T08:19:36.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.634+0000 7f49d48fa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d006d2f0 0x7f49d00b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.634+0000 7f49d48fa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d006d2f0 0x7f49d00b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52540/0 (socket says 172.21.15.40:52540) 2023-11-08T08:19:36.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.634+0000 7f49d48fa700 1 -- 172.21.15.40:0/188523215 learned_addr learned my addr 172.21.15.40:0/188523215 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:36.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.634+0000 7f49d50fb700 1 --2- 172.21.15.40:0/188523215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49d0153890 0x7f49d00c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.634+0000 7f49cffff700 1 --2- 172.21.15.40:0/188523215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d0163cb0 0x7f49d01f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.638+0000 7f49cffff700 1 -- 172.21.15.40:0/188523215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49d0153890 msgr2=0x7f49d00c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.638+0000 7f49cffff700 1 --2- 172.21.15.40:0/188523215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49d0153890 0x7f49d00c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.638+0000 7f49cffff700 1 -- 172.21.15.40:0/188523215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d006d2f0 msgr2=0x7f49d00b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.638+0000 7f49cffff700 1 --2- 172.21.15.40:0/188523215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d006d2f0 0x7f49d00b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.638+0000 7f49cffff700 1 -- 172.21.15.40:0/188523215 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c00094b0 con 0x7f49d0163cb0 2023-11-08T08:19:36.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.638+0000 7f49cffff700 1 --2- 172.21.15.40:0/188523215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d0163cb0 0x7f49d01f5800 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f49d006fbe0 tx=0x7f49c0004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.638+0000 7f49cd7fa700 1 -- 172.21.15.40:0/188523215 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0004550 con 0x7f49d0163cb0 2023-11-08T08:19:36.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.638+0000 7f49d6b5e700 1 -- 172.21.15.40:0/188523215 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d00cc780 con 0x7f49d0163cb0 2023-11-08T08:19:36.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.638+0000 7f49d6b5e700 1 -- 172.21.15.40:0/188523215 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d00cc4a0 con 0x7f49d0163cb0 2023-11-08T08:19:36.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.642+0000 7fb683fff700 1 Processor -- start 2023-11-08T08:19:36.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.642+0000 7f49cd7fa700 1 -- 172.21.15.40:0/188523215 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c001f070 con 0x7f49d0163cb0 2023-11-08T08:19:36.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.642+0000 7fb683fff700 1 -- start start 2023-11-08T08:19:36.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.642+0000 7f49cd7fa700 1 -- 172.21.15.40:0/188523215 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0004550 con 0x7f49d0163cb0 2023-11-08T08:19:36.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.642+0000 7f49d6b5e700 1 -- 172.21.15.40:0/188523215 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d0069e40 con 0x7f49d0163cb0 2023-11-08T08:19:36.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.642+0000 7fb683fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb684163ac0 0x7fb6840ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.642+0000 7fb683fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6840703c0 0x7fb68406b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.642+0000 7fb683fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb68406ad00 con 0x7fb6840c3dc0 2023-11-08T08:19:36.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.642+0000 7fb683fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6840cfda0 con 0x7fb684163ac0 2023-11-08T08:19:36.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.642+0000 7fb683fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6840ce8a0 con 0x7fb6840703c0 2023-11-08T08:19:36.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.646+0000 7fb682ffd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb684163ac0 0x7fb6840ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.646+0000 7fb6837fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6840703c0 0x7fb68406b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.646+0000 7fb67a7fc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb6840c3dc0 0x7fb68406fcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49786/0 (socket says 172.21.15.40:49786) 2023-11-08T08:19:36.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.646+0000 7fb67a7fc700 1 -- 172.21.15.40:0/3157441164 learned_addr learned my addr 172.21.15.40:0/3157441164 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:36.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.646+0000 7f49cd7fa700 1 -- 172.21.15.40:0/188523215 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f49c000f050 con 0x7f49d0163cb0 2023-11-08T08:19:36.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.646+0000 7f49cd7fa700 1 --2- 172.21.15.40:0/188523215 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f49bc064fa0 0x7f49bc067450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.646+0000 7f49cd7fa700 1 -- 172.21.15.40:0/188523215 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f49c0021070 con 0x7f49d0163cb0 2023-11-08T08:19:36.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.646+0000 7fb682ffd700 1 -- 172.21.15.40:0/3157441164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6840703c0 msgr2=0x7fb68406b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:36.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.646+0000 7fb682ffd700 1 --2- 172.21.15.40:0/3157441164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6840703c0 0x7fb68406b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.646+0000 7f49d48fa700 1 --2- 172.21.15.40:0/188523215 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f49bc064fa0 0x7f49bc067450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.646+0000 7fb682ffd700 1 -- 172.21.15.40:0/3157441164 >> v1:172.21.15.40:6789/0 conn(0x7fb6840c3dc0 legacy=0x7fb68406fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.646+0000 7fb682ffd700 1 -- 172.21.15.40:0/3157441164 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb684164ed0 con 0x7fb684163ac0 2023-11-08T08:19:36.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.646+0000 7fb682ffd700 1 --2- 172.21.15.40:0/3157441164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb684163ac0 0x7fb6840ce360 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb674009d90 tx=0x7fb67400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afc7bbb704cda965 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.650+0000 7fb6827fc700 1 -- 172.21.15.40:0/3157441164 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67400e070 con 0x7fb684163ac0 2023-11-08T08:19:36.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.650+0000 7fb6827fc700 1 -- 172.21.15.40:0/3157441164 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67400bb20 con 0x7fb684163ac0 2023-11-08T08:19:36.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.650+0000 7fb683fff700 1 -- 172.21.15.40:0/3157441164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb684163ac0 msgr2=0x7fb6840ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.650+0000 7fb6827fc700 1 -- 172.21.15.40:0/3157441164 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67400e070 con 0x7fb684163ac0 2023-11-08T08:19:36.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.650+0000 7fb683fff700 1 --2- 172.21.15.40:0/3157441164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb684163ac0 0x7fb6840ce360 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb674009d90 tx=0x7fb67400b330 comp rx=0 tx=0).stop 2023-11-08T08:19:36.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.650+0000 7fb683fff700 1 -- 172.21.15.40:0/3157441164 shutdown_connections 2023-11-08T08:19:36.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.650+0000 7fb683fff700 1 --2- 172.21.15.40:0/3157441164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6840703c0 0x7fb68406b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.650+0000 7fb683fff700 1 --2- 172.21.15.40:0/3157441164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb684163ac0 0x7fb6840ce360 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.650+0000 7fb683fff700 1 -- 172.21.15.40:0/3157441164 >> 172.21.15.40:0/3157441164 conn(0x7fb6840cde50 msgr2=0x7fb6840d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:36.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.650+0000 7fb683fff700 1 -- 172.21.15.40:0/3157441164 shutdown_connections 2023-11-08T08:19:36.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.654+0000 7f49d48fa700 1 --2- 172.21.15.40:0/188523215 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f49bc064fa0 0x7f49bc067450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f49c8005cb0 tx=0x7f49c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.650+0000 7fb683fff700 1 -- 172.21.15.40:0/3157441164 wait complete. 2023-11-08T08:19:36.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.654+0000 7f49cd7fa700 1 -- 172.21.15.40:0/188523215 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f49c00514f0 con 0x7f49d0163cb0 2023-11-08T08:19:36.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.654+0000 7fb683fff700 1 Processor -- start 2023-11-08T08:19:36.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.654+0000 7fb683fff700 1 -- start start 2023-11-08T08:19:36.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.658+0000 7fb683fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6840703c0 0x7fb6840c5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.658+0000 7fb683fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb684163ac0 0x7fb684153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.658+0000 7fb683fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6841f58b0 0x7fb6840ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.658+0000 7fb683fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6840c90d0 con 0x7fb6841f58b0 2023-11-08T08:19:36.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.658+0000 7fb683fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6840c7c20 con 0x7fb684163ac0 2023-11-08T08:19:36.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.658+0000 7fb683fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6841f5de0 con 0x7fb6840703c0 2023-11-08T08:19:36.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.658+0000 7fb682ffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6840703c0 0x7fb6840c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.658+0000 7fb682ffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6840703c0 0x7fb6840c5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:53658/0 (socket says 172.21.15.40:53658) 2023-11-08T08:19:36.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.658+0000 7fb682ffd700 1 -- 172.21.15.40:0/4276151423 learned_addr learned my addr 172.21.15.40:0/4276151423 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:36.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.658+0000 7fb6837fe700 1 --2- 172.21.15.40:0/4276151423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6841f58b0 0x7fb6840ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.658+0000 7fb682ffd700 1 -- 172.21.15.40:0/4276151423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb684163ac0 msgr2=0x7fb684153bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-08T08:19:36.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.658+0000 7fb682ffd700 1 --2- 172.21.15.40:0/4276151423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb684163ac0 0x7fb684153bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.658+0000 7fb682ffd700 1 -- 172.21.15.40:0/4276151423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6841f58b0 msgr2=0x7fb6840ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.658+0000 7fb682ffd700 1 --2- 172.21.15.40:0/4276151423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6841f58b0 0x7fb6840ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.658+0000 7fb682ffd700 1 -- 172.21.15.40:0/4276151423 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb674009a70 con 0x7fb6840703c0 2023-11-08T08:19:36.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.658+0000 7fb6837fe700 1 --2- 172.21.15.40:0/4276151423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6841f58b0 0x7fb6840ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:36.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.658+0000 7f8e1b1d9700 1 Processor -- start 2023-11-08T08:19:36.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.658+0000 7f8e1b1d9700 1 -- start start 2023-11-08T08:19:36.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.658+0000 7f8e1b1d9700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8e140895b0 0x7f8e14070580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.662+0000 7f8e1b1d9700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e14070ac0 con 0x7f8e140cde90 2023-11-08T08:19:36.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.662+0000 7f8e18f75700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8e140cde90 0x7f8e1406cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49796/0 (socket says 172.21.15.40:49796) 2023-11-08T08:19:36.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.662+0000 7f8e18f75700 1 -- 172.21.15.40:0/2837091409 learned_addr learned my addr 172.21.15.40:0/2837091409 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:36.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.662+0000 7f8e1b1d9700 1 -- 172.21.15.40:0/2837091409 --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e140ce260 con 0x7f8e14070130 2023-11-08T08:19:36.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.662+0000 7f8e1b1d9700 1 -- 172.21.15.40:0/2837091409 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e140ce500 con 0x7f8e140895b0 2023-11-08T08:19:36.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.662+0000 7f8e19776700 1 --2- 172.21.15.40:0/2837091409 >> v2:172.21.15.169:3300/0 conn(0x7f8e140895b0 0x7f8e14070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.662+0000 7fb682ffd700 1 --2- 172.21.15.40:0/4276151423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6840703c0 0x7fb6840c5610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb674014040 tx=0x7fb674004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.662+0000 7fb67bfff700 1 -- 172.21.15.40:0/4276151423 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb674012840 con 0x7fb6840703c0 2023-11-08T08:19:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.662+0000 7fb683fff700 1 -- 172.21.15.40:0/4276151423 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6840d0320 con 0x7fb6840703c0 2023-11-08T08:19:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.662+0000 7fb683fff700 1 -- 172.21.15.40:0/4276151423 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6840d0040 con 0x7fb6840703c0 2023-11-08T08:19:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.662+0000 7fb67bfff700 1 -- 172.21.15.40:0/4276151423 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb674020070 con 0x7fb6840703c0 2023-11-08T08:19:36.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.662+0000 7fb67bfff700 1 -- 172.21.15.40:0/4276151423 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb674012840 con 0x7fb6840703c0 2023-11-08T08:19:36.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.662+0000 7f8e19776700 1 -- 172.21.15.40:0/2837091409 >> v1:172.21.15.169:6789/0 conn(0x7f8e14070130 legacy=0x7f8e1406bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.662+0000 7f8e19776700 1 -- 172.21.15.40:0/2837091409 >> v1:172.21.15.40:6789/0 conn(0x7f8e140cde90 legacy=0x7f8e1406cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.662+0000 7f8e19776700 1 -- 172.21.15.40:0/2837091409 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1406aa10 con 0x7f8e140895b0 2023-11-08T08:19:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.662+0000 7f8e137fe700 1 -- 172.21.15.40:0/2837091409 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225638050 0 0) 0x7f8e04001110 con 0x7f8e140cde90 2023-11-08T08:19:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.662+0000 7f8e19776700 1 --2- 172.21.15.40:0/2837091409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e140895b0 0x7f8e14070580 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c009d90 tx=0x7f8e0c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c78ba3d2233bd8c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.666+0000 7f8e137fe700 1 -- 172.21.15.40:0/2837091409 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c00e070 con 0x7f8e140895b0 2023-11-08T08:19:36.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.666+0000 7f8e137fe700 1 -- 172.21.15.40:0/2837091409 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c00bbd0 con 0x7f8e140895b0 2023-11-08T08:19:36.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.666+0000 7f8e137fe700 1 -- 172.21.15.40:0/2837091409 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c012430 con 0x7f8e140895b0 2023-11-08T08:19:36.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.666+0000 7fab44d3e700 1 Processor -- start 2023-11-08T08:19:36.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.666+0000 7fb6797fa700 1 -- 172.21.15.40:0/4276151423 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb684163e90 con 0x7fb6840703c0 2023-11-08T08:19:36.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.666+0000 7fab44d3e700 1 -- start start 2023-11-08T08:19:36.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.666+0000 7f8e1b1d9700 1 -- 172.21.15.40:0/2837091409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e140895b0 msgr2=0x7f8e14070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.666+0000 7f8e1b1d9700 1 --2- 172.21.15.40:0/2837091409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e140895b0 0x7f8e14070580 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c009d90 tx=0x7f8e0c00b330 comp rx=0 tx=0).stop 2023-11-08T08:19:36.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.666+0000 7f8e1b1d9700 1 -- 172.21.15.40:0/2837091409 shutdown_connections 2023-11-08T08:19:36.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.666+0000 7f8e1b1d9700 1 --2- 172.21.15.40:0/2837091409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e140895b0 0x7f8e14070580 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.666+0000 7f8e1b1d9700 1 -- 172.21.15.40:0/2837091409 >> 172.21.15.40:0/2837091409 conn(0x7f8e140ce6c0 msgr2=0x7f8e140cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:36.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.666+0000 7f8e1b1d9700 1 -- 172.21.15.40:0/2837091409 shutdown_connections 2023-11-08T08:19:36.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.666+0000 7f8e1b1d9700 1 -- 172.21.15.40:0/2837091409 wait complete. 2023-11-08T08:19:36.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.666+0000 7fab44d3e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fab40161130 0x7fab40163510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.666+0000 7fab44d3e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab4006a450 con 0x7fab4006cb00 2023-11-08T08:19:36.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.666+0000 7fab44d3e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab400ccc20 con 0x7fab40070080 2023-11-08T08:19:36.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.666+0000 7fab44d3e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab40158760 con 0x7fab40161130 2023-11-08T08:19:36.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.670+0000 7fab3ffff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fab40161130 0x7fab40163510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.670+0000 7fab3ffff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fab40161130 0x7fab40163510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33810/0 (socket says 172.21.15.40:33810) 2023-11-08T08:19:36.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.670+0000 7fab3ffff700 1 -- 172.21.15.40:0/2922167481 learned_addr learned my addr 172.21.15.40:0/2922167481 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:36.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.670+0000 7f8e1b1d9700 1 Processor -- start 2023-11-08T08:19:36.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.670+0000 7fab3e7fc700 1 -- 172.21.15.40:0/2922167481 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055372298 0 0) 0x7fab400ccc20 con 0x7fab40070080 2023-11-08T08:19:36.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.670+0000 7fab3e7fc700 1 -- 172.21.15.40:0/2922167481 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab2c0035d0 con 0x7fab40070080 2023-11-08T08:19:36.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.670+0000 7f8e1b1d9700 1 -- start start 2023-11-08T08:19:36.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.670+0000 7fab3ffff700 1 -- 172.21.15.40:0/2922167481 >> v1:172.21.15.169:6789/0 conn(0x7fab40070080 legacy=0x7fab40070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.670+0000 7fab3ffff700 1 -- 172.21.15.40:0/2922167481 >> v1:172.21.15.40:6789/0 conn(0x7fab4006cb00 legacy=0x7fab400cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:36.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.670+0000 7fab3ffff700 1 -- 172.21.15.40:0/2922167481 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab40069e00 con 0x7fab40161130 2023-11-08T08:19:36.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.670+0000 7fab3ffff700 1 --2- 172.21.15.40:0/2922167481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab40161130 0x7fab40163510 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fab30008910 tx=0x7fab3000d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a8ef58ed4fbc6f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7f8e1b1d9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e14070130 0x7f8e140b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.670+0000 7fab3e7fc700 1 -- 172.21.15.40:0/2922167481 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30010070 con 0x7fab40161130 2023-11-08T08:19:36.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7f8e1b1d9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e140895b0 0x7f8e140b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7fab3e7fc700 1 -- 172.21.15.40:0/2922167481 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3000d990 con 0x7fab40161130 2023-11-08T08:19:36.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7fab3e7fc700 1 -- 172.21.15.40:0/2922167481 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3000dd20 con 0x7fab40161130 2023-11-08T08:19:36.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7f8e1b1d9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e140cde90 0x7f8e140c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7f8e1b1d9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e140b14f0 con 0x7f8e14070130 2023-11-08T08:19:36.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7f8e13fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e140895b0 0x7f8e140b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7f8e13fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e140895b0 0x7f8e140b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:53668/0 (socket says 172.21.15.40:53668) 2023-11-08T08:19:36.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7f8e13fff700 1 -- 172.21.15.40:0/3517905865 learned_addr learned my addr 172.21.15.40:0/3517905865 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:36.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7f8e1b1d9700 1 -- 172.21.15.40:0/3517905865 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e140c64c0 con 0x7f8e140cde90 2023-11-08T08:19:36.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7f8e1b1d9700 1 -- 172.21.15.40:0/3517905865 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e140c67a0 con 0x7f8e140895b0 2023-11-08T08:19:36.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.670+0000 7fb67bfff700 1 -- 172.21.15.40:0/4276151423 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fb6740129a0 con 0x7fb6840703c0 2023-11-08T08:19:36.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7fb67bfff700 1 --2- 172.21.15.40:0/4276151423 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb664064fa0 0x7fb664067450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7fb67bfff700 1 -- 172.21.15.40:0/4276151423 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb674022070 con 0x7fb6840703c0 2023-11-08T08:19:36.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7fab44d3e700 1 -- 172.21.15.40:0/2922167481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab40161130 msgr2=0x7fab40163510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7fab44d3e700 1 --2- 172.21.15.40:0/2922167481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab40161130 0x7fab40163510 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fab30008910 tx=0x7fab3000d330 comp rx=0 tx=0).stop 2023-11-08T08:19:36.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7f8e19776700 1 --2- 172.21.15.40:0/3517905865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e140cde90 0x7f8e140c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7fb67a7fc700 1 --2- 172.21.15.40:0/4276151423 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb664064fa0 0x7fb664067450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7f8e18f75700 1 --2- 172.21.15.40:0/3517905865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e14070130 0x7f8e140b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7fab44d3e700 1 -- 172.21.15.40:0/2922167481 shutdown_connections 2023-11-08T08:19:36.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7fab44d3e700 1 --2- 172.21.15.40:0/2922167481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab40161130 0x7fab40163510 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7fab44d3e700 1 -- 172.21.15.40:0/2922167481 >> 172.21.15.40:0/2922167481 conn(0x7fab400b0f90 msgr2=0x7fab400ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:36.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7fb67a7fc700 1 --2- 172.21.15.40:0/4276151423 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb664064fa0 0x7fb664067450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb6840c5540 tx=0x7fb66c007af0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7f8e18f75700 1 -- 172.21.15.40:0/3517905865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e140895b0 msgr2=0x7f8e140b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7f8e18f75700 1 --2- 172.21.15.40:0/3517905865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e140895b0 0x7f8e140b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7f8e18f75700 1 -- 172.21.15.40:0/3517905865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e140cde90 msgr2=0x7f8e140c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7f8e18f75700 1 --2- 172.21.15.40:0/3517905865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e140cde90 0x7f8e140c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7f8e18f75700 1 -- 172.21.15.40:0/3517905865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0c009a70 con 0x7f8e14070130 2023-11-08T08:19:36.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.678+0000 7f8e18f75700 1 --2- 172.21.15.40:0/3517905865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e14070130 0x7f8e140b3b50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8e04000c00 tx=0x7f8e0400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.678+0000 7fb67bfff700 1 -- 172.21.15.40:0/4276151423 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fb674052900 con 0x7fb6840703c0 2023-11-08T08:19:36.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.678+0000 7f8e117fa700 1 -- 172.21.15.40:0/3517905865 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04004ca0 con 0x7f8e14070130 2023-11-08T08:19:36.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.678+0000 7f8e117fa700 1 -- 172.21.15.40:0/3517905865 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e04004e00 con 0x7f8e14070130 2023-11-08T08:19:36.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.678+0000 7f8e117fa700 1 -- 172.21.15.40:0/3517905865 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e040072f0 con 0x7f8e14070130 2023-11-08T08:19:36.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7fab44d3e700 1 -- 172.21.15.40:0/2922167481 shutdown_connections 2023-11-08T08:19:36.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.674+0000 7fab44d3e700 1 -- 172.21.15.40:0/2922167481 wait complete. 2023-11-08T08:19:36.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.678+0000 7f8e1b1d9700 1 -- 172.21.15.40:0/3517905865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1406d6e0 con 0x7f8e14070130 2023-11-08T08:19:36.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.678+0000 7f8e1b1d9700 1 -- 172.21.15.40:0/3517905865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e140cc430 con 0x7f8e14070130 2023-11-08T08:19:36.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.678+0000 7fab44d3e700 1 Processor -- start 2023-11-08T08:19:36.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.682+0000 7f8e1b1d9700 1 -- 172.21.15.40:0/3517905865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e140addf0 con 0x7f8e14070130 2023-11-08T08:19:36.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.682+0000 7f8e117fa700 1 -- 172.21.15.40:0/3517905865 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f8e0401a020 con 0x7f8e14070130 2023-11-08T08:19:36.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.682+0000 7f8e117fa700 1 --2- 172.21.15.40:0/3517905865 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8e00064f30 0x7f8e000673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.682+0000 7f8e117fa700 1 -- 172.21.15.40:0/3517905865 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8e04081a40 con 0x7f8e14070130 2023-11-08T08:19:36.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.682+0000 7f8e13fff700 1 --2- 172.21.15.40:0/3517905865 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8e00064f30 0x7f8e000673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.686+0000 7fab44d3e700 1 -- start start 2023-11-08T08:19:36.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.686+0000 7f8e13fff700 1 --2- 172.21.15.40:0/3517905865 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8e00064f30 0x7f8e000673e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8e140b3aa0 tx=0x7f8e08009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.686+0000 7fab44d3e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab4006cb00 0x7fab401e65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.686+0000 7fab44d3e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab40070080 0x7fab400c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.686+0000 7fab44d3e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab40161130 0x7fab40151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.686+0000 7fab44d3e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab40158950 con 0x7fab4006cb00 2023-11-08T08:19:36.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.686+0000 7fab44d3e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab40158bc0 con 0x7fab40161130 2023-11-08T08:19:36.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.686+0000 7fab44d3e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab40158ea0 con 0x7fab40070080 2023-11-08T08:19:36.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.686+0000 7fab3ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab40161130 0x7fab40151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.686+0000 7fab3ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab40161130 0x7fab40151c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33836/0 (socket says 172.21.15.40:33836) 2023-11-08T08:19:36.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.686+0000 7fab3ffff700 1 -- 172.21.15.40:0/3173214173 learned_addr learned my addr 172.21.15.40:0/3173214173 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:36.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.686+0000 7fab3effd700 1 --2- 172.21.15.40:0/3173214173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab40070080 0x7fab400c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.686+0000 7fab3ffff700 1 -- 172.21.15.40:0/3173214173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab40070080 msgr2=0x7fab400c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.686+0000 7fab3ffff700 1 --2- 172.21.15.40:0/3173214173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab40070080 0x7fab400c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.686+0000 7fab3ffff700 1 -- 172.21.15.40:0/3173214173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab4006cb00 msgr2=0x7fab401e65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.686+0000 7fab3ffff700 1 --2- 172.21.15.40:0/3173214173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab4006cb00 0x7fab401e65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.686+0000 7fab3ffff700 1 -- 172.21.15.40:0/3173214173 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab300085f0 con 0x7fab40161130 2023-11-08T08:19:36.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.690+0000 7fab3ffff700 1 --2- 172.21.15.40:0/3173214173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab40161130 0x7fab40151c40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fab30004000 tx=0x7fab30004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.690+0000 7fab37fff700 1 -- 172.21.15.40:0/3173214173 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab300044c0 con 0x7fab40161130 2023-11-08T08:19:36.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.690+0000 7fab44d3e700 1 -- 172.21.15.40:0/3173214173 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab400c8da0 con 0x7fab40161130 2023-11-08T08:19:36.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.690+0000 7fab37fff700 1 -- 172.21.15.40:0/3173214173 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3000db00 con 0x7fab40161130 2023-11-08T08:19:36.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.690+0000 7fab37fff700 1 -- 172.21.15.40:0/3173214173 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab300036c0 con 0x7fab40161130 2023-11-08T08:19:36.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.690+0000 7fab44d3e700 1 -- 172.21.15.40:0/3173214173 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4006a0e0 con 0x7fab40161130 2023-11-08T08:19:36.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.690+0000 7fab44d3e700 1 -- 172.21.15.40:0/3173214173 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab24005310 con 0x7fab40161130 2023-11-08T08:19:36.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.694+0000 7f8e117fa700 1 -- 172.21.15.40:0/3517905865 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f8e0404f990 con 0x7f8e14070130 2023-11-08T08:19:36.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.694+0000 7fab37fff700 1 -- 172.21.15.40:0/3173214173 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fab30014070 con 0x7fab40161130 2023-11-08T08:19:36.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.694+0000 7fab37fff700 1 --2- 172.21.15.40:0/3173214173 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fab20064ee0 0x7fab20067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:36.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.694+0000 7fab37fff700 1 -- 172.21.15.40:0/3173214173 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fab30082ef0 con 0x7fab40161130 2023-11-08T08:19:36.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.698+0000 7fab3f7fe700 1 --2- 172.21.15.40:0/3173214173 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fab20064ee0 0x7fab20067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:36.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.698+0000 7fab3f7fe700 1 --2- 172.21.15.40:0/3173214173 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fab20064ee0 0x7fab20067390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fab380012e0 tx=0x7fab38001760 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:36.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.706+0000 7fab37fff700 1 -- 172.21.15.40:0/3173214173 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fab30050cb0 con 0x7fab40161130 2023-11-08T08:19:36.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.950+0000 7fafe6f2d700 1 -- 172.21.15.40:0/970029501 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fafd8005840 con 0x7fafd8035c00 2023-11-08T08:19:36.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.950+0000 7fafdd7fa700 1 -- 172.21.15.40:0/970029501 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fafd4052820 con 0x7fafd8035c00 2023-11-08T08:19:36.955 INFO:teuthology.orchestra.run.smithi040.stdout:55834574860 2023-11-08T08:19:36.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.954+0000 7fafe6f2d700 1 -- 172.21.15.40:0/970029501 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fafc4064ee0 msgr2=0x7fafc4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.954+0000 7fafe6f2d700 1 --2- 172.21.15.40:0/970029501 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fafc4064ee0 0x7fafc4067390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fafd80384c0 tx=0x7fafcc009040 comp rx=0 tx=0).stop 2023-11-08T08:19:36.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.954+0000 7fafe6f2d700 1 -- 172.21.15.40:0/970029501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd8035c00 msgr2=0x7fafd8050320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.954+0000 7fafe6f2d700 1 --2- 172.21.15.40:0/970029501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd8035c00 0x7fafd8050320 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fafd400c500 tx=0x7fafd400c810 comp rx=0 tx=0).stop 2023-11-08T08:19:36.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.954+0000 7fafe6f2d700 1 -- 172.21.15.40:0/970029501 shutdown_connections 2023-11-08T08:19:36.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.954+0000 7fafe6f2d700 1 --2- 172.21.15.40:0/970029501 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fafc4064ee0 0x7fafc4067390 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.954+0000 7fafe6f2d700 1 --2- 172.21.15.40:0/970029501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd80e6d60 0x7fafd8051470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.954+0000 7fafe6f2d700 1 --2- 172.21.15.40:0/970029501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafd8057460 0x7fafd804c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.954+0000 7fafe6f2d700 1 --2- 172.21.15.40:0/970029501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd8035c00 0x7fafd8050320 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.954+0000 7fafe6f2d700 1 -- 172.21.15.40:0/970029501 >> 172.21.15.40:0/970029501 conn(0x7fafd8009f40 msgr2=0x7fafd8004e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:36.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.958+0000 7fafe6f2d700 1 -- 172.21.15.40:0/970029501 shutdown_connections 2023-11-08T08:19:36.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.958+0000 7fafe6f2d700 1 -- 172.21.15.40:0/970029501 wait complete. 2023-11-08T08:19:36.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.962+0000 7f33b550d700 1 -- 172.21.15.40:0/834729771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f33b0073cb0 con 0x7f33b0069e00 2023-11-08T08:19:36.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.966+0000 7f33977fe700 1 -- 172.21.15.40:0/834729771 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f33a4055140 con 0x7f33b0069e00 2023-11-08T08:19:36.973 INFO:teuthology.orchestra.run.smithi040.stdout:55834574858 2023-11-08T08:19:36.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.970+0000 7f33b550d700 1 -- 172.21.15.40:0/834729771 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f339c064f30 msgr2=0x7f339c0673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.970+0000 7f33b550d700 1 --2- 172.21.15.40:0/834729771 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f339c064f30 0x7f339c0673e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f33b015a390 tx=0x7f33a0009040 comp rx=0 tx=0).stop 2023-11-08T08:19:36.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.970+0000 7f33b550d700 1 -- 172.21.15.40:0/834729771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0069e00 msgr2=0x7f33b00ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.970+0000 7f33b550d700 1 --2- 172.21.15.40:0/834729771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0069e00 0x7f33b00ccf70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f33a4000c00 tx=0x7f33a4004770 comp rx=0 tx=0).stop 2023-11-08T08:19:36.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.970+0000 7f33b550d700 1 -- 172.21.15.40:0/834729771 shutdown_connections 2023-11-08T08:19:36.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.970+0000 7f33b550d700 1 --2- 172.21.15.40:0/834729771 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f339c064f30 0x7f339c0673e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.970+0000 7f33b550d700 1 --2- 172.21.15.40:0/834729771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33b015c640 0x7f33b00c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.970+0000 7f33b550d700 1 --2- 172.21.15.40:0/834729771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f33b0071aa0 0x7f33b00b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.970+0000 7f33b550d700 1 --2- 172.21.15.40:0/834729771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b0069e00 0x7f33b00ccf70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.970+0000 7f33b550d700 1 -- 172.21.15.40:0/834729771 >> 172.21.15.40:0/834729771 conn(0x7f33b00b6520 msgr2=0x7f33b00b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:36.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.970+0000 7f33b550d700 1 -- 172.21.15.40:0/834729771 shutdown_connections 2023-11-08T08:19:36.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.970+0000 7f33b550d700 1 -- 172.21.15.40:0/834729771 wait complete. 2023-11-08T08:19:36.977 INFO:teuthology.orchestra.run.smithi040.stdout:55834574860 2023-11-08T08:19:36.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.970+0000 7f848f59e700 1 -- 172.21.15.40:0/3623561651 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f84740055c0 con 0x7f84880d6530 2023-11-08T08:19:36.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.970+0000 7f847effd700 1 -- 172.21.15.40:0/3623561651 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8484053680 con 0x7f84880d6530 2023-11-08T08:19:36.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.974+0000 7f54e7630700 1 -- 172.21.15.40:0/956480771 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f54e015d1e0 con 0x7f54e00710d0 2023-11-08T08:19:36.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.974+0000 7f54ddffb700 1 -- 172.21.15.40:0/956480771 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f54e015d1e0 con 0x7f54e00710d0 2023-11-08T08:19:36.979 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2023-11-08T08:19:36.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f848f59e700 1 -- 172.21.15.40:0/3623561651 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8470064fb0 msgr2=0x7f8470067460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f848f59e700 1 --2- 172.21.15.40:0/3623561651 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8470064fb0 0x7f8470067460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8480001a30 tx=0x7f84800079a0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f848f59e700 1 -- 172.21.15.40:0/3623561651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84880d6530 msgr2=0x7f84880d1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f848f59e700 1 --2- 172.21.15.40:0/3623561651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84880d6530 0x7f84880d1540 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8488004320 tx=0x7f8484011eb0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f848f59e700 1 -- 172.21.15.40:0/3623561651 shutdown_connections 2023-11-08T08:19:36.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f848f59e700 1 --2- 172.21.15.40:0/3623561651 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8470064fb0 0x7f8470067460 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8480001a30 tx=0x7f84800079a0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f848f59e700 1 --2- 172.21.15.40:0/3623561651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8488050970 0x7f84880d6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f848f59e700 1 --2- 172.21.15.40:0/3623561651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84880d6530 0x7f84880d1540 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f848f59e700 1 --2- 172.21.15.40:0/3623561651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8478005220 0x7f84880d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f848f59e700 1 -- 172.21.15.40:0/3623561651 >> 172.21.15.40:0/3623561651 conn(0x7f8488055c20 msgr2=0x7f84880d5720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:36.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f848f59e700 1 -- 172.21.15.40:0/3623561651 shutdown_connections 2023-11-08T08:19:36.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f848f59e700 1 -- 172.21.15.40:0/3623561651 wait complete. 2023-11-08T08:19:36.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f54e7630700 1 -- 172.21.15.40:0/956480771 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f54cc064f30 msgr2=0x7f54cc0673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f54e7630700 1 --2- 172.21.15.40:0/956480771 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f54cc064f30 0x7f54cc0673e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f54e00b3f50 tx=0x7f54d40042e0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f54e7630700 1 -- 172.21.15.40:0/956480771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54e00710d0 msgr2=0x7f54e0151510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:36.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f54e7630700 1 --2- 172.21.15.40:0/956480771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54e00710d0 0x7f54e0151510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f54d800c730 tx=0x7f54d800ca40 comp rx=0 tx=0).stop 2023-11-08T08:19:36.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f54e7630700 1 -- 172.21.15.40:0/956480771 shutdown_connections 2023-11-08T08:19:36.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f54e7630700 1 --2- 172.21.15.40:0/956480771 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f54cc064f30 0x7f54cc0673e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f54e7630700 1 --2- 172.21.15.40:0/956480771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54e01587b0 0x7f54e015b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f54e7630700 1 --2- 172.21.15.40:0/956480771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54e00ccc60 0x7f54e015b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f54e7630700 1 --2- 172.21.15.40:0/956480771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54e00710d0 0x7f54e0151510 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:36.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f54e7630700 1 -- 172.21.15.40:0/956480771 >> 172.21.15.40:0/956480771 conn(0x7f54e00ce6c0 msgr2=0x7f54e01578d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:36.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f54e7630700 1 -- 172.21.15.40:0/956480771 shutdown_connections 2023-11-08T08:19:36.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.978+0000 7f54e7630700 1 -- 172.21.15.40:0/956480771 wait complete. 2023-11-08T08:19:36.990 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.3 2023-11-08T08:19:36.990 DEBUG:teuthology.parallel:result is None 2023-11-08T08:19:36.994 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2023-11-08T08:19:36.995 DEBUG:teuthology.parallel:result is None 2023-11-08T08:19:36.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.994+0000 7f49d6b5e700 1 -- 172.21.15.40:0/188523215 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f49d00726c0 con 0x7f49d0163cb0 2023-11-08T08:19:36.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.994+0000 7f49cd7fa700 1 -- 172.21.15.40:0/188523215 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f49c0015020 con 0x7f49d0163cb0 2023-11-08T08:19:36.999 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2023-11-08T08:19:37.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.998+0000 7f49d6b5e700 1 -- 172.21.15.40:0/188523215 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f49bc064fa0 msgr2=0x7f49bc067450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:37.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.998+0000 7f49d6b5e700 1 --2- 172.21.15.40:0/188523215 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f49bc064fa0 0x7f49bc067450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f49c8005cb0 tx=0x7f49c8009040 comp rx=0 tx=0).stop 2023-11-08T08:19:37.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.998+0000 7f49d6b5e700 1 -- 172.21.15.40:0/188523215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d0163cb0 msgr2=0x7f49d01f5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:37.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.998+0000 7f49d6b5e700 1 --2- 172.21.15.40:0/188523215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d0163cb0 0x7f49d01f5800 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f49d006fbe0 tx=0x7f49c0004770 comp rx=0 tx=0).stop 2023-11-08T08:19:37.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.998+0000 7f49d6b5e700 1 -- 172.21.15.40:0/188523215 shutdown_connections 2023-11-08T08:19:37.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.998+0000 7f49d6b5e700 1 --2- 172.21.15.40:0/188523215 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f49bc064fa0 0x7f49bc067450 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.998+0000 7f49d6b5e700 1 --2- 172.21.15.40:0/188523215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49d0153890 0x7f49d00c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.998+0000 7f49d6b5e700 1 --2- 172.21.15.40:0/188523215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d0163cb0 0x7f49d01f5800 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.998+0000 7f49d6b5e700 1 --2- 172.21.15.40:0/188523215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49d006d2f0 0x7f49d00b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.998+0000 7f49d6b5e700 1 -- 172.21.15.40:0/188523215 >> 172.21.15.40:0/188523215 conn(0x7f49d00cf350 msgr2=0x7f49d00af870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:37.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.998+0000 7f49d6b5e700 1 -- 172.21.15.40:0/188523215 shutdown_connections 2023-11-08T08:19:37.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:36.998+0000 7f49d6b5e700 1 -- 172.21.15.40:0/188523215 wait complete. 2023-11-08T08:19:37.010 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.6 2023-11-08T08:19:37.011 DEBUG:teuthology.parallel:result is None 2023-11-08T08:19:37.011 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.0 2023-11-08T08:19:37.011 DEBUG:teuthology.parallel:result is None 2023-11-08T08:19:37.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.014+0000 7fb6797fa700 1 -- 172.21.15.40:0/4276151423 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb684073cb0 con 0x7fb6840703c0 2023-11-08T08:19:37.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.014+0000 7f8e1b1d9700 1 -- 172.21.15.40:0/3517905865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f8e14070ac0 con 0x7f8e14070130 2023-11-08T08:19:37.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.014+0000 7fb67bfff700 1 -- 172.21.15.40:0/4276151423 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb67402c020 con 0x7fb6840703c0 2023-11-08T08:19:37.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.014+0000 7f8e117fa700 1 -- 172.21.15.40:0/3517905865 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8e14070ac0 con 0x7f8e14070130 2023-11-08T08:19:37.019 INFO:teuthology.orchestra.run.smithi040.stdout:55834574860 2023-11-08T08:19:37.020 INFO:teuthology.orchestra.run.smithi040.stdout:55834574859 2023-11-08T08:19:37.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7f8e1b1d9700 1 -- 172.21.15.40:0/3517905865 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8e00064f30 msgr2=0x7f8e000673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:37.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7f8e1b1d9700 1 --2- 172.21.15.40:0/3517905865 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8e00064f30 0x7f8e000673e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8e140b3aa0 tx=0x7f8e08009040 comp rx=0 tx=0).stop 2023-11-08T08:19:37.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7f8e1b1d9700 1 -- 172.21.15.40:0/3517905865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e14070130 msgr2=0x7f8e140b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:37.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7f8e1b1d9700 1 --2- 172.21.15.40:0/3517905865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e14070130 0x7f8e140b3b50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8e04000c00 tx=0x7f8e0400ca30 comp rx=0 tx=0).stop 2023-11-08T08:19:37.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7fb6797fa700 1 -- 172.21.15.40:0/4276151423 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb664064fa0 msgr2=0x7fb664067450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:37.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7fb6797fa700 1 --2- 172.21.15.40:0/4276151423 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb664064fa0 0x7fb664067450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb6840c5540 tx=0x7fb66c007af0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7fb6797fa700 1 -- 172.21.15.40:0/4276151423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6840703c0 msgr2=0x7fb6840c5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:37.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7fb6797fa700 1 --2- 172.21.15.40:0/4276151423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6840703c0 0x7fb6840c5610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb674014040 tx=0x7fb674004af0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7f8e1b1d9700 1 -- 172.21.15.40:0/3517905865 shutdown_connections 2023-11-08T08:19:37.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7f8e1b1d9700 1 --2- 172.21.15.40:0/3517905865 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8e00064f30 0x7f8e000673e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7f8e1b1d9700 1 --2- 172.21.15.40:0/3517905865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e140cde90 0x7f8e140c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7fb6797fa700 1 -- 172.21.15.40:0/4276151423 shutdown_connections 2023-11-08T08:19:37.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7fb6797fa700 1 --2- 172.21.15.40:0/4276151423 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb664064fa0 0x7fb664067450 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7fb6797fa700 1 --2- 172.21.15.40:0/4276151423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6841f58b0 0x7fb6840ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7fb6797fa700 1 --2- 172.21.15.40:0/4276151423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb684163ac0 0x7fb684153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7f8e1b1d9700 1 --2- 172.21.15.40:0/3517905865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e140895b0 0x7f8e140b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7f8e1b1d9700 1 --2- 172.21.15.40:0/3517905865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e14070130 0x7f8e140b3b50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7f8e1b1d9700 1 -- 172.21.15.40:0/3517905865 >> 172.21.15.40:0/3517905865 conn(0x7f8e140ce6c0 msgr2=0x7f8e1416c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:37.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7fb6797fa700 1 --2- 172.21.15.40:0/4276151423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6840703c0 0x7fb6840c5610 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7fb6797fa700 1 -- 172.21.15.40:0/4276151423 >> 172.21.15.40:0/4276151423 conn(0x7fb6840cde50 msgr2=0x7fb684162ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:37.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7fb6797fa700 1 -- 172.21.15.40:0/4276151423 shutdown_connections 2023-11-08T08:19:37.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7fb6797fa700 1 -- 172.21.15.40:0/4276151423 wait complete. 2023-11-08T08:19:37.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.018+0000 7f8e1b1d9700 1 -- 172.21.15.40:0/3517905865 shutdown_connections 2023-11-08T08:19:37.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.022+0000 7f8e1b1d9700 1 -- 172.21.15.40:0/3517905865 wait complete. 2023-11-08T08:19:37.028 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.4 2023-11-08T08:19:37.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.022+0000 7fab44d3e700 1 -- 172.21.15.40:0/3173214173 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fab240055c0 con 0x7fab40161130 2023-11-08T08:19:37.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.026+0000 7fab37fff700 1 -- 172.21.15.40:0/3173214173 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fab3001b090 con 0x7fab40161130 2023-11-08T08:19:37.029 DEBUG:teuthology.parallel:result is None 2023-11-08T08:19:37.029 INFO:teuthology.orchestra.run.smithi040.stdout:55834574858 2023-11-08T08:19:37.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.026+0000 7fab44d3e700 1 -- 172.21.15.40:0/3173214173 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fab20064ee0 msgr2=0x7fab20067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:37.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.026+0000 7fab44d3e700 1 --2- 172.21.15.40:0/3173214173 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fab20064ee0 0x7fab20067390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fab380012e0 tx=0x7fab38001760 comp rx=0 tx=0).stop 2023-11-08T08:19:37.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.026+0000 7fab44d3e700 1 -- 172.21.15.40:0/3173214173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab40161130 msgr2=0x7fab40151c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:37.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.026+0000 7fab44d3e700 1 --2- 172.21.15.40:0/3173214173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab40161130 0x7fab40151c40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fab30004000 tx=0x7fab30004770 comp rx=0 tx=0).stop 2023-11-08T08:19:37.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.026+0000 7fab44d3e700 1 -- 172.21.15.40:0/3173214173 shutdown_connections 2023-11-08T08:19:37.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.026+0000 7fab44d3e700 1 --2- 172.21.15.40:0/3173214173 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fab20064ee0 0x7fab20067390 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.026+0000 7fab44d3e700 1 --2- 172.21.15.40:0/3173214173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fab40161130 0x7fab40151c40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.026+0000 7fab44d3e700 1 --2- 172.21.15.40:0/3173214173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fab40070080 0x7fab400c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.026+0000 7fab44d3e700 1 --2- 172.21.15.40:0/3173214173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab4006cb00 0x7fab401e65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.026+0000 7fab44d3e700 1 -- 172.21.15.40:0/3173214173 >> 172.21.15.40:0/3173214173 conn(0x7fab400b0f90 msgr2=0x7fab40070450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:37.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.026+0000 7fab44d3e700 1 -- 172.21.15.40:0/3173214173 shutdown_connections 2023-11-08T08:19:37.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.026+0000 7fab44d3e700 1 -- 172.21.15.40:0/3173214173 wait complete. 2023-11-08T08:19:37.040 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.7 2023-11-08T08:19:37.040 DEBUG:teuthology.parallel:result is None 2023-11-08T08:19:37.044 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2023-11-08T08:19:37.044 DEBUG:teuthology.parallel:result is None 2023-11-08T08:19:37.051 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.1 2023-11-08T08:19:37.051 DEBUG:teuthology.parallel:result is None 2023-11-08T08:19:37.051 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-08T08:19:37.052 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T08:19:37.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.262+0000 7fed590cb700 1 Processor -- start 2023-11-08T08:19:37.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.262+0000 7fed590cb700 1 -- start start 2023-11-08T08:19:37.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.262+0000 7fed590cb700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed5415c1a0 0x7fed5406c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:37.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.262+0000 7fed590cb700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed540cdeb0 0x7fed54070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:37.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.262+0000 7fed590cb700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed540adf90 con 0x7fed540705c0 2023-11-08T08:19:37.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.262+0000 7fed590cb700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed54069bf0 con 0x7fed540cdeb0 2023-11-08T08:19:37.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.262+0000 7fed590cb700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed54070a40 con 0x7fed5415c1a0 2023-11-08T08:19:37.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.262+0000 7fed537fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed540cdeb0 0x7fed54070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:37.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.262+0000 7fed537fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed540cdeb0 0x7fed54070080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52574/0 (socket says 172.21.15.40:52574) 2023-11-08T08:19:37.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.262+0000 7fed537fe700 1 -- 172.21.15.40:0/1397090825 learned_addr learned my addr 172.21.15.40:0/1397090825 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:37.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.266+0000 7fed53fff700 1 --2- 172.21.15.40:0/1397090825 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed5415c1a0 0x7fed5406c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:37.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.266+0000 7fed537fe700 1 -- 172.21.15.40:0/1397090825 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed5415c1a0 msgr2=0x7fed5406c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.266+0000 7fed537fe700 1 --2- 172.21.15.40:0/1397090825 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed5415c1a0 0x7fed5406c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.266+0000 7fed537fe700 1 -- 172.21.15.40:0/1397090825 >> v1:172.21.15.169:6789/0 conn(0x7fed540705c0 legacy=0x7fed5415fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.266+0000 7fed537fe700 1 -- 172.21.15.40:0/1397090825 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed540ce370 con 0x7fed540cdeb0 2023-11-08T08:19:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.266+0000 7fed537fe700 1 --2- 172.21.15.40:0/1397090825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed540cdeb0 0x7fed54070080 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fed48009910 tx=0x7fed48009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb20fd43f92bf66 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.266+0000 7fed52ffd700 1 -- 172.21.15.40:0/1397090825 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed48011830 con 0x7fed540cdeb0 2023-11-08T08:19:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.266+0000 7fed52ffd700 1 -- 172.21.15.40:0/1397090825 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed48011990 con 0x7fed540cdeb0 2023-11-08T08:19:37.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.266+0000 7fed52ffd700 1 -- 172.21.15.40:0/1397090825 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed48011d40 con 0x7fed540cdeb0 2023-11-08T08:19:37.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.266+0000 7fed590cb700 1 -- 172.21.15.40:0/1397090825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed540cdeb0 msgr2=0x7fed54070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:37.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.266+0000 7fed590cb700 1 --2- 172.21.15.40:0/1397090825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed540cdeb0 0x7fed54070080 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fed48009910 tx=0x7fed48009c20 comp rx=0 tx=0).stop 2023-11-08T08:19:37.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.266+0000 7fed590cb700 1 -- 172.21.15.40:0/1397090825 shutdown_connections 2023-11-08T08:19:37.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.266+0000 7fed590cb700 1 --2- 172.21.15.40:0/1397090825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed540cdeb0 0x7fed54070080 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.266+0000 7fed590cb700 1 --2- 172.21.15.40:0/1397090825 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed5415c1a0 0x7fed5406c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.266+0000 7fed590cb700 1 -- 172.21.15.40:0/1397090825 >> 172.21.15.40:0/1397090825 conn(0x7fed540b0f80 msgr2=0x7fed540ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:37.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.266+0000 7fed590cb700 1 -- 172.21.15.40:0/1397090825 shutdown_connections 2023-11-08T08:19:37.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.266+0000 7fed590cb700 1 -- 172.21.15.40:0/1397090825 wait complete. 2023-11-08T08:19:37.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.270+0000 7fed590cb700 1 Processor -- start 2023-11-08T08:19:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.270+0000 7fed590cb700 1 -- start start 2023-11-08T08:19:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed590cb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed540705c0 0x7fed54155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed590cb700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed540cdeb0 0x7fed5415f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed590cb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed5415c1a0 0x7fed540b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed590cb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed541563a0 con 0x7fed540705c0 2023-11-08T08:19:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed590cb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed541f6000 con 0x7fed5415c1a0 2023-11-08T08:19:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed590cb700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed541f62e0 con 0x7fed540cdeb0 2023-11-08T08:19:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed53fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed540705c0 0x7fed54155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:37.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed53fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed540705c0 0x7fed54155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52584/0 (socket says 172.21.15.40:52584) 2023-11-08T08:19:37.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed53fff700 1 -- 172.21.15.40:0/3062779518 learned_addr learned my addr 172.21.15.40:0/3062779518 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:37.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed537fe700 1 --2- 172.21.15.40:0/3062779518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed540cdeb0 0x7fed5415f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:37.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed588ca700 1 --2- 172.21.15.40:0/3062779518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed5415c1a0 0x7fed540b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:37.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed53fff700 1 -- 172.21.15.40:0/3062779518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed540cdeb0 msgr2=0x7fed5415f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:37.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed53fff700 1 --2- 172.21.15.40:0/3062779518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed540cdeb0 0x7fed5415f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed53fff700 1 -- 172.21.15.40:0/3062779518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed5415c1a0 msgr2=0x7fed540b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:37.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed53fff700 1 --2- 172.21.15.40:0/3062779518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed5415c1a0 0x7fed540b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed53fff700 1 -- 172.21.15.40:0/3062779518 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed48009570 con 0x7fed540705c0 2023-11-08T08:19:37.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed588ca700 1 --2- 172.21.15.40:0/3062779518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed5415c1a0 0x7fed540b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:37.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed53fff700 1 --2- 172.21.15.40:0/3062779518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed540705c0 0x7fed54155e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed4400e9d0 tx=0x7fed4400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:37.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed50ff9700 1 -- 172.21.15.40:0/3062779518 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44009720 con 0x7fed540705c0 2023-11-08T08:19:37.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed50ff9700 1 -- 172.21.15.40:0/3062779518 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4400c850 con 0x7fed540705c0 2023-11-08T08:19:37.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed590cb700 1 -- 172.21.15.40:0/3062779518 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed540cc780 con 0x7fed540705c0 2023-11-08T08:19:37.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed50ff9700 1 -- 172.21.15.40:0/3062779518 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4400cbc0 con 0x7fed540705c0 2023-11-08T08:19:37.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.274+0000 7fed590cb700 1 -- 172.21.15.40:0/3062779518 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed540cc4a0 con 0x7fed540705c0 2023-11-08T08:19:37.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.278+0000 7fed590cb700 1 -- 172.21.15.40:0/3062779518 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed5406cdf0 con 0x7fed540705c0 2023-11-08T08:19:37.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.278+0000 7fed50ff9700 1 -- 172.21.15.40:0/3062779518 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fed44019030 con 0x7fed540705c0 2023-11-08T08:19:37.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.282+0000 7fed50ff9700 1 --2- 172.21.15.40:0/3062779518 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fed40064ee0 0x7fed40067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:37.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.282+0000 7fed50ff9700 1 -- 172.21.15.40:0/3062779518 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fed440821d0 con 0x7fed540705c0 2023-11-08T08:19:37.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.282+0000 7fed537fe700 1 --2- 172.21.15.40:0/3062779518 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fed40064ee0 0x7fed40067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:37.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.286+0000 7fed537fe700 1 --2- 172.21.15.40:0/3062779518 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fed40064ee0 0x7fed40067390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fed540ce2a0 tx=0x7fed4801d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:37.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.290+0000 7fed50ff9700 1 -- 172.21.15.40:0/3062779518 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fed4404ff90 con 0x7fed540705c0 2023-11-08T08:19:37.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.446+0000 7fed590cb700 1 -- 172.21.15.40:0/3062779518 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fed541f6ef0 con 0x7fed40064ee0 2023-11-08T08:19:37.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.466+0000 7fed50ff9700 1 -- 172.21.15.40:0/3062779518 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367118 (secure 0 0 0) 0x7fed48011630 con 0x7fed40064ee0 2023-11-08T08:19:37.470 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:37.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.474+0000 7fed590cb700 1 -- 172.21.15.40:0/3062779518 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fed40064ee0 msgr2=0x7fed40067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:37.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.474+0000 7fed590cb700 1 --2- 172.21.15.40:0/3062779518 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fed40064ee0 0x7fed40067390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fed540ce2a0 tx=0x7fed4801d040 comp rx=0 tx=0).stop 2023-11-08T08:19:37.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.474+0000 7fed590cb700 1 -- 172.21.15.40:0/3062779518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed540705c0 msgr2=0x7fed54155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:37.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.474+0000 7fed590cb700 1 --2- 172.21.15.40:0/3062779518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed540705c0 0x7fed54155e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed4400e9d0 tx=0x7fed4400ece0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.474+0000 7fed590cb700 1 -- 172.21.15.40:0/3062779518 shutdown_connections 2023-11-08T08:19:37.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.474+0000 7fed590cb700 1 --2- 172.21.15.40:0/3062779518 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fed40064ee0 0x7fed40067390 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.474+0000 7fed590cb700 1 --2- 172.21.15.40:0/3062779518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed5415c1a0 0x7fed540b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.474+0000 7fed590cb700 1 --2- 172.21.15.40:0/3062779518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed540cdeb0 0x7fed5415f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.474+0000 7fed590cb700 1 --2- 172.21.15.40:0/3062779518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed540705c0 0x7fed54155e60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.474+0000 7fed590cb700 1 -- 172.21.15.40:0/3062779518 >> 172.21.15.40:0/3062779518 conn(0x7fed540b0f80 msgr2=0x7fed540ad610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:37.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.474+0000 7fed590cb700 1 -- 172.21.15.40:0/3062779518 shutdown_connections 2023-11-08T08:19:37.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.474+0000 7fed590cb700 1 -- 172.21.15.40:0/3062779518 wait complete. 2023-11-08T08:19:37.480 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-11-08T08:19:37.500 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2023-11-08T08:19:35.891534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594895,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67784,"kb_used_data":3592,"kb_used_omap":0,"kb_used_meta":64128,"kb_avail":754906936,"statfs":{"total":773094113280,"available":773024702464,"internally_reserved":0,"allocated":3678208,"data_stored":2186366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65667072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004821"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.370829+0000","last_change":"2023-11-08T08:19:08.519864+0000","last_active":"2023-11-08T08:19:09.370829+0000","last_peered":"2023-11-08T08:19:09.370829+0000","last_clean":"2023-11-08T08:19:09.370829+0000","last_became_active":"2023-11-08T08:19:08.518845+0000","last_became_peered":"2023-11-08T08:19:08.518845+0000","last_unstale":"2023-11-08T08:19:09.370829+0000","last_undegraded":"2023-11-08T08:19:09.370829+0000","last_fullsized":"2023-11-08T08:19:09.370829+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:47:58.348898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.496570+0000","last_change":"2023-11-08T08:19:04.015635+0000","last_active":"2023-11-08T08:19:08.496570+0000","last_peered":"2023-11-08T08:19:08.496570+0000","last_clean":"2023-11-08T08:19:08.496570+0000","last_became_active":"2023-11-08T08:19:04.014748+0000","last_became_peered":"2023-11-08T08:19:04.014748+0000","last_unstale":"2023-11-08T08:19:08.496570+0000","last_undegraded":"2023-11-08T08:19:08.496570+0000","last_fullsized":"2023-11-08T08:19:08.496570+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:01:41.280977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513402+0000","last_change":"2023-11-08T08:19:04.505994+0000","last_active":"2023-11-08T08:19:08.513402+0000","last_peered":"2023-11-08T08:19:08.513402+0000","last_clean":"2023-11-08T08:19:08.513402+0000","last_became_active":"2023-11-08T08:19:04.505596+0000","last_became_peered":"2023-11-08T08:19:04.505596+0000","last_unstale":"2023-11-08T08:19:08.513402+0000","last_undegraded":"2023-11-08T08:19:08.513402+0000","last_fullsized":"2023-11-08T08:19:08.513402+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:06:47.395577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.992842+0000","last_change":"2023-11-08T08:19:08.526866+0000","last_active":"2023-11-08T08:19:08.992842+0000","last_peered":"2023-11-08T08:19:08.992842+0000","last_clean":"2023-11-08T08:19:08.992842+0000","last_became_active":"2023-11-08T08:19:08.526405+0000","last_became_peered":"2023-11-08T08:19:08.526405+0000","last_unstale":"2023-11-08T08:19:08.992842+0000","last_undegraded":"2023-11-08T08:19:08.992842+0000","last_fullsized":"2023-11-08T08:19:08.992842+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:54:12.699998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519574+0000","last_change":"2023-11-08T08:19:04.021634+0000","last_active":"2023-11-08T08:19:08.519574+0000","last_peered":"2023-11-08T08:19:08.519574+0000","last_clean":"2023-11-08T08:19:08.519574+0000","last_became_active":"2023-11-08T08:19:04.017916+0000","last_became_peered":"2023-11-08T08:19:04.017916+0000","last_unstale":"2023-11-08T08:19:08.519574+0000","last_undegraded":"2023-11-08T08:19:08.519574+0000","last_fullsized":"2023-11-08T08:19:08.519574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:58:24.706350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518078+0000","last_change":"2023-11-08T08:19:04.481276+0000","last_active":"2023-11-08T08:19:08.518078+0000","last_peered":"2023-11-08T08:19:08.518078+0000","last_clean":"2023-11-08T08:19:08.518078+0000","last_became_active":"2023-11-08T08:19:04.480340+0000","last_became_peered":"2023-11-08T08:19:04.480340+0000","last_unstale":"2023-11-08T08:19:08.518078+0000","last_undegraded":"2023-11-08T08:19:08.518078+0000","last_fullsized":"2023-11-08T08:19:08.518078+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:04:55.302575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372099+0000","last_change":"2023-11-08T08:19:08.528726+0000","last_active":"2023-11-08T08:19:09.372099+0000","last_peered":"2023-11-08T08:19:09.372099+0000","last_clean":"2023-11-08T08:19:09.372099+0000","last_became_active":"2023-11-08T08:19:08.527853+0000","last_became_peered":"2023-11-08T08:19:08.527853+0000","last_unstale":"2023-11-08T08:19:09.372099+0000","last_undegraded":"2023-11-08T08:19:09.372099+0000","last_fullsized":"2023-11-08T08:19:09.372099+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:16:14.032597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519819+0000","last_change":"2023-11-08T08:19:04.025513+0000","last_active":"2023-11-08T08:19:08.519819+0000","last_peered":"2023-11-08T08:19:08.519819+0000","last_clean":"2023-11-08T08:19:08.519819+0000","last_became_active":"2023-11-08T08:19:04.025220+0000","last_became_peered":"2023-11-08T08:19:04.025220+0000","last_unstale":"2023-11-08T08:19:08.519819+0000","last_undegraded":"2023-11-08T08:19:08.519819+0000","last_fullsized":"2023-11-08T08:19:08.519819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:55:07.041721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512027+0000","last_change":"2023-11-08T08:19:04.512364+0000","last_active":"2023-11-08T08:19:08.512027+0000","last_peered":"2023-11-08T08:19:08.512027+0000","last_clean":"2023-11-08T08:19:08.512027+0000","last_became_active":"2023-11-08T08:19:04.511958+0000","last_became_peered":"2023-11-08T08:19:04.511958+0000","last_unstale":"2023-11-08T08:19:08.512027+0000","last_undegraded":"2023-11-08T08:19:08.512027+0000","last_fullsized":"2023-11-08T08:19:08.512027+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:32:57.245760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.994907+0000","last_change":"2023-11-08T08:19:08.527583+0000","last_active":"2023-11-08T08:19:08.994907+0000","last_peered":"2023-11-08T08:19:08.994907+0000","last_clean":"2023-11-08T08:19:08.994907+0000","last_became_active":"2023-11-08T08:19:08.527054+0000","last_became_peered":"2023-11-08T08:19:08.527054+0000","last_unstale":"2023-11-08T08:19:08.994907+0000","last_undegraded":"2023-11-08T08:19:08.994907+0000","last_fullsized":"2023-11-08T08:19:08.994907+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:08.797102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519426+0000","last_change":"2023-11-08T08:19:04.025416+0000","last_active":"2023-11-08T08:19:08.519426+0000","last_peered":"2023-11-08T08:19:08.519426+0000","last_clean":"2023-11-08T08:19:08.519426+0000","last_became_active":"2023-11-08T08:19:04.025091+0000","last_became_peered":"2023-11-08T08:19:04.025091+0000","last_unstale":"2023-11-08T08:19:08.519426+0000","last_undegraded":"2023-11-08T08:19:08.519426+0000","last_fullsized":"2023-11-08T08:19:08.519426+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:07:35.132845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.511963+0000","last_change":"2023-11-08T08:19:04.512794+0000","last_active":"2023-11-08T08:19:08.511963+0000","last_peered":"2023-11-08T08:19:08.511963+0000","last_clean":"2023-11-08T08:19:08.511963+0000","last_became_active":"2023-11-08T08:19:04.512499+0000","last_became_peered":"2023-11-08T08:19:04.512499+0000","last_unstale":"2023-11-08T08:19:08.511963+0000","last_undegraded":"2023-11-08T08:19:08.511963+0000","last_fullsized":"2023-11-08T08:19:08.511963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:05:16.516877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372287+0000","last_change":"2023-11-08T08:19:08.539289+0000","last_active":"2023-11-08T08:19:09.372287+0000","last_peered":"2023-11-08T08:19:09.372287+0000","last_clean":"2023-11-08T08:19:09.372287+0000","last_became_active":"2023-11-08T08:19:08.532011+0000","last_became_peered":"2023-11-08T08:19:08.532011+0000","last_unstale":"2023-11-08T08:19:09.372287+0000","last_undegraded":"2023-11-08T08:19:09.372287+0000","last_fullsized":"2023-11-08T08:19:09.372287+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:56:38.412909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.495705+0000","last_change":"2023-11-08T08:19:04.024246+0000","last_active":"2023-11-08T08:19:08.495705+0000","last_peered":"2023-11-08T08:19:08.495705+0000","last_clean":"2023-11-08T08:19:08.495705+0000","last_became_active":"2023-11-08T08:19:04.023681+0000","last_became_peered":"2023-11-08T08:19:04.023681+0000","last_unstale":"2023-11-08T08:19:08.495705+0000","last_undegraded":"2023-11-08T08:19:08.495705+0000","last_fullsized":"2023-11-08T08:19:08.495705+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:50:41.342112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.509382+0000","last_change":"2023-11-08T08:19:04.494535+0000","last_active":"2023-11-08T08:19:08.509382+0000","last_peered":"2023-11-08T08:19:08.509382+0000","last_clean":"2023-11-08T08:19:08.509382+0000","last_became_active":"2023-11-08T08:19:04.494226+0000","last_became_peered":"2023-11-08T08:19:04.494226+0000","last_unstale":"2023-11-08T08:19:08.509382+0000","last_undegraded":"2023-11-08T08:19:08.509382+0000","last_fullsized":"2023-11-08T08:19:08.509382+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:18:15.282329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.993497+0000","last_change":"2023-11-08T08:19:08.532723+0000","last_active":"2023-11-08T08:19:08.993497+0000","last_peered":"2023-11-08T08:19:08.993497+0000","last_clean":"2023-11-08T08:19:08.993497+0000","last_became_active":"2023-11-08T08:19:08.532320+0000","last_became_peered":"2023-11-08T08:19:08.532320+0000","last_unstale":"2023-11-08T08:19:08.993497+0000","last_undegraded":"2023-11-08T08:19:08.993497+0000","last_fullsized":"2023-11-08T08:19:08.993497+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:10:25.517705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512234+0000","last_change":"2023-11-08T08:19:04.020285+0000","last_active":"2023-11-08T08:19:08.512234+0000","last_peered":"2023-11-08T08:19:08.512234+0000","last_clean":"2023-11-08T08:19:08.512234+0000","last_became_active":"2023-11-08T08:19:04.019791+0000","last_became_peered":"2023-11-08T08:19:04.019791+0000","last_unstale":"2023-11-08T08:19:08.512234+0000","last_undegraded":"2023-11-08T08:19:08.512234+0000","last_fullsized":"2023-11-08T08:19:08.512234+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:21:09.218618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521337+0000","last_change":"2023-11-08T08:19:04.497059+0000","last_active":"2023-11-08T08:19:08.521337+0000","last_peered":"2023-11-08T08:19:08.521337+0000","last_clean":"2023-11-08T08:19:08.521337+0000","last_became_active":"2023-11-08T08:19:04.495199+0000","last_became_peered":"2023-11-08T08:19:04.495199+0000","last_unstale":"2023-11-08T08:19:08.521337+0000","last_undegraded":"2023-11-08T08:19:08.521337+0000","last_fullsized":"2023-11-08T08:19:08.521337+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:56:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.374791+0000","last_change":"2023-11-08T08:19:08.530445+0000","last_active":"2023-11-08T08:19:09.374791+0000","last_peered":"2023-11-08T08:19:09.374791+0000","last_clean":"2023-11-08T08:19:09.374791+0000","last_became_active":"2023-11-08T08:19:08.530020+0000","last_became_peered":"2023-11-08T08:19:08.530020+0000","last_unstale":"2023-11-08T08:19:09.374791+0000","last_undegraded":"2023-11-08T08:19:09.374791+0000","last_fullsized":"2023-11-08T08:19:09.374791+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:24:13.984326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.502373+0000","last_change":"2023-11-08T08:19:04.018882+0000","last_active":"2023-11-08T08:19:08.502373+0000","last_peered":"2023-11-08T08:19:08.502373+0000","last_clean":"2023-11-08T08:19:08.502373+0000","last_became_active":"2023-11-08T08:19:04.018537+0000","last_became_peered":"2023-11-08T08:19:04.018537+0000","last_unstale":"2023-11-08T08:19:08.502373+0000","last_undegraded":"2023-11-08T08:19:08.502373+0000","last_fullsized":"2023-11-08T08:19:08.502373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:51:22.228427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.496833+0000","last_change":"2023-11-08T08:19:04.503358+0000","last_active":"2023-11-08T08:19:08.496833+0000","last_peered":"2023-11-08T08:19:08.496833+0000","last_clean":"2023-11-08T08:19:08.496833+0000","last_became_active":"2023-11-08T08:19:04.502856+0000","last_became_peered":"2023-11-08T08:19:04.502856+0000","last_unstale":"2023-11-08T08:19:08.496833+0000","last_undegraded":"2023-11-08T08:19:08.496833+0000","last_fullsized":"2023-11-08T08:19:08.496833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.989197+0000","last_change":"2023-11-08T08:19:08.531064+0000","last_active":"2023-11-08T08:19:08.989197+0000","last_peered":"2023-11-08T08:19:08.989197+0000","last_clean":"2023-11-08T08:19:08.989197+0000","last_became_active":"2023-11-08T08:19:08.530517+0000","last_became_peered":"2023-11-08T08:19:08.530517+0000","last_unstale":"2023-11-08T08:19:08.989197+0000","last_undegraded":"2023-11-08T08:19:08.989197+0000","last_fullsized":"2023-11-08T08:19:08.989197+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:51:11.743430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517381+0000","last_change":"2023-11-08T08:19:04.022640+0000","last_active":"2023-11-08T08:19:08.517381+0000","last_peered":"2023-11-08T08:19:08.517381+0000","last_clean":"2023-11-08T08:19:08.517381+0000","last_became_active":"2023-11-08T08:19:04.022244+0000","last_became_peered":"2023-11-08T08:19:04.022244+0000","last_unstale":"2023-11-08T08:19:08.517381+0000","last_undegraded":"2023-11-08T08:19:08.517381+0000","last_fullsized":"2023-11-08T08:19:08.517381+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:50:41.342112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521762+0000","last_change":"2023-11-08T08:19:04.497516+0000","last_active":"2023-11-08T08:19:08.521762+0000","last_peered":"2023-11-08T08:19:08.521762+0000","last_clean":"2023-11-08T08:19:08.521762+0000","last_became_active":"2023-11-08T08:19:04.493572+0000","last_became_peered":"2023-11-08T08:19:04.493572+0000","last_unstale":"2023-11-08T08:19:08.521762+0000","last_undegraded":"2023-11-08T08:19:08.521762+0000","last_fullsized":"2023-11-08T08:19:08.521762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:57:06.487072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517281+0000","last_change":"2023-11-08T08:19:04.495610+0000","last_active":"2023-11-08T08:19:08.517281+0000","last_peered":"2023-11-08T08:19:08.517281+0000","last_clean":"2023-11-08T08:19:08.517281+0000","last_became_active":"2023-11-08T08:19:04.495307+0000","last_became_peered":"2023-11-08T08:19:04.495307+0000","last_unstale":"2023-11-08T08:19:08.517281+0000","last_undegraded":"2023-11-08T08:19:08.517281+0000","last_fullsized":"2023-11-08T08:19:08.517281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.496569+0000","last_change":"2023-11-08T08:19:04.015753+0000","last_active":"2023-11-08T08:19:08.496569+0000","last_peered":"2023-11-08T08:19:08.496569+0000","last_clean":"2023-11-08T08:19:08.496569+0000","last_became_active":"2023-11-08T08:19:04.015173+0000","last_became_peered":"2023-11-08T08:19:04.015173+0000","last_unstale":"2023-11-08T08:19:08.496569+0000","last_undegraded":"2023-11-08T08:19:08.496569+0000","last_fullsized":"2023-11-08T08:19:08.496569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:58:24.706350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.373895+0000","last_change":"2023-11-08T08:19:08.525297+0000","last_active":"2023-11-08T08:19:09.373895+0000","last_peered":"2023-11-08T08:19:09.373895+0000","last_clean":"2023-11-08T08:19:09.373895+0000","last_became_active":"2023-11-08T08:19:08.524571+0000","last_became_peered":"2023-11-08T08:19:08.524571+0000","last_unstale":"2023-11-08T08:19:09.373895+0000","last_undegraded":"2023-11-08T08:19:09.373895+0000","last_fullsized":"2023-11-08T08:19:09.373895+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:33:00.754113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517964+0000","last_change":"2023-11-08T08:19:04.500363+0000","last_active":"2023-11-08T08:19:08.517964+0000","last_peered":"2023-11-08T08:19:08.517964+0000","last_clean":"2023-11-08T08:19:08.517964+0000","last_became_active":"2023-11-08T08:19:04.500049+0000","last_became_peered":"2023-11-08T08:19:04.500049+0000","last_unstale":"2023-11-08T08:19:08.517964+0000","last_undegraded":"2023-11-08T08:19:08.517964+0000","last_fullsized":"2023-11-08T08:19:08.517964+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:06:47.395577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521546+0000","last_change":"2023-11-08T08:19:04.020555+0000","last_active":"2023-11-08T08:19:08.521546+0000","last_peered":"2023-11-08T08:19:08.521546+0000","last_clean":"2023-11-08T08:19:08.521546+0000","last_became_active":"2023-11-08T08:19:04.020053+0000","last_became_peered":"2023-11-08T08:19:04.020053+0000","last_unstale":"2023-11-08T08:19:08.521546+0000","last_undegraded":"2023-11-08T08:19:08.521546+0000","last_fullsized":"2023-11-08T08:19:08.521546+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:01:41.280977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.994150+0000","last_change":"2023-11-08T08:19:08.526638+0000","last_active":"2023-11-08T08:19:08.994150+0000","last_peered":"2023-11-08T08:19:08.994150+0000","last_clean":"2023-11-08T08:19:08.994150+0000","last_became_active":"2023-11-08T08:19:08.526219+0000","last_became_peered":"2023-11-08T08:19:08.526219+0000","last_unstale":"2023-11-08T08:19:08.994150+0000","last_undegraded":"2023-11-08T08:19:08.994150+0000","last_fullsized":"2023-11-08T08:19:08.994150+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:56:38.412909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.500591+0000","last_change":"2023-11-08T08:19:04.022271+0000","last_active":"2023-11-08T08:19:08.500591+0000","last_peered":"2023-11-08T08:19:08.500591+0000","last_clean":"2023-11-08T08:19:08.500591+0000","last_became_active":"2023-11-08T08:19:04.021306+0000","last_became_peered":"2023-11-08T08:19:04.021306+0000","last_unstale":"2023-11-08T08:19:08.500591+0000","last_undegraded":"2023-11-08T08:19:08.500591+0000","last_fullsized":"2023-11-08T08:19:08.500591+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:26:26.874402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.502798+0000","last_change":"2023-11-08T08:19:04.494816+0000","last_active":"2023-11-08T08:19:08.502798+0000","last_peered":"2023-11-08T08:19:08.502798+0000","last_clean":"2023-11-08T08:19:08.502798+0000","last_became_active":"2023-11-08T08:19:04.494321+0000","last_became_peered":"2023-11-08T08:19:04.494321+0000","last_unstale":"2023-11-08T08:19:08.502798+0000","last_undegraded":"2023-11-08T08:19:08.502798+0000","last_fullsized":"2023-11-08T08:19:08.502798+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:04:55.302575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372527+0000","last_change":"2023-11-08T08:19:08.542779+0000","last_active":"2023-11-08T08:19:09.372527+0000","last_peered":"2023-11-08T08:19:09.372527+0000","last_clean":"2023-11-08T08:19:09.372527+0000","last_became_active":"2023-11-08T08:19:08.542399+0000","last_became_peered":"2023-11-08T08:19:08.542399+0000","last_unstale":"2023-11-08T08:19:09.372527+0000","last_undegraded":"2023-11-08T08:19:09.372527+0000","last_fullsized":"2023-11-08T08:19:09.372527+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:16:14.032597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:21.997708+0000","last_change":"2023-11-08T08:19:04.018152+0000","last_active":"2023-11-08T08:19:21.997708+0000","last_peered":"2023-11-08T08:19:21.997708+0000","last_clean":"2023-11-08T08:19:21.997708+0000","last_became_active":"2023-11-08T08:19:04.017695+0000","last_became_peered":"2023-11-08T08:19:04.017695+0000","last_unstale":"2023-11-08T08:19:21.997708+0000","last_undegraded":"2023-11-08T08:19:21.997708+0000","last_fullsized":"2023-11-08T08:19:21.997708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:55:20.782396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3745,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512046+0000","last_change":"2023-11-08T08:19:04.501576+0000","last_active":"2023-11-08T08:19:08.512046+0000","last_peered":"2023-11-08T08:19:08.512046+0000","last_clean":"2023-11-08T08:19:08.512046+0000","last_became_active":"2023-11-08T08:19:04.500829+0000","last_became_peered":"2023-11-08T08:19:04.500829+0000","last_unstale":"2023-11-08T08:19:08.512046+0000","last_undegraded":"2023-11-08T08:19:08.512046+0000","last_fullsized":"2023-11-08T08:19:08.512046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372324+0000","last_change":"2023-11-08T08:19:08.536704+0000","last_active":"2023-11-08T08:19:09.372324+0000","last_peered":"2023-11-08T08:19:09.372324+0000","last_clean":"2023-11-08T08:19:09.372324+0000","last_became_active":"2023-11-08T08:19:08.536339+0000","last_became_peered":"2023-11-08T08:19:08.536339+0000","last_unstale":"2023-11-08T08:19:09.372324+0000","last_undegraded":"2023-11-08T08:19:09.372324+0000","last_fullsized":"2023-11-08T08:19:09.372324+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:51:11.743430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513610+0000","last_change":"2023-11-08T08:19:04.496877+0000","last_active":"2023-11-08T08:19:08.513610+0000","last_peered":"2023-11-08T08:19:08.513610+0000","last_clean":"2023-11-08T08:19:08.513610+0000","last_became_active":"2023-11-08T08:19:04.496505+0000","last_became_peered":"2023-11-08T08:19:04.496505+0000","last_unstale":"2023-11-08T08:19:08.513610+0000","last_undegraded":"2023-11-08T08:19:08.513610+0000","last_fullsized":"2023-11-08T08:19:08.513610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:59:35.582106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521817+0000","last_change":"2023-11-08T08:18:58.771310+0000","last_active":"2023-11-08T08:19:08.521817+0000","last_peered":"2023-11-08T08:19:08.521817+0000","last_clean":"2023-11-08T08:19:08.521817+0000","last_became_active":"2023-11-08T08:18:58.770891+0000","last_became_peered":"2023-11-08T08:18:58.770891+0000","last_unstale":"2023-11-08T08:19:08.521817+0000","last_undegraded":"2023-11-08T08:19:08.521817+0000","last_fullsized":"2023-11-08T08:19:08.521817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"27'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.969187+0000","last_change":"2023-11-08T08:19:04.012926+0000","last_active":"2023-11-08T08:19:08.969187+0000","last_peered":"2023-11-08T08:19:08.969187+0000","last_clean":"2023-11-08T08:19:08.969187+0000","last_became_active":"2023-11-08T08:19:04.012634+0000","last_became_peered":"2023-11-08T08:19:04.012634+0000","last_unstale":"2023-11-08T08:19:08.969187+0000","last_undegraded":"2023-11-08T08:19:08.969187+0000","last_fullsized":"2023-11-08T08:19:08.969187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:07:35.132845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.371641+0000","last_change":"2023-11-08T08:19:08.532617+0000","last_active":"2023-11-08T08:19:09.371641+0000","last_peered":"2023-11-08T08:19:09.371641+0000","last_clean":"2023-11-08T08:19:09.371641+0000","last_became_active":"2023-11-08T08:19:08.532177+0000","last_became_peered":"2023-11-08T08:19:08.532177+0000","last_unstale":"2023-11-08T08:19:09.371641+0000","last_undegraded":"2023-11-08T08:19:09.371641+0000","last_fullsized":"2023-11-08T08:19:09.371641+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:08.797102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517321+0000","last_change":"2023-11-08T08:19:04.504132+0000","last_active":"2023-11-08T08:19:08.517321+0000","last_peered":"2023-11-08T08:19:08.517321+0000","last_clean":"2023-11-08T08:19:08.517321+0000","last_became_active":"2023-11-08T08:19:04.503747+0000","last_became_peered":"2023-11-08T08:19:04.503747+0000","last_unstale":"2023-11-08T08:19:08.517321+0000","last_undegraded":"2023-11-08T08:19:08.517321+0000","last_fullsized":"2023-11-08T08:19:08.517321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:27:40.872420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503547+0000","last_change":"2023-11-08T08:18:58.770483+0000","last_active":"2023-11-08T08:19:08.503547+0000","last_peered":"2023-11-08T08:19:08.503547+0000","last_clean":"2023-11-08T08:19:08.503547+0000","last_became_active":"2023-11-08T08:18:58.770040+0000","last_became_peered":"2023-11-08T08:18:58.770040+0000","last_unstale":"2023-11-08T08:19:08.503547+0000","last_undegraded":"2023-11-08T08:19:08.503547+0000","last_fullsized":"2023-11-08T08:19:08.503547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.520152+0000","last_change":"2023-11-08T08:19:04.022667+0000","last_active":"2023-11-08T08:19:08.520152+0000","last_peered":"2023-11-08T08:19:08.520152+0000","last_clean":"2023-11-08T08:19:08.520152+0000","last_became_active":"2023-11-08T08:19:04.022430+0000","last_became_peered":"2023-11-08T08:19:04.022430+0000","last_unstale":"2023-11-08T08:19:08.520152+0000","last_undegraded":"2023-11-08T08:19:08.520152+0000","last_fullsized":"2023-11-08T08:19:08.520152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:15:32.560862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.992931+0000","last_change":"2023-11-08T08:19:08.531983+0000","last_active":"2023-11-08T08:19:08.992931+0000","last_peered":"2023-11-08T08:19:08.992931+0000","last_clean":"2023-11-08T08:19:08.992931+0000","last_became_active":"2023-11-08T08:19:08.531584+0000","last_became_peered":"2023-11-08T08:19:08.531584+0000","last_unstale":"2023-11-08T08:19:08.992931+0000","last_undegraded":"2023-11-08T08:19:08.992931+0000","last_fullsized":"2023-11-08T08:19:08.992931+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:56:34.397672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.495387+0000","last_change":"2023-11-08T08:19:04.481283+0000","last_active":"2023-11-08T08:19:08.495387+0000","last_peered":"2023-11-08T08:19:08.495387+0000","last_clean":"2023-11-08T08:19:08.495387+0000","last_became_active":"2023-11-08T08:19:04.480992+0000","last_became_peered":"2023-11-08T08:19:04.480992+0000","last_unstale":"2023-11-08T08:19:08.495387+0000","last_undegraded":"2023-11-08T08:19:08.495387+0000","last_fullsized":"2023-11-08T08:19:08.495387+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.500946+0000","last_change":"2023-11-08T08:18:58.766848+0000","last_active":"2023-11-08T08:19:08.500946+0000","last_peered":"2023-11-08T08:19:08.500946+0000","last_clean":"2023-11-08T08:19:08.500946+0000","last_became_active":"2023-11-08T08:18:58.766378+0000","last_became_peered":"2023-11-08T08:18:58.766378+0000","last_unstale":"2023-11-08T08:19:08.500946+0000","last_undegraded":"2023-11-08T08:19:08.500946+0000","last_fullsized":"2023-11-08T08:19:08.500946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503466+0000","last_change":"2023-11-08T08:19:04.018717+0000","last_active":"2023-11-08T08:19:08.503466+0000","last_peered":"2023-11-08T08:19:08.503466+0000","last_clean":"2023-11-08T08:19:08.503466+0000","last_became_active":"2023-11-08T08:19:04.018355+0000","last_became_peered":"2023-11-08T08:19:04.018355+0000","last_unstale":"2023-11-08T08:19:08.503466+0000","last_undegraded":"2023-11-08T08:19:08.503466+0000","last_fullsized":"2023-11-08T08:19:08.503466+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:21:09.218618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.374050+0000","last_change":"2023-11-08T08:19:08.539068+0000","last_active":"2023-11-08T08:19:09.374050+0000","last_peered":"2023-11-08T08:19:09.374050+0000","last_clean":"2023-11-08T08:19:09.374050+0000","last_became_active":"2023-11-08T08:19:08.538763+0000","last_became_peered":"2023-11-08T08:19:08.538763+0000","last_unstale":"2023-11-08T08:19:09.374050+0000","last_undegraded":"2023-11-08T08:19:09.374050+0000","last_fullsized":"2023-11-08T08:19:09.374050+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:27:44.380774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.502722+0000","last_change":"2023-11-08T08:19:04.508515+0000","last_active":"2023-11-08T08:19:08.502722+0000","last_peered":"2023-11-08T08:19:08.502722+0000","last_clean":"2023-11-08T08:19:08.502722+0000","last_became_active":"2023-11-08T08:19:04.508227+0000","last_became_peered":"2023-11-08T08:19:04.508227+0000","last_unstale":"2023-11-08T08:19:08.502722+0000","last_undegraded":"2023-11-08T08:19:08.502722+0000","last_fullsized":"2023-11-08T08:19:08.502722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:48:06.180083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503646+0000","last_change":"2023-11-08T08:18:58.769764+0000","last_active":"2023-11-08T08:19:08.503646+0000","last_peered":"2023-11-08T08:19:08.503646+0000","last_clean":"2023-11-08T08:19:08.503646+0000","last_became_active":"2023-11-08T08:18:58.769357+0000","last_became_peered":"2023-11-08T08:18:58.769357+0000","last_unstale":"2023-11-08T08:19:08.503646+0000","last_undegraded":"2023-11-08T08:19:08.503646+0000","last_fullsized":"2023-11-08T08:19:08.503646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:13.748094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519826+0000","last_change":"2023-11-08T08:19:04.019534+0000","last_active":"2023-11-08T08:19:08.519826+0000","last_peered":"2023-11-08T08:19:08.519826+0000","last_clean":"2023-11-08T08:19:08.519826+0000","last_became_active":"2023-11-08T08:19:04.019269+0000","last_became_peered":"2023-11-08T08:19:04.019269+0000","last_unstale":"2023-11-08T08:19:08.519826+0000","last_undegraded":"2023-11-08T08:19:08.519826+0000","last_fullsized":"2023-11-08T08:19:08.519826+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:18:14.269118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.989010+0000","last_change":"2023-11-08T08:19:08.534164+0000","last_active":"2023-11-08T08:19:08.989010+0000","last_peered":"2023-11-08T08:19:08.989010+0000","last_clean":"2023-11-08T08:19:08.989010+0000","last_became_active":"2023-11-08T08:19:08.533749+0000","last_became_peered":"2023-11-08T08:19:08.533749+0000","last_unstale":"2023-11-08T08:19:08.989010+0000","last_undegraded":"2023-11-08T08:19:08.989010+0000","last_fullsized":"2023-11-08T08:19:08.989010+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:44:30.267694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503356+0000","last_change":"2023-11-08T08:19:04.492859+0000","last_active":"2023-11-08T08:19:08.503356+0000","last_peered":"2023-11-08T08:19:08.503356+0000","last_clean":"2023-11-08T08:19:08.503356+0000","last_became_active":"2023-11-08T08:19:04.492564+0000","last_became_peered":"2023-11-08T08:19:04.492564+0000","last_unstale":"2023-11-08T08:19:08.503356+0000","last_undegraded":"2023-11-08T08:19:08.503356+0000","last_fullsized":"2023-11-08T08:19:08.503356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512115+0000","last_change":"2023-11-08T08:19:04.024444+0000","last_active":"2023-11-08T08:19:08.512115+0000","last_peered":"2023-11-08T08:19:08.512115+0000","last_clean":"2023-11-08T08:19:08.512115+0000","last_became_active":"2023-11-08T08:19:04.024046+0000","last_became_peered":"2023-11-08T08:19:04.024046+0000","last_unstale":"2023-11-08T08:19:08.512115+0000","last_undegraded":"2023-11-08T08:19:08.512115+0000","last_fullsized":"2023-11-08T08:19:08.512115+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:15:32.560862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.514051+0000","last_change":"2023-11-08T08:18:58.771460+0000","last_active":"2023-11-08T08:19:08.514051+0000","last_peered":"2023-11-08T08:19:08.514051+0000","last_clean":"2023-11-08T08:19:08.514051+0000","last_became_active":"2023-11-08T08:18:58.770648+0000","last_became_peered":"2023-11-08T08:18:58.770648+0000","last_unstale":"2023-11-08T08:19:08.514051+0000","last_undegraded":"2023-11-08T08:19:08.514051+0000","last_fullsized":"2023-11-08T08:19:08.514051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.988677+0000","last_change":"2023-11-08T08:19:08.525326+0000","last_active":"2023-11-08T08:19:08.988677+0000","last_peered":"2023-11-08T08:19:08.988677+0000","last_clean":"2023-11-08T08:19:08.988677+0000","last_became_active":"2023-11-08T08:19:08.524957+0000","last_became_peered":"2023-11-08T08:19:08.524957+0000","last_unstale":"2023-11-08T08:19:08.988677+0000","last_undegraded":"2023-11-08T08:19:08.988677+0000","last_fullsized":"2023-11-08T08:19:08.988677+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:08.797102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503646+0000","last_change":"2023-11-08T08:19:04.503414+0000","last_active":"2023-11-08T08:19:08.503646+0000","last_peered":"2023-11-08T08:19:08.503646+0000","last_clean":"2023-11-08T08:19:08.503646+0000","last_became_active":"2023-11-08T08:19:04.503122+0000","last_became_peered":"2023-11-08T08:19:04.503122+0000","last_unstale":"2023-11-08T08:19:08.503646+0000","last_undegraded":"2023-11-08T08:19:08.503646+0000","last_fullsized":"2023-11-08T08:19:08.503646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501677+0000","last_change":"2023-11-08T08:19:04.020937+0000","last_active":"2023-11-08T08:19:08.501677+0000","last_peered":"2023-11-08T08:19:08.501677+0000","last_clean":"2023-11-08T08:19:08.501677+0000","last_became_active":"2023-11-08T08:19:04.020517+0000","last_became_peered":"2023-11-08T08:19:04.020517+0000","last_unstale":"2023-11-08T08:19:08.501677+0000","last_undegraded":"2023-11-08T08:19:08.501677+0000","last_fullsized":"2023-11-08T08:19:08.501677+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:50:41.342112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.509869+0000","last_change":"2023-11-08T08:18:58.973732+0000","last_active":"2023-11-08T08:19:08.509869+0000","last_peered":"2023-11-08T08:19:08.509869+0000","last_clean":"2023-11-08T08:19:08.509869+0000","last_became_active":"2023-11-08T08:18:58.973007+0000","last_became_peered":"2023-11-08T08:18:58.973007+0000","last_unstale":"2023-11-08T08:19:08.509869+0000","last_undegraded":"2023-11-08T08:19:08.509869+0000","last_fullsized":"2023-11-08T08:19:08.509869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.375576+0000","last_change":"2023-11-08T08:19:08.542391+0000","last_active":"2023-11-08T08:19:09.375576+0000","last_peered":"2023-11-08T08:19:09.375576+0000","last_clean":"2023-11-08T08:19:09.375576+0000","last_became_active":"2023-11-08T08:19:08.538962+0000","last_became_peered":"2023-11-08T08:19:08.538962+0000","last_unstale":"2023-11-08T08:19:09.375576+0000","last_undegraded":"2023-11-08T08:19:09.375576+0000","last_fullsized":"2023-11-08T08:19:09.375576+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:05:20.025231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.496994+0000","last_change":"2023-11-08T08:19:04.495234+0000","last_active":"2023-11-08T08:19:08.496994+0000","last_peered":"2023-11-08T08:19:08.496994+0000","last_clean":"2023-11-08T08:19:08.496994+0000","last_became_active":"2023-11-08T08:19:04.494991+0000","last_became_peered":"2023-11-08T08:19:04.494991+0000","last_unstale":"2023-11-08T08:19:08.496994+0000","last_undegraded":"2023-11-08T08:19:08.496994+0000","last_fullsized":"2023-11-08T08:19:08.496994+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:55:43.865351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.494564+0000","last_change":"2023-11-08T08:19:04.025597+0000","last_active":"2023-11-08T08:19:08.494564+0000","last_peered":"2023-11-08T08:19:08.494564+0000","last_clean":"2023-11-08T08:19:08.494564+0000","last_became_active":"2023-11-08T08:19:04.025292+0000","last_became_peered":"2023-11-08T08:19:04.025292+0000","last_unstale":"2023-11-08T08:19:08.494564+0000","last_undegraded":"2023-11-08T08:19:08.494564+0000","last_fullsized":"2023-11-08T08:19:08.494564+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:48:40.803835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.499836+0000","last_change":"2023-11-08T08:18:58.770136+0000","last_active":"2023-11-08T08:19:08.499836+0000","last_peered":"2023-11-08T08:19:08.499836+0000","last_clean":"2023-11-08T08:19:08.499836+0000","last_became_active":"2023-11-08T08:18:58.769797+0000","last_became_peered":"2023-11-08T08:18:58.769797+0000","last_unstale":"2023-11-08T08:19:08.499836+0000","last_undegraded":"2023-11-08T08:19:08.499836+0000","last_fullsized":"2023-11-08T08:19:08.499836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:29:12.107128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372601+0000","last_change":"2023-11-08T08:19:08.532719+0000","last_active":"2023-11-08T08:19:09.372601+0000","last_peered":"2023-11-08T08:19:09.372601+0000","last_clean":"2023-11-08T08:19:09.372601+0000","last_became_active":"2023-11-08T08:19:08.532282+0000","last_became_peered":"2023-11-08T08:19:08.532282+0000","last_unstale":"2023-11-08T08:19:09.372601+0000","last_undegraded":"2023-11-08T08:19:09.372601+0000","last_fullsized":"2023-11-08T08:19:09.372601+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:28:49.369446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.520613+0000","last_change":"2023-11-08T08:19:04.506360+0000","last_active":"2023-11-08T08:19:08.520613+0000","last_peered":"2023-11-08T08:19:08.520613+0000","last_clean":"2023-11-08T08:19:08.520613+0000","last_became_active":"2023-11-08T08:19:04.506117+0000","last_became_peered":"2023-11-08T08:19:04.506117+0000","last_unstale":"2023-11-08T08:19:08.520613+0000","last_undegraded":"2023-11-08T08:19:08.520613+0000","last_fullsized":"2023-11-08T08:19:08.520613+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:17:34.354695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517669+0000","last_change":"2023-11-08T08:19:04.020276+0000","last_active":"2023-11-08T08:19:08.517669+0000","last_peered":"2023-11-08T08:19:08.517669+0000","last_clean":"2023-11-08T08:19:08.517669+0000","last_became_active":"2023-11-08T08:19:04.019739+0000","last_became_peered":"2023-11-08T08:19:04.019739+0000","last_unstale":"2023-11-08T08:19:08.517669+0000","last_undegraded":"2023-11-08T08:19:08.517669+0000","last_fullsized":"2023-11-08T08:19:08.517669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:48:40.803835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519500+0000","last_change":"2023-11-08T08:18:58.777140+0000","last_active":"2023-11-08T08:19:08.519500+0000","last_peered":"2023-11-08T08:19:08.519500+0000","last_clean":"2023-11-08T08:19:08.519500+0000","last_became_active":"2023-11-08T08:18:58.776834+0000","last_became_peered":"2023-11-08T08:18:58.776834+0000","last_unstale":"2023-11-08T08:19:08.519500+0000","last_undegraded":"2023-11-08T08:19:08.519500+0000","last_fullsized":"2023-11-08T08:19:08.519500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.499953+0000","last_change":"2023-11-08T08:18:53.780089+0000","last_active":"2023-11-08T08:19:08.499953+0000","last_peered":"2023-11-08T08:19:08.499953+0000","last_clean":"2023-11-08T08:19:08.499953+0000","last_became_active":"2023-11-08T08:18:53.779533+0000","last_became_peered":"2023-11-08T08:18:53.779533+0000","last_unstale":"2023-11-08T08:19:08.499953+0000","last_undegraded":"2023-11-08T08:19:08.499953+0000","last_fullsized":"2023-11-08T08:19:08.499953+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_clean_scrub_stamp":"2023-11-08T08:18:52.744693+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:58:24.877070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.994953+0000","last_change":"2023-11-08T08:19:08.535027+0000","last_active":"2023-11-08T08:19:08.994953+0000","last_peered":"2023-11-08T08:19:08.994953+0000","last_clean":"2023-11-08T08:19:08.994953+0000","last_became_active":"2023-11-08T08:19:08.534620+0000","last_became_peered":"2023-11-08T08:19:08.534620+0000","last_unstale":"2023-11-08T08:19:08.994953+0000","last_undegraded":"2023-11-08T08:19:08.994953+0000","last_fullsized":"2023-11-08T08:19:08.994953+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:51:11.743430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503552+0000","last_change":"2023-11-08T08:19:04.503360+0000","last_active":"2023-11-08T08:19:08.503552+0000","last_peered":"2023-11-08T08:19:08.503552+0000","last_clean":"2023-11-08T08:19:08.503552+0000","last_became_active":"2023-11-08T08:19:04.502756+0000","last_became_peered":"2023-11-08T08:19:04.502756+0000","last_unstale":"2023-11-08T08:19:08.503552+0000","last_undegraded":"2023-11-08T08:19:08.503552+0000","last_fullsized":"2023-11-08T08:19:08.503552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:57:06.487072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517506+0000","last_change":"2023-11-08T08:19:04.025872+0000","last_active":"2023-11-08T08:19:08.517506+0000","last_peered":"2023-11-08T08:19:08.517506+0000","last_clean":"2023-11-08T08:19:08.517506+0000","last_became_active":"2023-11-08T08:19:04.025567+0000","last_became_peered":"2023-11-08T08:19:04.025567+0000","last_unstale":"2023-11-08T08:19:08.517506+0000","last_undegraded":"2023-11-08T08:19:08.517506+0000","last_fullsized":"2023-11-08T08:19:08.517506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:52:24.620964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.994736+0000","last_change":"2023-11-08T08:19:08.523353+0000","last_active":"2023-11-08T08:19:08.994736+0000","last_peered":"2023-11-08T08:19:08.994736+0000","last_clean":"2023-11-08T08:19:08.994736+0000","last_became_active":"2023-11-08T08:19:08.522741+0000","last_became_peered":"2023-11-08T08:19:08.522741+0000","last_unstale":"2023-11-08T08:19:08.994736+0000","last_undegraded":"2023-11-08T08:19:08.994736+0000","last_fullsized":"2023-11-08T08:19:08.994736+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:01:48.615826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512369+0000","last_change":"2023-11-08T08:19:04.512228+0000","last_active":"2023-11-08T08:19:08.512369+0000","last_peered":"2023-11-08T08:19:08.512369+0000","last_clean":"2023-11-08T08:19:08.512369+0000","last_became_active":"2023-11-08T08:19:04.501319+0000","last_became_peered":"2023-11-08T08:19:04.501319+0000","last_unstale":"2023-11-08T08:19:08.512369+0000","last_undegraded":"2023-11-08T08:19:08.512369+0000","last_fullsized":"2023-11-08T08:19:08.512369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:27:36.453040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513674+0000","last_change":"2023-11-08T08:19:04.012809+0000","last_active":"2023-11-08T08:19:08.513674+0000","last_peered":"2023-11-08T08:19:08.513674+0000","last_clean":"2023-11-08T08:19:08.513674+0000","last_became_active":"2023-11-08T08:19:04.012289+0000","last_became_peered":"2023-11-08T08:19:04.012289+0000","last_unstale":"2023-11-08T08:19:08.513674+0000","last_undegraded":"2023-11-08T08:19:08.513674+0000","last_fullsized":"2023-11-08T08:19:08.513674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:52:24.620964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372268+0000","last_change":"2023-11-08T08:19:08.538675+0000","last_active":"2023-11-08T08:19:09.372268+0000","last_peered":"2023-11-08T08:19:09.372268+0000","last_clean":"2023-11-08T08:19:09.372268+0000","last_became_active":"2023-11-08T08:19:08.537804+0000","last_became_peered":"2023-11-08T08:19:08.537804+0000","last_unstale":"2023-11-08T08:19:09.372268+0000","last_undegraded":"2023-11-08T08:19:09.372268+0000","last_fullsized":"2023-11-08T08:19:09.372268+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:32:44.139418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.516288+0000","last_change":"2023-11-08T08:19:04.498523+0000","last_active":"2023-11-08T08:19:08.516288+0000","last_peered":"2023-11-08T08:19:08.516288+0000","last_clean":"2023-11-08T08:19:08.516288+0000","last_became_active":"2023-11-08T08:19:04.497819+0000","last_became_peered":"2023-11-08T08:19:04.497819+0000","last_unstale":"2023-11-08T08:19:08.516288+0000","last_undegraded":"2023-11-08T08:19:08.516288+0000","last_fullsized":"2023-11-08T08:19:08.516288+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:55:43.865351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518299+0000","last_change":"2023-11-08T08:19:04.024906+0000","last_active":"2023-11-08T08:19:08.518299+0000","last_peered":"2023-11-08T08:19:08.518299+0000","last_clean":"2023-11-08T08:19:08.518299+0000","last_became_active":"2023-11-08T08:19:04.024603+0000","last_became_peered":"2023-11-08T08:19:04.024603+0000","last_unstale":"2023-11-08T08:19:08.518299+0000","last_undegraded":"2023-11-08T08:19:08.518299+0000","last_fullsized":"2023-11-08T08:19:08.518299+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:21:09.218618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372046+0000","last_change":"2023-11-08T08:19:08.525624+0000","last_active":"2023-11-08T08:19:09.372046+0000","last_peered":"2023-11-08T08:19:09.372046+0000","last_clean":"2023-11-08T08:19:09.372046+0000","last_became_active":"2023-11-08T08:19:08.524202+0000","last_became_peered":"2023-11-08T08:19:08.524202+0000","last_unstale":"2023-11-08T08:19:09.372046+0000","last_undegraded":"2023-11-08T08:19:09.372046+0000","last_fullsized":"2023-11-08T08:19:09.372046+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:08.797102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513312+0000","last_change":"2023-11-08T08:19:04.511393+0000","last_active":"2023-11-08T08:19:08.513312+0000","last_peered":"2023-11-08T08:19:08.513312+0000","last_clean":"2023-11-08T08:19:08.513312+0000","last_became_active":"2023-11-08T08:19:04.510749+0000","last_became_peered":"2023-11-08T08:19:04.510749+0000","last_unstale":"2023-11-08T08:19:08.513312+0000","last_undegraded":"2023-11-08T08:19:08.513312+0000","last_fullsized":"2023-11-08T08:19:08.513312+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:09:30.686343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518295+0000","last_change":"2023-11-08T08:19:04.022458+0000","last_active":"2023-11-08T08:19:08.518295+0000","last_peered":"2023-11-08T08:19:08.518295+0000","last_clean":"2023-11-08T08:19:08.518295+0000","last_became_active":"2023-11-08T08:19:04.022003+0000","last_became_peered":"2023-11-08T08:19:04.022003+0000","last_unstale":"2023-11-08T08:19:08.518295+0000","last_undegraded":"2023-11-08T08:19:08.518295+0000","last_fullsized":"2023-11-08T08:19:08.518295+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:21:09.218618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.375400+0000","last_change":"2023-11-08T08:19:08.536502+0000","last_active":"2023-11-08T08:19:09.375400+0000","last_peered":"2023-11-08T08:19:09.375400+0000","last_clean":"2023-11-08T08:19:09.375400+0000","last_became_active":"2023-11-08T08:19:08.536112+0000","last_became_peered":"2023-11-08T08:19:08.536112+0000","last_unstale":"2023-11-08T08:19:09.375400+0000","last_undegraded":"2023-11-08T08:19:09.375400+0000","last_fullsized":"2023-11-08T08:19:09.375400+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:26:20.245634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.495202+0000","last_change":"2023-11-08T08:19:04.493387+0000","last_active":"2023-11-08T08:19:08.495202+0000","last_peered":"2023-11-08T08:19:08.495202+0000","last_clean":"2023-11-08T08:19:08.495202+0000","last_became_active":"2023-11-08T08:19:04.493131+0000","last_became_peered":"2023-11-08T08:19:04.493131+0000","last_unstale":"2023-11-08T08:19:08.495202+0000","last_undegraded":"2023-11-08T08:19:08.495202+0000","last_fullsized":"2023-11-08T08:19:08.495202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:56:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519525+0000","last_change":"2023-11-08T08:19:04.021637+0000","last_active":"2023-11-08T08:19:08.519525+0000","last_peered":"2023-11-08T08:19:08.519525+0000","last_clean":"2023-11-08T08:19:08.519525+0000","last_became_active":"2023-11-08T08:19:04.018995+0000","last_became_peered":"2023-11-08T08:19:04.018995+0000","last_unstale":"2023-11-08T08:19:08.519525+0000","last_undegraded":"2023-11-08T08:19:08.519525+0000","last_fullsized":"2023-11-08T08:19:08.519525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:44.433066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.375204+0000","last_change":"2023-11-08T08:19:08.535800+0000","last_active":"2023-11-08T08:19:09.375204+0000","last_peered":"2023-11-08T08:19:09.375204+0000","last_clean":"2023-11-08T08:19:09.375204+0000","last_became_active":"2023-11-08T08:19:08.529039+0000","last_became_peered":"2023-11-08T08:19:08.529039+0000","last_unstale":"2023-11-08T08:19:09.375204+0000","last_undegraded":"2023-11-08T08:19:09.375204+0000","last_fullsized":"2023-11-08T08:19:09.375204+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:55:58.092159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518136+0000","last_change":"2023-11-08T08:19:04.497093+0000","last_active":"2023-11-08T08:19:08.518136+0000","last_peered":"2023-11-08T08:19:08.518136+0000","last_clean":"2023-11-08T08:19:08.518136+0000","last_became_active":"2023-11-08T08:19:04.496619+0000","last_became_peered":"2023-11-08T08:19:04.496619+0000","last_unstale":"2023-11-08T08:19:08.518136+0000","last_undegraded":"2023-11-08T08:19:08.518136+0000","last_fullsized":"2023-11-08T08:19:08.518136+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:27:39.386956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.500739+0000","last_change":"2023-11-08T08:19:04.015072+0000","last_active":"2023-11-08T08:19:08.500739+0000","last_peered":"2023-11-08T08:19:08.500739+0000","last_clean":"2023-11-08T08:19:08.500739+0000","last_became_active":"2023-11-08T08:19:04.014727+0000","last_became_peered":"2023-11-08T08:19:04.014727+0000","last_unstale":"2023-11-08T08:19:08.500739+0000","last_undegraded":"2023-11-08T08:19:08.500739+0000","last_fullsized":"2023-11-08T08:19:08.500739+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:17:33.341483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.375246+0000","last_change":"2023-11-08T08:19:08.534568+0000","last_active":"2023-11-08T08:19:09.375246+0000","last_peered":"2023-11-08T08:19:09.375246+0000","last_clean":"2023-11-08T08:19:09.375246+0000","last_became_active":"2023-11-08T08:19:08.534139+0000","last_became_peered":"2023-11-08T08:19:08.534139+0000","last_unstale":"2023-11-08T08:19:09.375246+0000","last_undegraded":"2023-11-08T08:19:09.375246+0000","last_fullsized":"2023-11-08T08:19:09.375246+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:59:39.090459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513527+0000","last_change":"2023-11-08T08:19:04.512845+0000","last_active":"2023-11-08T08:19:08.513527+0000","last_peered":"2023-11-08T08:19:08.513527+0000","last_clean":"2023-11-08T08:19:08.513527+0000","last_became_active":"2023-11-08T08:19:04.512499+0000","last_became_peered":"2023-11-08T08:19:04.512499+0000","last_unstale":"2023-11-08T08:19:08.513527+0000","last_undegraded":"2023-11-08T08:19:08.513527+0000","last_fullsized":"2023-11-08T08:19:08.513527+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:24:10.475972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501411+0000","last_change":"2023-11-08T08:19:04.018033+0000","last_active":"2023-11-08T08:19:08.501411+0000","last_peered":"2023-11-08T08:19:08.501411+0000","last_clean":"2023-11-08T08:19:08.501411+0000","last_became_active":"2023-11-08T08:19:04.017059+0000","last_became_peered":"2023-11-08T08:19:04.017059+0000","last_unstale":"2023-11-08T08:19:08.501411+0000","last_undegraded":"2023-11-08T08:19:08.501411+0000","last_fullsized":"2023-11-08T08:19:08.501411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:44.433066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.369706+0000","last_change":"2023-11-08T08:19:08.539326+0000","last_active":"2023-11-08T08:19:09.369706+0000","last_peered":"2023-11-08T08:19:09.369706+0000","last_clean":"2023-11-08T08:19:09.369706+0000","last_became_active":"2023-11-08T08:19:08.538846+0000","last_became_peered":"2023-11-08T08:19:08.538846+0000","last_unstale":"2023-11-08T08:19:09.369706+0000","last_undegraded":"2023-11-08T08:19:09.369706+0000","last_fullsized":"2023-11-08T08:19:09.369706+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:33:00.754113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513023+0000","last_change":"2023-11-08T08:19:04.503981+0000","last_active":"2023-11-08T08:19:08.513023+0000","last_peered":"2023-11-08T08:19:08.513023+0000","last_clean":"2023-11-08T08:19:08.513023+0000","last_became_active":"2023-11-08T08:19:04.503651+0000","last_became_peered":"2023-11-08T08:19:04.503651+0000","last_unstale":"2023-11-08T08:19:08.513023+0000","last_undegraded":"2023-11-08T08:19:08.513023+0000","last_fullsized":"2023-11-08T08:19:08.513023+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:55:43.865351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.500289+0000","last_change":"2023-11-08T08:19:04.022166+0000","last_active":"2023-11-08T08:19:08.500289+0000","last_peered":"2023-11-08T08:19:08.500289+0000","last_clean":"2023-11-08T08:19:08.500289+0000","last_became_active":"2023-11-08T08:19:04.017289+0000","last_became_peered":"2023-11-08T08:19:04.017289+0000","last_unstale":"2023-11-08T08:19:08.500289+0000","last_undegraded":"2023-11-08T08:19:08.500289+0000","last_fullsized":"2023-11-08T08:19:08.500289+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:51:22.228427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.995287+0000","last_change":"2023-11-08T08:19:08.534923+0000","last_active":"2023-11-08T08:19:08.995287+0000","last_peered":"2023-11-08T08:19:08.995287+0000","last_clean":"2023-11-08T08:19:08.995287+0000","last_became_active":"2023-11-08T08:19:08.530869+0000","last_became_peered":"2023-11-08T08:19:08.530869+0000","last_unstale":"2023-11-08T08:19:08.995287+0000","last_undegraded":"2023-11-08T08:19:08.995287+0000","last_fullsized":"2023-11-08T08:19:08.995287+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:47:09.660706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372246+0000","last_change":"2023-11-08T08:19:08.541714+0000","last_active":"2023-11-08T08:19:09.372246+0000","last_peered":"2023-11-08T08:19:09.372246+0000","last_clean":"2023-11-08T08:19:09.372246+0000","last_became_active":"2023-11-08T08:19:08.537859+0000","last_became_peered":"2023-11-08T08:19:08.537859+0000","last_unstale":"2023-11-08T08:19:09.372246+0000","last_undegraded":"2023-11-08T08:19:09.372246+0000","last_fullsized":"2023-11-08T08:19:09.372246+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:33:11.313632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518613+0000","last_change":"2023-11-08T08:19:04.008921+0000","last_active":"2023-11-08T08:19:08.518613+0000","last_peered":"2023-11-08T08:19:08.518613+0000","last_clean":"2023-11-08T08:19:08.518613+0000","last_became_active":"2023-11-08T08:19:04.008616+0000","last_became_peered":"2023-11-08T08:19:04.008616+0000","last_unstale":"2023-11-08T08:19:08.518613+0000","last_undegraded":"2023-11-08T08:19:08.518613+0000","last_fullsized":"2023-11-08T08:19:08.518613+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:50:41.342112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.520011+0000","last_change":"2023-11-08T08:19:04.501127+0000","last_active":"2023-11-08T08:19:08.520011+0000","last_peered":"2023-11-08T08:19:08.520011+0000","last_clean":"2023-11-08T08:19:08.520011+0000","last_became_active":"2023-11-08T08:19:04.500705+0000","last_became_peered":"2023-11-08T08:19:04.500705+0000","last_unstale":"2023-11-08T08:19:08.520011+0000","last_undegraded":"2023-11-08T08:19:08.520011+0000","last_fullsized":"2023-11-08T08:19:08.520011+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:25.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372698+0000","last_change":"2023-11-08T08:19:08.533064+0000","last_active":"2023-11-08T08:19:09.372698+0000","last_peered":"2023-11-08T08:19:09.372698+0000","last_clean":"2023-11-08T08:19:09.372698+0000","last_became_active":"2023-11-08T08:19:08.532631+0000","last_became_peered":"2023-11-08T08:19:08.532631+0000","last_unstale":"2023-11-08T08:19:09.372698+0000","last_undegraded":"2023-11-08T08:19:09.372698+0000","last_fullsized":"2023-11-08T08:19:09.372698+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:32:44.139418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501706+0000","last_change":"2023-11-08T08:19:04.017775+0000","last_active":"2023-11-08T08:19:08.501706+0000","last_peered":"2023-11-08T08:19:08.501706+0000","last_clean":"2023-11-08T08:19:08.501706+0000","last_became_active":"2023-11-08T08:19:04.017572+0000","last_became_peered":"2023-11-08T08:19:04.017572+0000","last_unstale":"2023-11-08T08:19:08.501706+0000","last_undegraded":"2023-11-08T08:19:08.501706+0000","last_fullsized":"2023-11-08T08:19:08.501706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:55:07.041721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.497148+0000","last_change":"2023-11-08T08:19:04.488153+0000","last_active":"2023-11-08T08:19:08.497148+0000","last_peered":"2023-11-08T08:19:08.497148+0000","last_clean":"2023-11-08T08:19:08.497148+0000","last_became_active":"2023-11-08T08:19:04.487871+0000","last_became_peered":"2023-11-08T08:19:04.487871+0000","last_unstale":"2023-11-08T08:19:08.497148+0000","last_undegraded":"2023-11-08T08:19:08.497148+0000","last_fullsized":"2023-11-08T08:19:08.497148+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:06:47.395577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.494806+0000","last_change":"2023-11-08T08:19:04.011650+0000","last_active":"2023-11-08T08:19:08.494806+0000","last_peered":"2023-11-08T08:19:08.494806+0000","last_clean":"2023-11-08T08:19:08.494806+0000","last_became_active":"2023-11-08T08:19:04.011071+0000","last_became_peered":"2023-11-08T08:19:04.011071+0000","last_unstale":"2023-11-08T08:19:08.494806+0000","last_undegraded":"2023-11-08T08:19:08.494806+0000","last_fullsized":"2023-11-08T08:19:08.494806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:51:22.228427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519773+0000","last_change":"2023-11-08T08:19:04.505594+0000","last_active":"2023-11-08T08:19:08.519773+0000","last_peered":"2023-11-08T08:19:08.519773+0000","last_clean":"2023-11-08T08:19:08.519773+0000","last_became_active":"2023-11-08T08:19:04.505258+0000","last_became_peered":"2023-11-08T08:19:04.505258+0000","last_unstale":"2023-11-08T08:19:08.519773+0000","last_undegraded":"2023-11-08T08:19:08.519773+0000","last_fullsized":"2023-11-08T08:19:08.519773+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:56:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.373555+0000","last_change":"2023-11-08T08:19:08.529766+0000","last_active":"2023-11-08T08:19:09.373555+0000","last_peered":"2023-11-08T08:19:09.373555+0000","last_clean":"2023-11-08T08:19:09.373555+0000","last_became_active":"2023-11-08T08:19:08.529334+0000","last_became_peered":"2023-11-08T08:19:08.529334+0000","last_unstale":"2023-11-08T08:19:09.373555+0000","last_undegraded":"2023-11-08T08:19:09.373555+0000","last_fullsized":"2023-11-08T08:19:09.373555+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:28:49.369446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501076+0000","last_change":"2023-11-08T08:19:04.020027+0000","last_active":"2023-11-08T08:19:08.501076+0000","last_peered":"2023-11-08T08:19:08.501076+0000","last_clean":"2023-11-08T08:19:08.501076+0000","last_became_active":"2023-11-08T08:19:04.019727+0000","last_became_peered":"2023-11-08T08:19:04.019727+0000","last_unstale":"2023-11-08T08:19:08.501076+0000","last_undegraded":"2023-11-08T08:19:08.501076+0000","last_fullsized":"2023-11-08T08:19:08.501076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:18:14.269118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519487+0000","last_change":"2023-11-08T08:19:04.495283+0000","last_active":"2023-11-08T08:19:08.519487+0000","last_peered":"2023-11-08T08:19:08.519487+0000","last_clean":"2023-11-08T08:19:08.519487+0000","last_became_active":"2023-11-08T08:19:04.494955+0000","last_became_peered":"2023-11-08T08:19:04.494955+0000","last_unstale":"2023-11-08T08:19:08.519487+0000","last_undegraded":"2023-11-08T08:19:08.519487+0000","last_fullsized":"2023-11-08T08:19:08.519487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372061+0000","last_change":"2023-11-08T08:19:08.532374+0000","last_active":"2023-11-08T08:19:09.372061+0000","last_peered":"2023-11-08T08:19:09.372061+0000","last_clean":"2023-11-08T08:19:09.372061+0000","last_became_active":"2023-11-08T08:19:08.531953+0000","last_became_peered":"2023-11-08T08:19:08.531953+0000","last_unstale":"2023-11-08T08:19:09.372061+0000","last_undegraded":"2023-11-08T08:19:09.372061+0000","last_fullsized":"2023-11-08T08:19:09.372061+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:54:33.577798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518515+0000","last_change":"2023-11-08T08:19:04.491042+0000","last_active":"2023-11-08T08:19:08.518515+0000","last_peered":"2023-11-08T08:19:08.518515+0000","last_clean":"2023-11-08T08:19:08.518515+0000","last_became_active":"2023-11-08T08:19:04.490734+0000","last_became_peered":"2023-11-08T08:19:04.490734+0000","last_unstale":"2023-11-08T08:19:08.518515+0000","last_undegraded":"2023-11-08T08:19:08.518515+0000","last_fullsized":"2023-11-08T08:19:08.518515+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:26:27.887614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517935+0000","last_change":"2023-11-08T08:19:04.507871+0000","last_active":"2023-11-08T08:19:08.517935+0000","last_peered":"2023-11-08T08:19:08.517935+0000","last_clean":"2023-11-08T08:19:08.517935+0000","last_became_active":"2023-11-08T08:19:04.507600+0000","last_became_peered":"2023-11-08T08:19:04.507600+0000","last_unstale":"2023-11-08T08:19:08.517935+0000","last_undegraded":"2023-11-08T08:19:08.517935+0000","last_fullsized":"2023-11-08T08:19:08.517935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:55:43.865351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512136+0000","last_change":"2023-11-08T08:19:04.505023+0000","last_active":"2023-11-08T08:19:08.512136+0000","last_peered":"2023-11-08T08:19:08.512136+0000","last_clean":"2023-11-08T08:19:08.512136+0000","last_became_active":"2023-11-08T08:19:04.504750+0000","last_became_peered":"2023-11-08T08:19:04.504750+0000","last_unstale":"2023-11-08T08:19:08.512136+0000","last_undegraded":"2023-11-08T08:19:08.512136+0000","last_fullsized":"2023-11-08T08:19:08.512136+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:25.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518063+0000","last_change":"2023-11-08T08:19:04.504115+0000","last_active":"2023-11-08T08:19:08.518063+0000","last_peered":"2023-11-08T08:19:08.518063+0000","last_clean":"2023-11-08T08:19:08.518063+0000","last_became_active":"2023-11-08T08:19:04.503841+0000","last_became_peered":"2023-11-08T08:19:04.503841+0000","last_unstale":"2023-11-08T08:19:08.518063+0000","last_undegraded":"2023-11-08T08:19:08.518063+0000","last_fullsized":"2023-11-08T08:19:08.518063+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:33:12.861277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503562+0000","last_change":"2023-11-08T08:19:04.504534+0000","last_active":"2023-11-08T08:19:08.503562+0000","last_peered":"2023-11-08T08:19:08.503562+0000","last_clean":"2023-11-08T08:19:08.503562+0000","last_became_active":"2023-11-08T08:19:04.504283+0000","last_became_peered":"2023-11-08T08:19:04.504283+0000","last_unstale":"2023-11-08T08:19:08.503562+0000","last_undegraded":"2023-11-08T08:19:08.503562+0000","last_fullsized":"2023-11-08T08:19:08.503562+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:56:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519721+0000","last_change":"2023-11-08T08:19:04.503978+0000","last_active":"2023-11-08T08:19:08.519721+0000","last_peered":"2023-11-08T08:19:08.519721+0000","last_clean":"2023-11-08T08:19:08.519721+0000","last_became_active":"2023-11-08T08:19:04.503653+0000","last_became_peered":"2023-11-08T08:19:04.503653+0000","last_unstale":"2023-11-08T08:19:08.519721+0000","last_undegraded":"2023-11-08T08:19:08.519721+0000","last_fullsized":"2023-11-08T08:19:08.519721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503754+0000","last_change":"2023-11-08T08:19:04.491309+0000","last_active":"2023-11-08T08:19:08.503754+0000","last_peered":"2023-11-08T08:19:08.503754+0000","last_clean":"2023-11-08T08:19:08.503754+0000","last_became_active":"2023-11-08T08:19:04.491035+0000","last_became_peered":"2023-11-08T08:19:04.491035+0000","last_unstale":"2023-11-08T08:19:08.503754+0000","last_undegraded":"2023-11-08T08:19:08.503754+0000","last_fullsized":"2023-11-08T08:19:08.503754+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:18:36.796288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503455+0000","last_change":"2023-11-08T08:19:04.503092+0000","last_active":"2023-11-08T08:19:08.503455+0000","last_peered":"2023-11-08T08:19:08.503455+0000","last_clean":"2023-11-08T08:19:08.503455+0000","last_became_active":"2023-11-08T08:19:04.502751+0000","last_became_peered":"2023-11-08T08:19:04.502751+0000","last_unstale":"2023-11-08T08:19:08.503455+0000","last_undegraded":"2023-11-08T08:19:08.503455+0000","last_fullsized":"2023-11-08T08:19:08.503455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:40:44.425321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.499633+0000","last_change":"2023-11-08T08:19:04.509538+0000","last_active":"2023-11-08T08:19:08.499633+0000","last_peered":"2023-11-08T08:19:08.499633+0000","last_clean":"2023-11-08T08:19:08.499633+0000","last_became_active":"2023-11-08T08:19:04.509190+0000","last_became_peered":"2023-11-08T08:19:04.509190+0000","last_unstale":"2023-11-08T08:19:08.499633+0000","last_undegraded":"2023-11-08T08:19:08.499633+0000","last_fullsized":"2023-11-08T08:19:08.499633+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:04:55.302575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517870+0000","last_change":"2023-11-08T08:19:04.489238+0000","last_active":"2023-11-08T08:19:08.517870+0000","last_peered":"2023-11-08T08:19:08.517870+0000","last_clean":"2023-11-08T08:19:08.517870+0000","last_became_active":"2023-11-08T08:19:04.488998+0000","last_became_peered":"2023-11-08T08:19:04.488998+0000","last_unstale":"2023-11-08T08:19:08.517870+0000","last_undegraded":"2023-11-08T08:19:08.517870+0000","last_fullsized":"2023-11-08T08:19:08.517870+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:27:36.453040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.494671+0000","last_change":"2023-11-08T08:19:04.499236+0000","last_active":"2023-11-08T08:19:08.494671+0000","last_peered":"2023-11-08T08:19:08.494671+0000","last_clean":"2023-11-08T08:19:08.494671+0000","last_became_active":"2023-11-08T08:19:04.498978+0000","last_became_peered":"2023-11-08T08:19:04.498978+0000","last_unstale":"2023-11-08T08:19:08.494671+0000","last_undegraded":"2023-11-08T08:19:08.494671+0000","last_fullsized":"2023-11-08T08:19:08.494671+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:04:55.302575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.500995+0000","last_change":"2023-11-08T08:19:04.513191+0000","last_active":"2023-11-08T08:19:08.500995+0000","last_peered":"2023-11-08T08:19:08.500995+0000","last_clean":"2023-11-08T08:19:08.500995+0000","last_became_active":"2023-11-08T08:19:04.512519+0000","last_became_peered":"2023-11-08T08:19:04.512519+0000","last_unstale":"2023-11-08T08:19:08.500995+0000","last_undegraded":"2023-11-08T08:19:08.500995+0000","last_fullsized":"2023-11-08T08:19:08.500995+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:56:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513262+0000","last_change":"2023-11-08T08:19:04.510593+0000","last_active":"2023-11-08T08:19:08.513262+0000","last_peered":"2023-11-08T08:19:08.513262+0000","last_clean":"2023-11-08T08:19:08.513262+0000","last_became_active":"2023-11-08T08:19:04.501361+0000","last_became_peered":"2023-11-08T08:19:04.501361+0000","last_unstale":"2023-11-08T08:19:08.513262+0000","last_undegraded":"2023-11-08T08:19:08.513262+0000","last_fullsized":"2023-11-08T08:19:08.513262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.509488+0000","last_change":"2023-11-08T08:19:04.497954+0000","last_active":"2023-11-08T08:19:08.509488+0000","last_peered":"2023-11-08T08:19:08.509488+0000","last_clean":"2023-11-08T08:19:08.509488+0000","last_became_active":"2023-11-08T08:19:04.497675+0000","last_became_peered":"2023-11-08T08:19:04.497675+0000","last_unstale":"2023-11-08T08:19:08.509488+0000","last_undegraded":"2023-11-08T08:19:08.509488+0000","last_fullsized":"2023-11-08T08:19:08.509488+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:07:36.146057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512026+0000","last_change":"2023-11-08T08:19:04.494551+0000","last_active":"2023-11-08T08:19:08.512026+0000","last_peered":"2023-11-08T08:19:08.512026+0000","last_clean":"2023-11-08T08:19:08.512026+0000","last_became_active":"2023-11-08T08:19:04.494261+0000","last_became_peered":"2023-11-08T08:19:04.494261+0000","last_unstale":"2023-11-08T08:19:08.512026+0000","last_undegraded":"2023-11-08T08:19:08.512026+0000","last_fullsized":"2023-11-08T08:19:08.512026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:27:40.872420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.496353+0000","last_change":"2023-11-08T08:19:04.486999+0000","last_active":"2023-11-08T08:19:08.496353+0000","last_peered":"2023-11-08T08:19:08.496353+0000","last_clean":"2023-11-08T08:19:08.496353+0000","last_became_active":"2023-11-08T08:19:04.486746+0000","last_became_peered":"2023-11-08T08:19:04.486746+0000","last_unstale":"2023-11-08T08:19:08.496353+0000","last_undegraded":"2023-11-08T08:19:08.496353+0000","last_fullsized":"2023-11-08T08:19:08.496353+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:27:36.453040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513306+0000","last_change":"2023-11-08T08:19:04.506263+0000","last_active":"2023-11-08T08:19:08.513306+0000","last_peered":"2023-11-08T08:19:08.513306+0000","last_clean":"2023-11-08T08:19:08.513306+0000","last_became_active":"2023-11-08T08:19:04.505930+0000","last_became_peered":"2023-11-08T08:19:04.505930+0000","last_unstale":"2023-11-08T08:19:08.513306+0000","last_undegraded":"2023-11-08T08:19:08.513306+0000","last_fullsized":"2023-11-08T08:19:08.513306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:41:09.071125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501581+0000","last_change":"2023-11-08T08:19:04.514677+0000","last_active":"2023-11-08T08:19:08.501581+0000","last_peered":"2023-11-08T08:19:08.501581+0000","last_clean":"2023-11-08T08:19:08.501581+0000","last_became_active":"2023-11-08T08:19:04.511465+0000","last_became_peered":"2023-11-08T08:19:04.511465+0000","last_unstale":"2023-11-08T08:19:08.501581+0000","last_undegraded":"2023-11-08T08:19:08.501581+0000","last_fullsized":"2023-11-08T08:19:08.501581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513976+0000","last_change":"2023-11-08T08:19:04.490773+0000","last_active":"2023-11-08T08:19:08.513976+0000","last_peered":"2023-11-08T08:19:08.513976+0000","last_clean":"2023-11-08T08:19:08.513976+0000","last_became_active":"2023-11-08T08:19:04.490474+0000","last_became_peered":"2023-11-08T08:19:04.490474+0000","last_unstale":"2023-11-08T08:19:08.513976+0000","last_undegraded":"2023-11-08T08:19:08.513976+0000","last_fullsized":"2023-11-08T08:19:08.513976+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512799+0000","last_change":"2023-11-08T08:19:04.487159+0000","last_active":"2023-11-08T08:19:08.512799+0000","last_peered":"2023-11-08T08:19:08.512799+0000","last_clean":"2023-11-08T08:19:08.512799+0000","last_became_active":"2023-11-08T08:19:04.486861+0000","last_became_peered":"2023-11-08T08:19:04.486861+0000","last_unstale":"2023-11-08T08:19:08.512799+0000","last_undegraded":"2023-11-08T08:19:08.512799+0000","last_fullsized":"2023-11-08T08:19:08.512799+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:45.446278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.509301+0000","last_change":"2023-11-08T08:19:04.491757+0000","last_active":"2023-11-08T08:19:08.509301+0000","last_peered":"2023-11-08T08:19:08.509301+0000","last_clean":"2023-11-08T08:19:08.509301+0000","last_became_active":"2023-11-08T08:19:04.491359+0000","last_became_peered":"2023-11-08T08:19:04.491359+0000","last_unstale":"2023-11-08T08:19:08.509301+0000","last_undegraded":"2023-11-08T08:19:08.509301+0000","last_fullsized":"2023-11-08T08:19:08.509301+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:01:42.294189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.516256+0000","last_change":"2023-11-08T08:19:04.498748+0000","last_active":"2023-11-08T08:19:08.516256+0000","last_peered":"2023-11-08T08:19:08.516256+0000","last_clean":"2023-11-08T08:19:08.516256+0000","last_became_active":"2023-11-08T08:19:04.498208+0000","last_became_peered":"2023-11-08T08:19:04.498208+0000","last_unstale":"2023-11-08T08:19:08.516256+0000","last_undegraded":"2023-11-08T08:19:08.516256+0000","last_fullsized":"2023-11-08T08:19:08.516256+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:33:12.861277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521468+0000","last_change":"2023-11-08T08:19:04.490052+0000","last_active":"2023-11-08T08:19:08.521468+0000","last_peered":"2023-11-08T08:19:08.521468+0000","last_clean":"2023-11-08T08:19:08.521468+0000","last_became_active":"2023-11-08T08:19:04.489758+0000","last_became_peered":"2023-11-08T08:19:04.489758+0000","last_unstale":"2023-11-08T08:19:08.521468+0000","last_undegraded":"2023-11-08T08:19:08.521468+0000","last_fullsized":"2023-11-08T08:19:08.521468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:25.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.495092+0000","last_change":"2023-11-08T08:19:04.489823+0000","last_active":"2023-11-08T08:19:08.495092+0000","last_peered":"2023-11-08T08:19:08.495092+0000","last_clean":"2023-11-08T08:19:08.495092+0000","last_became_active":"2023-11-08T08:19:04.489559+0000","last_became_peered":"2023-11-08T08:19:04.489559+0000","last_unstale":"2023-11-08T08:19:08.495092+0000","last_undegraded":"2023-11-08T08:19:08.495092+0000","last_fullsized":"2023-11-08T08:19:08.495092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:25.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512940+0000","last_change":"2023-11-08T08:19:04.509192+0000","last_active":"2023-11-08T08:19:08.512940+0000","last_peered":"2023-11-08T08:19:08.512940+0000","last_clean":"2023-11-08T08:19:08.512940+0000","last_became_active":"2023-11-08T08:19:04.508856+0000","last_became_peered":"2023-11-08T08:19:04.508856+0000","last_unstale":"2023-11-08T08:19:08.512940+0000","last_undegraded":"2023-11-08T08:19:08.512940+0000","last_fullsized":"2023-11-08T08:19:08.512940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:27:36.453040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501455+0000","last_change":"2023-11-08T08:19:04.510274+0000","last_active":"2023-11-08T08:19:08.501455+0000","last_peered":"2023-11-08T08:19:08.501455+0000","last_clean":"2023-11-08T08:19:08.501455+0000","last_became_active":"2023-11-08T08:19:04.510052+0000","last_became_peered":"2023-11-08T08:19:04.510052+0000","last_unstale":"2023-11-08T08:19:08.501455+0000","last_undegraded":"2023-11-08T08:19:08.501455+0000","last_fullsized":"2023-11-08T08:19:08.501455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.511920+0000","last_change":"2023-11-08T08:19:04.507388+0000","last_active":"2023-11-08T08:19:08.511920+0000","last_peered":"2023-11-08T08:19:08.511920+0000","last_clean":"2023-11-08T08:19:08.511920+0000","last_became_active":"2023-11-08T08:19:04.506655+0000","last_became_peered":"2023-11-08T08:19:04.506655+0000","last_unstale":"2023-11-08T08:19:08.511920+0000","last_undegraded":"2023-11-08T08:19:08.511920+0000","last_fullsized":"2023-11-08T08:19:08.511920+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:26:16.737280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.495536+0000","last_change":"2023-11-08T08:19:04.484402+0000","last_active":"2023-11-08T08:19:08.495536+0000","last_peered":"2023-11-08T08:19:08.495536+0000","last_clean":"2023-11-08T08:19:08.495536+0000","last_became_active":"2023-11-08T08:19:04.480030+0000","last_became_peered":"2023-11-08T08:19:04.480030+0000","last_unstale":"2023-11-08T08:19:08.495536+0000","last_undegraded":"2023-11-08T08:19:08.495536+0000","last_fullsized":"2023-11-08T08:19:08.495536+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:27:40.872420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517912+0000","last_change":"2023-11-08T08:19:04.500239+0000","last_active":"2023-11-08T08:19:08.517912+0000","last_peered":"2023-11-08T08:19:08.517912+0000","last_clean":"2023-11-08T08:19:08.517912+0000","last_became_active":"2023-11-08T08:19:04.499933+0000","last_became_peered":"2023-11-08T08:19:04.499933+0000","last_unstale":"2023-11-08T08:19:08.517912+0000","last_undegraded":"2023-11-08T08:19:08.517912+0000","last_fullsized":"2023-11-08T08:19:08.517912+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:45.446278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.516154+0000","last_change":"2023-11-08T08:19:04.507285+0000","last_active":"2023-11-08T08:19:08.516154+0000","last_peered":"2023-11-08T08:19:08.516154+0000","last_clean":"2023-11-08T08:19:08.516154+0000","last_became_active":"2023-11-08T08:19:04.506999+0000","last_became_peered":"2023-11-08T08:19:04.506999+0000","last_unstale":"2023-11-08T08:19:08.516154+0000","last_undegraded":"2023-11-08T08:19:08.516154+0000","last_fullsized":"2023-11-08T08:19:08.516154+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:55:54.583805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521418+0000","last_change":"2023-11-08T08:19:04.496585+0000","last_active":"2023-11-08T08:19:08.521418+0000","last_peered":"2023-11-08T08:19:08.521418+0000","last_clean":"2023-11-08T08:19:08.521418+0000","last_became_active":"2023-11-08T08:19:04.496307+0000","last_became_peered":"2023-11-08T08:19:04.496307+0000","last_unstale":"2023-11-08T08:19:08.521418+0000","last_undegraded":"2023-11-08T08:19:08.521418+0000","last_fullsized":"2023-11-08T08:19:08.521418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:06:47.395577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.509596+0000","last_change":"2023-11-08T08:19:04.487836+0000","last_active":"2023-11-08T08:19:08.509596+0000","last_peered":"2023-11-08T08:19:08.509596+0000","last_clean":"2023-11-08T08:19:08.509596+0000","last_became_active":"2023-11-08T08:19:04.487548+0000","last_became_peered":"2023-11-08T08:19:04.487548+0000","last_unstale":"2023-11-08T08:19:08.509596+0000","last_undegraded":"2023-11-08T08:19:08.509596+0000","last_fullsized":"2023-11-08T08:19:08.509596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:55:08.054932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4527,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9296,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362544,"statfs":{"total":96636764160,"available":96627245056,"internally_reserved":0,"allocated":925696,"data_stored":718901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6949999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3080000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8176,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363664,"statfs":{"total":96636764160,"available":96628391936,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.464}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.258}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8184,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363656,"statfs":{"total":96636764160,"available":96628383744,"internally_reserved":0,"allocated":311296,"data_stored":125208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4790000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.532}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8260,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363580,"statfs":{"total":96636764160,"available":96628305920,"internally_reserved":0,"allocated":323584,"data_stored":128373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5169999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.579}]}]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8244,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363596,"statfs":{"total":96636764160,"available":96628322304,"internally_reserved":0,"allocated":307200,"data_stored":124662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5660000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.623}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8112,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363728,"statfs":{"total":96636764160,"available":96628457472,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4419999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.641}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8240,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363600,"statfs":{"total":96636764160,"available":96628326400,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.373}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4830000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9272,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362568,"statfs":{"total":96636764160,"available":96627269632,"internally_reserved":0,"allocated":901120,"data_stored":715506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.288}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T08:19:37.507 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T08:19:37.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.718+0000 7f22d38ee700 1 Processor -- start 2023-11-08T08:19:37.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.718+0000 7f22d38ee700 1 -- start start 2023-11-08T08:19:37.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.718+0000 7f22d38ee700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f22cc15c3b0 0x7f22cc0d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:37.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.718+0000 7f22d38ee700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f22cc0cb6e0 0x7f22cc06c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:37.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.718+0000 7f22d38ee700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22cc0cebb0 con 0x7f22cc06c040 2023-11-08T08:19:37.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.718+0000 7f22d38ee700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22cc0ce5f0 con 0x7f22cc0cb6e0 2023-11-08T08:19:37.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.718+0000 7f22d38ee700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22cc0ce8d0 con 0x7f22cc15c3b0 2023-11-08T08:19:37.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.718+0000 7f22d168a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f22cc15c3b0 0x7f22cc0d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:37.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.718+0000 7f22d1e8b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f22cc0cb6e0 0x7f22cc06c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:37.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.722+0000 7f22d168a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f22cc15c3b0 0x7f22cc0d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:45306/0 (socket says 172.21.15.40:45306) 2023-11-08T08:19:37.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.722+0000 7f22d168a700 1 -- 172.21.15.40:0/2660025143 learned_addr learned my addr 172.21.15.40:0/2660025143 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:37.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.722+0000 7f22d0e89700 1 --1- 172.21.15.40:0/2660025143 >> v1:172.21.15.169:6789/0 conn(0x7f22cc06c040 0x7f22cc0cafd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:43532/0 (socket says 172.21.15.40:43532) 2023-11-08T08:19:37.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.722+0000 7f22d168a700 1 -- 172.21.15.40:0/2660025143 >> v2:172.21.15.40:3300/0 conn(0x7f22cc0cb6e0 msgr2=0x7f22cc06c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:37.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.722+0000 7f22d168a700 1 --2- 172.21.15.40:0/2660025143 >> v2:172.21.15.40:3300/0 conn(0x7f22cc0cb6e0 0x7f22cc06c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.722+0000 7f22d168a700 1 -- 172.21.15.40:0/2660025143 >> v1:172.21.15.169:6789/0 conn(0x7f22cc06c040 legacy=0x7f22cc0cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:37.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.722+0000 7f22d168a700 1 -- 172.21.15.40:0/2660025143 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc06b840 con 0x7f22cc15c3b0 2023-11-08T08:19:37.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.722+0000 7f22d168a700 1 --2- 172.21.15.40:0/2660025143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22cc15c3b0 0x7f22cc0d00b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f22cc06f0a0 tx=0x7f22c800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35e9ca43b815fe42 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:37.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.722+0000 7f22c3fff700 1 -- 172.21.15.40:0/2660025143 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c800e070 con 0x7f22cc15c3b0 2023-11-08T08:19:37.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.722+0000 7f22c3fff700 1 -- 172.21.15.40:0/2660025143 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c800b910 con 0x7f22cc15c3b0 2023-11-08T08:19:37.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.722+0000 7f22c3fff700 1 -- 172.21.15.40:0/2660025143 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c800bcc0 con 0x7f22cc15c3b0 2023-11-08T08:19:37.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.722+0000 7f22d38ee700 1 -- 172.21.15.40:0/2660025143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22cc15c3b0 msgr2=0x7f22cc0d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:37.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.722+0000 7f22d38ee700 1 --2- 172.21.15.40:0/2660025143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22cc15c3b0 0x7f22cc0d00b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f22cc06f0a0 tx=0x7f22c800b330 comp rx=0 tx=0).stop 2023-11-08T08:19:37.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.722+0000 7f22d38ee700 1 -- 172.21.15.40:0/2660025143 shutdown_connections 2023-11-08T08:19:37.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.722+0000 7f22d38ee700 1 --2- 172.21.15.40:0/2660025143 >> v2:172.21.15.40:3300/0 conn(0x7f22cc0cb6e0 0x7f22cc06c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.722+0000 7f22d38ee700 1 --2- 172.21.15.40:0/2660025143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22cc15c3b0 0x7f22cc0d00b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.722+0000 7f22d38ee700 1 -- 172.21.15.40:0/2660025143 >> 172.21.15.40:0/2660025143 conn(0x7f22cc06e520 msgr2=0x7f22cc06e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:37.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.722+0000 7f22d38ee700 1 -- 172.21.15.40:0/2660025143 shutdown_connections 2023-11-08T08:19:37.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.722+0000 7f22d38ee700 1 -- 172.21.15.40:0/2660025143 wait complete. 2023-11-08T08:19:37.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.726+0000 7f22d38ee700 1 Processor -- start 2023-11-08T08:19:37.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.726+0000 7f22d38ee700 1 -- start start 2023-11-08T08:19:37.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.726+0000 7f22d38ee700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22cc06c040 0x7f22cc0b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:37.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.726+0000 7f22d38ee700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22cc0cb6e0 0x7f22cc0c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:37.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.726+0000 7f22d38ee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22cc15c3b0 0x7f22cc14a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:37.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.726+0000 7f22d38ee700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc0c4db0 con 0x7f22cc15c3b0 2023-11-08T08:19:37.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.726+0000 7f22d38ee700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc0c5090 con 0x7f22cc0cb6e0 2023-11-08T08:19:37.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.726+0000 7f22d38ee700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc0c5370 con 0x7f22cc06c040 2023-11-08T08:19:37.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.726+0000 7f22d1e8b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22cc15c3b0 0x7f22cc14a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:37.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.726+0000 7f22d1e8b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22cc15c3b0 0x7f22cc14a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55432/0 (socket says 172.21.15.40:55432) 2023-11-08T08:19:37.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.726+0000 7f22d1e8b700 1 -- 172.21.15.40:0/1755537621 learned_addr learned my addr 172.21.15.40:0/1755537621 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:37.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.726+0000 7f22d0e89700 1 --2- 172.21.15.40:0/1755537621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22cc0cb6e0 0x7f22cc0c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:37.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.726+0000 7f22d168a700 1 --2- 172.21.15.40:0/1755537621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22cc06c040 0x7f22cc0b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:37.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.730+0000 7f22d0e89700 1 -- 172.21.15.40:0/1755537621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22cc06c040 msgr2=0x7f22cc0b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:37.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.730+0000 7f22d0e89700 1 --2- 172.21.15.40:0/1755537621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22cc06c040 0x7f22cc0b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.730+0000 7f22d0e89700 1 -- 172.21.15.40:0/1755537621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22cc15c3b0 msgr2=0x7f22cc14a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:37.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.730+0000 7f22d0e89700 1 --2- 172.21.15.40:0/1755537621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22cc15c3b0 0x7f22cc14a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.730+0000 7f22d0e89700 1 -- 172.21.15.40:0/1755537621 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c8009b30 con 0x7f22cc0cb6e0 2023-11-08T08:19:37.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.730+0000 7f22d0e89700 1 --2- 172.21.15.40:0/1755537621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22cc0cb6e0 0x7f22cc0c4310 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00c500 tx=0x7f22bc00c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:37.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.730+0000 7f22c1ffb700 1 -- 172.21.15.40:0/1755537621 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc005cd0 con 0x7f22cc0cb6e0 2023-11-08T08:19:37.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.730+0000 7f22c1ffb700 1 -- 172.21.15.40:0/1755537621 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22bc005e30 con 0x7f22cc0cb6e0 2023-11-08T08:19:37.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.730+0000 7f22d38ee700 1 -- 172.21.15.40:0/1755537621 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22cc0d05f0 con 0x7f22cc0cb6e0 2023-11-08T08:19:37.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.730+0000 7f22c1ffb700 1 -- 172.21.15.40:0/1755537621 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc0072d0 con 0x7f22cc0cb6e0 2023-11-08T08:19:37.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.730+0000 7f22d38ee700 1 -- 172.21.15.40:0/1755537621 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22cc06bb20 con 0x7f22cc0cb6e0 2023-11-08T08:19:37.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.730+0000 7f22d38ee700 1 -- 172.21.15.40:0/1755537621 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22cc15fe30 con 0x7f22cc0cb6e0 2023-11-08T08:19:37.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.734+0000 7f22c1ffb700 1 -- 172.21.15.40:0/1755537621 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f22bc019030 con 0x7f22cc0cb6e0 2023-11-08T08:19:37.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.734+0000 7f22c1ffb700 1 --2- 172.21.15.40:0/1755537621 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f22b0064ee0 0x7f22b0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:37.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.734+0000 7f22c1ffb700 1 -- 172.21.15.40:0/1755537621 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f22bc0813c0 con 0x7f22cc0cb6e0 2023-11-08T08:19:37.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.734+0000 7f22d168a700 1 --2- 172.21.15.40:0/1755537621 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f22b0064ee0 0x7f22b0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:37.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.734+0000 7f22d168a700 1 --2- 172.21.15.40:0/1755537621 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f22b0064ee0 0x7f22b0067390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f22c8000f80 tx=0x7f22c8009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:37.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.742+0000 7f22c1ffb700 1 -- 172.21.15.40:0/1755537621 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f22bc04a400 con 0x7f22cc0cb6e0 2023-11-08T08:19:37.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.910+0000 7f22d38ee700 1 -- 172.21.15.40:0/1755537621 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22cc0726c0 con 0x7f22b0064ee0 2023-11-08T08:19:37.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.930+0000 7f22c1ffb700 1 -- 172.21.15.40:0/1755537621 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367138 (secure 0 0 0) 0x7f22cc0726c0 con 0x7f22b0064ee0 2023-11-08T08:19:37.942 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:37.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.938+0000 7f22d38ee700 1 -- 172.21.15.40:0/1755537621 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f22b0064ee0 msgr2=0x7f22b0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:37.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.938+0000 7f22d38ee700 1 --2- 172.21.15.40:0/1755537621 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f22b0064ee0 0x7f22b0067390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f22c8000f80 tx=0x7f22c8009000 comp rx=0 tx=0).stop 2023-11-08T08:19:37.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.938+0000 7f22d38ee700 1 -- 172.21.15.40:0/1755537621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22cc0cb6e0 msgr2=0x7f22cc0c4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:37.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.938+0000 7f22d38ee700 1 --2- 172.21.15.40:0/1755537621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22cc0cb6e0 0x7f22cc0c4310 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00c500 tx=0x7f22bc00c810 comp rx=0 tx=0).stop 2023-11-08T08:19:37.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.942+0000 7f22d38ee700 1 -- 172.21.15.40:0/1755537621 shutdown_connections 2023-11-08T08:19:37.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.942+0000 7f22d38ee700 1 --2- 172.21.15.40:0/1755537621 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f22b0064ee0 0x7f22b0067390 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.942+0000 7f22d38ee700 1 --2- 172.21.15.40:0/1755537621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22cc15c3b0 0x7f22cc14a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.942+0000 7f22d38ee700 1 --2- 172.21.15.40:0/1755537621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22cc0cb6e0 0x7f22cc0c4310 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.942+0000 7f22d38ee700 1 --2- 172.21.15.40:0/1755537621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22cc06c040 0x7f22cc0b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:37.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.942+0000 7f22d38ee700 1 -- 172.21.15.40:0/1755537621 >> 172.21.15.40:0/1755537621 conn(0x7f22cc06e520 msgr2=0x7f22cc0b4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:37.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.942+0000 7f22d38ee700 1 -- 172.21.15.40:0/1755537621 shutdown_connections 2023-11-08T08:19:37.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:37.942+0000 7f22d38ee700 1 -- 172.21.15.40:0/1755537621 wait complete. 2023-11-08T08:19:37.947 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-11-08T08:19:37.967 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":45,"stamp":"2023-11-08T08:19:37.892775+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594895,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67784,"kb_used_data":3592,"kb_used_omap":0,"kb_used_meta":64128,"kb_avail":754906936,"statfs":{"total":773094113280,"available":773024702464,"internally_reserved":0,"allocated":3678208,"data_stored":2186366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65667072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005441"},"pg_stats":[{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.370829+0000","last_change":"2023-11-08T08:19:08.519864+0000","last_active":"2023-11-08T08:19:09.370829+0000","last_peered":"2023-11-08T08:19:09.370829+0000","last_clean":"2023-11-08T08:19:09.370829+0000","last_became_active":"2023-11-08T08:19:08.518845+0000","last_became_peered":"2023-11-08T08:19:08.518845+0000","last_unstale":"2023-11-08T08:19:09.370829+0000","last_undegraded":"2023-11-08T08:19:09.370829+0000","last_fullsized":"2023-11-08T08:19:09.370829+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:47:58.348898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.496570+0000","last_change":"2023-11-08T08:19:04.015635+0000","last_active":"2023-11-08T08:19:08.496570+0000","last_peered":"2023-11-08T08:19:08.496570+0000","last_clean":"2023-11-08T08:19:08.496570+0000","last_became_active":"2023-11-08T08:19:04.014748+0000","last_became_peered":"2023-11-08T08:19:04.014748+0000","last_unstale":"2023-11-08T08:19:08.496570+0000","last_undegraded":"2023-11-08T08:19:08.496570+0000","last_fullsized":"2023-11-08T08:19:08.496570+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:01:41.280977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513402+0000","last_change":"2023-11-08T08:19:04.505994+0000","last_active":"2023-11-08T08:19:08.513402+0000","last_peered":"2023-11-08T08:19:08.513402+0000","last_clean":"2023-11-08T08:19:08.513402+0000","last_became_active":"2023-11-08T08:19:04.505596+0000","last_became_peered":"2023-11-08T08:19:04.505596+0000","last_unstale":"2023-11-08T08:19:08.513402+0000","last_undegraded":"2023-11-08T08:19:08.513402+0000","last_fullsized":"2023-11-08T08:19:08.513402+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:06:47.395577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.992842+0000","last_change":"2023-11-08T08:19:08.526866+0000","last_active":"2023-11-08T08:19:08.992842+0000","last_peered":"2023-11-08T08:19:08.992842+0000","last_clean":"2023-11-08T08:19:08.992842+0000","last_became_active":"2023-11-08T08:19:08.526405+0000","last_became_peered":"2023-11-08T08:19:08.526405+0000","last_unstale":"2023-11-08T08:19:08.992842+0000","last_undegraded":"2023-11-08T08:19:08.992842+0000","last_fullsized":"2023-11-08T08:19:08.992842+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:54:12.699998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519574+0000","last_change":"2023-11-08T08:19:04.021634+0000","last_active":"2023-11-08T08:19:08.519574+0000","last_peered":"2023-11-08T08:19:08.519574+0000","last_clean":"2023-11-08T08:19:08.519574+0000","last_became_active":"2023-11-08T08:19:04.017916+0000","last_became_peered":"2023-11-08T08:19:04.017916+0000","last_unstale":"2023-11-08T08:19:08.519574+0000","last_undegraded":"2023-11-08T08:19:08.519574+0000","last_fullsized":"2023-11-08T08:19:08.519574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:58:24.706350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518078+0000","last_change":"2023-11-08T08:19:04.481276+0000","last_active":"2023-11-08T08:19:08.518078+0000","last_peered":"2023-11-08T08:19:08.518078+0000","last_clean":"2023-11-08T08:19:08.518078+0000","last_became_active":"2023-11-08T08:19:04.480340+0000","last_became_peered":"2023-11-08T08:19:04.480340+0000","last_unstale":"2023-11-08T08:19:08.518078+0000","last_undegraded":"2023-11-08T08:19:08.518078+0000","last_fullsized":"2023-11-08T08:19:08.518078+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:04:55.302575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372099+0000","last_change":"2023-11-08T08:19:08.528726+0000","last_active":"2023-11-08T08:19:09.372099+0000","last_peered":"2023-11-08T08:19:09.372099+0000","last_clean":"2023-11-08T08:19:09.372099+0000","last_became_active":"2023-11-08T08:19:08.527853+0000","last_became_peered":"2023-11-08T08:19:08.527853+0000","last_unstale":"2023-11-08T08:19:09.372099+0000","last_undegraded":"2023-11-08T08:19:09.372099+0000","last_fullsized":"2023-11-08T08:19:09.372099+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:16:14.032597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519819+0000","last_change":"2023-11-08T08:19:04.025513+0000","last_active":"2023-11-08T08:19:08.519819+0000","last_peered":"2023-11-08T08:19:08.519819+0000","last_clean":"2023-11-08T08:19:08.519819+0000","last_became_active":"2023-11-08T08:19:04.025220+0000","last_became_peered":"2023-11-08T08:19:04.025220+0000","last_unstale":"2023-11-08T08:19:08.519819+0000","last_undegraded":"2023-11-08T08:19:08.519819+0000","last_fullsized":"2023-11-08T08:19:08.519819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:55:07.041721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512027+0000","last_change":"2023-11-08T08:19:04.512364+0000","last_active":"2023-11-08T08:19:08.512027+0000","last_peered":"2023-11-08T08:19:08.512027+0000","last_clean":"2023-11-08T08:19:08.512027+0000","last_became_active":"2023-11-08T08:19:04.511958+0000","last_became_peered":"2023-11-08T08:19:04.511958+0000","last_unstale":"2023-11-08T08:19:08.512027+0000","last_undegraded":"2023-11-08T08:19:08.512027+0000","last_fullsized":"2023-11-08T08:19:08.512027+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:32:57.245760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.994907+0000","last_change":"2023-11-08T08:19:08.527583+0000","last_active":"2023-11-08T08:19:08.994907+0000","last_peered":"2023-11-08T08:19:08.994907+0000","last_clean":"2023-11-08T08:19:08.994907+0000","last_became_active":"2023-11-08T08:19:08.527054+0000","last_became_peered":"2023-11-08T08:19:08.527054+0000","last_unstale":"2023-11-08T08:19:08.994907+0000","last_undegraded":"2023-11-08T08:19:08.994907+0000","last_fullsized":"2023-11-08T08:19:08.994907+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:08.797102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519426+0000","last_change":"2023-11-08T08:19:04.025416+0000","last_active":"2023-11-08T08:19:08.519426+0000","last_peered":"2023-11-08T08:19:08.519426+0000","last_clean":"2023-11-08T08:19:08.519426+0000","last_became_active":"2023-11-08T08:19:04.025091+0000","last_became_peered":"2023-11-08T08:19:04.025091+0000","last_unstale":"2023-11-08T08:19:08.519426+0000","last_undegraded":"2023-11-08T08:19:08.519426+0000","last_fullsized":"2023-11-08T08:19:08.519426+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:07:35.132845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.511963+0000","last_change":"2023-11-08T08:19:04.512794+0000","last_active":"2023-11-08T08:19:08.511963+0000","last_peered":"2023-11-08T08:19:08.511963+0000","last_clean":"2023-11-08T08:19:08.511963+0000","last_became_active":"2023-11-08T08:19:04.512499+0000","last_became_peered":"2023-11-08T08:19:04.512499+0000","last_unstale":"2023-11-08T08:19:08.511963+0000","last_undegraded":"2023-11-08T08:19:08.511963+0000","last_fullsized":"2023-11-08T08:19:08.511963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:05:16.516877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372287+0000","last_change":"2023-11-08T08:19:08.539289+0000","last_active":"2023-11-08T08:19:09.372287+0000","last_peered":"2023-11-08T08:19:09.372287+0000","last_clean":"2023-11-08T08:19:09.372287+0000","last_became_active":"2023-11-08T08:19:08.532011+0000","last_became_peered":"2023-11-08T08:19:08.532011+0000","last_unstale":"2023-11-08T08:19:09.372287+0000","last_undegraded":"2023-11-08T08:19:09.372287+0000","last_fullsized":"2023-11-08T08:19:09.372287+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:56:38.412909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.495705+0000","last_change":"2023-11-08T08:19:04.024246+0000","last_active":"2023-11-08T08:19:08.495705+0000","last_peered":"2023-11-08T08:19:08.495705+0000","last_clean":"2023-11-08T08:19:08.495705+0000","last_became_active":"2023-11-08T08:19:04.023681+0000","last_became_peered":"2023-11-08T08:19:04.023681+0000","last_unstale":"2023-11-08T08:19:08.495705+0000","last_undegraded":"2023-11-08T08:19:08.495705+0000","last_fullsized":"2023-11-08T08:19:08.495705+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:50:41.342112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.509382+0000","last_change":"2023-11-08T08:19:04.494535+0000","last_active":"2023-11-08T08:19:08.509382+0000","last_peered":"2023-11-08T08:19:08.509382+0000","last_clean":"2023-11-08T08:19:08.509382+0000","last_became_active":"2023-11-08T08:19:04.494226+0000","last_became_peered":"2023-11-08T08:19:04.494226+0000","last_unstale":"2023-11-08T08:19:08.509382+0000","last_undegraded":"2023-11-08T08:19:08.509382+0000","last_fullsized":"2023-11-08T08:19:08.509382+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:18:15.282329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.993497+0000","last_change":"2023-11-08T08:19:08.532723+0000","last_active":"2023-11-08T08:19:08.993497+0000","last_peered":"2023-11-08T08:19:08.993497+0000","last_clean":"2023-11-08T08:19:08.993497+0000","last_became_active":"2023-11-08T08:19:08.532320+0000","last_became_peered":"2023-11-08T08:19:08.532320+0000","last_unstale":"2023-11-08T08:19:08.993497+0000","last_undegraded":"2023-11-08T08:19:08.993497+0000","last_fullsized":"2023-11-08T08:19:08.993497+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:10:25.517705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512234+0000","last_change":"2023-11-08T08:19:04.020285+0000","last_active":"2023-11-08T08:19:08.512234+0000","last_peered":"2023-11-08T08:19:08.512234+0000","last_clean":"2023-11-08T08:19:08.512234+0000","last_became_active":"2023-11-08T08:19:04.019791+0000","last_became_peered":"2023-11-08T08:19:04.019791+0000","last_unstale":"2023-11-08T08:19:08.512234+0000","last_undegraded":"2023-11-08T08:19:08.512234+0000","last_fullsized":"2023-11-08T08:19:08.512234+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:21:09.218618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521337+0000","last_change":"2023-11-08T08:19:04.497059+0000","last_active":"2023-11-08T08:19:08.521337+0000","last_peered":"2023-11-08T08:19:08.521337+0000","last_clean":"2023-11-08T08:19:08.521337+0000","last_became_active":"2023-11-08T08:19:04.495199+0000","last_became_peered":"2023-11-08T08:19:04.495199+0000","last_unstale":"2023-11-08T08:19:08.521337+0000","last_undegraded":"2023-11-08T08:19:08.521337+0000","last_fullsized":"2023-11-08T08:19:08.521337+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:56:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.374791+0000","last_change":"2023-11-08T08:19:08.530445+0000","last_active":"2023-11-08T08:19:09.374791+0000","last_peered":"2023-11-08T08:19:09.374791+0000","last_clean":"2023-11-08T08:19:09.374791+0000","last_became_active":"2023-11-08T08:19:08.530020+0000","last_became_peered":"2023-11-08T08:19:08.530020+0000","last_unstale":"2023-11-08T08:19:09.374791+0000","last_undegraded":"2023-11-08T08:19:09.374791+0000","last_fullsized":"2023-11-08T08:19:09.374791+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:24:13.984326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.502373+0000","last_change":"2023-11-08T08:19:04.018882+0000","last_active":"2023-11-08T08:19:08.502373+0000","last_peered":"2023-11-08T08:19:08.502373+0000","last_clean":"2023-11-08T08:19:08.502373+0000","last_became_active":"2023-11-08T08:19:04.018537+0000","last_became_peered":"2023-11-08T08:19:04.018537+0000","last_unstale":"2023-11-08T08:19:08.502373+0000","last_undegraded":"2023-11-08T08:19:08.502373+0000","last_fullsized":"2023-11-08T08:19:08.502373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:51:22.228427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.496833+0000","last_change":"2023-11-08T08:19:04.503358+0000","last_active":"2023-11-08T08:19:08.496833+0000","last_peered":"2023-11-08T08:19:08.496833+0000","last_clean":"2023-11-08T08:19:08.496833+0000","last_became_active":"2023-11-08T08:19:04.502856+0000","last_became_peered":"2023-11-08T08:19:04.502856+0000","last_unstale":"2023-11-08T08:19:08.496833+0000","last_undegraded":"2023-11-08T08:19:08.496833+0000","last_fullsized":"2023-11-08T08:19:08.496833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.989197+0000","last_change":"2023-11-08T08:19:08.531064+0000","last_active":"2023-11-08T08:19:08.989197+0000","last_peered":"2023-11-08T08:19:08.989197+0000","last_clean":"2023-11-08T08:19:08.989197+0000","last_became_active":"2023-11-08T08:19:08.530517+0000","last_became_peered":"2023-11-08T08:19:08.530517+0000","last_unstale":"2023-11-08T08:19:08.989197+0000","last_undegraded":"2023-11-08T08:19:08.989197+0000","last_fullsized":"2023-11-08T08:19:08.989197+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:51:11.743430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517381+0000","last_change":"2023-11-08T08:19:04.022640+0000","last_active":"2023-11-08T08:19:08.517381+0000","last_peered":"2023-11-08T08:19:08.517381+0000","last_clean":"2023-11-08T08:19:08.517381+0000","last_became_active":"2023-11-08T08:19:04.022244+0000","last_became_peered":"2023-11-08T08:19:04.022244+0000","last_unstale":"2023-11-08T08:19:08.517381+0000","last_undegraded":"2023-11-08T08:19:08.517381+0000","last_fullsized":"2023-11-08T08:19:08.517381+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:50:41.342112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521762+0000","last_change":"2023-11-08T08:19:04.497516+0000","last_active":"2023-11-08T08:19:08.521762+0000","last_peered":"2023-11-08T08:19:08.521762+0000","last_clean":"2023-11-08T08:19:08.521762+0000","last_became_active":"2023-11-08T08:19:04.493572+0000","last_became_peered":"2023-11-08T08:19:04.493572+0000","last_unstale":"2023-11-08T08:19:08.521762+0000","last_undegraded":"2023-11-08T08:19:08.521762+0000","last_fullsized":"2023-11-08T08:19:08.521762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:57:06.487072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517281+0000","last_change":"2023-11-08T08:19:04.495610+0000","last_active":"2023-11-08T08:19:08.517281+0000","last_peered":"2023-11-08T08:19:08.517281+0000","last_clean":"2023-11-08T08:19:08.517281+0000","last_became_active":"2023-11-08T08:19:04.495307+0000","last_became_peered":"2023-11-08T08:19:04.495307+0000","last_unstale":"2023-11-08T08:19:08.517281+0000","last_undegraded":"2023-11-08T08:19:08.517281+0000","last_fullsized":"2023-11-08T08:19:08.517281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.496569+0000","last_change":"2023-11-08T08:19:04.015753+0000","last_active":"2023-11-08T08:19:08.496569+0000","last_peered":"2023-11-08T08:19:08.496569+0000","last_clean":"2023-11-08T08:19:08.496569+0000","last_became_active":"2023-11-08T08:19:04.015173+0000","last_became_peered":"2023-11-08T08:19:04.015173+0000","last_unstale":"2023-11-08T08:19:08.496569+0000","last_undegraded":"2023-11-08T08:19:08.496569+0000","last_fullsized":"2023-11-08T08:19:08.496569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:58:24.706350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.373895+0000","last_change":"2023-11-08T08:19:08.525297+0000","last_active":"2023-11-08T08:19:09.373895+0000","last_peered":"2023-11-08T08:19:09.373895+0000","last_clean":"2023-11-08T08:19:09.373895+0000","last_became_active":"2023-11-08T08:19:08.524571+0000","last_became_peered":"2023-11-08T08:19:08.524571+0000","last_unstale":"2023-11-08T08:19:09.373895+0000","last_undegraded":"2023-11-08T08:19:09.373895+0000","last_fullsized":"2023-11-08T08:19:09.373895+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:33:00.754113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517964+0000","last_change":"2023-11-08T08:19:04.500363+0000","last_active":"2023-11-08T08:19:08.517964+0000","last_peered":"2023-11-08T08:19:08.517964+0000","last_clean":"2023-11-08T08:19:08.517964+0000","last_became_active":"2023-11-08T08:19:04.500049+0000","last_became_peered":"2023-11-08T08:19:04.500049+0000","last_unstale":"2023-11-08T08:19:08.517964+0000","last_undegraded":"2023-11-08T08:19:08.517964+0000","last_fullsized":"2023-11-08T08:19:08.517964+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:06:47.395577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521546+0000","last_change":"2023-11-08T08:19:04.020555+0000","last_active":"2023-11-08T08:19:08.521546+0000","last_peered":"2023-11-08T08:19:08.521546+0000","last_clean":"2023-11-08T08:19:08.521546+0000","last_became_active":"2023-11-08T08:19:04.020053+0000","last_became_peered":"2023-11-08T08:19:04.020053+0000","last_unstale":"2023-11-08T08:19:08.521546+0000","last_undegraded":"2023-11-08T08:19:08.521546+0000","last_fullsized":"2023-11-08T08:19:08.521546+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:01:41.280977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.994150+0000","last_change":"2023-11-08T08:19:08.526638+0000","last_active":"2023-11-08T08:19:08.994150+0000","last_peered":"2023-11-08T08:19:08.994150+0000","last_clean":"2023-11-08T08:19:08.994150+0000","last_became_active":"2023-11-08T08:19:08.526219+0000","last_became_peered":"2023-11-08T08:19:08.526219+0000","last_unstale":"2023-11-08T08:19:08.994150+0000","last_undegraded":"2023-11-08T08:19:08.994150+0000","last_fullsized":"2023-11-08T08:19:08.994150+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:56:38.412909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.500591+0000","last_change":"2023-11-08T08:19:04.022271+0000","last_active":"2023-11-08T08:19:08.500591+0000","last_peered":"2023-11-08T08:19:08.500591+0000","last_clean":"2023-11-08T08:19:08.500591+0000","last_became_active":"2023-11-08T08:19:04.021306+0000","last_became_peered":"2023-11-08T08:19:04.021306+0000","last_unstale":"2023-11-08T08:19:08.500591+0000","last_undegraded":"2023-11-08T08:19:08.500591+0000","last_fullsized":"2023-11-08T08:19:08.500591+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:26:26.874402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.502798+0000","last_change":"2023-11-08T08:19:04.494816+0000","last_active":"2023-11-08T08:19:08.502798+0000","last_peered":"2023-11-08T08:19:08.502798+0000","last_clean":"2023-11-08T08:19:08.502798+0000","last_became_active":"2023-11-08T08:19:04.494321+0000","last_became_peered":"2023-11-08T08:19:04.494321+0000","last_unstale":"2023-11-08T08:19:08.502798+0000","last_undegraded":"2023-11-08T08:19:08.502798+0000","last_fullsized":"2023-11-08T08:19:08.502798+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:04:55.302575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372527+0000","last_change":"2023-11-08T08:19:08.542779+0000","last_active":"2023-11-08T08:19:09.372527+0000","last_peered":"2023-11-08T08:19:09.372527+0000","last_clean":"2023-11-08T08:19:09.372527+0000","last_became_active":"2023-11-08T08:19:08.542399+0000","last_became_peered":"2023-11-08T08:19:08.542399+0000","last_unstale":"2023-11-08T08:19:09.372527+0000","last_undegraded":"2023-11-08T08:19:09.372527+0000","last_fullsized":"2023-11-08T08:19:09.372527+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:16:14.032597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:21.997708+0000","last_change":"2023-11-08T08:19:04.018152+0000","last_active":"2023-11-08T08:19:21.997708+0000","last_peered":"2023-11-08T08:19:21.997708+0000","last_clean":"2023-11-08T08:19:21.997708+0000","last_became_active":"2023-11-08T08:19:04.017695+0000","last_became_peered":"2023-11-08T08:19:04.017695+0000","last_unstale":"2023-11-08T08:19:21.997708+0000","last_undegraded":"2023-11-08T08:19:21.997708+0000","last_fullsized":"2023-11-08T08:19:21.997708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:55:20.782396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3745,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512046+0000","last_change":"2023-11-08T08:19:04.501576+0000","last_active":"2023-11-08T08:19:08.512046+0000","last_peered":"2023-11-08T08:19:08.512046+0000","last_clean":"2023-11-08T08:19:08.512046+0000","last_became_active":"2023-11-08T08:19:04.500829+0000","last_became_peered":"2023-11-08T08:19:04.500829+0000","last_unstale":"2023-11-08T08:19:08.512046+0000","last_undegraded":"2023-11-08T08:19:08.512046+0000","last_fullsized":"2023-11-08T08:19:08.512046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372324+0000","last_change":"2023-11-08T08:19:08.536704+0000","last_active":"2023-11-08T08:19:09.372324+0000","last_peered":"2023-11-08T08:19:09.372324+0000","last_clean":"2023-11-08T08:19:09.372324+0000","last_became_active":"2023-11-08T08:19:08.536339+0000","last_became_peered":"2023-11-08T08:19:08.536339+0000","last_unstale":"2023-11-08T08:19:09.372324+0000","last_undegraded":"2023-11-08T08:19:09.372324+0000","last_fullsized":"2023-11-08T08:19:09.372324+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:51:11.743430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513610+0000","last_change":"2023-11-08T08:19:04.496877+0000","last_active":"2023-11-08T08:19:08.513610+0000","last_peered":"2023-11-08T08:19:08.513610+0000","last_clean":"2023-11-08T08:19:08.513610+0000","last_became_active":"2023-11-08T08:19:04.496505+0000","last_became_peered":"2023-11-08T08:19:04.496505+0000","last_unstale":"2023-11-08T08:19:08.513610+0000","last_undegraded":"2023-11-08T08:19:08.513610+0000","last_fullsized":"2023-11-08T08:19:08.513610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:59:35.582106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521817+0000","last_change":"2023-11-08T08:18:58.771310+0000","last_active":"2023-11-08T08:19:08.521817+0000","last_peered":"2023-11-08T08:19:08.521817+0000","last_clean":"2023-11-08T08:19:08.521817+0000","last_became_active":"2023-11-08T08:18:58.770891+0000","last_became_peered":"2023-11-08T08:18:58.770891+0000","last_unstale":"2023-11-08T08:19:08.521817+0000","last_undegraded":"2023-11-08T08:19:08.521817+0000","last_fullsized":"2023-11-08T08:19:08.521817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"27'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.969187+0000","last_change":"2023-11-08T08:19:04.012926+0000","last_active":"2023-11-08T08:19:08.969187+0000","last_peered":"2023-11-08T08:19:08.969187+0000","last_clean":"2023-11-08T08:19:08.969187+0000","last_became_active":"2023-11-08T08:19:04.012634+0000","last_became_peered":"2023-11-08T08:19:04.012634+0000","last_unstale":"2023-11-08T08:19:08.969187+0000","last_undegraded":"2023-11-08T08:19:08.969187+0000","last_fullsized":"2023-11-08T08:19:08.969187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:07:35.132845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.371641+0000","last_change":"2023-11-08T08:19:08.532617+0000","last_active":"2023-11-08T08:19:09.371641+0000","last_peered":"2023-11-08T08:19:09.371641+0000","last_clean":"2023-11-08T08:19:09.371641+0000","last_became_active":"2023-11-08T08:19:08.532177+0000","last_became_peered":"2023-11-08T08:19:08.532177+0000","last_unstale":"2023-11-08T08:19:09.371641+0000","last_undegraded":"2023-11-08T08:19:09.371641+0000","last_fullsized":"2023-11-08T08:19:09.371641+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:08.797102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517321+0000","last_change":"2023-11-08T08:19:04.504132+0000","last_active":"2023-11-08T08:19:08.517321+0000","last_peered":"2023-11-08T08:19:08.517321+0000","last_clean":"2023-11-08T08:19:08.517321+0000","last_became_active":"2023-11-08T08:19:04.503747+0000","last_became_peered":"2023-11-08T08:19:04.503747+0000","last_unstale":"2023-11-08T08:19:08.517321+0000","last_undegraded":"2023-11-08T08:19:08.517321+0000","last_fullsized":"2023-11-08T08:19:08.517321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:27:40.872420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503547+0000","last_change":"2023-11-08T08:18:58.770483+0000","last_active":"2023-11-08T08:19:08.503547+0000","last_peered":"2023-11-08T08:19:08.503547+0000","last_clean":"2023-11-08T08:19:08.503547+0000","last_became_active":"2023-11-08T08:18:58.770040+0000","last_became_peered":"2023-11-08T08:18:58.770040+0000","last_unstale":"2023-11-08T08:19:08.503547+0000","last_undegraded":"2023-11-08T08:19:08.503547+0000","last_fullsized":"2023-11-08T08:19:08.503547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.520152+0000","last_change":"2023-11-08T08:19:04.022667+0000","last_active":"2023-11-08T08:19:08.520152+0000","last_peered":"2023-11-08T08:19:08.520152+0000","last_clean":"2023-11-08T08:19:08.520152+0000","last_became_active":"2023-11-08T08:19:04.022430+0000","last_became_peered":"2023-11-08T08:19:04.022430+0000","last_unstale":"2023-11-08T08:19:08.520152+0000","last_undegraded":"2023-11-08T08:19:08.520152+0000","last_fullsized":"2023-11-08T08:19:08.520152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:15:32.560862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.992931+0000","last_change":"2023-11-08T08:19:08.531983+0000","last_active":"2023-11-08T08:19:08.992931+0000","last_peered":"2023-11-08T08:19:08.992931+0000","last_clean":"2023-11-08T08:19:08.992931+0000","last_became_active":"2023-11-08T08:19:08.531584+0000","last_became_peered":"2023-11-08T08:19:08.531584+0000","last_unstale":"2023-11-08T08:19:08.992931+0000","last_undegraded":"2023-11-08T08:19:08.992931+0000","last_fullsized":"2023-11-08T08:19:08.992931+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:56:34.397672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.495387+0000","last_change":"2023-11-08T08:19:04.481283+0000","last_active":"2023-11-08T08:19:08.495387+0000","last_peered":"2023-11-08T08:19:08.495387+0000","last_clean":"2023-11-08T08:19:08.495387+0000","last_became_active":"2023-11-08T08:19:04.480992+0000","last_became_peered":"2023-11-08T08:19:04.480992+0000","last_unstale":"2023-11-08T08:19:08.495387+0000","last_undegraded":"2023-11-08T08:19:08.495387+0000","last_fullsized":"2023-11-08T08:19:08.495387+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.500946+0000","last_change":"2023-11-08T08:18:58.766848+0000","last_active":"2023-11-08T08:19:08.500946+0000","last_peered":"2023-11-08T08:19:08.500946+0000","last_clean":"2023-11-08T08:19:08.500946+0000","last_became_active":"2023-11-08T08:18:58.766378+0000","last_became_peered":"2023-11-08T08:18:58.766378+0000","last_unstale":"2023-11-08T08:19:08.500946+0000","last_undegraded":"2023-11-08T08:19:08.500946+0000","last_fullsized":"2023-11-08T08:19:08.500946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503466+0000","last_change":"2023-11-08T08:19:04.018717+0000","last_active":"2023-11-08T08:19:08.503466+0000","last_peered":"2023-11-08T08:19:08.503466+0000","last_clean":"2023-11-08T08:19:08.503466+0000","last_became_active":"2023-11-08T08:19:04.018355+0000","last_became_peered":"2023-11-08T08:19:04.018355+0000","last_unstale":"2023-11-08T08:19:08.503466+0000","last_undegraded":"2023-11-08T08:19:08.503466+0000","last_fullsized":"2023-11-08T08:19:08.503466+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:21:09.218618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.374050+0000","last_change":"2023-11-08T08:19:08.539068+0000","last_active":"2023-11-08T08:19:09.374050+0000","last_peered":"2023-11-08T08:19:09.374050+0000","last_clean":"2023-11-08T08:19:09.374050+0000","last_became_active":"2023-11-08T08:19:08.538763+0000","last_became_peered":"2023-11-08T08:19:08.538763+0000","last_unstale":"2023-11-08T08:19:09.374050+0000","last_undegraded":"2023-11-08T08:19:09.374050+0000","last_fullsized":"2023-11-08T08:19:09.374050+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:27:44.380774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.502722+0000","last_change":"2023-11-08T08:19:04.508515+0000","last_active":"2023-11-08T08:19:08.502722+0000","last_peered":"2023-11-08T08:19:08.502722+0000","last_clean":"2023-11-08T08:19:08.502722+0000","last_became_active":"2023-11-08T08:19:04.508227+0000","last_became_peered":"2023-11-08T08:19:04.508227+0000","last_unstale":"2023-11-08T08:19:08.502722+0000","last_undegraded":"2023-11-08T08:19:08.502722+0000","last_fullsized":"2023-11-08T08:19:08.502722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:48:06.180083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503646+0000","last_change":"2023-11-08T08:18:58.769764+0000","last_active":"2023-11-08T08:19:08.503646+0000","last_peered":"2023-11-08T08:19:08.503646+0000","last_clean":"2023-11-08T08:19:08.503646+0000","last_became_active":"2023-11-08T08:18:58.769357+0000","last_became_peered":"2023-11-08T08:18:58.769357+0000","last_unstale":"2023-11-08T08:19:08.503646+0000","last_undegraded":"2023-11-08T08:19:08.503646+0000","last_fullsized":"2023-11-08T08:19:08.503646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:13.748094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519826+0000","last_change":"2023-11-08T08:19:04.019534+0000","last_active":"2023-11-08T08:19:08.519826+0000","last_peered":"2023-11-08T08:19:08.519826+0000","last_clean":"2023-11-08T08:19:08.519826+0000","last_became_active":"2023-11-08T08:19:04.019269+0000","last_became_peered":"2023-11-08T08:19:04.019269+0000","last_unstale":"2023-11-08T08:19:08.519826+0000","last_undegraded":"2023-11-08T08:19:08.519826+0000","last_fullsized":"2023-11-08T08:19:08.519826+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:18:14.269118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.989010+0000","last_change":"2023-11-08T08:19:08.534164+0000","last_active":"2023-11-08T08:19:08.989010+0000","last_peered":"2023-11-08T08:19:08.989010+0000","last_clean":"2023-11-08T08:19:08.989010+0000","last_became_active":"2023-11-08T08:19:08.533749+0000","last_became_peered":"2023-11-08T08:19:08.533749+0000","last_unstale":"2023-11-08T08:19:08.989010+0000","last_undegraded":"2023-11-08T08:19:08.989010+0000","last_fullsized":"2023-11-08T08:19:08.989010+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:44:30.267694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503356+0000","last_change":"2023-11-08T08:19:04.492859+0000","last_active":"2023-11-08T08:19:08.503356+0000","last_peered":"2023-11-08T08:19:08.503356+0000","last_clean":"2023-11-08T08:19:08.503356+0000","last_became_active":"2023-11-08T08:19:04.492564+0000","last_became_peered":"2023-11-08T08:19:04.492564+0000","last_unstale":"2023-11-08T08:19:08.503356+0000","last_undegraded":"2023-11-08T08:19:08.503356+0000","last_fullsized":"2023-11-08T08:19:08.503356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512115+0000","last_change":"2023-11-08T08:19:04.024444+0000","last_active":"2023-11-08T08:19:08.512115+0000","last_peered":"2023-11-08T08:19:08.512115+0000","last_clean":"2023-11-08T08:19:08.512115+0000","last_became_active":"2023-11-08T08:19:04.024046+0000","last_became_peered":"2023-11-08T08:19:04.024046+0000","last_unstale":"2023-11-08T08:19:08.512115+0000","last_undegraded":"2023-11-08T08:19:08.512115+0000","last_fullsized":"2023-11-08T08:19:08.512115+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:15:32.560862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.514051+0000","last_change":"2023-11-08T08:18:58.771460+0000","last_active":"2023-11-08T08:19:08.514051+0000","last_peered":"2023-11-08T08:19:08.514051+0000","last_clean":"2023-11-08T08:19:08.514051+0000","last_became_active":"2023-11-08T08:18:58.770648+0000","last_became_peered":"2023-11-08T08:18:58.770648+0000","last_unstale":"2023-11-08T08:19:08.514051+0000","last_undegraded":"2023-11-08T08:19:08.514051+0000","last_fullsized":"2023-11-08T08:19:08.514051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.988677+0000","last_change":"2023-11-08T08:19:08.525326+0000","last_active":"2023-11-08T08:19:08.988677+0000","last_peered":"2023-11-08T08:19:08.988677+0000","last_clean":"2023-11-08T08:19:08.988677+0000","last_became_active":"2023-11-08T08:19:08.524957+0000","last_became_peered":"2023-11-08T08:19:08.524957+0000","last_unstale":"2023-11-08T08:19:08.988677+0000","last_undegraded":"2023-11-08T08:19:08.988677+0000","last_fullsized":"2023-11-08T08:19:08.988677+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:08.797102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503646+0000","last_change":"2023-11-08T08:19:04.503414+0000","last_active":"2023-11-08T08:19:08.503646+0000","last_peered":"2023-11-08T08:19:08.503646+0000","last_clean":"2023-11-08T08:19:08.503646+0000","last_became_active":"2023-11-08T08:19:04.503122+0000","last_became_peered":"2023-11-08T08:19:04.503122+0000","last_unstale":"2023-11-08T08:19:08.503646+0000","last_undegraded":"2023-11-08T08:19:08.503646+0000","last_fullsized":"2023-11-08T08:19:08.503646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501677+0000","last_change":"2023-11-08T08:19:04.020937+0000","last_active":"2023-11-08T08:19:08.501677+0000","last_peered":"2023-11-08T08:19:08.501677+0000","last_clean":"2023-11-08T08:19:08.501677+0000","last_became_active":"2023-11-08T08:19:04.020517+0000","last_became_peered":"2023-11-08T08:19:04.020517+0000","last_unstale":"2023-11-08T08:19:08.501677+0000","last_undegraded":"2023-11-08T08:19:08.501677+0000","last_fullsized":"2023-11-08T08:19:08.501677+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:50:41.342112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.509869+0000","last_change":"2023-11-08T08:18:58.973732+0000","last_active":"2023-11-08T08:19:08.509869+0000","last_peered":"2023-11-08T08:19:08.509869+0000","last_clean":"2023-11-08T08:19:08.509869+0000","last_became_active":"2023-11-08T08:18:58.973007+0000","last_became_peered":"2023-11-08T08:18:58.973007+0000","last_unstale":"2023-11-08T08:19:08.509869+0000","last_undegraded":"2023-11-08T08:19:08.509869+0000","last_fullsized":"2023-11-08T08:19:08.509869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.375576+0000","last_change":"2023-11-08T08:19:08.542391+0000","last_active":"2023-11-08T08:19:09.375576+0000","last_peered":"2023-11-08T08:19:09.375576+0000","last_clean":"2023-11-08T08:19:09.375576+0000","last_became_active":"2023-11-08T08:19:08.538962+0000","last_became_peered":"2023-11-08T08:19:08.538962+0000","last_unstale":"2023-11-08T08:19:09.375576+0000","last_undegraded":"2023-11-08T08:19:09.375576+0000","last_fullsized":"2023-11-08T08:19:09.375576+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:05:20.025231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.496994+0000","last_change":"2023-11-08T08:19:04.495234+0000","last_active":"2023-11-08T08:19:08.496994+0000","last_peered":"2023-11-08T08:19:08.496994+0000","last_clean":"2023-11-08T08:19:08.496994+0000","last_became_active":"2023-11-08T08:19:04.494991+0000","last_became_peered":"2023-11-08T08:19:04.494991+0000","last_unstale":"2023-11-08T08:19:08.496994+0000","last_undegraded":"2023-11-08T08:19:08.496994+0000","last_fullsized":"2023-11-08T08:19:08.496994+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:55:43.865351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.494564+0000","last_change":"2023-11-08T08:19:04.025597+0000","last_active":"2023-11-08T08:19:08.494564+0000","last_peered":"2023-11-08T08:19:08.494564+0000","last_clean":"2023-11-08T08:19:08.494564+0000","last_became_active":"2023-11-08T08:19:04.025292+0000","last_became_peered":"2023-11-08T08:19:04.025292+0000","last_unstale":"2023-11-08T08:19:08.494564+0000","last_undegraded":"2023-11-08T08:19:08.494564+0000","last_fullsized":"2023-11-08T08:19:08.494564+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:48:40.803835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.499836+0000","last_change":"2023-11-08T08:18:58.770136+0000","last_active":"2023-11-08T08:19:08.499836+0000","last_peered":"2023-11-08T08:19:08.499836+0000","last_clean":"2023-11-08T08:19:08.499836+0000","last_became_active":"2023-11-08T08:18:58.769797+0000","last_became_peered":"2023-11-08T08:18:58.769797+0000","last_unstale":"2023-11-08T08:19:08.499836+0000","last_undegraded":"2023-11-08T08:19:08.499836+0000","last_fullsized":"2023-11-08T08:19:08.499836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:29:12.107128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372601+0000","last_change":"2023-11-08T08:19:08.532719+0000","last_active":"2023-11-08T08:19:09.372601+0000","last_peered":"2023-11-08T08:19:09.372601+0000","last_clean":"2023-11-08T08:19:09.372601+0000","last_became_active":"2023-11-08T08:19:08.532282+0000","last_became_peered":"2023-11-08T08:19:08.532282+0000","last_unstale":"2023-11-08T08:19:09.372601+0000","last_undegraded":"2023-11-08T08:19:09.372601+0000","last_fullsized":"2023-11-08T08:19:09.372601+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:28:49.369446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.520613+0000","last_change":"2023-11-08T08:19:04.506360+0000","last_active":"2023-11-08T08:19:08.520613+0000","last_peered":"2023-11-08T08:19:08.520613+0000","last_clean":"2023-11-08T08:19:08.520613+0000","last_became_active":"2023-11-08T08:19:04.506117+0000","last_became_peered":"2023-11-08T08:19:04.506117+0000","last_unstale":"2023-11-08T08:19:08.520613+0000","last_undegraded":"2023-11-08T08:19:08.520613+0000","last_fullsized":"2023-11-08T08:19:08.520613+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:17:34.354695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517669+0000","last_change":"2023-11-08T08:19:04.020276+0000","last_active":"2023-11-08T08:19:08.517669+0000","last_peered":"2023-11-08T08:19:08.517669+0000","last_clean":"2023-11-08T08:19:08.517669+0000","last_became_active":"2023-11-08T08:19:04.019739+0000","last_became_peered":"2023-11-08T08:19:04.019739+0000","last_unstale":"2023-11-08T08:19:08.517669+0000","last_undegraded":"2023-11-08T08:19:08.517669+0000","last_fullsized":"2023-11-08T08:19:08.517669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:48:40.803835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519500+0000","last_change":"2023-11-08T08:18:58.777140+0000","last_active":"2023-11-08T08:19:08.519500+0000","last_peered":"2023-11-08T08:19:08.519500+0000","last_clean":"2023-11-08T08:19:08.519500+0000","last_became_active":"2023-11-08T08:18:58.776834+0000","last_became_peered":"2023-11-08T08:18:58.776834+0000","last_unstale":"2023-11-08T08:19:08.519500+0000","last_undegraded":"2023-11-08T08:19:08.519500+0000","last_fullsized":"2023-11-08T08:19:08.519500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.499953+0000","last_change":"2023-11-08T08:18:53.780089+0000","last_active":"2023-11-08T08:19:08.499953+0000","last_peered":"2023-11-08T08:19:08.499953+0000","last_clean":"2023-11-08T08:19:08.499953+0000","last_became_active":"2023-11-08T08:18:53.779533+0000","last_became_peered":"2023-11-08T08:18:53.779533+0000","last_unstale":"2023-11-08T08:19:08.499953+0000","last_undegraded":"2023-11-08T08:19:08.499953+0000","last_fullsized":"2023-11-08T08:19:08.499953+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_clean_scrub_stamp":"2023-11-08T08:18:52.744693+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:58:24.877070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.994953+0000","last_change":"2023-11-08T08:19:08.535027+0000","last_active":"2023-11-08T08:19:08.994953+0000","last_peered":"2023-11-08T08:19:08.994953+0000","last_clean":"2023-11-08T08:19:08.994953+0000","last_became_active":"2023-11-08T08:19:08.534620+0000","last_became_peered":"2023-11-08T08:19:08.534620+0000","last_unstale":"2023-11-08T08:19:08.994953+0000","last_undegraded":"2023-11-08T08:19:08.994953+0000","last_fullsized":"2023-11-08T08:19:08.994953+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:51:11.743430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503552+0000","last_change":"2023-11-08T08:19:04.503360+0000","last_active":"2023-11-08T08:19:08.503552+0000","last_peered":"2023-11-08T08:19:08.503552+0000","last_clean":"2023-11-08T08:19:08.503552+0000","last_became_active":"2023-11-08T08:19:04.502756+0000","last_became_peered":"2023-11-08T08:19:04.502756+0000","last_unstale":"2023-11-08T08:19:08.503552+0000","last_undegraded":"2023-11-08T08:19:08.503552+0000","last_fullsized":"2023-11-08T08:19:08.503552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:57:06.487072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517506+0000","last_change":"2023-11-08T08:19:04.025872+0000","last_active":"2023-11-08T08:19:08.517506+0000","last_peered":"2023-11-08T08:19:08.517506+0000","last_clean":"2023-11-08T08:19:08.517506+0000","last_became_active":"2023-11-08T08:19:04.025567+0000","last_became_peered":"2023-11-08T08:19:04.025567+0000","last_unstale":"2023-11-08T08:19:08.517506+0000","last_undegraded":"2023-11-08T08:19:08.517506+0000","last_fullsized":"2023-11-08T08:19:08.517506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:52:24.620964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.994736+0000","last_change":"2023-11-08T08:19:08.523353+0000","last_active":"2023-11-08T08:19:08.994736+0000","last_peered":"2023-11-08T08:19:08.994736+0000","last_clean":"2023-11-08T08:19:08.994736+0000","last_became_active":"2023-11-08T08:19:08.522741+0000","last_became_peered":"2023-11-08T08:19:08.522741+0000","last_unstale":"2023-11-08T08:19:08.994736+0000","last_undegraded":"2023-11-08T08:19:08.994736+0000","last_fullsized":"2023-11-08T08:19:08.994736+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:01:48.615826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512369+0000","last_change":"2023-11-08T08:19:04.512228+0000","last_active":"2023-11-08T08:19:08.512369+0000","last_peered":"2023-11-08T08:19:08.512369+0000","last_clean":"2023-11-08T08:19:08.512369+0000","last_became_active":"2023-11-08T08:19:04.501319+0000","last_became_peered":"2023-11-08T08:19:04.501319+0000","last_unstale":"2023-11-08T08:19:08.512369+0000","last_undegraded":"2023-11-08T08:19:08.512369+0000","last_fullsized":"2023-11-08T08:19:08.512369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:27:36.453040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513674+0000","last_change":"2023-11-08T08:19:04.012809+0000","last_active":"2023-11-08T08:19:08.513674+0000","last_peered":"2023-11-08T08:19:08.513674+0000","last_clean":"2023-11-08T08:19:08.513674+0000","last_became_active":"2023-11-08T08:19:04.012289+0000","last_became_peered":"2023-11-08T08:19:04.012289+0000","last_unstale":"2023-11-08T08:19:08.513674+0000","last_undegraded":"2023-11-08T08:19:08.513674+0000","last_fullsized":"2023-11-08T08:19:08.513674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:52:24.620964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372268+0000","last_change":"2023-11-08T08:19:08.538675+0000","last_active":"2023-11-08T08:19:09.372268+0000","last_peered":"2023-11-08T08:19:09.372268+0000","last_clean":"2023-11-08T08:19:09.372268+0000","last_became_active":"2023-11-08T08:19:08.537804+0000","last_became_peered":"2023-11-08T08:19:08.537804+0000","last_unstale":"2023-11-08T08:19:09.372268+0000","last_undegraded":"2023-11-08T08:19:09.372268+0000","last_fullsized":"2023-11-08T08:19:09.372268+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:32:44.139418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.516288+0000","last_change":"2023-11-08T08:19:04.498523+0000","last_active":"2023-11-08T08:19:08.516288+0000","last_peered":"2023-11-08T08:19:08.516288+0000","last_clean":"2023-11-08T08:19:08.516288+0000","last_became_active":"2023-11-08T08:19:04.497819+0000","last_became_peered":"2023-11-08T08:19:04.497819+0000","last_unstale":"2023-11-08T08:19:08.516288+0000","last_undegraded":"2023-11-08T08:19:08.516288+0000","last_fullsized":"2023-11-08T08:19:08.516288+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:55:43.865351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518299+0000","last_change":"2023-11-08T08:19:04.024906+0000","last_active":"2023-11-08T08:19:08.518299+0000","last_peered":"2023-11-08T08:19:08.518299+0000","last_clean":"2023-11-08T08:19:08.518299+0000","last_became_active":"2023-11-08T08:19:04.024603+0000","last_became_peered":"2023-11-08T08:19:04.024603+0000","last_unstale":"2023-11-08T08:19:08.518299+0000","last_undegraded":"2023-11-08T08:19:08.518299+0000","last_fullsized":"2023-11-08T08:19:08.518299+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:21:09.218618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372046+0000","last_change":"2023-11-08T08:19:08.525624+0000","last_active":"2023-11-08T08:19:09.372046+0000","last_peered":"2023-11-08T08:19:09.372046+0000","last_clean":"2023-11-08T08:19:09.372046+0000","last_became_active":"2023-11-08T08:19:08.524202+0000","last_became_peered":"2023-11-08T08:19:08.524202+0000","last_unstale":"2023-11-08T08:19:09.372046+0000","last_undegraded":"2023-11-08T08:19:09.372046+0000","last_fullsized":"2023-11-08T08:19:09.372046+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:08.797102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513312+0000","last_change":"2023-11-08T08:19:04.511393+0000","last_active":"2023-11-08T08:19:08.513312+0000","last_peered":"2023-11-08T08:19:08.513312+0000","last_clean":"2023-11-08T08:19:08.513312+0000","last_became_active":"2023-11-08T08:19:04.510749+0000","last_became_peered":"2023-11-08T08:19:04.510749+0000","last_unstale":"2023-11-08T08:19:08.513312+0000","last_undegraded":"2023-11-08T08:19:08.513312+0000","last_fullsized":"2023-11-08T08:19:08.513312+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:09:30.686343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518295+0000","last_change":"2023-11-08T08:19:04.022458+0000","last_active":"2023-11-08T08:19:08.518295+0000","last_peered":"2023-11-08T08:19:08.518295+0000","last_clean":"2023-11-08T08:19:08.518295+0000","last_became_active":"2023-11-08T08:19:04.022003+0000","last_became_peered":"2023-11-08T08:19:04.022003+0000","last_unstale":"2023-11-08T08:19:08.518295+0000","last_undegraded":"2023-11-08T08:19:08.518295+0000","last_fullsized":"2023-11-08T08:19:08.518295+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:21:09.218618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.375400+0000","last_change":"2023-11-08T08:19:08.536502+0000","last_active":"2023-11-08T08:19:09.375400+0000","last_peered":"2023-11-08T08:19:09.375400+0000","last_clean":"2023-11-08T08:19:09.375400+0000","last_became_active":"2023-11-08T08:19:08.536112+0000","last_became_peered":"2023-11-08T08:19:08.536112+0000","last_unstale":"2023-11-08T08:19:09.375400+0000","last_undegraded":"2023-11-08T08:19:09.375400+0000","last_fullsized":"2023-11-08T08:19:09.375400+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:26:20.245634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.495202+0000","last_change":"2023-11-08T08:19:04.493387+0000","last_active":"2023-11-08T08:19:08.495202+0000","last_peered":"2023-11-08T08:19:08.495202+0000","last_clean":"2023-11-08T08:19:08.495202+0000","last_became_active":"2023-11-08T08:19:04.493131+0000","last_became_peered":"2023-11-08T08:19:04.493131+0000","last_unstale":"2023-11-08T08:19:08.495202+0000","last_undegraded":"2023-11-08T08:19:08.495202+0000","last_fullsized":"2023-11-08T08:19:08.495202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:56:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519525+0000","last_change":"2023-11-08T08:19:04.021637+0000","last_active":"2023-11-08T08:19:08.519525+0000","last_peered":"2023-11-08T08:19:08.519525+0000","last_clean":"2023-11-08T08:19:08.519525+0000","last_became_active":"2023-11-08T08:19:04.018995+0000","last_became_peered":"2023-11-08T08:19:04.018995+0000","last_unstale":"2023-11-08T08:19:08.519525+0000","last_undegraded":"2023-11-08T08:19:08.519525+0000","last_fullsized":"2023-11-08T08:19:08.519525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:44.433066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.375204+0000","last_change":"2023-11-08T08:19:08.535800+0000","last_active":"2023-11-08T08:19:09.375204+0000","last_peered":"2023-11-08T08:19:09.375204+0000","last_clean":"2023-11-08T08:19:09.375204+0000","last_became_active":"2023-11-08T08:19:08.529039+0000","last_became_peered":"2023-11-08T08:19:08.529039+0000","last_unstale":"2023-11-08T08:19:09.375204+0000","last_undegraded":"2023-11-08T08:19:09.375204+0000","last_fullsized":"2023-11-08T08:19:09.375204+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:55:58.092159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518136+0000","last_change":"2023-11-08T08:19:04.497093+0000","last_active":"2023-11-08T08:19:08.518136+0000","last_peered":"2023-11-08T08:19:08.518136+0000","last_clean":"2023-11-08T08:19:08.518136+0000","last_became_active":"2023-11-08T08:19:04.496619+0000","last_became_peered":"2023-11-08T08:19:04.496619+0000","last_unstale":"2023-11-08T08:19:08.518136+0000","last_undegraded":"2023-11-08T08:19:08.518136+0000","last_fullsized":"2023-11-08T08:19:08.518136+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:27:39.386956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.500739+0000","last_change":"2023-11-08T08:19:04.015072+0000","last_active":"2023-11-08T08:19:08.500739+0000","last_peered":"2023-11-08T08:19:08.500739+0000","last_clean":"2023-11-08T08:19:08.500739+0000","last_became_active":"2023-11-08T08:19:04.014727+0000","last_became_peered":"2023-11-08T08:19:04.014727+0000","last_unstale":"2023-11-08T08:19:08.500739+0000","last_undegraded":"2023-11-08T08:19:08.500739+0000","last_fullsized":"2023-11-08T08:19:08.500739+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:17:33.341483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.375246+0000","last_change":"2023-11-08T08:19:08.534568+0000","last_active":"2023-11-08T08:19:09.375246+0000","last_peered":"2023-11-08T08:19:09.375246+0000","last_clean":"2023-11-08T08:19:09.375246+0000","last_became_active":"2023-11-08T08:19:08.534139+0000","last_became_peered":"2023-11-08T08:19:08.534139+0000","last_unstale":"2023-11-08T08:19:09.375246+0000","last_undegraded":"2023-11-08T08:19:09.375246+0000","last_fullsized":"2023-11-08T08:19:09.375246+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:59:39.090459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513527+0000","last_change":"2023-11-08T08:19:04.512845+0000","last_active":"2023-11-08T08:19:08.513527+0000","last_peered":"2023-11-08T08:19:08.513527+0000","last_clean":"2023-11-08T08:19:08.513527+0000","last_became_active":"2023-11-08T08:19:04.512499+0000","last_became_peered":"2023-11-08T08:19:04.512499+0000","last_unstale":"2023-11-08T08:19:08.513527+0000","last_undegraded":"2023-11-08T08:19:08.513527+0000","last_fullsized":"2023-11-08T08:19:08.513527+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:24:10.475972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501411+0000","last_change":"2023-11-08T08:19:04.018033+0000","last_active":"2023-11-08T08:19:08.501411+0000","last_peered":"2023-11-08T08:19:08.501411+0000","last_clean":"2023-11-08T08:19:08.501411+0000","last_became_active":"2023-11-08T08:19:04.017059+0000","last_became_peered":"2023-11-08T08:19:04.017059+0000","last_unstale":"2023-11-08T08:19:08.501411+0000","last_undegraded":"2023-11-08T08:19:08.501411+0000","last_fullsized":"2023-11-08T08:19:08.501411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:44.433066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.369706+0000","last_change":"2023-11-08T08:19:08.539326+0000","last_active":"2023-11-08T08:19:09.369706+0000","last_peered":"2023-11-08T08:19:09.369706+0000","last_clean":"2023-11-08T08:19:09.369706+0000","last_became_active":"2023-11-08T08:19:08.538846+0000","last_became_peered":"2023-11-08T08:19:08.538846+0000","last_unstale":"2023-11-08T08:19:09.369706+0000","last_undegraded":"2023-11-08T08:19:09.369706+0000","last_fullsized":"2023-11-08T08:19:09.369706+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:33:00.754113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513023+0000","last_change":"2023-11-08T08:19:04.503981+0000","last_active":"2023-11-08T08:19:08.513023+0000","last_peered":"2023-11-08T08:19:08.513023+0000","last_clean":"2023-11-08T08:19:08.513023+0000","last_became_active":"2023-11-08T08:19:04.503651+0000","last_became_peered":"2023-11-08T08:19:04.503651+0000","last_unstale":"2023-11-08T08:19:08.513023+0000","last_undegraded":"2023-11-08T08:19:08.513023+0000","last_fullsized":"2023-11-08T08:19:08.513023+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:55:43.865351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.500289+0000","last_change":"2023-11-08T08:19:04.022166+0000","last_active":"2023-11-08T08:19:08.500289+0000","last_peered":"2023-11-08T08:19:08.500289+0000","last_clean":"2023-11-08T08:19:08.500289+0000","last_became_active":"2023-11-08T08:19:04.017289+0000","last_became_peered":"2023-11-08T08:19:04.017289+0000","last_unstale":"2023-11-08T08:19:08.500289+0000","last_undegraded":"2023-11-08T08:19:08.500289+0000","last_fullsized":"2023-11-08T08:19:08.500289+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:51:22.228427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.995287+0000","last_change":"2023-11-08T08:19:08.534923+0000","last_active":"2023-11-08T08:19:08.995287+0000","last_peered":"2023-11-08T08:19:08.995287+0000","last_clean":"2023-11-08T08:19:08.995287+0000","last_became_active":"2023-11-08T08:19:08.530869+0000","last_became_peered":"2023-11-08T08:19:08.530869+0000","last_unstale":"2023-11-08T08:19:08.995287+0000","last_undegraded":"2023-11-08T08:19:08.995287+0000","last_fullsized":"2023-11-08T08:19:08.995287+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:47:09.660706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372246+0000","last_change":"2023-11-08T08:19:08.541714+0000","last_active":"2023-11-08T08:19:09.372246+0000","last_peered":"2023-11-08T08:19:09.372246+0000","last_clean":"2023-11-08T08:19:09.372246+0000","last_became_active":"2023-11-08T08:19:08.537859+0000","last_became_peered":"2023-11-08T08:19:08.537859+0000","last_unstale":"2023-11-08T08:19:09.372246+0000","last_undegraded":"2023-11-08T08:19:09.372246+0000","last_fullsized":"2023-11-08T08:19:09.372246+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:33:11.313632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518613+0000","last_change":"2023-11-08T08:19:04.008921+0000","last_active":"2023-11-08T08:19:08.518613+0000","last_peered":"2023-11-08T08:19:08.518613+0000","last_clean":"2023-11-08T08:19:08.518613+0000","last_became_active":"2023-11-08T08:19:04.008616+0000","last_became_peered":"2023-11-08T08:19:04.008616+0000","last_unstale":"2023-11-08T08:19:08.518613+0000","last_undegraded":"2023-11-08T08:19:08.518613+0000","last_fullsized":"2023-11-08T08:19:08.518613+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:50:41.342112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.520011+0000","last_change":"2023-11-08T08:19:04.501127+0000","last_active":"2023-11-08T08:19:08.520011+0000","last_peered":"2023-11-08T08:19:08.520011+0000","last_clean":"2023-11-08T08:19:08.520011+0000","last_became_active":"2023-11-08T08:19:04.500705+0000","last_became_peered":"2023-11-08T08:19:04.500705+0000","last_unstale":"2023-11-08T08:19:08.520011+0000","last_undegraded":"2023-11-08T08:19:08.520011+0000","last_fullsized":"2023-11-08T08:19:08.520011+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:25.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372698+0000","last_change":"2023-11-08T08:19:08.533064+0000","last_active":"2023-11-08T08:19:09.372698+0000","last_peered":"2023-11-08T08:19:09.372698+0000","last_clean":"2023-11-08T08:19:09.372698+0000","last_became_active":"2023-11-08T08:19:08.532631+0000","last_became_peered":"2023-11-08T08:19:08.532631+0000","last_unstale":"2023-11-08T08:19:09.372698+0000","last_undegraded":"2023-11-08T08:19:09.372698+0000","last_fullsized":"2023-11-08T08:19:09.372698+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:32:44.139418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501706+0000","last_change":"2023-11-08T08:19:04.017775+0000","last_active":"2023-11-08T08:19:08.501706+0000","last_peered":"2023-11-08T08:19:08.501706+0000","last_clean":"2023-11-08T08:19:08.501706+0000","last_became_active":"2023-11-08T08:19:04.017572+0000","last_became_peered":"2023-11-08T08:19:04.017572+0000","last_unstale":"2023-11-08T08:19:08.501706+0000","last_undegraded":"2023-11-08T08:19:08.501706+0000","last_fullsized":"2023-11-08T08:19:08.501706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:55:07.041721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.497148+0000","last_change":"2023-11-08T08:19:04.488153+0000","last_active":"2023-11-08T08:19:08.497148+0000","last_peered":"2023-11-08T08:19:08.497148+0000","last_clean":"2023-11-08T08:19:08.497148+0000","last_became_active":"2023-11-08T08:19:04.487871+0000","last_became_peered":"2023-11-08T08:19:04.487871+0000","last_unstale":"2023-11-08T08:19:08.497148+0000","last_undegraded":"2023-11-08T08:19:08.497148+0000","last_fullsized":"2023-11-08T08:19:08.497148+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:06:47.395577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.494806+0000","last_change":"2023-11-08T08:19:04.011650+0000","last_active":"2023-11-08T08:19:08.494806+0000","last_peered":"2023-11-08T08:19:08.494806+0000","last_clean":"2023-11-08T08:19:08.494806+0000","last_became_active":"2023-11-08T08:19:04.011071+0000","last_became_peered":"2023-11-08T08:19:04.011071+0000","last_unstale":"2023-11-08T08:19:08.494806+0000","last_undegraded":"2023-11-08T08:19:08.494806+0000","last_fullsized":"2023-11-08T08:19:08.494806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:51:22.228427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519773+0000","last_change":"2023-11-08T08:19:04.505594+0000","last_active":"2023-11-08T08:19:08.519773+0000","last_peered":"2023-11-08T08:19:08.519773+0000","last_clean":"2023-11-08T08:19:08.519773+0000","last_became_active":"2023-11-08T08:19:04.505258+0000","last_became_peered":"2023-11-08T08:19:04.505258+0000","last_unstale":"2023-11-08T08:19:08.519773+0000","last_undegraded":"2023-11-08T08:19:08.519773+0000","last_fullsized":"2023-11-08T08:19:08.519773+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:56:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.373555+0000","last_change":"2023-11-08T08:19:08.529766+0000","last_active":"2023-11-08T08:19:09.373555+0000","last_peered":"2023-11-08T08:19:09.373555+0000","last_clean":"2023-11-08T08:19:09.373555+0000","last_became_active":"2023-11-08T08:19:08.529334+0000","last_became_peered":"2023-11-08T08:19:08.529334+0000","last_unstale":"2023-11-08T08:19:09.373555+0000","last_undegraded":"2023-11-08T08:19:09.373555+0000","last_fullsized":"2023-11-08T08:19:09.373555+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:28:49.369446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501076+0000","last_change":"2023-11-08T08:19:04.020027+0000","last_active":"2023-11-08T08:19:08.501076+0000","last_peered":"2023-11-08T08:19:08.501076+0000","last_clean":"2023-11-08T08:19:08.501076+0000","last_became_active":"2023-11-08T08:19:04.019727+0000","last_became_peered":"2023-11-08T08:19:04.019727+0000","last_unstale":"2023-11-08T08:19:08.501076+0000","last_undegraded":"2023-11-08T08:19:08.501076+0000","last_fullsized":"2023-11-08T08:19:08.501076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:02.957310+0000","last_clean_scrub_stamp":"2023-11-08T08:19:02.957310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:18:14.269118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519487+0000","last_change":"2023-11-08T08:19:04.495283+0000","last_active":"2023-11-08T08:19:08.519487+0000","last_peered":"2023-11-08T08:19:08.519487+0000","last_clean":"2023-11-08T08:19:08.519487+0000","last_became_active":"2023-11-08T08:19:04.494955+0000","last_became_peered":"2023-11-08T08:19:04.494955+0000","last_unstale":"2023-11-08T08:19:08.519487+0000","last_undegraded":"2023-11-08T08:19:08.519487+0000","last_fullsized":"2023-11-08T08:19:08.519487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-08T08:19:09.372061+0000","last_change":"2023-11-08T08:19:08.532374+0000","last_active":"2023-11-08T08:19:09.372061+0000","last_peered":"2023-11-08T08:19:09.372061+0000","last_clean":"2023-11-08T08:19:09.372061+0000","last_became_active":"2023-11-08T08:19:08.531953+0000","last_became_peered":"2023-11-08T08:19:08.531953+0000","last_unstale":"2023-11-08T08:19:09.372061+0000","last_undegraded":"2023-11-08T08:19:09.372061+0000","last_fullsized":"2023-11-08T08:19:09.372061+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:07.478875+0000","last_clean_scrub_stamp":"2023-11-08T08:19:07.478875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:54:33.577798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518515+0000","last_change":"2023-11-08T08:19:04.491042+0000","last_active":"2023-11-08T08:19:08.518515+0000","last_peered":"2023-11-08T08:19:08.518515+0000","last_clean":"2023-11-08T08:19:08.518515+0000","last_became_active":"2023-11-08T08:19:04.490734+0000","last_became_peered":"2023-11-08T08:19:04.490734+0000","last_unstale":"2023-11-08T08:19:08.518515+0000","last_undegraded":"2023-11-08T08:19:08.518515+0000","last_fullsized":"2023-11-08T08:19:08.518515+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:26:27.887614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517935+0000","last_change":"2023-11-08T08:19:04.507871+0000","last_active":"2023-11-08T08:19:08.517935+0000","last_peered":"2023-11-08T08:19:08.517935+0000","last_clean":"2023-11-08T08:19:08.517935+0000","last_became_active":"2023-11-08T08:19:04.507600+0000","last_became_peered":"2023-11-08T08:19:04.507600+0000","last_unstale":"2023-11-08T08:19:08.517935+0000","last_undegraded":"2023-11-08T08:19:08.517935+0000","last_fullsized":"2023-11-08T08:19:08.517935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:55:43.865351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512136+0000","last_change":"2023-11-08T08:19:04.505023+0000","last_active":"2023-11-08T08:19:08.512136+0000","last_peered":"2023-11-08T08:19:08.512136+0000","last_clean":"2023-11-08T08:19:08.512136+0000","last_became_active":"2023-11-08T08:19:04.504750+0000","last_became_peered":"2023-11-08T08:19:04.504750+0000","last_unstale":"2023-11-08T08:19:08.512136+0000","last_undegraded":"2023-11-08T08:19:08.512136+0000","last_fullsized":"2023-11-08T08:19:08.512136+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:25.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.518063+0000","last_change":"2023-11-08T08:19:04.504115+0000","last_active":"2023-11-08T08:19:08.518063+0000","last_peered":"2023-11-08T08:19:08.518063+0000","last_clean":"2023-11-08T08:19:08.518063+0000","last_became_active":"2023-11-08T08:19:04.503841+0000","last_became_peered":"2023-11-08T08:19:04.503841+0000","last_unstale":"2023-11-08T08:19:08.518063+0000","last_undegraded":"2023-11-08T08:19:08.518063+0000","last_fullsized":"2023-11-08T08:19:08.518063+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:33:12.861277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503562+0000","last_change":"2023-11-08T08:19:04.504534+0000","last_active":"2023-11-08T08:19:08.503562+0000","last_peered":"2023-11-08T08:19:08.503562+0000","last_clean":"2023-11-08T08:19:08.503562+0000","last_became_active":"2023-11-08T08:19:04.504283+0000","last_became_peered":"2023-11-08T08:19:04.504283+0000","last_unstale":"2023-11-08T08:19:08.503562+0000","last_undegraded":"2023-11-08T08:19:08.503562+0000","last_fullsized":"2023-11-08T08:19:08.503562+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:56:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.519721+0000","last_change":"2023-11-08T08:19:04.503978+0000","last_active":"2023-11-08T08:19:08.519721+0000","last_peered":"2023-11-08T08:19:08.519721+0000","last_clean":"2023-11-08T08:19:08.519721+0000","last_became_active":"2023-11-08T08:19:04.503653+0000","last_became_peered":"2023-11-08T08:19:04.503653+0000","last_unstale":"2023-11-08T08:19:08.519721+0000","last_undegraded":"2023-11-08T08:19:08.519721+0000","last_fullsized":"2023-11-08T08:19:08.519721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503754+0000","last_change":"2023-11-08T08:19:04.491309+0000","last_active":"2023-11-08T08:19:08.503754+0000","last_peered":"2023-11-08T08:19:08.503754+0000","last_clean":"2023-11-08T08:19:08.503754+0000","last_became_active":"2023-11-08T08:19:04.491035+0000","last_became_peered":"2023-11-08T08:19:04.491035+0000","last_unstale":"2023-11-08T08:19:08.503754+0000","last_undegraded":"2023-11-08T08:19:08.503754+0000","last_fullsized":"2023-11-08T08:19:08.503754+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:18:36.796288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.503455+0000","last_change":"2023-11-08T08:19:04.503092+0000","last_active":"2023-11-08T08:19:08.503455+0000","last_peered":"2023-11-08T08:19:08.503455+0000","last_clean":"2023-11-08T08:19:08.503455+0000","last_became_active":"2023-11-08T08:19:04.502751+0000","last_became_peered":"2023-11-08T08:19:04.502751+0000","last_unstale":"2023-11-08T08:19:08.503455+0000","last_undegraded":"2023-11-08T08:19:08.503455+0000","last_fullsized":"2023-11-08T08:19:08.503455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:40:44.425321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.499633+0000","last_change":"2023-11-08T08:19:04.509538+0000","last_active":"2023-11-08T08:19:08.499633+0000","last_peered":"2023-11-08T08:19:08.499633+0000","last_clean":"2023-11-08T08:19:08.499633+0000","last_became_active":"2023-11-08T08:19:04.509190+0000","last_became_peered":"2023-11-08T08:19:04.509190+0000","last_unstale":"2023-11-08T08:19:08.499633+0000","last_undegraded":"2023-11-08T08:19:08.499633+0000","last_fullsized":"2023-11-08T08:19:08.499633+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:04:55.302575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517870+0000","last_change":"2023-11-08T08:19:04.489238+0000","last_active":"2023-11-08T08:19:08.517870+0000","last_peered":"2023-11-08T08:19:08.517870+0000","last_clean":"2023-11-08T08:19:08.517870+0000","last_became_active":"2023-11-08T08:19:04.488998+0000","last_became_peered":"2023-11-08T08:19:04.488998+0000","last_unstale":"2023-11-08T08:19:08.517870+0000","last_undegraded":"2023-11-08T08:19:08.517870+0000","last_fullsized":"2023-11-08T08:19:08.517870+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:27:36.453040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.494671+0000","last_change":"2023-11-08T08:19:04.499236+0000","last_active":"2023-11-08T08:19:08.494671+0000","last_peered":"2023-11-08T08:19:08.494671+0000","last_clean":"2023-11-08T08:19:08.494671+0000","last_became_active":"2023-11-08T08:19:04.498978+0000","last_became_peered":"2023-11-08T08:19:04.498978+0000","last_unstale":"2023-11-08T08:19:08.494671+0000","last_undegraded":"2023-11-08T08:19:08.494671+0000","last_fullsized":"2023-11-08T08:19:08.494671+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:04:55.302575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.500995+0000","last_change":"2023-11-08T08:19:04.513191+0000","last_active":"2023-11-08T08:19:08.500995+0000","last_peered":"2023-11-08T08:19:08.500995+0000","last_clean":"2023-11-08T08:19:08.500995+0000","last_became_active":"2023-11-08T08:19:04.512519+0000","last_became_peered":"2023-11-08T08:19:04.512519+0000","last_unstale":"2023-11-08T08:19:08.500995+0000","last_undegraded":"2023-11-08T08:19:08.500995+0000","last_fullsized":"2023-11-08T08:19:08.500995+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:56:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513262+0000","last_change":"2023-11-08T08:19:04.510593+0000","last_active":"2023-11-08T08:19:08.513262+0000","last_peered":"2023-11-08T08:19:08.513262+0000","last_clean":"2023-11-08T08:19:08.513262+0000","last_became_active":"2023-11-08T08:19:04.501361+0000","last_became_peered":"2023-11-08T08:19:04.501361+0000","last_unstale":"2023-11-08T08:19:08.513262+0000","last_undegraded":"2023-11-08T08:19:08.513262+0000","last_fullsized":"2023-11-08T08:19:08.513262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.509488+0000","last_change":"2023-11-08T08:19:04.497954+0000","last_active":"2023-11-08T08:19:08.509488+0000","last_peered":"2023-11-08T08:19:08.509488+0000","last_clean":"2023-11-08T08:19:08.509488+0000","last_became_active":"2023-11-08T08:19:04.497675+0000","last_became_peered":"2023-11-08T08:19:04.497675+0000","last_unstale":"2023-11-08T08:19:08.509488+0000","last_undegraded":"2023-11-08T08:19:08.509488+0000","last_fullsized":"2023-11-08T08:19:08.509488+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:07:36.146057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512026+0000","last_change":"2023-11-08T08:19:04.494551+0000","last_active":"2023-11-08T08:19:08.512026+0000","last_peered":"2023-11-08T08:19:08.512026+0000","last_clean":"2023-11-08T08:19:08.512026+0000","last_became_active":"2023-11-08T08:19:04.494261+0000","last_became_peered":"2023-11-08T08:19:04.494261+0000","last_unstale":"2023-11-08T08:19:08.512026+0000","last_undegraded":"2023-11-08T08:19:08.512026+0000","last_fullsized":"2023-11-08T08:19:08.512026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:27:40.872420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.496353+0000","last_change":"2023-11-08T08:19:04.486999+0000","last_active":"2023-11-08T08:19:08.496353+0000","last_peered":"2023-11-08T08:19:08.496353+0000","last_clean":"2023-11-08T08:19:08.496353+0000","last_became_active":"2023-11-08T08:19:04.486746+0000","last_became_peered":"2023-11-08T08:19:04.486746+0000","last_unstale":"2023-11-08T08:19:08.496353+0000","last_undegraded":"2023-11-08T08:19:08.496353+0000","last_fullsized":"2023-11-08T08:19:08.496353+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:27:36.453040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513306+0000","last_change":"2023-11-08T08:19:04.506263+0000","last_active":"2023-11-08T08:19:08.513306+0000","last_peered":"2023-11-08T08:19:08.513306+0000","last_clean":"2023-11-08T08:19:08.513306+0000","last_became_active":"2023-11-08T08:19:04.505930+0000","last_became_peered":"2023-11-08T08:19:04.505930+0000","last_unstale":"2023-11-08T08:19:08.513306+0000","last_undegraded":"2023-11-08T08:19:08.513306+0000","last_fullsized":"2023-11-08T08:19:08.513306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:41:09.071125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501581+0000","last_change":"2023-11-08T08:19:04.514677+0000","last_active":"2023-11-08T08:19:08.501581+0000","last_peered":"2023-11-08T08:19:08.501581+0000","last_clean":"2023-11-08T08:19:08.501581+0000","last_became_active":"2023-11-08T08:19:04.511465+0000","last_became_peered":"2023-11-08T08:19:04.511465+0000","last_unstale":"2023-11-08T08:19:08.501581+0000","last_undegraded":"2023-11-08T08:19:08.501581+0000","last_fullsized":"2023-11-08T08:19:08.501581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:35.964411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.513976+0000","last_change":"2023-11-08T08:19:04.490773+0000","last_active":"2023-11-08T08:19:08.513976+0000","last_peered":"2023-11-08T08:19:08.513976+0000","last_clean":"2023-11-08T08:19:08.513976+0000","last_became_active":"2023-11-08T08:19:04.490474+0000","last_became_peered":"2023-11-08T08:19:04.490474+0000","last_unstale":"2023-11-08T08:19:08.513976+0000","last_undegraded":"2023-11-08T08:19:08.513976+0000","last_fullsized":"2023-11-08T08:19:08.513976+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512799+0000","last_change":"2023-11-08T08:19:04.487159+0000","last_active":"2023-11-08T08:19:08.512799+0000","last_peered":"2023-11-08T08:19:08.512799+0000","last_clean":"2023-11-08T08:19:08.512799+0000","last_became_active":"2023-11-08T08:19:04.486861+0000","last_became_peered":"2023-11-08T08:19:04.486861+0000","last_unstale":"2023-11-08T08:19:08.512799+0000","last_undegraded":"2023-11-08T08:19:08.512799+0000","last_fullsized":"2023-11-08T08:19:08.512799+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:45.446278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.509301+0000","last_change":"2023-11-08T08:19:04.491757+0000","last_active":"2023-11-08T08:19:08.509301+0000","last_peered":"2023-11-08T08:19:08.509301+0000","last_clean":"2023-11-08T08:19:08.509301+0000","last_became_active":"2023-11-08T08:19:04.491359+0000","last_became_peered":"2023-11-08T08:19:04.491359+0000","last_unstale":"2023-11-08T08:19:08.509301+0000","last_undegraded":"2023-11-08T08:19:08.509301+0000","last_fullsized":"2023-11-08T08:19:08.509301+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:01:42.294189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.516256+0000","last_change":"2023-11-08T08:19:04.498748+0000","last_active":"2023-11-08T08:19:08.516256+0000","last_peered":"2023-11-08T08:19:08.516256+0000","last_clean":"2023-11-08T08:19:08.516256+0000","last_became_active":"2023-11-08T08:19:04.498208+0000","last_became_peered":"2023-11-08T08:19:04.498208+0000","last_unstale":"2023-11-08T08:19:08.516256+0000","last_undegraded":"2023-11-08T08:19:08.516256+0000","last_fullsized":"2023-11-08T08:19:08.516256+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:33:12.861277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521468+0000","last_change":"2023-11-08T08:19:04.490052+0000","last_active":"2023-11-08T08:19:08.521468+0000","last_peered":"2023-11-08T08:19:08.521468+0000","last_clean":"2023-11-08T08:19:08.521468+0000","last_became_active":"2023-11-08T08:19:04.489758+0000","last_became_peered":"2023-11-08T08:19:04.489758+0000","last_unstale":"2023-11-08T08:19:08.521468+0000","last_undegraded":"2023-11-08T08:19:08.521468+0000","last_fullsized":"2023-11-08T08:19:08.521468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:25.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.495092+0000","last_change":"2023-11-08T08:19:04.489823+0000","last_active":"2023-11-08T08:19:08.495092+0000","last_peered":"2023-11-08T08:19:08.495092+0000","last_clean":"2023-11-08T08:19:08.495092+0000","last_became_active":"2023-11-08T08:19:04.489559+0000","last_became_peered":"2023-11-08T08:19:04.489559+0000","last_unstale":"2023-11-08T08:19:08.495092+0000","last_undegraded":"2023-11-08T08:19:08.495092+0000","last_fullsized":"2023-11-08T08:19:08.495092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:25.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.512940+0000","last_change":"2023-11-08T08:19:04.509192+0000","last_active":"2023-11-08T08:19:08.512940+0000","last_peered":"2023-11-08T08:19:08.512940+0000","last_clean":"2023-11-08T08:19:08.512940+0000","last_became_active":"2023-11-08T08:19:04.508856+0000","last_became_peered":"2023-11-08T08:19:04.508856+0000","last_unstale":"2023-11-08T08:19:08.512940+0000","last_undegraded":"2023-11-08T08:19:08.512940+0000","last_fullsized":"2023-11-08T08:19:08.512940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:27:36.453040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.501455+0000","last_change":"2023-11-08T08:19:04.510274+0000","last_active":"2023-11-08T08:19:08.501455+0000","last_peered":"2023-11-08T08:19:08.501455+0000","last_clean":"2023-11-08T08:19:08.501455+0000","last_became_active":"2023-11-08T08:19:04.510052+0000","last_became_peered":"2023-11-08T08:19:04.510052+0000","last_unstale":"2023-11-08T08:19:08.501455+0000","last_undegraded":"2023-11-08T08:19:08.501455+0000","last_fullsized":"2023-11-08T08:19:08.501455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:11:11.305869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.511920+0000","last_change":"2023-11-08T08:19:04.507388+0000","last_active":"2023-11-08T08:19:08.511920+0000","last_peered":"2023-11-08T08:19:08.511920+0000","last_clean":"2023-11-08T08:19:08.511920+0000","last_became_active":"2023-11-08T08:19:04.506655+0000","last_became_peered":"2023-11-08T08:19:04.506655+0000","last_unstale":"2023-11-08T08:19:08.511920+0000","last_undegraded":"2023-11-08T08:19:08.511920+0000","last_fullsized":"2023-11-08T08:19:08.511920+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:26:16.737280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.495536+0000","last_change":"2023-11-08T08:19:04.484402+0000","last_active":"2023-11-08T08:19:08.495536+0000","last_peered":"2023-11-08T08:19:08.495536+0000","last_clean":"2023-11-08T08:19:08.495536+0000","last_became_active":"2023-11-08T08:19:04.480030+0000","last_became_peered":"2023-11-08T08:19:04.480030+0000","last_unstale":"2023-11-08T08:19:08.495536+0000","last_undegraded":"2023-11-08T08:19:08.495536+0000","last_fullsized":"2023-11-08T08:19:08.495536+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:27:40.872420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.517912+0000","last_change":"2023-11-08T08:19:04.500239+0000","last_active":"2023-11-08T08:19:08.517912+0000","last_peered":"2023-11-08T08:19:08.517912+0000","last_clean":"2023-11-08T08:19:08.517912+0000","last_became_active":"2023-11-08T08:19:04.499933+0000","last_became_peered":"2023-11-08T08:19:04.499933+0000","last_unstale":"2023-11-08T08:19:08.517912+0000","last_undegraded":"2023-11-08T08:19:08.517912+0000","last_fullsized":"2023-11-08T08:19:08.517912+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:45.446278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.516154+0000","last_change":"2023-11-08T08:19:04.507285+0000","last_active":"2023-11-08T08:19:08.516154+0000","last_peered":"2023-11-08T08:19:08.516154+0000","last_clean":"2023-11-08T08:19:08.516154+0000","last_became_active":"2023-11-08T08:19:04.506999+0000","last_became_peered":"2023-11-08T08:19:04.506999+0000","last_unstale":"2023-11-08T08:19:08.516154+0000","last_undegraded":"2023-11-08T08:19:08.516154+0000","last_fullsized":"2023-11-08T08:19:08.516154+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:55:54.583805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.521418+0000","last_change":"2023-11-08T08:19:04.496585+0000","last_active":"2023-11-08T08:19:08.521418+0000","last_peered":"2023-11-08T08:19:08.521418+0000","last_clean":"2023-11-08T08:19:08.521418+0000","last_became_active":"2023-11-08T08:19:04.496307+0000","last_became_peered":"2023-11-08T08:19:04.496307+0000","last_unstale":"2023-11-08T08:19:08.521418+0000","last_undegraded":"2023-11-08T08:19:08.521418+0000","last_fullsized":"2023-11-08T08:19:08.521418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:06:47.395577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-08T08:19:08.509596+0000","last_change":"2023-11-08T08:19:04.487836+0000","last_active":"2023-11-08T08:19:08.509596+0000","last_peered":"2023-11-08T08:19:08.509596+0000","last_clean":"2023-11-08T08:19:08.509596+0000","last_became_active":"2023-11-08T08:19:04.487548+0000","last_became_peered":"2023-11-08T08:19:04.487548+0000","last_unstale":"2023-11-08T08:19:08.509596+0000","last_undegraded":"2023-11-08T08:19:08.509596+0000","last_fullsized":"2023-11-08T08:19:08.509596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:19:03.970522+0000","last_clean_scrub_stamp":"2023-11-08T08:19:03.970522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:55:08.054932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4527,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9296,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362544,"statfs":{"total":96636764160,"available":96627245056,"internally_reserved":0,"allocated":925696,"data_stored":718901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6949999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3080000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]}]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8176,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363664,"statfs":{"total":96636764160,"available":96628391936,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9370000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1059999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8184,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363656,"statfs":{"total":96636764160,"available":96628383744,"internally_reserved":0,"allocated":311296,"data_stored":125208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4790000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.532}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8260,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363580,"statfs":{"total":96636764160,"available":96628305920,"internally_reserved":0,"allocated":323584,"data_stored":128373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.99}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8244,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363596,"statfs":{"total":96636764160,"available":96628322304,"internally_reserved":0,"allocated":307200,"data_stored":124662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5660000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.623}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8112,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363728,"statfs":{"total":96636764160,"available":96628457472,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.532}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8240,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363600,"statfs":{"total":96636764160,"available":96628326400,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.373}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4830000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9272,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362568,"statfs":{"total":96636764160,"available":96627269632,"internally_reserved":0,"allocated":901120,"data_stored":715506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.288}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T08:19:37.974 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-08T08:19:37.974 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-08T08:19:37.975 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-08T08:19:37.975 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-08T08:19:38.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda50e2f700 1 Processor -- start 2023-11-08T08:19:38.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda50e2f700 1 -- start start 2023-11-08T08:19:38.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda50e2f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fda44036e80 0x7fda440ea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:38.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda50e2f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda44052bf0 con 0x7fda440dd280 2023-11-08T08:19:38.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda50e2f700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda440528c0 con 0x7fda44050760 2023-11-08T08:19:38.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda50e2f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda440dd650 con 0x7fda44036e80 2023-11-08T08:19:38.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4b7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fda440dd280 0x7fda440dfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40594/0 (socket says 172.21.15.40:40594) 2023-11-08T08:19:38.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4b7fe700 1 -- 172.21.15.40:0/474606337 learned_addr learned my addr 172.21.15.40:0/474606337 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:38.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4bfff700 1 --2- 172.21.15.40:0/474606337 >> v2:172.21.15.40:3300/0 conn(0x7fda44036e80 0x7fda440ea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:38.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4a7fc700 1 -- 172.21.15.40:0/474606337 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3339344885 0 0) 0x7fda44052bf0 con 0x7fda440dd280 2023-11-08T08:19:38.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4a7fc700 1 -- 172.21.15.40:0/474606337 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda340035d0 con 0x7fda440dd280 2023-11-08T08:19:38.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4a7fc700 1 -- 172.21.15.40:0/474606337 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996777128 0 0) 0x7fda440528c0 con 0x7fda44050760 2023-11-08T08:19:38.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4a7fc700 1 -- 172.21.15.40:0/474606337 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda44052bf0 con 0x7fda44050760 2023-11-08T08:19:38.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4a7fc700 1 -- 172.21.15.40:0/474606337 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2314395761 0 0) 0x7fda340035d0 con 0x7fda440dd280 2023-11-08T08:19:38.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4a7fc700 1 -- 172.21.15.40:0/474606337 >> v2:172.21.15.40:3300/0 conn(0x7fda44036e80 msgr2=0x7fda440ea2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:38.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4a7fc700 1 --2- 172.21.15.40:0/474606337 >> v2:172.21.15.40:3300/0 conn(0x7fda44036e80 0x7fda440ea2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4a7fc700 1 -- 172.21.15.40:0/474606337 >> v1:172.21.15.169:6789/0 conn(0x7fda44050760 legacy=0x7fda44036770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:38.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4a7fc700 1 -- 172.21.15.40:0/474606337 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda440e0430 con 0x7fda440dd280 2023-11-08T08:19:38.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4a7fc700 1 -- 172.21.15.40:0/474606337 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7fda40001670 con 0x7fda440dd280 2023-11-08T08:19:38.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4bfff700 1 --2- 172.21.15.40:0/474606337 >> v2:172.21.15.40:3300/0 conn(0x7fda44036e80 0x7fda440ea2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:38.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4a7fc700 1 -- 172.21.15.40:0/474606337 >> v1:172.21.15.40:6789/0 conn(0x7fda440dd280 legacy=0x7fda440dfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:38.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4a7fc700 1 --2- 172.21.15.40:0/474606337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda34003c00 0x7fda34003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:38.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4a7fc700 1 --2- 172.21.15.40:0/474606337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda34004870 0x7fda34004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:38.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4a7fc700 1 --2- 172.21.15.40:0/474606337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda34005220 0x7fda3400dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:38.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4a7fc700 1 -- 172.21.15.40:0/474606337 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda440dd650 con 0x7fda34003c00 2023-11-08T08:19:38.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4a7fc700 1 -- 172.21.15.40:0/474606337 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3400e310 con 0x7fda34005220 2023-11-08T08:19:38.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.170+0000 7fda4a7fc700 1 -- 172.21.15.40:0/474606337 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3400e4a0 con 0x7fda34004870 2023-11-08T08:19:38.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda4b7fe700 1 --2- 172.21.15.40:0/474606337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda34003c00 0x7fda34003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:38.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda4bfff700 1 --2- 172.21.15.40:0/474606337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda34005220 0x7fda3400dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:38.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda4affd700 1 --2- 172.21.15.40:0/474606337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda34004870 0x7fda34004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:38.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda4bfff700 1 -- 172.21.15.40:0/474606337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda34004870 msgr2=0x7fda34004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:38.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda4bfff700 1 --2- 172.21.15.40:0/474606337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda34004870 0x7fda34004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda4bfff700 1 -- 172.21.15.40:0/474606337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda34003c00 msgr2=0x7fda34003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:38.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda4bfff700 1 --2- 172.21.15.40:0/474606337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda34003c00 0x7fda34003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda4bfff700 1 -- 172.21.15.40:0/474606337 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda340035d0 con 0x7fda34005220 2023-11-08T08:19:38.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda4b7fe700 1 --2- 172.21.15.40:0/474606337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda34003c00 0x7fda34003ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:38.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda4bfff700 1 -- 172.21.15.40:0/474606337 reap_dead start 2023-11-08T08:19:38.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda4bfff700 1 --2- 172.21.15.40:0/474606337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda34005220 0x7fda3400dc40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda4c02cad0 tx=0x7fda340058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a224e49c3fa89c9a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:38.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda4a7fc700 1 -- 172.21.15.40:0/474606337 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c08e070 con 0x7fda34005220 2023-11-08T08:19:38.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda4a7fc700 1 -- 172.21.15.40:0/474606337 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4c072b80 con 0x7fda34005220 2023-11-08T08:19:38.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda4a7fc700 1 -- 172.21.15.40:0/474606337 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c066b40 con 0x7fda34005220 2023-11-08T08:19:38.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda50e2f700 1 -- 172.21.15.40:0/474606337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda34005220 msgr2=0x7fda3400dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:38.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda50e2f700 1 --2- 172.21.15.40:0/474606337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda34005220 0x7fda3400dc40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda4c02cad0 tx=0x7fda340058a0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda50e2f700 1 -- 172.21.15.40:0/474606337 shutdown_connections 2023-11-08T08:19:38.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda50e2f700 1 --2- 172.21.15.40:0/474606337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda34005220 0x7fda3400dc40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda50e2f700 1 -- 172.21.15.40:0/474606337 >> 172.21.15.40:0/474606337 conn(0x7fda44051770 msgr2=0x7fda44051b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:38.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda50e2f700 1 -- 172.21.15.40:0/474606337 shutdown_connections 2023-11-08T08:19:38.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.174+0000 7fda50e2f700 1 -- 172.21.15.40:0/474606337 wait complete. 2023-11-08T08:19:38.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.178+0000 7fda50e2f700 1 Processor -- start 2023-11-08T08:19:38.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.178+0000 7fda50e2f700 1 -- start start 2023-11-08T08:19:38.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda50e2f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda34005220 0x7fda4404e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:38.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda50e2f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda4404a260 0x7fda4404a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:38.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda50e2f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda4404bf80 0x7fda4404c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:38.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda50e2f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4404d930 con 0x7fda4404bf80 2023-11-08T08:19:38.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda50e2f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4404ac10 con 0x7fda4404a260 2023-11-08T08:19:38.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda50e2f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4404ccb0 con 0x7fda34005220 2023-11-08T08:19:38.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda4bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda4404bf80 0x7fda4404c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda4bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda4404bf80 0x7fda4404c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55450/0 (socket says 172.21.15.40:55450) 2023-11-08T08:19:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda4bfff700 1 -- 172.21.15.40:0/3449320468 learned_addr learned my addr 172.21.15.40:0/3449320468 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda4b7fe700 1 --2- 172.21.15.40:0/3449320468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda34005220 0x7fda4404e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda4affd700 1 --2- 172.21.15.40:0/3449320468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda4404a260 0x7fda4404a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:38.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda4bfff700 1 -- 172.21.15.40:0/3449320468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda34005220 msgr2=0x7fda4404e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:38.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda4bfff700 1 --2- 172.21.15.40:0/3449320468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda34005220 0x7fda4404e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda4bfff700 1 -- 172.21.15.40:0/3449320468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda4404a260 msgr2=0x7fda4404a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:38.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda4bfff700 1 --2- 172.21.15.40:0/3449320468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda4404a260 0x7fda4404a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda4bfff700 1 -- 172.21.15.40:0/3449320468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda440e09f0 con 0x7fda4404bf80 2023-11-08T08:19:38.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda4b7fe700 1 --2- 172.21.15.40:0/3449320468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda34005220 0x7fda4404e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:38.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda4affd700 1 --2- 172.21.15.40:0/3449320468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda4404a260 0x7fda4404a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:38.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda4bfff700 1 --2- 172.21.15.40:0/3449320468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda4404bf80 0x7fda4404c3f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fda4c063810 tx=0x7fda4c0678c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:38.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda33fff700 1 -- 172.21.15.40:0/3449320468 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c073270 con 0x7fda4404bf80 2023-11-08T08:19:38.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda33fff700 1 -- 172.21.15.40:0/3449320468 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4c0733d0 con 0x7fda4404bf80 2023-11-08T08:19:38.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda50e2f700 1 -- 172.21.15.40:0/3449320468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda440e0710 con 0x7fda4404bf80 2023-11-08T08:19:38.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda33fff700 1 -- 172.21.15.40:0/3449320468 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c09d460 con 0x7fda4404bf80 2023-11-08T08:19:38.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda50e2f700 1 -- 172.21.15.40:0/3449320468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda44036770 con 0x7fda4404bf80 2023-11-08T08:19:38.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.182+0000 7fda50e2f700 1 -- 172.21.15.40:0/3449320468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda440cda40 con 0x7fda4404bf80 2023-11-08T08:19:38.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.186+0000 7fda33fff700 1 -- 172.21.15.40:0/3449320468 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fda4c0a4030 con 0x7fda4404bf80 2023-11-08T08:19:38.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.186+0000 7fda33fff700 1 --2- 172.21.15.40:0/3449320468 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fda38064f10 0x7fda380673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:38.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.186+0000 7fda33fff700 1 -- 172.21.15.40:0/3449320468 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fda4c072040 con 0x7fda4404bf80 2023-11-08T08:19:38.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.186+0000 7fda4b7fe700 1 --2- 172.21.15.40:0/3449320468 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fda38064f10 0x7fda380673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:38.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.190+0000 7fda4b7fe700 1 --2- 172.21.15.40:0/3449320468 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fda38064f10 0x7fda380673c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fda40000c00 tx=0x7fda40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:38.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.194+0000 7fda33fff700 1 -- 172.21.15.40:0/3449320468 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fda440cda40 con 0x7fda4404bf80 2023-11-08T08:19:38.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.402+0000 7fda50e2f700 1 -- 172.21.15.40:0/3449320468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fda440cda40 con 0x7fda4404bf80 2023-11-08T08:19:38.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.402+0000 7fda33fff700 1 -- 172.21.15.40:0/3449320468 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fda4c0a2040 con 0x7fda4404bf80 2023-11-08T08:19:38.406 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:38.406 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-08T08:19:38.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.406+0000 7fda50e2f700 1 -- 172.21.15.40:0/3449320468 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fda38064f10 msgr2=0x7fda380673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:38.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.406+0000 7fda50e2f700 1 --2- 172.21.15.40:0/3449320468 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fda38064f10 0x7fda380673c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fda40000c00 tx=0x7fda40009040 comp rx=0 tx=0).stop 2023-11-08T08:19:38.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.406+0000 7fda50e2f700 1 -- 172.21.15.40:0/3449320468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda4404bf80 msgr2=0x7fda4404c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:38.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.406+0000 7fda50e2f700 1 --2- 172.21.15.40:0/3449320468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda4404bf80 0x7fda4404c3f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fda4c063810 tx=0x7fda4c0678c0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.406+0000 7fda50e2f700 1 -- 172.21.15.40:0/3449320468 shutdown_connections 2023-11-08T08:19:38.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.406+0000 7fda50e2f700 1 --2- 172.21.15.40:0/3449320468 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fda38064f10 0x7fda380673c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.406+0000 7fda50e2f700 1 --2- 172.21.15.40:0/3449320468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda4404bf80 0x7fda4404c3f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.406+0000 7fda50e2f700 1 --2- 172.21.15.40:0/3449320468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda4404a260 0x7fda4404a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.406+0000 7fda50e2f700 1 --2- 172.21.15.40:0/3449320468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda34005220 0x7fda4404e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.406+0000 7fda50e2f700 1 -- 172.21.15.40:0/3449320468 >> 172.21.15.40:0/3449320468 conn(0x7fda44051770 msgr2=0x7fda440dc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:38.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.406+0000 7fda50e2f700 1 -- 172.21.15.40:0/3449320468 shutdown_connections 2023-11-08T08:19:38.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.406+0000 7fda50e2f700 1 -- 172.21.15.40:0/3449320468 wait complete. 2023-11-08T08:19:38.433 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-08T08:19:38.433 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:19:38.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.606+0000 7f103df13700 1 Processor -- start 2023-11-08T08:19:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.606+0000 7f103df13700 1 -- start start 2023-11-08T08:19:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.606+0000 7f103df13700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f103815a480 0x7f10380cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.606+0000 7f103df13700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1038069d60 0x7f103806a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.606+0000 7f103df13700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f103815a850 con 0x7f10380c8d10 2023-11-08T08:19:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.606+0000 7f103df13700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10380ce060 con 0x7f103815a480 2023-11-08T08:19:38.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.606+0000 7f103df13700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10380ce300 con 0x7f1038069d60 2023-11-08T08:19:38.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.606+0000 7f102ffff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f10380c8d10 0x7f10380cc7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40610/0 (socket says 172.21.15.40:40610) 2023-11-08T08:19:38.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.606+0000 7f102ffff700 1 -- 172.21.15.40:0/2455606760 learned_addr learned my addr 172.21.15.40:0/2455606760 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:38.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.606+0000 7f10377fe700 1 --2- 172.21.15.40:0/2455606760 >> v2:172.21.15.169:3300/0 conn(0x7f103815a480 0x7f10380cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:38.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.606+0000 7f1037fff700 1 --2- 172.21.15.40:0/2455606760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1038069d60 0x7f103806a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:38.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.606+0000 7f1036ffd700 1 -- 172.21.15.40:0/2455606760 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673029797 0 0) 0x7f103815a850 con 0x7f10380c8d10 2023-11-08T08:19:38.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.606+0000 7f10377fe700 1 -- 172.21.15.40:0/2455606760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1038069d60 msgr2=0x7f103806a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:38.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.606+0000 7f10377fe700 1 --2- 172.21.15.40:0/2455606760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1038069d60 0x7f103806a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.606+0000 7f10377fe700 1 -- 172.21.15.40:0/2455606760 >> v1:172.21.15.40:6789/0 conn(0x7f10380c8d10 legacy=0x7f10380cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:38.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.606+0000 7f10377fe700 1 -- 172.21.15.40:0/2455606760 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10380cada0 con 0x7f103815a480 2023-11-08T08:19:38.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.610+0000 7f10377fe700 1 --2- 172.21.15.40:0/2455606760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f103815a480 0x7f10380cc290 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1028009850 tx=0x7f1028009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3786515f4878205c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:38.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.610+0000 7f1036ffd700 1 -- 172.21.15.40:0/2455606760 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028011830 con 0x7f103815a480 2023-11-08T08:19:38.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.610+0000 7f1036ffd700 1 -- 172.21.15.40:0/2455606760 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1028011990 con 0x7f103815a480 2023-11-08T08:19:38.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.610+0000 7f1036ffd700 1 -- 172.21.15.40:0/2455606760 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028011d40 con 0x7f103815a480 2023-11-08T08:19:38.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.610+0000 7f103df13700 1 -- 172.21.15.40:0/2455606760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f103815a480 msgr2=0x7f10380cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:38.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.610+0000 7f103df13700 1 --2- 172.21.15.40:0/2455606760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f103815a480 0x7f10380cc290 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1028009850 tx=0x7f1028009b60 comp rx=0 tx=0).stop 2023-11-08T08:19:38.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.610+0000 7f103df13700 1 -- 172.21.15.40:0/2455606760 shutdown_connections 2023-11-08T08:19:38.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.610+0000 7f103df13700 1 --2- 172.21.15.40:0/2455606760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1038069d60 0x7f103806a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.610+0000 7f103df13700 1 --2- 172.21.15.40:0/2455606760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f103815a480 0x7f10380cc290 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.610+0000 7f103df13700 1 -- 172.21.15.40:0/2455606760 >> 172.21.15.40:0/2455606760 conn(0x7f10380cf350 msgr2=0x7f10380b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:38.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.610+0000 7f103df13700 1 -- 172.21.15.40:0/2455606760 shutdown_connections 2023-11-08T08:19:38.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.610+0000 7f103df13700 1 -- 172.21.15.40:0/2455606760 wait complete. 2023-11-08T08:19:38.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f103df13700 1 Processor -- start 2023-11-08T08:19:38.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f103df13700 1 -- start start 2023-11-08T08:19:38.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f103df13700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1038069d60 0x7f103806c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:38.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f103df13700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10380c8d10 0x7f10380c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:38.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f103df13700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f103815a480 0x7f1038153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:38.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f103df13700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10380c6ab0 con 0x7f103815a480 2023-11-08T08:19:38.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f103df13700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10380c5cd0 con 0x7f1038069d60 2023-11-08T08:19:38.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f103df13700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10380c5f30 con 0x7f10380c8d10 2023-11-08T08:19:38.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f1037fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f103815a480 0x7f1038153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:38.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f1037fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f103815a480 0x7f1038153c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55466/0 (socket says 172.21.15.40:55466) 2023-11-08T08:19:38.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f1037fff700 1 -- 172.21.15.40:0/1160649852 learned_addr learned my addr 172.21.15.40:0/1160649852 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:38.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f102ffff700 1 --2- 172.21.15.40:0/1160649852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10380c8d10 0x7f10380c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:38.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f10377fe700 1 --2- 172.21.15.40:0/1160649852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1038069d60 0x7f103806c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:38.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f1037fff700 1 -- 172.21.15.40:0/1160649852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10380c8d10 msgr2=0x7f10380c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:38.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f1037fff700 1 --2- 172.21.15.40:0/1160649852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10380c8d10 0x7f10380c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f1037fff700 1 -- 172.21.15.40:0/1160649852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1038069d60 msgr2=0x7f103806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:38.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f1037fff700 1 --2- 172.21.15.40:0/1160649852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1038069d60 0x7f103806c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f1037fff700 1 -- 172.21.15.40:0/1160649852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10280094b0 con 0x7f103815a480 2023-11-08T08:19:38.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f10377fe700 1 --2- 172.21.15.40:0/1160649852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1038069d60 0x7f103806c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:38.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f1037fff700 1 --2- 172.21.15.40:0/1160649852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f103815a480 0x7f1038153c90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f103000efc0 tx=0x7f103000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:38.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f1034ff9700 1 -- 172.21.15.40:0/1160649852 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1030009ba0 con 0x7f103815a480 2023-11-08T08:19:38.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f1034ff9700 1 -- 172.21.15.40:0/1160649852 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103000f040 con 0x7f103815a480 2023-11-08T08:19:38.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f103df13700 1 -- 172.21.15.40:0/1160649852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10380add50 con 0x7f103815a480 2023-11-08T08:19:38.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.614+0000 7f1034ff9700 1 -- 172.21.15.40:0/1160649852 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1030014480 con 0x7f103815a480 2023-11-08T08:19:38.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.618+0000 7f103df13700 1 -- 172.21.15.40:0/1160649852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10380caa30 con 0x7f103815a480 2023-11-08T08:19:38.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.618+0000 7f103df13700 1 -- 172.21.15.40:0/1160649852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10380ca6e0 con 0x7f103815a480 2023-11-08T08:19:38.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.618+0000 7f1034ff9700 1 -- 172.21.15.40:0/1160649852 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f10300145e0 con 0x7f103815a480 2023-11-08T08:19:38.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.622+0000 7f1034ff9700 1 --2- 172.21.15.40:0/1160649852 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1018064ee0 0x7f1018067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:38.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.622+0000 7f1034ff9700 1 -- 172.21.15.40:0/1160649852 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1030082f40 con 0x7f103815a480 2023-11-08T08:19:38.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.622+0000 7f10377fe700 1 --2- 172.21.15.40:0/1160649852 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1018064ee0 0x7f1018067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:38.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.622+0000 7f10377fe700 1 --2- 172.21.15.40:0/1160649852 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1018064ee0 0x7f1018067390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f10280090b0 tx=0x7f102801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:38.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.630+0000 7f1034ff9700 1 -- 172.21.15.40:0/1160649852 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1030050d00 con 0x7f103815a480 2023-11-08T08:19:38.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.798+0000 7f103df13700 1 -- 172.21.15.40:0/1160649852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10381f21b0 con 0x7f103815a480 2023-11-08T08:19:38.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.802+0000 7f1034ff9700 1 -- 172.21.15.40:0/1160649852 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19652 (secure 0 0 0) 0x7f1030018090 con 0x7f103815a480 2023-11-08T08:19:38.806 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:38.806 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":28,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:08.980158+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:19:38.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.802+0000 7f103df13700 1 -- 172.21.15.40:0/1160649852 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1018064ee0 msgr2=0x7f1018067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:38.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.802+0000 7f103df13700 1 --2- 172.21.15.40:0/1160649852 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1018064ee0 0x7f1018067390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f10280090b0 tx=0x7f102801c040 comp rx=0 tx=0).stop 2023-11-08T08:19:38.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.806+0000 7f103df13700 1 -- 172.21.15.40:0/1160649852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f103815a480 msgr2=0x7f1038153c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:38.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.806+0000 7f103df13700 1 --2- 172.21.15.40:0/1160649852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f103815a480 0x7f1038153c90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f103000efc0 tx=0x7f103000c3a0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.806+0000 7f103df13700 1 -- 172.21.15.40:0/1160649852 shutdown_connections 2023-11-08T08:19:38.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.806+0000 7f103df13700 1 --2- 172.21.15.40:0/1160649852 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1018064ee0 0x7f1018067390 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.806+0000 7f103df13700 1 --2- 172.21.15.40:0/1160649852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f103815a480 0x7f1038153c90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.806+0000 7f103df13700 1 --2- 172.21.15.40:0/1160649852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10380c8d10 0x7f10380c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.806+0000 7f103df13700 1 --2- 172.21.15.40:0/1160649852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1038069d60 0x7f103806c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:38.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.806+0000 7f103df13700 1 -- 172.21.15.40:0/1160649852 >> 172.21.15.40:0/1160649852 conn(0x7f10380cf350 msgr2=0x7f10380b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:38.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.806+0000 7f103df13700 1 -- 172.21.15.40:0/1160649852 shutdown_connections 2023-11-08T08:19:38.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:38.806+0000 7f103df13700 1 -- 172.21.15.40:0/1160649852 wait complete. 2023-11-08T08:19:38.833 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T08:19:02.627691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T08:19:03.473821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T08:19:06.539942+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T08:19:38.834 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:19:39.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.014+0000 7f846ef9a700 1 Processor -- start 2023-11-08T08:19:39.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.014+0000 7f846ef9a700 1 -- start start 2023-11-08T08:19:39.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.014+0000 7f846ef9a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8468070620 0x7f84680b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:39.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.018+0000 7f846ef9a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f846815c520 con 0x7f8468069bf0 2023-11-08T08:19:39.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.018+0000 7f846ef9a700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f846806cc50 con 0x7f846815c150 2023-11-08T08:19:39.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.018+0000 7f846ef9a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84680b4240 con 0x7f8468070620 2023-11-08T08:19:39.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.018+0000 7f846d797700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8468070620 0x7f84680b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:39.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.018+0000 7f846e799700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8468069bf0 0x7f8468069ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40626/0 (socket says 172.21.15.40:40626) 2023-11-08T08:19:39.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.018+0000 7f846d797700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8468070620 0x7f84680b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55472/0 (socket says 172.21.15.40:55472) 2023-11-08T08:19:39.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.018+0000 7f846e799700 1 -- 172.21.15.40:0/831506199 learned_addr learned my addr 172.21.15.40:0/831506199 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:39.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.018+0000 7f846d797700 1 -- 172.21.15.40:0/831506199 >> v1:172.21.15.169:6789/0 conn(0x7f846815c150 legacy=0x7f846806ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:39.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.018+0000 7f846d797700 1 -- 172.21.15.40:0/831506199 >> v1:172.21.15.40:6789/0 conn(0x7f8468069bf0 legacy=0x7f8468069ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:39.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.018+0000 7f846d797700 1 -- 172.21.15.40:0/831506199 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84680d2450 con 0x7f8468070620 2023-11-08T08:19:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.018+0000 7f846d797700 1 --2- 172.21.15.40:0/831506199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8468070620 0x7f84680b3d00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8458009840 tx=0x7f8458009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcd5dc3685ae688 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.018+0000 7f846cf96700 1 -- 172.21.15.40:0/831506199 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8458005e50 con 0x7f8468070620 2023-11-08T08:19:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.018+0000 7f846cf96700 1 -- 172.21.15.40:0/831506199 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8458006030 con 0x7f8468070620 2023-11-08T08:19:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.018+0000 7f846cf96700 1 -- 172.21.15.40:0/831506199 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84580063c0 con 0x7f8468070620 2023-11-08T08:19:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.018+0000 7f846ef9a700 1 -- 172.21.15.40:0/831506199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8468070620 msgr2=0x7f84680b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:39.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.018+0000 7f846ef9a700 1 --2- 172.21.15.40:0/831506199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8468070620 0x7f84680b3d00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8458009840 tx=0x7f8458009b50 comp rx=0 tx=0).stop 2023-11-08T08:19:39.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.022+0000 7f846ef9a700 1 -- 172.21.15.40:0/831506199 shutdown_connections 2023-11-08T08:19:39.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.022+0000 7f846ef9a700 1 --2- 172.21.15.40:0/831506199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8468070620 0x7f84680b3d00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.022+0000 7f846ef9a700 1 -- 172.21.15.40:0/831506199 >> 172.21.15.40:0/831506199 conn(0x7f84680d3680 msgr2=0x7f84680cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:39.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.022+0000 7f846ef9a700 1 -- 172.21.15.40:0/831506199 shutdown_connections 2023-11-08T08:19:39.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.022+0000 7f846ef9a700 1 -- 172.21.15.40:0/831506199 wait complete. 2023-11-08T08:19:39.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.022+0000 7f846ef9a700 1 Processor -- start 2023-11-08T08:19:39.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.022+0000 7f846ef9a700 1 -- start start 2023-11-08T08:19:39.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f846ef9a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8468069bf0 0x7f846814c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:39.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f846ef9a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8468070620 0x7f846815ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:39.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f846ef9a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f846815c150 0x7f8468159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:39.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f846ef9a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84680cdff0 con 0x7f8468070620 2023-11-08T08:19:39.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f846ef9a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8468158d50 con 0x7f846815c150 2023-11-08T08:19:39.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f846ef9a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8468159030 con 0x7f8468069bf0 2023-11-08T08:19:39.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f846d797700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8468070620 0x7f846815ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:39.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f846d797700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8468070620 0x7f846815ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55480/0 (socket says 172.21.15.40:55480) 2023-11-08T08:19:39.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f846d797700 1 -- 172.21.15.40:0/3917198962 learned_addr learned my addr 172.21.15.40:0/3917198962 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:39.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f846e799700 1 --2- 172.21.15.40:0/3917198962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f846815c150 0x7f8468159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:39.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f846df98700 1 --2- 172.21.15.40:0/3917198962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8468069bf0 0x7f846814c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:39.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f846d797700 1 -- 172.21.15.40:0/3917198962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8468069bf0 msgr2=0x7f846814c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:39.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f846d797700 1 --2- 172.21.15.40:0/3917198962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8468069bf0 0x7f846814c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f846d797700 1 -- 172.21.15.40:0/3917198962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f846815c150 msgr2=0x7f8468159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:39.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f846d797700 1 --2- 172.21.15.40:0/3917198962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f846815c150 0x7f8468159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f846d797700 1 -- 172.21.15.40:0/3917198962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8458009450 con 0x7f8468070620 2023-11-08T08:19:39.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f846d797700 1 --2- 172.21.15.40:0/3917198962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8468070620 0x7f846815ffb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8458009b90 tx=0x7f8458004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:39.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f845e7fc700 1 -- 172.21.15.40:0/3917198962 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8458003ad0 con 0x7f8468070620 2023-11-08T08:19:39.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f845e7fc700 1 -- 172.21.15.40:0/3917198962 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8458003c30 con 0x7f8468070620 2023-11-08T08:19:39.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f846ef9a700 1 -- 172.21.15.40:0/3917198962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84680d05e0 con 0x7f8468070620 2023-11-08T08:19:39.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f845e7fc700 1 -- 172.21.15.40:0/3917198962 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84580061a0 con 0x7f8468070620 2023-11-08T08:19:39.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.026+0000 7f846ef9a700 1 -- 172.21.15.40:0/3917198962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84680d2730 con 0x7f8468070620 2023-11-08T08:19:39.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.030+0000 7f846ef9a700 1 -- 172.21.15.40:0/3917198962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84680d2140 con 0x7f8468070620 2023-11-08T08:19:39.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.030+0000 7f845e7fc700 1 -- 172.21.15.40:0/3917198962 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f845800f050 con 0x7f8468070620 2023-11-08T08:19:39.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.030+0000 7f845e7fc700 1 --2- 172.21.15.40:0/3917198962 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f844c064ee0 0x7f844c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:39.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.030+0000 7f845e7fc700 1 -- 172.21.15.40:0/3917198962 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8458083cd0 con 0x7f8468070620 2023-11-08T08:19:39.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.034+0000 7f846df98700 1 --2- 172.21.15.40:0/3917198962 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f844c064ee0 0x7f844c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:39.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.034+0000 7f846df98700 1 --2- 172.21.15.40:0/3917198962 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f844c064ee0 0x7f844c067390 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8464000f60 tx=0x7f8464009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:39.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.038+0000 7f845e7fc700 1 -- 172.21.15.40:0/3917198962 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f8458051b00 con 0x7f8468070620 2023-11-08T08:19:39.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.214+0000 7f846ef9a700 1 -- 172.21.15.40:0/3917198962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84680d0150 con 0x7f8468070620 2023-11-08T08:19:39.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.214+0000 7f845e7fc700 1 -- 172.21.15.40:0/3917198962 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f845802a0a0 con 0x7f8468070620 2023-11-08T08:19:39.218 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:19:39.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.214+0000 7f846ef9a700 1 -- 172.21.15.40:0/3917198962 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f844c064ee0 msgr2=0x7f844c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:39.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.214+0000 7f846ef9a700 1 --2- 172.21.15.40:0/3917198962 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f844c064ee0 0x7f844c067390 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8464000f60 tx=0x7f8464009040 comp rx=0 tx=0).stop 2023-11-08T08:19:39.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.214+0000 7f846ef9a700 1 -- 172.21.15.40:0/3917198962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8468070620 msgr2=0x7f846815ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:39.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.214+0000 7f846ef9a700 1 --2- 172.21.15.40:0/3917198962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8468070620 0x7f846815ffb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8458009b90 tx=0x7f8458004770 comp rx=0 tx=0).stop 2023-11-08T08:19:39.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.214+0000 7f846ef9a700 1 -- 172.21.15.40:0/3917198962 shutdown_connections 2023-11-08T08:19:39.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.218+0000 7f846ef9a700 1 --2- 172.21.15.40:0/3917198962 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f844c064ee0 0x7f844c067390 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.218+0000 7f846ef9a700 1 --2- 172.21.15.40:0/3917198962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f846815c150 0x7f8468159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.218+0000 7f846ef9a700 1 --2- 172.21.15.40:0/3917198962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8468070620 0x7f846815ffb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.218+0000 7f846ef9a700 1 --2- 172.21.15.40:0/3917198962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8468069bf0 0x7f846814c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.218+0000 7f846ef9a700 1 -- 172.21.15.40:0/3917198962 >> 172.21.15.40:0/3917198962 conn(0x7f84680d3680 msgr2=0x7f84680b7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:39.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.218+0000 7f846ef9a700 1 -- 172.21.15.40:0/3917198962 shutdown_connections 2023-11-08T08:19:39.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.218+0000 7f846ef9a700 1 -- 172.21.15.40:0/3917198962 wait complete. 2023-11-08T08:19:39.241 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:19:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.422+0000 7fdcdb59e700 1 Processor -- start 2023-11-08T08:19:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.422+0000 7fdcdb59e700 1 -- start start 2023-11-08T08:19:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.422+0000 7fdcdb59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcdc159f10 0x7fdcdc070a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.422+0000 7fdcdb59e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdcdc0c8c90 0x7fdcdc06b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.422+0000 7fdcdb59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcdc070600 con 0x7fdcdc15a2e0 2023-11-08T08:19:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.422+0000 7fdcdb59e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcdc0c9100 con 0x7fdcdc0c8c90 2023-11-08T08:19:39.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.422+0000 7fdcdb59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc06b840 con 0x7fdcdc159f10 2023-11-08T08:19:39.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.422+0000 7fdcdad9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdcdc15a2e0 0x7fdcdc15a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40638/0 (socket says 172.21.15.40:40638) 2023-11-08T08:19:39.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.422+0000 7fdcdad9d700 1 -- 172.21.15.40:0/3121779264 learned_addr learned my addr 172.21.15.40:0/3121779264 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:39.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.422+0000 7fdcd1d9b700 1 --2- 172.21.15.40:0/3121779264 >> v2:172.21.15.169:3300/0 conn(0x7fdcdc0c8c90 0x7fdcdc06b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:39.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.422+0000 7fdcda59c700 1 --2- 172.21.15.40:0/3121779264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcdc159f10 0x7fdcdc070a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:39.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.426+0000 7fdcd1d9b700 1 -- 172.21.15.40:0/3121779264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcdc159f10 msgr2=0x7fdcdc070a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:39.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.426+0000 7fdcd1d9b700 1 --2- 172.21.15.40:0/3121779264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcdc159f10 0x7fdcdc070a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.426+0000 7fdcd1d9b700 1 -- 172.21.15.40:0/3121779264 >> v1:172.21.15.40:6789/0 conn(0x7fdcdc15a2e0 legacy=0x7fdcdc15a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:39.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.426+0000 7fdcd9d9b700 1 -- 172.21.15.40:0/3121779264 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 862663838 0 0) 0x7fdcdc070600 con 0x7fdcdc15a2e0 2023-11-08T08:19:39.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.426+0000 7fdcd1d9b700 1 -- 172.21.15.40:0/3121779264 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc15ae30 con 0x7fdcdc0c8c90 2023-11-08T08:19:39.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.426+0000 7fdcd1d9b700 1 --2- 172.21.15.40:0/3121779264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcdc0c8c90 0x7fdcdc06b300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009850 tx=0x7fdcc4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbfe817d13d87b66 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:39.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.426+0000 7fdcd9d9b700 1 -- 172.21.15.40:0/3121779264 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4011830 con 0x7fdcdc0c8c90 2023-11-08T08:19:39.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.426+0000 7fdcd9d9b700 1 -- 172.21.15.40:0/3121779264 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc4011990 con 0x7fdcdc0c8c90 2023-11-08T08:19:39.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.426+0000 7fdcd9d9b700 1 -- 172.21.15.40:0/3121779264 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4011d40 con 0x7fdcdc0c8c90 2023-11-08T08:19:39.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.426+0000 7fdcdb59e700 1 -- 172.21.15.40:0/3121779264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcdc0c8c90 msgr2=0x7fdcdc06b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:39.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.426+0000 7fdcdb59e700 1 --2- 172.21.15.40:0/3121779264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcdc0c8c90 0x7fdcdc06b300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009850 tx=0x7fdcc4009b60 comp rx=0 tx=0).stop 2023-11-08T08:19:39.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.426+0000 7fdcdb59e700 1 -- 172.21.15.40:0/3121779264 shutdown_connections 2023-11-08T08:19:39.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.426+0000 7fdcdb59e700 1 --2- 172.21.15.40:0/3121779264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcdc0c8c90 0x7fdcdc06b300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.426+0000 7fdcdb59e700 1 --2- 172.21.15.40:0/3121779264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcdc159f10 0x7fdcdc070a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.426+0000 7fdcdb59e700 1 -- 172.21.15.40:0/3121779264 >> 172.21.15.40:0/3121779264 conn(0x7fdcdc0cf280 msgr2=0x7fdcdc0cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:39.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.426+0000 7fdcdb59e700 1 -- 172.21.15.40:0/3121779264 shutdown_connections 2023-11-08T08:19:39.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.426+0000 7fdcdb59e700 1 -- 172.21.15.40:0/3121779264 wait complete. 2023-11-08T08:19:39.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.430+0000 7fdcdb59e700 1 Processor -- start 2023-11-08T08:19:39.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcdb59e700 1 -- start start 2023-11-08T08:19:39.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcdb59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcdc0c8c90 0x7fdcdc0b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:39.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcdb59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcdc159f10 0x7fdcdc15cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:39.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcdb59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcdc15a2e0 0x7fdcdc14d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:39.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcdb59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc15d100 con 0x7fdcdc15a2e0 2023-11-08T08:19:39.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcdb59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc153a30 con 0x7fdcdc0c8c90 2023-11-08T08:19:39.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcdb59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc15c730 con 0x7fdcdc159f10 2023-11-08T08:19:39.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcdad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcdc15a2e0 0x7fdcdc14d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:39.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcdad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcdc15a2e0 0x7fdcdc14d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55486/0 (socket says 172.21.15.40:55486) 2023-11-08T08:19:39.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcdad9d700 1 -- 172.21.15.40:0/2855461187 learned_addr learned my addr 172.21.15.40:0/2855461187 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:39.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcd1d9b700 1 --2- 172.21.15.40:0/2855461187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcdc159f10 0x7fdcdc15cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:39.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcda59c700 1 --2- 172.21.15.40:0/2855461187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcdc0c8c90 0x7fdcdc0b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:39.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcdad9d700 1 -- 172.21.15.40:0/2855461187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcdc159f10 msgr2=0x7fdcdc15cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:39.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcdad9d700 1 --2- 172.21.15.40:0/2855461187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcdc159f10 0x7fdcdc15cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcdad9d700 1 -- 172.21.15.40:0/2855461187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcdc0c8c90 msgr2=0x7fdcdc0b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:39.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcdad9d700 1 --2- 172.21.15.40:0/2855461187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcdc0c8c90 0x7fdcdc0b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcdad9d700 1 -- 172.21.15.40:0/2855461187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc40094b0 con 0x7fdcdc15a2e0 2023-11-08T08:19:39.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcd1d9b700 1 --2- 172.21.15.40:0/2855461187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcdc159f10 0x7fdcdc15cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:39.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcdad9d700 1 --2- 172.21.15.40:0/2855461187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcdc15a2e0 0x7fdcdc14d200 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400cb90 tx=0x7fdcd400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:39.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcd37fe700 1 -- 172.21.15.40:0/2855461187 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd400ab30 con 0x7fdcdc15a2e0 2023-11-08T08:19:39.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcd37fe700 1 -- 172.21.15.40:0/2855461187 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd400ac90 con 0x7fdcdc15a2e0 2023-11-08T08:19:39.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcd37fe700 1 -- 172.21.15.40:0/2855461187 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd40070d0 con 0x7fdcdc15a2e0 2023-11-08T08:19:39.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.434+0000 7fdcdb59e700 1 -- 172.21.15.40:0/2855461187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcdc06ffb0 con 0x7fdcdc15a2e0 2023-11-08T08:19:39.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.438+0000 7fdcdb59e700 1 -- 172.21.15.40:0/2855461187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcdc0afcc0 con 0x7fdcdc15a2e0 2023-11-08T08:19:39.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.438+0000 7fdcdb59e700 1 -- 172.21.15.40:0/2855461187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcdc070290 con 0x7fdcdc15a2e0 2023-11-08T08:19:39.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.438+0000 7fdcd37fe700 1 -- 172.21.15.40:0/2855461187 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fdcd4010070 con 0x7fdcdc15a2e0 2023-11-08T08:19:39.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.442+0000 7fdcd37fe700 1 --2- 172.21.15.40:0/2855461187 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdcbc064ee0 0x7fdcbc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:39.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.442+0000 7fdcd37fe700 1 -- 172.21.15.40:0/2855461187 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdcd4081c80 con 0x7fdcdc15a2e0 2023-11-08T08:19:39.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.442+0000 7fdcda59c700 1 --2- 172.21.15.40:0/2855461187 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdcbc064ee0 0x7fdcbc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:39.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.442+0000 7fdcda59c700 1 --2- 172.21.15.40:0/2855461187 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdcbc064ee0 0x7fdcbc067390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009b80 tx=0x7fdccc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:39.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.446+0000 7fdcd37fe700 1 -- 172.21.15.40:0/2855461187 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fdcd404f9e0 con 0x7fdcdc15a2e0 2023-11-08T08:19:39.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.622+0000 7fdcdb59e700 1 -- 172.21.15.40:0/2855461187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdcdc06cf30 con 0x7fdcdc15a2e0 2023-11-08T08:19:39.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.622+0000 7fdcd37fe700 1 -- 172.21.15.40:0/2855461187 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdcd4018020 con 0x7fdcdc15a2e0 2023-11-08T08:19:39.627 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:19:39.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.626+0000 7fdcdb59e700 1 -- 172.21.15.40:0/2855461187 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdcbc064ee0 msgr2=0x7fdcbc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:39.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.626+0000 7fdcdb59e700 1 --2- 172.21.15.40:0/2855461187 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdcbc064ee0 0x7fdcbc067390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009b80 tx=0x7fdccc008040 comp rx=0 tx=0).stop 2023-11-08T08:19:39.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.626+0000 7fdcdb59e700 1 -- 172.21.15.40:0/2855461187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcdc15a2e0 msgr2=0x7fdcdc14d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:39.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.626+0000 7fdcdb59e700 1 --2- 172.21.15.40:0/2855461187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcdc15a2e0 0x7fdcdc14d200 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400cb90 tx=0x7fdcd400cf50 comp rx=0 tx=0).stop 2023-11-08T08:19:39.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.626+0000 7fdcdb59e700 1 -- 172.21.15.40:0/2855461187 shutdown_connections 2023-11-08T08:19:39.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.626+0000 7fdcdb59e700 1 --2- 172.21.15.40:0/2855461187 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdcbc064ee0 0x7fdcbc067390 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.626+0000 7fdcdb59e700 1 --2- 172.21.15.40:0/2855461187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcdc15a2e0 0x7fdcdc14d200 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.626+0000 7fdcdb59e700 1 --2- 172.21.15.40:0/2855461187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdcdc159f10 0x7fdcdc15cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.626+0000 7fdcdb59e700 1 --2- 172.21.15.40:0/2855461187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcdc0c8c90 0x7fdcdc0b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.626+0000 7fdcdb59e700 1 -- 172.21.15.40:0/2855461187 >> 172.21.15.40:0/2855461187 conn(0x7fdcdc0cf280 msgr2=0x7fdcdc0cb560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:39.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.626+0000 7fdcdb59e700 1 -- 172.21.15.40:0/2855461187 shutdown_connections 2023-11-08T08:19:39.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.626+0000 7fdcdb59e700 1 -- 172.21.15.40:0/2855461187 wait complete. 2023-11-08T08:19:39.651 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T08:19:39.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.834+0000 7f52acd69700 1 Processor -- start 2023-11-08T08:19:39.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.834+0000 7f52acd69700 1 -- start start 2023-11-08T08:19:39.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.834+0000 7f52acd69700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f52a815d6b0 0x7f52a80ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:39.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.834+0000 7f52acd69700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f52a8160770 0x7f52a80b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:39.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.834+0000 7f52acd69700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52a815da80 con 0x7f52a80b11a0 2023-11-08T08:19:39.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.834+0000 7f52acd69700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a80b1b80 con 0x7f52a8160770 2023-11-08T08:19:39.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.834+0000 7f52acd69700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a815ce90 con 0x7f52a815d6b0 2023-11-08T08:19:39.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.838+0000 7f52a7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f52a8160770 0x7f52a80b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:39.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.838+0000 7f52a7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f52a8160770 0x7f52a80b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55494/0 (socket says 172.21.15.40:55494) 2023-11-08T08:19:39.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.838+0000 7f52a7fff700 1 -- 172.21.15.40:0/845257049 learned_addr learned my addr 172.21.15.40:0/845257049 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:39.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.838+0000 7f52a77fe700 1 --2- 172.21.15.40:0/845257049 >> v2:172.21.15.169:3300/0 conn(0x7f52a815d6b0 0x7f52a80ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:39.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.838+0000 7f52a7fff700 1 -- 172.21.15.40:0/845257049 >> v2:172.21.15.169:3300/0 conn(0x7f52a815d6b0 msgr2=0x7f52a80ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:39.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.838+0000 7f52a7fff700 1 --2- 172.21.15.40:0/845257049 >> v2:172.21.15.169:3300/0 conn(0x7f52a815d6b0 0x7f52a80ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.838+0000 7f52a7fff700 1 -- 172.21.15.40:0/845257049 >> v1:172.21.15.169:6789/0 conn(0x7f52a80b11a0 legacy=0x7f52a8160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:39.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.838+0000 7f52a7fff700 1 -- 172.21.15.40:0/845257049 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a80c9e10 con 0x7f52a8160770 2023-11-08T08:19:39.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.838+0000 7f52a7fff700 1 --2- 172.21.15.40:0/845257049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a8160770 0x7f52a80b1640 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f52a806d2c0 tx=0x7f52a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98097668ca8964f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:39.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.842+0000 7f52a67fc700 1 -- 172.21.15.40:0/845257049 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a000e070 con 0x7f52a8160770 2023-11-08T08:19:39.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.842+0000 7f52a67fc700 1 -- 172.21.15.40:0/845257049 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a000b910 con 0x7f52a8160770 2023-11-08T08:19:39.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.842+0000 7f52a67fc700 1 -- 172.21.15.40:0/845257049 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a000bcc0 con 0x7f52a8160770 2023-11-08T08:19:39.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.842+0000 7f52acd69700 1 -- 172.21.15.40:0/845257049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a8160770 msgr2=0x7f52a80b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:39.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.842+0000 7f52acd69700 1 --2- 172.21.15.40:0/845257049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a8160770 0x7f52a80b1640 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f52a806d2c0 tx=0x7f52a000b330 comp rx=0 tx=0).stop 2023-11-08T08:19:39.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.842+0000 7f52acd69700 1 -- 172.21.15.40:0/845257049 shutdown_connections 2023-11-08T08:19:39.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.842+0000 7f52acd69700 1 --2- 172.21.15.40:0/845257049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a8160770 0x7f52a80b1640 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.842+0000 7f52acd69700 1 --2- 172.21.15.40:0/845257049 >> v2:172.21.15.169:3300/0 conn(0x7f52a815d6b0 0x7f52a80ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.842+0000 7f52acd69700 1 -- 172.21.15.40:0/845257049 >> 172.21.15.40:0/845257049 conn(0x7f52a80cc1a0 msgr2=0x7f52a80cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.842+0000 7f52acd69700 1 -- 172.21.15.40:0/845257049 shutdown_connections 2023-11-08T08:19:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.842+0000 7f52acd69700 1 -- 172.21.15.40:0/845257049 wait complete. 2023-11-08T08:19:39.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.846+0000 7f52acd69700 1 Processor -- start 2023-11-08T08:19:39.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.846+0000 7f52acd69700 1 -- start start 2023-11-08T08:19:39.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f52acd69700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a80b11a0 0x7f52a814cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f52acd69700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52a815d6b0 0x7f52a806a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f52acd69700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52a8160770 0x7f52a815e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f52acd69700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a806b770 con 0x7f52a80b11a0 2023-11-08T08:19:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f52acd69700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a806ba50 con 0x7f52a8160770 2023-11-08T08:19:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f52acd69700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a806bd30 con 0x7f52a815d6b0 2023-11-08T08:19:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f52a77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a80b11a0 0x7f52a814cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f52a77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a80b11a0 0x7f52a814cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55506/0 (socket says 172.21.15.40:55506) 2023-11-08T08:19:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f52a77fe700 1 -- 172.21.15.40:0/2724713256 learned_addr learned my addr 172.21.15.40:0/2724713256 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f52a7fff700 1 --2- 172.21.15.40:0/2724713256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52a8160770 0x7f52a815e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:39.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f52a6ffd700 1 --2- 172.21.15.40:0/2724713256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52a815d6b0 0x7f52a806a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:39.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f52a77fe700 1 -- 172.21.15.40:0/2724713256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52a815d6b0 msgr2=0x7f52a806a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f52a77fe700 1 --2- 172.21.15.40:0/2724713256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52a815d6b0 0x7f52a806a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f52a77fe700 1 -- 172.21.15.40:0/2724713256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52a8160770 msgr2=0x7f52a815e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f52a77fe700 1 --2- 172.21.15.40:0/2724713256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52a8160770 0x7f52a815e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f52a77fe700 1 -- 172.21.15.40:0/2724713256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a0009b30 con 0x7f52a80b11a0 2023-11-08T08:19:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f52a77fe700 1 --2- 172.21.15.40:0/2724713256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a80b11a0 0x7f52a814cf10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f529800e9d0 tx=0x7f529800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f528ffff700 1 -- 172.21.15.40:0/2724713256 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5298009720 con 0x7f52a80b11a0 2023-11-08T08:19:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f528ffff700 1 -- 172.21.15.40:0/2724713256 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529800ee70 con 0x7f52a80b11a0 2023-11-08T08:19:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f528ffff700 1 -- 172.21.15.40:0/2724713256 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529800ca50 con 0x7f52a80b11a0 2023-11-08T08:19:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.850+0000 7f52acd69700 1 -- 172.21.15.40:0/2724713256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a80ca0b0 con 0x7f52a80b11a0 2023-11-08T08:19:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.854+0000 7f52acd69700 1 -- 172.21.15.40:0/2724713256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a8160c70 con 0x7f52a80b11a0 2023-11-08T08:19:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.854+0000 7f52acd69700 1 -- 172.21.15.40:0/2724713256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a80ca390 con 0x7f52a80b11a0 2023-11-08T08:19:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.858+0000 7f528ffff700 1 -- 172.21.15.40:0/2724713256 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5298019030 con 0x7f52a80b11a0 2023-11-08T08:19:39.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.858+0000 7f528ffff700 1 --2- 172.21.15.40:0/2724713256 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5294064ee0 0x7f5294067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:39.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.858+0000 7f528ffff700 1 -- 172.21.15.40:0/2724713256 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5298082030 con 0x7f52a80b11a0 2023-11-08T08:19:39.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.858+0000 7f52a6ffd700 1 --2- 172.21.15.40:0/2724713256 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5294064ee0 0x7f5294067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:39.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.858+0000 7f52a6ffd700 1 --2- 172.21.15.40:0/2724713256 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5294064ee0 0x7f5294067390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f52a80b21c0 tx=0x7f529c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:39.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:39.862+0000 7f528ffff700 1 -- 172.21.15.40:0/2724713256 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f529804fee0 con 0x7f52a80b11a0 2023-11-08T08:19:40.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.042+0000 7f52acd69700 1 -- 172.21.15.40:0/2724713256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52a8073d20 con 0x7f52a80b11a0 2023-11-08T08:19:40.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.042+0000 7f528ffff700 1 -- 172.21.15.40:0/2724713256 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5298053500 con 0x7f52a80b11a0 2023-11-08T08:19:40.047 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:19:40.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.046+0000 7f52acd69700 1 -- 172.21.15.40:0/2724713256 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5294064ee0 msgr2=0x7f5294067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:40.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.046+0000 7f52acd69700 1 --2- 172.21.15.40:0/2724713256 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5294064ee0 0x7f5294067390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f52a80b21c0 tx=0x7f529c009040 comp rx=0 tx=0).stop 2023-11-08T08:19:40.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.046+0000 7f52acd69700 1 -- 172.21.15.40:0/2724713256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a80b11a0 msgr2=0x7f52a814cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:40.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.046+0000 7f52acd69700 1 --2- 172.21.15.40:0/2724713256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a80b11a0 0x7f52a814cf10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f529800e9d0 tx=0x7f529800ece0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.046+0000 7f52acd69700 1 -- 172.21.15.40:0/2724713256 shutdown_connections 2023-11-08T08:19:40.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.046+0000 7f52acd69700 1 --2- 172.21.15.40:0/2724713256 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5294064ee0 0x7f5294067390 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.046+0000 7f52acd69700 1 --2- 172.21.15.40:0/2724713256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52a8160770 0x7f52a815e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.046+0000 7f52acd69700 1 --2- 172.21.15.40:0/2724713256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52a815d6b0 0x7f52a806a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.046+0000 7f52acd69700 1 --2- 172.21.15.40:0/2724713256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a80b11a0 0x7f52a814cf10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.046+0000 7f52acd69700 1 -- 172.21.15.40:0/2724713256 >> 172.21.15.40:0/2724713256 conn(0x7f52a80cc1a0 msgr2=0x7f52a80d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:40.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.046+0000 7f52acd69700 1 -- 172.21.15.40:0/2724713256 shutdown_connections 2023-11-08T08:19:40.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.046+0000 7f52acd69700 1 -- 172.21.15.40:0/2724713256 wait complete. 2023-11-08T08:19:40.074 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T08:19:40.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.258+0000 7eff7cccb700 1 Processor -- start 2023-11-08T08:19:40.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.258+0000 7eff7cccb700 1 -- start start 2023-11-08T08:19:40.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.258+0000 7eff7cccb700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7eff7815c150 0x7eff780b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:40.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.258+0000 7eff7cccb700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7eff7806ff10 0x7eff78070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:40.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.258+0000 7eff7cccb700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff7815c520 con 0x7eff7815fc30 2023-11-08T08:19:40.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.258+0000 7eff7cccb700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff780b4240 con 0x7eff7815c150 2023-11-08T08:19:40.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.258+0000 7eff7cccb700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff780708a0 con 0x7eff7806ff10 2023-11-08T08:19:40.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.262+0000 7eff777fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7eff7815c150 0x7eff780b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.262+0000 7eff777fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7eff7815c150 0x7eff780b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55520/0 (socket says 172.21.15.40:55520) 2023-11-08T08:19:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.262+0000 7eff777fe700 1 -- 172.21.15.40:0/1067909430 learned_addr learned my addr 172.21.15.40:0/1067909430 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.262+0000 7eff6effd700 1 --2- 172.21.15.40:0/1067909430 >> v2:172.21.15.169:3300/0 conn(0x7eff7806ff10 0x7eff78070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:40.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.262+0000 7eff777fe700 1 -- 172.21.15.40:0/1067909430 >> v2:172.21.15.169:3300/0 conn(0x7eff7806ff10 msgr2=0x7eff78070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:40.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.262+0000 7eff777fe700 1 --2- 172.21.15.40:0/1067909430 >> v2:172.21.15.169:3300/0 conn(0x7eff7806ff10 0x7eff78070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.262+0000 7eff777fe700 1 -- 172.21.15.40:0/1067909430 >> v1:172.21.15.169:6789/0 conn(0x7eff7815fc30 legacy=0x7eff78160050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:40.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.262+0000 7eff777fe700 1 -- 172.21.15.40:0/1067909430 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7806ac00 con 0x7eff7815c150 2023-11-08T08:19:40.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.262+0000 7eff777fe700 1 --2- 172.21.15.40:0/1067909430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff7815c150 0x7eff780b3d00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7eff78066d30 tx=0x7eff68009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704967b4054ad174 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:40.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.262+0000 7eff76ffd700 1 -- 172.21.15.40:0/1067909430 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68011830 con 0x7eff7815c150 2023-11-08T08:19:40.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.262+0000 7eff76ffd700 1 -- 172.21.15.40:0/1067909430 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff68011990 con 0x7eff7815c150 2023-11-08T08:19:40.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.262+0000 7eff76ffd700 1 -- 172.21.15.40:0/1067909430 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68011d20 con 0x7eff7815c150 2023-11-08T08:19:40.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.262+0000 7eff7cccb700 1 -- 172.21.15.40:0/1067909430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff7815c150 msgr2=0x7eff780b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:40.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.262+0000 7eff7cccb700 1 --2- 172.21.15.40:0/1067909430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff7815c150 0x7eff780b3d00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7eff78066d30 tx=0x7eff68009bf0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.266+0000 7eff7cccb700 1 -- 172.21.15.40:0/1067909430 shutdown_connections 2023-11-08T08:19:40.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.266+0000 7eff7cccb700 1 --2- 172.21.15.40:0/1067909430 >> v2:172.21.15.169:3300/0 conn(0x7eff7806ff10 0x7eff78070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.266+0000 7eff7cccb700 1 --2- 172.21.15.40:0/1067909430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff7815c150 0x7eff780b3d00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.266+0000 7eff7cccb700 1 -- 172.21.15.40:0/1067909430 >> 172.21.15.40:0/1067909430 conn(0x7eff780d3680 msgr2=0x7eff780cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:40.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.266+0000 7eff7cccb700 1 -- 172.21.15.40:0/1067909430 shutdown_connections 2023-11-08T08:19:40.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.266+0000 7eff7cccb700 1 -- 172.21.15.40:0/1067909430 wait complete. 2023-11-08T08:19:40.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.266+0000 7eff7cccb700 1 Processor -- start 2023-11-08T08:19:40.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.270+0000 7eff7cccb700 1 -- start start 2023-11-08T08:19:40.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.270+0000 7eff7cccb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff7806ff10 0x7eff7815ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:40.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.270+0000 7eff7cccb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff7815c150 0x7eff781f0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:40.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.270+0000 7eff7cccb700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff7815fc30 0x7eff781f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:40.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.270+0000 7eff7cccb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7815db60 con 0x7eff7806ff10 2023-11-08T08:19:40.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.270+0000 7eff7cccb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7815de40 con 0x7eff7815c150 2023-11-08T08:19:40.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.270+0000 7eff7cccb700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7815e120 con 0x7eff7815fc30 2023-11-08T08:19:40.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.270+0000 7eff777fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff7806ff10 0x7eff7815ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:40.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.270+0000 7eff777fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff7806ff10 0x7eff7815ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55536/0 (socket says 172.21.15.40:55536) 2023-11-08T08:19:40.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.270+0000 7eff777fe700 1 -- 172.21.15.40:0/278778568 learned_addr learned my addr 172.21.15.40:0/278778568 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:40.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.270+0000 7eff6effd700 1 --2- 172.21.15.40:0/278778568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff7815c150 0x7eff781f0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:40.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.270+0000 7eff77fff700 1 --2- 172.21.15.40:0/278778568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff7815fc30 0x7eff781f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:40.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.270+0000 7eff777fe700 1 -- 172.21.15.40:0/278778568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff7815fc30 msgr2=0x7eff781f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:40.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.270+0000 7eff777fe700 1 --2- 172.21.15.40:0/278778568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff7815fc30 0x7eff781f4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.270+0000 7eff777fe700 1 -- 172.21.15.40:0/278778568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff7815c150 msgr2=0x7eff781f0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:40.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.270+0000 7eff777fe700 1 --2- 172.21.15.40:0/278778568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff7815c150 0x7eff781f0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.270+0000 7eff777fe700 1 -- 172.21.15.40:0/278778568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff68009570 con 0x7eff7806ff10 2023-11-08T08:19:40.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.270+0000 7eff77fff700 1 --2- 172.21.15.40:0/278778568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff7815fc30 0x7eff781f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:40.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.270+0000 7eff777fe700 1 --2- 172.21.15.40:0/278778568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff7806ff10 0x7eff7815ec00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7eff78073200 tx=0x7eff68004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:40.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.274+0000 7eff74ff9700 1 -- 172.21.15.40:0/278778568 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68004550 con 0x7eff7806ff10 2023-11-08T08:19:40.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.274+0000 7eff74ff9700 1 -- 172.21.15.40:0/278778568 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6801f070 con 0x7eff7806ff10 2023-11-08T08:19:40.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.274+0000 7eff7cccb700 1 -- 172.21.15.40:0/278778568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff780d2450 con 0x7eff7806ff10 2023-11-08T08:19:40.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.274+0000 7eff74ff9700 1 -- 172.21.15.40:0/278778568 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68003900 con 0x7eff7806ff10 2023-11-08T08:19:40.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.274+0000 7eff7cccb700 1 -- 172.21.15.40:0/278778568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7806cc50 con 0x7eff7806ff10 2023-11-08T08:19:40.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.274+0000 7eff7cccb700 1 -- 172.21.15.40:0/278778568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff780d2730 con 0x7eff7806ff10 2023-11-08T08:19:40.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.278+0000 7eff74ff9700 1 -- 172.21.15.40:0/278778568 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7eff6800f050 con 0x7eff7806ff10 2023-11-08T08:19:40.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.278+0000 7eff74ff9700 1 --2- 172.21.15.40:0/278778568 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7eff58064ee0 0x7eff58067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:40.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.278+0000 7eff74ff9700 1 -- 172.21.15.40:0/278778568 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7eff68021070 con 0x7eff7806ff10 2023-11-08T08:19:40.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.278+0000 7eff6effd700 1 --2- 172.21.15.40:0/278778568 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7eff58064ee0 0x7eff58067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:40.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.282+0000 7eff6effd700 1 --2- 172.21.15.40:0/278778568 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7eff58064ee0 0x7eff58067390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7eff780d1670 tx=0x7eff60008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:40.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.286+0000 7eff74ff9700 1 -- 172.21.15.40:0/278778568 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7eff680515e0 con 0x7eff7806ff10 2023-11-08T08:19:40.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.462+0000 7eff7cccb700 1 -- 172.21.15.40:0/278778568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7eff78073d20 con 0x7eff7806ff10 2023-11-08T08:19:40.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.462+0000 7eff74ff9700 1 -- 172.21.15.40:0/278778568 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7eff6802a020 con 0x7eff7806ff10 2023-11-08T08:19:40.466 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-11-08T08:19:40.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.466+0000 7eff7cccb700 1 -- 172.21.15.40:0/278778568 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7eff58064ee0 msgr2=0x7eff58067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:40.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.466+0000 7eff7cccb700 1 --2- 172.21.15.40:0/278778568 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7eff58064ee0 0x7eff58067390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7eff780d1670 tx=0x7eff60008040 comp rx=0 tx=0).stop 2023-11-08T08:19:40.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.466+0000 7eff7cccb700 1 -- 172.21.15.40:0/278778568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff7806ff10 msgr2=0x7eff7815ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:40.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.466+0000 7eff7cccb700 1 --2- 172.21.15.40:0/278778568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff7806ff10 0x7eff7815ec00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7eff78073200 tx=0x7eff68004770 comp rx=0 tx=0).stop 2023-11-08T08:19:40.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.466+0000 7eff7cccb700 1 -- 172.21.15.40:0/278778568 shutdown_connections 2023-11-08T08:19:40.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.466+0000 7eff7cccb700 1 --2- 172.21.15.40:0/278778568 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7eff58064ee0 0x7eff58067390 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.466+0000 7eff7cccb700 1 --2- 172.21.15.40:0/278778568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff7815fc30 0x7eff781f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.466+0000 7eff7cccb700 1 --2- 172.21.15.40:0/278778568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff7815c150 0x7eff781f0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.466+0000 7eff7cccb700 1 --2- 172.21.15.40:0/278778568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff7806ff10 0x7eff7815ec00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.466+0000 7eff7cccb700 1 -- 172.21.15.40:0/278778568 >> 172.21.15.40:0/278778568 conn(0x7eff780d3680 msgr2=0x7eff780c8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:40.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.466+0000 7eff7cccb700 1 -- 172.21.15.40:0/278778568 shutdown_connections 2023-11-08T08:19:40.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.466+0000 7eff7cccb700 1 -- 172.21.15.40:0/278778568 wait complete. 2023-11-08T08:19:40.495 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T08:19:40.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.674+0000 7f22b15f9700 1 Processor -- start 2023-11-08T08:19:40.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.674+0000 7f22b15f9700 1 -- start start 2023-11-08T08:19:40.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.674+0000 7f22b15f9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0cb630 0x7f22ac06bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:40.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.674+0000 7f22b15f9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f22ac06aae0 0x7f22ac0c9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:40.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.674+0000 7f22b15f9700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ac06d180 con 0x7f22ac15a030 2023-11-08T08:19:40.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.674+0000 7f22b15f9700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ac15a400 con 0x7f22ac06aae0 2023-11-08T08:19:40.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.674+0000 7f22b15f9700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac069f10 con 0x7f22ac0cb630 2023-11-08T08:19:40.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.674+0000 7f22b0df8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f22ac06aae0 0x7f22ac0c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:40.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.674+0000 7f22a37fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0cb630 0x7f22ac06bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:40.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.674+0000 7f22abfff700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f22ac15a030 0x7f22ac0caf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:43584/0 (socket says 172.21.15.40:43584) 2023-11-08T08:19:40.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.674+0000 7f22abfff700 1 -- 172.21.15.40:0/4067332638 learned_addr learned my addr 172.21.15.40:0/4067332638 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:40.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.678+0000 7f22b0df8700 1 -- 172.21.15.40:0/4067332638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0cb630 msgr2=0x7f22ac06bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:40.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.678+0000 7f22b0df8700 1 --2- 172.21.15.40:0/4067332638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0cb630 0x7f22ac06bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.678+0000 7f22b0df8700 1 -- 172.21.15.40:0/4067332638 >> v1:172.21.15.169:6789/0 conn(0x7f22ac15a030 legacy=0x7f22ac0caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:40.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.678+0000 7f22b0df8700 1 -- 172.21.15.40:0/4067332638 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac06af00 con 0x7f22ac06aae0 2023-11-08T08:19:40.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.678+0000 7f22a37fe700 1 --2- 172.21.15.40:0/4067332638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0cb630 0x7f22ac06bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:40.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.678+0000 7f22b0df8700 1 --2- 172.21.15.40:0/4067332638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ac06aae0 0x7f22ac0c9620 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f22a4009850 tx=0x7f22a4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc8110e7091b8b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:40.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.678+0000 7f22ab7fe700 1 -- 172.21.15.40:0/4067332638 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a4011830 con 0x7f22ac06aae0 2023-11-08T08:19:40.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.678+0000 7f22ab7fe700 1 -- 172.21.15.40:0/4067332638 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a4011990 con 0x7f22ac06aae0 2023-11-08T08:19:40.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.678+0000 7f22ab7fe700 1 -- 172.21.15.40:0/4067332638 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a4011d40 con 0x7f22ac06aae0 2023-11-08T08:19:40.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.678+0000 7f22b15f9700 1 -- 172.21.15.40:0/4067332638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ac06aae0 msgr2=0x7f22ac0c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:40.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.678+0000 7f22b15f9700 1 --2- 172.21.15.40:0/4067332638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ac06aae0 0x7f22ac0c9620 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f22a4009850 tx=0x7f22a4009b60 comp rx=0 tx=0).stop 2023-11-08T08:19:40.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.678+0000 7f22b15f9700 1 -- 172.21.15.40:0/4067332638 shutdown_connections 2023-11-08T08:19:40.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.678+0000 7f22b15f9700 1 --2- 172.21.15.40:0/4067332638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac0cb630 0x7f22ac06bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.678+0000 7f22b15f9700 1 --2- 172.21.15.40:0/4067332638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ac06aae0 0x7f22ac0c9620 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.678+0000 7f22b15f9700 1 -- 172.21.15.40:0/4067332638 >> 172.21.15.40:0/4067332638 conn(0x7f22ac0ce510 msgr2=0x7f22ac0ce910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:40.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.678+0000 7f22b15f9700 1 -- 172.21.15.40:0/4067332638 shutdown_connections 2023-11-08T08:19:40.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.678+0000 7f22b15f9700 1 -- 172.21.15.40:0/4067332638 wait complete. 2023-11-08T08:19:40.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.682+0000 7f22b15f9700 1 Processor -- start 2023-11-08T08:19:40.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.682+0000 7f22b15f9700 1 -- start start 2023-11-08T08:19:40.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22b15f9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac06aae0 0x7f22ac0c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:40.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22b15f9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ac0cb630 0x7f22ac0afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:40.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22b15f9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22ac15a030 0x7f22ac0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:40.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22b15f9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac0c3040 con 0x7f22ac0cb630 2023-11-08T08:19:40.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22b15f9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac158270 con 0x7f22ac15a030 2023-11-08T08:19:40.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22b15f9700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac157f90 con 0x7f22ac06aae0 2023-11-08T08:19:40.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22a37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ac0cb630 0x7f22ac0afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:40.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22a37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ac0cb630 0x7f22ac0afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55558/0 (socket says 172.21.15.40:55558) 2023-11-08T08:19:40.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22a37fe700 1 -- 172.21.15.40:0/2980585249 learned_addr learned my addr 172.21.15.40:0/2980585249 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:40.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22abfff700 1 --2- 172.21.15.40:0/2980585249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac06aae0 0x7f22ac0c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:40.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22b0df8700 1 --2- 172.21.15.40:0/2980585249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22ac15a030 0x7f22ac0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:40.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22a37fe700 1 -- 172.21.15.40:0/2980585249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac06aae0 msgr2=0x7f22ac0c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:40.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22a37fe700 1 --2- 172.21.15.40:0/2980585249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac06aae0 0x7f22ac0c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22a37fe700 1 -- 172.21.15.40:0/2980585249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22ac15a030 msgr2=0x7f22ac0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:40.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22a37fe700 1 --2- 172.21.15.40:0/2980585249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22ac15a030 0x7f22ac0b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22a37fe700 1 -- 172.21.15.40:0/2980585249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22940094b0 con 0x7f22ac0cb630 2023-11-08T08:19:40.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22a37fe700 1 --2- 172.21.15.40:0/2980585249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ac0cb630 0x7f22ac0afff0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f229400ebe0 tx=0x7f2294009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:40.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22a97fa700 1 -- 172.21.15.40:0/2980585249 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229400c8b0 con 0x7f22ac0cb630 2023-11-08T08:19:40.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22a97fa700 1 -- 172.21.15.40:0/2980585249 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229400ca10 con 0x7f22ac0cb630 2023-11-08T08:19:40.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22a97fa700 1 -- 172.21.15.40:0/2980585249 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229400cdc0 con 0x7f22ac0cb630 2023-11-08T08:19:40.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22b15f9700 1 -- 172.21.15.40:0/2980585249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a40094b0 con 0x7f22ac0cb630 2023-11-08T08:19:40.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.686+0000 7f22b15f9700 1 -- 172.21.15.40:0/2980585249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ac06c850 con 0x7f22ac0cb630 2023-11-08T08:19:40.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.690+0000 7f22b15f9700 1 -- 172.21.15.40:0/2980585249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ac06c570 con 0x7f22ac0cb630 2023-11-08T08:19:40.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.690+0000 7f22a97fa700 1 -- 172.21.15.40:0/2980585249 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f2294019030 con 0x7f22ac0cb630 2023-11-08T08:19:40.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.694+0000 7f22a97fa700 1 --2- 172.21.15.40:0/2980585249 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f228c064ee0 0x7f228c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:40.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.694+0000 7f22a97fa700 1 -- 172.21.15.40:0/2980585249 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2294081f30 con 0x7f22ac0cb630 2023-11-08T08:19:40.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.694+0000 7f22abfff700 1 --2- 172.21.15.40:0/2980585249 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f228c064ee0 0x7f228c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:40.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.694+0000 7f22abfff700 1 --2- 172.21.15.40:0/2980585249 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f228c064ee0 0x7f228c067390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f229c005cb0 tx=0x7f229c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:40.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.698+0000 7f22a97fa700 1 -- 172.21.15.40:0/2980585249 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f229404fc90 con 0x7f22ac0cb630 2023-11-08T08:19:40.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.870+0000 7f22b15f9700 1 -- 172.21.15.40:0/2980585249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f22ac06cb30 con 0x7f22ac0cb630 2023-11-08T08:19:40.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.870+0000 7f22a97fa700 1 -- 172.21.15.40:0/2980585249 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f22940532b0 con 0x7f22ac0cb630 2023-11-08T08:19:40.875 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:19:40.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.874+0000 7f22b15f9700 1 -- 172.21.15.40:0/2980585249 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f228c064ee0 msgr2=0x7f228c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:40.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.874+0000 7f22b15f9700 1 --2- 172.21.15.40:0/2980585249 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f228c064ee0 0x7f228c067390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f229c005cb0 tx=0x7f229c009040 comp rx=0 tx=0).stop 2023-11-08T08:19:40.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.874+0000 7f22b15f9700 1 -- 172.21.15.40:0/2980585249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ac0cb630 msgr2=0x7f22ac0afff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:40.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.874+0000 7f22b15f9700 1 --2- 172.21.15.40:0/2980585249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ac0cb630 0x7f22ac0afff0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f229400ebe0 tx=0x7f2294009e70 comp rx=0 tx=0).stop 2023-11-08T08:19:40.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.878+0000 7f22b15f9700 1 -- 172.21.15.40:0/2980585249 shutdown_connections 2023-11-08T08:19:40.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.878+0000 7f22b15f9700 1 --2- 172.21.15.40:0/2980585249 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f228c064ee0 0x7f228c067390 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.878+0000 7f22b15f9700 1 --2- 172.21.15.40:0/2980585249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22ac15a030 0x7f22ac0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.878+0000 7f22b15f9700 1 --2- 172.21.15.40:0/2980585249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ac0cb630 0x7f22ac0afff0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.878+0000 7f22b15f9700 1 --2- 172.21.15.40:0/2980585249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22ac06aae0 0x7f22ac0c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:40.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.878+0000 7f22b15f9700 1 -- 172.21.15.40:0/2980585249 >> 172.21.15.40:0/2980585249 conn(0x7f22ac0ce510 msgr2=0x7f22ac071870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:40.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.878+0000 7f22b15f9700 1 -- 172.21.15.40:0/2980585249 shutdown_connections 2023-11-08T08:19:40.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:40.878+0000 7f22b15f9700 1 -- 172.21.15.40:0/2980585249 wait complete. 2023-11-08T08:19:40.904 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:19:41.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f861d098700 1 Processor -- start 2023-11-08T08:19:41.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f861d098700 1 -- start start 2023-11-08T08:19:41.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f861d098700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f86180cdeb0 0x7f8618070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f861d098700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86180705c0 0x7f861806a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f861d098700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86180709d0 con 0x7f861815c1a0 2023-11-08T08:19:41.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f861d098700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8618069bf0 con 0x7f86180705c0 2023-11-08T08:19:41.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f861d098700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86180adf90 con 0x7f86180cdeb0 2023-11-08T08:19:41.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f8617fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f861815c1a0 0x7f861806c8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40652/0 (socket says 172.21.15.40:40652) 2023-11-08T08:19:41.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f8617fff700 1 -- 172.21.15.40:0/3653541440 learned_addr learned my addr 172.21.15.40:0/3653541440 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:41.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f861c897700 1 --2- 172.21.15.40:0/3653541440 >> v2:172.21.15.40:3300/0 conn(0x7f86180705c0 0x7f861806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f860f7fe700 1 --2- 172.21.15.40:0/3653541440 >> v2:172.21.15.169:3300/0 conn(0x7f86180cdeb0 0x7f8618070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f86177fe700 1 -- 172.21.15.40:0/3653541440 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3460042009 0 0) 0x7f86180709d0 con 0x7f861815c1a0 2023-11-08T08:19:41.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f86177fe700 1 -- 172.21.15.40:0/3653541440 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86040035d0 con 0x7f861815c1a0 2023-11-08T08:19:41.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f86177fe700 1 -- 172.21.15.40:0/3653541440 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 716599034 0 0) 0x7f86040035d0 con 0x7f861815c1a0 2023-11-08T08:19:41.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f86177fe700 1 -- 172.21.15.40:0/3653541440 >> v2:172.21.15.169:3300/0 conn(0x7f86180cdeb0 msgr2=0x7f8618070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:41.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f86177fe700 1 --2- 172.21.15.40:0/3653541440 >> v2:172.21.15.169:3300/0 conn(0x7f86180cdeb0 0x7f8618070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f86177fe700 1 -- 172.21.15.40:0/3653541440 >> v2:172.21.15.40:3300/0 conn(0x7f86180705c0 msgr2=0x7f861806a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:41.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f86177fe700 1 --2- 172.21.15.40:0/3653541440 >> v2:172.21.15.40:3300/0 conn(0x7f86180705c0 0x7f861806a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f86177fe700 1 -- 172.21.15.40:0/3653541440 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86180ce370 con 0x7f861815c1a0 2023-11-08T08:19:41.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f86177fe700 1 -- 172.21.15.40:0/3653541440 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f8608001670 con 0x7f861815c1a0 2023-11-08T08:19:41.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f86177fe700 1 -- 172.21.15.40:0/3653541440 >> v1:172.21.15.40:6789/0 conn(0x7f861815c1a0 legacy=0x7f861806c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:41.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f86177fe700 1 --2- 172.21.15.40:0/3653541440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8604003a20 0x7f8604003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f86177fe700 1 --2- 172.21.15.40:0/3653541440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8604004870 0x7f8604004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f86177fe700 1 --2- 172.21.15.40:0/3653541440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8604005220 0x7f860400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f86177fe700 1 -- 172.21.15.40:0/3653541440 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8618069bf0 con 0x7f8604003a20 2023-11-08T08:19:41.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f86177fe700 1 -- 172.21.15.40:0/3653541440 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86180adf90 con 0x7f8604004870 2023-11-08T08:19:41.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.078+0000 7f86177fe700 1 -- 172.21.15.40:0/3653541440 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f860400c020 con 0x7f8604005220 2023-11-08T08:19:41.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f8617fff700 1 --2- 172.21.15.40:0/3653541440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8604003a20 0x7f8604003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f860f7fe700 1 --2- 172.21.15.40:0/3653541440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8604004870 0x7f8604004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f861c897700 1 --2- 172.21.15.40:0/3653541440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8604005220 0x7f860400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f8617fff700 1 -- 172.21.15.40:0/3653541440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8604005220 msgr2=0x7f860400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:41.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f8617fff700 1 --2- 172.21.15.40:0/3653541440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8604005220 0x7f860400ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f8617fff700 1 -- 172.21.15.40:0/3653541440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8604004870 msgr2=0x7f8604004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:41.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f8617fff700 1 --2- 172.21.15.40:0/3653541440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8604004870 0x7f8604004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f8617fff700 1 -- 172.21.15.40:0/3653541440 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86040035d0 con 0x7f8604003a20 2023-11-08T08:19:41.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f860f7fe700 1 --2- 172.21.15.40:0/3653541440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8604004870 0x7f8604004ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:41.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f861c897700 1 -- 172.21.15.40:0/3653541440 reap_dead start 2023-11-08T08:19:41.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f8617fff700 1 --2- 172.21.15.40:0/3653541440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8604003a20 0x7f8604003e30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f860800c2f0 tx=0x7f86040058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4daa7284ac123f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:41.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f86177fe700 1 -- 172.21.15.40:0/3653541440 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860800ce30 con 0x7f8604003a20 2023-11-08T08:19:41.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f86177fe700 1 -- 172.21.15.40:0/3653541440 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8608005980 con 0x7f8604003a20 2023-11-08T08:19:41.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f86177fe700 1 -- 172.21.15.40:0/3653541440 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8608005d30 con 0x7f8604003a20 2023-11-08T08:19:41.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f861d098700 1 -- 172.21.15.40:0/3653541440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8604003a20 msgr2=0x7f8604003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:41.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f861d098700 1 --2- 172.21.15.40:0/3653541440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8604003a20 0x7f8604003e30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f860800c2f0 tx=0x7f86040058a0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f861d098700 1 -- 172.21.15.40:0/3653541440 shutdown_connections 2023-11-08T08:19:41.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f861d098700 1 --2- 172.21.15.40:0/3653541440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8604003a20 0x7f8604003e30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f861d098700 1 -- 172.21.15.40:0/3653541440 >> 172.21.15.40:0/3653541440 conn(0x7f86180b0f80 msgr2=0x7f86180ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:41.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f861d098700 1 -- 172.21.15.40:0/3653541440 shutdown_connections 2023-11-08T08:19:41.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.082+0000 7f861d098700 1 -- 172.21.15.40:0/3653541440 wait complete. 2023-11-08T08:19:41.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.086+0000 7f861d098700 1 Processor -- start 2023-11-08T08:19:41.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.086+0000 7f861d098700 1 -- start start 2023-11-08T08:19:41.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f861d098700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8604003a20 0x7f8618154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f861d098700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f861815a9a0 0x7f8618156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f861d098700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f861815ae10 0x7f8618156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f861d098700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861815d8e0 con 0x7f861815ae10 2023-11-08T08:19:41.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f861d098700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861815cc60 con 0x7f861815a9a0 2023-11-08T08:19:41.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f861d098700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f861815cf40 con 0x7f8604003a20 2023-11-08T08:19:41.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f861c897700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f861815ae10 0x7f8618156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f861c897700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f861815ae10 0x7f8618156cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55572/0 (socket says 172.21.15.40:55572) 2023-11-08T08:19:41.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f861c897700 1 -- 172.21.15.40:0/2793741550 learned_addr learned my addr 172.21.15.40:0/2793741550 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:41.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f8617fff700 1 --2- 172.21.15.40:0/2793741550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8604003a20 0x7f8618154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f860f7fe700 1 --2- 172.21.15.40:0/2793741550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f861815a9a0 0x7f8618156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f861c897700 1 -- 172.21.15.40:0/2793741550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8604003a20 msgr2=0x7f8618154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:41.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f861c897700 1 --2- 172.21.15.40:0/2793741550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8604003a20 0x7f8618154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f861c897700 1 -- 172.21.15.40:0/2793741550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f861815a9a0 msgr2=0x7f8618156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:41.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f861c897700 1 --2- 172.21.15.40:0/2793741550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f861815a9a0 0x7f8618156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f861c897700 1 -- 172.21.15.40:0/2793741550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86180cc780 con 0x7f861815ae10 2023-11-08T08:19:41.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f861c897700 1 --2- 172.21.15.40:0/2793741550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f861815ae10 0x7f8618156cc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8618071500 tx=0x7f861000f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:41.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f86157fa700 1 -- 172.21.15.40:0/2793741550 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861000fcb0 con 0x7f861815ae10 2023-11-08T08:19:41.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f86157fa700 1 -- 172.21.15.40:0/2793741550 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8610011040 con 0x7f861815ae10 2023-11-08T08:19:41.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f86157fa700 1 -- 172.21.15.40:0/2793741550 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8610017400 con 0x7f861815ae10 2023-11-08T08:19:41.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f861d098700 1 -- 172.21.15.40:0/2793741550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86180cc4a0 con 0x7f861815ae10 2023-11-08T08:19:41.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.090+0000 7f861d098700 1 -- 172.21.15.40:0/2793741550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f861815c570 con 0x7f861815ae10 2023-11-08T08:19:41.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.094+0000 7f861d098700 1 -- 172.21.15.40:0/2793741550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8618073cb0 con 0x7f861815ae10 2023-11-08T08:19:41.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.094+0000 7f86157fa700 1 -- 172.21.15.40:0/2793741550 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f861001e030 con 0x7f861815ae10 2023-11-08T08:19:41.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.098+0000 7f86157fa700 1 --2- 172.21.15.40:0/2793741550 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f85f8064ee0 0x7f85f8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.098+0000 7f86157fa700 1 -- 172.21.15.40:0/2793741550 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f86100821a0 con 0x7f861815ae10 2023-11-08T08:19:41.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.098+0000 7f8617fff700 1 --2- 172.21.15.40:0/2793741550 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f85f8064ee0 0x7f85f8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.098+0000 7f8617fff700 1 --2- 172.21.15.40:0/2793741550 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f85f8064ee0 0x7f85f8067390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8608008fd0 tx=0x7f860800a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:41.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.102+0000 7f86157fa700 1 -- 172.21.15.40:0/2793741550 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f861004ff60 con 0x7f861815ae10 2023-11-08T08:19:41.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.314+0000 7f861d098700 1 -- 172.21.15.40:0/2793741550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86180711c0 con 0x7f861815ae10 2023-11-08T08:19:41.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.314+0000 7f86157fa700 1 -- 172.21.15.40:0/2793741550 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4641 (secure 0 0 0) 0x7f8610053580 con 0x7f861815ae10 2023-11-08T08:19:41.319 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:41.319 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/196211087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":196211087},{"type":"v1","addr":"172.21.15.40:6834","nonce":196211087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/67127106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":67127106},{"type":"v1","addr":"172.21.15.40:6835","nonce":67127106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1231240563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1231240563},{"type":"v1","addr":"172.21.15.169:6835","nonce":1231240563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:19:04.456508+0000","modified":"2023-11-08T08:19:31.023792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6837/1989495204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1989495204},{"type":"v1","addr":"172.21.15.169:6837","nonce":1989495204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T08:19:41.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.318+0000 7f861d098700 1 -- 172.21.15.40:0/2793741550 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f85f8064ee0 msgr2=0x7f85f8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:41.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.318+0000 7f861d098700 1 --2- 172.21.15.40:0/2793741550 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f85f8064ee0 0x7f85f8067390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8608008fd0 tx=0x7f860800a000 comp rx=0 tx=0).stop 2023-11-08T08:19:41.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.318+0000 7f861d098700 1 -- 172.21.15.40:0/2793741550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f861815ae10 msgr2=0x7f8618156cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:41.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.318+0000 7f861d098700 1 --2- 172.21.15.40:0/2793741550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f861815ae10 0x7f8618156cc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8618071500 tx=0x7f861000f3b0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.318+0000 7f861d098700 1 -- 172.21.15.40:0/2793741550 shutdown_connections 2023-11-08T08:19:41.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.318+0000 7f861d098700 1 --2- 172.21.15.40:0/2793741550 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f85f8064ee0 0x7f85f8067390 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.318+0000 7f861d098700 1 --2- 172.21.15.40:0/2793741550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f861815ae10 0x7f8618156cc0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.318+0000 7f861d098700 1 --2- 172.21.15.40:0/2793741550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f861815a9a0 0x7f8618156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.318+0000 7f861d098700 1 --2- 172.21.15.40:0/2793741550 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8604003a20 0x7f8618154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.318+0000 7f861d098700 1 -- 172.21.15.40:0/2793741550 >> 172.21.15.40:0/2793741550 conn(0x7f86180b0f80 msgr2=0x7f86180cad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:41.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.318+0000 7f861d098700 1 -- 172.21.15.40:0/2793741550 shutdown_connections 2023-11-08T08:19:41.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.318+0000 7f861d098700 1 -- 172.21.15.40:0/2793741550 wait complete. 2023-11-08T08:19:41.327 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 9 2023-11-08T08:19:41.746 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T08:19:04.456508+0000', 'modified': '2023-11-08T08:19:31.023792+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4291}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4291': {'gid': 4291, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6837/1989495204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6836', 'nonce': 1989495204}, {'type': 'v1', 'addr': '172.21.15.169:6837', 'nonce': 1989495204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T08:19:41.747 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T08:19:41.747 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-08T08:19:41.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.930+0000 7fdbba5d9700 1 Processor -- start 2023-11-08T08:19:41.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.930+0000 7fdbba5d9700 1 -- start start 2023-11-08T08:19:41.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.930+0000 7fdbba5d9700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdbb4156790 0x7fdbb40cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.930+0000 7fdbba5d9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdbb4156b60 0x7fdbb40ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.930+0000 7fdbba5d9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb415fea0 0x7fdbb41602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.930+0000 7fdbba5d9700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb40ce540 con 0x7fdbb4156b60 2023-11-08T08:19:41.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.930+0000 7fdbba5d9700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb406b0d0 con 0x7fdbb4156790 2023-11-08T08:19:41.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.930+0000 7fdbba5d9700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb4160820 con 0x7fdbb415fea0 2023-11-08T08:19:41.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbb37fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdbb4156b60 0x7fdbb40ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbb37fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdbb4156b60 0x7fdbb40ce000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55578/0 (socket says 172.21.15.40:55578) 2023-11-08T08:19:41.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbb37fe700 1 -- 172.21.15.40:0/1732932788 learned_addr learned my addr 172.21.15.40:0/1732932788 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:41.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbb3fff700 1 --2- 172.21.15.40:0/1732932788 >> v2:172.21.15.169:3300/0 conn(0x7fdbb4156790 0x7fdbb40cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbb8b76700 1 --2- 172.21.15.40:0/1732932788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb415fea0 0x7fdbb41602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbb3fff700 1 -- 172.21.15.40:0/1732932788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb415fea0 msgr2=0x7fdbb41602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbb3fff700 1 --2- 172.21.15.40:0/1732932788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb415fea0 0x7fdbb41602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbb3fff700 1 -- 172.21.15.40:0/1732932788 >> v2:172.21.15.40:3300/0 conn(0x7fdbb4156b60 msgr2=0x7fdbb40ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:41.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbb3fff700 1 --2- 172.21.15.40:0/1732932788 >> v2:172.21.15.40:3300/0 conn(0x7fdbb4156b60 0x7fdbb40ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbb3fff700 1 -- 172.21.15.40:0/1732932788 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb406b560 con 0x7fdbb4156790 2023-11-08T08:19:41.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbb37fe700 1 --2- 172.21.15.40:0/1732932788 >> v2:172.21.15.40:3300/0 conn(0x7fdbb4156b60 0x7fdbb40ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:41.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbb3fff700 1 --2- 172.21.15.40:0/1732932788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb4156790 0x7fdbb40cc290 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdbb406e250 tx=0x7fdba4009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bff866e11a75f3ca server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:41.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbb2ffd700 1 -- 172.21.15.40:0/1732932788 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba4011830 con 0x7fdbb4156790 2023-11-08T08:19:41.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbb2ffd700 1 -- 172.21.15.40:0/1732932788 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba4011990 con 0x7fdbb4156790 2023-11-08T08:19:41.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbb2ffd700 1 -- 172.21.15.40:0/1732932788 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba4011d20 con 0x7fdbb4156790 2023-11-08T08:19:41.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbba5d9700 1 -- 172.21.15.40:0/1732932788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb4156790 msgr2=0x7fdbb40cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:41.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbba5d9700 1 --2- 172.21.15.40:0/1732932788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb4156790 0x7fdbb40cc290 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdbb406e250 tx=0x7fdba4009b30 comp rx=0 tx=0).stop 2023-11-08T08:19:41.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbba5d9700 1 -- 172.21.15.40:0/1732932788 shutdown_connections 2023-11-08T08:19:41.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbba5d9700 1 --2- 172.21.15.40:0/1732932788 >> v2:172.21.15.40:3300/0 conn(0x7fdbb4156b60 0x7fdbb40ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbba5d9700 1 --2- 172.21.15.40:0/1732932788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb415fea0 0x7fdbb41602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbba5d9700 1 --2- 172.21.15.40:0/1732932788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb4156790 0x7fdbb40cc290 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbba5d9700 1 -- 172.21.15.40:0/1732932788 >> 172.21.15.40:0/1732932788 conn(0x7fdbb40cf350 msgr2=0x7fdbb40b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:41.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbba5d9700 1 -- 172.21.15.40:0/1732932788 shutdown_connections 2023-11-08T08:19:41.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.934+0000 7fdbba5d9700 1 -- 172.21.15.40:0/1732932788 wait complete. 2023-11-08T08:19:41.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.938+0000 7fdbba5d9700 1 Processor -- start 2023-11-08T08:19:41.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.938+0000 7fdbba5d9700 1 -- start start 2023-11-08T08:19:41.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.938+0000 7fdbba5d9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb4156790 0x7fdbb4144ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.938+0000 7fdbba5d9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb4156b60 0x7fdbb40c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.938+0000 7fdbba5d9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbb415fea0 0x7fdbb40b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.938+0000 7fdbba5d9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb40c4290 con 0x7fdbb415fea0 2023-11-08T08:19:41.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.938+0000 7fdbba5d9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb40afad0 con 0x7fdbb4156790 2023-11-08T08:19:41.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.938+0000 7fdbba5d9700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb40afdb0 con 0x7fdbb4156b60 2023-11-08T08:19:41.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.938+0000 7fdbb3fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb4156790 0x7fdbb4144ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.938+0000 7fdbb8b76700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbb415fea0 0x7fdbb40b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.938+0000 7fdbb3fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb4156790 0x7fdbb4144ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:45494/0 (socket says 172.21.15.40:45494) 2023-11-08T08:19:41.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.938+0000 7fdbb8b76700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbb415fea0 0x7fdbb40b38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55594/0 (socket says 172.21.15.40:55594) 2023-11-08T08:19:41.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.938+0000 7fdbb3fff700 1 -- 172.21.15.40:0/87193642 learned_addr learned my addr 172.21.15.40:0/87193642 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:41.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.942+0000 7fdbb37fe700 1 --2- 172.21.15.40:0/87193642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb4156b60 0x7fdbb40c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.942+0000 7fdbb8b76700 1 -- 172.21.15.40:0/87193642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb4156b60 msgr2=0x7fdbb40c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:41.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.942+0000 7fdbb8b76700 1 --2- 172.21.15.40:0/87193642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb4156b60 0x7fdbb40c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.942+0000 7fdbb8b76700 1 -- 172.21.15.40:0/87193642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb4156790 msgr2=0x7fdbb4144ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:41.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.942+0000 7fdbb8b76700 1 --2- 172.21.15.40:0/87193642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb4156790 0x7fdbb4144ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.942+0000 7fdbb8b76700 1 -- 172.21.15.40:0/87193642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9c009570 con 0x7fdbb415fea0 2023-11-08T08:19:41.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.942+0000 7fdbb3fff700 1 --2- 172.21.15.40:0/87193642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb4156790 0x7fdbb4144ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:41.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.942+0000 7fdbb8b76700 1 --2- 172.21.15.40:0/87193642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbb415fea0 0x7fdbb40b38e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00eba0 tx=0x7fdbac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:41.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.942+0000 7fdbb0ff9700 1 -- 172.21.15.40:0/87193642 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac00c970 con 0x7fdbb415fea0 2023-11-08T08:19:41.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.942+0000 7fdbb0ff9700 1 -- 172.21.15.40:0/87193642 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbac00f040 con 0x7fdbb415fea0 2023-11-08T08:19:41.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.942+0000 7fdbb0ff9700 1 -- 172.21.15.40:0/87193642 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac00cd10 con 0x7fdbb415fea0 2023-11-08T08:19:41.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.942+0000 7fdbba5d9700 1 -- 172.21.15.40:0/87193642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba40094b0 con 0x7fdbb415fea0 2023-11-08T08:19:41.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.942+0000 7fdbba5d9700 1 -- 172.21.15.40:0/87193642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb406a7a0 con 0x7fdbb415fea0 2023-11-08T08:19:41.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.946+0000 7fdbb0ff9700 1 -- 172.21.15.40:0/87193642 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fdbac014480 con 0x7fdbb415fea0 2023-11-08T08:19:41.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.946+0000 7fdbb0ff9700 1 --2- 172.21.15.40:0/87193642 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdb94064f30 0x7fdb940673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.946+0000 7fdbb0ff9700 1 -- 172.21.15.40:0/87193642 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdbac082c40 con 0x7fdbb415fea0 2023-11-08T08:19:41.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.946+0000 7fdbb3fff700 1 --2- 172.21.15.40:0/87193642 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdb94064f30 0x7fdb940673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.950+0000 7fdbb3fff700 1 --2- 172.21.15.40:0/87193642 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdb94064f30 0x7fdb940673e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdba4009fd0 tx=0x7fdba4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:41.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.950+0000 7fdbbd861740 1 Processor -- start 2023-11-08T08:19:41.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.950+0000 7fdbbd861740 1 -- start start 2023-11-08T08:19:41.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.950+0000 7fdbbd861740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x11bfae0 0x146fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.950+0000 7fdbbd861740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x11bf710 0x1470340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.950+0000 7fdbbd861740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x120a790 0x1470880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.950+0000 7fdbbd861740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x132eae0 con 0x120a790 2023-11-08T08:19:41.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.950+0000 7fdbbd861740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x139ebd0 con 0x11bf710 2023-11-08T08:19:41.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.950+0000 7fdbbd861740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1397ed0 con 0x11bfae0 2023-11-08T08:19:41.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.950+0000 7fdbb8b76700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x120a790 0x1470880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.950+0000 7fdbb8b76700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x120a790 0x1470880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55604/0 (socket says 172.21.15.40:55604) 2023-11-08T08:19:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.950+0000 7fdbb8b76700 1 -- 172.21.15.40:0/2763290491 learned_addr learned my addr 172.21.15.40:0/2763290491 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.950+0000 7fdbb3fff700 1 --2- 172.21.15.40:0/2763290491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x11bfae0 0x146fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.950+0000 7fdbb37fe700 1 --2- 172.21.15.40:0/2763290491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x11bf710 0x1470340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.950+0000 7fdbb8b76700 1 -- 172.21.15.40:0/2763290491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x11bfae0 msgr2=0x146fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:41.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.950+0000 7fdbb8b76700 1 --2- 172.21.15.40:0/2763290491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x11bfae0 0x146fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.950+0000 7fdbb8b76700 1 -- 172.21.15.40:0/2763290491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x11bf710 msgr2=0x1470340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:41.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.950+0000 7fdbb8b76700 1 --2- 172.21.15.40:0/2763290491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x11bf710 0x1470340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.950+0000 7fdbb8b76700 1 -- 172.21.15.40:0/2763290491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x13bdcc0 con 0x120a790 2023-11-08T08:19:41.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.954+0000 7fdbb8b76700 1 --2- 172.21.15.40:0/2763290491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x120a790 0x1470880 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdbac054140 tx=0x7fdbac054170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd70629e84f710f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:41.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.954+0000 7fdba9ffb700 1 -- 172.21.15.40:0/2763290491 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac05a2c0 con 0x120a790 2023-11-08T08:19:41.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.954+0000 7fdba9ffb700 1 -- 172.21.15.40:0/2763290491 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbac053bd0 con 0x120a790 2023-11-08T08:19:41.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.954+0000 7fdba9ffb700 1 -- 172.21.15.40:0/2763290491 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac05c340 con 0x120a790 2023-11-08T08:19:41.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.954+0000 7fdbbd861740 1 -- 172.21.15.40:0/2763290491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x120a790 msgr2=0x1470880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:41.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.954+0000 7fdbbd861740 1 --2- 172.21.15.40:0/2763290491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x120a790 0x1470880 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdbac054140 tx=0x7fdbac054170 comp rx=0 tx=0).stop 2023-11-08T08:19:41.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.954+0000 7fdbbd861740 1 -- 172.21.15.40:0/2763290491 shutdown_connections 2023-11-08T08:19:41.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.954+0000 7fdbbd861740 1 --2- 172.21.15.40:0/2763290491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x120a790 0x1470880 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.954+0000 7fdbbd861740 1 --2- 172.21.15.40:0/2763290491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x11bf710 0x1470340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.954+0000 7fdbbd861740 1 --2- 172.21.15.40:0/2763290491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x11bfae0 0x146fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.954+0000 7fdbbd861740 1 -- 172.21.15.40:0/2763290491 >> 172.21.15.40:0/2763290491 conn(0x11bfeb0 msgr2=0x1458700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:41.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.954+0000 7fdbbd861740 1 -- 172.21.15.40:0/2763290491 shutdown_connections 2023-11-08T08:19:41.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.954+0000 7fdbbd861740 1 -- 172.21.15.40:0/2763290491 wait complete. 2023-11-08T08:19:41.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.954+0000 7fdbbd861740 1 -- start start 2023-11-08T08:19:41.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.954+0000 7fdbbd861740 1 Processor -- start 2023-11-08T08:19:41.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbbd861740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x11bf710 0x1444700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbbd861740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bfae0 0x1454b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbbd861740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x120a790 0x1431aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbbd861740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x1244b20 con 0x11bfae0 2023-11-08T08:19:41.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbbd861740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1445c20 con 0x11bf710 2023-11-08T08:19:41.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbbd861740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1446210 con 0x120a790 2023-11-08T08:19:41.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbbd861740 10 client.0 _ll_register_callbacks cb 0x13f40d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:19:41.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbb37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bfae0 0x1454b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbb37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bfae0 0x1454b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55614/0 (socket says 172.21.15.40:55614) 2023-11-08T08:19:41.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbb37fe700 1 -- 172.21.15.40:0/2608061653 learned_addr learned my addr 172.21.15.40:0/2608061653 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:41.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbb8b76700 1 --2- 172.21.15.40:0/2608061653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x120a790 0x1431aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbb3fff700 1 --2- 172.21.15.40:0/2608061653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x11bf710 0x1444700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbb37fe700 1 -- 172.21.15.40:0/2608061653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x120a790 msgr2=0x1431aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:41.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbb37fe700 1 --2- 172.21.15.40:0/2608061653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x120a790 0x1431aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbb37fe700 1 -- 172.21.15.40:0/2608061653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x11bf710 msgr2=0x1444700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:41.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbb37fe700 1 --2- 172.21.15.40:0/2608061653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x11bf710 0x1444700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbb37fe700 1 -- 172.21.15.40:0/2608061653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb406a7a0 con 0x11bfae0 2023-11-08T08:19:41.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbb37fe700 1 --2- 172.21.15.40:0/2608061653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bfae0 0x1454b20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c014920 tx=0x7fdb9c014950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77882b5d3b8f2462 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:41.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdb92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:19:41.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdb92ffd700 1 -- 172.21.15.40:0/2608061653 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c004030 con 0x11bfae0 2023-11-08T08:19:41.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdb92ffd700 1 -- 172.21.15.40:0/2608061653 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdb9c004190 con 0x11bfae0 2023-11-08T08:19:41.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbbd861740 1 -- 172.21.15.40:0/2608061653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1390f50 con 0x11bfae0 2023-11-08T08:19:41.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdb92ffd700 1 -- 172.21.15.40:0/2608061653 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb9c004350 con 0x11bfae0 2023-11-08T08:19:41.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbbd861740 10 client.4803 fetch_fsmap learned FSMap version 9 2023-11-08T08:19:41.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbbd861740 1 -- 172.21.15.40:0/2608061653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x120b9e0 con 0x11bfae0 2023-11-08T08:19:41.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdb92ffd700 1 -- 172.21.15.40:0/2608061653 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1390f50 con 0x11bfae0 2023-11-08T08:19:41.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdb92ffd700 1 -- 172.21.15.40:0/2608061653 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 9) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fdb9c0036c0 con 0x11bfae0 2023-11-08T08:19:41.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbbd861740 10 client.4803 fetch_fsmap finished waiting for FSMap version 9 2023-11-08T08:19:41.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbbd861740 10 client.4803 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T08:19:41.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbbd861740 20 client.4803 populate_metadata read hostname 'smithi040' 2023-11-08T08:19:41.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbbd861740 1 --2- 172.21.15.40:0/2608061653 >> [v2:172.21.15.169:6836/1989495204,v1:172.21.15.169:6837/1989495204] conn(0x14453e0 0x148a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbbd861740 4 client.4803 mds_command: new command op to 4291 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-08T08:19:41.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.958+0000 7fdbbd861740 1 -- 172.21.15.40:0/2608061653 --> [v2:172.21.15.169:6836/1989495204,v1:172.21.15.169:6837/1989495204] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1390c70 con 0x14453e0 2023-11-08T08:19:41.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.962+0000 7fdbb3fff700 1 --2- 172.21.15.40:0/2608061653 >> [v2:172.21.15.169:6836/1989495204,v1:172.21.15.169:6837/1989495204] conn(0x14453e0 0x148a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.962+0000 7fdbb3fff700 1 --2- 172.21.15.40:0/2608061653 >> [v2:172.21.15.169:6836/1989495204,v1:172.21.15.169:6837/1989495204] conn(0x14453e0 0x148a1b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37f724bda53e95a1 server_cookie=a6a1fc048ecbdeea in_seq=0 out_seq=0 2023-11-08T08:19:41.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.962+0000 7fdb92ffd700 10 client.4803 ms_handle_connect on v2:172.21.15.169:6836/1989495204 2023-11-08T08:19:41.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.962+0000 7fdb92ffd700 1 -- 172.21.15.40:0/2608061653 <== mds.0 v2:172.21.15.169:6836/1989495204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x1390c70 con 0x14453e0 2023-11-08T08:19:41.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.962+0000 7fdb92ffd700 10 client.4803 handle_command_reply: tid=0 2023-11-08T08:19:41.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.962+0000 7fdbbd861740 1 client.4803 shutdown 2023-11-08T08:19:41.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.962+0000 7fdbbd861740 1 -- 172.21.15.40:0/2608061653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bfae0 msgr2=0x1454b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:41.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.962+0000 7fdbbd861740 1 --2- 172.21.15.40:0/2608061653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bfae0 0x1454b20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c014920 tx=0x7fdb9c014950 comp rx=0 tx=0).stop 2023-11-08T08:19:41.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.962+0000 7fdbbd861740 1 -- 172.21.15.40:0/2608061653 shutdown_connections 2023-11-08T08:19:41.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.962+0000 7fdbbd861740 1 --2- 172.21.15.40:0/2608061653 >> [v2:172.21.15.169:6836/1989495204,v1:172.21.15.169:6837/1989495204] conn(0x14453e0 0x148a1b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.962+0000 7fdbbd861740 1 --2- 172.21.15.40:0/2608061653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x120a790 0x1431aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.962+0000 7fdbbd861740 1 --2- 172.21.15.40:0/2608061653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bfae0 0x1454b20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.962+0000 7fdbbd861740 1 --2- 172.21.15.40:0/2608061653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x11bf710 0x1444700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:41.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.962+0000 7fdbbd861740 1 -- 172.21.15.40:0/2608061653 >> 172.21.15.40:0/2608061653 conn(0x11bfeb0 msgr2=0x1458800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:41.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.962+0000 7fdb92ffd700 0 client.4803 ms_handle_reset on v2:172.21.15.169:6836/1989495204 2023-11-08T08:19:41.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.966+0000 7fdbbd861740 1 -- 172.21.15.40:0/2608061653 shutdown_connections 2023-11-08T08:19:41.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.966+0000 7fdbbd861740 1 -- 172.21.15.40:0/2608061653 wait complete. 2023-11-08T08:19:41.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.966+0000 7fdbbd861740 20 client.4803 trim_cache size 0 max 16384 2023-11-08T08:19:41.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.986+0000 7fdbbd861740 1 Processor -- start 2023-11-08T08:19:41.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.986+0000 7fdbbd861740 1 -- start start 2023-11-08T08:19:41.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.986+0000 7fdbbd861740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x11bf710 0x1458b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.986+0000 7fdbbd861740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bfae0 0x1459080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.986+0000 7fdbbd861740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x120a790 0x1457230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:41.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.986+0000 7fdbbd861740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x1396d60 con 0x11bfae0 2023-11-08T08:19:41.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.986+0000 7fdbbd861740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1470f90 con 0x120a790 2023-11-08T08:19:41.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.986+0000 7fdbbd861740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x139cb40 con 0x11bf710 2023-11-08T08:19:41.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.986+0000 7fdbb37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bfae0 0x1459080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.986+0000 7fdbb37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bfae0 0x1459080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55628/0 (socket says 172.21.15.40:55628) 2023-11-08T08:19:41.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.986+0000 7fdbb37fe700 1 -- 172.21.15.40:0/661247597 learned_addr learned my addr 172.21.15.40:0/661247597 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:41.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.986+0000 7fdbb3fff700 1 --2- 172.21.15.40:0/661247597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x11bf710 0x1458b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.986+0000 7fdbb8b76700 1 --2- 172.21.15.40:0/661247597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x120a790 0x1457230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:41.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.986+0000 7fdbb37fe700 1 -- 172.21.15.40:0/661247597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x11bf710 msgr2=0x1458b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:41.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.986+0000 7fdbb37fe700 1 --2- 172.21.15.40:0/661247597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x11bf710 0x1458b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:42.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.986+0000 7fdbb37fe700 1 -- 172.21.15.40:0/661247597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x120a790 msgr2=0x1457230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:42.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.986+0000 7fdbb37fe700 1 --2- 172.21.15.40:0/661247597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x120a790 0x1457230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:42.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.986+0000 7fdbb37fe700 1 -- 172.21.15.40:0/661247597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x135e8a0 con 0x11bfae0 2023-11-08T08:19:42.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.990+0000 7fdbb37fe700 1 --2- 172.21.15.40:0/661247597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bfae0 0x1459080 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0148f0 tx=0x7fdb9c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bdd32a2bef8ace server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:42.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.990+0000 7fdbaa7fc700 1 -- 172.21.15.40:0/661247597 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c0036c0 con 0x11bfae0 2023-11-08T08:19:42.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.990+0000 7fdbaa7fc700 1 -- 172.21.15.40:0/661247597 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9c003b80 con 0x11bfae0 2023-11-08T08:19:42.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.990+0000 7fdbaa7fc700 1 -- 172.21.15.40:0/661247597 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c003840 con 0x11bfae0 2023-11-08T08:19:42.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.990+0000 7fdbbd861740 1 -- 172.21.15.40:0/661247597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bfae0 msgr2=0x1459080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:42.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.990+0000 7fdbbd861740 1 --2- 172.21.15.40:0/661247597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bfae0 0x1459080 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0148f0 tx=0x7fdb9c017000 comp rx=0 tx=0).stop 2023-11-08T08:19:42.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.990+0000 7fdbbd861740 1 -- 172.21.15.40:0/661247597 shutdown_connections 2023-11-08T08:19:42.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.990+0000 7fdbbd861740 1 --2- 172.21.15.40:0/661247597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x120a790 0x1457230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:42.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.990+0000 7fdbbd861740 1 --2- 172.21.15.40:0/661247597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bfae0 0x1459080 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:42.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.990+0000 7fdbbd861740 1 --2- 172.21.15.40:0/661247597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x11bf710 0x1458b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:42.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.990+0000 7fdbbd861740 1 -- 172.21.15.40:0/661247597 >> 172.21.15.40:0/661247597 conn(0x11bfeb0 msgr2=0x1454930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:42.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.990+0000 7fdbbd861740 1 -- 172.21.15.40:0/661247597 shutdown_connections 2023-11-08T08:19:42.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.990+0000 7fdbbd861740 1 -- 172.21.15.40:0/661247597 wait complete. 2023-11-08T08:19:42.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.990+0000 7fdbbd861740 1 -- start start 2023-11-08T08:19:42.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbbd861740 1 Processor -- start 2023-11-08T08:19:42.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbbd861740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bf710 0x142f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:42.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbbd861740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x11bfae0 0x142ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:42.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbbd861740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x120a790 0x142b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:42.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbbd861740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x139aa30 con 0x11bf710 2023-11-08T08:19:42.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbbd861740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1368ec0 con 0x11bfae0 2023-11-08T08:19:42.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbbd861740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x139fe60 con 0x120a790 2023-11-08T08:19:42.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbbd861740 10 client.0 _ll_register_callbacks cb 0x13442d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:19:42.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbb3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bf710 0x142f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:42.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbb3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bf710 0x142f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55638/0 (socket says 172.21.15.40:55638) 2023-11-08T08:19:42.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbb3fff700 1 -- 172.21.15.40:0/2202938692 learned_addr learned my addr 172.21.15.40:0/2202938692 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:42.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbb37fe700 1 --2- 172.21.15.40:0/2202938692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x11bfae0 0x142ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:42.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbb3fff700 1 -- 172.21.15.40:0/2202938692 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x120a790 msgr2=0x142b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:42.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbb8b76700 1 --2- 172.21.15.40:0/2202938692 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x120a790 0x142b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:42.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbb3fff700 1 --2- 172.21.15.40:0/2202938692 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x120a790 0x142b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:42.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbb3fff700 1 -- 172.21.15.40:0/2202938692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x11bfae0 msgr2=0x142ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:42.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbb3fff700 1 --2- 172.21.15.40:0/2202938692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x11bfae0 0x142ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:42.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbb3fff700 1 -- 172.21.15.40:0/2202938692 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x120b9e0 con 0x11bf710 2023-11-08T08:19:42.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbb37fe700 1 --2- 172.21.15.40:0/2202938692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x11bfae0 0x142ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:42.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbb8b76700 1 --2- 172.21.15.40:0/2202938692 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x120a790 0x142b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:42.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbb3fff700 1 --2- 172.21.15.40:0/2202938692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bf710 0x142f2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdbb406b8f0 tx=0x7fdba40047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a1e2883fbac4af server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:42.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdb92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:19:42.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdb92ffd700 1 -- 172.21.15.40:0/2202938692 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba4039030 con 0x11bf710 2023-11-08T08:19:42.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdbbd861740 1 -- 172.21.15.40:0/2202938692 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbac053bd0 con 0x11bf710 2023-11-08T08:19:42.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.994+0000 7fdb92ffd700 1 -- 172.21.15.40:0/2202938692 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdba403a070 con 0x11bf710 2023-11-08T08:19:42.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.998+0000 7fdb92ffd700 1 -- 172.21.15.40:0/2202938692 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdba402e040 con 0x11bf710 2023-11-08T08:19:42.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.998+0000 7fdbbd861740 10 client.4809 fetch_fsmap learned FSMap version 9 2023-11-08T08:19:42.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.998+0000 7fdbbd861740 1 -- 172.21.15.40:0/2202938692 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x1344380 con 0x11bf710 2023-11-08T08:19:42.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.998+0000 7fdb92ffd700 1 -- 172.21.15.40:0/2202938692 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbac053bd0 con 0x11bf710 2023-11-08T08:19:42.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.998+0000 7fdb92ffd700 1 -- 172.21.15.40:0/2202938692 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 9) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fdba4035d00 con 0x11bf710 2023-11-08T08:19:42.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.998+0000 7fdbbd861740 10 client.4809 fetch_fsmap finished waiting for FSMap version 9 2023-11-08T08:19:42.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.998+0000 7fdbbd861740 10 client.4809 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T08:19:42.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.998+0000 7fdbbd861740 20 client.4809 populate_metadata read hostname 'smithi040' 2023-11-08T08:19:42.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.998+0000 7fdbbd861740 1 --2- 172.21.15.40:0/2202938692 >> [v2:172.21.15.169:6836/1989495204,v1:172.21.15.169:6837/1989495204] conn(0x14df520 0x14df8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:42.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.998+0000 7fdbbd861740 4 client.4809 mds_command: new command op to 4291 tid=0[{"prefix": "status"}] 2023-11-08T08:19:42.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.998+0000 7fdbbd861740 1 -- 172.21.15.40:0/2202938692 --> [v2:172.21.15.169:6836/1989495204,v1:172.21.15.169:6837/1989495204] -- command(tid 0: {"prefix": "status"}) v1 -- 0x123c260 con 0x14df520 2023-11-08T08:19:42.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.998+0000 7fdbb37fe700 1 --2- 172.21.15.40:0/2202938692 >> [v2:172.21.15.169:6836/1989495204,v1:172.21.15.169:6837/1989495204] conn(0x14df520 0x14df8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:42.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.998+0000 7fdbb37fe700 1 --2- 172.21.15.40:0/2202938692 >> [v2:172.21.15.169:6836/1989495204,v1:172.21.15.169:6837/1989495204] conn(0x14df520 0x14df8f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ec8fe9c4065db9 server_cookie=aca072d10124e1ad in_seq=0 out_seq=0 2023-11-08T08:19:42.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.998+0000 7fdb92ffd700 10 client.4809 ms_handle_connect on v2:172.21.15.169:6836/1989495204 2023-11-08T08:19:42.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.998+0000 7fdb92ffd700 1 -- 172.21.15.40:0/2202938692 <== mds.0 v2:172.21.15.169:6836/1989495204 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x123c260 con 0x14df520 2023-11-08T08:19:42.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.998+0000 7fdb92ffd700 10 client.4809 handle_command_reply: tid=0 2023-11-08T08:19:42.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.998+0000 7fdbbd861740 1 client.4809 shutdown 2023-11-08T08:19:42.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.998+0000 7fdbbd861740 1 -- 172.21.15.40:0/2202938692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bf710 msgr2=0x142f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:42.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:41.998+0000 7fdbbd861740 1 --2- 172.21.15.40:0/2202938692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bf710 0x142f2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdbb406b8f0 tx=0x7fdba40047e0 comp rx=0 tx=0).stop 2023-11-08T08:19:42.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbbd861740 1 -- 172.21.15.40:0/2202938692 shutdown_connections 2023-11-08T08:19:42.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbbd861740 1 --2- 172.21.15.40:0/2202938692 >> [v2:172.21.15.169:6836/1989495204,v1:172.21.15.169:6837/1989495204] conn(0x14df520 0x14df8f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:42.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbbd861740 1 --2- 172.21.15.40:0/2202938692 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x120a790 0x142b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:42.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbbd861740 1 --2- 172.21.15.40:0/2202938692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x11bfae0 0x142ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:42.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbbd861740 1 --2- 172.21.15.40:0/2202938692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x11bf710 0x142f2f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:42.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdb92ffd700 0 client.4809 ms_handle_reset on v2:172.21.15.169:6836/1989495204 2023-11-08T08:19:42.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbbd861740 1 -- 172.21.15.40:0/2202938692 >> 172.21.15.40:0/2202938692 conn(0x11bfeb0 msgr2=0x1458930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:42.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbbd861740 1 -- 172.21.15.40:0/2202938692 shutdown_connections 2023-11-08T08:19:42.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbbd861740 1 -- 172.21.15.40:0/2202938692 wait complete. 2023-11-08T08:19:42.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbbd861740 20 client.4809 trim_cache size 0 max 16384 2023-11-08T08:19:42.017 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:19:42.018 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "54ed45f4-d5b0-41c3-8c3b-b5a18b844f25", 2023-11-08T08:19:42.018 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2023-11-08T08:19:42.018 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4291, 2023-11-08T08:19:42.018 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-11-08T08:19:42.018 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-11-08T08:19:42.019 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-11-08T08:19:42.019 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 37.537712519000003, 2023-11-08T08:19:42.019 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 9, 2023-11-08T08:19:42.019 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 28, 2023-11-08T08:19:42.019 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 28, 2023-11-08T08:19:42.020 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 43.041183050000001 2023-11-08T08:19:42.020 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:19:42.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbba5d9700 1 -- 172.21.15.40:0/87193642 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdb94064f30 msgr2=0x7fdb940673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:42.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbba5d9700 1 --2- 172.21.15.40:0/87193642 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdb94064f30 0x7fdb940673e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdba4009fd0 tx=0x7fdba4008030 comp rx=0 tx=0).stop 2023-11-08T08:19:42.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbba5d9700 1 -- 172.21.15.40:0/87193642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbb415fea0 msgr2=0x7fdbb40b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:42.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbba5d9700 1 --2- 172.21.15.40:0/87193642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbb415fea0 0x7fdbb40b38e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00eba0 tx=0x7fdbac00eeb0 comp rx=0 tx=0).stop 2023-11-08T08:19:42.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbba5d9700 1 -- 172.21.15.40:0/87193642 shutdown_connections 2023-11-08T08:19:42.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbba5d9700 1 --2- 172.21.15.40:0/87193642 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdb94064f30 0x7fdb940673e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:42.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbba5d9700 1 --2- 172.21.15.40:0/87193642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbb415fea0 0x7fdbb40b38e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:42.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbba5d9700 1 --2- 172.21.15.40:0/87193642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb4156b60 0x7fdbb40c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:42.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbba5d9700 1 --2- 172.21.15.40:0/87193642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb4156790 0x7fdbb4144ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:42.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbba5d9700 1 -- 172.21.15.40:0/87193642 >> 172.21.15.40:0/87193642 conn(0x7fdbb40cf350 msgr2=0x7fdbb415da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:42.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbba5d9700 1 -- 172.21.15.40:0/87193642 shutdown_connections 2023-11-08T08:19:42.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.002+0000 7fdbba5d9700 1 -- 172.21.15.40:0/87193642 wait complete. 2023-11-08T08:19:42.031 INFO:teuthology.run_tasks:Running task kclient... 2023-11-08T08:19:42.048 INFO:tasks.kclient:Mounting kernel clients... 2023-11-08T08:19:42.049 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-08T08:19:42.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:42.050 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-11-08T08:19:42.070 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-11-08T08:19:42.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:42.072 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-08T08:19:42.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:42.152 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2023-11-08T08:19:42.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:42.361 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-11-08T08:19:42.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:42.376 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2023-11-08T08:19:42.463 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2023-11-08T08:19:42.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T08:19:42.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:42.468 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-11-08T08:19:42.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:42.490 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2023-11-08T08:19:42.570 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2023-11-08T08:19:42.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T08:19:42.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:42.574 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-11-08T08:19:42.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:42.633 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2023-11-08T08:19:42.718 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2023-11-08T08:19:42.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T08:19:42.722 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-11-08T08:19:42.722 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-08T08:19:42.722 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T08:19:42.723 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T08:19:42.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.906+0000 7f87ed10e700 1 Processor -- start 2023-11-08T08:19:42.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.906+0000 7f87ed10e700 1 -- start start 2023-11-08T08:19:42.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.910+0000 7f87ed10e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87e815c1a0 0x7f87e806c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:42.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.910+0000 7f87ed10e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e80adf90 con 0x7f87e80cdeb0 2023-11-08T08:19:42.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.910+0000 7f87ed10e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e80ce370 con 0x7f87e8070790 2023-11-08T08:19:42.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.910+0000 7f87ed10e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e8069bf0 con 0x7f87e815c1a0 2023-11-08T08:19:42.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.910+0000 7f87e77fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f87e80cdeb0 0x7f87e8070080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40660/0 (socket says 172.21.15.40:40660) 2023-11-08T08:19:42.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.910+0000 7f87ec90d700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f87e8070790 0x7f87e806a010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:43586/0 (socket says 172.21.15.40:43586) 2023-11-08T08:19:42.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.910+0000 7f87e77fe700 1 -- 172.21.15.40:0/74884514 learned_addr learned my addr 172.21.15.40:0/74884514 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:42.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.910+0000 7f87e7fff700 1 --2- 172.21.15.40:0/74884514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87e815c1a0 0x7f87e806c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:42.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.910+0000 7f87e6ffd700 1 -- 172.21.15.40:0/74884514 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585027275 0 0) 0x7f87e80ce370 con 0x7f87e8070790 2023-11-08T08:19:42.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.910+0000 7f87e6ffd700 1 -- 172.21.15.40:0/74884514 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87d00035d0 con 0x7f87e8070790 2023-11-08T08:19:42.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.910+0000 7f87e6ffd700 1 -- 172.21.15.40:0/74884514 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638609650 0 0) 0x7f87e80adf90 con 0x7f87e80cdeb0 2023-11-08T08:19:42.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.910+0000 7f87e6ffd700 1 -- 172.21.15.40:0/74884514 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87e80ce370 con 0x7f87e80cdeb0 2023-11-08T08:19:42.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.910+0000 7f87e6ffd700 1 -- 172.21.15.40:0/74884514 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1389307961 0 0) 0x7f87d00035d0 con 0x7f87e8070790 2023-11-08T08:19:42.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.910+0000 7f87e7fff700 1 -- 172.21.15.40:0/74884514 >> v1:172.21.15.169:6789/0 conn(0x7f87e8070790 legacy=0x7f87e806a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:42.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.910+0000 7f87e7fff700 1 -- 172.21.15.40:0/74884514 >> v1:172.21.15.40:6789/0 conn(0x7f87e80cdeb0 legacy=0x7f87e8070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:42.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.910+0000 7f87e7fff700 1 -- 172.21.15.40:0/74884514 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e80cc4a0 con 0x7f87e815c1a0 2023-11-08T08:19:42.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.914+0000 7f87e7fff700 1 --2- 172.21.15.40:0/74884514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87e815c1a0 0x7f87e806c8b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009880 tx=0x7f87d8009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66cbcf4281faa269 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:42.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.914+0000 7f87e6ffd700 1 -- 172.21.15.40:0/74884514 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc001610 con 0x7f87e815c1a0 2023-11-08T08:19:42.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.914+0000 7f87e6ffd700 1 -- 172.21.15.40:0/74884514 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d8010590 con 0x7f87e815c1a0 2023-11-08T08:19:42.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.914+0000 7f87e6ffd700 1 -- 172.21.15.40:0/74884514 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e00019f0 con 0x7f87e815c1a0 2023-11-08T08:19:42.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.914+0000 7f87ed10e700 1 -- 172.21.15.40:0/74884514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87e815c1a0 msgr2=0x7f87e806c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:42.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.914+0000 7f87ed10e700 1 --2- 172.21.15.40:0/74884514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87e815c1a0 0x7f87e806c8b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009880 tx=0x7f87d8009b90 comp rx=0 tx=0).stop 2023-11-08T08:19:42.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.914+0000 7f87ed10e700 1 -- 172.21.15.40:0/74884514 shutdown_connections 2023-11-08T08:19:42.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.914+0000 7f87ed10e700 1 --2- 172.21.15.40:0/74884514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87e815c1a0 0x7f87e806c8b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:42.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.914+0000 7f87ed10e700 1 -- 172.21.15.40:0/74884514 >> 172.21.15.40:0/74884514 conn(0x7f87e80b0f80 msgr2=0x7f87e80ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:42.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.914+0000 7f87ed10e700 1 -- 172.21.15.40:0/74884514 shutdown_connections 2023-11-08T08:19:42.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.914+0000 7f87ed10e700 1 -- 172.21.15.40:0/74884514 wait complete. 2023-11-08T08:19:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.918+0000 7f87ed10e700 1 Processor -- start 2023-11-08T08:19:42.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.918+0000 7f87ed10e700 1 -- start start 2023-11-08T08:19:42.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.918+0000 7f87ed10e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87e8070790 0x7f87e815a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:42.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.918+0000 7f87ed10e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87e80cdeb0 0x7f87e815ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:42.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.918+0000 7f87ed10e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87e815c1a0 0x7f87e8154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:42.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.918+0000 7f87ed10e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e815adb0 con 0x7f87e8070790 2023-11-08T08:19:42.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.918+0000 7f87ed10e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e81597d0 con 0x7f87e80cdeb0 2023-11-08T08:19:42.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.918+0000 7f87ed10e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e8159ab0 con 0x7f87e815c1a0 2023-11-08T08:19:42.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.918+0000 7f87e7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87e8070790 0x7f87e815a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:42.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.918+0000 7f87e7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87e8070790 0x7f87e815a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55646/0 (socket says 172.21.15.40:55646) 2023-11-08T08:19:42.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.918+0000 7f87e7fff700 1 -- 172.21.15.40:0/1610427988 learned_addr learned my addr 172.21.15.40:0/1610427988 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:42.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.918+0000 7f87ec90d700 1 --2- 172.21.15.40:0/1610427988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87e815c1a0 0x7f87e8154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:42.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.922+0000 7f87e77fe700 1 --2- 172.21.15.40:0/1610427988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87e80cdeb0 0x7f87e815ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:42.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.922+0000 7f87e7fff700 1 -- 172.21.15.40:0/1610427988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87e815c1a0 msgr2=0x7f87e8154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:42.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.922+0000 7f87e7fff700 1 --2- 172.21.15.40:0/1610427988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87e815c1a0 0x7f87e8154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:42.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.922+0000 7f87e7fff700 1 -- 172.21.15.40:0/1610427988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87e80cdeb0 msgr2=0x7f87e815ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:42.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.922+0000 7f87e7fff700 1 --2- 172.21.15.40:0/1610427988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87e80cdeb0 0x7f87e815ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:42.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.922+0000 7f87e7fff700 1 -- 172.21.15.40:0/1610427988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d80094b0 con 0x7f87e8070790 2023-11-08T08:19:42.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.922+0000 7f87ec90d700 1 --2- 172.21.15.40:0/1610427988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87e815c1a0 0x7f87e8154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:42.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.922+0000 7f87e77fe700 1 --2- 172.21.15.40:0/1610427988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87e80cdeb0 0x7f87e815ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.922+0000 7f87e7fff700 1 --2- 172.21.15.40:0/1610427988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87e8070790 0x7f87e815a870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f87d8012000 tx=0x7f87d8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.922+0000 7f87e4ff9700 1 -- 172.21.15.40:0/1610427988 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8003e00 con 0x7f87e8070790 2023-11-08T08:19:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.922+0000 7f87e4ff9700 1 -- 172.21.15.40:0/1610427988 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d8010700 con 0x7f87e8070790 2023-11-08T08:19:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.922+0000 7f87ed10e700 1 -- 172.21.15.40:0/1610427988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e806cdf0 con 0x7f87e8070790 2023-11-08T08:19:42.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.922+0000 7f87e4ff9700 1 -- 172.21.15.40:0/1610427988 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8010ab0 con 0x7f87e8070790 2023-11-08T08:19:42.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.922+0000 7f87ed10e700 1 -- 172.21.15.40:0/1610427988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e80cc780 con 0x7f87e8070790 2023-11-08T08:19:42.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.922+0000 7f87ed10e700 1 -- 172.21.15.40:0/1610427988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e815c570 con 0x7f87e8070790 2023-11-08T08:19:42.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.926+0000 7f87e4ff9700 1 -- 172.21.15.40:0/1610427988 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f87d802c070 con 0x7f87e8070790 2023-11-08T08:19:42.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.926+0000 7f87e4ff9700 1 --2- 172.21.15.40:0/1610427988 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f87d4064ee0 0x7f87d4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:42.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.926+0000 7f87e4ff9700 1 -- 172.21.15.40:0/1610427988 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f87d8083390 con 0x7f87e8070790 2023-11-08T08:19:42.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.926+0000 7f87e77fe700 1 --2- 172.21.15.40:0/1610427988 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f87d4064ee0 0x7f87d4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:42.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.930+0000 7f87e77fe700 1 --2- 172.21.15.40:0/1610427988 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f87d4064ee0 0x7f87d4067390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f87dc001eb0 tx=0x7f87dc0016b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:42.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:42.934+0000 7f87e4ff9700 1 -- 172.21.15.40:0/1610427988 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f87d8051180 con 0x7f87e8070790 2023-11-08T08:19:43.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:43.146+0000 7f87ed10e700 1 -- 172.21.15.40:0/1610427988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f87e8073cb0 con 0x7f87e8070790 2023-11-08T08:19:43.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:43.146+0000 7f87e4ff9700 1 -- 172.21.15.40:0/1610427988 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f87d80547a0 con 0x7f87e8070790 2023-11-08T08:19:43.149 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-08T08:19:43.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:43.150+0000 7f87ed10e700 1 -- 172.21.15.40:0/1610427988 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f87d4064ee0 msgr2=0x7f87d4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:43.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:43.150+0000 7f87ed10e700 1 --2- 172.21.15.40:0/1610427988 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f87d4064ee0 0x7f87d4067390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f87dc001eb0 tx=0x7f87dc0016b0 comp rx=0 tx=0).stop 2023-11-08T08:19:43.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:43.150+0000 7f87ed10e700 1 -- 172.21.15.40:0/1610427988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87e8070790 msgr2=0x7f87e815a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:43.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:43.150+0000 7f87ed10e700 1 --2- 172.21.15.40:0/1610427988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87e8070790 0x7f87e815a870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f87d8012000 tx=0x7f87d8004770 comp rx=0 tx=0).stop 2023-11-08T08:19:43.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:43.150+0000 7f87ed10e700 1 -- 172.21.15.40:0/1610427988 shutdown_connections 2023-11-08T08:19:43.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:43.150+0000 7f87ed10e700 1 --2- 172.21.15.40:0/1610427988 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f87d4064ee0 0x7f87d4067390 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:43.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:43.150+0000 7f87ed10e700 1 --2- 172.21.15.40:0/1610427988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87e815c1a0 0x7f87e8154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:43.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:43.150+0000 7f87ed10e700 1 --2- 172.21.15.40:0/1610427988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87e80cdeb0 0x7f87e815ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:43.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:43.150+0000 7f87ed10e700 1 --2- 172.21.15.40:0/1610427988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87e8070790 0x7f87e815a870 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:43.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:43.150+0000 7f87ed10e700 1 -- 172.21.15.40:0/1610427988 >> 172.21.15.40:0/1610427988 conn(0x7f87e80b0f80 msgr2=0x7f87e806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:43.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:43.150+0000 7f87ed10e700 1 -- 172.21.15.40:0/1610427988 shutdown_connections 2023-11-08T08:19:43.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:43.150+0000 7f87ed10e700 1 -- 172.21.15.40:0/1610427988 wait complete. 2023-11-08T08:19:43.177 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T08:19:43.178 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T08:19:43.178 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-11-08T08:19:43.178 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T08:19:43.179 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T08:19:43.179 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T08:19:43.179 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T08:19:43.179 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T08:19:43.180 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T08:19:43.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:43.181 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-11-08T08:19:43.198 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T08:19:43.199 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T08:19:43.199 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T08:19:43.199 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:43.199 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-11-08T08:19:43.200 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:43.200 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:19:43.200 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:43.200 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2023-11-08T08:19:43.200 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:19:43.201 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:43.201 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2023-11-08T08:19:43.201 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T08:19:43.201 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:43.201 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2023-11-08T08:19:43.202 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-08T08:19:43.202 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42337sec preferred_lft 42337sec 2023-11-08T08:19:43.202 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link 2023-11-08T08:19:43.202 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:43.203 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:19:43.203 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:43.203 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2023-11-08T08:19:43.203 INFO:teuthology.orchestra.run.smithi040.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-08T08:19:43.204 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 02:42:94:e4:3d:2e brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:43.204 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-08T08:19:43.204 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:43.204 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-08T08:19:43.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:43.205 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-11-08T08:19:43.205 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-11-08T08:19:43.206 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add name ceph-brx type bridge 2023-11-08T08:19:43.206 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr flush dev ceph-brx 2023-11-08T08:19:43.206 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set ceph-brx up 2023-11-08T08:19:43.207 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-08T08:19:43.207 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-11-08T08:19:43.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:43.360 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T08:19:43.398 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-11-08T08:19:43.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:43.402 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-11-08T08:19:43.469 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-11-08T08:19:43.470 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T08:19:43.470 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-08T08:19:43.470 INFO:teuthology.orchestra.run.smithi040.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-08T08:19:43.470 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-08T08:19:43.471 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T08:19:43.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:43.472 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-11-08T08:19:43.472 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-11-08T08:19:43.473 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-08T08:19:43.473 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-08T08:19:43.473 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-08T08:19:43.474 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-11-08T08:19:43.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:43.616 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-11-08T08:19:43.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:43.635 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2023-11-08T08:19:43.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:43.698 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-11-08T08:19:43.698 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-11-08T08:19:43.699 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T08:19:43.699 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-08T08:19:43.699 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-11-08T08:19:43.815 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-08T08:19:43.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:43.816 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-11-08T08:19:43.816 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-11-08T08:19:43.816 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-08T08:19:43.816 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-08T08:19:43.817 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-08T08:19:43.817 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-08T08:19:43.817 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-08T08:19:43.818 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-11-08T08:19:43.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:43.991 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-11-08T08:19:43.991 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-11-08T08:19:43.991 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2023-11-08T08:19:43.992 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.0 master ceph-brx 2023-11-08T08:19:43.992 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-11-08T08:19:44.068 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T08:19:44.069 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T08:19:44.085 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T08:19:44.086 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T08:19:44.086 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T08:19:44.151 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-08T08:19:44.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:44.152 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:19:44.289 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:19:44.289 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T08:19:44.425 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:19:44.426 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T08:19:44.426 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-08T08:19:44.426 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@54ed45f4-d5b0-41c3-8c3b-b5a18b844f25.cephfs=/ 2023-11-08T08:19:44.426 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2023-11-08T08:19:44.435 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T08:19:44.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:44.436 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T08:19:44.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:44.472 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T08:19:44.548 INFO:teuthology.orchestra.run.smithi040.stdout:client4613 2023-11-08T08:19:44.549 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:19:44.549 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:19:44.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:44.616 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T08:19:44.688 INFO:teuthology.orchestra.run.smithi040.stdout:client4613 2023-11-08T08:19:44.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:44.689 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T08:19:44.764 INFO:teuthology.orchestra.run.smithi040.stdout:client4613 2023-11-08T08:19:44.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:44.765 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/54ed45f4-d5b0-41c3-8c3b-b5a18b844f25.client4613/status') 2023-11-08T08:19:44.848 INFO:teuthology.orchestra.run.smithi040.stdout:instance: client.4613 (3)172.21.15.40:0/1390694828 2023-11-08T08:19:44.849 INFO:teuthology.orchestra.run.smithi040.stdout:blocklisted: false 2023-11-08T08:19:44.849 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records in 2023-11-08T08:19:44.849 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records out 2023-11-08T08:19:44.850 INFO:teuthology.orchestra.run.smithi040.stderr:70 bytes copied, 0.000166516 s, 420 kB/s 2023-11-08T08:19:44.851 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-11-08T08:19:44.852 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-08T08:19:44.852 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-08T08:19:44.852 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T08:19:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.058+0000 7fb143fff700 1 Processor -- start 2023-11-08T08:19:45.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.058+0000 7fb143fff700 1 -- start start 2023-11-08T08:19:45.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.062+0000 7fb143fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb14406cb00 0x7fb144167080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:45.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.062+0000 7fb143fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1440ce600 con 0x7fb1440cea70 2023-11-08T08:19:45.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.062+0000 7fb143fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14406afc0 con 0x7fb144159b10 2023-11-08T08:19:45.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.062+0000 7fb143fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1440ccb80 con 0x7fb14406cb00 2023-11-08T08:19:45.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.062+0000 7fb1427fc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb1440cea70 0x7fb14406c3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40662/0 (socket says 172.21.15.40:40662) 2023-11-08T08:19:45.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.062+0000 7fb1427fc700 1 -- 172.21.15.40:0/3479733993 learned_addr learned my addr 172.21.15.40:0/3479733993 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:45.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.062+0000 7fb142ffd700 1 --1- 172.21.15.40:0/3479733993 >> v1:172.21.15.169:6789/0 conn(0x7fb144159b10 0x7fb144071690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:43594/0 (socket says 172.21.15.40:43594) 2023-11-08T08:19:45.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.062+0000 7fb1437fe700 1 --2- 172.21.15.40:0/3479733993 >> v2:172.21.15.169:3300/0 conn(0x7fb14406cb00 0x7fb144167080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:45.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.062+0000 7fb141ffb700 1 -- 172.21.15.40:0/3479733993 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843868088 0 0) 0x7fb14406afc0 con 0x7fb144159b10 2023-11-08T08:19:45.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.062+0000 7fb141ffb700 1 -- 172.21.15.40:0/3479733993 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb12c0035d0 con 0x7fb144159b10 2023-11-08T08:19:45.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.062+0000 7fb141ffb700 1 -- 172.21.15.40:0/3479733993 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408777236 0 0) 0x7fb1440ce600 con 0x7fb1440cea70 2023-11-08T08:19:45.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.062+0000 7fb141ffb700 1 -- 172.21.15.40:0/3479733993 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb14406afc0 con 0x7fb1440cea70 2023-11-08T08:19:45.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.066+0000 7fb1437fe700 1 -- 172.21.15.40:0/3479733993 >> v1:172.21.15.169:6789/0 conn(0x7fb144159b10 legacy=0x7fb144071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:45.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.066+0000 7fb1437fe700 1 -- 172.21.15.40:0/3479733993 >> v1:172.21.15.40:6789/0 conn(0x7fb1440cea70 legacy=0x7fb14406c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:45.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.066+0000 7fb1437fe700 1 -- 172.21.15.40:0/3479733993 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14406b210 con 0x7fb14406cb00 2023-11-08T08:19:45.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.066+0000 7fb1437fe700 1 --2- 172.21.15.40:0/3479733993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb14406cb00 0x7fb144167080 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb13c009d90 tx=0x7fb13c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1e3fb28f5700d0c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:45.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.066+0000 7fb141ffb700 1 -- 172.21.15.40:0/3479733993 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c00e070 con 0x7fb14406cb00 2023-11-08T08:19:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.066+0000 7fb141ffb700 1 -- 172.21.15.40:0/3479733993 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13c00bb20 con 0x7fb14406cb00 2023-11-08T08:19:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.066+0000 7fb141ffb700 1 -- 172.21.15.40:0/3479733993 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c00be90 con 0x7fb14406cb00 2023-11-08T08:19:45.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.066+0000 7fb143fff700 1 -- 172.21.15.40:0/3479733993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb14406cb00 msgr2=0x7fb144167080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:45.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.066+0000 7fb143fff700 1 --2- 172.21.15.40:0/3479733993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb14406cb00 0x7fb144167080 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb13c009d90 tx=0x7fb13c00b330 comp rx=0 tx=0).stop 2023-11-08T08:19:45.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.066+0000 7fb143fff700 1 -- 172.21.15.40:0/3479733993 shutdown_connections 2023-11-08T08:19:45.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.066+0000 7fb143fff700 1 --2- 172.21.15.40:0/3479733993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb14406cb00 0x7fb144167080 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:45.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.066+0000 7fb143fff700 1 -- 172.21.15.40:0/3479733993 >> 172.21.15.40:0/3479733993 conn(0x7fb1440b0a20 msgr2=0x7fb1440b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:45.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.066+0000 7fb143fff700 1 -- 172.21.15.40:0/3479733993 shutdown_connections 2023-11-08T08:19:45.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.066+0000 7fb143fff700 1 -- 172.21.15.40:0/3479733993 wait complete. 2023-11-08T08:19:45.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.070+0000 7fb143fff700 1 Processor -- start 2023-11-08T08:19:45.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.070+0000 7fb143fff700 1 -- start start 2023-11-08T08:19:45.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb143fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb14406cb00 0x7fb1441f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:45.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb143fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1440cea70 0x7fb1440b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:45.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb143fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb144159b10 0x7fb1441542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:45.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb143fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1441f2a70 con 0x7fb14406cb00 2023-11-08T08:19:45.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb143fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14415a600 con 0x7fb144159b10 2023-11-08T08:19:45.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb143fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14415a8e0 con 0x7fb1440cea70 2023-11-08T08:19:45.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb142ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb14406cb00 0x7fb1441f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:45.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb142ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb14406cb00 0x7fb1441f2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55662/0 (socket says 172.21.15.40:55662) 2023-11-08T08:19:45.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb142ffd700 1 -- 172.21.15.40:0/1384133191 learned_addr learned my addr 172.21.15.40:0/1384133191 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:45.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb1437fe700 1 --2- 172.21.15.40:0/1384133191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb144159b10 0x7fb1441542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:45.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb1427fc700 1 --2- 172.21.15.40:0/1384133191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1440cea70 0x7fb1440b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:45.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb142ffd700 1 -- 172.21.15.40:0/1384133191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1440cea70 msgr2=0x7fb1440b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:45.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb142ffd700 1 --2- 172.21.15.40:0/1384133191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1440cea70 0x7fb1440b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:45.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb142ffd700 1 -- 172.21.15.40:0/1384133191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb144159b10 msgr2=0x7fb1441542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:45.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb142ffd700 1 --2- 172.21.15.40:0/1384133191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb144159b10 0x7fb1441542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:45.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb142ffd700 1 -- 172.21.15.40:0/1384133191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c009a70 con 0x7fb14406cb00 2023-11-08T08:19:45.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb1427fc700 1 --2- 172.21.15.40:0/1384133191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1440cea70 0x7fb1440b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:45.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb142ffd700 1 --2- 172.21.15.40:0/1384133191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb14406cb00 0x7fb1441f2d70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb13400c4d0 tx=0x7fb13400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:45.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb12bfff700 1 -- 172.21.15.40:0/1384133191 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134005cd0 con 0x7fb14406cb00 2023-11-08T08:19:45.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb12bfff700 1 -- 172.21.15.40:0/1384133191 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb134005e30 con 0x7fb14406cb00 2023-11-08T08:19:45.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb12bfff700 1 -- 172.21.15.40:0/1384133191 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134007310 con 0x7fb14406cb00 2023-11-08T08:19:45.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb143fff700 1 -- 172.21.15.40:0/1384133191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1440cd0d0 con 0x7fb14406cb00 2023-11-08T08:19:45.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb143fff700 1 -- 172.21.15.40:0/1384133191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14406b4f0 con 0x7fb14406cb00 2023-11-08T08:19:45.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.074+0000 7fb143fff700 1 -- 172.21.15.40:0/1384133191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb144070be0 con 0x7fb14406cb00 2023-11-08T08:19:45.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.078+0000 7fb12bfff700 1 -- 172.21.15.40:0/1384133191 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fb134019030 con 0x7fb14406cb00 2023-11-08T08:19:45.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.078+0000 7fb12bfff700 1 --2- 172.21.15.40:0/1384133191 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb130064f10 0x7fb1300673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:45.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.078+0000 7fb12bfff700 1 -- 172.21.15.40:0/1384133191 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb134081580 con 0x7fb14406cb00 2023-11-08T08:19:45.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.082+0000 7fb1427fc700 1 --2- 172.21.15.40:0/1384133191 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb130064f10 0x7fb1300673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:45.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.082+0000 7fb1427fc700 1 --2- 172.21.15.40:0/1384133191 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb130064f10 0x7fb1300673c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb14415a520 tx=0x7fb138009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:45.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.086+0000 7fb12bfff700 1 -- 172.21.15.40:0/1384133191 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fb13404f300 con 0x7fb14406cb00 2023-11-08T08:19:45.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.298+0000 7fb143fff700 1 -- 172.21.15.40:0/1384133191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb144073cb0 con 0x7fb14406cb00 2023-11-08T08:19:45.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.298+0000 7fb12bfff700 1 -- 172.21.15.40:0/1384133191 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fb134017070 con 0x7fb14406cb00 2023-11-08T08:19:45.302 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-08T08:19:45.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.302+0000 7fb143fff700 1 -- 172.21.15.40:0/1384133191 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb130064f10 msgr2=0x7fb1300673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:45.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.302+0000 7fb143fff700 1 --2- 172.21.15.40:0/1384133191 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb130064f10 0x7fb1300673c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb14415a520 tx=0x7fb138009040 comp rx=0 tx=0).stop 2023-11-08T08:19:45.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.302+0000 7fb143fff700 1 -- 172.21.15.40:0/1384133191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb14406cb00 msgr2=0x7fb1441f2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:45.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.302+0000 7fb143fff700 1 --2- 172.21.15.40:0/1384133191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb14406cb00 0x7fb1441f2d70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb13400c4d0 tx=0x7fb13400c7e0 comp rx=0 tx=0).stop 2023-11-08T08:19:45.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.302+0000 7fb143fff700 1 -- 172.21.15.40:0/1384133191 shutdown_connections 2023-11-08T08:19:45.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.302+0000 7fb143fff700 1 --2- 172.21.15.40:0/1384133191 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb130064f10 0x7fb1300673c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:45.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.302+0000 7fb143fff700 1 --2- 172.21.15.40:0/1384133191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb144159b10 0x7fb1441542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:45.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.302+0000 7fb143fff700 1 --2- 172.21.15.40:0/1384133191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1440cea70 0x7fb1440b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:45.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.302+0000 7fb143fff700 1 --2- 172.21.15.40:0/1384133191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb14406cb00 0x7fb1441f2d70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:45.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.302+0000 7fb143fff700 1 -- 172.21.15.40:0/1384133191 >> 172.21.15.40:0/1384133191 conn(0x7fb1440b0a20 msgr2=0x7fb1440b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:45.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.302+0000 7fb143fff700 1 -- 172.21.15.40:0/1384133191 shutdown_connections 2023-11-08T08:19:45.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:45.302+0000 7fb143fff700 1 -- 172.21.15.40:0/1384133191 wait complete. 2023-11-08T08:19:45.330 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T08:19:45.331 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T08:19:45.331 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-11-08T08:19:45.331 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-08T08:19:45.332 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-08T08:19:45.332 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T08:19:45.332 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T08:19:45.332 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T08:19:45.333 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-08T08:19:45.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:45.333 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-11-08T08:19:45.350 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T08:19:45.350 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T08:19:45.350 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T08:19:45.351 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:45.351 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-11-08T08:19:45.351 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:45.351 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:19:45.351 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:45.352 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2023-11-08T08:19:45.352 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:19:45.352 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:45.352 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2023-11-08T08:19:45.353 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T08:19:45.353 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:45.353 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2023-11-08T08:19:45.353 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-08T08:19:45.353 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42335sec preferred_lft 42335sec 2023-11-08T08:19:45.354 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link 2023-11-08T08:19:45.354 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:45.354 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:19:45.354 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:45.355 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2023-11-08T08:19:45.355 INFO:teuthology.orchestra.run.smithi040.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-08T08:19:45.355 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 02:42:94:e4:3d:2e brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:45.355 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-08T08:19:45.356 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:45.356 INFO:teuthology.orchestra.run.smithi040.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T08:19:45.356 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0e:a3:2a:89:6f:47 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:45.356 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T08:19:45.356 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:45.357 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a4cc:6bff:fe07:287b/64 scope link 2023-11-08T08:19:45.357 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:45.357 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T08:19:45.357 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0e:a3:2a:89:6f:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T08:19:45.357 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ca3:2aff:fe89:6f47/64 scope link tentative 2023-11-08T08:19:45.357 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:45.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:45.359 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T08:19:45.439 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-11-08T08:19:45.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:45.442 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-11-08T08:19:45.509 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-11-08T08:19:45.509 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T08:19:45.510 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-08T08:19:45.510 INFO:teuthology.orchestra.run.smithi040.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-08T08:19:45.510 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-08T08:19:45.510 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T08:19:45.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:45.511 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-11-08T08:19:45.511 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-11-08T08:19:45.512 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-08T08:19:45.512 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-08T08:19:45.512 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-08T08:19:45.512 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-11-08T08:19:45.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:45.656 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-11-08T08:19:45.673 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T08:19:45.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:45.676 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2023-11-08T08:19:45.737 INFO:teuthology.orchestra.run.smithi040.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-08T08:19:45.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:45.739 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-11-08T08:19:45.739 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-11-08T08:19:45.740 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T08:19:45.740 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-08T08:19:45.740 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-11-08T08:19:45.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:45.857 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-08T08:19:45.898 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T08:19:45.898 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T08:19:45.899 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T08:19:45.899 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:45.899 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-11-08T08:19:45.899 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:45.900 INFO:teuthology.orchestra.run.smithi040.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T08:19:45.900 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether be:87:bd:57:b2:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-08T08:19:45.901 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-08T08:19:45.901 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:45.901 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::bc87:bdff:fe57:b202/64 scope link 2023-11-08T08:19:45.901 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:45.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:45.904 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-08T08:19:45.986 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T08:19:45.986 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T08:19:45.986 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T08:19:45.987 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:45.987 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-11-08T08:19:45.988 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:45.988 INFO:teuthology.orchestra.run.smithi040.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T08:19:45.988 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether be:87:bd:57:b2:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-08T08:19:45.989 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-08T08:19:45.989 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:45.989 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::bc87:bdff:fe57:b202/64 scope link 2023-11-08T08:19:45.990 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:45.991 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-08T08:19:45.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:45.993 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-11-08T08:19:45.993 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-11-08T08:19:45.994 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-08T08:19:45.994 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-08T08:19:45.994 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-08T08:19:45.995 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-08T08:19:45.995 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-08T08:19:45.995 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-11-08T08:19:46.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:46.195 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-11-08T08:19:46.195 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-11-08T08:19:46.196 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.1 up 2023-11-08T08:19:46.196 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.1 master ceph-brx 2023-11-08T08:19:46.196 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-11-08T08:19:46.265 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T08:19:46.266 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-08T08:19:46.282 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-08T08:19:46.283 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T08:19:46.283 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-08T08:19:46.340 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-11-08T08:19:46.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:46.341 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:19:46.477 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:19:46.477 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T08:19:46.505 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:19:46.506 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T08:19:46.506 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-08T08:19:46.506 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 2@54ed45f4-d5b0-41c3-8c3b-b5a18b844f25.cephfs=/ 2023-11-08T08:19:46.507 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.40:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2023-11-08T08:19:46.512 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T08:19:46.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:46.513 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-08T08:19:46.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:46.547 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T08:19:46.615 INFO:teuthology.orchestra.run.smithi040.stdout:client4619 2023-11-08T08:19:46.615 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:19:46.616 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:19:46.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:46.675 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T08:19:46.748 INFO:teuthology.orchestra.run.smithi040.stdout:client4619 2023-11-08T08:19:46.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:46.749 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T08:19:46.819 INFO:teuthology.orchestra.run.smithi040.stdout:client4619 2023-11-08T08:19:46.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:46.820 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/54ed45f4-d5b0-41c3-8c3b-b5a18b844f25.client4619/status') 2023-11-08T08:19:46.906 INFO:teuthology.orchestra.run.smithi040.stdout:instance: client.4619 (3)172.21.15.40:0/606124117 2023-11-08T08:19:46.907 INFO:teuthology.orchestra.run.smithi040.stdout:blocklisted: false 2023-11-08T08:19:46.907 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records in 2023-11-08T08:19:46.907 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records out 2023-11-08T08:19:46.907 INFO:teuthology.orchestra.run.smithi040.stderr:69 bytes copied, 0.000164917 s, 418 kB/s 2023-11-08T08:19:46.910 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-11-08T08:19:46.910 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-08T08:19:46.910 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-08T08:19:46.911 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T08:19:47.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.138+0000 7f81b2a9f700 1 Processor -- start 2023-11-08T08:19:47.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.138+0000 7f81b2a9f700 1 -- start start 2023-11-08T08:19:47.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.142+0000 7f81b2a9f700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f81ac15c640 0x7f81ac0d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:47.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.142+0000 7f81b2a9f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ac06c600 con 0x7f81ac069e00 2023-11-08T08:19:47.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.142+0000 7f81b2a9f700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ac0722f0 con 0x7f81ac071aa0 2023-11-08T08:19:47.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.142+0000 7f81b2a9f700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ac06cef0 con 0x7f81ac15c640 2023-11-08T08:19:47.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.142+0000 7f81b229e700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f81ac069e00 0x7f81ac0d0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40666/0 (socket says 172.21.15.40:40666) 2023-11-08T08:19:47.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.142+0000 7f81b229e700 1 -- 172.21.15.40:0/3874283348 learned_addr learned my addr 172.21.15.40:0/3874283348 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:47.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.142+0000 7f81b1a9d700 1 --2- 172.21.15.40:0/3874283348 >> v2:172.21.15.169:3300/0 conn(0x7f81ac15c640 0x7f81ac0d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:47.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.142+0000 7f81b1a9d700 1 -- 172.21.15.40:0/3874283348 >> v1:172.21.15.169:6789/0 conn(0x7f81ac071aa0 legacy=0x7f81ac06d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:47.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.142+0000 7f81b1a9d700 1 -- 172.21.15.40:0/3874283348 >> v1:172.21.15.40:6789/0 conn(0x7f81ac069e00 legacy=0x7f81ac0d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:47.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.142+0000 7f81b1a9d700 1 -- 172.21.15.40:0/3874283348 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac06da20 con 0x7f81ac15c640 2023-11-08T08:19:47.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.142+0000 7f81b0a9b700 1 -- 172.21.15.40:0/3874283348 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181721277 0 0) 0x7f81ac06c600 con 0x7f81ac069e00 2023-11-08T08:19:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.142+0000 7f81b1a9d700 1 --2- 172.21.15.40:0/3874283348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81ac15c640 0x7f81ac0d31e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009850 tx=0x7f81a8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1724aa6df3cfe730 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.142+0000 7f81b0a9b700 1 -- 172.21.15.40:0/3874283348 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8011830 con 0x7f81ac15c640 2023-11-08T08:19:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.142+0000 7f81b0a9b700 1 -- 172.21.15.40:0/3874283348 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a8011990 con 0x7f81ac15c640 2023-11-08T08:19:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.142+0000 7f81b0a9b700 1 -- 172.21.15.40:0/3874283348 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8011d20 con 0x7f81ac15c640 2023-11-08T08:19:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.146+0000 7f81b2a9f700 1 -- 172.21.15.40:0/3874283348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81ac15c640 msgr2=0x7f81ac0d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:47.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.146+0000 7f81b2a9f700 1 --2- 172.21.15.40:0/3874283348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81ac15c640 0x7f81ac0d31e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009850 tx=0x7f81a8009b60 comp rx=0 tx=0).stop 2023-11-08T08:19:47.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.146+0000 7f81b2a9f700 1 -- 172.21.15.40:0/3874283348 shutdown_connections 2023-11-08T08:19:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.146+0000 7f81b2a9f700 1 --2- 172.21.15.40:0/3874283348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81ac15c640 0x7f81ac0d31e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.146+0000 7f81b2a9f700 1 -- 172.21.15.40:0/3874283348 >> 172.21.15.40:0/3874283348 conn(0x7f81ac0b6520 msgr2=0x7f81ac071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.146+0000 7f81b2a9f700 1 -- 172.21.15.40:0/3874283348 shutdown_connections 2023-11-08T08:19:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.146+0000 7f81b2a9f700 1 -- 172.21.15.40:0/3874283348 wait complete. 2023-11-08T08:19:47.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.146+0000 7f81b2a9f700 1 Processor -- start 2023-11-08T08:19:47.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.146+0000 7f81b2a9f700 1 -- start start 2023-11-08T08:19:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81b2a9f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81ac069e00 0x7f81ac0b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81b2a9f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81ac071aa0 0x7f81ac1f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81b2a9f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81ac15c640 0x7f81ac0ce750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81b2a9f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac0b3030 con 0x7f81ac071aa0 2023-11-08T08:19:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81b2a9f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac0cd7a0 con 0x7f81ac069e00 2023-11-08T08:19:47.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81b2a9f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac0b2e20 con 0x7f81ac15c640 2023-11-08T08:19:47.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81b129c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81ac071aa0 0x7f81ac1f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:47.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81b129c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81ac071aa0 0x7f81ac1f4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55678/0 (socket says 172.21.15.40:55678) 2023-11-08T08:19:47.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81b129c700 1 -- 172.21.15.40:0/2423936727 learned_addr learned my addr 172.21.15.40:0/2423936727 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:47.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81b1a9d700 1 --2- 172.21.15.40:0/2423936727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81ac069e00 0x7f81ac0b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:47.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81b229e700 1 --2- 172.21.15.40:0/2423936727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81ac15c640 0x7f81ac0ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:47.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81b129c700 1 -- 172.21.15.40:0/2423936727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81ac15c640 msgr2=0x7f81ac0ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:47.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81b129c700 1 --2- 172.21.15.40:0/2423936727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81ac15c640 0x7f81ac0ce750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:47.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81b129c700 1 -- 172.21.15.40:0/2423936727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81ac069e00 msgr2=0x7f81ac0b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:47.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81b129c700 1 --2- 172.21.15.40:0/2423936727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81ac069e00 0x7f81ac0b28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:47.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81b129c700 1 -- 172.21.15.40:0/2423936727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a80094b0 con 0x7f81ac071aa0 2023-11-08T08:19:47.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81b229e700 1 --2- 172.21.15.40:0/2423936727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81ac15c640 0x7f81ac0ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:47.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81b129c700 1 --2- 172.21.15.40:0/2423936727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81ac071aa0 0x7f81ac1f4cf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f819c00c500 tx=0x7f819c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:47.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81a27fc700 1 -- 172.21.15.40:0/2423936727 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c0052d0 con 0x7f81ac071aa0 2023-11-08T08:19:47.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81a27fc700 1 -- 172.21.15.40:0/2423936727 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819c005430 con 0x7f81ac071aa0 2023-11-08T08:19:47.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81a27fc700 1 -- 172.21.15.40:0/2423936727 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c0055a0 con 0x7f81ac071aa0 2023-11-08T08:19:47.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81b2a9f700 1 -- 172.21.15.40:0/2423936727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ac072010 con 0x7f81ac071aa0 2023-11-08T08:19:47.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.150+0000 7f81b2a9f700 1 -- 172.21.15.40:0/2423936727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ac15ca10 con 0x7f81ac071aa0 2023-11-08T08:19:47.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.154+0000 7f81b2a9f700 1 -- 172.21.15.40:0/2423936727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ac06c830 con 0x7f81ac071aa0 2023-11-08T08:19:47.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.154+0000 7f81a27fc700 1 -- 172.21.15.40:0/2423936727 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f819c019030 con 0x7f81ac071aa0 2023-11-08T08:19:47.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.158+0000 7f81a27fc700 1 --2- 172.21.15.40:0/2423936727 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8190064ee0 0x7f8190067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:47.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.158+0000 7f81a27fc700 1 -- 172.21.15.40:0/2423936727 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f819c081500 con 0x7f81ac071aa0 2023-11-08T08:19:47.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.158+0000 7f81b1a9d700 1 --2- 172.21.15.40:0/2423936727 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8190064ee0 0x7f8190067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:47.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.158+0000 7f81b1a9d700 1 --2- 172.21.15.40:0/2423936727 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8190064ee0 0x7f8190067390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f81a80090b0 tx=0x7f81a8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:47.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.162+0000 7f81a27fc700 1 -- 172.21.15.40:0/2423936727 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f819c04f280 con 0x7f81ac071aa0 2023-11-08T08:19:47.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.378+0000 7f81b2a9f700 1 -- 172.21.15.40:0/2423936727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f81ac073cb0 con 0x7f81ac071aa0 2023-11-08T08:19:47.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.382+0000 7f81a27fc700 1 -- 172.21.15.40:0/2423936727 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f819c0528a0 con 0x7f81ac071aa0 2023-11-08T08:19:47.384 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-08T08:19:47.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.382+0000 7f81b2a9f700 1 -- 172.21.15.40:0/2423936727 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8190064ee0 msgr2=0x7f8190067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:47.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.382+0000 7f81b2a9f700 1 --2- 172.21.15.40:0/2423936727 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8190064ee0 0x7f8190067390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f81a80090b0 tx=0x7f81a8008030 comp rx=0 tx=0).stop 2023-11-08T08:19:47.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.382+0000 7f81b2a9f700 1 -- 172.21.15.40:0/2423936727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81ac071aa0 msgr2=0x7f81ac1f4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:47.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.382+0000 7f81b2a9f700 1 --2- 172.21.15.40:0/2423936727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81ac071aa0 0x7f81ac1f4cf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f819c00c500 tx=0x7f819c00c810 comp rx=0 tx=0).stop 2023-11-08T08:19:47.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.382+0000 7f81b2a9f700 1 -- 172.21.15.40:0/2423936727 shutdown_connections 2023-11-08T08:19:47.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.382+0000 7f81b2a9f700 1 --2- 172.21.15.40:0/2423936727 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8190064ee0 0x7f8190067390 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:47.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.382+0000 7f81b2a9f700 1 --2- 172.21.15.40:0/2423936727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81ac15c640 0x7f81ac0ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:47.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.382+0000 7f81b2a9f700 1 --2- 172.21.15.40:0/2423936727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81ac071aa0 0x7f81ac1f4cf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:47.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.382+0000 7f81b2a9f700 1 --2- 172.21.15.40:0/2423936727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81ac069e00 0x7f81ac0b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:47.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.382+0000 7f81b2a9f700 1 -- 172.21.15.40:0/2423936727 >> 172.21.15.40:0/2423936727 conn(0x7f81ac0b6520 msgr2=0x7f81ac0b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:47.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.382+0000 7f81b2a9f700 1 -- 172.21.15.40:0/2423936727 shutdown_connections 2023-11-08T08:19:47.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:47.382+0000 7f81b2a9f700 1 -- 172.21.15.40:0/2423936727 wait complete. 2023-11-08T08:19:47.409 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T08:19:47.409 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T08:19:47.409 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2023-11-08T08:19:47.409 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-08T08:19:47.410 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-08T08:19:47.410 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T08:19:47.410 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T08:19:47.410 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T08:19:47.411 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-08T08:19:47.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:47.413 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2023-11-08T08:19:47.432 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T08:19:47.432 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T08:19:47.432 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T08:19:47.432 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:47.432 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-11-08T08:19:47.433 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:47.433 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:19:47.433 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:47.433 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2023-11-08T08:19:47.433 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:19:47.434 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:47.434 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2023-11-08T08:19:47.434 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:19:47.434 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:47.435 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T08:19:47.435 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:47.435 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-08T08:19:47.435 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42337sec preferred_lft 42337sec 2023-11-08T08:19:47.435 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2023-11-08T08:19:47.435 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:47.436 INFO:teuthology.orchestra.run.smithi169.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-08T08:19:47.436 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 02:42:fa:e3:02:50 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:47.436 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-08T08:19:47.436 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:47.437 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-08T08:19:47.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:47.437 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-11-08T08:19:47.438 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-11-08T08:19:47.438 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add name ceph-brx type bridge 2023-11-08T08:19:47.438 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr flush dev ceph-brx 2023-11-08T08:19:47.438 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set ceph-brx up 2023-11-08T08:19:47.439 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-08T08:19:47.439 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-11-08T08:19:47.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:47.588 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T08:19:47.621 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-11-08T08:19:47.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:47.625 DEBUG:teuthology.orchestra.run.smithi169:> route 2023-11-08T08:19:47.708 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2023-11-08T08:19:47.708 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T08:19:47.708 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-08T08:19:47.708 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T08:19:47.709 INFO:teuthology.orchestra.run.smithi169.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-08T08:19:47.709 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-08T08:19:47.709 INFO:teuthology.orchestra.run.smithi169.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-08T08:19:47.709 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T08:19:47.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:47.710 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-11-08T08:19:47.710 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-11-08T08:19:47.711 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T08:19:47.711 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T08:19:47.711 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T08:19:47.712 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-11-08T08:19:47.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:47.844 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-11-08T08:19:47.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:47.863 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list-id 2023-11-08T08:19:47.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:47.919 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-11-08T08:19:47.919 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-11-08T08:19:47.920 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T08:19:47.920 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-08T08:19:47.920 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-11-08T08:19:48.039 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-08T08:19:48.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:48.040 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-11-08T08:19:48.041 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-11-08T08:19:48.041 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-08T08:19:48.041 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-08T08:19:48.042 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-08T08:19:48.042 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-08T08:19:48.042 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-08T08:19:48.042 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-11-08T08:19:48.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:48.218 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-11-08T08:19:48.218 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-11-08T08:19:48.219 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2023-11-08T08:19:48.219 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set dev brx.0 master ceph-brx 2023-11-08T08:19:48.219 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-11-08T08:19:48.288 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T08:19:48.289 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-08T08:19:48.304 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-08T08:19:48.305 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T08:19:48.305 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-08T08:19:48.370 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-08T08:19:48.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:48.371 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:19:48.508 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:19:48.508 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T08:19:48.639 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:19:48.640 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T08:19:48.640 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-08T08:19:48.640 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with new device syntax: 1@54ed45f4-d5b0-41c3-8c3b-b5a18b844f25.cephfs=/ 2023-11-08T08:19:48.640 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.40:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2023-11-08T08:19:48.645 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T08:19:48.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:48.646 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-08T08:19:48.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:48.678 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T08:19:48.756 INFO:teuthology.orchestra.run.smithi169.stdout:client4830 2023-11-08T08:19:48.757 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:19:48.757 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:19:48.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:48.820 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T08:19:48.891 INFO:teuthology.orchestra.run.smithi169.stdout:client4830 2023-11-08T08:19:48.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:48.892 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T08:19:48.958 INFO:teuthology.orchestra.run.smithi169.stdout:client4830 2023-11-08T08:19:48.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:48.959 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/54ed45f4-d5b0-41c3-8c3b-b5a18b844f25.client4830/status') 2023-11-08T08:19:49.041 INFO:teuthology.orchestra.run.smithi169.stdout:instance: client.4830 (3)172.21.15.169:0/3983204608 2023-11-08T08:19:49.041 INFO:teuthology.orchestra.run.smithi169.stdout:blocklisted: false 2023-11-08T08:19:49.042 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records in 2023-11-08T08:19:49.042 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records out 2023-11-08T08:19:49.042 INFO:teuthology.orchestra.run.smithi169.stderr:71 bytes copied, 0.000174073 s, 408 kB/s 2023-11-08T08:19:49.043 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-11-08T08:19:49.044 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-08T08:19:49.044 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-08T08:19:49.044 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T08:19:49.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.218+0000 7f98d0d37700 1 Processor -- start 2023-11-08T08:19:49.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.222+0000 7f98d0d37700 1 -- start start 2023-11-08T08:19:49.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.222+0000 7f98d0d37700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f98cc06d4c0 0x7f98cc16b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:49.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.222+0000 7f98d0d37700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98cc164080 con 0x7f98cc0cdfb0 2023-11-08T08:19:49.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.222+0000 7f98d0d37700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98cc069bf0 con 0x7f98cc163cb0 2023-11-08T08:19:49.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.222+0000 7f98d0d37700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98cc0cc290 con 0x7f98cc06d4c0 2023-11-08T08:19:49.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.222+0000 7f98c3fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f98cc0cdfb0 0x7f98cc06cdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58462/0 (socket says 172.21.15.40:58462) 2023-11-08T08:19:49.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.222+0000 7f98c3fff700 1 -- 172.21.15.40:0/902612987 learned_addr learned my addr 172.21.15.40:0/902612987 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:49.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.226+0000 7f98cad9d700 1 --2- 172.21.15.40:0/902612987 >> v2:172.21.15.169:3300/0 conn(0x7f98cc06d4c0 0x7f98cc16b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:49.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.226+0000 7f98c9d9b700 1 -- 172.21.15.40:0/902612987 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748536193 0 0) 0x7f98cc069bf0 con 0x7f98cc163cb0 2023-11-08T08:19:49.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.226+0000 7f98c9d9b700 1 -- 172.21.15.40:0/902612987 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98b80035d0 con 0x7f98cc163cb0 2023-11-08T08:19:49.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.226+0000 7f98c9d9b700 1 -- 172.21.15.40:0/902612987 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411364656 0 0) 0x7f98cc164080 con 0x7f98cc0cdfb0 2023-11-08T08:19:49.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.226+0000 7f98c9d9b700 1 -- 172.21.15.40:0/902612987 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98cc069bf0 con 0x7f98cc0cdfb0 2023-11-08T08:19:49.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.226+0000 7f98cad9d700 1 -- 172.21.15.40:0/902612987 >> v1:172.21.15.169:6789/0 conn(0x7f98cc163cb0 legacy=0x7f98cc164cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:49.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.226+0000 7f98cad9d700 1 -- 172.21.15.40:0/902612987 >> v1:172.21.15.40:6789/0 conn(0x7f98cc0cdfb0 legacy=0x7f98cc06cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:49.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.226+0000 7f98cad9d700 1 -- 172.21.15.40:0/902612987 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc0cc4a0 con 0x7f98cc06d4c0 2023-11-08T08:19:49.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.226+0000 7f98cad9d700 1 --2- 172.21.15.40:0/902612987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98cc06d4c0 0x7f98cc16b630 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009d90 tx=0x7f98c400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb1257d6d6442c69 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:49.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.226+0000 7f98c9d9b700 1 -- 172.21.15.40:0/902612987 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c400e070 con 0x7f98cc06d4c0 2023-11-08T08:19:49.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.226+0000 7f98c9d9b700 1 -- 172.21.15.40:0/902612987 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c400bb20 con 0x7f98cc06d4c0 2023-11-08T08:19:49.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.226+0000 7f98c9d9b700 1 -- 172.21.15.40:0/902612987 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4012430 con 0x7f98cc06d4c0 2023-11-08T08:19:49.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.226+0000 7f98d0d37700 1 -- 172.21.15.40:0/902612987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98cc06d4c0 msgr2=0x7f98cc16b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:49.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.226+0000 7f98d0d37700 1 --2- 172.21.15.40:0/902612987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98cc06d4c0 0x7f98cc16b630 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009d90 tx=0x7f98c400b330 comp rx=0 tx=0).stop 2023-11-08T08:19:49.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.226+0000 7f98d0d37700 1 -- 172.21.15.40:0/902612987 shutdown_connections 2023-11-08T08:19:49.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.226+0000 7f98d0d37700 1 --2- 172.21.15.40:0/902612987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98cc06d4c0 0x7f98cc16b630 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:49.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.226+0000 7f98d0d37700 1 -- 172.21.15.40:0/902612987 >> 172.21.15.40:0/902612987 conn(0x7f98cc0cf350 msgr2=0x7f98cc0b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.226+0000 7f98d0d37700 1 -- 172.21.15.40:0/902612987 shutdown_connections 2023-11-08T08:19:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.230+0000 7f98d0d37700 1 -- 172.21.15.40:0/902612987 wait complete. 2023-11-08T08:19:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.230+0000 7f98d0d37700 1 Processor -- start 2023-11-08T08:19:49.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.230+0000 7f98d0d37700 1 -- start start 2023-11-08T08:19:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98d0d37700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98cc06d4c0 0x7f98cc15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:49.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98d0d37700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98cc0cdfb0 0x7f98cc0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:49.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98d0d37700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc163cb0 0x7f98cc0c4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:49.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98d0d37700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0b2b10 con 0x7f98cc163cb0 2023-11-08T08:19:49.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98d0d37700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0b21e0 con 0x7f98cc06d4c0 2023-11-08T08:19:49.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98d0d37700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0b24c0 con 0x7f98cc0cdfb0 2023-11-08T08:19:49.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98cad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc163cb0 0x7f98cc0c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:49.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98cad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc163cb0 0x7f98cc0c4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34836/0 (socket says 172.21.15.40:34836) 2023-11-08T08:19:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98cad9d700 1 -- 172.21.15.40:0/166633764 learned_addr learned my addr 172.21.15.40:0/166633764 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98c3fff700 1 --2- 172.21.15.40:0/166633764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98cc0cdfb0 0x7f98cc0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98ca59c700 1 --2- 172.21.15.40:0/166633764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98cc06d4c0 0x7f98cc15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98cad9d700 1 -- 172.21.15.40:0/166633764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98cc0cdfb0 msgr2=0x7f98cc0b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98cad9d700 1 --2- 172.21.15.40:0/166633764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98cc0cdfb0 0x7f98cc0b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:49.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98cad9d700 1 -- 172.21.15.40:0/166633764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98cc06d4c0 msgr2=0x7f98cc15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:49.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98cad9d700 1 --2- 172.21.15.40:0/166633764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98cc06d4c0 0x7f98cc15afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:49.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98cad9d700 1 -- 172.21.15.40:0/166633764 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c4009a70 con 0x7f98cc163cb0 2023-11-08T08:19:49.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98c3fff700 1 --2- 172.21.15.40:0/166633764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98cc0cdfb0 0x7f98cc0b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:49.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98ca59c700 1 --2- 172.21.15.40:0/166633764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98cc06d4c0 0x7f98cc15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:49.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98cad9d700 1 --2- 172.21.15.40:0/166633764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc163cb0 0x7f98cc0c4170 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f98c4013000 tx=0x7f98c4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:49.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98c2ffd700 1 -- 172.21.15.40:0/166633764 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c401b810 con 0x7f98cc163cb0 2023-11-08T08:19:49.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98c2ffd700 1 -- 172.21.15.40:0/166633764 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c4020070 con 0x7f98cc163cb0 2023-11-08T08:19:49.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98c2ffd700 1 -- 172.21.15.40:0/166633764 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c401bbb0 con 0x7f98cc163cb0 2023-11-08T08:19:49.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98d0d37700 1 -- 172.21.15.40:0/166633764 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98cc069e40 con 0x7f98cc163cb0 2023-11-08T08:19:49.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.234+0000 7f98d0d37700 1 -- 172.21.15.40:0/166633764 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc0cc780 con 0x7f98cc163cb0 2023-11-08T08:19:49.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.238+0000 7f98d0d37700 1 -- 172.21.15.40:0/166633764 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98cc06a120 con 0x7f98cc163cb0 2023-11-08T08:19:49.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.238+0000 7f98c2ffd700 1 -- 172.21.15.40:0/166633764 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f98c402e070 con 0x7f98cc163cb0 2023-11-08T08:19:49.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.242+0000 7f98c2ffd700 1 --2- 172.21.15.40:0/166633764 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98ac064ee0 0x7f98ac067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:49.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.242+0000 7f98c2ffd700 1 -- 172.21.15.40:0/166633764 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f98c4022070 con 0x7f98cc163cb0 2023-11-08T08:19:49.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.242+0000 7f98ca59c700 1 --2- 172.21.15.40:0/166633764 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98ac064ee0 0x7f98ac067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:49.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.242+0000 7f98ca59c700 1 --2- 172.21.15.40:0/166633764 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98ac064ee0 0x7f98ac067390 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0011e0 tx=0x7f98bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:49.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.246+0000 7f98c2ffd700 1 -- 172.21.15.40:0/166633764 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f98c40524e0 con 0x7f98cc163cb0 2023-11-08T08:19:49.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.454+0000 7f98d0d37700 1 -- 172.21.15.40:0/166633764 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f98cc0726c0 con 0x7f98cc163cb0 2023-11-08T08:19:49.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.458+0000 7f98c2ffd700 1 -- 172.21.15.40:0/166633764 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f98c402c070 con 0x7f98cc163cb0 2023-11-08T08:19:49.460 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-08T08:19:49.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.458+0000 7f98d0d37700 1 -- 172.21.15.40:0/166633764 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98ac064ee0 msgr2=0x7f98ac067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:49.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.458+0000 7f98d0d37700 1 --2- 172.21.15.40:0/166633764 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98ac064ee0 0x7f98ac067390 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0011e0 tx=0x7f98bc009040 comp rx=0 tx=0).stop 2023-11-08T08:19:49.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.458+0000 7f98d0d37700 1 -- 172.21.15.40:0/166633764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc163cb0 msgr2=0x7f98cc0c4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:49.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.458+0000 7f98d0d37700 1 --2- 172.21.15.40:0/166633764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc163cb0 0x7f98cc0c4170 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f98c4013000 tx=0x7f98c4004af0 comp rx=0 tx=0).stop 2023-11-08T08:19:49.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.458+0000 7f98d0d37700 1 -- 172.21.15.40:0/166633764 shutdown_connections 2023-11-08T08:19:49.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.458+0000 7f98d0d37700 1 --2- 172.21.15.40:0/166633764 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98ac064ee0 0x7f98ac067390 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:49.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.458+0000 7f98d0d37700 1 --2- 172.21.15.40:0/166633764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98cc163cb0 0x7f98cc0c4170 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:49.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.458+0000 7f98d0d37700 1 --2- 172.21.15.40:0/166633764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98cc0cdfb0 0x7f98cc0b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:49.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.458+0000 7f98d0d37700 1 --2- 172.21.15.40:0/166633764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98cc06d4c0 0x7f98cc15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:49.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.458+0000 7f98d0d37700 1 -- 172.21.15.40:0/166633764 >> 172.21.15.40:0/166633764 conn(0x7f98cc0cf350 msgr2=0x7f98cc06bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:49.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.458+0000 7f98d0d37700 1 -- 172.21.15.40:0/166633764 shutdown_connections 2023-11-08T08:19:49.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:49.458+0000 7f98d0d37700 1 -- 172.21.15.40:0/166633764 wait complete. 2023-11-08T08:19:49.488 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T08:19:49.489 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T08:19:49.489 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2023-11-08T08:19:49.489 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-08T08:19:49.489 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-08T08:19:49.490 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T08:19:49.490 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T08:19:49.490 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T08:19:49.490 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-08T08:19:49.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:49.491 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2023-11-08T08:19:49.510 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T08:19:49.510 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T08:19:49.510 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T08:19:49.510 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:49.511 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-11-08T08:19:49.511 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:49.511 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:19:49.511 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:49.512 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2023-11-08T08:19:49.512 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:19:49.512 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:49.512 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2023-11-08T08:19:49.512 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:19:49.513 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:49.513 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T08:19:49.513 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:49.513 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-08T08:19:49.513 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42335sec preferred_lft 42335sec 2023-11-08T08:19:49.514 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2023-11-08T08:19:49.514 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:49.514 INFO:teuthology.orchestra.run.smithi169.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-08T08:19:49.514 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 02:42:fa:e3:02:50 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:49.514 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-08T08:19:49.515 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:49.515 INFO:teuthology.orchestra.run.smithi169.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T08:19:49.515 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether fe:4b:52:bf:04:66 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:19:49.515 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T08:19:49.516 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:49.516 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::2412:36ff:fe6f:b42/64 scope link tentative 2023-11-08T08:19:49.516 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:49.517 INFO:teuthology.orchestra.run.smithi169.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T08:19:49.517 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether fe:4b:52:bf:04:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T08:19:49.517 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::fc4b:52ff:febf:466/64 scope link 2023-11-08T08:19:49.518 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:49.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:49.518 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T08:19:49.600 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-11-08T08:19:49.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:49.603 DEBUG:teuthology.orchestra.run.smithi169:> route 2023-11-08T08:19:49.670 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2023-11-08T08:19:49.670 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T08:19:49.674 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-08T08:19:49.674 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T08:19:49.674 INFO:teuthology.orchestra.run.smithi169.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-08T08:19:49.674 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-11-08T08:19:49.675 INFO:teuthology.orchestra.run.smithi169.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-08T08:19:49.675 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T08:19:49.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:49.676 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-11-08T08:19:49.677 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-11-08T08:19:49.677 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T08:19:49.677 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T08:19:49.678 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T08:19:49.678 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-11-08T08:19:49.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:49.811 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-11-08T08:19:49.828 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-08T08:19:49.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:49.830 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list-id 2023-11-08T08:19:49.889 INFO:teuthology.orchestra.run.smithi169.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-08T08:19:49.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:49.891 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-11-08T08:19:49.891 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-11-08T08:19:49.891 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-08T08:19:49.892 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-08T08:19:49.892 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-11-08T08:19:50.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:50.008 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-08T08:19:50.047 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T08:19:50.047 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T08:19:50.047 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T08:19:50.047 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:50.048 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-11-08T08:19:50.048 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:50.048 INFO:teuthology.orchestra.run.smithi169.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T08:19:50.048 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 62:a3:74:c0:e1:d8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-08T08:19:50.048 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-08T08:19:50.049 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:50.049 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::60a3:74ff:fec0:e1d8/64 scope link 2023-11-08T08:19:50.049 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:50.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:50.051 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-08T08:19:50.134 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T08:19:50.134 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T08:19:50.134 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T08:19:50.135 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:50.135 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-11-08T08:19:50.135 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:50.135 INFO:teuthology.orchestra.run.smithi169.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T08:19:50.135 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 62:a3:74:c0:e1:d8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-08T08:19:50.136 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-08T08:19:50.136 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:50.136 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::60a3:74ff:fec0:e1d8/64 scope link 2023-11-08T08:19:50.136 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:19:50.138 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-08T08:19:50.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:50.139 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-11-08T08:19:50.139 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-11-08T08:19:50.139 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-08T08:19:50.140 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-08T08:19:50.140 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-08T08:19:50.140 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-08T08:19:50.141 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-08T08:19:50.141 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-11-08T08:19:50.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:50.329 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-11-08T08:19:50.329 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-11-08T08:19:50.330 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.1 up 2023-11-08T08:19:50.330 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set dev brx.1 master ceph-brx 2023-11-08T08:19:50.330 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-11-08T08:19:50.400 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T08:19:50.400 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-08T08:19:50.413 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-08T08:19:50.414 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T08:19:50.414 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-08T08:19:50.472 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-11-08T08:19:50.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:50.473 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:19:50.597 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:19:50.598 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T08:19:50.618 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:19:50.619 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T08:19:50.619 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-08T08:19:50.619 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with new device syntax: 3@54ed45f4-d5b0-41c3-8c3b-b5a18b844f25.cephfs=/ 2023-11-08T08:19:50.619 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.40:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2023-11-08T08:19:50.624 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T08:19:50.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:50.625 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-08T08:19:50.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:50.659 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T08:19:50.731 INFO:teuthology.orchestra.run.smithi169.stdout:client4628 2023-11-08T08:19:50.732 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:19:50.732 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:19:50.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:50.795 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T08:19:50.870 INFO:teuthology.orchestra.run.smithi169.stdout:client4628 2023-11-08T08:19:50.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:50.871 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T08:19:50.940 INFO:teuthology.orchestra.run.smithi169.stdout:client4628 2023-11-08T08:19:50.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:19:50.941 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/54ed45f4-d5b0-41c3-8c3b-b5a18b844f25.client4628/status') 2023-11-08T08:19:51.027 INFO:teuthology.orchestra.run.smithi169.stdout:instance: client.4628 (3)192.168.0.2:0/3542434295 2023-11-08T08:19:51.027 INFO:teuthology.orchestra.run.smithi169.stdout:blocklisted: false 2023-11-08T08:19:51.027 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records in 2023-11-08T08:19:51.028 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records out 2023-11-08T08:19:51.028 INFO:teuthology.orchestra.run.smithi169.stderr:69 bytes copied, 0.00021149 s, 326 kB/s 2023-11-08T08:19:51.030 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-08T08:19:51.048 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:19:51.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.226+0000 7feac9834700 1 Processor -- start 2023-11-08T08:19:51.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.226+0000 7feac9834700 1 -- start start 2023-11-08T08:19:51.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.226+0000 7feac9834700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feac40cf8f0 0x7feac4159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:51.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.226+0000 7feac9834700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac40cebe0 con 0x7feac40b4b30 2023-11-08T08:19:51.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.226+0000 7feac9834700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac40cdfd0 con 0x7feac40b1b30 2023-11-08T08:19:51.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.226+0000 7feac9834700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac415a430 con 0x7feac40cf8f0 2023-11-08T08:19:51.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.226+0000 7feac3fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7feac40b4b30 0x7feac40b4f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58472/0 (socket says 172.21.15.40:58472) 2023-11-08T08:19:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.226+0000 7feac3fff700 1 -- 172.21.15.40:0/2077326521 learned_addr learned my addr 172.21.15.40:0/2077326521 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.226+0000 7feac27fc700 1 --2- 172.21.15.40:0/2077326521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feac40cf8f0 0x7feac4159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.226+0000 7feac1ffb700 1 -- 172.21.15.40:0/2077326521 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332772545 0 0) 0x7feac40cebe0 con 0x7feac40b4b30 2023-11-08T08:19:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.226+0000 7feac1ffb700 1 -- 172.21.15.40:0/2077326521 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaac0035d0 con 0x7feac40b4b30 2023-11-08T08:19:51.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac1ffb700 1 -- 172.21.15.40:0/2077326521 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318362989 0 0) 0x7feac40cdfd0 con 0x7feac40b1b30 2023-11-08T08:19:51.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac1ffb700 1 -- 172.21.15.40:0/2077326521 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feac40cebe0 con 0x7feac40b1b30 2023-11-08T08:19:51.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac1ffb700 1 -- 172.21.15.40:0/2077326521 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4167204668 0 0) 0x7feaac0035d0 con 0x7feac40b4b30 2023-11-08T08:19:51.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac1ffb700 1 -- 172.21.15.40:0/2077326521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feac40cf8f0 msgr2=0x7feac4159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:51.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac1ffb700 1 --2- 172.21.15.40:0/2077326521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feac40cf8f0 0x7feac4159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:51.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac1ffb700 1 -- 172.21.15.40:0/2077326521 >> v1:172.21.15.169:6789/0 conn(0x7feac40b1b30 legacy=0x7feac40b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:51.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac1ffb700 1 -- 172.21.15.40:0/2077326521 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac40ce2b0 con 0x7feac40b4b30 2023-11-08T08:19:51.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac1ffb700 1 -- 172.21.15.40:0/2077326521 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7feabc0019f0 con 0x7feac40b4b30 2023-11-08T08:19:51.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac1ffb700 1 -- 172.21.15.40:0/2077326521 >> v1:172.21.15.40:6789/0 conn(0x7feac40b4b30 legacy=0x7feac40b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:51.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac1ffb700 1 --2- 172.21.15.40:0/2077326521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaac003be0 0x7feaac003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac1ffb700 1 --2- 172.21.15.40:0/2077326521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaac004870 0x7feaac004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac1ffb700 1 --2- 172.21.15.40:0/2077326521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feac40cf8f0 0x7feaac005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac1ffb700 1 -- 172.21.15.40:0/2077326521 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac415a430 con 0x7feaac003be0 2023-11-08T08:19:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac1ffb700 1 -- 172.21.15.40:0/2077326521 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaac00bae0 con 0x7feaac004870 2023-11-08T08:19:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac1ffb700 1 -- 172.21.15.40:0/2077326521 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaac00bc50 con 0x7feac40cf8f0 2023-11-08T08:19:51.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac2ffd700 1 --2- 172.21.15.40:0/2077326521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaac003be0 0x7feaac003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:51.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac27fc700 1 --2- 172.21.15.40:0/2077326521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaac004870 0x7feaac004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:51.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac2ffd700 1 -- 172.21.15.40:0/2077326521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feac40cf8f0 msgr2=0x7feaac005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:51.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac2ffd700 1 --2- 172.21.15.40:0/2077326521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feac40cf8f0 0x7feaac005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:51.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac2ffd700 1 -- 172.21.15.40:0/2077326521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaac004870 msgr2=0x7feaac004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:51.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac2ffd700 1 --2- 172.21.15.40:0/2077326521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaac004870 0x7feaac004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:51.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac2ffd700 1 -- 172.21.15.40:0/2077326521 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaac0035d0 con 0x7feaac003be0 2023-11-08T08:19:51.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac27fc700 1 --2- 172.21.15.40:0/2077326521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaac004870 0x7feaac004ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:51.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac2ffd700 1 --2- 172.21.15.40:0/2077326521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaac003be0 0x7feaac003fd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feab4001960 tx=0x7feaac0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65c703f842bb91a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:51.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac1ffb700 1 -- 172.21.15.40:0/2077326521 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4001450 con 0x7feaac003be0 2023-11-08T08:19:51.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac1ffb700 1 -- 172.21.15.40:0/2077326521 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab40052f0 con 0x7feaac003be0 2023-11-08T08:19:51.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac1ffb700 1 -- 172.21.15.40:0/2077326521 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40036a0 con 0x7feaac003be0 2023-11-08T08:19:51.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac9834700 1 -- 172.21.15.40:0/2077326521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaac003be0 msgr2=0x7feaac003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:51.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac9834700 1 --2- 172.21.15.40:0/2077326521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaac003be0 0x7feaac003fd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feab4001960 tx=0x7feaac0058a0 comp rx=0 tx=0).stop 2023-11-08T08:19:51.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac3fff700 1 -- 172.21.15.40:0/2077326521 reap_dead start 2023-11-08T08:19:51.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac9834700 1 -- 172.21.15.40:0/2077326521 shutdown_connections 2023-11-08T08:19:51.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac9834700 1 -- 172.21.15.40:0/2077326521 >> 172.21.15.40:0/2077326521 conn(0x7feac40ccb30 msgr2=0x7feac40ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:51.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac9834700 1 -- 172.21.15.40:0/2077326521 shutdown_connections 2023-11-08T08:19:51.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.230+0000 7feac9834700 1 -- 172.21.15.40:0/2077326521 wait complete. 2023-11-08T08:19:51.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.234+0000 7feac9834700 1 Processor -- start 2023-11-08T08:19:51.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.234+0000 7feac9834700 1 -- start start 2023-11-08T08:19:51.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.234+0000 7feac9834700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaac003be0 0x7feac406b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:51.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.234+0000 7feac9834700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feac41f0c00 0x7feac41f1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:51.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.234+0000 7feac9834700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac4142260 0x7feac41426d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:51.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.234+0000 7feac9834700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac41f15b0 con 0x7feac4142260 2023-11-08T08:19:51.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.234+0000 7feac9834700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac406be50 con 0x7feaac003be0 2023-11-08T08:19:51.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.234+0000 7feac9834700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac406c0d0 con 0x7feac41f0c00 2023-11-08T08:19:51.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.234+0000 7feac3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac4142260 0x7feac41426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:51.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.234+0000 7feac3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac4142260 0x7feac41426d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34852/0 (socket says 172.21.15.40:34852) 2023-11-08T08:19:51.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.234+0000 7feac3fff700 1 -- 172.21.15.40:0/2786136 learned_addr learned my addr 172.21.15.40:0/2786136 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:51.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.234+0000 7feac2ffd700 1 --2- 172.21.15.40:0/2786136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaac003be0 0x7feac406b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:51.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.234+0000 7feac27fc700 1 --2- 172.21.15.40:0/2786136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feac41f0c00 0x7feac41f1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:51.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.238+0000 7feac3fff700 1 -- 172.21.15.40:0/2786136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feac41f0c00 msgr2=0x7feac41f1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:51.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.238+0000 7feac3fff700 1 --2- 172.21.15.40:0/2786136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feac41f0c00 0x7feac41f1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:51.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.238+0000 7feac3fff700 1 -- 172.21.15.40:0/2786136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaac003be0 msgr2=0x7feac406b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:51.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.238+0000 7feac3fff700 1 --2- 172.21.15.40:0/2786136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaac003be0 0x7feac406b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:51.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.238+0000 7feac3fff700 1 -- 172.21.15.40:0/2786136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac40ce870 con 0x7feac4142260 2023-11-08T08:19:51.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.238+0000 7feac3fff700 1 --2- 172.21.15.40:0/2786136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac4142260 0x7feac41426d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feac40af7d0 tx=0x7feabc00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:51.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.238+0000 7feaabfff700 1 -- 172.21.15.40:0/2786136 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc00a9f0 con 0x7feac4142260 2023-11-08T08:19:51.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.238+0000 7feaabfff700 1 -- 172.21.15.40:0/2786136 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feabc00ab70 con 0x7feac4142260 2023-11-08T08:19:51.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.238+0000 7feaabfff700 1 -- 172.21.15.40:0/2786136 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc007030 con 0x7feac4142260 2023-11-08T08:19:51.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.238+0000 7feac9834700 1 -- 172.21.15.40:0/2786136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac40ce590 con 0x7feac4142260 2023-11-08T08:19:51.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.238+0000 7feac9834700 1 -- 172.21.15.40:0/2786136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac40cdc80 con 0x7feac4142260 2023-11-08T08:19:51.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.238+0000 7feac9834700 1 -- 172.21.15.40:0/2786136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac40726c0 con 0x7feac4142260 2023-11-08T08:19:51.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.242+0000 7feaabfff700 1 -- 172.21.15.40:0/2786136 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7feabc010070 con 0x7feac4142260 2023-11-08T08:19:51.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.242+0000 7feaabfff700 1 --2- 172.21.15.40:0/2786136 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feab0064f00 0x7feab00673b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:51.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.242+0000 7feaabfff700 1 -- 172.21.15.40:0/2786136 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7feabc0816b0 con 0x7feac4142260 2023-11-08T08:19:51.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.242+0000 7feac2ffd700 1 --2- 172.21.15.40:0/2786136 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feab0064f00 0x7feab00673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:51.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.246+0000 7feac2ffd700 1 --2- 172.21.15.40:0/2786136 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feab0064f00 0x7feab00673b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feac406b700 tx=0x7feab400a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:51.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.250+0000 7feaabfff700 1 -- 172.21.15.40:0/2786136 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7feabc04f520 con 0x7feac4142260 2023-11-08T08:19:51.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.434+0000 7feac9834700 1 -- 172.21.15.40:0/2786136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feac40cf430 con 0x7feac4142260 2023-11-08T08:19:51.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.438+0000 7feaabfff700 1 -- 172.21.15.40:0/2786136 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19748 (secure 0 0 0) 0x7feabc018090 con 0x7feac4142260 2023-11-08T08:19:51.441 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:51.441 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":30,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:45.885827+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:19:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.438+0000 7feac9834700 1 -- 172.21.15.40:0/2786136 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feab0064f00 msgr2=0x7feab00673b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.438+0000 7feac9834700 1 --2- 172.21.15.40:0/2786136 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feab0064f00 0x7feab00673b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feac406b700 tx=0x7feab400a000 comp rx=0 tx=0).stop 2023-11-08T08:19:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.438+0000 7feac9834700 1 -- 172.21.15.40:0/2786136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac4142260 msgr2=0x7feac41426d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:51.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.438+0000 7feac9834700 1 --2- 172.21.15.40:0/2786136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac4142260 0x7feac41426d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feac40af7d0 tx=0x7feabc00cf20 comp rx=0 tx=0).stop 2023-11-08T08:19:51.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.442+0000 7feac9834700 1 -- 172.21.15.40:0/2786136 shutdown_connections 2023-11-08T08:19:51.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.442+0000 7feac9834700 1 --2- 172.21.15.40:0/2786136 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feab0064f00 0x7feab00673b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:51.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.442+0000 7feac9834700 1 --2- 172.21.15.40:0/2786136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac4142260 0x7feac41426d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:51.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.442+0000 7feac9834700 1 --2- 172.21.15.40:0/2786136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feac41f0c00 0x7feac41f1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:51.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.442+0000 7feac9834700 1 --2- 172.21.15.40:0/2786136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaac003be0 0x7feac406b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:51.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.442+0000 7feac9834700 1 -- 172.21.15.40:0/2786136 >> 172.21.15.40:0/2786136 conn(0x7feac40ccb30 msgr2=0x7feac406fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:51.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.442+0000 7feac9834700 1 -- 172.21.15.40:0/2786136 shutdown_connections 2023-11-08T08:19:51.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.442+0000 7feac9834700 1 -- 172.21.15.40:0/2786136 wait complete. 2023-11-08T08:19:51.467 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T08:19:02.627691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T08:19:03.473821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T08:19:06.539942+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T08:19:51.468 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:19:51.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.654+0000 7fb3ccfa8700 1 Processor -- start 2023-11-08T08:19:51.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.654+0000 7fb3ccfa8700 1 -- start start 2023-11-08T08:19:51.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.658+0000 7fb3ccfa8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3c8070620 0x7fb3c80b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:51.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.658+0000 7fb3ccfa8700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3c815c520 con 0x7fb3c815c150 2023-11-08T08:19:51.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.658+0000 7fb3ccfa8700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3c806cc50 con 0x7fb3c8069bf0 2023-11-08T08:19:51.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.658+0000 7fb3ccfa8700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3c80b4240 con 0x7fb3c8070620 2023-11-08T08:19:51.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.658+0000 7fb3c6ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3c8070620 0x7fb3c80b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:51.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.658+0000 7fb3c6ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3c8070620 0x7fb3c80b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34868/0 (socket says 172.21.15.40:34868) 2023-11-08T08:19:51.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.658+0000 7fb3c6ffd700 1 -- 172.21.15.40:0/3635018325 learned_addr learned my addr 172.21.15.40:0/3635018325 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:51.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.658+0000 7fb3c6ffd700 1 -- 172.21.15.40:0/3635018325 >> v1:172.21.15.169:6789/0 conn(0x7fb3c8069bf0 legacy=0x7fb3c8069ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:51.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.658+0000 7fb3c6ffd700 1 -- 172.21.15.40:0/3635018325 >> v1:172.21.15.40:6789/0 conn(0x7fb3c815c150 legacy=0x7fb3c806ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:51.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.658+0000 7fb3c6ffd700 1 -- 172.21.15.40:0/3635018325 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c80d2450 con 0x7fb3c8070620 2023-11-08T08:19:51.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.658+0000 7fb3c67fc700 1 -- 172.21.15.40:0/3635018325 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824512510 0 0) 0x7fb3c815c520 con 0x7fb3c815c150 2023-11-08T08:19:51.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.658+0000 7fb3c6ffd700 1 --2- 172.21.15.40:0/3635018325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c8070620 0x7fb3c80b3d00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009910 tx=0x7fb3bc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48311225fa7a01b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:51.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.662+0000 7fb3c67fc700 1 -- 172.21.15.40:0/3635018325 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc005e50 con 0x7fb3c8070620 2023-11-08T08:19:51.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.662+0000 7fb3c67fc700 1 -- 172.21.15.40:0/3635018325 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3bc009e60 con 0x7fb3c8070620 2023-11-08T08:19:51.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.662+0000 7fb3c67fc700 1 -- 172.21.15.40:0/3635018325 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc00b870 con 0x7fb3c8070620 2023-11-08T08:19:51.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.662+0000 7fb3ccfa8700 1 -- 172.21.15.40:0/3635018325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c8070620 msgr2=0x7fb3c80b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:51.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.662+0000 7fb3ccfa8700 1 --2- 172.21.15.40:0/3635018325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c8070620 0x7fb3c80b3d00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009910 tx=0x7fb3bc009c20 comp rx=0 tx=0).stop 2023-11-08T08:19:51.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.662+0000 7fb3ccfa8700 1 -- 172.21.15.40:0/3635018325 shutdown_connections 2023-11-08T08:19:51.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.662+0000 7fb3ccfa8700 1 --2- 172.21.15.40:0/3635018325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c8070620 0x7fb3c80b3d00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:51.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.662+0000 7fb3ccfa8700 1 -- 172.21.15.40:0/3635018325 >> 172.21.15.40:0/3635018325 conn(0x7fb3c80d3680 msgr2=0x7fb3c80cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:51.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.662+0000 7fb3ccfa8700 1 -- 172.21.15.40:0/3635018325 shutdown_connections 2023-11-08T08:19:51.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.662+0000 7fb3ccfa8700 1 -- 172.21.15.40:0/3635018325 wait complete. 2023-11-08T08:19:51.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.666+0000 7fb3ccfa8700 1 Processor -- start 2023-11-08T08:19:51.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.666+0000 7fb3ccfa8700 1 -- start start 2023-11-08T08:19:51.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.666+0000 7fb3ccfa8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3c8069bf0 0x7fb3c814c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:51.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.666+0000 7fb3ccfa8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c8070620 0x7fb3c80b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:51.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.666+0000 7fb3ccfa8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3c815c150 0x7fb3c80b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:51.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.666+0000 7fb3ccfa8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c80caa90 con 0x7fb3c8070620 2023-11-08T08:19:51.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.666+0000 7fb3ccfa8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c80cb330 con 0x7fb3c815c150 2023-11-08T08:19:51.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.666+0000 7fb3ccfa8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c80cde40 con 0x7fb3c8069bf0 2023-11-08T08:19:51.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.666+0000 7fb3c6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c8070620 0x7fb3c80b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:51.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.666+0000 7fb3c6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c8070620 0x7fb3c80b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34880/0 (socket says 172.21.15.40:34880) 2023-11-08T08:19:51.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.666+0000 7fb3c6ffd700 1 -- 172.21.15.40:0/2866496525 learned_addr learned my addr 172.21.15.40:0/2866496525 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:51.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.666+0000 7fb3c7fff700 1 --2- 172.21.15.40:0/2866496525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3c815c150 0x7fb3c80b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:51.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.666+0000 7fb3c77fe700 1 --2- 172.21.15.40:0/2866496525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3c8069bf0 0x7fb3c814c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:51.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.670+0000 7fb3c6ffd700 1 -- 172.21.15.40:0/2866496525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3c8069bf0 msgr2=0x7fb3c814c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:51.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.670+0000 7fb3c6ffd700 1 --2- 172.21.15.40:0/2866496525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3c8069bf0 0x7fb3c814c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:51.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.670+0000 7fb3c6ffd700 1 -- 172.21.15.40:0/2866496525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3c815c150 msgr2=0x7fb3c80b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:51.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.670+0000 7fb3c6ffd700 1 --2- 172.21.15.40:0/2866496525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3c815c150 0x7fb3c80b4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:51.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.670+0000 7fb3c6ffd700 1 -- 172.21.15.40:0/2866496525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3bc009520 con 0x7fb3c8070620 2023-11-08T08:19:51.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.670+0000 7fb3c6ffd700 1 --2- 172.21.15.40:0/2866496525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c8070620 0x7fb3c80b4610 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc012000 tx=0x7fb3bc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:51.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.670+0000 7fb3affff700 1 -- 172.21.15.40:0/2866496525 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc00bdd0 con 0x7fb3c8070620 2023-11-08T08:19:51.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.670+0000 7fb3affff700 1 -- 172.21.15.40:0/2866496525 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3bc020070 con 0x7fb3c8070620 2023-11-08T08:19:51.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.670+0000 7fb3ccfa8700 1 -- 172.21.15.40:0/2866496525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c80d05e0 con 0x7fb3c8070620 2023-11-08T08:19:51.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.670+0000 7fb3affff700 1 -- 172.21.15.40:0/2866496525 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc006220 con 0x7fb3c8070620 2023-11-08T08:19:51.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.670+0000 7fb3ccfa8700 1 -- 172.21.15.40:0/2866496525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c80d2730 con 0x7fb3c8070620 2023-11-08T08:19:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.670+0000 7fb3ccfa8700 1 -- 172.21.15.40:0/2866496525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c80d2140 con 0x7fb3c8070620 2023-11-08T08:19:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.674+0000 7fb3affff700 1 -- 172.21.15.40:0/2866496525 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fb3bc02b020 con 0x7fb3c8070620 2023-11-08T08:19:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.674+0000 7fb3affff700 1 --2- 172.21.15.40:0/2866496525 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb3b4064ee0 0x7fb3b4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.674+0000 7fb3affff700 1 -- 172.21.15.40:0/2866496525 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb3bc083a60 con 0x7fb3c8070620 2023-11-08T08:19:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.674+0000 7fb3c77fe700 1 --2- 172.21.15.40:0/2866496525 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb3b4064ee0 0x7fb3b4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.678+0000 7fb3c77fe700 1 --2- 172.21.15.40:0/2866496525 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb3b4064ee0 0x7fb3b4067390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb3c808e350 tx=0x7fb3b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:51.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.682+0000 7fb3affff700 1 -- 172.21.15.40:0/2866496525 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fb3bc0518d0 con 0x7fb3c8070620 2023-11-08T08:19:51.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.850+0000 7fb3ccfa8700 1 -- 172.21.15.40:0/2866496525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb3c8159540 con 0x7fb3c8070620 2023-11-08T08:19:51.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.854+0000 7fb3affff700 1 -- 172.21.15.40:0/2866496525 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb3bc054ef0 con 0x7fb3c8070620 2023-11-08T08:19:51.857 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:19:51.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.854+0000 7fb3ccfa8700 1 -- 172.21.15.40:0/2866496525 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb3b4064ee0 msgr2=0x7fb3b4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:51.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.854+0000 7fb3ccfa8700 1 --2- 172.21.15.40:0/2866496525 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb3b4064ee0 0x7fb3b4067390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb3c808e350 tx=0x7fb3b8009040 comp rx=0 tx=0).stop 2023-11-08T08:19:51.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.854+0000 7fb3ccfa8700 1 -- 172.21.15.40:0/2866496525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c8070620 msgr2=0x7fb3c80b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:51.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.854+0000 7fb3ccfa8700 1 --2- 172.21.15.40:0/2866496525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c8070620 0x7fb3c80b4610 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc012000 tx=0x7fb3bc004770 comp rx=0 tx=0).stop 2023-11-08T08:19:51.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.858+0000 7fb3ccfa8700 1 -- 172.21.15.40:0/2866496525 shutdown_connections 2023-11-08T08:19:51.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.858+0000 7fb3ccfa8700 1 --2- 172.21.15.40:0/2866496525 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb3b4064ee0 0x7fb3b4067390 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:51.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.858+0000 7fb3ccfa8700 1 --2- 172.21.15.40:0/2866496525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3c815c150 0x7fb3c80b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:51.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.858+0000 7fb3ccfa8700 1 --2- 172.21.15.40:0/2866496525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c8070620 0x7fb3c80b4610 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:51.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.858+0000 7fb3ccfa8700 1 --2- 172.21.15.40:0/2866496525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3c8069bf0 0x7fb3c814c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:51.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.858+0000 7fb3ccfa8700 1 -- 172.21.15.40:0/2866496525 >> 172.21.15.40:0/2866496525 conn(0x7fb3c80d3680 msgr2=0x7fb3c806a710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:51.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.858+0000 7fb3ccfa8700 1 -- 172.21.15.40:0/2866496525 shutdown_connections 2023-11-08T08:19:51.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:51.858+0000 7fb3ccfa8700 1 -- 172.21.15.40:0/2866496525 wait complete. 2023-11-08T08:19:51.882 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:19:52.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.062+0000 7fdffad45700 1 Processor -- start 2023-11-08T08:19:52.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.062+0000 7fdffad45700 1 -- start start 2023-11-08T08:19:52.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.066+0000 7fdffad45700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdff40c7e50 0x7fdff41612d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:52.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.066+0000 7fdffad45700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdff40b20a0 0x7fdff4070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:52.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.066+0000 7fdffad45700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff406a600 con 0x7fdff41607f0 2023-11-08T08:19:52.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.066+0000 7fdffad45700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff40c79c0 con 0x7fdff40c7e50 2023-11-08T08:19:52.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.066+0000 7fdffad45700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff41619e0 con 0x7fdff40b20a0 2023-11-08T08:19:52.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.066+0000 7fdff9542700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdff40c7e50 0x7fdff41612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:52.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.066+0000 7fdff9542700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdff40c7e50 0x7fdff41612d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34882/0 (socket says 172.21.15.40:34882) 2023-11-08T08:19:52.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.066+0000 7fdff9542700 1 -- 172.21.15.40:0/3475602730 learned_addr learned my addr 172.21.15.40:0/3475602730 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:52.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.066+0000 7fdff9542700 1 -- 172.21.15.40:0/3475602730 >> v2:172.21.15.169:3300/0 conn(0x7fdff40b20a0 msgr2=0x7fdff4070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:52.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.066+0000 7fdff9542700 1 --2- 172.21.15.40:0/3475602730 >> v2:172.21.15.169:3300/0 conn(0x7fdff40b20a0 0x7fdff4070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.066+0000 7fdff9542700 1 -- 172.21.15.40:0/3475602730 >> v1:172.21.15.40:6789/0 conn(0x7fdff41607f0 legacy=0x7fdff4160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:52.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.066+0000 7fdff9542700 1 -- 172.21.15.40:0/3475602730 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff406a850 con 0x7fdff40c7e50 2023-11-08T08:19:52.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.070+0000 7fdff9542700 1 --2- 172.21.15.40:0/3475602730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff40c7e50 0x7fdff41612d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdff406fc00 tx=0x7fdfe4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30009468ac1d143e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:52.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.070+0000 7fdff8d41700 1 -- 172.21.15.40:0/3475602730 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe400b8a0 con 0x7fdff40c7e50 2023-11-08T08:19:52.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.070+0000 7fdff8d41700 1 -- 172.21.15.40:0/3475602730 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe400ba00 con 0x7fdff40c7e50 2023-11-08T08:19:52.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.070+0000 7fdff8d41700 1 -- 172.21.15.40:0/3475602730 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe400bd90 con 0x7fdff40c7e50 2023-11-08T08:19:52.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.070+0000 7fdffad45700 1 -- 172.21.15.40:0/3475602730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff40c7e50 msgr2=0x7fdff41612d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:52.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.070+0000 7fdffad45700 1 --2- 172.21.15.40:0/3475602730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff40c7e50 0x7fdff41612d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdff406fc00 tx=0x7fdfe4009bf0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.070+0000 7fdffad45700 1 -- 172.21.15.40:0/3475602730 shutdown_connections 2023-11-08T08:19:52.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.070+0000 7fdffad45700 1 --2- 172.21.15.40:0/3475602730 >> v2:172.21.15.169:3300/0 conn(0x7fdff40b20a0 0x7fdff4070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.070+0000 7fdffad45700 1 --2- 172.21.15.40:0/3475602730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff40c7e50 0x7fdff41612d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.070+0000 7fdffad45700 1 -- 172.21.15.40:0/3475602730 >> 172.21.15.40:0/3475602730 conn(0x7fdff40b53a0 msgr2=0x7fdff406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:52.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.070+0000 7fdffad45700 1 -- 172.21.15.40:0/3475602730 shutdown_connections 2023-11-08T08:19:52.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.070+0000 7fdffad45700 1 -- 172.21.15.40:0/3475602730 wait complete. 2023-11-08T08:19:52.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.070+0000 7fdffad45700 1 Processor -- start 2023-11-08T08:19:52.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.074+0000 7fdffad45700 1 -- start start 2023-11-08T08:19:52.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.074+0000 7fdffad45700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff40b20a0 0x7fdff41fa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:52.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.074+0000 7fdffad45700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdff40c7e50 0x7fdff415ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:52.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.074+0000 7fdffad45700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdff41607f0 0x7fdff415d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:52.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.074+0000 7fdffad45700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff415f150 con 0x7fdff40b20a0 2023-11-08T08:19:52.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.074+0000 7fdffad45700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff41638f0 con 0x7fdff41607f0 2023-11-08T08:19:52.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.074+0000 7fdffad45700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff4162ce0 con 0x7fdff40c7e50 2023-11-08T08:19:52.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.074+0000 7fdff9d43700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff40b20a0 0x7fdff41fa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:52.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.074+0000 7fdff9d43700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff40b20a0 0x7fdff41fa500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34896/0 (socket says 172.21.15.40:34896) 2023-11-08T08:19:52.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.074+0000 7fdff9d43700 1 -- 172.21.15.40:0/2769195733 learned_addr learned my addr 172.21.15.40:0/2769195733 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:52.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.074+0000 7fdff9542700 1 --2- 172.21.15.40:0/2769195733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdff40c7e50 0x7fdff415ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:52.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.074+0000 7fdffa544700 1 --2- 172.21.15.40:0/2769195733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdff41607f0 0x7fdff415d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:52.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.074+0000 7fdff9d43700 1 -- 172.21.15.40:0/2769195733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdff40c7e50 msgr2=0x7fdff415ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:52.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.074+0000 7fdff9d43700 1 --2- 172.21.15.40:0/2769195733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdff40c7e50 0x7fdff415ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.074+0000 7fdff9d43700 1 -- 172.21.15.40:0/2769195733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdff41607f0 msgr2=0x7fdff415d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:52.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.074+0000 7fdff9d43700 1 --2- 172.21.15.40:0/2769195733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdff41607f0 0x7fdff415d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.074+0000 7fdff9d43700 1 -- 172.21.15.40:0/2769195733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe4009570 con 0x7fdff40b20a0 2023-11-08T08:19:52.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.074+0000 7fdff9d43700 1 --2- 172.21.15.40:0/2769195733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff40b20a0 0x7fdff41fa500 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdff000c410 tx=0x7fdff000c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:52.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.078+0000 7fdfea7fc700 1 -- 172.21.15.40:0/2769195733 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff00052d0 con 0x7fdff40b20a0 2023-11-08T08:19:52.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.078+0000 7fdfea7fc700 1 -- 172.21.15.40:0/2769195733 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff0005430 con 0x7fdff40b20a0 2023-11-08T08:19:52.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.078+0000 7fdfea7fc700 1 -- 172.21.15.40:0/2769195733 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff00055a0 con 0x7fdff40b20a0 2023-11-08T08:19:52.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.078+0000 7fdffad45700 1 -- 172.21.15.40:0/2769195733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff406cd80 con 0x7fdff40b20a0 2023-11-08T08:19:52.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.078+0000 7fdffad45700 1 -- 172.21.15.40:0/2769195733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff406ab30 con 0x7fdff40b20a0 2023-11-08T08:19:52.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.078+0000 7fdffad45700 1 -- 172.21.15.40:0/2769195733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff40c9630 con 0x7fdff40b20a0 2023-11-08T08:19:52.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.078+0000 7fdfea7fc700 1 -- 172.21.15.40:0/2769195733 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fdff0010050 con 0x7fdff40b20a0 2023-11-08T08:19:52.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.082+0000 7fdfea7fc700 1 --2- 172.21.15.40:0/2769195733 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdfd8064f10 0x7fdfd80673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:52.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.082+0000 7fdfea7fc700 1 -- 172.21.15.40:0/2769195733 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdff0081290 con 0x7fdff40b20a0 2023-11-08T08:19:52.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.082+0000 7fdff9542700 1 --2- 172.21.15.40:0/2769195733 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdfd8064f10 0x7fdfd80673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:52.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.082+0000 7fdff9542700 1 --2- 172.21.15.40:0/2769195733 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdfd8064f10 0x7fdfd80673c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdff40c82b0 tx=0x7fdfe4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:52.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.090+0000 7fdfea7fc700 1 -- 172.21.15.40:0/2769195733 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fdff0049400 con 0x7fdff40b20a0 2023-11-08T08:19:52.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.266+0000 7fdffad45700 1 -- 172.21.15.40:0/2769195733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdff40ccf80 con 0x7fdff40b20a0 2023-11-08T08:19:52.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.266+0000 7fdfea7fc700 1 -- 172.21.15.40:0/2769195733 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdff004efb0 con 0x7fdff40b20a0 2023-11-08T08:19:52.269 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:19:52.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.270+0000 7fdffad45700 1 -- 172.21.15.40:0/2769195733 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdfd8064f10 msgr2=0x7fdfd80673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:52.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.270+0000 7fdffad45700 1 --2- 172.21.15.40:0/2769195733 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdfd8064f10 0x7fdfd80673c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdff40c82b0 tx=0x7fdfe4008030 comp rx=0 tx=0).stop 2023-11-08T08:19:52.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.270+0000 7fdffad45700 1 -- 172.21.15.40:0/2769195733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff40b20a0 msgr2=0x7fdff41fa500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:52.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.270+0000 7fdffad45700 1 --2- 172.21.15.40:0/2769195733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff40b20a0 0x7fdff41fa500 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdff000c410 tx=0x7fdff000c720 comp rx=0 tx=0).stop 2023-11-08T08:19:52.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.270+0000 7fdffad45700 1 -- 172.21.15.40:0/2769195733 shutdown_connections 2023-11-08T08:19:52.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.270+0000 7fdffad45700 1 --2- 172.21.15.40:0/2769195733 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdfd8064f10 0x7fdfd80673c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.270+0000 7fdffad45700 1 --2- 172.21.15.40:0/2769195733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdff41607f0 0x7fdff415d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.270+0000 7fdffad45700 1 --2- 172.21.15.40:0/2769195733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdff40c7e50 0x7fdff415ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.270+0000 7fdffad45700 1 --2- 172.21.15.40:0/2769195733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff40b20a0 0x7fdff41fa500 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.270+0000 7fdffad45700 1 -- 172.21.15.40:0/2769195733 >> 172.21.15.40:0/2769195733 conn(0x7fdff40b53a0 msgr2=0x7fdff40d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:52.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.270+0000 7fdffad45700 1 -- 172.21.15.40:0/2769195733 shutdown_connections 2023-11-08T08:19:52.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.270+0000 7fdffad45700 1 -- 172.21.15.40:0/2769195733 wait complete. 2023-11-08T08:19:52.292 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T08:19:52.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.466+0000 7ff8108d0700 1 Processor -- start 2023-11-08T08:19:52.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.466+0000 7ff8108d0700 1 -- start start 2023-11-08T08:19:52.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.466+0000 7ff8108d0700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff80c15e190 0x7ff80c0cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:52.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff8108d0700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff80c06ff10 0x7ff80c070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:52.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff8108d0700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80c0708b0 con 0x7ff80c161560 2023-11-08T08:19:52.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff8108d0700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff80c15e560 con 0x7ff80c15e190 2023-11-08T08:19:52.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff8108d0700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff80c06be80 con 0x7ff80c06ff10 2023-11-08T08:19:52.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff80ad9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff80c15e190 0x7ff80c0cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:52.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff80ad9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff80c15e190 0x7ff80c0cdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34900/0 (socket says 172.21.15.40:34900) 2023-11-08T08:19:52.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff80ad9d700 1 -- 172.21.15.40:0/1828017232 learned_addr learned my addr 172.21.15.40:0/1828017232 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:52.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff80b59e700 1 --2- 172.21.15.40:0/1828017232 >> v2:172.21.15.169:3300/0 conn(0x7ff80c06ff10 0x7ff80c070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:52.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff80ad9d700 1 -- 172.21.15.40:0/1828017232 >> v2:172.21.15.169:3300/0 conn(0x7ff80c06ff10 msgr2=0x7ff80c070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:52.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff80ad9d700 1 --2- 172.21.15.40:0/1828017232 >> v2:172.21.15.169:3300/0 conn(0x7ff80c06ff10 0x7ff80c070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff80ad9d700 1 -- 172.21.15.40:0/1828017232 >> v1:172.21.15.40:6789/0 conn(0x7ff80c161560 legacy=0x7ff80c069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:52.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff80ad9d700 1 -- 172.21.15.40:0/1828017232 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80c0ce330 con 0x7ff80c15e190 2023-11-08T08:19:52.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff809d9b700 1 -- 172.21.15.40:0/1828017232 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266667977 0 0) 0x7ff80c0708b0 con 0x7ff80c161560 2023-11-08T08:19:52.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff80ad9d700 1 --2- 172.21.15.40:0/1828017232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff80c15e190 0x7ff80c0cdc50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff80c06bbe0 tx=0x7ff7fc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ef0c93128cadff server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff809d9b700 1 -- 172.21.15.40:0/1828017232 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc011830 con 0x7ff80c15e190 2023-11-08T08:19:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff809d9b700 1 -- 172.21.15.40:0/1828017232 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7fc011990 con 0x7ff80c15e190 2023-11-08T08:19:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff809d9b700 1 -- 172.21.15.40:0/1828017232 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc011d20 con 0x7ff80c15e190 2023-11-08T08:19:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff8108d0700 1 -- 172.21.15.40:0/1828017232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff80c15e190 msgr2=0x7ff80c0cdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff8108d0700 1 --2- 172.21.15.40:0/1828017232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff80c15e190 0x7ff80c0cdc50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff80c06bbe0 tx=0x7ff7fc009bf0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff8108d0700 1 -- 172.21.15.40:0/1828017232 shutdown_connections 2023-11-08T08:19:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff8108d0700 1 --2- 172.21.15.40:0/1828017232 >> v2:172.21.15.169:3300/0 conn(0x7ff80c06ff10 0x7ff80c070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.470+0000 7ff8108d0700 1 --2- 172.21.15.40:0/1828017232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff80c15e190 0x7ff80c0cdc50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.474+0000 7ff8108d0700 1 -- 172.21.15.40:0/1828017232 >> 172.21.15.40:0/1828017232 conn(0x7ff80c0cf280 msgr2=0x7ff80c0cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:52.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.474+0000 7ff8108d0700 1 -- 172.21.15.40:0/1828017232 shutdown_connections 2023-11-08T08:19:52.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.474+0000 7ff8108d0700 1 -- 172.21.15.40:0/1828017232 wait complete. 2023-11-08T08:19:52.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.474+0000 7ff8108d0700 1 Processor -- start 2023-11-08T08:19:52.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.474+0000 7ff8108d0700 1 -- start start 2023-11-08T08:19:52.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff8108d0700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff80c06ff10 0x7ff80c1587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:52.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff8108d0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff80c15e190 0x7ff80c162950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:52.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff8108d0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff80c161560 0x7ff80c162e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:52.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff8108d0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c159220 con 0x7ff80c15e190 2023-11-08T08:19:52.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff8108d0700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c0c5630 con 0x7ff80c161560 2023-11-08T08:19:52.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff8108d0700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c0c5910 con 0x7ff80c06ff10 2023-11-08T08:19:52.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff80a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff80c15e190 0x7ff80c162950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:52.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff80a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff80c15e190 0x7ff80c162950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34906/0 (socket says 172.21.15.40:34906) 2023-11-08T08:19:52.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff80a59c700 1 -- 172.21.15.40:0/3737831915 learned_addr learned my addr 172.21.15.40:0/3737831915 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:52.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff80b59e700 1 --2- 172.21.15.40:0/3737831915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff80c161560 0x7ff80c162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:52.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff80ad9d700 1 --2- 172.21.15.40:0/3737831915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff80c06ff10 0x7ff80c1587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:52.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff80a59c700 1 -- 172.21.15.40:0/3737831915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff80c06ff10 msgr2=0x7ff80c1587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:52.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff80a59c700 1 --2- 172.21.15.40:0/3737831915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff80c06ff10 0x7ff80c1587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff80a59c700 1 -- 172.21.15.40:0/3737831915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff80c161560 msgr2=0x7ff80c162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:52.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff80a59c700 1 --2- 172.21.15.40:0/3737831915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff80c161560 0x7ff80c162e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff80a59c700 1 -- 172.21.15.40:0/3737831915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7fc009570 con 0x7ff80c15e190 2023-11-08T08:19:52.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff80a59c700 1 --2- 172.21.15.40:0/3737831915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff80c15e190 0x7ff80c162950 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff8000011e0 tx=0x7ff80000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:52.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff7f37fe700 1 -- 172.21.15.40:0/3737831915 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8000054e0 con 0x7ff80c15e190 2023-11-08T08:19:52.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff7f37fe700 1 -- 172.21.15.40:0/3737831915 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8000036a0 con 0x7ff80c15e190 2023-11-08T08:19:52.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff7f37fe700 1 -- 172.21.15.40:0/3737831915 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800003a50 con 0x7ff80c15e190 2023-11-08T08:19:52.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff8108d0700 1 -- 172.21.15.40:0/3737831915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80c0afc70 con 0x7ff80c15e190 2023-11-08T08:19:52.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.478+0000 7ff8108d0700 1 -- 172.21.15.40:0/3737831915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80c0af990 con 0x7ff80c15e190 2023-11-08T08:19:52.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.482+0000 7ff8108d0700 1 -- 172.21.15.40:0/3737831915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff80c0aff50 con 0x7ff80c15e190 2023-11-08T08:19:52.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.482+0000 7ff7f37fe700 1 -- 172.21.15.40:0/3737831915 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff800019030 con 0x7ff80c15e190 2023-11-08T08:19:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.482+0000 7ff7f37fe700 1 --2- 172.21.15.40:0/3737831915 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff7f8064ee0 0x7ff7f8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.482+0000 7ff7f37fe700 1 -- 172.21.15.40:0/3737831915 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff800081580 con 0x7ff80c15e190 2023-11-08T08:19:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.486+0000 7ff80ad9d700 1 --2- 172.21.15.40:0/3737831915 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff7f8064ee0 0x7ff7f8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:52.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.486+0000 7ff80ad9d700 1 --2- 172.21.15.40:0/3737831915 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff7f8064ee0 0x7ff7f8067390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc000c00 tx=0x7ff7fc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:52.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.490+0000 7ff7f37fe700 1 -- 172.21.15.40:0/3737831915 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ff80004f3f0 con 0x7ff80c15e190 2023-11-08T08:19:52.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.666+0000 7ff8108d0700 1 -- 172.21.15.40:0/3737831915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff80c073d20 con 0x7ff80c15e190 2023-11-08T08:19:52.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.666+0000 7ff7f37fe700 1 -- 172.21.15.40:0/3737831915 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff800017020 con 0x7ff80c15e190 2023-11-08T08:19:52.669 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:19:52.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.670+0000 7ff8108d0700 1 -- 172.21.15.40:0/3737831915 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff7f8064ee0 msgr2=0x7ff7f8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:52.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.670+0000 7ff8108d0700 1 --2- 172.21.15.40:0/3737831915 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff7f8064ee0 0x7ff7f8067390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc000c00 tx=0x7ff7fc008030 comp rx=0 tx=0).stop 2023-11-08T08:19:52.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.670+0000 7ff8108d0700 1 -- 172.21.15.40:0/3737831915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff80c15e190 msgr2=0x7ff80c162950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:52.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.670+0000 7ff8108d0700 1 --2- 172.21.15.40:0/3737831915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff80c15e190 0x7ff80c162950 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff8000011e0 tx=0x7ff80000ca30 comp rx=0 tx=0).stop 2023-11-08T08:19:52.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.670+0000 7ff8108d0700 1 -- 172.21.15.40:0/3737831915 shutdown_connections 2023-11-08T08:19:52.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.670+0000 7ff8108d0700 1 --2- 172.21.15.40:0/3737831915 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff7f8064ee0 0x7ff7f8067390 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.670+0000 7ff8108d0700 1 --2- 172.21.15.40:0/3737831915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff80c161560 0x7ff80c162e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.670+0000 7ff8108d0700 1 --2- 172.21.15.40:0/3737831915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff80c15e190 0x7ff80c162950 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.670+0000 7ff8108d0700 1 --2- 172.21.15.40:0/3737831915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff80c06ff10 0x7ff80c1587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.670+0000 7ff8108d0700 1 -- 172.21.15.40:0/3737831915 >> 172.21.15.40:0/3737831915 conn(0x7ff80c0cf280 msgr2=0x7ff80c070370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:52.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.670+0000 7ff8108d0700 1 -- 172.21.15.40:0/3737831915 shutdown_connections 2023-11-08T08:19:52.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.670+0000 7ff8108d0700 1 -- 172.21.15.40:0/3737831915 wait complete. 2023-11-08T08:19:52.697 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T08:19:52.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.882+0000 7f7173fff700 1 Processor -- start 2023-11-08T08:19:52.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.882+0000 7f7173fff700 1 -- start start 2023-11-08T08:19:52.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.886+0000 7f7173fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71740cefc0 0x7f71740ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:52.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.886+0000 7f7173fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71740ca550 con 0x7f7174160950 2023-11-08T08:19:52.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.886+0000 7f7173fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71741611a0 con 0x7f71740b1240 2023-11-08T08:19:52.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.886+0000 7f7173fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71740c9f40 con 0x7f71740cefc0 2023-11-08T08:19:52.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.886+0000 7f71737fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7174160950 0x7f71740cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58504/0 (socket says 172.21.15.40:58504) 2023-11-08T08:19:52.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.886+0000 7f71737fe700 1 -- 172.21.15.40:0/3530027594 learned_addr learned my addr 172.21.15.40:0/3530027594 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:52.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.886+0000 7f7172ffd700 1 --2- 172.21.15.40:0/3530027594 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71740cefc0 0x7f71740ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:52.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.886+0000 7f7171ffb700 1 -- 172.21.15.40:0/3530027594 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562454037 0 0) 0x7f71741611a0 con 0x7f71740b1240 2023-11-08T08:19:52.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.886+0000 7f7171ffb700 1 -- 172.21.15.40:0/3530027594 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f715c0035d0 con 0x7f71740b1240 2023-11-08T08:19:52.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.886+0000 7f7171ffb700 1 -- 172.21.15.40:0/3530027594 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213536062 0 0) 0x7f71740ca550 con 0x7f7174160950 2023-11-08T08:19:52.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.886+0000 7f7171ffb700 1 -- 172.21.15.40:0/3530027594 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71741611a0 con 0x7f7174160950 2023-11-08T08:19:52.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.886+0000 7f7172ffd700 1 -- 172.21.15.40:0/3530027594 >> v1:172.21.15.169:6789/0 conn(0x7f71740b1240 legacy=0x7f7174160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:52.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.886+0000 7f7172ffd700 1 -- 172.21.15.40:0/3530027594 >> v1:172.21.15.40:6789/0 conn(0x7f7174160950 legacy=0x7f71740cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:52.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.886+0000 7f7172ffd700 1 -- 172.21.15.40:0/3530027594 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7174160ec0 con 0x7f71740cefc0 2023-11-08T08:19:52.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.886+0000 7f7172ffd700 1 --2- 172.21.15.40:0/3530027594 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71740cefc0 0x7f71740ca9e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7164009880 tx=0x7f7164009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca316e4dbdbeeaca server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:52.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.890+0000 7f7171ffb700 1 -- 172.21.15.40:0/3530027594 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164011830 con 0x7f71740cefc0 2023-11-08T08:19:52.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.890+0000 7f7171ffb700 1 -- 172.21.15.40:0/3530027594 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7164011990 con 0x7f71740cefc0 2023-11-08T08:19:52.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.890+0000 7f7171ffb700 1 -- 172.21.15.40:0/3530027594 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164011d20 con 0x7f71740cefc0 2023-11-08T08:19:52.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.890+0000 7f7173fff700 1 -- 172.21.15.40:0/3530027594 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71740cefc0 msgr2=0x7f71740ca9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:52.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.890+0000 7f7173fff700 1 --2- 172.21.15.40:0/3530027594 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71740cefc0 0x7f71740ca9e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7164009880 tx=0x7f7164009b90 comp rx=0 tx=0).stop 2023-11-08T08:19:52.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.890+0000 7f7173fff700 1 -- 172.21.15.40:0/3530027594 shutdown_connections 2023-11-08T08:19:52.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.890+0000 7f7173fff700 1 --2- 172.21.15.40:0/3530027594 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71740cefc0 0x7f71740ca9e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.890+0000 7f7173fff700 1 -- 172.21.15.40:0/3530027594 >> 172.21.15.40:0/3530027594 conn(0x7f71740cc240 msgr2=0x7f71740cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:52.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.890+0000 7f7173fff700 1 -- 172.21.15.40:0/3530027594 shutdown_connections 2023-11-08T08:19:52.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.890+0000 7f7173fff700 1 -- 172.21.15.40:0/3530027594 wait complete. 2023-11-08T08:19:52.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.890+0000 7f7173fff700 1 Processor -- start 2023-11-08T08:19:52.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.890+0000 7f7173fff700 1 -- start start 2023-11-08T08:19:52.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f7173fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71740b1240 0x7f71741f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:52.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f7173fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71740cefc0 0x7f717415ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:52.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f7173fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7174160950 0x7f71740b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:52.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f7173fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71740ac780 con 0x7f71740b1240 2023-11-08T08:19:52.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f7173fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71741f3030 con 0x7f71740cefc0 2023-11-08T08:19:52.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f7173fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71740ab860 con 0x7f7174160950 2023-11-08T08:19:52.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f7172ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71740b1240 0x7f71741f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:52.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f7172ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71740b1240 0x7f71741f2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34908/0 (socket says 172.21.15.40:34908) 2023-11-08T08:19:52.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f7172ffd700 1 -- 172.21.15.40:0/547419300 learned_addr learned my addr 172.21.15.40:0/547419300 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:52.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f71737fe700 1 --2- 172.21.15.40:0/547419300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7174160950 0x7f71740b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:52.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f71727fc700 1 --2- 172.21.15.40:0/547419300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71740cefc0 0x7f717415ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:52.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f7172ffd700 1 -- 172.21.15.40:0/547419300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7174160950 msgr2=0x7f71740b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:52.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f7172ffd700 1 --2- 172.21.15.40:0/547419300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7174160950 0x7f71740b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f7172ffd700 1 -- 172.21.15.40:0/547419300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71740cefc0 msgr2=0x7f717415ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:52.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f7172ffd700 1 --2- 172.21.15.40:0/547419300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71740cefc0 0x7f717415ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:52.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f7172ffd700 1 -- 172.21.15.40:0/547419300 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71640094b0 con 0x7f71740b1240 2023-11-08T08:19:52.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f71727fc700 1 --2- 172.21.15.40:0/547419300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71740cefc0 0x7f717415ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:52.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f7172ffd700 1 --2- 172.21.15.40:0/547419300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71740b1240 0x7f71741f2af0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f71640090b0 tx=0x7f7164004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:52.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f715bfff700 1 -- 172.21.15.40:0/547419300 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716401a8c0 con 0x7f71740b1240 2023-11-08T08:19:52.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f715bfff700 1 -- 172.21.15.40:0/547419300 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716401e070 con 0x7f71740b1240 2023-11-08T08:19:52.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f7173fff700 1 -- 172.21.15.40:0/547419300 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71740caf20 con 0x7f71740b1240 2023-11-08T08:19:52.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f715bfff700 1 -- 172.21.15.40:0/547419300 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716401ac40 con 0x7f71740b1240 2023-11-08T08:19:52.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.894+0000 7f7173fff700 1 -- 172.21.15.40:0/547419300 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71740ca1e0 con 0x7f71740b1240 2023-11-08T08:19:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.898+0000 7f7173fff700 1 -- 172.21.15.40:0/547419300 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71740cf390 con 0x7f71740b1240 2023-11-08T08:19:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.898+0000 7f715bfff700 1 -- 172.21.15.40:0/547419300 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f716400f050 con 0x7f71740b1240 2023-11-08T08:19:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.898+0000 7f715bfff700 1 --2- 172.21.15.40:0/547419300 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7160064ee0 0x7f7160067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.898+0000 7f715bfff700 1 -- 172.21.15.40:0/547419300 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7164083160 con 0x7f71740b1240 2023-11-08T08:19:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.902+0000 7f71727fc700 1 --2- 172.21.15.40:0/547419300 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7160064ee0 0x7f7160067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:52.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.902+0000 7f71727fc700 1 --2- 172.21.15.40:0/547419300 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7160064ee0 0x7f7160067390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f71680011e0 tx=0x7f7168009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:52.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:52.906+0000 7f715bfff700 1 -- 172.21.15.40:0/547419300 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7164050ee0 con 0x7f71740b1240 2023-11-08T08:19:53.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.082+0000 7f7173fff700 1 -- 172.21.15.40:0/547419300 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f71740ac9f0 con 0x7f71740b1240 2023-11-08T08:19:53.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.086+0000 7f715bfff700 1 -- 172.21.15.40:0/547419300 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f71640200a0 con 0x7f71740b1240 2023-11-08T08:19:53.088 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-11-08T08:19:53.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.090+0000 7f7173fff700 1 -- 172.21.15.40:0/547419300 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7160064ee0 msgr2=0x7f7160067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:53.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.090+0000 7f7173fff700 1 --2- 172.21.15.40:0/547419300 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7160064ee0 0x7f7160067390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f71680011e0 tx=0x7f7168009040 comp rx=0 tx=0).stop 2023-11-08T08:19:53.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.090+0000 7f7173fff700 1 -- 172.21.15.40:0/547419300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71740b1240 msgr2=0x7f71741f2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:53.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.090+0000 7f7173fff700 1 --2- 172.21.15.40:0/547419300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71740b1240 0x7f71741f2af0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f71640090b0 tx=0x7f7164004770 comp rx=0 tx=0).stop 2023-11-08T08:19:53.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.090+0000 7f7173fff700 1 -- 172.21.15.40:0/547419300 shutdown_connections 2023-11-08T08:19:53.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.090+0000 7f7173fff700 1 --2- 172.21.15.40:0/547419300 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7160064ee0 0x7f7160067390 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.090+0000 7f7173fff700 1 --2- 172.21.15.40:0/547419300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7174160950 0x7f71740b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.090+0000 7f7173fff700 1 --2- 172.21.15.40:0/547419300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71740cefc0 0x7f717415ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.090+0000 7f7173fff700 1 --2- 172.21.15.40:0/547419300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71740b1240 0x7f71741f2af0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.090+0000 7f7173fff700 1 -- 172.21.15.40:0/547419300 >> 172.21.15.40:0/547419300 conn(0x7f71740cc240 msgr2=0x7f717415e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:53.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.090+0000 7f7173fff700 1 -- 172.21.15.40:0/547419300 shutdown_connections 2023-11-08T08:19:53.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.090+0000 7f7173fff700 1 -- 172.21.15.40:0/547419300 wait complete. 2023-11-08T08:19:53.121 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T08:19:53.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.298+0000 7f7efd158700 1 Processor -- start 2023-11-08T08:19:53.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.298+0000 7f7efd158700 1 -- start start 2023-11-08T08:19:53.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.298+0000 7f7efd158700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7ef815c150 0x7f7ef80b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:53.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.298+0000 7f7efd158700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ef8070360 0x7f7ef8160340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:53.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.298+0000 7f7efd158700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ef815c520 con 0x7f7ef806ff10 2023-11-08T08:19:53.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.298+0000 7f7efd158700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ef80b4240 con 0x7f7ef815c150 2023-11-08T08:19:53.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.298+0000 7f7efd158700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef806ac00 con 0x7f7ef8070360 2023-11-08T08:19:53.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.298+0000 7f7ef7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7ef815c150 0x7f7ef80b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:53.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.302+0000 7f7ef7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7ef815c150 0x7f7ef80b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34914/0 (socket says 172.21.15.40:34914) 2023-11-08T08:19:53.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.302+0000 7f7ef7fff700 1 -- 172.21.15.40:0/3654414118 learned_addr learned my addr 172.21.15.40:0/3654414118 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:53.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.302+0000 7f7efc957700 1 --2- 172.21.15.40:0/3654414118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ef8070360 0x7f7ef8160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:53.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.302+0000 7f7ef7fff700 1 -- 172.21.15.40:0/3654414118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ef8070360 msgr2=0x7f7ef8160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:53.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.302+0000 7f7ef7fff700 1 --2- 172.21.15.40:0/3654414118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ef8070360 0x7f7ef8160340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.302+0000 7f7ef7fff700 1 -- 172.21.15.40:0/3654414118 >> v1:172.21.15.169:6789/0 conn(0x7f7ef806ff10 legacy=0x7f7ef815fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:53.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.302+0000 7f7ef7fff700 1 -- 172.21.15.40:0/3654414118 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef806cc50 con 0x7f7ef815c150 2023-11-08T08:19:53.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.302+0000 7f7ef7fff700 1 --2- 172.21.15.40:0/3654414118 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ef815c150 0x7f7ef80b3d00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009910 tx=0x7f7ee8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fbcb83f1edab06 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:53.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.302+0000 7f7ef6ffd700 1 -- 172.21.15.40:0/3654414118 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee800ba60 con 0x7f7ef815c150 2023-11-08T08:19:53.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.302+0000 7f7ef6ffd700 1 -- 172.21.15.40:0/3654414118 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee800bbc0 con 0x7f7ef815c150 2023-11-08T08:19:53.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.302+0000 7f7ef6ffd700 1 -- 172.21.15.40:0/3654414118 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8009e60 con 0x7f7ef815c150 2023-11-08T08:19:53.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.302+0000 7f7efd158700 1 -- 172.21.15.40:0/3654414118 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ef815c150 msgr2=0x7f7ef80b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:53.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.302+0000 7f7efd158700 1 --2- 172.21.15.40:0/3654414118 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ef815c150 0x7f7ef80b3d00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009910 tx=0x7f7ee8009c20 comp rx=0 tx=0).stop 2023-11-08T08:19:53.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.302+0000 7f7efd158700 1 -- 172.21.15.40:0/3654414118 shutdown_connections 2023-11-08T08:19:53.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.302+0000 7f7efd158700 1 --2- 172.21.15.40:0/3654414118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ef8070360 0x7f7ef8160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.302+0000 7f7efd158700 1 --2- 172.21.15.40:0/3654414118 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ef815c150 0x7f7ef80b3d00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.302+0000 7f7efd158700 1 -- 172.21.15.40:0/3654414118 >> 172.21.15.40:0/3654414118 conn(0x7f7ef80d3680 msgr2=0x7f7ef80cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:53.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.302+0000 7f7efd158700 1 -- 172.21.15.40:0/3654414118 shutdown_connections 2023-11-08T08:19:53.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.302+0000 7f7efd158700 1 -- 172.21.15.40:0/3654414118 wait complete. 2023-11-08T08:19:53.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.306+0000 7f7efd158700 1 Processor -- start 2023-11-08T08:19:53.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.306+0000 7f7efd158700 1 -- start start 2023-11-08T08:19:53.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.306+0000 7f7efd158700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ef806ff10 0x7f7ef8143300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:53.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.306+0000 7f7efd158700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ef8070360 0x7f7ef80c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:53.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.306+0000 7f7efd158700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ef815c150 0x7f7ef80c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:53.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.306+0000 7f7efd158700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef8143840 con 0x7f7ef8070360 2023-11-08T08:19:53.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.306+0000 7f7efd158700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef8143f40 con 0x7f7ef806ff10 2023-11-08T08:19:53.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.306+0000 7f7efd158700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef80b70e0 con 0x7f7ef815c150 2023-11-08T08:19:53.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.310+0000 7f7ef7fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ef806ff10 0x7f7ef8143300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:53.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.310+0000 7f7ef7fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ef806ff10 0x7f7ef8143300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:44948/0 (socket says 172.21.15.40:44948) 2023-11-08T08:19:53.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.310+0000 7f7ef7fff700 1 -- 172.21.15.40:0/3862576211 learned_addr learned my addr 172.21.15.40:0/3862576211 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:53.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.310+0000 7f7ef77fe700 1 --2- 172.21.15.40:0/3862576211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ef8070360 0x7f7ef80c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:53.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.310+0000 7f7efc957700 1 --2- 172.21.15.40:0/3862576211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ef815c150 0x7f7ef80c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:53.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.310+0000 7f7ef7fff700 1 -- 172.21.15.40:0/3862576211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ef815c150 msgr2=0x7f7ef80c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:53.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.310+0000 7f7ef7fff700 1 --2- 172.21.15.40:0/3862576211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ef815c150 0x7f7ef80c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.310+0000 7f7ef7fff700 1 -- 172.21.15.40:0/3862576211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ef8070360 msgr2=0x7f7ef80c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:53.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.310+0000 7f7ef7fff700 1 --2- 172.21.15.40:0/3862576211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ef8070360 0x7f7ef80c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.310+0000 7f7ef7fff700 1 -- 172.21.15.40:0/3862576211 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee8009570 con 0x7f7ef806ff10 2023-11-08T08:19:53.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.310+0000 7f7ef7fff700 1 --2- 172.21.15.40:0/3862576211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ef806ff10 0x7f7ef8143300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8014040 tx=0x7f7ee8004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:53.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.310+0000 7f7ef4ff9700 1 -- 172.21.15.40:0/3862576211 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8004510 con 0x7f7ef806ff10 2023-11-08T08:19:53.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.310+0000 7f7ef4ff9700 1 -- 172.21.15.40:0/3862576211 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee801e070 con 0x7f7ef806ff10 2023-11-08T08:19:53.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.310+0000 7f7ef4ff9700 1 -- 172.21.15.40:0/3862576211 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8003900 con 0x7f7ef806ff10 2023-11-08T08:19:53.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.310+0000 7f7efd158700 1 -- 172.21.15.40:0/3862576211 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef80d2730 con 0x7f7ef806ff10 2023-11-08T08:19:53.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.310+0000 7f7efd158700 1 -- 172.21.15.40:0/3862576211 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef80d2450 con 0x7f7ef806ff10 2023-11-08T08:19:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.314+0000 7f7efd158700 1 -- 172.21.15.40:0/3862576211 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef80d05e0 con 0x7f7ef806ff10 2023-11-08T08:19:53.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.314+0000 7f7ef4ff9700 1 -- 172.21.15.40:0/3862576211 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7ee800f050 con 0x7f7ef806ff10 2023-11-08T08:19:53.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.314+0000 7f7ef4ff9700 1 --2- 172.21.15.40:0/3862576211 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7ee4064ee0 0x7f7ee4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:53.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.314+0000 7f7ef4ff9700 1 -- 172.21.15.40:0/3862576211 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7ee8020070 con 0x7f7ef806ff10 2023-11-08T08:19:53.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.318+0000 7f7ef77fe700 1 --2- 172.21.15.40:0/3862576211 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7ee4064ee0 0x7f7ee4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:53.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.318+0000 7f7ef77fe700 1 --2- 172.21.15.40:0/3862576211 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7ee4064ee0 0x7f7ee4067390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7ef80b6cc0 tx=0x7f7eec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:53.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.326+0000 7f7ef4ff9700 1 -- 172.21.15.40:0/3862576211 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7ee80514d0 con 0x7f7ef806ff10 2023-11-08T08:19:53.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.498+0000 7f7efd158700 1 -- 172.21.15.40:0/3862576211 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7ef8073d20 con 0x7f7ef806ff10 2023-11-08T08:19:53.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.498+0000 7f7ef4ff9700 1 -- 172.21.15.40:0/3862576211 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7ee802a020 con 0x7f7ef806ff10 2023-11-08T08:19:53.502 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:19:53.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.502+0000 7f7efd158700 1 -- 172.21.15.40:0/3862576211 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7ee4064ee0 msgr2=0x7f7ee4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:53.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.502+0000 7f7efd158700 1 --2- 172.21.15.40:0/3862576211 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7ee4064ee0 0x7f7ee4067390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7ef80b6cc0 tx=0x7f7eec009040 comp rx=0 tx=0).stop 2023-11-08T08:19:53.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.502+0000 7f7efd158700 1 -- 172.21.15.40:0/3862576211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ef806ff10 msgr2=0x7f7ef8143300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:53.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.502+0000 7f7efd158700 1 --2- 172.21.15.40:0/3862576211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ef806ff10 0x7f7ef8143300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8014040 tx=0x7f7ee8004770 comp rx=0 tx=0).stop 2023-11-08T08:19:53.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.502+0000 7f7efd158700 1 -- 172.21.15.40:0/3862576211 shutdown_connections 2023-11-08T08:19:53.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.502+0000 7f7efd158700 1 --2- 172.21.15.40:0/3862576211 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7ee4064ee0 0x7f7ee4067390 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.502+0000 7f7efd158700 1 --2- 172.21.15.40:0/3862576211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ef815c150 0x7f7ef80c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.502+0000 7f7efd158700 1 --2- 172.21.15.40:0/3862576211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ef8070360 0x7f7ef80c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.502+0000 7f7efd158700 1 --2- 172.21.15.40:0/3862576211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ef806ff10 0x7f7ef8143300 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.502+0000 7f7efd158700 1 -- 172.21.15.40:0/3862576211 >> 172.21.15.40:0/3862576211 conn(0x7f7ef80d3680 msgr2=0x7f7ef80b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:53.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.502+0000 7f7efd158700 1 -- 172.21.15.40:0/3862576211 shutdown_connections 2023-11-08T08:19:53.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.502+0000 7f7efd158700 1 -- 172.21.15.40:0/3862576211 wait complete. 2023-11-08T08:19:53.533 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:19:53.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.710+0000 7f354b59e700 1 Processor -- start 2023-11-08T08:19:53.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.714+0000 7f354b59e700 1 -- start start 2023-11-08T08:19:53.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.714+0000 7f354b59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f354c15c640 0x7f354c0d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:53.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.714+0000 7f354b59e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f354c071aa0 0x7f354c06d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:53.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.714+0000 7f354b59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f354c069e00 0x7f354c06a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:53.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.714+0000 7f354b59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f354c06cef0 con 0x7f354c15c640 2023-11-08T08:19:53.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.714+0000 7f354b59e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f354c06a7c0 con 0x7f354c071aa0 2023-11-08T08:19:53.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.714+0000 7f354b59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354c06c600 con 0x7f354c069e00 2023-11-08T08:19:53.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.714+0000 7f354a59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f354c15c640 0x7f354c0d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:53.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.714+0000 7f354a59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f354c15c640 0x7f354c0d31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34928/0 (socket says 172.21.15.40:34928) 2023-11-08T08:19:53.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.714+0000 7f354a59c700 1 -- 172.21.15.40:0/1835752640 learned_addr learned my addr 172.21.15.40:0/1835752640 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:53.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.714+0000 7f3541d9b700 1 --2- 172.21.15.40:0/1835752640 >> v2:172.21.15.169:3300/0 conn(0x7f354c071aa0 0x7f354c06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:53.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f354a59c700 1 -- 172.21.15.40:0/1835752640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f354c069e00 msgr2=0x7f354c06a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:53.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f354ad9d700 1 --2- 172.21.15.40:0/1835752640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f354c069e00 0x7f354c06a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:53.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f354a59c700 1 --2- 172.21.15.40:0/1835752640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f354c069e00 0x7f354c06a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f354a59c700 1 -- 172.21.15.40:0/1835752640 >> v2:172.21.15.169:3300/0 conn(0x7f354c071aa0 msgr2=0x7f354c06d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:53.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f354a59c700 1 --2- 172.21.15.40:0/1835752640 >> v2:172.21.15.169:3300/0 conn(0x7f354c071aa0 0x7f354c06d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f354a59c700 1 -- 172.21.15.40:0/1835752640 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c0722f0 con 0x7f354c15c640 2023-11-08T08:19:53.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f354ad9d700 1 --2- 172.21.15.40:0/1835752640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f354c069e00 0x7f354c06a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:53.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f3541d9b700 1 --2- 172.21.15.40:0/1835752640 >> v2:172.21.15.169:3300/0 conn(0x7f354c071aa0 0x7f354c06d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:53.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f354a59c700 1 --2- 172.21.15.40:0/1835752640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c15c640 0x7f354c0d31e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f354c070e70 tx=0x7f353c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78424b75afc8f5b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:53.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f3549d9b700 1 -- 172.21.15.40:0/1835752640 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c00b8a0 con 0x7f354c15c640 2023-11-08T08:19:53.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f3549d9b700 1 -- 172.21.15.40:0/1835752640 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f353c00ba00 con 0x7f354c15c640 2023-11-08T08:19:53.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f3549d9b700 1 -- 172.21.15.40:0/1835752640 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c00bd90 con 0x7f354c15c640 2023-11-08T08:19:53.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f354b59e700 1 -- 172.21.15.40:0/1835752640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c15c640 msgr2=0x7f354c0d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:53.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f354b59e700 1 --2- 172.21.15.40:0/1835752640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c15c640 0x7f354c0d31e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f354c070e70 tx=0x7f353c009bf0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f354b59e700 1 -- 172.21.15.40:0/1835752640 shutdown_connections 2023-11-08T08:19:53.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f354b59e700 1 --2- 172.21.15.40:0/1835752640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f354c069e00 0x7f354c06a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f354b59e700 1 --2- 172.21.15.40:0/1835752640 >> v2:172.21.15.169:3300/0 conn(0x7f354c071aa0 0x7f354c06d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f354b59e700 1 --2- 172.21.15.40:0/1835752640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c15c640 0x7f354c0d31e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f354b59e700 1 -- 172.21.15.40:0/1835752640 >> 172.21.15.40:0/1835752640 conn(0x7f354c0b6520 msgr2=0x7f354c071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:53.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f354b59e700 1 -- 172.21.15.40:0/1835752640 shutdown_connections 2023-11-08T08:19:53.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.718+0000 7f354b59e700 1 -- 172.21.15.40:0/1835752640 wait complete. 2023-11-08T08:19:53.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.722+0000 7f354b59e700 1 Processor -- start 2023-11-08T08:19:53.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.722+0000 7f354b59e700 1 -- start start 2023-11-08T08:19:53.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f354b59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f354c069e00 0x7f354c15f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:53.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f354b59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f354c071aa0 0x7f354c0cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:53.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f354b59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c15c640 0x7f354c15cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:53.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f354b59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c0cc630 con 0x7f354c15c640 2023-11-08T08:19:53.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f354b59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c15d0e0 con 0x7f354c069e00 2023-11-08T08:19:53.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f354b59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354c148f80 con 0x7f354c071aa0 2023-11-08T08:19:53.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f354ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c15c640 0x7f354c15cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:53.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f354ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c15c640 0x7f354c15cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34930/0 (socket says 172.21.15.40:34930) 2023-11-08T08:19:53.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f354ad9d700 1 -- 172.21.15.40:0/3138610969 learned_addr learned my addr 172.21.15.40:0/3138610969 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:53.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f354a59c700 1 --2- 172.21.15.40:0/3138610969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f354c069e00 0x7f354c15f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:53.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f3541d9b700 1 --2- 172.21.15.40:0/3138610969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f354c071aa0 0x7f354c0cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:53.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f354ad9d700 1 -- 172.21.15.40:0/3138610969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f354c071aa0 msgr2=0x7f354c0cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:53.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f354ad9d700 1 --2- 172.21.15.40:0/3138610969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f354c071aa0 0x7f354c0cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f354ad9d700 1 -- 172.21.15.40:0/3138610969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f354c069e00 msgr2=0x7f354c15f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:53.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f354ad9d700 1 --2- 172.21.15.40:0/3138610969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f354c069e00 0x7f354c15f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f354ad9d700 1 -- 172.21.15.40:0/3138610969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f353c009570 con 0x7f354c15c640 2023-11-08T08:19:53.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f354ad9d700 1 --2- 172.21.15.40:0/3138610969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c15c640 0x7f354c15cba0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f354400cd10 tx=0x7f354400a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:53.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f35437fe700 1 -- 172.21.15.40:0/3138610969 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354400ce60 con 0x7f354c15c640 2023-11-08T08:19:53.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f35437fe700 1 -- 172.21.15.40:0/3138610969 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354400f480 con 0x7f354c15c640 2023-11-08T08:19:53.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f35437fe700 1 -- 172.21.15.40:0/3138610969 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354400f810 con 0x7f354c15c640 2023-11-08T08:19:53.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f354b59e700 1 -- 172.21.15.40:0/3138610969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354c072010 con 0x7f354c15c640 2023-11-08T08:19:53.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.726+0000 7f354b59e700 1 -- 172.21.15.40:0/3138610969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354c15ca10 con 0x7f354c15c640 2023-11-08T08:19:53.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.730+0000 7f354b59e700 1 -- 172.21.15.40:0/3138610969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354c06c830 con 0x7f354c15c640 2023-11-08T08:19:53.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.730+0000 7f35437fe700 1 -- 172.21.15.40:0/3138610969 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f3544009030 con 0x7f354c15c640 2023-11-08T08:19:53.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.730+0000 7f35437fe700 1 --2- 172.21.15.40:0/3138610969 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f352c064ee0 0x7f352c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:53.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.730+0000 7f35437fe700 1 -- 172.21.15.40:0/3138610969 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3544082a10 con 0x7f354c15c640 2023-11-08T08:19:53.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.730+0000 7f354a59c700 1 --2- 172.21.15.40:0/3138610969 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f352c064ee0 0x7f352c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:53.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.734+0000 7f354a59c700 1 --2- 172.21.15.40:0/3138610969 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f352c064ee0 0x7f352c067390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f353c000c00 tx=0x7f353c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:53.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.738+0000 7f35437fe700 1 -- 172.21.15.40:0/3138610969 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f3544050880 con 0x7f354c15c640 2023-11-08T08:19:53.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.910+0000 7f354b59e700 1 -- 172.21.15.40:0/3138610969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f354c073cb0 con 0x7f354c15c640 2023-11-08T08:19:53.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.914+0000 7f35437fe700 1 -- 172.21.15.40:0/3138610969 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19748 (secure 0 0 0) 0x7f3544053ea0 con 0x7f354c15c640 2023-11-08T08:19:53.919 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:53.919 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":30,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:45.885827+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:19:53.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.918+0000 7f354b59e700 1 -- 172.21.15.40:0/3138610969 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f352c064ee0 msgr2=0x7f352c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:53.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.918+0000 7f354b59e700 1 --2- 172.21.15.40:0/3138610969 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f352c064ee0 0x7f352c067390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f353c000c00 tx=0x7f353c008030 comp rx=0 tx=0).stop 2023-11-08T08:19:53.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.918+0000 7f354b59e700 1 -- 172.21.15.40:0/3138610969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c15c640 msgr2=0x7f354c15cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:53.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.918+0000 7f354b59e700 1 --2- 172.21.15.40:0/3138610969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c15c640 0x7f354c15cba0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f354400cd10 tx=0x7f354400a560 comp rx=0 tx=0).stop 2023-11-08T08:19:53.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.918+0000 7f354b59e700 1 -- 172.21.15.40:0/3138610969 shutdown_connections 2023-11-08T08:19:53.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.918+0000 7f354b59e700 1 --2- 172.21.15.40:0/3138610969 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f352c064ee0 0x7f352c067390 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.918+0000 7f354b59e700 1 --2- 172.21.15.40:0/3138610969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f354c15c640 0x7f354c15cba0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.918+0000 7f354b59e700 1 --2- 172.21.15.40:0/3138610969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f354c071aa0 0x7f354c0cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.918+0000 7f354b59e700 1 --2- 172.21.15.40:0/3138610969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f354c069e00 0x7f354c15f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:53.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.918+0000 7f354b59e700 1 -- 172.21.15.40:0/3138610969 >> 172.21.15.40:0/3138610969 conn(0x7f354c0b6520 msgr2=0x7f354c15b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:53.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.918+0000 7f354b59e700 1 -- 172.21.15.40:0/3138610969 shutdown_connections 2023-11-08T08:19:53.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:53.918+0000 7f354b59e700 1 -- 172.21.15.40:0/3138610969 wait complete. 2023-11-08T08:19:53.945 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T08:19:02.627691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T08:19:03.473821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T08:19:06.539942+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T08:19:53.946 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:19:54.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.122+0000 7f08e60bc700 1 Processor -- start 2023-11-08T08:19:54.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.122+0000 7f08e60bc700 1 -- start start 2023-11-08T08:19:54.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.126+0000 7f08e60bc700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f08e015c360 0x7f08e00ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:54.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.126+0000 7f08e60bc700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e00cd110 con 0x7f08e006cbc0 2023-11-08T08:19:54.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.126+0000 7f08e60bc700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e00cd340 con 0x7f08e006c590 2023-11-08T08:19:54.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.126+0000 7f08e60bc700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e00c9a40 con 0x7f08e015c360 2023-11-08T08:19:54.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.126+0000 7f08e48b9700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f08e006cbc0 0x7f08e006be80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58510/0 (socket says 172.21.15.40:58510) 2023-11-08T08:19:54.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.126+0000 7f08e48b9700 1 -- 172.21.15.40:0/2816412559 learned_addr learned my addr 172.21.15.40:0/2816412559 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:54.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.126+0000 7f08e50ba700 1 --2- 172.21.15.40:0/2816412559 >> v2:172.21.15.169:3300/0 conn(0x7f08e015c360 0x7f08e00ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:54.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.126+0000 7f08d7fff700 1 -- 172.21.15.40:0/2816412559 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401087697 0 0) 0x7f08e00cd110 con 0x7f08e006cbc0 2023-11-08T08:19:54.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.126+0000 7f08d7fff700 1 -- 172.21.15.40:0/2816412559 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08cc0035d0 con 0x7f08e006cbc0 2023-11-08T08:19:54.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.126+0000 7f08d7fff700 1 -- 172.21.15.40:0/2816412559 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968516008 0 0) 0x7f08e00cd340 con 0x7f08e006c590 2023-11-08T08:19:54.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.126+0000 7f08d7fff700 1 -- 172.21.15.40:0/2816412559 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08e00cd110 con 0x7f08e006c590 2023-11-08T08:19:54.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.126+0000 7f08e50ba700 1 -- 172.21.15.40:0/2816412559 >> v1:172.21.15.169:6789/0 conn(0x7f08e006c590 legacy=0x7f08e00c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:54.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.126+0000 7f08e50ba700 1 -- 172.21.15.40:0/2816412559 >> v1:172.21.15.40:6789/0 conn(0x7f08e006cbc0 legacy=0x7f08e006be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:54.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.126+0000 7f08e50ba700 1 -- 172.21.15.40:0/2816412559 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e00cd620 con 0x7f08e015c360 2023-11-08T08:19:54.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.126+0000 7f08e50ba700 1 --2- 172.21.15.40:0/2816412559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e015c360 0x7f08e00ccbd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009850 tx=0x7f08dc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76d31f2f61fe145c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:54.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.130+0000 7f08d7fff700 1 -- 172.21.15.40:0/2816412559 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc011830 con 0x7f08e015c360 2023-11-08T08:19:54.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.130+0000 7f08d7fff700 1 -- 172.21.15.40:0/2816412559 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc011990 con 0x7f08e015c360 2023-11-08T08:19:54.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.130+0000 7f08d7fff700 1 -- 172.21.15.40:0/2816412559 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc011d40 con 0x7f08e015c360 2023-11-08T08:19:54.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.130+0000 7f08e60bc700 1 -- 172.21.15.40:0/2816412559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e015c360 msgr2=0x7f08e00ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:54.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.130+0000 7f08e60bc700 1 --2- 172.21.15.40:0/2816412559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e015c360 0x7f08e00ccbd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009850 tx=0x7f08dc009b60 comp rx=0 tx=0).stop 2023-11-08T08:19:54.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.130+0000 7f08e60bc700 1 -- 172.21.15.40:0/2816412559 shutdown_connections 2023-11-08T08:19:54.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.130+0000 7f08e60bc700 1 --2- 172.21.15.40:0/2816412559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e015c360 0x7f08e00ccbd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.130+0000 7f08e60bc700 1 -- 172.21.15.40:0/2816412559 >> 172.21.15.40:0/2816412559 conn(0x7f08e00cf280 msgr2=0x7f08e00cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:54.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.130+0000 7f08e60bc700 1 -- 172.21.15.40:0/2816412559 shutdown_connections 2023-11-08T08:19:54.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.130+0000 7f08e60bc700 1 -- 172.21.15.40:0/2816412559 wait complete. 2023-11-08T08:19:54.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08e60bc700 1 Processor -- start 2023-11-08T08:19:54.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08e60bc700 1 -- start start 2023-11-08T08:19:54.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08e60bc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e006c590 0x7f08e01f3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:54.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08e60bc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08e006cbc0 0x7f08e015af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:54.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08e60bc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e015c360 0x7f08e00afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:54.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08e60bc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e0157260 con 0x7f08e006cbc0 2023-11-08T08:19:54.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08e60bc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e01f23b0 con 0x7f08e015c360 2023-11-08T08:19:54.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08e60bc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e01f2690 con 0x7f08e006c590 2023-11-08T08:19:54.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08e48b9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08e006cbc0 0x7f08e015af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:54.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08e48b9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08e006cbc0 0x7f08e015af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34944/0 (socket says 172.21.15.40:34944) 2023-11-08T08:19:54.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08e48b9700 1 -- 172.21.15.40:0/3315072862 learned_addr learned my addr 172.21.15.40:0/3315072862 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:54.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08e58bb700 1 --2- 172.21.15.40:0/3315072862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e015c360 0x7f08e00afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:54.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08e50ba700 1 --2- 172.21.15.40:0/3315072862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e006c590 0x7f08e01f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:54.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08e48b9700 1 -- 172.21.15.40:0/3315072862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e006c590 msgr2=0x7f08e01f3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:54.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08e48b9700 1 --2- 172.21.15.40:0/3315072862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e006c590 0x7f08e01f3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08e48b9700 1 -- 172.21.15.40:0/3315072862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e015c360 msgr2=0x7f08e00afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:54.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08e48b9700 1 --2- 172.21.15.40:0/3315072862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e015c360 0x7f08e00afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08e48b9700 1 -- 172.21.15.40:0/3315072862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc0094b0 con 0x7f08e006cbc0 2023-11-08T08:19:54.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08e48b9700 1 --2- 172.21.15.40:0/3315072862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08e006cbc0 0x7f08e015af30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f08d000c4d0 tx=0x7f08d000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:54.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08d5ffb700 1 -- 172.21.15.40:0/3315072862 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0005d90 con 0x7f08e006cbc0 2023-11-08T08:19:54.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08d5ffb700 1 -- 172.21.15.40:0/3315072862 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d00070d0 con 0x7f08e006cbc0 2023-11-08T08:19:54.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08e60bc700 1 -- 172.21.15.40:0/3315072862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e015c730 con 0x7f08e006cbc0 2023-11-08T08:19:54.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.134+0000 7f08d5ffb700 1 -- 172.21.15.40:0/3315072862 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0007460 con 0x7f08e006cbc0 2023-11-08T08:19:54.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.138+0000 7f08e60bc700 1 -- 172.21.15.40:0/3315072862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e00c9660 con 0x7f08e006cbc0 2023-11-08T08:19:54.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.138+0000 7f08e60bc700 1 -- 172.21.15.40:0/3315072862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e00c9350 con 0x7f08e006cbc0 2023-11-08T08:19:54.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.138+0000 7f08d5ffb700 1 -- 172.21.15.40:0/3315072862 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f08d0019030 con 0x7f08e006cbc0 2023-11-08T08:19:54.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.142+0000 7f08d5ffb700 1 --2- 172.21.15.40:0/3315072862 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f08c4064ee0 0x7f08c4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:54.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.142+0000 7f08d5ffb700 1 -- 172.21.15.40:0/3315072862 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f08d00815b0 con 0x7f08e006cbc0 2023-11-08T08:19:54.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.142+0000 7f08e50ba700 1 --2- 172.21.15.40:0/3315072862 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f08c4064ee0 0x7f08c4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:54.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.142+0000 7f08e50ba700 1 --2- 172.21.15.40:0/3315072862 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f08c4064ee0 0x7f08c4067390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0090b0 tx=0x7f08dc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:54.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.146+0000 7f08d5ffb700 1 -- 172.21.15.40:0/3315072862 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f08d004f420 con 0x7f08e006cbc0 2023-11-08T08:19:54.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.322+0000 7f08e60bc700 1 -- 172.21.15.40:0/3315072862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f08e0073d20 con 0x7f08e006cbc0 2023-11-08T08:19:54.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.322+0000 7f08d5ffb700 1 -- 172.21.15.40:0/3315072862 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f08d0052a40 con 0x7f08e006cbc0 2023-11-08T08:19:54.327 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:19:54.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.326+0000 7f08e60bc700 1 -- 172.21.15.40:0/3315072862 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f08c4064ee0 msgr2=0x7f08c4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:54.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.326+0000 7f08e60bc700 1 --2- 172.21.15.40:0/3315072862 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f08c4064ee0 0x7f08c4067390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0090b0 tx=0x7f08dc008030 comp rx=0 tx=0).stop 2023-11-08T08:19:54.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.326+0000 7f08e60bc700 1 -- 172.21.15.40:0/3315072862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08e006cbc0 msgr2=0x7f08e015af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:54.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.326+0000 7f08e60bc700 1 --2- 172.21.15.40:0/3315072862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08e006cbc0 0x7f08e015af30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f08d000c4d0 tx=0x7f08d000c890 comp rx=0 tx=0).stop 2023-11-08T08:19:54.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.326+0000 7f08e60bc700 1 -- 172.21.15.40:0/3315072862 shutdown_connections 2023-11-08T08:19:54.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.326+0000 7f08e60bc700 1 --2- 172.21.15.40:0/3315072862 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f08c4064ee0 0x7f08c4067390 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.326+0000 7f08e60bc700 1 --2- 172.21.15.40:0/3315072862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e015c360 0x7f08e00afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.326+0000 7f08e60bc700 1 --2- 172.21.15.40:0/3315072862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08e006cbc0 0x7f08e015af30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.326+0000 7f08e60bc700 1 --2- 172.21.15.40:0/3315072862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e006c590 0x7f08e01f3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.326+0000 7f08e60bc700 1 -- 172.21.15.40:0/3315072862 >> 172.21.15.40:0/3315072862 conn(0x7f08e00cf280 msgr2=0x7f08e01525b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:54.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.330+0000 7f08e60bc700 1 -- 172.21.15.40:0/3315072862 shutdown_connections 2023-11-08T08:19:54.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.330+0000 7f08e60bc700 1 -- 172.21.15.40:0/3315072862 wait complete. 2023-11-08T08:19:54.354 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:19:54.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.542+0000 7f1d9dcd5700 1 Processor -- start 2023-11-08T08:19:54.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.542+0000 7f1d9dcd5700 1 -- start start 2023-11-08T08:19:54.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.546+0000 7f1d9dcd5700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1d9815c310 0x7f1d980d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:54.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.546+0000 7f1d9dcd5700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d980cd080 con 0x7f1d980707d0 2023-11-08T08:19:54.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.546+0000 7f1d9dcd5700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d980d21a0 con 0x7f1d9806b830 2023-11-08T08:19:54.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.546+0000 7f1d9dcd5700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d9806d240 con 0x7f1d9815c310 2023-11-08T08:19:54.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.546+0000 7f1d9d4d4700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1d980707d0 0x7f1d98069bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58516/0 (socket says 172.21.15.40:58516) 2023-11-08T08:19:54.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.546+0000 7f1d9d4d4700 1 -- 172.21.15.40:0/2896156561 learned_addr learned my addr 172.21.15.40:0/2896156561 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:54.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.546+0000 7f1d9ccd3700 1 --2- 172.21.15.40:0/2896156561 >> v2:172.21.15.169:3300/0 conn(0x7f1d9815c310 0x7f1d980d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:54.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.546+0000 7f1d977fe700 1 -- 172.21.15.40:0/2896156561 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739751093 0 0) 0x7f1d980cd080 con 0x7f1d980707d0 2023-11-08T08:19:54.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.546+0000 7f1d977fe700 1 -- 172.21.15.40:0/2896156561 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d880035d0 con 0x7f1d980707d0 2023-11-08T08:19:54.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.546+0000 7f1d977fe700 1 -- 172.21.15.40:0/2896156561 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2504481880 0 0) 0x7f1d980d21a0 con 0x7f1d9806b830 2023-11-08T08:19:54.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.546+0000 7f1d977fe700 1 -- 172.21.15.40:0/2896156561 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d980cd080 con 0x7f1d9806b830 2023-11-08T08:19:54.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.546+0000 7f1d9ccd3700 1 -- 172.21.15.40:0/2896156561 >> v1:172.21.15.169:6789/0 conn(0x7f1d9806b830 legacy=0x7f1d980700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:54.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.546+0000 7f1d9ccd3700 1 -- 172.21.15.40:0/2896156561 >> v1:172.21.15.40:6789/0 conn(0x7f1d980707d0 legacy=0x7f1d98069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:54.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.546+0000 7f1d977fe700 1 -- 172.21.15.40:0/2896156561 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4174015894 0 0) 0x7f1d880035d0 con 0x7f1d980707d0 2023-11-08T08:19:54.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.546+0000 7f1d9ccd3700 1 -- 172.21.15.40:0/2896156561 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d980d1900 con 0x7f1d9815c310 2023-11-08T08:19:54.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.546+0000 7f1d9ccd3700 1 --2- 172.21.15.40:0/2896156561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d9815c310 0x7f1d980d1310 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009850 tx=0x7f1d8c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f49d2a1121f8a8f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:54.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.550+0000 7f1d977fe700 1 -- 172.21.15.40:0/2896156561 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c011830 con 0x7f1d9815c310 2023-11-08T08:19:54.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.550+0000 7f1d977fe700 1 -- 172.21.15.40:0/2896156561 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c011990 con 0x7f1d9815c310 2023-11-08T08:19:54.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.550+0000 7f1d977fe700 1 -- 172.21.15.40:0/2896156561 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c011d40 con 0x7f1d9815c310 2023-11-08T08:19:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.550+0000 7f1d9dcd5700 1 -- 172.21.15.40:0/2896156561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d9815c310 msgr2=0x7f1d980d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.550+0000 7f1d9dcd5700 1 --2- 172.21.15.40:0/2896156561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d9815c310 0x7f1d980d1310 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009850 tx=0x7f1d8c009b60 comp rx=0 tx=0).stop 2023-11-08T08:19:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.550+0000 7f1d9dcd5700 1 -- 172.21.15.40:0/2896156561 shutdown_connections 2023-11-08T08:19:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.550+0000 7f1d9dcd5700 1 --2- 172.21.15.40:0/2896156561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d9815c310 0x7f1d980d1310 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.550+0000 7f1d9dcd5700 1 -- 172.21.15.40:0/2896156561 >> 172.21.15.40:0/2896156561 conn(0x7f1d980d0140 msgr2=0x7f1d980d2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:54.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.550+0000 7f1d9dcd5700 1 -- 172.21.15.40:0/2896156561 shutdown_connections 2023-11-08T08:19:54.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.550+0000 7f1d9dcd5700 1 -- 172.21.15.40:0/2896156561 wait complete. 2023-11-08T08:19:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.554+0000 7f1d9dcd5700 1 Processor -- start 2023-11-08T08:19:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.554+0000 7f1d9dcd5700 1 -- start start 2023-11-08T08:19:54.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.554+0000 7f1d9dcd5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d9806b830 0x7f1d9806a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:54.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.554+0000 7f1d9dcd5700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d980707d0 0x7f1d9814cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:54.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.554+0000 7f1d9dcd5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d9815c310 0x7f1d9814d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:54.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.554+0000 7f1d9dcd5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9815d270 con 0x7f1d9806b830 2023-11-08T08:19:54.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.554+0000 7f1d9dcd5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9814d550 con 0x7f1d9815c310 2023-11-08T08:19:54.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.554+0000 7f1d9dcd5700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d980c7400 con 0x7f1d980707d0 2023-11-08T08:19:54.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.554+0000 7f1d9ccd3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d9806b830 0x7f1d9806a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:54.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.554+0000 7f1d9ccd3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d9806b830 0x7f1d9806a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34958/0 (socket says 172.21.15.40:34958) 2023-11-08T08:19:54.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.554+0000 7f1d9ccd3700 1 -- 172.21.15.40:0/1815936188 learned_addr learned my addr 172.21.15.40:0/1815936188 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:54.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.554+0000 7f1d97fff700 1 --2- 172.21.15.40:0/1815936188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d980707d0 0x7f1d9814cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:54.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.554+0000 7f1d9d4d4700 1 --2- 172.21.15.40:0/1815936188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d9815c310 0x7f1d9814d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:54.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.558+0000 7f1d9ccd3700 1 -- 172.21.15.40:0/1815936188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d980707d0 msgr2=0x7f1d9814cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:54.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.558+0000 7f1d9ccd3700 1 --2- 172.21.15.40:0/1815936188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d980707d0 0x7f1d9814cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.558+0000 7f1d9ccd3700 1 -- 172.21.15.40:0/1815936188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d9815c310 msgr2=0x7f1d9814d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:54.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.558+0000 7f1d9ccd3700 1 --2- 172.21.15.40:0/1815936188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d9815c310 0x7f1d9814d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.558+0000 7f1d9ccd3700 1 -- 172.21.15.40:0/1815936188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c0094b0 con 0x7f1d9806b830 2023-11-08T08:19:54.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.558+0000 7f1d9ccd3700 1 --2- 172.21.15.40:0/1815936188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d9806b830 0x7f1d9806a140 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1d98071980 tx=0x7f1d8c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:54.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.558+0000 7f1d957fa700 1 -- 172.21.15.40:0/1815936188 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c01a990 con 0x7f1d9806b830 2023-11-08T08:19:54.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.558+0000 7f1d957fa700 1 -- 172.21.15.40:0/1815936188 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c01f070 con 0x7f1d9806b830 2023-11-08T08:19:54.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.558+0000 7f1d957fa700 1 -- 172.21.15.40:0/1815936188 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c01ad30 con 0x7f1d9806b830 2023-11-08T08:19:54.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.558+0000 7f1d9dcd5700 1 -- 172.21.15.40:0/1815936188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9806d450 con 0x7f1d9806b830 2023-11-08T08:19:54.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.558+0000 7f1d9dcd5700 1 -- 172.21.15.40:0/1815936188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9815c6e0 con 0x7f1d9806b830 2023-11-08T08:19:54.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.558+0000 7f1d9dcd5700 1 -- 172.21.15.40:0/1815936188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9806d730 con 0x7f1d9806b830 2023-11-08T08:19:54.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.562+0000 7f1d957fa700 1 -- 172.21.15.40:0/1815936188 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1d8c00f050 con 0x7f1d9806b830 2023-11-08T08:19:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.562+0000 7f1d957fa700 1 --2- 172.21.15.40:0/1815936188 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d7c064ee0 0x7f1d7c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.562+0000 7f1d957fa700 1 -- 172.21.15.40:0/1815936188 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1d8c083250 con 0x7f1d9806b830 2023-11-08T08:19:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.562+0000 7f1d97fff700 1 --2- 172.21.15.40:0/1815936188 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d7c064ee0 0x7f1d7c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:54.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.566+0000 7f1d97fff700 1 --2- 172.21.15.40:0/1815936188 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d7c064ee0 0x7f1d7c067390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1d980ccfd0 tx=0x7f1d84009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:54.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.570+0000 7f1d957fa700 1 -- 172.21.15.40:0/1815936188 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1d8c050fd0 con 0x7f1d9806b830 2023-11-08T08:19:54.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.742+0000 7f1d9dcd5700 1 -- 172.21.15.40:0/1815936188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1d9815d3b0 con 0x7f1d9806b830 2023-11-08T08:19:54.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.742+0000 7f1d957fa700 1 -- 172.21.15.40:0/1815936188 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1d8c0210a0 con 0x7f1d9806b830 2023-11-08T08:19:54.747 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:19:54.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.746+0000 7f1d9dcd5700 1 -- 172.21.15.40:0/1815936188 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d7c064ee0 msgr2=0x7f1d7c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:54.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.746+0000 7f1d9dcd5700 1 --2- 172.21.15.40:0/1815936188 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d7c064ee0 0x7f1d7c067390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1d980ccfd0 tx=0x7f1d84009040 comp rx=0 tx=0).stop 2023-11-08T08:19:54.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.746+0000 7f1d9dcd5700 1 -- 172.21.15.40:0/1815936188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d9806b830 msgr2=0x7f1d9806a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:54.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.746+0000 7f1d9dcd5700 1 --2- 172.21.15.40:0/1815936188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d9806b830 0x7f1d9806a140 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1d98071980 tx=0x7f1d8c004770 comp rx=0 tx=0).stop 2023-11-08T08:19:54.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.750+0000 7f1d9dcd5700 1 -- 172.21.15.40:0/1815936188 shutdown_connections 2023-11-08T08:19:54.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.750+0000 7f1d9dcd5700 1 --2- 172.21.15.40:0/1815936188 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d7c064ee0 0x7f1d7c067390 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.750+0000 7f1d9dcd5700 1 --2- 172.21.15.40:0/1815936188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d9815c310 0x7f1d9814d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.750+0000 7f1d9dcd5700 1 --2- 172.21.15.40:0/1815936188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d980707d0 0x7f1d9814cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.750+0000 7f1d9dcd5700 1 --2- 172.21.15.40:0/1815936188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d9806b830 0x7f1d9806a140 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.750+0000 7f1d9dcd5700 1 -- 172.21.15.40:0/1815936188 >> 172.21.15.40:0/1815936188 conn(0x7f1d980d0140 msgr2=0x7f1d98157030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:54.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.750+0000 7f1d9dcd5700 1 -- 172.21.15.40:0/1815936188 shutdown_connections 2023-11-08T08:19:54.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.750+0000 7f1d9dcd5700 1 -- 172.21.15.40:0/1815936188 wait complete. 2023-11-08T08:19:54.775 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T08:19:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.966+0000 7f6f7d8c4700 1 Processor -- start 2023-11-08T08:19:54.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.966+0000 7f6f7d8c4700 1 -- start start 2023-11-08T08:19:54.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.966+0000 7f6f7d8c4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6f7815e190 0x7f6f780cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:54.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.966+0000 7f6f7d8c4700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f78161560 0x7f6f78069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:54.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.966+0000 7f6f7d8c4700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6f7806ff10 0x7f6f78070390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:54.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.966+0000 7f6f7d8c4700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f780708d0 con 0x7f6f7815e190 2023-11-08T08:19:54.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7d8c4700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f7815e560 con 0x7f6f7806ff10 2023-11-08T08:19:54.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7d8c4700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7806be80 con 0x7f6f78161560 2023-11-08T08:19:54.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7c8c2700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6f7815e190 0x7f6f780cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:54.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7c8c2700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6f7815e190 0x7f6f780cdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34974/0 (socket says 172.21.15.40:34974) 2023-11-08T08:19:54.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7c8c2700 1 -- 172.21.15.40:0/1303309837 learned_addr learned my addr 172.21.15.40:0/1303309837 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:54.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f77fff700 1 --2- 172.21.15.40:0/1303309837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f78161560 0x7f6f78069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:54.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7d0c3700 1 --2- 172.21.15.40:0/1303309837 >> v2:172.21.15.169:3300/0 conn(0x7f6f7806ff10 0x7f6f78070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:54.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7c8c2700 1 -- 172.21.15.40:0/1303309837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f78161560 msgr2=0x7f6f78069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:54.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7c8c2700 1 --2- 172.21.15.40:0/1303309837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f78161560 0x7f6f78069bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7c8c2700 1 -- 172.21.15.40:0/1303309837 >> v2:172.21.15.169:3300/0 conn(0x7f6f7806ff10 msgr2=0x7f6f78070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:54.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7c8c2700 1 --2- 172.21.15.40:0/1303309837 >> v2:172.21.15.169:3300/0 conn(0x7f6f7806ff10 0x7f6f78070390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7c8c2700 1 -- 172.21.15.40:0/1303309837 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f780ce330 con 0x7f6f7815e190 2023-11-08T08:19:54.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f77fff700 1 --2- 172.21.15.40:0/1303309837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f78161560 0x7f6f78069bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:54.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7c8c2700 1 --2- 172.21.15.40:0/1303309837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f7815e190 0x7f6f780cdc50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6f780b4800 tx=0x7f6f6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9100fcfaebbff1d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:54.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f777fe700 1 -- 172.21.15.40:0/1303309837 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c00e070 con 0x7f6f7815e190 2023-11-08T08:19:54.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f777fe700 1 -- 172.21.15.40:0/1303309837 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c00b9c0 con 0x7f6f7815e190 2023-11-08T08:19:54.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f777fe700 1 -- 172.21.15.40:0/1303309837 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c00bd70 con 0x7f6f7815e190 2023-11-08T08:19:54.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7d8c4700 1 -- 172.21.15.40:0/1303309837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f7815e190 msgr2=0x7f6f780cdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:54.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7d8c4700 1 --2- 172.21.15.40:0/1303309837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f7815e190 0x7f6f780cdc50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6f780b4800 tx=0x7f6f6c00b330 comp rx=0 tx=0).stop 2023-11-08T08:19:54.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7d8c4700 1 -- 172.21.15.40:0/1303309837 shutdown_connections 2023-11-08T08:19:54.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7d8c4700 1 --2- 172.21.15.40:0/1303309837 >> v2:172.21.15.169:3300/0 conn(0x7f6f7806ff10 0x7f6f78070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7d8c4700 1 --2- 172.21.15.40:0/1303309837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f78161560 0x7f6f78069bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7d8c4700 1 --2- 172.21.15.40:0/1303309837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f7815e190 0x7f6f780cdc50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7d8c4700 1 -- 172.21.15.40:0/1303309837 >> 172.21.15.40:0/1303309837 conn(0x7f6f780cf280 msgr2=0x7f6f780cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:54.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7d8c4700 1 -- 172.21.15.40:0/1303309837 shutdown_connections 2023-11-08T08:19:54.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.970+0000 7f6f7d8c4700 1 -- 172.21.15.40:0/1303309837 wait complete. 2023-11-08T08:19:54.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.974+0000 7f6f7d8c4700 1 Processor -- start 2023-11-08T08:19:54.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.974+0000 7f6f7d8c4700 1 -- start start 2023-11-08T08:19:54.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.974+0000 7f6f7d8c4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f7806ff10 0x7f6f780c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:54.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.974+0000 7f6f7d8c4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f7815e190 0x7f6f780c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:54.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.974+0000 7f6f7d8c4700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f78161560 0x7f6f780c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:54.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.974+0000 7f6f7d8c4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f780c5b40 con 0x7f6f7806ff10 2023-11-08T08:19:54.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.974+0000 7f6f7d8c4700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f780c7920 con 0x7f6f7815e190 2023-11-08T08:19:54.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.974+0000 7f6f7d8c4700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f780c7c00 con 0x7f6f78161560 2023-11-08T08:19:54.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.974+0000 7f6f7c8c2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f7806ff10 0x7f6f780c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:54.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.974+0000 7f6f7c8c2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f7806ff10 0x7f6f780c70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34988/0 (socket says 172.21.15.40:34988) 2023-11-08T08:19:54.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.974+0000 7f6f7c8c2700 1 -- 172.21.15.40:0/1659380601 learned_addr learned my addr 172.21.15.40:0/1659380601 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:54.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.974+0000 7f6f7d0c3700 1 --2- 172.21.15.40:0/1659380601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f78161560 0x7f6f780c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:54.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.974+0000 7f6f77fff700 1 --2- 172.21.15.40:0/1659380601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f7815e190 0x7f6f780c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:54.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.974+0000 7f6f7c8c2700 1 -- 172.21.15.40:0/1659380601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f78161560 msgr2=0x7f6f780c5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:54.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.974+0000 7f6f7c8c2700 1 --2- 172.21.15.40:0/1659380601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f78161560 0x7f6f780c5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.974+0000 7f6f7c8c2700 1 -- 172.21.15.40:0/1659380601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f7815e190 msgr2=0x7f6f780c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:54.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.974+0000 7f6f7c8c2700 1 --2- 172.21.15.40:0/1659380601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f7815e190 0x7f6f780c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:54.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.974+0000 7f6f7c8c2700 1 -- 172.21.15.40:0/1659380601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6c009b30 con 0x7f6f7806ff10 2023-11-08T08:19:54.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.978+0000 7f6f7c8c2700 1 --2- 172.21.15.40:0/1659380601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f7806ff10 0x7f6f780c70f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00b370 tx=0x7f6f6c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:54.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.978+0000 7f6f757fa700 1 -- 172.21.15.40:0/1659380601 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c0099b0 con 0x7f6f7806ff10 2023-11-08T08:19:54.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.978+0000 7f6f757fa700 1 -- 172.21.15.40:0/1659380601 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c004250 con 0x7f6f7806ff10 2023-11-08T08:19:54.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.978+0000 7f6f757fa700 1 -- 172.21.15.40:0/1659380601 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c0045e0 con 0x7f6f7806ff10 2023-11-08T08:19:54.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.978+0000 7f6f7d8c4700 1 -- 172.21.15.40:0/1659380601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f780afc70 con 0x7f6f7806ff10 2023-11-08T08:19:54.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.978+0000 7f6f7d8c4700 1 -- 172.21.15.40:0/1659380601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f780af990 con 0x7f6f7806ff10 2023-11-08T08:19:54.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.978+0000 7f6f7d8c4700 1 -- 172.21.15.40:0/1659380601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f780aff50 con 0x7f6f7806ff10 2023-11-08T08:19:54.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.982+0000 7f6f757fa700 1 -- 172.21.15.40:0/1659380601 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f6f6c004740 con 0x7f6f7806ff10 2023-11-08T08:19:54.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.982+0000 7f6f757fa700 1 --2- 172.21.15.40:0/1659380601 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6f5c064ee0 0x7f6f5c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:54.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.982+0000 7f6f757fa700 1 -- 172.21.15.40:0/1659380601 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6f6c083900 con 0x7f6f7806ff10 2023-11-08T08:19:54.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.982+0000 7f6f77fff700 1 --2- 172.21.15.40:0/1659380601 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6f5c064ee0 0x7f6f5c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:54.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.982+0000 7f6f77fff700 1 --2- 172.21.15.40:0/1659380601 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6f5c064ee0 0x7f6f5c067390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6f7815e7b0 tx=0x7f6f64008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:54.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:54.986+0000 7f6f757fa700 1 -- 172.21.15.40:0/1659380601 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f6f6c051680 con 0x7f6f7806ff10 2023-11-08T08:19:55.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.162+0000 7f6f7d8c4700 1 -- 172.21.15.40:0/1659380601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6f780c5c80 con 0x7f6f7806ff10 2023-11-08T08:19:55.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.162+0000 7f6f757fa700 1 -- 172.21.15.40:0/1659380601 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6f6c020020 con 0x7f6f7806ff10 2023-11-08T08:19:55.166 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:19:55.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.166+0000 7f6f7d8c4700 1 -- 172.21.15.40:0/1659380601 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6f5c064ee0 msgr2=0x7f6f5c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:55.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.166+0000 7f6f7d8c4700 1 --2- 172.21.15.40:0/1659380601 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6f5c064ee0 0x7f6f5c067390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6f7815e7b0 tx=0x7f6f64008040 comp rx=0 tx=0).stop 2023-11-08T08:19:55.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.166+0000 7f6f7d8c4700 1 -- 172.21.15.40:0/1659380601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f7806ff10 msgr2=0x7f6f780c70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:55.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.166+0000 7f6f7d8c4700 1 --2- 172.21.15.40:0/1659380601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f7806ff10 0x7f6f780c70f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00b370 tx=0x7f6f6c004af0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.166+0000 7f6f7d8c4700 1 -- 172.21.15.40:0/1659380601 shutdown_connections 2023-11-08T08:19:55.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.166+0000 7f6f7d8c4700 1 --2- 172.21.15.40:0/1659380601 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6f5c064ee0 0x7f6f5c067390 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.166+0000 7f6f7d8c4700 1 --2- 172.21.15.40:0/1659380601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f78161560 0x7f6f780c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.166+0000 7f6f7d8c4700 1 --2- 172.21.15.40:0/1659380601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f7815e190 0x7f6f780c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.166+0000 7f6f7d8c4700 1 --2- 172.21.15.40:0/1659380601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f7806ff10 0x7f6f780c70f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.166+0000 7f6f7d8c4700 1 -- 172.21.15.40:0/1659380601 >> 172.21.15.40:0/1659380601 conn(0x7f6f780cf280 msgr2=0x7f6f78070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:55.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.166+0000 7f6f7d8c4700 1 -- 172.21.15.40:0/1659380601 shutdown_connections 2023-11-08T08:19:55.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.166+0000 7f6f7d8c4700 1 -- 172.21.15.40:0/1659380601 wait complete. 2023-11-08T08:19:55.191 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T08:19:55.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.374+0000 7f9e5c7ba700 1 Processor -- start 2023-11-08T08:19:55.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.374+0000 7f9e5c7ba700 1 -- start start 2023-11-08T08:19:55.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.378+0000 7f9e5c7ba700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e5415c3c0 0x7f9e540cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:55.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.378+0000 7f9e5c7ba700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9e540cc820 0x7f9e540ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:55.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.378+0000 7f9e5c7ba700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e540cdfe0 con 0x7f9e541609d0 2023-11-08T08:19:55.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.378+0000 7f9e5c7ba700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e540cdd70 con 0x7f9e540cc820 2023-11-08T08:19:55.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.378+0000 7f9e5c7ba700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e540cd1e0 con 0x7f9e5415c3c0 2023-11-08T08:19:55.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.378+0000 7f9e5ad57700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9e540cc820 0x7f9e540ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:55.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e5a556700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e5415c3c0 0x7f9e540cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:55.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e5a556700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e5415c3c0 0x7f9e540cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:50414/0 (socket says 172.21.15.40:50414) 2023-11-08T08:19:55.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e5a556700 1 -- 172.21.15.40:0/2845259538 learned_addr learned my addr 172.21.15.40:0/2845259538 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:55.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e59d55700 1 --1- 172.21.15.40:0/2845259538 >> v1:172.21.15.40:6789/0 conn(0x7f9e541609d0 0x7f9e54069dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58524/0 (socket says 172.21.15.40:58524) 2023-11-08T08:19:55.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e5ad57700 1 -- 172.21.15.40:0/2845259538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e5415c3c0 msgr2=0x7f9e540cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:55.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e5ad57700 1 --2- 172.21.15.40:0/2845259538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e5415c3c0 0x7f9e540cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e5ad57700 1 -- 172.21.15.40:0/2845259538 >> v1:172.21.15.40:6789/0 conn(0x7f9e541609d0 legacy=0x7f9e54069dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:55.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e5ad57700 1 -- 172.21.15.40:0/2845259538 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e540ca720 con 0x7f9e540cc820 2023-11-08T08:19:55.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e59554700 1 -- 172.21.15.40:0/2845259538 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2242109036 0 0) 0x7f9e540cdfe0 con 0x7f9e541609d0 2023-11-08T08:19:55.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e5a556700 1 --2- 172.21.15.40:0/2845259538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e5415c3c0 0x7f9e540cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:55.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e5ad57700 1 --2- 172.21.15.40:0/2845259538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e540cc820 0x7f9e540ccca0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009d90 tx=0x7f9e4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee140545107f9b4a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:55.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e59554700 1 -- 172.21.15.40:0/2845259538 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c00e070 con 0x7f9e540cc820 2023-11-08T08:19:55.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e59554700 1 -- 172.21.15.40:0/2845259538 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4c00bb20 con 0x7f9e540cc820 2023-11-08T08:19:55.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e59554700 1 -- 172.21.15.40:0/2845259538 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c012430 con 0x7f9e540cc820 2023-11-08T08:19:55.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e5c7ba700 1 -- 172.21.15.40:0/2845259538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e540cc820 msgr2=0x7f9e540ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:55.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e5c7ba700 1 --2- 172.21.15.40:0/2845259538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e540cc820 0x7f9e540ccca0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009d90 tx=0x7f9e4c00b330 comp rx=0 tx=0).stop 2023-11-08T08:19:55.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e5c7ba700 1 -- 172.21.15.40:0/2845259538 shutdown_connections 2023-11-08T08:19:55.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e5c7ba700 1 --2- 172.21.15.40:0/2845259538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e540cc820 0x7f9e540ccca0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e5c7ba700 1 --2- 172.21.15.40:0/2845259538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e5415c3c0 0x7f9e540cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e5c7ba700 1 -- 172.21.15.40:0/2845259538 >> 172.21.15.40:0/2845259538 conn(0x7f9e5406e2e0 msgr2=0x7f9e5406e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:55.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e5c7ba700 1 -- 172.21.15.40:0/2845259538 shutdown_connections 2023-11-08T08:19:55.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.382+0000 7f9e5c7ba700 1 -- 172.21.15.40:0/2845259538 wait complete. 2023-11-08T08:19:55.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.386+0000 7f9e5c7ba700 1 Processor -- start 2023-11-08T08:19:55.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.386+0000 7f9e5c7ba700 1 -- start start 2023-11-08T08:19:55.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.386+0000 7f9e5c7ba700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e540cc820 0x7f9e541f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:55.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.386+0000 7f9e5c7ba700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e5415c3c0 0x7f9e541f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:55.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.386+0000 7f9e5c7ba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e541609d0 0x7f9e540c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:55.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.386+0000 7f9e5c7ba700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e540b11a0 con 0x7f9e541609d0 2023-11-08T08:19:55.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.386+0000 7f9e5c7ba700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e541f2850 con 0x7f9e540cc820 2023-11-08T08:19:55.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.386+0000 7f9e5c7ba700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e541f4c00 con 0x7f9e5415c3c0 2023-11-08T08:19:55.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.386+0000 7f9e5ad57700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e541609d0 0x7f9e540c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:55.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.386+0000 7f9e5a556700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e540cc820 0x7f9e541f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:55.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.386+0000 7f9e5ad57700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e541609d0 0x7f9e540c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35012/0 (socket says 172.21.15.40:35012) 2023-11-08T08:19:55.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.386+0000 7f9e5ad57700 1 -- 172.21.15.40:0/2921666281 learned_addr learned my addr 172.21.15.40:0/2921666281 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:55.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.386+0000 7f9e59d55700 1 --2- 172.21.15.40:0/2921666281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e5415c3c0 0x7f9e541f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:55.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.390+0000 7f9e5ad57700 1 -- 172.21.15.40:0/2921666281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e5415c3c0 msgr2=0x7f9e541f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:55.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.390+0000 7f9e5ad57700 1 --2- 172.21.15.40:0/2921666281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e5415c3c0 0x7f9e541f46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.390+0000 7f9e5ad57700 1 -- 172.21.15.40:0/2921666281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e540cc820 msgr2=0x7f9e541f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:55.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.390+0000 7f9e5ad57700 1 --2- 172.21.15.40:0/2921666281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e540cc820 0x7f9e541f2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.390+0000 7f9e5ad57700 1 -- 172.21.15.40:0/2921666281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e50009570 con 0x7f9e541609d0 2023-11-08T08:19:55.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.390+0000 7f9e5ad57700 1 --2- 172.21.15.40:0/2921666281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e541609d0 0x7f9e540c9090 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9e540713d0 tx=0x7f9e4c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:55.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.390+0000 7f9e4affd700 1 -- 172.21.15.40:0/2921666281 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c012430 con 0x7f9e541609d0 2023-11-08T08:19:55.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.390+0000 7f9e4affd700 1 -- 172.21.15.40:0/2921666281 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4c020070 con 0x7f9e541609d0 2023-11-08T08:19:55.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.390+0000 7f9e4affd700 1 -- 172.21.15.40:0/2921666281 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c004280 con 0x7f9e541609d0 2023-11-08T08:19:55.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.390+0000 7f9e5c7ba700 1 -- 172.21.15.40:0/2921666281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4c009a70 con 0x7f9e541609d0 2023-11-08T08:19:55.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.390+0000 7f9e5c7ba700 1 -- 172.21.15.40:0/2921666281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e540cae40 con 0x7f9e541609d0 2023-11-08T08:19:55.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.390+0000 7f9e5c7ba700 1 -- 172.21.15.40:0/2921666281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e540cab60 con 0x7f9e541609d0 2023-11-08T08:19:55.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.394+0000 7f9e4affd700 1 -- 172.21.15.40:0/2921666281 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f9e4c02e070 con 0x7f9e541609d0 2023-11-08T08:19:55.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.394+0000 7f9e4affd700 1 --2- 172.21.15.40:0/2921666281 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9e38064ee0 0x7f9e38067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:55.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.394+0000 7f9e4affd700 1 -- 172.21.15.40:0/2921666281 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9e4c022070 con 0x7f9e541609d0 2023-11-08T08:19:55.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.394+0000 7f9e5a556700 1 --2- 172.21.15.40:0/2921666281 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9e38064ee0 0x7f9e38067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:55.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.398+0000 7f9e5a556700 1 --2- 172.21.15.40:0/2921666281 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9e38064ee0 0x7f9e38067390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9e5406ed40 tx=0x7f9e500092e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:55.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.402+0000 7f9e4affd700 1 -- 172.21.15.40:0/2921666281 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f9e4c051c40 con 0x7f9e541609d0 2023-11-08T08:19:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.574+0000 7f9e5c7ba700 1 -- 172.21.15.40:0/2921666281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9e5415c790 con 0x7f9e541609d0 2023-11-08T08:19:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.578+0000 7f9e4affd700 1 -- 172.21.15.40:0/2921666281 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9e4c02c020 con 0x7f9e541609d0 2023-11-08T08:19:55.581 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-11-08T08:19:55.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.578+0000 7f9e5c7ba700 1 -- 172.21.15.40:0/2921666281 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9e38064ee0 msgr2=0x7f9e38067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:55.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.578+0000 7f9e5c7ba700 1 --2- 172.21.15.40:0/2921666281 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9e38064ee0 0x7f9e38067390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9e5406ed40 tx=0x7f9e500092e0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.578+0000 7f9e5c7ba700 1 -- 172.21.15.40:0/2921666281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e541609d0 msgr2=0x7f9e540c9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:55.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.578+0000 7f9e5c7ba700 1 --2- 172.21.15.40:0/2921666281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e541609d0 0x7f9e540c9090 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9e540713d0 tx=0x7f9e4c004af0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.578+0000 7f9e5c7ba700 1 -- 172.21.15.40:0/2921666281 shutdown_connections 2023-11-08T08:19:55.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.578+0000 7f9e5c7ba700 1 --2- 172.21.15.40:0/2921666281 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9e38064ee0 0x7f9e38067390 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.578+0000 7f9e5c7ba700 1 --2- 172.21.15.40:0/2921666281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e541609d0 0x7f9e540c9090 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.578+0000 7f9e5c7ba700 1 --2- 172.21.15.40:0/2921666281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e5415c3c0 0x7f9e541f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.578+0000 7f9e5c7ba700 1 --2- 172.21.15.40:0/2921666281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e540cc820 0x7f9e541f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.578+0000 7f9e5c7ba700 1 -- 172.21.15.40:0/2921666281 >> 172.21.15.40:0/2921666281 conn(0x7f9e5406e2e0 msgr2=0x7f9e540c6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:55.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.582+0000 7f9e5c7ba700 1 -- 172.21.15.40:0/2921666281 shutdown_connections 2023-11-08T08:19:55.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.582+0000 7f9e5c7ba700 1 -- 172.21.15.40:0/2921666281 wait complete. 2023-11-08T08:19:55.604 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T08:19:55.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.786+0000 7f5b1a769700 1 Processor -- start 2023-11-08T08:19:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.786+0000 7f5b1a769700 1 -- start start 2023-11-08T08:19:55.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.790+0000 7f5b1a769700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b14070620 0x7f5b140b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:55.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.790+0000 7f5b1a769700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5b14069bf0 0x7f5b1406a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:55.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.790+0000 7f5b1a769700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b1415c520 con 0x7f5b1415c150 2023-11-08T08:19:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.790+0000 7f5b1a769700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b140b4240 con 0x7f5b14069bf0 2023-11-08T08:19:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.790+0000 7f5b1a769700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1406cc50 con 0x7f5b14070620 2023-11-08T08:19:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.790+0000 7f5b19767700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f5b1415c150 0x7f5b1406ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:46572/0 (socket says 172.21.15.40:46572) 2023-11-08T08:19:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.790+0000 7f5b19767700 1 -- 172.21.15.40:0/4171602278 learned_addr learned my addr 172.21.15.40:0/4171602278 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.790+0000 7f5b18f66700 1 --2- 172.21.15.40:0/4171602278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b14070620 0x7f5b140b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.790+0000 7f5b19f68700 1 --2- 172.21.15.40:0/4171602278 >> v2:172.21.15.40:3300/0 conn(0x7f5b14069bf0 0x7f5b1406a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.790+0000 7f5b18f66700 1 -- 172.21.15.40:0/4171602278 >> v2:172.21.15.40:3300/0 conn(0x7f5b14069bf0 msgr2=0x7f5b1406a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:55.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.790+0000 7f5b18f66700 1 --2- 172.21.15.40:0/4171602278 >> v2:172.21.15.40:3300/0 conn(0x7f5b14069bf0 0x7f5b1406a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.790+0000 7f5b0bfff700 1 -- 172.21.15.40:0/4171602278 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 535172896 0 0) 0x7f5b1415c520 con 0x7f5b1415c150 2023-11-08T08:19:55.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.790+0000 7f5b18f66700 1 -- 172.21.15.40:0/4171602278 >> v1:172.21.15.169:6789/0 conn(0x7f5b1415c150 legacy=0x7f5b1406ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:55.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.790+0000 7f5b18f66700 1 -- 172.21.15.40:0/4171602278 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b140d2450 con 0x7f5b14070620 2023-11-08T08:19:55.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.790+0000 7f5b18f66700 1 --2- 172.21.15.40:0/4171602278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b14070620 0x7f5b140b3d00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009840 tx=0x7f5b04009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e743acb87b02655 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:55.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.790+0000 7f5b0bfff700 1 -- 172.21.15.40:0/4171602278 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04005e50 con 0x7f5b14070620 2023-11-08T08:19:55.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.790+0000 7f5b0bfff700 1 -- 172.21.15.40:0/4171602278 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b04006030 con 0x7f5b14070620 2023-11-08T08:19:55.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.790+0000 7f5b0bfff700 1 -- 172.21.15.40:0/4171602278 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b040063c0 con 0x7f5b14070620 2023-11-08T08:19:55.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.794+0000 7f5b1a769700 1 -- 172.21.15.40:0/4171602278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b14070620 msgr2=0x7f5b140b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:55.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.794+0000 7f5b1a769700 1 --2- 172.21.15.40:0/4171602278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b14070620 0x7f5b140b3d00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009840 tx=0x7f5b04009b50 comp rx=0 tx=0).stop 2023-11-08T08:19:55.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.794+0000 7f5b1a769700 1 -- 172.21.15.40:0/4171602278 shutdown_connections 2023-11-08T08:19:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.794+0000 7f5b1a769700 1 --2- 172.21.15.40:0/4171602278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b14070620 0x7f5b140b3d00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.794+0000 7f5b1a769700 1 --2- 172.21.15.40:0/4171602278 >> v2:172.21.15.40:3300/0 conn(0x7f5b14069bf0 0x7f5b1406a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.794+0000 7f5b1a769700 1 -- 172.21.15.40:0/4171602278 >> 172.21.15.40:0/4171602278 conn(0x7f5b140d3680 msgr2=0x7f5b140cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.794+0000 7f5b1a769700 1 -- 172.21.15.40:0/4171602278 shutdown_connections 2023-11-08T08:19:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.794+0000 7f5b1a769700 1 -- 172.21.15.40:0/4171602278 wait complete. 2023-11-08T08:19:55.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.794+0000 7f5b1a769700 1 Processor -- start 2023-11-08T08:19:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.794+0000 7f5b1a769700 1 -- start start 2023-11-08T08:19:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b1a769700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b14069bf0 0x7f5b141f1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b1a769700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b14070620 0x7f5b141453e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b1a769700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b1415c150 0x7f5b140b6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b1a769700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140cbcc0 con 0x7f5b14070620 2023-11-08T08:19:55.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b1a769700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140cb0b0 con 0x7f5b1415c150 2023-11-08T08:19:55.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b1a769700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140cb390 con 0x7f5b14069bf0 2023-11-08T08:19:55.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b18f66700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b14070620 0x7f5b141453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:55.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b19767700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b14069bf0 0x7f5b141f1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:55.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b18f66700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b14070620 0x7f5b141453e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35038/0 (socket says 172.21.15.40:35038) 2023-11-08T08:19:55.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b18f66700 1 -- 172.21.15.40:0/2776442954 learned_addr learned my addr 172.21.15.40:0/2776442954 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:55.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b19f68700 1 --2- 172.21.15.40:0/2776442954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b1415c150 0x7f5b140b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:55.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b18f66700 1 -- 172.21.15.40:0/2776442954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b14069bf0 msgr2=0x7f5b141f1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:55.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b18f66700 1 --2- 172.21.15.40:0/2776442954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b14069bf0 0x7f5b141f1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b18f66700 1 -- 172.21.15.40:0/2776442954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b1415c150 msgr2=0x7f5b140b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:55.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b18f66700 1 --2- 172.21.15.40:0/2776442954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b1415c150 0x7f5b140b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b18f66700 1 -- 172.21.15.40:0/2776442954 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b04009450 con 0x7f5b14070620 2023-11-08T08:19:55.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b19767700 1 --2- 172.21.15.40:0/2776442954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b14069bf0 0x7f5b141f1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:19:55.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b18f66700 1 --2- 172.21.15.40:0/2776442954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b14070620 0x7f5b141453e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5b04000c00 tx=0x7f5b04004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:55.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b09ffb700 1 -- 172.21.15.40:0/2776442954 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b040038c0 con 0x7f5b14070620 2023-11-08T08:19:55.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b09ffb700 1 -- 172.21.15.40:0/2776442954 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b04003a40 con 0x7f5b14070620 2023-11-08T08:19:55.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b1a769700 1 -- 172.21.15.40:0/2776442954 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b140d05e0 con 0x7f5b14070620 2023-11-08T08:19:55.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b09ffb700 1 -- 172.21.15.40:0/2776442954 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04003df0 con 0x7f5b14070620 2023-11-08T08:19:55.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.798+0000 7f5b1a769700 1 -- 172.21.15.40:0/2776442954 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b140d2730 con 0x7f5b14070620 2023-11-08T08:19:55.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.802+0000 7f5b1a769700 1 -- 172.21.15.40:0/2776442954 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b140d2140 con 0x7f5b14070620 2023-11-08T08:19:55.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.802+0000 7f5b09ffb700 1 -- 172.21.15.40:0/2776442954 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5b0400f050 con 0x7f5b14070620 2023-11-08T08:19:55.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.802+0000 7f5b09ffb700 1 --2- 172.21.15.40:0/2776442954 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5af8064ee0 0x7f5af8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:55.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.806+0000 7f5b09ffb700 1 -- 172.21.15.40:0/2776442954 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5b04083c30 con 0x7f5b14070620 2023-11-08T08:19:55.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.806+0000 7f5b19767700 1 --2- 172.21.15.40:0/2776442954 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5af8064ee0 0x7f5af8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:55.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.806+0000 7f5b19767700 1 --2- 172.21.15.40:0/2776442954 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5af8064ee0 0x7f5af8067390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5b10000f60 tx=0x7f5b10007e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:55.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.810+0000 7f5b09ffb700 1 -- 172.21.15.40:0/2776442954 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5b040519b0 con 0x7f5b14070620 2023-11-08T08:19:55.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.986+0000 7f5b1a769700 1 -- 172.21.15.40:0/2776442954 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5b140cbe70 con 0x7f5b14070620 2023-11-08T08:19:55.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.986+0000 7f5b09ffb700 1 -- 172.21.15.40:0/2776442954 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5b04054fd0 con 0x7f5b14070620 2023-11-08T08:19:55.990 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:19:55.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.990+0000 7f5b1a769700 1 -- 172.21.15.40:0/2776442954 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5af8064ee0 msgr2=0x7f5af8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:55.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.990+0000 7f5b1a769700 1 --2- 172.21.15.40:0/2776442954 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5af8064ee0 0x7f5af8067390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5b10000f60 tx=0x7f5b10007e90 comp rx=0 tx=0).stop 2023-11-08T08:19:55.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.990+0000 7f5b1a769700 1 -- 172.21.15.40:0/2776442954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b14070620 msgr2=0x7f5b141453e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:55.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.990+0000 7f5b1a769700 1 --2- 172.21.15.40:0/2776442954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b14070620 0x7f5b141453e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5b04000c00 tx=0x7f5b04004770 comp rx=0 tx=0).stop 2023-11-08T08:19:55.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.990+0000 7f5b1a769700 1 -- 172.21.15.40:0/2776442954 shutdown_connections 2023-11-08T08:19:55.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.990+0000 7f5b1a769700 1 --2- 172.21.15.40:0/2776442954 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5af8064ee0 0x7f5af8067390 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.990+0000 7f5b1a769700 1 --2- 172.21.15.40:0/2776442954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b1415c150 0x7f5b140b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.990+0000 7f5b1a769700 1 --2- 172.21.15.40:0/2776442954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b14070620 0x7f5b141453e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.990+0000 7f5b1a769700 1 --2- 172.21.15.40:0/2776442954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b14069bf0 0x7f5b141f1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:55.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.990+0000 7f5b1a769700 1 -- 172.21.15.40:0/2776442954 >> 172.21.15.40:0/2776442954 conn(0x7f5b140d3680 msgr2=0x7f5b140c8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:55.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.990+0000 7f5b1a769700 1 -- 172.21.15.40:0/2776442954 shutdown_connections 2023-11-08T08:19:55.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:55.990+0000 7f5b1a769700 1 -- 172.21.15.40:0/2776442954 wait complete. 2023-11-08T08:19:56.014 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:19:56.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.202+0000 7ff80f59e700 1 Processor -- start 2023-11-08T08:19:56.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.202+0000 7ff80f59e700 1 -- start start 2023-11-08T08:19:56.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.202+0000 7ff80f59e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff810071aa0 0x7ff81006d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:56.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.202+0000 7ff80f59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff810069e00 0x7ff81006a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:56.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.202+0000 7ff80f59e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81006c600 con 0x7ff81015c640 2023-11-08T08:19:56.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.202+0000 7ff80f59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81006cef0 con 0x7ff810069e00 2023-11-08T08:19:56.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.202+0000 7ff80f59e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8100d38f0 con 0x7ff810071aa0 2023-11-08T08:19:56.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.202+0000 7ff80ed9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff810069e00 0x7ff81006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:56.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.206+0000 7ff80dd9b700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff810071aa0 0x7ff81006d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:56.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.206+0000 7ff80ed9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff810069e00 0x7ff81006a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35046/0 (socket says 172.21.15.40:35046) 2023-11-08T08:19:56.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.206+0000 7ff80ed9d700 1 -- 172.21.15.40:0/1272191123 learned_addr learned my addr 172.21.15.40:0/1272191123 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:56.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.206+0000 7ff80dd9b700 1 -- 172.21.15.40:0/1272191123 >> v2:172.21.15.40:3300/0 conn(0x7ff810069e00 msgr2=0x7ff81006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:56.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.206+0000 7ff80dd9b700 1 --2- 172.21.15.40:0/1272191123 >> v2:172.21.15.40:3300/0 conn(0x7ff810069e00 0x7ff81006a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.206+0000 7ff80dd9b700 1 -- 172.21.15.40:0/1272191123 >> v1:172.21.15.169:6789/0 conn(0x7ff81015c640 legacy=0x7ff8100d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:56.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.206+0000 7ff80dd9b700 1 -- 172.21.15.40:0/1272191123 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8100722f0 con 0x7ff810071aa0 2023-11-08T08:19:56.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.206+0000 7ff80dd9b700 1 --2- 172.21.15.40:0/1272191123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff810071aa0 0x7ff81006d310 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff810070e70 tx=0x7ff804009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f0c73229884f517 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:56.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.206+0000 7ff80d59a700 1 -- 172.21.15.40:0/1272191123 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff804011830 con 0x7ff810071aa0 2023-11-08T08:19:56.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.206+0000 7ff80d59a700 1 -- 172.21.15.40:0/1272191123 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff804011990 con 0x7ff810071aa0 2023-11-08T08:19:56.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.206+0000 7ff80d59a700 1 -- 172.21.15.40:0/1272191123 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff804011d20 con 0x7ff810071aa0 2023-11-08T08:19:56.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.206+0000 7ff80f59e700 1 -- 172.21.15.40:0/1272191123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff810071aa0 msgr2=0x7ff81006d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:56.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.206+0000 7ff80f59e700 1 --2- 172.21.15.40:0/1272191123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff810071aa0 0x7ff81006d310 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff810070e70 tx=0x7ff804009b30 comp rx=0 tx=0).stop 2023-11-08T08:19:56.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.206+0000 7ff80f59e700 1 -- 172.21.15.40:0/1272191123 shutdown_connections 2023-11-08T08:19:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.206+0000 7ff80f59e700 1 --2- 172.21.15.40:0/1272191123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff810071aa0 0x7ff81006d310 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.206+0000 7ff80f59e700 1 --2- 172.21.15.40:0/1272191123 >> v2:172.21.15.40:3300/0 conn(0x7ff810069e00 0x7ff81006a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.206+0000 7ff80f59e700 1 -- 172.21.15.40:0/1272191123 >> 172.21.15.40:0/1272191123 conn(0x7ff8100b6520 msgr2=0x7ff810071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.206+0000 7ff80f59e700 1 -- 172.21.15.40:0/1272191123 shutdown_connections 2023-11-08T08:19:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.206+0000 7ff80f59e700 1 -- 172.21.15.40:0/1272191123 wait complete. 2023-11-08T08:19:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.210+0000 7ff80f59e700 1 Processor -- start 2023-11-08T08:19:56.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.210+0000 7ff80f59e700 1 -- start start 2023-11-08T08:19:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80f59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff810069e00 0x7ff81015f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:56.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80f59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff810071aa0 0x7ff8101f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:56.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80f59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff81015c640 0x7ff8100cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:56.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80f59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff810158c10 con 0x7ff810069e00 2023-11-08T08:19:56.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80f59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8100c75d0 con 0x7ff810071aa0 2023-11-08T08:19:56.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80f59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81015acc0 con 0x7ff81015c640 2023-11-08T08:19:56.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff810069e00 0x7ff81015f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:56.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff810069e00 0x7ff81015f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35048/0 (socket says 172.21.15.40:35048) 2023-11-08T08:19:56.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80e59c700 1 -- 172.21.15.40:0/260404262 learned_addr learned my addr 172.21.15.40:0/260404262 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:56.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80dd9b700 1 --2- 172.21.15.40:0/260404262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff810071aa0 0x7ff8101f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:56.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80ed9d700 1 --2- 172.21.15.40:0/260404262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff81015c640 0x7ff8100cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:56.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80e59c700 1 -- 172.21.15.40:0/260404262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff81015c640 msgr2=0x7ff8100cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:56.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80e59c700 1 --2- 172.21.15.40:0/260404262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff81015c640 0x7ff8100cefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80e59c700 1 -- 172.21.15.40:0/260404262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff810071aa0 msgr2=0x7ff8101f14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:56.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80e59c700 1 --2- 172.21.15.40:0/260404262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff810071aa0 0x7ff8101f14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80e59c700 1 -- 172.21.15.40:0/260404262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8040094b0 con 0x7ff810069e00 2023-11-08T08:19:56.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80dd9b700 1 --2- 172.21.15.40:0/260404262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff810071aa0 0x7ff8101f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:56.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80ed9d700 1 --2- 172.21.15.40:0/260404262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff81015c640 0x7ff8100cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:56.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80e59c700 1 --2- 172.21.15.40:0/260404262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff810069e00 0x7ff81015f8d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff80000c530 tx=0x7ff80000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:56.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff7feffd700 1 -- 172.21.15.40:0/260404262 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8000052d0 con 0x7ff810069e00 2023-11-08T08:19:56.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff7feffd700 1 -- 172.21.15.40:0/260404262 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff800005430 con 0x7ff810069e00 2023-11-08T08:19:56.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff7feffd700 1 -- 172.21.15.40:0/260404262 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8000055a0 con 0x7ff810069e00 2023-11-08T08:19:56.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80f59e700 1 -- 172.21.15.40:0/260404262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff810072010 con 0x7ff810069e00 2023-11-08T08:19:56.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.214+0000 7ff80f59e700 1 -- 172.21.15.40:0/260404262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81015ca10 con 0x7ff810069e00 2023-11-08T08:19:56.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.218+0000 7ff80f59e700 1 -- 172.21.15.40:0/260404262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81006c830 con 0x7ff810069e00 2023-11-08T08:19:56.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.218+0000 7ff7feffd700 1 -- 172.21.15.40:0/260404262 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff800019030 con 0x7ff810069e00 2023-11-08T08:19:56.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.218+0000 7ff7feffd700 1 --2- 172.21.15.40:0/260404262 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff7f0064ee0 0x7ff7f0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:56.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.218+0000 7ff7feffd700 1 -- 172.21.15.40:0/260404262 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff800081690 con 0x7ff810069e00 2023-11-08T08:19:56.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.222+0000 7ff80dd9b700 1 --2- 172.21.15.40:0/260404262 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff7f0064ee0 0x7ff7f0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:56.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.222+0000 7ff80dd9b700 1 --2- 172.21.15.40:0/260404262 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff7f0064ee0 0x7ff7f0067390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff8100c9650 tx=0x7ff804008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:56.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.222+0000 7ff7feffd700 1 -- 172.21.15.40:0/260404262 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ff80004f410 con 0x7ff810069e00 2023-11-08T08:19:56.329 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:19:56.337 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:19:56.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.402+0000 7ff80f59e700 1 -- 172.21.15.40:0/260404262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff810073cb0 con 0x7ff810069e00 2023-11-08T08:19:56.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.406+0000 7ff7feffd700 1 -- 172.21.15.40:0/260404262 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19748 (secure 0 0 0) 0x7ff800052a30 con 0x7ff810069e00 2023-11-08T08:19:56.409 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:56.410 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":30,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:45.885827+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:19:56.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.406+0000 7ff80f59e700 1 -- 172.21.15.40:0/260404262 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff7f0064ee0 msgr2=0x7ff7f0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:56.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.406+0000 7ff80f59e700 1 --2- 172.21.15.40:0/260404262 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff7f0064ee0 0x7ff7f0067390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff8100c9650 tx=0x7ff804008030 comp rx=0 tx=0).stop 2023-11-08T08:19:56.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.406+0000 7ff80f59e700 1 -- 172.21.15.40:0/260404262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff810069e00 msgr2=0x7ff81015f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:56.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.406+0000 7ff80f59e700 1 --2- 172.21.15.40:0/260404262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff810069e00 0x7ff81015f8d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff80000c530 tx=0x7ff80000c8f0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.406+0000 7ff80f59e700 1 -- 172.21.15.40:0/260404262 shutdown_connections 2023-11-08T08:19:56.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.406+0000 7ff80f59e700 1 --2- 172.21.15.40:0/260404262 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff7f0064ee0 0x7ff7f0067390 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.406+0000 7ff80f59e700 1 --2- 172.21.15.40:0/260404262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff81015c640 0x7ff8100cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.406+0000 7ff80f59e700 1 --2- 172.21.15.40:0/260404262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff810071aa0 0x7ff8101f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.406+0000 7ff80f59e700 1 --2- 172.21.15.40:0/260404262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff810069e00 0x7ff81015f8d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.406+0000 7ff80f59e700 1 -- 172.21.15.40:0/260404262 >> 172.21.15.40:0/260404262 conn(0x7ff8100b6520 msgr2=0x7ff8100b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:56.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.406+0000 7ff80f59e700 1 -- 172.21.15.40:0/260404262 shutdown_connections 2023-11-08T08:19:56.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.406+0000 7ff80f59e700 1 -- 172.21.15.40:0/260404262 wait complete. 2023-11-08T08:19:56.433 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T08:19:02.627691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T08:19:03.473821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T08:19:06.539942+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T08:19:56.434 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:19:56.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.666+0000 7f22eae62700 1 Processor -- start 2023-11-08T08:19:56.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.666+0000 7f22eae62700 1 -- start start 2023-11-08T08:19:56.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22eae62700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f22e40707d0 0x7f22e4167100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:56.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22eae62700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e40cd080 con 0x7f22e415c310 2023-11-08T08:19:56.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22eae62700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e40d21a0 con 0x7f22e40d1310 2023-11-08T08:19:56.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22eae62700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e406d240 con 0x7f22e40707d0 2023-11-08T08:19:56.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22e8bfe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f22e415c310 0x7f22e406b830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58538/0 (socket says 172.21.15.40:58538) 2023-11-08T08:19:56.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22e8bfe700 1 -- 172.21.15.40:0/2364167722 learned_addr learned my addr 172.21.15.40:0/2364167722 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:56.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22e93ff700 1 --2- 172.21.15.40:0/2364167722 >> v2:172.21.15.169:3300/0 conn(0x7f22e40707d0 0x7f22e4167100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:56.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22e37fe700 1 -- 172.21.15.40:0/2364167722 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1762688655 0 0) 0x7f22e40cd080 con 0x7f22e415c310 2023-11-08T08:19:56.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22e37fe700 1 -- 172.21.15.40:0/2364167722 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22d40035d0 con 0x7f22e415c310 2023-11-08T08:19:56.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22e37fe700 1 -- 172.21.15.40:0/2364167722 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3958161261 0 0) 0x7f22d40035d0 con 0x7f22e415c310 2023-11-08T08:19:56.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22e37fe700 1 -- 172.21.15.40:0/2364167722 >> v2:172.21.15.169:3300/0 conn(0x7f22e40707d0 msgr2=0x7f22e4167100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:56.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22e37fe700 1 --2- 172.21.15.40:0/2364167722 >> v2:172.21.15.169:3300/0 conn(0x7f22e40707d0 0x7f22e4167100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22e37fe700 1 -- 172.21.15.40:0/2364167722 >> v1:172.21.15.169:6789/0 conn(0x7f22e40d1310 legacy=0x7f22e40700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:56.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22e37fe700 1 -- 172.21.15.40:0/2364167722 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e40d1900 con 0x7f22e415c310 2023-11-08T08:19:56.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22e37fe700 1 -- 172.21.15.40:0/2364167722 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f22d8001670 con 0x7f22e415c310 2023-11-08T08:19:56.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22e37fe700 1 -- 172.21.15.40:0/2364167722 >> v1:172.21.15.40:6789/0 conn(0x7f22e415c310 legacy=0x7f22e406b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:56.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22e37fe700 1 --2- 172.21.15.40:0/2364167722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22d4003a20 0x7f22d4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:56.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22e37fe700 1 --2- 172.21.15.40:0/2364167722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22d4004870 0x7f22d4004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:56.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22e37fe700 1 --2- 172.21.15.40:0/2364167722 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22d4005220 0x7f22d400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:56.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22e37fe700 1 -- 172.21.15.40:0/2364167722 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e406d240 con 0x7f22d4004870 2023-11-08T08:19:56.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22e37fe700 1 -- 172.21.15.40:0/2364167722 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d400c130 con 0x7f22d4003a20 2023-11-08T08:19:56.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.670+0000 7f22e37fe700 1 -- 172.21.15.40:0/2364167722 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d400c2f0 con 0x7f22d4005220 2023-11-08T08:19:56.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.674+0000 7f22e3fff700 1 --2- 172.21.15.40:0/2364167722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22d4004870 0x7f22d4004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:56.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.674+0000 7f22e8bfe700 1 --2- 172.21.15.40:0/2364167722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22d4003a20 0x7f22d4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:56.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.674+0000 7f22e93ff700 1 --2- 172.21.15.40:0/2364167722 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22d4005220 0x7f22d400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:56.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.674+0000 7f22e3fff700 1 -- 172.21.15.40:0/2364167722 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22d4005220 msgr2=0x7f22d400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:56.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.674+0000 7f22e3fff700 1 --2- 172.21.15.40:0/2364167722 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22d4005220 0x7f22d400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.674+0000 7f22e3fff700 1 -- 172.21.15.40:0/2364167722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22d4003a20 msgr2=0x7f22d4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:56.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.674+0000 7f22e3fff700 1 --2- 172.21.15.40:0/2364167722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22d4003a20 0x7f22d4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.674+0000 7f22e3fff700 1 -- 172.21.15.40:0/2364167722 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d40035d0 con 0x7f22d4004870 2023-11-08T08:19:56.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.674+0000 7f22e93ff700 1 -- 172.21.15.40:0/2364167722 reap_dead start 2023-11-08T08:19:56.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.674+0000 7f22e3fff700 1 --2- 172.21.15.40:0/2364167722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22d4004870 0x7f22d4004ce0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f22d00017e0 tx=0x7f22d40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1a0da114830aab server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:56.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.674+0000 7f22e37fe700 1 -- 172.21.15.40:0/2364167722 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d00052d0 con 0x7f22d4004870 2023-11-08T08:19:56.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.674+0000 7f22e37fe700 1 -- 172.21.15.40:0/2364167722 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d0005450 con 0x7f22d4004870 2023-11-08T08:19:56.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.674+0000 7f22e37fe700 1 -- 172.21.15.40:0/2364167722 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d00038e0 con 0x7f22d4004870 2023-11-08T08:19:56.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.674+0000 7f22eae62700 1 -- 172.21.15.40:0/2364167722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22d4004870 msgr2=0x7f22d4004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:56.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.674+0000 7f22eae62700 1 --2- 172.21.15.40:0/2364167722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22d4004870 0x7f22d4004ce0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f22d00017e0 tx=0x7f22d40058a0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.674+0000 7f22eae62700 1 -- 172.21.15.40:0/2364167722 shutdown_connections 2023-11-08T08:19:56.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.674+0000 7f22eae62700 1 --2- 172.21.15.40:0/2364167722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22d4004870 0x7f22d4004ce0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.674+0000 7f22eae62700 1 -- 172.21.15.40:0/2364167722 >> 172.21.15.40:0/2364167722 conn(0x7f22e40d0140 msgr2=0x7f22e40d2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:56.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.674+0000 7f22eae62700 1 -- 172.21.15.40:0/2364167722 shutdown_connections 2023-11-08T08:19:56.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.674+0000 7f22eae62700 1 -- 172.21.15.40:0/2364167722 wait complete. 2023-11-08T08:19:56.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.678+0000 7f22eae62700 1 Processor -- start 2023-11-08T08:19:56.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.678+0000 7f22eae62700 1 -- start start 2023-11-08T08:19:56.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.678+0000 7f22eae62700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22d4004870 0x7f22e4153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:56.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.678+0000 7f22eae62700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22e415eb10 0x7f22e415d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:56.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.678+0000 7f22eae62700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22e415ddc0 0x7f22e415b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:56.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.678+0000 7f22eae62700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4153780 con 0x7f22e415eb10 2023-11-08T08:19:56.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.678+0000 7f22eae62700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40b71e0 con 0x7f22e415ddc0 2023-11-08T08:19:56.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.678+0000 7f22eae62700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40c6560 con 0x7f22d4004870 2023-11-08T08:19:56.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.682+0000 7f22e3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22e415eb10 0x7f22e415d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:56.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.682+0000 7f22e3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22e415eb10 0x7f22e415d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35072/0 (socket says 172.21.15.40:35072) 2023-11-08T08:19:56.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.682+0000 7f22e3fff700 1 -- 172.21.15.40:0/2229285120 learned_addr learned my addr 172.21.15.40:0/2229285120 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:56.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.682+0000 7f22e8bfe700 1 --2- 172.21.15.40:0/2229285120 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22d4004870 0x7f22e4153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:56.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.682+0000 7f22e93ff700 1 --2- 172.21.15.40:0/2229285120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22e415ddc0 0x7f22e415b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:56.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.682+0000 7f22e3fff700 1 -- 172.21.15.40:0/2229285120 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22d4004870 msgr2=0x7f22e4153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:56.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.682+0000 7f22e3fff700 1 --2- 172.21.15.40:0/2229285120 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22d4004870 0x7f22e4153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.682+0000 7f22e3fff700 1 -- 172.21.15.40:0/2229285120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22e415ddc0 msgr2=0x7f22e415b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:56.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.682+0000 7f22e3fff700 1 --2- 172.21.15.40:0/2229285120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22e415ddc0 0x7f22e415b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.682+0000 7f22e3fff700 1 -- 172.21.15.40:0/2229285120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e406d450 con 0x7f22e415eb10 2023-11-08T08:19:56.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.682+0000 7f22e8bfe700 1 --2- 172.21.15.40:0/2229285120 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22d4004870 0x7f22e4153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:56.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.682+0000 7f22e3fff700 1 --2- 172.21.15.40:0/2229285120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22e415eb10 0x7f22e415d880 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f22d400c100 tx=0x7f22d0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:56.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.682+0000 7f22e17fa700 1 -- 172.21.15.40:0/2229285120 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0006e80 con 0x7f22e415eb10 2023-11-08T08:19:56.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.682+0000 7f22eae62700 1 -- 172.21.15.40:0/2229285120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e415c6e0 con 0x7f22e415eb10 2023-11-08T08:19:56.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.682+0000 7f22e17fa700 1 -- 172.21.15.40:0/2229285120 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d0001290 con 0x7f22e415eb10 2023-11-08T08:19:56.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.682+0000 7f22e17fa700 1 -- 172.21.15.40:0/2229285120 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0007280 con 0x7f22e415eb10 2023-11-08T08:19:56.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.682+0000 7f22eae62700 1 -- 172.21.15.40:0/2229285120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e406d730 con 0x7f22e415eb10 2023-11-08T08:19:56.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.682+0000 7f22eae62700 1 -- 172.21.15.40:0/2229285120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e40b6f90 con 0x7f22e415eb10 2023-11-08T08:19:56.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.686+0000 7f22e17fa700 1 -- 172.21.15.40:0/2229285120 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f22d002d070 con 0x7f22e415eb10 2023-11-08T08:19:56.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.686+0000 7f22e17fa700 1 --2- 172.21.15.40:0/2229285120 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f22c8064ee0 0x7f22c8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:56.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.686+0000 7f22e17fa700 1 -- 172.21.15.40:0/2229285120 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f22d0082b30 con 0x7f22e415eb10 2023-11-08T08:19:56.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.686+0000 7f22e8bfe700 1 --2- 172.21.15.40:0/2229285120 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f22c8064ee0 0x7f22c8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:56.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.686+0000 7f22e8bfe700 1 --2- 172.21.15.40:0/2229285120 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f22c8064ee0 0x7f22c8067390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f22e4071280 tx=0x7f22d8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:56.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.694+0000 7f22e17fa700 1 -- 172.21.15.40:0/2229285120 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f22d0050830 con 0x7f22e415eb10 2023-11-08T08:19:56.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.874+0000 7f22eae62700 1 -- 172.21.15.40:0/2229285120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f22e4073d20 con 0x7f22e415eb10 2023-11-08T08:19:56.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.874+0000 7f22e17fa700 1 -- 172.21.15.40:0/2229285120 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f22d0053e50 con 0x7f22e415eb10 2023-11-08T08:19:56.879 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:19:56.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.878+0000 7f22eae62700 1 -- 172.21.15.40:0/2229285120 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f22c8064ee0 msgr2=0x7f22c8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:56.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.878+0000 7f22eae62700 1 --2- 172.21.15.40:0/2229285120 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f22c8064ee0 0x7f22c8067390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f22e4071280 tx=0x7f22d8009040 comp rx=0 tx=0).stop 2023-11-08T08:19:56.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.878+0000 7f22eae62700 1 -- 172.21.15.40:0/2229285120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22e415eb10 msgr2=0x7f22e415d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:56.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.878+0000 7f22eae62700 1 --2- 172.21.15.40:0/2229285120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22e415eb10 0x7f22e415d880 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f22d400c100 tx=0x7f22d0001120 comp rx=0 tx=0).stop 2023-11-08T08:19:56.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.878+0000 7f22eae62700 1 -- 172.21.15.40:0/2229285120 shutdown_connections 2023-11-08T08:19:56.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.878+0000 7f22eae62700 1 --2- 172.21.15.40:0/2229285120 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f22c8064ee0 0x7f22c8067390 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.878+0000 7f22eae62700 1 --2- 172.21.15.40:0/2229285120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22e415ddc0 0x7f22e415b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.878+0000 7f22eae62700 1 --2- 172.21.15.40:0/2229285120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22e415eb10 0x7f22e415d880 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.878+0000 7f22eae62700 1 --2- 172.21.15.40:0/2229285120 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22d4004870 0x7f22e4153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:56.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.878+0000 7f22eae62700 1 -- 172.21.15.40:0/2229285120 >> 172.21.15.40:0/2229285120 conn(0x7f22e40d0140 msgr2=0x7f22e4159900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:56.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.882+0000 7f22eae62700 1 -- 172.21.15.40:0/2229285120 shutdown_connections 2023-11-08T08:19:56.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:56.882+0000 7f22eae62700 1 -- 172.21.15.40:0/2229285120 wait complete. 2023-11-08T08:19:56.907 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:19:57.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.090+0000 7fdc0c9b7700 1 Processor -- start 2023-11-08T08:19:57.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.090+0000 7fdc0c9b7700 1 -- start start 2023-11-08T08:19:57.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.094+0000 7fdc0c9b7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdc080b3d00 0x7fdc0806a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:57.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.094+0000 7fdc0c9b7700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc0815c520 con 0x7fdc08070620 2023-11-08T08:19:57.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.094+0000 7fdc0c9b7700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc0806cc50 con 0x7fdc0815c150 2023-11-08T08:19:57.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.094+0000 7fdc0c9b7700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc080d2450 con 0x7fdc080b3d00 2023-11-08T08:19:57.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.094+0000 7fdc07fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdc080b3d00 0x7fdc0806a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:57.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.094+0000 7fdbfffff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdc08070620 0x7fdc08069bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58540/0 (socket says 172.21.15.40:58540) 2023-11-08T08:19:57.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.094+0000 7fdc077fe700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fdc0815c150 0x7fdc0806ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:46610/0 (socket says 172.21.15.40:46610) 2023-11-08T08:19:57.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.094+0000 7fdbfffff700 1 -- 172.21.15.40:0/1587540680 learned_addr learned my addr 172.21.15.40:0/1587540680 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:57.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.094+0000 7fdc07fff700 1 -- 172.21.15.40:0/1587540680 >> v1:172.21.15.169:6789/0 conn(0x7fdc0815c150 legacy=0x7fdc0806ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:57.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.094+0000 7fdc07fff700 1 -- 172.21.15.40:0/1587540680 >> v1:172.21.15.40:6789/0 conn(0x7fdc08070620 legacy=0x7fdc08069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:57.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.094+0000 7fdc07fff700 1 -- 172.21.15.40:0/1587540680 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc080d2730 con 0x7fdc080b3d00 2023-11-08T08:19:57.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.098+0000 7fdc07fff700 1 --2- 172.21.15.40:0/1587540680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc080b3d00 0x7fdc0806a300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009d90 tx=0x7fdc0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936b354df946b54e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:57.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.098+0000 7fdc06ffd700 1 -- 172.21.15.40:0/1587540680 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0000e070 con 0x7fdc080b3d00 2023-11-08T08:19:57.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.098+0000 7fdc06ffd700 1 -- 172.21.15.40:0/1587540680 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0000bbd0 con 0x7fdc080b3d00 2023-11-08T08:19:57.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.098+0000 7fdc06ffd700 1 -- 172.21.15.40:0/1587540680 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc00012430 con 0x7fdc080b3d00 2023-11-08T08:19:57.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.098+0000 7fdc0c9b7700 1 -- 172.21.15.40:0/1587540680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc080b3d00 msgr2=0x7fdc0806a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:57.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.098+0000 7fdc0c9b7700 1 --2- 172.21.15.40:0/1587540680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc080b3d00 0x7fdc0806a300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009d90 tx=0x7fdc0000b330 comp rx=0 tx=0).stop 2023-11-08T08:19:57.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.098+0000 7fdc0c9b7700 1 -- 172.21.15.40:0/1587540680 shutdown_connections 2023-11-08T08:19:57.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.098+0000 7fdc0c9b7700 1 --2- 172.21.15.40:0/1587540680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc080b3d00 0x7fdc0806a300 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:57.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.098+0000 7fdc0c9b7700 1 -- 172.21.15.40:0/1587540680 >> 172.21.15.40:0/1587540680 conn(0x7fdc080d3680 msgr2=0x7fdc080cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:57.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.098+0000 7fdc0c9b7700 1 -- 172.21.15.40:0/1587540680 shutdown_connections 2023-11-08T08:19:57.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.098+0000 7fdc0c9b7700 1 -- 172.21.15.40:0/1587540680 wait complete. 2023-11-08T08:19:57.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.098+0000 7fdc0c9b7700 1 Processor -- start 2023-11-08T08:19:57.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdc0c9b7700 1 -- start start 2023-11-08T08:19:57.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdc0c9b7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc08070620 0x7fdc080cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:57.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdc0c9b7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc080b3d00 0x7fdc080b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:57.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdc0c9b7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc0815c150 0x7fdc081f0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:57.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdc0c9b7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc080b7fe0 con 0x7fdc080b3d00 2023-11-08T08:19:57.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdc0c9b7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0813fd90 con 0x7fdc0815c150 2023-11-08T08:19:57.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdc0c9b7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc080b73d0 con 0x7fdc08070620 2023-11-08T08:19:57.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdbfffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc080b3d00 0x7fdc080b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:57.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdbfffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc080b3d00 0x7fdc080b69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35080/0 (socket says 172.21.15.40:35080) 2023-11-08T08:19:57.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdbfffff700 1 -- 172.21.15.40:0/972958103 learned_addr learned my addr 172.21.15.40:0/972958103 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:57.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdc07fff700 1 --2- 172.21.15.40:0/972958103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc0815c150 0x7fdc081f0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:57.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdc077fe700 1 --2- 172.21.15.40:0/972958103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc08070620 0x7fdc080cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:57.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdbfffff700 1 -- 172.21.15.40:0/972958103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc08070620 msgr2=0x7fdc080cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:57.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdbfffff700 1 --2- 172.21.15.40:0/972958103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc08070620 0x7fdc080cd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:57.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdbfffff700 1 -- 172.21.15.40:0/972958103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc0815c150 msgr2=0x7fdc081f0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:57.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdbfffff700 1 --2- 172.21.15.40:0/972958103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc0815c150 0x7fdc081f0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:57.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdbfffff700 1 -- 172.21.15.40:0/972958103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc00009a70 con 0x7fdc080b3d00 2023-11-08T08:19:57.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdbfffff700 1 --2- 172.21.15.40:0/972958103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc080b3d00 0x7fdc080b69a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000c530 tx=0x7fdbf000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:57.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdc04ff9700 1 -- 172.21.15.40:0/972958103 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00074e0 con 0x7fdc080b3d00 2023-11-08T08:19:57.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdc04ff9700 1 -- 172.21.15.40:0/972958103 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf0007640 con 0x7fdc080b3d00 2023-11-08T08:19:57.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdc0c9b7700 1 -- 172.21.15.40:0/972958103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc080d2140 con 0x7fdc080b3d00 2023-11-08T08:19:57.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdc04ff9700 1 -- 172.21.15.40:0/972958103 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0005a90 con 0x7fdc080b3d00 2023-11-08T08:19:57.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.102+0000 7fdc0c9b7700 1 -- 172.21.15.40:0/972958103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc080d05e0 con 0x7fdc080b3d00 2023-11-08T08:19:57.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.106+0000 7fdc0c9b7700 1 -- 172.21.15.40:0/972958103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc08070cd0 con 0x7fdc080b3d00 2023-11-08T08:19:57.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.106+0000 7fdc04ff9700 1 -- 172.21.15.40:0/972958103 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fdbf0019030 con 0x7fdc080b3d00 2023-11-08T08:19:57.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.110+0000 7fdc04ff9700 1 --2- 172.21.15.40:0/972958103 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdbe8064ee0 0x7fdbe8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:57.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.110+0000 7fdc04ff9700 1 -- 172.21.15.40:0/972958103 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdbf0081530 con 0x7fdc080b3d00 2023-11-08T08:19:57.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.110+0000 7fdc077fe700 1 --2- 172.21.15.40:0/972958103 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdbe8064ee0 0x7fdbe8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:57.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.110+0000 7fdc077fe700 1 --2- 172.21.15.40:0/972958103 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdbe8064ee0 0x7fdbe8067390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8000f60 tx=0x7fdbf8007ef0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:57.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.114+0000 7fdc04ff9700 1 -- 172.21.15.40:0/972958103 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fdbf004f2b0 con 0x7fdc080b3d00 2023-11-08T08:19:57.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.286+0000 7fdc0c9b7700 1 -- 172.21.15.40:0/972958103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdc0815b900 con 0x7fdc080b3d00 2023-11-08T08:19:57.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.286+0000 7fdc04ff9700 1 -- 172.21.15.40:0/972958103 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdbf0017020 con 0x7fdc080b3d00 2023-11-08T08:19:57.292 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:19:57.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.290+0000 7fdc0c9b7700 1 -- 172.21.15.40:0/972958103 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdbe8064ee0 msgr2=0x7fdbe8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:57.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.290+0000 7fdc0c9b7700 1 --2- 172.21.15.40:0/972958103 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdbe8064ee0 0x7fdbe8067390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8000f60 tx=0x7fdbf8007ef0 comp rx=0 tx=0).stop 2023-11-08T08:19:57.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.290+0000 7fdc0c9b7700 1 -- 172.21.15.40:0/972958103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc080b3d00 msgr2=0x7fdc080b69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:57.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.290+0000 7fdc0c9b7700 1 --2- 172.21.15.40:0/972958103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc080b3d00 0x7fdc080b69a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000c530 tx=0x7fdbf000c840 comp rx=0 tx=0).stop 2023-11-08T08:19:57.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.290+0000 7fdc0c9b7700 1 -- 172.21.15.40:0/972958103 shutdown_connections 2023-11-08T08:19:57.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.290+0000 7fdc0c9b7700 1 --2- 172.21.15.40:0/972958103 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdbe8064ee0 0x7fdbe8067390 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:57.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.290+0000 7fdc0c9b7700 1 --2- 172.21.15.40:0/972958103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc0815c150 0x7fdc081f0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:57.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.290+0000 7fdc0c9b7700 1 --2- 172.21.15.40:0/972958103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc080b3d00 0x7fdc080b69a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:57.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.290+0000 7fdc0c9b7700 1 --2- 172.21.15.40:0/972958103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc08070620 0x7fdc080cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:57.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.290+0000 7fdc0c9b7700 1 -- 172.21.15.40:0/972958103 >> 172.21.15.40:0/972958103 conn(0x7fdc080d3680 msgr2=0x7fdc0815b100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:57.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.290+0000 7fdc0c9b7700 1 -- 172.21.15.40:0/972958103 shutdown_connections 2023-11-08T08:19:57.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.290+0000 7fdc0c9b7700 1 -- 172.21.15.40:0/972958103 wait complete. 2023-11-08T08:19:57.319 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T08:19:57.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.502+0000 7fb2adc5a700 1 Processor -- start 2023-11-08T08:19:57.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.502+0000 7fb2adc5a700 1 -- start start 2023-11-08T08:19:57.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.506+0000 7fb2adc5a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a8070620 0x7fb2a80b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:57.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.506+0000 7fb2adc5a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb2a8069bf0 0x7fb2a806a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:57.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.506+0000 7fb2adc5a700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a815c520 con 0x7fb2a815c150 2023-11-08T08:19:57.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.506+0000 7fb2adc5a700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a80b4240 con 0x7fb2a8069bf0 2023-11-08T08:19:57.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.506+0000 7fb2adc5a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a806cc50 con 0x7fb2a8070620 2023-11-08T08:19:57.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.506+0000 7fb2acc58700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fb2a815c150 0x7fb2a806ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:43752/0 (socket says 172.21.15.40:43752) 2023-11-08T08:19:57.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.506+0000 7fb2ad459700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb2a8069bf0 0x7fb2a806a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:57.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.506+0000 7fb2a7fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a8070620 0x7fb2a80b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:57.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.506+0000 7fb2acc58700 1 -- 172.21.15.40:0/1376925045 learned_addr learned my addr 172.21.15.40:0/1376925045 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:57.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.506+0000 7fb2a77fe700 1 -- 172.21.15.40:0/1376925045 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2652191356 0 0) 0x7fb2a815c520 con 0x7fb2a815c150 2023-11-08T08:19:57.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.506+0000 7fb2a77fe700 1 -- 172.21.15.40:0/1376925045 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2980035d0 con 0x7fb2a815c150 2023-11-08T08:19:57.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.506+0000 7fb2a7fff700 1 -- 172.21.15.40:0/1376925045 >> v2:172.21.15.169:3300/0 conn(0x7fb2a8069bf0 msgr2=0x7fb2a806a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:57.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.506+0000 7fb2a7fff700 1 --2- 172.21.15.40:0/1376925045 >> v2:172.21.15.169:3300/0 conn(0x7fb2a8069bf0 0x7fb2a806a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:57.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.506+0000 7fb2a7fff700 1 -- 172.21.15.40:0/1376925045 >> v1:172.21.15.169:6789/0 conn(0x7fb2a815c150 legacy=0x7fb2a806ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:57.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.506+0000 7fb2a7fff700 1 -- 172.21.15.40:0/1376925045 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a80d2450 con 0x7fb2a8070620 2023-11-08T08:19:57.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.506+0000 7fb2a7fff700 1 --2- 172.21.15.40:0/1376925045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a8070620 0x7fb2a80b3d00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb294009840 tx=0x7fb294009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73b3d445779cab84 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:57.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.506+0000 7fb2a77fe700 1 -- 172.21.15.40:0/1376925045 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294005e50 con 0x7fb2a8070620 2023-11-08T08:19:57.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.510+0000 7fb2a77fe700 1 -- 172.21.15.40:0/1376925045 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb294006030 con 0x7fb2a8070620 2023-11-08T08:19:57.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.510+0000 7fb2a77fe700 1 -- 172.21.15.40:0/1376925045 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2940063c0 con 0x7fb2a8070620 2023-11-08T08:19:57.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.510+0000 7fb2adc5a700 1 -- 172.21.15.40:0/1376925045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a8070620 msgr2=0x7fb2a80b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:57.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.510+0000 7fb2adc5a700 1 --2- 172.21.15.40:0/1376925045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a8070620 0x7fb2a80b3d00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb294009840 tx=0x7fb294009b50 comp rx=0 tx=0).stop 2023-11-08T08:19:57.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.510+0000 7fb2adc5a700 1 -- 172.21.15.40:0/1376925045 shutdown_connections 2023-11-08T08:19:57.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.510+0000 7fb2adc5a700 1 --2- 172.21.15.40:0/1376925045 >> v2:172.21.15.169:3300/0 conn(0x7fb2a8069bf0 0x7fb2a806a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:57.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.510+0000 7fb2adc5a700 1 --2- 172.21.15.40:0/1376925045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a8070620 0x7fb2a80b3d00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:57.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.510+0000 7fb2adc5a700 1 -- 172.21.15.40:0/1376925045 >> 172.21.15.40:0/1376925045 conn(0x7fb2a80d3680 msgr2=0x7fb2a80cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:57.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.510+0000 7fb2adc5a700 1 -- 172.21.15.40:0/1376925045 shutdown_connections 2023-11-08T08:19:57.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.510+0000 7fb2adc5a700 1 -- 172.21.15.40:0/1376925045 wait complete. 2023-11-08T08:19:57.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.514+0000 7fb2adc5a700 1 Processor -- start 2023-11-08T08:19:57.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.514+0000 7fb2adc5a700 1 -- start start 2023-11-08T08:19:57.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.514+0000 7fb2adc5a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a8069bf0 0x7fb2a80cf880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:57.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.514+0000 7fb2adc5a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a8070620 0x7fb2a81599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:57.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.514+0000 7fb2adc5a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2a815c150 0x7fb2a80cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:57.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.514+0000 7fb2adc5a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a80cfdc0 con 0x7fb2a8070620 2023-11-08T08:19:57.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.514+0000 7fb2adc5a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a814a9a0 con 0x7fb2a815c150 2023-11-08T08:19:57.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.514+0000 7fb2adc5a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a81f5900 con 0x7fb2a8069bf0 2023-11-08T08:19:57.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.514+0000 7fb2a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a8070620 0x7fb2a81599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:57.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.514+0000 7fb2a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a8070620 0x7fb2a81599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35938/0 (socket says 172.21.15.40:35938) 2023-11-08T08:19:57.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.514+0000 7fb2a7fff700 1 -- 172.21.15.40:0/1620115315 learned_addr learned my addr 172.21.15.40:0/1620115315 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:57.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.514+0000 7fb2acc58700 1 --2- 172.21.15.40:0/1620115315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a8069bf0 0x7fb2a80cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:57.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.514+0000 7fb2ad459700 1 --2- 172.21.15.40:0/1620115315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2a815c150 0x7fb2a80cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:57.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.514+0000 7fb2a7fff700 1 -- 172.21.15.40:0/1620115315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a8069bf0 msgr2=0x7fb2a80cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:57.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.514+0000 7fb2a7fff700 1 --2- 172.21.15.40:0/1620115315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a8069bf0 0x7fb2a80cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:57.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.514+0000 7fb2a7fff700 1 -- 172.21.15.40:0/1620115315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2a815c150 msgr2=0x7fb2a80cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:57.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.514+0000 7fb2a7fff700 1 --2- 172.21.15.40:0/1620115315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2a815c150 0x7fb2a80cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:57.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.514+0000 7fb2a7fff700 1 -- 172.21.15.40:0/1620115315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb294009450 con 0x7fb2a8070620 2023-11-08T08:19:57.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.518+0000 7fb2a7fff700 1 --2- 172.21.15.40:0/1620115315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a8070620 0x7fb2a81599a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb294009b90 tx=0x7fb294004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:57.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.518+0000 7fb2a57fa700 1 -- 172.21.15.40:0/1620115315 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294003ad0 con 0x7fb2a8070620 2023-11-08T08:19:57.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.518+0000 7fb2a57fa700 1 -- 172.21.15.40:0/1620115315 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb294003c50 con 0x7fb2a8070620 2023-11-08T08:19:57.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.518+0000 7fb2adc5a700 1 -- 172.21.15.40:0/1620115315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a80d05e0 con 0x7fb2a8070620 2023-11-08T08:19:57.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.518+0000 7fb2a57fa700 1 -- 172.21.15.40:0/1620115315 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2940045d0 con 0x7fb2a8070620 2023-11-08T08:19:57.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.518+0000 7fb2adc5a700 1 -- 172.21.15.40:0/1620115315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a80d2730 con 0x7fb2a8070620 2023-11-08T08:19:57.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.518+0000 7fb2adc5a700 1 -- 172.21.15.40:0/1620115315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a80d2140 con 0x7fb2a8070620 2023-11-08T08:19:57.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.522+0000 7fb2a57fa700 1 -- 172.21.15.40:0/1620115315 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fb29400f050 con 0x7fb2a8070620 2023-11-08T08:19:57.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.522+0000 7fb2a57fa700 1 --2- 172.21.15.40:0/1620115315 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb28c064ee0 0x7fb28c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:57.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.522+0000 7fb2a57fa700 1 -- 172.21.15.40:0/1620115315 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb294083cd0 con 0x7fb2a8070620 2023-11-08T08:19:57.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.522+0000 7fb2acc58700 1 --2- 172.21.15.40:0/1620115315 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb28c064ee0 0x7fb28c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:57.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.526+0000 7fb2acc58700 1 --2- 172.21.15.40:0/1620115315 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb28c064ee0 0x7fb28c067390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0011e0 tx=0x7fb29c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:57.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.526+0000 7fb2a57fa700 1 -- 172.21.15.40:0/1620115315 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fb294051a50 con 0x7fb2a8070620 2023-11-08T08:19:57.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.698+0000 7fb2adc5a700 1 -- 172.21.15.40:0/1620115315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb2a8073d20 con 0x7fb2a8070620 2023-11-08T08:19:57.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.698+0000 7fb2a57fa700 1 -- 172.21.15.40:0/1620115315 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb294055070 con 0x7fb2a8070620 2023-11-08T08:19:57.704 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:19:57.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.702+0000 7fb2adc5a700 1 -- 172.21.15.40:0/1620115315 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb28c064ee0 msgr2=0x7fb28c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:57.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.702+0000 7fb2adc5a700 1 --2- 172.21.15.40:0/1620115315 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb28c064ee0 0x7fb28c067390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0011e0 tx=0x7fb29c009040 comp rx=0 tx=0).stop 2023-11-08T08:19:57.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.702+0000 7fb2adc5a700 1 -- 172.21.15.40:0/1620115315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a8070620 msgr2=0x7fb2a81599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:57.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.702+0000 7fb2adc5a700 1 --2- 172.21.15.40:0/1620115315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a8070620 0x7fb2a81599a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb294009b90 tx=0x7fb294004770 comp rx=0 tx=0).stop 2023-11-08T08:19:57.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.702+0000 7fb2adc5a700 1 -- 172.21.15.40:0/1620115315 shutdown_connections 2023-11-08T08:19:57.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.702+0000 7fb2adc5a700 1 --2- 172.21.15.40:0/1620115315 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb28c064ee0 0x7fb28c067390 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:57.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.702+0000 7fb2adc5a700 1 --2- 172.21.15.40:0/1620115315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb2a815c150 0x7fb2a80cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:57.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.702+0000 7fb2adc5a700 1 --2- 172.21.15.40:0/1620115315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a8070620 0x7fb2a81599a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:57.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.702+0000 7fb2adc5a700 1 --2- 172.21.15.40:0/1620115315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb2a8069bf0 0x7fb2a80cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:57.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.702+0000 7fb2adc5a700 1 -- 172.21.15.40:0/1620115315 >> 172.21.15.40:0/1620115315 conn(0x7fb2a80d3680 msgr2=0x7fb2a80b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:57.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.702+0000 7fb2adc5a700 1 -- 172.21.15.40:0/1620115315 shutdown_connections 2023-11-08T08:19:57.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.702+0000 7fb2adc5a700 1 -- 172.21.15.40:0/1620115315 wait complete. 2023-11-08T08:19:57.730 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T08:19:57.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.914+0000 7fc206425700 1 Processor -- start 2023-11-08T08:19:57.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.914+0000 7fc206425700 1 -- start start 2023-11-08T08:19:57.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.914+0000 7fc206425700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc20015c700 0x7fc2000702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:57.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.914+0000 7fc206425700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc20006cf00 con 0x7fc2000b1be0 2023-11-08T08:19:57.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.914+0000 7fc206425700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc20006ac00 con 0x7fc2000b2130 2023-11-08T08:19:57.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.914+0000 7fc206425700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2000cd360 con 0x7fc20015c700 2023-11-08T08:19:57.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.914+0000 7fc1fffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc20015c700 0x7fc2000702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:57.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.914+0000 7fc1fffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc20015c700 0x7fc2000702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:40754/0 (socket says 172.21.15.40:40754) 2023-11-08T08:19:57.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.914+0000 7fc1ff7fe700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fc2000b2130 0x7fc20006c0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:43754/0 (socket says 172.21.15.40:43754) 2023-11-08T08:19:57.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.914+0000 7fc1fffff700 1 -- 172.21.15.40:0/2868619806 learned_addr learned my addr 172.21.15.40:0/2868619806 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:57.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.914+0000 7fc2049c2700 1 --1- 172.21.15.40:0/2868619806 >> v1:172.21.15.40:6789/0 conn(0x7fc2000b1be0 0x7fc20006c7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41852/0 (socket says 172.21.15.40:41852) 2023-11-08T08:19:57.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.918+0000 7fc1feffd700 1 -- 172.21.15.40:0/2868619806 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1116819229 0 0) 0x7fc20006ac00 con 0x7fc2000b2130 2023-11-08T08:19:57.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.918+0000 7fc1feffd700 1 -- 172.21.15.40:0/2868619806 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1e80035d0 con 0x7fc2000b2130 2023-11-08T08:19:57.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.918+0000 7fc1fffff700 1 -- 172.21.15.40:0/2868619806 >> v1:172.21.15.169:6789/0 conn(0x7fc2000b2130 legacy=0x7fc20006c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:57.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.918+0000 7fc1feffd700 1 -- 172.21.15.40:0/2868619806 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1653559924 0 0) 0x7fc20006cf00 con 0x7fc2000b1be0 2023-11-08T08:19:57.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.918+0000 7fc1fffff700 1 -- 172.21.15.40:0/2868619806 >> v1:172.21.15.40:6789/0 conn(0x7fc2000b1be0 legacy=0x7fc20006c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:57.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.918+0000 7fc1fffff700 1 -- 172.21.15.40:0/2868619806 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc200160190 con 0x7fc20015c700 2023-11-08T08:19:57.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.918+0000 7fc1fffff700 1 --2- 172.21.15.40:0/2868619806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc20015c700 0x7fc2000702f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009e80 tx=0x7fc1f000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccc75f5ab7704233 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:57.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.918+0000 7fc1feffd700 1 -- 172.21.15.40:0/2868619806 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f000e070 con 0x7fc20015c700 2023-11-08T08:19:57.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.918+0000 7fc1feffd700 1 -- 172.21.15.40:0/2868619806 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f000b910 con 0x7fc20015c700 2023-11-08T08:19:57.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.918+0000 7fc1feffd700 1 -- 172.21.15.40:0/2868619806 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f000bcc0 con 0x7fc20015c700 2023-11-08T08:19:57.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.918+0000 7fc206425700 1 -- 172.21.15.40:0/2868619806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc20015c700 msgr2=0x7fc2000702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:57.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.918+0000 7fc206425700 1 --2- 172.21.15.40:0/2868619806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc20015c700 0x7fc2000702f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009e80 tx=0x7fc1f000b330 comp rx=0 tx=0).stop 2023-11-08T08:19:57.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.918+0000 7fc206425700 1 -- 172.21.15.40:0/2868619806 shutdown_connections 2023-11-08T08:19:57.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.918+0000 7fc206425700 1 --2- 172.21.15.40:0/2868619806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc20015c700 0x7fc2000702f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:57.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.918+0000 7fc206425700 1 -- 172.21.15.40:0/2868619806 >> 172.21.15.40:0/2868619806 conn(0x7fc2000d2d10 msgr2=0x7fc2000d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:57.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.918+0000 7fc206425700 1 -- 172.21.15.40:0/2868619806 shutdown_connections 2023-11-08T08:19:57.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.918+0000 7fc206425700 1 -- 172.21.15.40:0/2868619806 wait complete. 2023-11-08T08:19:57.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.922+0000 7fc206425700 1 Processor -- start 2023-11-08T08:19:57.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.922+0000 7fc206425700 1 -- start start 2023-11-08T08:19:57.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc206425700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2000b1be0 0x7fc20014ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:57.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc206425700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2000b2130 0x7fc2000c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:57.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc206425700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc20015c700 0x7fc2000cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:57.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc206425700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20014ee60 con 0x7fc2000b1be0 2023-11-08T08:19:57.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc206425700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20014f140 con 0x7fc2000b2130 2023-11-08T08:19:57.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc206425700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20014f420 con 0x7fc20015c700 2023-11-08T08:19:57.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc1fffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2000b1be0 0x7fc20014ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:57.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc1fffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2000b1be0 0x7fc20014ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35942/0 (socket says 172.21.15.40:35942) 2023-11-08T08:19:57.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc1fffff700 1 -- 172.21.15.40:0/3174939596 learned_addr learned my addr 172.21.15.40:0/3174939596 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:57.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc1ff7fe700 1 --2- 172.21.15.40:0/3174939596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2000b2130 0x7fc2000c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:57.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc1fffff700 1 -- 172.21.15.40:0/3174939596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc20015c700 msgr2=0x7fc2000cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:57.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc2049c2700 1 --2- 172.21.15.40:0/3174939596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc20015c700 0x7fc2000cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:57.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc1fffff700 1 --2- 172.21.15.40:0/3174939596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc20015c700 0x7fc2000cbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:57.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc1fffff700 1 -- 172.21.15.40:0/3174939596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2000b2130 msgr2=0x7fc2000c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:57.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc1fffff700 1 --2- 172.21.15.40:0/3174939596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2000b2130 0x7fc2000c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:57.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc1fffff700 1 -- 172.21.15.40:0/3174939596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f0009b30 con 0x7fc2000b1be0 2023-11-08T08:19:57.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc1ff7fe700 1 --2- 172.21.15.40:0/3174939596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2000b2130 0x7fc2000c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:57.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc1fffff700 1 --2- 172.21.15.40:0/3174939596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2000b1be0 0x7fc20014ff00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc2000d3a50 tx=0x7fc1f0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:57.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc1fcff9700 1 -- 172.21.15.40:0/3174939596 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f001c640 con 0x7fc2000b1be0 2023-11-08T08:19:57.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc1fcff9700 1 -- 172.21.15.40:0/3174939596 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f001c7a0 con 0x7fc2000b1be0 2023-11-08T08:19:57.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc1fcff9700 1 -- 172.21.15.40:0/3174939596 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f001cb10 con 0x7fc2000b1be0 2023-11-08T08:19:57.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc206425700 1 -- 172.21.15.40:0/3174939596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc200160750 con 0x7fc2000b1be0 2023-11-08T08:19:57.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.926+0000 7fc206425700 1 -- 172.21.15.40:0/3174939596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc200160470 con 0x7fc2000b1be0 2023-11-08T08:19:57.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.930+0000 7fc206425700 1 -- 172.21.15.40:0/3174939596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc200160a30 con 0x7fc2000b1be0 2023-11-08T08:19:57.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.930+0000 7fc1fcff9700 1 -- 172.21.15.40:0/3174939596 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc1f0024030 con 0x7fc2000b1be0 2023-11-08T08:19:57.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.934+0000 7fc1fcff9700 1 --2- 172.21.15.40:0/3174939596 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc1ec064ee0 0x7fc1ec067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:57.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.934+0000 7fc1fcff9700 1 -- 172.21.15.40:0/3174939596 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc1f0083c70 con 0x7fc2000b1be0 2023-11-08T08:19:57.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.934+0000 7fc1ff7fe700 1 --2- 172.21.15.40:0/3174939596 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc1ec064ee0 0x7fc1ec067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:57.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.934+0000 7fc1ff7fe700 1 --2- 172.21.15.40:0/3174939596 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc1ec064ee0 0x7fc1ec067390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc2000b5590 tx=0x7fc1f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:57.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:57.942+0000 7fc1fcff9700 1 -- 172.21.15.40:0/3174939596 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc1f00519f0 con 0x7fc2000b1be0 2023-11-08T08:19:58.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.118+0000 7fc206425700 1 -- 172.21.15.40:0/3174939596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc2000726c0 con 0x7fc2000b1be0 2023-11-08T08:19:58.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.122+0000 7fc1fcff9700 1 -- 172.21.15.40:0/3174939596 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc1f0055010 con 0x7fc2000b1be0 2023-11-08T08:19:58.125 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-11-08T08:19:58.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.122+0000 7fc206425700 1 -- 172.21.15.40:0/3174939596 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc1ec064ee0 msgr2=0x7fc1ec067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:58.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.122+0000 7fc206425700 1 --2- 172.21.15.40:0/3174939596 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc1ec064ee0 0x7fc1ec067390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc2000b5590 tx=0x7fc1f4009040 comp rx=0 tx=0).stop 2023-11-08T08:19:58.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.122+0000 7fc206425700 1 -- 172.21.15.40:0/3174939596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2000b1be0 msgr2=0x7fc20014ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:58.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.122+0000 7fc206425700 1 --2- 172.21.15.40:0/3174939596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2000b1be0 0x7fc20014ff00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc2000d3a50 tx=0x7fc1f0004af0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.122+0000 7fc206425700 1 -- 172.21.15.40:0/3174939596 shutdown_connections 2023-11-08T08:19:58.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.122+0000 7fc206425700 1 --2- 172.21.15.40:0/3174939596 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc1ec064ee0 0x7fc1ec067390 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.122+0000 7fc206425700 1 --2- 172.21.15.40:0/3174939596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc20015c700 0x7fc2000cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.122+0000 7fc206425700 1 --2- 172.21.15.40:0/3174939596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2000b2130 0x7fc2000c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.122+0000 7fc206425700 1 --2- 172.21.15.40:0/3174939596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2000b1be0 0x7fc20014ff00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.122+0000 7fc206425700 1 -- 172.21.15.40:0/3174939596 >> 172.21.15.40:0/3174939596 conn(0x7fc2000d2d10 msgr2=0x7fc2000b29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:58.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.122+0000 7fc206425700 1 -- 172.21.15.40:0/3174939596 shutdown_connections 2023-11-08T08:19:58.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.126+0000 7fc206425700 1 -- 172.21.15.40:0/3174939596 wait complete. 2023-11-08T08:19:58.152 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T08:19:58.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.326+0000 7f827f308700 1 Processor -- start 2023-11-08T08:19:58.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.326+0000 7f827f308700 1 -- start start 2023-11-08T08:19:58.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.330+0000 7f827f308700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f827815e190 0x7f82780cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:58.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.330+0000 7f827f308700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8278161560 0x7f8278069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:58.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.330+0000 7f827f308700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f827806ff10 0x7f8278070390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:58.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.330+0000 7f827f308700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82780708d0 con 0x7f8278161560 2023-11-08T08:19:58.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.330+0000 7f827f308700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f827815e560 con 0x7f827806ff10 2023-11-08T08:19:58.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.330+0000 7f827f308700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827806be80 con 0x7f827815e190 2023-11-08T08:19:58.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.330+0000 7f827db05700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8278161560 0x7f8278069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:58.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.330+0000 7f827db05700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8278161560 0x7f8278069bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35950/0 (socket says 172.21.15.40:35950) 2023-11-08T08:19:58.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.330+0000 7f827db05700 1 -- 172.21.15.40:0/3200546123 learned_addr learned my addr 172.21.15.40:0/3200546123 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:58.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.330+0000 7f827e306700 1 --2- 172.21.15.40:0/3200546123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f827815e190 0x7f82780cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:58.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.330+0000 7f827eb07700 1 --2- 172.21.15.40:0/3200546123 >> v2:172.21.15.169:3300/0 conn(0x7f827806ff10 0x7f8278070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:58.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.330+0000 7f827db05700 1 -- 172.21.15.40:0/3200546123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f827815e190 msgr2=0x7f82780cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:58.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.330+0000 7f827db05700 1 --2- 172.21.15.40:0/3200546123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f827815e190 0x7f82780cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.330+0000 7f827db05700 1 -- 172.21.15.40:0/3200546123 >> v2:172.21.15.169:3300/0 conn(0x7f827806ff10 msgr2=0x7f8278070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:58.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.330+0000 7f827db05700 1 --2- 172.21.15.40:0/3200546123 >> v2:172.21.15.169:3300/0 conn(0x7f827806ff10 0x7f8278070390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.330+0000 7f827db05700 1 -- 172.21.15.40:0/3200546123 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82780ce330 con 0x7f8278161560 2023-11-08T08:19:58.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.330+0000 7f827e306700 1 --2- 172.21.15.40:0/3200546123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f827815e190 0x7f82780cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:58.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.330+0000 7f827eb07700 1 --2- 172.21.15.40:0/3200546123 >> v2:172.21.15.169:3300/0 conn(0x7f827806ff10 0x7f8278070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:58.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.330+0000 7f827db05700 1 --2- 172.21.15.40:0/3200546123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8278161560 0x7f8278069bf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f82780b4800 tx=0x7f8268009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0907fee5b79f52 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:58.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.334+0000 7f827d304700 1 -- 172.21.15.40:0/3200546123 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826800b8a0 con 0x7f8278161560 2023-11-08T08:19:58.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.334+0000 7f827d304700 1 -- 172.21.15.40:0/3200546123 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826800ba00 con 0x7f8278161560 2023-11-08T08:19:58.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.334+0000 7f827d304700 1 -- 172.21.15.40:0/3200546123 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826800bd90 con 0x7f8278161560 2023-11-08T08:19:58.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.334+0000 7f827f308700 1 -- 172.21.15.40:0/3200546123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8278161560 msgr2=0x7f8278069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:58.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.334+0000 7f827f308700 1 --2- 172.21.15.40:0/3200546123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8278161560 0x7f8278069bf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f82780b4800 tx=0x7f8268009bf0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.334+0000 7f827f308700 1 -- 172.21.15.40:0/3200546123 shutdown_connections 2023-11-08T08:19:58.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.334+0000 7f827f308700 1 --2- 172.21.15.40:0/3200546123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8278161560 0x7f8278069bf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.334+0000 7f827f308700 1 --2- 172.21.15.40:0/3200546123 >> v2:172.21.15.169:3300/0 conn(0x7f827806ff10 0x7f8278070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.334+0000 7f827f308700 1 --2- 172.21.15.40:0/3200546123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f827815e190 0x7f82780cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.334+0000 7f827f308700 1 -- 172.21.15.40:0/3200546123 >> 172.21.15.40:0/3200546123 conn(0x7f82780cf280 msgr2=0x7f82780cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:58.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.334+0000 7f827f308700 1 -- 172.21.15.40:0/3200546123 shutdown_connections 2023-11-08T08:19:58.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.334+0000 7f827f308700 1 -- 172.21.15.40:0/3200546123 wait complete. 2023-11-08T08:19:58.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.338+0000 7f827f308700 1 Processor -- start 2023-11-08T08:19:58.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.338+0000 7f827f308700 1 -- start start 2023-11-08T08:19:58.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827f308700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f827806ff10 0x7f8278153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:58.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827f308700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f827815e190 0x7f82781546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:58.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827f308700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8278161560 0x7f8278154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:58.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827f308700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8278154510 con 0x7f8278161560 2023-11-08T08:19:58.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827f308700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827815cb30 con 0x7f827815e190 2023-11-08T08:19:58.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827f308700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827815cdd0 con 0x7f827806ff10 2023-11-08T08:19:58.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827eb07700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8278161560 0x7f8278154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:58.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827eb07700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8278161560 0x7f8278154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35962/0 (socket says 172.21.15.40:35962) 2023-11-08T08:19:58.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827eb07700 1 -- 172.21.15.40:0/338917806 learned_addr learned my addr 172.21.15.40:0/338917806 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:58.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827db05700 1 --2- 172.21.15.40:0/338917806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f827815e190 0x7f82781546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:58.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827e306700 1 --2- 172.21.15.40:0/338917806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f827806ff10 0x7f8278153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:58.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827eb07700 1 -- 172.21.15.40:0/338917806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f827806ff10 msgr2=0x7f8278153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:58.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827eb07700 1 --2- 172.21.15.40:0/338917806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f827806ff10 0x7f8278153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827eb07700 1 -- 172.21.15.40:0/338917806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f827815e190 msgr2=0x7f82781546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:58.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827eb07700 1 --2- 172.21.15.40:0/338917806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f827815e190 0x7f82781546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827eb07700 1 -- 172.21.15.40:0/338917806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8268009570 con 0x7f8278161560 2023-11-08T08:19:58.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827db05700 1 --2- 172.21.15.40:0/338917806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f827815e190 0x7f82781546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:19:58.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827eb07700 1 --2- 172.21.15.40:0/338917806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8278161560 0x7f8278154c20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f827000cef0 tx=0x7f827000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:58.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f826effd700 1 -- 172.21.15.40:0/338917806 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827000ae50 con 0x7f8278161560 2023-11-08T08:19:58.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f826effd700 1 -- 172.21.15.40:0/338917806 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8270015440 con 0x7f8278161560 2023-11-08T08:19:58.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827f308700 1 -- 172.21.15.40:0/338917806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82780afc70 con 0x7f8278161560 2023-11-08T08:19:58.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f826effd700 1 -- 172.21.15.40:0/338917806 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82700157f0 con 0x7f8278161560 2023-11-08T08:19:58.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827f308700 1 -- 172.21.15.40:0/338917806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82780af990 con 0x7f8278161560 2023-11-08T08:19:58.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.342+0000 7f827f308700 1 -- 172.21.15.40:0/338917806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82780aff50 con 0x7f8278161560 2023-11-08T08:19:58.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.346+0000 7f826effd700 1 -- 172.21.15.40:0/338917806 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f827001c030 con 0x7f8278161560 2023-11-08T08:19:58.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.346+0000 7f826effd700 1 --2- 172.21.15.40:0/338917806 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f825c064ee0 0x7f825c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:58.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.346+0000 7f826effd700 1 -- 172.21.15.40:0/338917806 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8270082760 con 0x7f8278161560 2023-11-08T08:19:58.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.346+0000 7f827e306700 1 --2- 172.21.15.40:0/338917806 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f825c064ee0 0x7f825c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:58.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.350+0000 7f827e306700 1 --2- 172.21.15.40:0/338917806 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f825c064ee0 0x7f825c067390 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8274006fd0 tx=0x7f8274008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:58.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.354+0000 7f826effd700 1 -- 172.21.15.40:0/338917806 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f82700504e0 con 0x7f8278161560 2023-11-08T08:19:58.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.526+0000 7f827f308700 1 -- 172.21.15.40:0/338917806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8278073d20 con 0x7f8278161560 2023-11-08T08:19:58.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.526+0000 7f826effd700 1 -- 172.21.15.40:0/338917806 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f827001a0a0 con 0x7f8278161560 2023-11-08T08:19:58.530 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:19:58.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.530+0000 7f827f308700 1 -- 172.21.15.40:0/338917806 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f825c064ee0 msgr2=0x7f825c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:58.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.530+0000 7f827f308700 1 --2- 172.21.15.40:0/338917806 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f825c064ee0 0x7f825c067390 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8274006fd0 tx=0x7f8274008040 comp rx=0 tx=0).stop 2023-11-08T08:19:58.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.530+0000 7f827f308700 1 -- 172.21.15.40:0/338917806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8278161560 msgr2=0x7f8278154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:58.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.530+0000 7f827f308700 1 --2- 172.21.15.40:0/338917806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8278161560 0x7f8278154c20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f827000cef0 tx=0x7f827000a340 comp rx=0 tx=0).stop 2023-11-08T08:19:58.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.530+0000 7f827f308700 1 -- 172.21.15.40:0/338917806 shutdown_connections 2023-11-08T08:19:58.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.530+0000 7f827f308700 1 --2- 172.21.15.40:0/338917806 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f825c064ee0 0x7f825c067390 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.530+0000 7f827f308700 1 --2- 172.21.15.40:0/338917806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8278161560 0x7f8278154c20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.530+0000 7f827f308700 1 --2- 172.21.15.40:0/338917806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f827815e190 0x7f82781546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.530+0000 7f827f308700 1 --2- 172.21.15.40:0/338917806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f827806ff10 0x7f8278153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.530+0000 7f827f308700 1 -- 172.21.15.40:0/338917806 >> 172.21.15.40:0/338917806 conn(0x7f82780cf280 msgr2=0x7f827815f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:58.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.530+0000 7f827f308700 1 -- 172.21.15.40:0/338917806 shutdown_connections 2023-11-08T08:19:58.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.530+0000 7f827f308700 1 -- 172.21.15.40:0/338917806 wait complete. 2023-11-08T08:19:58.560 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:19:58.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.738+0000 7f7b9cb69700 1 Processor -- start 2023-11-08T08:19:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.738+0000 7f7b9cb69700 1 -- start start 2023-11-08T08:19:58.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.742+0000 7f7b9cb69700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b980704b0 0x7f7b98164960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:58.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.742+0000 7f7b9cb69700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b980ad6c0 con 0x7f7b98160710 2023-11-08T08:19:58.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.742+0000 7f7b9cb69700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b980adc60 con 0x7f7b98070080 2023-11-08T08:19:58.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.742+0000 7f7b9cb69700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9806b1c0 con 0x7f7b980704b0 2023-11-08T08:19:58.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.742+0000 7f7b977fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7b98160710 0x7f7b980cc320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41856/0 (socket says 172.21.15.40:41856) 2023-11-08T08:19:58.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.742+0000 7f7b977fe700 1 -- 172.21.15.40:0/4121952447 learned_addr learned my addr 172.21.15.40:0/4121952447 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:58.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.742+0000 7f7b97fff700 1 --2- 172.21.15.40:0/4121952447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b980704b0 0x7f7b98164960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:58.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.742+0000 7f7b967fc700 1 -- 172.21.15.40:0/4121952447 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3405246571 0 0) 0x7f7b980ad6c0 con 0x7f7b98160710 2023-11-08T08:19:58.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.742+0000 7f7b967fc700 1 -- 172.21.15.40:0/4121952447 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b800035d0 con 0x7f7b98160710 2023-11-08T08:19:58.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.742+0000 7f7b967fc700 1 -- 172.21.15.40:0/4121952447 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572604080 0 0) 0x7f7b980adc60 con 0x7f7b98070080 2023-11-08T08:19:58.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.742+0000 7f7b967fc700 1 -- 172.21.15.40:0/4121952447 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b980ad6c0 con 0x7f7b98070080 2023-11-08T08:19:58.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.742+0000 7f7b97fff700 1 -- 172.21.15.40:0/4121952447 >> v1:172.21.15.169:6789/0 conn(0x7f7b98070080 legacy=0x7f7b98164250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:58.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.742+0000 7f7b97fff700 1 -- 172.21.15.40:0/4121952447 >> v1:172.21.15.40:6789/0 conn(0x7f7b98160710 legacy=0x7f7b980cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:58.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.742+0000 7f7b97fff700 1 -- 172.21.15.40:0/4121952447 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b980cca30 con 0x7f7b980704b0 2023-11-08T08:19:58.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.742+0000 7f7b97fff700 1 --2- 172.21.15.40:0/4121952447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b980704b0 0x7f7b98164960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009dc0 tx=0x7f7b9000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca51e173620ca5e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:58.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.742+0000 7f7b967fc700 1 -- 172.21.15.40:0/4121952447 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9000e070 con 0x7f7b980704b0 2023-11-08T08:19:58.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.742+0000 7f7b967fc700 1 -- 172.21.15.40:0/4121952447 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9000bb20 con 0x7f7b980704b0 2023-11-08T08:19:58.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.742+0000 7f7b967fc700 1 -- 172.21.15.40:0/4121952447 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b90012430 con 0x7f7b980704b0 2023-11-08T08:19:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.746+0000 7f7b9cb69700 1 -- 172.21.15.40:0/4121952447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b980704b0 msgr2=0x7f7b98164960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.746+0000 7f7b9cb69700 1 --2- 172.21.15.40:0/4121952447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b980704b0 0x7f7b98164960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009dc0 tx=0x7f7b9000b330 comp rx=0 tx=0).stop 2023-11-08T08:19:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.746+0000 7f7b9cb69700 1 -- 172.21.15.40:0/4121952447 shutdown_connections 2023-11-08T08:19:58.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.746+0000 7f7b9cb69700 1 --2- 172.21.15.40:0/4121952447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b980704b0 0x7f7b98164960 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.746+0000 7f7b9cb69700 1 -- 172.21.15.40:0/4121952447 >> 172.21.15.40:0/4121952447 conn(0x7f7b980b0f80 msgr2=0x7f7b980ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:58.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.746+0000 7f7b9cb69700 1 -- 172.21.15.40:0/4121952447 shutdown_connections 2023-11-08T08:19:58.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.746+0000 7f7b9cb69700 1 -- 172.21.15.40:0/4121952447 wait complete. 2023-11-08T08:19:58.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.746+0000 7f7b9cb69700 1 Processor -- start 2023-11-08T08:19:58.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.746+0000 7f7b9cb69700 1 -- start start 2023-11-08T08:19:58.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b9cb69700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b98070080 0x7f7b9814ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:58.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b9cb69700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b980704b0 0x7f7b9814f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:58.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b9cb69700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b98160710 0x7f7b980c2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:58.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b9cb69700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9814f230 con 0x7f7b980704b0 2023-11-08T08:19:58.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b9cb69700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9815e800 con 0x7f7b98160710 2023-11-08T08:19:58.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b9cb69700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980c2d40 con 0x7f7b98070080 2023-11-08T08:19:58.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b96ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b980704b0 0x7f7b9814f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:58.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b96ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b980704b0 0x7f7b9814f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35970/0 (socket says 172.21.15.40:35970) 2023-11-08T08:19:58.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b96ffd700 1 -- 172.21.15.40:0/974873377 learned_addr learned my addr 172.21.15.40:0/974873377 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:58.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b977fe700 1 --2- 172.21.15.40:0/974873377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b98070080 0x7f7b9814ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:58.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b97fff700 1 --2- 172.21.15.40:0/974873377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b98160710 0x7f7b980c2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:58.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b96ffd700 1 -- 172.21.15.40:0/974873377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b98070080 msgr2=0x7f7b9814ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:58.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b96ffd700 1 --2- 172.21.15.40:0/974873377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b98070080 0x7f7b9814ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b96ffd700 1 -- 172.21.15.40:0/974873377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b98160710 msgr2=0x7f7b980c2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:58.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b96ffd700 1 --2- 172.21.15.40:0/974873377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b98160710 0x7f7b980c2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b96ffd700 1 -- 172.21.15.40:0/974873377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b90009a70 con 0x7f7b980704b0 2023-11-08T08:19:58.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b96ffd700 1 --2- 172.21.15.40:0/974873377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b980704b0 0x7f7b9814f6d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00c730 tx=0x7f7b8c00caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:58.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b7ffff700 1 -- 172.21.15.40:0/974873377 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0052d0 con 0x7f7b980704b0 2023-11-08T08:19:58.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b7ffff700 1 -- 172.21.15.40:0/974873377 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8c005430 con 0x7f7b980704b0 2023-11-08T08:19:58.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b7ffff700 1 -- 172.21.15.40:0/974873377 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0055a0 con 0x7f7b980704b0 2023-11-08T08:19:58.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b9cb69700 1 -- 172.21.15.40:0/974873377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9806b6b0 con 0x7f7b980704b0 2023-11-08T08:19:58.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.750+0000 7f7b9cb69700 1 -- 172.21.15.40:0/974873377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9806b3d0 con 0x7f7b980704b0 2023-11-08T08:19:58.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.754+0000 7f7b9cb69700 1 -- 172.21.15.40:0/974873377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b980ad8f0 con 0x7f7b980704b0 2023-11-08T08:19:58.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.754+0000 7f7b7ffff700 1 -- 172.21.15.40:0/974873377 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7b8c019030 con 0x7f7b980704b0 2023-11-08T08:19:58.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.754+0000 7f7b7ffff700 1 --2- 172.21.15.40:0/974873377 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7b84064ee0 0x7f7b84067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:58.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.754+0000 7f7b7ffff700 1 -- 172.21.15.40:0/974873377 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7b8c081610 con 0x7f7b980704b0 2023-11-08T08:19:58.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.758+0000 7f7b977fe700 1 --2- 172.21.15.40:0/974873377 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7b84064ee0 0x7f7b84067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:58.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.758+0000 7f7b977fe700 1 --2- 172.21.15.40:0/974873377 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7b84064ee0 0x7f7b84067390 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7b880011e0 tx=0x7f7b88009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:58.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.762+0000 7f7b7ffff700 1 -- 172.21.15.40:0/974873377 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7b8c04f480 con 0x7f7b980704b0 2023-11-08T08:19:58.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.942+0000 7f7b9cb69700 1 -- 172.21.15.40:0/974873377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b98073cb0 con 0x7f7b980704b0 2023-11-08T08:19:58.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.946+0000 7f7b7ffff700 1 -- 172.21.15.40:0/974873377 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19748 (secure 0 0 0) 0x7f7b8c052aa0 con 0x7f7b980704b0 2023-11-08T08:19:58.949 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:19:58.949 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":30,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:45.885827+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:19:58.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.946+0000 7f7b9cb69700 1 -- 172.21.15.40:0/974873377 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7b84064ee0 msgr2=0x7f7b84067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:58.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.946+0000 7f7b9cb69700 1 --2- 172.21.15.40:0/974873377 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7b84064ee0 0x7f7b84067390 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7b880011e0 tx=0x7f7b88009040 comp rx=0 tx=0).stop 2023-11-08T08:19:58.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.946+0000 7f7b9cb69700 1 -- 172.21.15.40:0/974873377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b980704b0 msgr2=0x7f7b9814f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:58.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.946+0000 7f7b9cb69700 1 --2- 172.21.15.40:0/974873377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b980704b0 0x7f7b9814f6d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00c730 tx=0x7f7b8c00caf0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.946+0000 7f7b9cb69700 1 -- 172.21.15.40:0/974873377 shutdown_connections 2023-11-08T08:19:58.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.946+0000 7f7b9cb69700 1 --2- 172.21.15.40:0/974873377 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7b84064ee0 0x7f7b84067390 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.946+0000 7f7b9cb69700 1 --2- 172.21.15.40:0/974873377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b98160710 0x7f7b980c2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.946+0000 7f7b9cb69700 1 --2- 172.21.15.40:0/974873377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b980704b0 0x7f7b9814f6d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.946+0000 7f7b9cb69700 1 --2- 172.21.15.40:0/974873377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b98070080 0x7f7b9814ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:58.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.946+0000 7f7b9cb69700 1 -- 172.21.15.40:0/974873377 >> 172.21.15.40:0/974873377 conn(0x7f7b980b0f80 msgr2=0x7f7b98069bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:58.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.950+0000 7f7b9cb69700 1 -- 172.21.15.40:0/974873377 shutdown_connections 2023-11-08T08:19:58.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:58.950+0000 7f7b9cb69700 1 -- 172.21.15.40:0/974873377 wait complete. 2023-11-08T08:19:58.975 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T08:19:02.627691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T08:19:03.473821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T08:19:06.539942+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T08:19:58.976 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:19:59.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.162+0000 7f41dc95d700 1 Processor -- start 2023-11-08T08:19:59.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.162+0000 7f41dc95d700 1 -- start start 2023-11-08T08:19:59.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.162+0000 7f41dc95d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41d8070620 0x7f41d80b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:59.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.162+0000 7f41dc95d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f41d8069bf0 0x7f41d806a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:59.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.162+0000 7f41dc95d700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d815c520 con 0x7f41d815c150 2023-11-08T08:19:59.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.162+0000 7f41dc95d700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41d80b4240 con 0x7f41d8069bf0 2023-11-08T08:19:59.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.162+0000 7f41dc95d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d806cc50 con 0x7f41d8070620 2023-11-08T08:19:59.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.162+0000 7f41d7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f41d8069bf0 0x7f41d806a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:59.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.162+0000 7f41d6ffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41d8070620 0x7f41d80b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:59.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.162+0000 7f41d7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f41d8069bf0 0x7f41d806a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35978/0 (socket says 172.21.15.40:35978) 2023-11-08T08:19:59.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.162+0000 7f41d7fff700 1 -- 172.21.15.40:0/657889111 learned_addr learned my addr 172.21.15.40:0/657889111 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:59.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.162+0000 7f41d77fe700 1 --1- 172.21.15.40:0/657889111 >> v1:172.21.15.169:6789/0 conn(0x7f41d815c150 0x7f41d806ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:43766/0 (socket says 172.21.15.40:43766) 2023-11-08T08:19:59.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.166+0000 7f41d7fff700 1 -- 172.21.15.40:0/657889111 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41d8070620 msgr2=0x7f41d80b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:19:59.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.166+0000 7f41d7fff700 1 --2- 172.21.15.40:0/657889111 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41d8070620 0x7f41d80b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:59.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.166+0000 7f41d7fff700 1 -- 172.21.15.40:0/657889111 >> v1:172.21.15.169:6789/0 conn(0x7f41d815c150 legacy=0x7f41d806ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:59.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.166+0000 7f41d7fff700 1 -- 172.21.15.40:0/657889111 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d80d2450 con 0x7f41d8069bf0 2023-11-08T08:19:59.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.166+0000 7f41d7fff700 1 --2- 172.21.15.40:0/657889111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d8069bf0 0x7f41d806a050 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009e50 tx=0x7f41d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15891ca96388dbb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:59.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.166+0000 7f41d67fc700 1 -- 172.21.15.40:0/657889111 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d000e070 con 0x7f41d8069bf0 2023-11-08T08:19:59.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.166+0000 7f41d67fc700 1 -- 172.21.15.40:0/657889111 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d000b910 con 0x7f41d8069bf0 2023-11-08T08:19:59.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.166+0000 7f41d67fc700 1 -- 172.21.15.40:0/657889111 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d000bcc0 con 0x7f41d8069bf0 2023-11-08T08:19:59.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.166+0000 7f41dc95d700 1 -- 172.21.15.40:0/657889111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d8069bf0 msgr2=0x7f41d806a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:59.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.166+0000 7f41dc95d700 1 --2- 172.21.15.40:0/657889111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d8069bf0 0x7f41d806a050 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009e50 tx=0x7f41d000b330 comp rx=0 tx=0).stop 2023-11-08T08:19:59.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.166+0000 7f41dc95d700 1 -- 172.21.15.40:0/657889111 shutdown_connections 2023-11-08T08:19:59.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.166+0000 7f41dc95d700 1 --2- 172.21.15.40:0/657889111 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41d8070620 0x7f41d80b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:59.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.166+0000 7f41dc95d700 1 --2- 172.21.15.40:0/657889111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d8069bf0 0x7f41d806a050 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:59.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.166+0000 7f41dc95d700 1 -- 172.21.15.40:0/657889111 >> 172.21.15.40:0/657889111 conn(0x7f41d80d3680 msgr2=0x7f41d80cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:59.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.166+0000 7f41dc95d700 1 -- 172.21.15.40:0/657889111 shutdown_connections 2023-11-08T08:19:59.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.166+0000 7f41dc95d700 1 -- 172.21.15.40:0/657889111 wait complete. 2023-11-08T08:19:59.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.170+0000 7f41dc95d700 1 Processor -- start 2023-11-08T08:19:59.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.170+0000 7f41dc95d700 1 -- start start 2023-11-08T08:19:59.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.170+0000 7f41dc95d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41d8069bf0 0x7f41d8158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:59.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.170+0000 7f41dc95d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d8070620 0x7f41d8146230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:59.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.170+0000 7f41dc95d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41d815c150 0x7f41d81f1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:59.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.170+0000 7f41dc95d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d80cd040 con 0x7f41d8070620 2023-11-08T08:19:59.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.170+0000 7f41dc95d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d80cd2b0 con 0x7f41d8069bf0 2023-11-08T08:19:59.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.170+0000 7f41dc95d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d80b7450 con 0x7f41d815c150 2023-11-08T08:19:59.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.170+0000 7f41d6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d8070620 0x7f41d8146230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:59.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.170+0000 7f41d6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d8070620 0x7f41d8146230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35984/0 (socket says 172.21.15.40:35984) 2023-11-08T08:19:59.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.170+0000 7f41d6ffd700 1 -- 172.21.15.40:0/2377875290 learned_addr learned my addr 172.21.15.40:0/2377875290 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:59.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.174+0000 7f41d77fe700 1 --2- 172.21.15.40:0/2377875290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41d8069bf0 0x7f41d8158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:59.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.174+0000 7f41d6ffd700 1 -- 172.21.15.40:0/2377875290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41d815c150 msgr2=0x7f41d81f1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:59.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.174+0000 7f41d7fff700 1 --2- 172.21.15.40:0/2377875290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41d815c150 0x7f41d81f1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:59.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.174+0000 7f41d6ffd700 1 --2- 172.21.15.40:0/2377875290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41d815c150 0x7f41d81f1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:59.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.174+0000 7f41d6ffd700 1 -- 172.21.15.40:0/2377875290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41d8069bf0 msgr2=0x7f41d8158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:59.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.174+0000 7f41d6ffd700 1 --2- 172.21.15.40:0/2377875290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41d8069bf0 0x7f41d8158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:59.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.174+0000 7f41d6ffd700 1 -- 172.21.15.40:0/2377875290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d0009b30 con 0x7f41d8070620 2023-11-08T08:19:59.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.174+0000 7f41d6ffd700 1 --2- 172.21.15.40:0/2377875290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d8070620 0x7f41d8146230 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00e980 tx=0x7f41cc00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:59.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.174+0000 7f41bffff700 1 -- 172.21.15.40:0/2377875290 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc009710 con 0x7f41d8070620 2023-11-08T08:19:59.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.174+0000 7f41bffff700 1 -- 172.21.15.40:0/2377875290 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41cc00c850 con 0x7f41d8070620 2023-11-08T08:19:59.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.174+0000 7f41bffff700 1 -- 172.21.15.40:0/2377875290 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc00cc00 con 0x7f41d8070620 2023-11-08T08:19:59.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.174+0000 7f41dc95d700 1 -- 172.21.15.40:0/2377875290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d80d05e0 con 0x7f41d8070620 2023-11-08T08:19:59.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.174+0000 7f41dc95d700 1 -- 172.21.15.40:0/2377875290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d80d2730 con 0x7f41d8070620 2023-11-08T08:19:59.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.174+0000 7f41dc95d700 1 -- 172.21.15.40:0/2377875290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d80d2140 con 0x7f41d8070620 2023-11-08T08:19:59.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.178+0000 7f41bffff700 1 -- 172.21.15.40:0/2377875290 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f41cc023070 con 0x7f41d8070620 2023-11-08T08:19:59.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.178+0000 7f41bffff700 1 --2- 172.21.15.40:0/2377875290 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f41c4064ee0 0x7f41c4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:59.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.178+0000 7f41bffff700 1 -- 172.21.15.40:0/2377875290 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f41cc082590 con 0x7f41d8070620 2023-11-08T08:19:59.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.178+0000 7f41d77fe700 1 --2- 172.21.15.40:0/2377875290 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f41c4064ee0 0x7f41c4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:59.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.182+0000 7f41d77fe700 1 --2- 172.21.15.40:0/2377875290 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f41c4064ee0 0x7f41c4067390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f41c8000f60 tx=0x7f41c8007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:59.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.186+0000 7f41bffff700 1 -- 172.21.15.40:0/2377875290 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f41cc050460 con 0x7f41d8070620 2023-11-08T08:19:59.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.374+0000 7f41dc95d700 1 -- 172.21.15.40:0/2377875290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f41d80b7610 con 0x7f41d8070620 2023-11-08T08:19:59.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.374+0000 7f41bffff700 1 -- 172.21.15.40:0/2377875290 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f41cc0210a0 con 0x7f41d8070620 2023-11-08T08:19:59.379 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:19:59.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.378+0000 7f41dc95d700 1 -- 172.21.15.40:0/2377875290 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f41c4064ee0 msgr2=0x7f41c4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:59.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.378+0000 7f41dc95d700 1 --2- 172.21.15.40:0/2377875290 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f41c4064ee0 0x7f41c4067390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f41c8000f60 tx=0x7f41c8007d50 comp rx=0 tx=0).stop 2023-11-08T08:19:59.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.378+0000 7f41dc95d700 1 -- 172.21.15.40:0/2377875290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d8070620 msgr2=0x7f41d8146230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:59.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.378+0000 7f41dc95d700 1 --2- 172.21.15.40:0/2377875290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d8070620 0x7f41d8146230 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00e980 tx=0x7f41cc00ed40 comp rx=0 tx=0).stop 2023-11-08T08:19:59.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.378+0000 7f41dc95d700 1 -- 172.21.15.40:0/2377875290 shutdown_connections 2023-11-08T08:19:59.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.378+0000 7f41dc95d700 1 --2- 172.21.15.40:0/2377875290 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f41c4064ee0 0x7f41c4067390 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:59.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.378+0000 7f41dc95d700 1 --2- 172.21.15.40:0/2377875290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41d815c150 0x7f41d81f1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:59.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.378+0000 7f41dc95d700 1 --2- 172.21.15.40:0/2377875290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41d8070620 0x7f41d8146230 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:59.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.378+0000 7f41dc95d700 1 --2- 172.21.15.40:0/2377875290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41d8069bf0 0x7f41d8158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:59.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.378+0000 7f41dc95d700 1 -- 172.21.15.40:0/2377875290 >> 172.21.15.40:0/2377875290 conn(0x7f41d80d3680 msgr2=0x7f41d80b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:59.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.378+0000 7f41dc95d700 1 -- 172.21.15.40:0/2377875290 shutdown_connections 2023-11-08T08:19:59.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.378+0000 7f41dc95d700 1 -- 172.21.15.40:0/2377875290 wait complete. 2023-11-08T08:19:59.405 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:19:59.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.586+0000 7fa187fff700 1 Processor -- start 2023-11-08T08:19:59.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.586+0000 7fa187fff700 1 -- start start 2023-11-08T08:19:59.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.586+0000 7fa187fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa188070620 0x7fa1880b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:59.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.586+0000 7fa187fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18815c520 con 0x7fa188069bf0 2023-11-08T08:19:59.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.586+0000 7fa187fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18806cc50 con 0x7fa18815c150 2023-11-08T08:19:59.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.586+0000 7fa187fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1880b4240 con 0x7fa188070620 2023-11-08T08:19:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.586+0000 7fa1867fc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa188070620 0x7fa1880b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.586+0000 7fa1877fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa188069bf0 0x7fa18806a050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41864/0 (socket says 172.21.15.40:41864) 2023-11-08T08:19:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.586+0000 7fa186ffd700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fa18815c150 0x7fa18806ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:43778/0 (socket says 172.21.15.40:43778) 2023-11-08T08:19:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.586+0000 7fa1877fe700 1 -- 172.21.15.40:0/1305684279 learned_addr learned my addr 172.21.15.40:0/1305684279 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:19:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.590+0000 7fa1867fc700 1 -- 172.21.15.40:0/1305684279 >> v1:172.21.15.169:6789/0 conn(0x7fa18815c150 legacy=0x7fa18806ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:59.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.590+0000 7fa185ffb700 1 -- 172.21.15.40:0/1305684279 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487899112 0 0) 0x7fa18815c520 con 0x7fa188069bf0 2023-11-08T08:19:59.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.590+0000 7fa1867fc700 1 -- 172.21.15.40:0/1305684279 >> v1:172.21.15.40:6789/0 conn(0x7fa188069bf0 legacy=0x7fa18806a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:59.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.590+0000 7fa1867fc700 1 -- 172.21.15.40:0/1305684279 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1880d2450 con 0x7fa188070620 2023-11-08T08:19:59.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.590+0000 7fa1867fc700 1 --2- 172.21.15.40:0/1305684279 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa188070620 0x7fa1880b3d00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0097f0 tx=0x7fa17c009b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59b7e2d9083e005a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:59.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.590+0000 7fa185ffb700 1 -- 172.21.15.40:0/1305684279 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c004230 con 0x7fa188070620 2023-11-08T08:19:59.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.590+0000 7fa185ffb700 1 -- 172.21.15.40:0/1305684279 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c004390 con 0x7fa188070620 2023-11-08T08:19:59.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.590+0000 7fa185ffb700 1 -- 172.21.15.40:0/1305684279 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c0036c0 con 0x7fa188070620 2023-11-08T08:19:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.590+0000 7fa187fff700 1 -- 172.21.15.40:0/1305684279 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa188070620 msgr2=0x7fa1880b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.590+0000 7fa187fff700 1 --2- 172.21.15.40:0/1305684279 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa188070620 0x7fa1880b3d00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0097f0 tx=0x7fa17c009b00 comp rx=0 tx=0).stop 2023-11-08T08:19:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.590+0000 7fa187fff700 1 -- 172.21.15.40:0/1305684279 shutdown_connections 2023-11-08T08:19:59.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.590+0000 7fa187fff700 1 --2- 172.21.15.40:0/1305684279 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa188070620 0x7fa1880b3d00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:59.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.590+0000 7fa187fff700 1 -- 172.21.15.40:0/1305684279 >> 172.21.15.40:0/1305684279 conn(0x7fa1880d3680 msgr2=0x7fa1880cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:59.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.590+0000 7fa187fff700 1 -- 172.21.15.40:0/1305684279 shutdown_connections 2023-11-08T08:19:59.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.590+0000 7fa187fff700 1 -- 172.21.15.40:0/1305684279 wait complete. 2023-11-08T08:19:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.594+0000 7fa187fff700 1 Processor -- start 2023-11-08T08:19:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.594+0000 7fa187fff700 1 -- start start 2023-11-08T08:19:59.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa187fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa188069bf0 0x7fa188158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:59.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa187fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa188070620 0x7fa1880cf230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:59.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa187fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa18815c150 0x7fa1880cf770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:59.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa187fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1880cfcb0 con 0x7fa188069bf0 2023-11-08T08:19:59.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa187fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1880cbae0 con 0x7fa188070620 2023-11-08T08:19:59.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa187fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1880ca760 con 0x7fa18815c150 2023-11-08T08:19:59.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa186ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa188069bf0 0x7fa188158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:59.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa186ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa188069bf0 0x7fa188158910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35996/0 (socket says 172.21.15.40:35996) 2023-11-08T08:19:59.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa186ffd700 1 -- 172.21.15.40:0/1710661159 learned_addr learned my addr 172.21.15.40:0/1710661159 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:19:59.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa1867fc700 1 --2- 172.21.15.40:0/1710661159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa188070620 0x7fa1880cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:59.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa1877fe700 1 --2- 172.21.15.40:0/1710661159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa18815c150 0x7fa1880cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:59.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa186ffd700 1 -- 172.21.15.40:0/1710661159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa18815c150 msgr2=0x7fa1880cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:59.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa186ffd700 1 --2- 172.21.15.40:0/1710661159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa18815c150 0x7fa1880cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:59.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa186ffd700 1 -- 172.21.15.40:0/1710661159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa188070620 msgr2=0x7fa1880cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:59.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa186ffd700 1 --2- 172.21.15.40:0/1710661159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa188070620 0x7fa1880cf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:59.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa186ffd700 1 -- 172.21.15.40:0/1710661159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c009450 con 0x7fa188069bf0 2023-11-08T08:19:59.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa1867fc700 1 --2- 172.21.15.40:0/1710661159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa188070620 0x7fa1880cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:59.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa1877fe700 1 --2- 172.21.15.40:0/1710661159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa18815c150 0x7fa1880cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:19:59.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa186ffd700 1 --2- 172.21.15.40:0/1710661159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa188069bf0 0x7fa188158910 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa17800c780 tx=0x7fa17800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:59.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa16f7fe700 1 -- 172.21.15.40:0/1710661159 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa178004ca0 con 0x7fa188069bf0 2023-11-08T08:19:59.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa16f7fe700 1 -- 172.21.15.40:0/1710661159 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa178004e00 con 0x7fa188069bf0 2023-11-08T08:19:59.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa187fff700 1 -- 172.21.15.40:0/1710661159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1880d05e0 con 0x7fa188069bf0 2023-11-08T08:19:59.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa16f7fe700 1 -- 172.21.15.40:0/1710661159 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa178007310 con 0x7fa188069bf0 2023-11-08T08:19:59.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.598+0000 7fa187fff700 1 -- 172.21.15.40:0/1710661159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1880d2730 con 0x7fa188069bf0 2023-11-08T08:19:59.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.602+0000 7fa187fff700 1 -- 172.21.15.40:0/1710661159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1880d2140 con 0x7fa188069bf0 2023-11-08T08:19:59.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.602+0000 7fa16f7fe700 1 -- 172.21.15.40:0/1710661159 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa17801a020 con 0x7fa188069bf0 2023-11-08T08:19:59.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.602+0000 7fa16f7fe700 1 --2- 172.21.15.40:0/1710661159 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa174064ee0 0x7fa174067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:19:59.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.602+0000 7fa16f7fe700 1 -- 172.21.15.40:0/1710661159 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa178081fb0 con 0x7fa188069bf0 2023-11-08T08:19:59.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.606+0000 7fa1867fc700 1 --2- 172.21.15.40:0/1710661159 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa174064ee0 0x7fa174067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:19:59.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.606+0000 7fa1867fc700 1 --2- 172.21.15.40:0/1710661159 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa174064ee0 0x7fa174067390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0097f0 tx=0x7fa17c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:19:59.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.606+0000 7fa16f7fe700 1 -- 172.21.15.40:0/1710661159 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa17804fd30 con 0x7fa188069bf0 2023-11-08T08:19:59.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.786+0000 7fa187fff700 1 -- 172.21.15.40:0/1710661159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa188160ab0 con 0x7fa188069bf0 2023-11-08T08:19:59.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.786+0000 7fa16f7fe700 1 -- 172.21.15.40:0/1710661159 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa178053350 con 0x7fa188069bf0 2023-11-08T08:19:59.791 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:19:59.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.790+0000 7fa187fff700 1 -- 172.21.15.40:0/1710661159 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa174064ee0 msgr2=0x7fa174067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:59.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.790+0000 7fa187fff700 1 --2- 172.21.15.40:0/1710661159 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa174064ee0 0x7fa174067390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0097f0 tx=0x7fa17c008030 comp rx=0 tx=0).stop 2023-11-08T08:19:59.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.790+0000 7fa187fff700 1 -- 172.21.15.40:0/1710661159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa188069bf0 msgr2=0x7fa188158910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:19:59.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.790+0000 7fa187fff700 1 --2- 172.21.15.40:0/1710661159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa188069bf0 0x7fa188158910 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa17800c780 tx=0x7fa17800cb40 comp rx=0 tx=0).stop 2023-11-08T08:19:59.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.790+0000 7fa187fff700 1 -- 172.21.15.40:0/1710661159 shutdown_connections 2023-11-08T08:19:59.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.790+0000 7fa187fff700 1 --2- 172.21.15.40:0/1710661159 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa174064ee0 0x7fa174067390 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:59.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.790+0000 7fa187fff700 1 --2- 172.21.15.40:0/1710661159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa18815c150 0x7fa1880cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:59.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.790+0000 7fa187fff700 1 --2- 172.21.15.40:0/1710661159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa188070620 0x7fa1880cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:59.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.790+0000 7fa187fff700 1 --2- 172.21.15.40:0/1710661159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa188069bf0 0x7fa188158910 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:19:59.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.790+0000 7fa187fff700 1 -- 172.21.15.40:0/1710661159 >> 172.21.15.40:0/1710661159 conn(0x7fa1880d3680 msgr2=0x7fa188155a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:19:59.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.790+0000 7fa187fff700 1 -- 172.21.15.40:0/1710661159 shutdown_connections 2023-11-08T08:19:59.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:19:59.790+0000 7fa187fff700 1 -- 172.21.15.40:0/1710661159 wait complete. 2023-11-08T08:19:59.817 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T08:20:00.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.018+0000 7f943a448700 1 Processor -- start 2023-11-08T08:20:00.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.018+0000 7f943a448700 1 -- start start 2023-11-08T08:20:00.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.022+0000 7f943a448700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94340c8c90 0x7f943415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:00.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.022+0000 7f943a448700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9434070600 con 0x7f9434070a90 2023-11-08T08:20:00.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.022+0000 7f943a448700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94340c9090 con 0x7f9434159f10 2023-11-08T08:20:00.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.022+0000 7f943a448700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94340afcc0 con 0x7f94340c8c90 2023-11-08T08:20:00.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.022+0000 7f9438c45700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9434070a90 0x7f943406be80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41880/0 (socket says 172.21.15.40:41880) 2023-11-08T08:20:00.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.022+0000 7f9438c45700 1 -- 172.21.15.40:0/1320445258 learned_addr learned my addr 172.21.15.40:0/1320445258 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:00.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.022+0000 7f9439c47700 1 --2- 172.21.15.40:0/1320445258 >> v2:172.21.15.40:3300/0 conn(0x7f94340c8c90 0x7f943415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:00.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.022+0000 7f942bfff700 1 -- 172.21.15.40:0/1320445258 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 362658219 0 0) 0x7f94340c9090 con 0x7f9434159f10 2023-11-08T08:20:00.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.022+0000 7f942bfff700 1 -- 172.21.15.40:0/1320445258 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94200035d0 con 0x7f9434159f10 2023-11-08T08:20:00.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.022+0000 7f942bfff700 1 -- 172.21.15.40:0/1320445258 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225678467 0 0) 0x7f9434070600 con 0x7f9434070a90 2023-11-08T08:20:00.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.022+0000 7f942bfff700 1 -- 172.21.15.40:0/1320445258 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94340c9090 con 0x7f9434070a90 2023-11-08T08:20:00.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.022+0000 7f9439c47700 1 -- 172.21.15.40:0/1320445258 >> v1:172.21.15.169:6789/0 conn(0x7f9434159f10 legacy=0x7f943415a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:00.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.022+0000 7f9439c47700 1 -- 172.21.15.40:0/1320445258 >> v1:172.21.15.40:6789/0 conn(0x7f9434070a90 legacy=0x7f943406be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:00.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.022+0000 7f9439c47700 1 -- 172.21.15.40:0/1320445258 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943406ffb0 con 0x7f94340c8c90 2023-11-08T08:20:00.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.026+0000 7f9439c47700 1 --2- 172.21.15.40:0/1320445258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94340c8c90 0x7f943415a9f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f942c009d90 tx=0x7f942c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9d900ea6462000 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:00.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.026+0000 7f942bfff700 1 -- 172.21.15.40:0/1320445258 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c00e070 con 0x7f94340c8c90 2023-11-08T08:20:00.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.026+0000 7f942bfff700 1 -- 172.21.15.40:0/1320445258 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942c00bb20 con 0x7f94340c8c90 2023-11-08T08:20:00.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.026+0000 7f942bfff700 1 -- 172.21.15.40:0/1320445258 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c012430 con 0x7f94340c8c90 2023-11-08T08:20:00.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.026+0000 7f943a448700 1 -- 172.21.15.40:0/1320445258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94340c8c90 msgr2=0x7f943415a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:00.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.026+0000 7f943a448700 1 --2- 172.21.15.40:0/1320445258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94340c8c90 0x7f943415a9f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f942c009d90 tx=0x7f942c00b330 comp rx=0 tx=0).stop 2023-11-08T08:20:00.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.026+0000 7f943a448700 1 -- 172.21.15.40:0/1320445258 shutdown_connections 2023-11-08T08:20:00.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.026+0000 7f943a448700 1 --2- 172.21.15.40:0/1320445258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94340c8c90 0x7f943415a9f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.026+0000 7f943a448700 1 -- 172.21.15.40:0/1320445258 >> 172.21.15.40:0/1320445258 conn(0x7f94340cf280 msgr2=0x7f94340cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:00.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.026+0000 7f943a448700 1 -- 172.21.15.40:0/1320445258 shutdown_connections 2023-11-08T08:20:00.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.026+0000 7f943a448700 1 -- 172.21.15.40:0/1320445258 wait complete. 2023-11-08T08:20:00.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.030+0000 7f943a448700 1 Processor -- start 2023-11-08T08:20:00.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.030+0000 7f943a448700 1 -- start start 2023-11-08T08:20:00.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.030+0000 7f943a448700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9434070a90 0x7f943415d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:00.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.030+0000 7f943a448700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94340c8c90 0x7f94341f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:00.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.030+0000 7f943a448700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9434159f10 0x7f943415a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:00.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.030+0000 7f943a448700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943415cc90 con 0x7f94340c8c90 2023-11-08T08:20:00.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.030+0000 7f943a448700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943415c080 con 0x7f9434159f10 2023-11-08T08:20:00.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.030+0000 7f943a448700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943415c360 con 0x7f9434070a90 2023-11-08T08:20:00.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.030+0000 7f9438c45700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94340c8c90 0x7f94341f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:00.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.030+0000 7f9438c45700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94340c8c90 0x7f94341f0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36018/0 (socket says 172.21.15.40:36018) 2023-11-08T08:20:00.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.030+0000 7f9438c45700 1 -- 172.21.15.40:0/995903924 learned_addr learned my addr 172.21.15.40:0/995903924 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:00.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.034+0000 7f9439c47700 1 --2- 172.21.15.40:0/995903924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9434159f10 0x7f943415a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:00.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.034+0000 7f9439446700 1 --2- 172.21.15.40:0/995903924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9434070a90 0x7f943415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:00.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.034+0000 7f9438c45700 1 -- 172.21.15.40:0/995903924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9434070a90 msgr2=0x7f943415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:00.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.034+0000 7f9438c45700 1 --2- 172.21.15.40:0/995903924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9434070a90 0x7f943415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.034+0000 7f9438c45700 1 -- 172.21.15.40:0/995903924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9434159f10 msgr2=0x7f943415a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:00.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.034+0000 7f9438c45700 1 --2- 172.21.15.40:0/995903924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9434159f10 0x7f943415a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.034+0000 7f9438c45700 1 -- 172.21.15.40:0/995903924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c009a70 con 0x7f94340c8c90 2023-11-08T08:20:00.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.034+0000 7f9439c47700 1 --2- 172.21.15.40:0/995903924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9434159f10 0x7f943415a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:00.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.034+0000 7f9438c45700 1 --2- 172.21.15.40:0/995903924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94340c8c90 0x7f94341f0eb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f942400c700 tx=0x7f942400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:00.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.034+0000 7f9429ffb700 1 -- 172.21.15.40:0/995903924 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424005d90 con 0x7f94340c8c90 2023-11-08T08:20:00.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.034+0000 7f9429ffb700 1 -- 172.21.15.40:0/995903924 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94240070d0 con 0x7f94340c8c90 2023-11-08T08:20:00.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.034+0000 7f943a448700 1 -- 172.21.15.40:0/995903924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94340affa0 con 0x7f94340c8c90 2023-11-08T08:20:00.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.034+0000 7f9429ffb700 1 -- 172.21.15.40:0/995903924 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424007480 con 0x7f94340c8c90 2023-11-08T08:20:00.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.034+0000 7f943a448700 1 -- 172.21.15.40:0/995903924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9434070290 con 0x7f94340c8c90 2023-11-08T08:20:00.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.034+0000 7f943a448700 1 -- 172.21.15.40:0/995903924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943406b300 con 0x7f94340c8c90 2023-11-08T08:20:00.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.038+0000 7f9429ffb700 1 -- 172.21.15.40:0/995903924 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f9424019030 con 0x7f94340c8c90 2023-11-08T08:20:00.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.038+0000 7f9429ffb700 1 --2- 172.21.15.40:0/995903924 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9418064ee0 0x7f9418067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:00.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.038+0000 7f9429ffb700 1 -- 172.21.15.40:0/995903924 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9424081580 con 0x7f94340c8c90 2023-11-08T08:20:00.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.038+0000 7f9439446700 1 --2- 172.21.15.40:0/995903924 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9418064ee0 0x7f9418067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:00.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.042+0000 7f9439446700 1 --2- 172.21.15.40:0/995903924 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9418064ee0 0x7f9418067390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9430007fd0 tx=0x7f9430009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:00.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.046+0000 7f9429ffb700 1 -- 172.21.15.40:0/995903924 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f942404f2a0 con 0x7f94340c8c90 2023-11-08T08:20:00.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.218+0000 7f943a448700 1 -- 172.21.15.40:0/995903924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f943406cf30 con 0x7f94340c8c90 2023-11-08T08:20:00.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.222+0000 7f9429ffb700 1 -- 172.21.15.40:0/995903924 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f94240528c0 con 0x7f94340c8c90 2023-11-08T08:20:00.225 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:20:00.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.226+0000 7f943a448700 1 -- 172.21.15.40:0/995903924 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9418064ee0 msgr2=0x7f9418067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:00.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.226+0000 7f943a448700 1 --2- 172.21.15.40:0/995903924 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9418064ee0 0x7f9418067390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9430007fd0 tx=0x7f9430009040 comp rx=0 tx=0).stop 2023-11-08T08:20:00.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.226+0000 7f943a448700 1 -- 172.21.15.40:0/995903924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94340c8c90 msgr2=0x7f94341f0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:00.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.226+0000 7f943a448700 1 --2- 172.21.15.40:0/995903924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94340c8c90 0x7f94341f0eb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f942400c700 tx=0x7f942400ca10 comp rx=0 tx=0).stop 2023-11-08T08:20:00.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.226+0000 7f943a448700 1 -- 172.21.15.40:0/995903924 shutdown_connections 2023-11-08T08:20:00.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.226+0000 7f943a448700 1 --2- 172.21.15.40:0/995903924 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9418064ee0 0x7f9418067390 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.226+0000 7f943a448700 1 --2- 172.21.15.40:0/995903924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9434159f10 0x7f943415a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.226+0000 7f943a448700 1 --2- 172.21.15.40:0/995903924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94340c8c90 0x7f94341f0eb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.226+0000 7f943a448700 1 --2- 172.21.15.40:0/995903924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9434070a90 0x7f943415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.226+0000 7f943a448700 1 -- 172.21.15.40:0/995903924 >> 172.21.15.40:0/995903924 conn(0x7f94340cf280 msgr2=0x7f9434155af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:00.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.226+0000 7f943a448700 1 -- 172.21.15.40:0/995903924 shutdown_connections 2023-11-08T08:20:00.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.226+0000 7f943a448700 1 -- 172.21.15.40:0/995903924 wait complete. 2023-11-08T08:20:00.254 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T08:20:00.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.438+0000 7f7e665a8700 1 Processor -- start 2023-11-08T08:20:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.438+0000 7f7e665a8700 1 -- start start 2023-11-08T08:20:00.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.442+0000 7f7e665a8700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7e6015c410 0x7f7e60160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.442+0000 7f7e665a8700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e600ca740 con 0x7f7e600cf2a0 2023-11-08T08:20:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.442+0000 7f7e665a8700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e60161020 con 0x7f7e600b1240 2023-11-08T08:20:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.442+0000 7f7e665a8700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e600ca9e0 con 0x7f7e6015c410 2023-11-08T08:20:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.442+0000 7f7e64da5700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7e600cf2a0 0x7f7e60069c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41896/0 (socket says 172.21.15.40:41896) 2023-11-08T08:20:00.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.442+0000 7f7e64da5700 1 -- 172.21.15.40:0/1230564701 learned_addr learned my addr 172.21.15.40:0/1230564701 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:00.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.442+0000 7f7e655a6700 1 --2- 172.21.15.40:0/1230564701 >> v2:172.21.15.169:3300/0 conn(0x7f7e6015c410 0x7f7e60160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:00.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.442+0000 7f7e57fff700 1 -- 172.21.15.40:0/1230564701 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2548179843 0 0) 0x7f7e600ca740 con 0x7f7e600cf2a0 2023-11-08T08:20:00.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.442+0000 7f7e57fff700 1 -- 172.21.15.40:0/1230564701 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e4c0035d0 con 0x7f7e600cf2a0 2023-11-08T08:20:00.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.442+0000 7f7e57fff700 1 -- 172.21.15.40:0/1230564701 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141742781 0 0) 0x7f7e60161020 con 0x7f7e600b1240 2023-11-08T08:20:00.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e57fff700 1 -- 172.21.15.40:0/1230564701 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e600ca740 con 0x7f7e600b1240 2023-11-08T08:20:00.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e57fff700 1 -- 172.21.15.40:0/1230564701 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4277381635 0 0) 0x7f7e4c0035d0 con 0x7f7e600cf2a0 2023-11-08T08:20:00.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e57fff700 1 -- 172.21.15.40:0/1230564701 >> v2:172.21.15.169:3300/0 conn(0x7f7e6015c410 msgr2=0x7f7e60160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:00.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e57fff700 1 --2- 172.21.15.40:0/1230564701 >> v2:172.21.15.169:3300/0 conn(0x7f7e6015c410 0x7f7e60160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e57fff700 1 -- 172.21.15.40:0/1230564701 >> v1:172.21.15.169:6789/0 conn(0x7f7e600b1240 legacy=0x7f7e600b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:00.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e57fff700 1 -- 172.21.15.40:0/1230564701 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e600cacc0 con 0x7f7e600cf2a0 2023-11-08T08:20:00.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e57fff700 1 -- 172.21.15.40:0/1230564701 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f7e50001670 con 0x7f7e600cf2a0 2023-11-08T08:20:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e57fff700 1 -- 172.21.15.40:0/1230564701 >> v1:172.21.15.40:6789/0 conn(0x7f7e600cf2a0 legacy=0x7f7e60069c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e57fff700 1 --2- 172.21.15.40:0/1230564701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e4c003bb0 0x7f7e4c003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e57fff700 1 --2- 172.21.15.40:0/1230564701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e4c004870 0x7f7e4c004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e57fff700 1 --2- 172.21.15.40:0/1230564701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e4c005220 0x7f7e4c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e57fff700 1 -- 172.21.15.40:0/1230564701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e600ca9e0 con 0x7f7e4c005220 2023-11-08T08:20:00.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e57fff700 1 -- 172.21.15.40:0/1230564701 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c00c080 con 0x7f7e4c004870 2023-11-08T08:20:00.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e57fff700 1 -- 172.21.15.40:0/1230564701 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c00c240 con 0x7f7e4c003bb0 2023-11-08T08:20:00.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e65da7700 1 --2- 172.21.15.40:0/1230564701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e4c005220 0x7f7e4c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:00.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e64da5700 1 --2- 172.21.15.40:0/1230564701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e4c004870 0x7f7e4c004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:00.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e655a6700 1 --2- 172.21.15.40:0/1230564701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e4c003bb0 0x7f7e4c003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:00.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e65da7700 1 -- 172.21.15.40:0/1230564701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e4c003bb0 msgr2=0x7f7e4c003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:00.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e65da7700 1 --2- 172.21.15.40:0/1230564701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e4c003bb0 0x7f7e4c003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e65da7700 1 -- 172.21.15.40:0/1230564701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e4c004870 msgr2=0x7f7e4c004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:00.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e65da7700 1 --2- 172.21.15.40:0/1230564701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e4c004870 0x7f7e4c004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e65da7700 1 -- 172.21.15.40:0/1230564701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e4c0035d0 con 0x7f7e4c005220 2023-11-08T08:20:00.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e64da5700 1 --2- 172.21.15.40:0/1230564701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e4c004870 0x7f7e4c004ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:00.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e65da7700 1 -- 172.21.15.40:0/1230564701 reap_dead start 2023-11-08T08:20:00.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e65da7700 1 --2- 172.21.15.40:0/1230564701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e4c005220 0x7f7e4c00ba30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800c2f0 tx=0x7f7e4c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2d2f8782636b7b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:00.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e57fff700 1 -- 172.21.15.40:0/1230564701 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58001f30 con 0x7f7e4c005220 2023-11-08T08:20:00.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e57fff700 1 -- 172.21.15.40:0/1230564701 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e58003a20 con 0x7f7e4c005220 2023-11-08T08:20:00.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e57fff700 1 -- 172.21.15.40:0/1230564701 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58003dd0 con 0x7f7e4c005220 2023-11-08T08:20:00.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e665a8700 1 -- 172.21.15.40:0/1230564701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e4c005220 msgr2=0x7f7e4c00ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:00.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e665a8700 1 --2- 172.21.15.40:0/1230564701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e4c005220 0x7f7e4c00ba30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800c2f0 tx=0x7f7e4c0058a0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e665a8700 1 -- 172.21.15.40:0/1230564701 shutdown_connections 2023-11-08T08:20:00.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e665a8700 1 --2- 172.21.15.40:0/1230564701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e4c005220 0x7f7e4c00ba30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e665a8700 1 -- 172.21.15.40:0/1230564701 >> 172.21.15.40:0/1230564701 conn(0x7f7e600cc240 msgr2=0x7f7e600cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:00.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e665a8700 1 -- 172.21.15.40:0/1230564701 shutdown_connections 2023-11-08T08:20:00.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.446+0000 7f7e665a8700 1 -- 172.21.15.40:0/1230564701 wait complete. 2023-11-08T08:20:00.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.450+0000 7f7e665a8700 1 Processor -- start 2023-11-08T08:20:00.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.450+0000 7f7e665a8700 1 -- start start 2023-11-08T08:20:00.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.450+0000 7f7e665a8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e4c005220 0x7f7e600b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:00.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.450+0000 7f7e665a8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e60153a60 0x7f7e600b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:00.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.450+0000 7f7e665a8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e60153eb0 0x7f7e600b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:00.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.450+0000 7f7e665a8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e600c6da0 con 0x7f7e60153eb0 2023-11-08T08:20:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.450+0000 7f7e665a8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e600b4be0 con 0x7f7e4c005220 2023-11-08T08:20:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.450+0000 7f7e665a8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e600b4620 con 0x7f7e60153a60 2023-11-08T08:20:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.450+0000 7f7e65da7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e60153eb0 0x7f7e600b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.450+0000 7f7e65da7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e60153eb0 0x7f7e600b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36030/0 (socket says 172.21.15.40:36030) 2023-11-08T08:20:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.450+0000 7f7e65da7700 1 -- 172.21.15.40:0/245078911 learned_addr learned my addr 172.21.15.40:0/245078911 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.454+0000 7f7e64da5700 1 --2- 172.21.15.40:0/245078911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e60153a60 0x7f7e600b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:00.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.454+0000 7f7e655a6700 1 --2- 172.21.15.40:0/245078911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e4c005220 0x7f7e600b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:00.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.454+0000 7f7e65da7700 1 -- 172.21.15.40:0/245078911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e60153a60 msgr2=0x7f7e600b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:00.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.454+0000 7f7e65da7700 1 --2- 172.21.15.40:0/245078911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e60153a60 0x7f7e600b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.454+0000 7f7e65da7700 1 -- 172.21.15.40:0/245078911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e4c005220 msgr2=0x7f7e600b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:00.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.454+0000 7f7e65da7700 1 --2- 172.21.15.40:0/245078911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e4c005220 0x7f7e600b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.454+0000 7f7e65da7700 1 -- 172.21.15.40:0/245078911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e600cde70 con 0x7f7e60153eb0 2023-11-08T08:20:00.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.454+0000 7f7e65da7700 1 --2- 172.21.15.40:0/245078911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e60153eb0 0x7f7e600b3d00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7e600cbb30 tx=0x7f7e58001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:00.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.454+0000 7f7e55ffb700 1 -- 172.21.15.40:0/245078911 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e580050d0 con 0x7f7e60153eb0 2023-11-08T08:20:00.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.454+0000 7f7e55ffb700 1 -- 172.21.15.40:0/245078911 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e58005230 con 0x7f7e60153eb0 2023-11-08T08:20:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.454+0000 7f7e665a8700 1 -- 172.21.15.40:0/245078911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e600cdb90 con 0x7f7e60153eb0 2023-11-08T08:20:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.454+0000 7f7e55ffb700 1 -- 172.21.15.40:0/245078911 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e580055c0 con 0x7f7e60153eb0 2023-11-08T08:20:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.454+0000 7f7e665a8700 1 -- 172.21.15.40:0/245078911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6015c7e0 con 0x7f7e60153eb0 2023-11-08T08:20:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.454+0000 7f7e665a8700 1 -- 172.21.15.40:0/245078911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e600c6ee0 con 0x7f7e60153eb0 2023-11-08T08:20:00.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.458+0000 7f7e55ffb700 1 -- 172.21.15.40:0/245078911 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7e58010050 con 0x7f7e60153eb0 2023-11-08T08:20:00.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.458+0000 7f7e55ffb700 1 --2- 172.21.15.40:0/245078911 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e44064ee0 0x7f7e44067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:00.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.458+0000 7f7e55ffb700 1 -- 172.21.15.40:0/245078911 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7e580837f0 con 0x7f7e60153eb0 2023-11-08T08:20:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.458+0000 7f7e655a6700 1 --2- 172.21.15.40:0/245078911 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e44064ee0 0x7f7e44067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.462+0000 7f7e655a6700 1 --2- 172.21.15.40:0/245078911 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e44064ee0 0x7f7e44067390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0045f0 tx=0x7f7e5c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.466+0000 7f7e55ffb700 1 -- 172.21.15.40:0/245078911 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7e600c6ee0 con 0x7f7e60153eb0 2023-11-08T08:20:00.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.654+0000 7f7e665a8700 1 -- 172.21.15.40:0/245078911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7e600c6ee0 con 0x7f7e60153eb0 2023-11-08T08:20:00.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.654+0000 7f7e55ffb700 1 -- 172.21.15.40:0/245078911 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7e600c6ee0 con 0x7f7e60153eb0 2023-11-08T08:20:00.658 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-11-08T08:20:00.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.658+0000 7f7e665a8700 1 -- 172.21.15.40:0/245078911 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e44064ee0 msgr2=0x7f7e44067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:00.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.658+0000 7f7e665a8700 1 --2- 172.21.15.40:0/245078911 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e44064ee0 0x7f7e44067390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0045f0 tx=0x7f7e5c008040 comp rx=0 tx=0).stop 2023-11-08T08:20:00.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.658+0000 7f7e665a8700 1 -- 172.21.15.40:0/245078911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e60153eb0 msgr2=0x7f7e600b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:00.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.658+0000 7f7e665a8700 1 --2- 172.21.15.40:0/245078911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e60153eb0 0x7f7e600b3d00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7e600cbb30 tx=0x7f7e58001650 comp rx=0 tx=0).stop 2023-11-08T08:20:00.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.658+0000 7f7e665a8700 1 -- 172.21.15.40:0/245078911 shutdown_connections 2023-11-08T08:20:00.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.658+0000 7f7e665a8700 1 --2- 172.21.15.40:0/245078911 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e44064ee0 0x7f7e44067390 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.658+0000 7f7e665a8700 1 --2- 172.21.15.40:0/245078911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e60153eb0 0x7f7e600b3d00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.658+0000 7f7e665a8700 1 --2- 172.21.15.40:0/245078911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e60153a60 0x7f7e600b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.658+0000 7f7e665a8700 1 --2- 172.21.15.40:0/245078911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e4c005220 0x7f7e600b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.658+0000 7f7e665a8700 1 -- 172.21.15.40:0/245078911 >> 172.21.15.40:0/245078911 conn(0x7f7e600cc240 msgr2=0x7f7e6006b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:00.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.658+0000 7f7e665a8700 1 -- 172.21.15.40:0/245078911 shutdown_connections 2023-11-08T08:20:00.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.658+0000 7f7e665a8700 1 -- 172.21.15.40:0/245078911 wait complete. 2023-11-08T08:20:00.684 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T08:20:00.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.870+0000 7f7e15fdc700 1 Processor -- start 2023-11-08T08:20:00.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.870+0000 7f7e15fdc700 1 -- start start 2023-11-08T08:20:00.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.874+0000 7f7e15fdc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7e1015c150 0x7f7e100b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.874+0000 7f7e15fdc700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e1015c520 con 0x7f7e1006ff10 2023-11-08T08:20:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.874+0000 7f7e15fdc700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e1006ac00 con 0x7f7e10070360 2023-11-08T08:20:00.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.874+0000 7f7e15fdc700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e100b4240 con 0x7f7e1015c150 2023-11-08T08:20:00.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.874+0000 7f7e14fda700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7e1015c150 0x7f7e100b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:00.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.874+0000 7f7e0ffff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7e1006ff10 0x7f7e1015fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41898/0 (socket says 172.21.15.40:41898) 2023-11-08T08:20:00.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.874+0000 7f7e0ffff700 1 -- 172.21.15.40:0/1048227080 learned_addr learned my addr 172.21.15.40:0/1048227080 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:00.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.874+0000 7f7e0f7fe700 1 -- 172.21.15.40:0/1048227080 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111513986 0 0) 0x7f7e1015c520 con 0x7f7e1006ff10 2023-11-08T08:20:00.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.874+0000 7f7e0f7fe700 1 -- 172.21.15.40:0/1048227080 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e000035d0 con 0x7f7e1006ff10 2023-11-08T08:20:00.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.874+0000 7f7e14fda700 1 -- 172.21.15.40:0/1048227080 >> v1:172.21.15.169:6789/0 conn(0x7f7e10070360 legacy=0x7f7e10160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:00.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.874+0000 7f7e14fda700 1 -- 172.21.15.40:0/1048227080 >> v1:172.21.15.40:6789/0 conn(0x7f7e1006ff10 legacy=0x7f7e1015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:00.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.874+0000 7f7e14fda700 1 -- 172.21.15.40:0/1048227080 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e10160a50 con 0x7f7e1015c150 2023-11-08T08:20:00.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.874+0000 7f7e14fda700 1 --2- 172.21.15.40:0/1048227080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e1015c150 0x7f7e100b3d00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009850 tx=0x7f7e04009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c360d143851560e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:00.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.874+0000 7f7e0f7fe700 1 -- 172.21.15.40:0/1048227080 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04011830 con 0x7f7e1015c150 2023-11-08T08:20:00.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.874+0000 7f7e0f7fe700 1 -- 172.21.15.40:0/1048227080 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e04011990 con 0x7f7e1015c150 2023-11-08T08:20:00.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.874+0000 7f7e0f7fe700 1 -- 172.21.15.40:0/1048227080 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04011d40 con 0x7f7e1015c150 2023-11-08T08:20:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.878+0000 7f7e15fdc700 1 -- 172.21.15.40:0/1048227080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e1015c150 msgr2=0x7f7e100b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.878+0000 7f7e15fdc700 1 --2- 172.21.15.40:0/1048227080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e1015c150 0x7f7e100b3d00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009850 tx=0x7f7e04009b60 comp rx=0 tx=0).stop 2023-11-08T08:20:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.878+0000 7f7e15fdc700 1 -- 172.21.15.40:0/1048227080 shutdown_connections 2023-11-08T08:20:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.878+0000 7f7e15fdc700 1 --2- 172.21.15.40:0/1048227080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e1015c150 0x7f7e100b3d00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.878+0000 7f7e15fdc700 1 -- 172.21.15.40:0/1048227080 >> 172.21.15.40:0/1048227080 conn(0x7f7e100d3680 msgr2=0x7f7e100cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.878+0000 7f7e15fdc700 1 -- 172.21.15.40:0/1048227080 shutdown_connections 2023-11-08T08:20:00.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.878+0000 7f7e15fdc700 1 -- 172.21.15.40:0/1048227080 wait complete. 2023-11-08T08:20:00.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.878+0000 7f7e15fdc700 1 Processor -- start 2023-11-08T08:20:00.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.882+0000 7f7e15fdc700 1 -- start start 2023-11-08T08:20:00.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.882+0000 7f7e15fdc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e1006ff10 0x7f7e101f5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:00.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.882+0000 7f7e15fdc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e10070360 0x7f7e10141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:00.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.882+0000 7f7e15fdc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e1015c150 0x7f7e101449d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:00.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.882+0000 7f7e15fdc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e100cccd0 con 0x7f7e1015c150 2023-11-08T08:20:00.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.882+0000 7f7e15fdc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e10148ce0 con 0x7f7e1006ff10 2023-11-08T08:20:00.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.882+0000 7f7e15fdc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e10145540 con 0x7f7e10070360 2023-11-08T08:20:00.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.882+0000 7f7e157db700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e1015c150 0x7f7e101449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:00.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.882+0000 7f7e157db700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e1015c150 0x7f7e101449d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36048/0 (socket says 172.21.15.40:36048) 2023-11-08T08:20:00.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.882+0000 7f7e157db700 1 -- 172.21.15.40:0/1490546252 learned_addr learned my addr 172.21.15.40:0/1490546252 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:00.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.882+0000 7f7e14fda700 1 --2- 172.21.15.40:0/1490546252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e1006ff10 0x7f7e101f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:00.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.882+0000 7f7e0ffff700 1 --2- 172.21.15.40:0/1490546252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e10070360 0x7f7e10141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:00.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.882+0000 7f7e157db700 1 -- 172.21.15.40:0/1490546252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e10070360 msgr2=0x7f7e10141fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:00.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.882+0000 7f7e157db700 1 --2- 172.21.15.40:0/1490546252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e10070360 0x7f7e10141fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.882+0000 7f7e157db700 1 -- 172.21.15.40:0/1490546252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e1006ff10 msgr2=0x7f7e101f5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:00.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.882+0000 7f7e157db700 1 --2- 172.21.15.40:0/1490546252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e1006ff10 0x7f7e101f5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:00.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.882+0000 7f7e157db700 1 -- 172.21.15.40:0/1490546252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e040094b0 con 0x7f7e1015c150 2023-11-08T08:20:00.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.882+0000 7f7e157db700 1 --2- 172.21.15.40:0/1490546252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e1015c150 0x7f7e101449d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800cc50 tx=0x7f7e0800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:00.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.886+0000 7f7e0d7fa700 1 -- 172.21.15.40:0/1490546252 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e08003c20 con 0x7f7e1015c150 2023-11-08T08:20:00.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.886+0000 7f7e0d7fa700 1 -- 172.21.15.40:0/1490546252 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e08003d80 con 0x7f7e1015c150 2023-11-08T08:20:00.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.886+0000 7f7e15fdc700 1 -- 172.21.15.40:0/1490546252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e100d2450 con 0x7f7e1015c150 2023-11-08T08:20:00.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.886+0000 7f7e0d7fa700 1 -- 172.21.15.40:0/1490546252 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0800ab60 con 0x7f7e1015c150 2023-11-08T08:20:00.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.886+0000 7f7e15fdc700 1 -- 172.21.15.40:0/1490546252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1006cc50 con 0x7f7e1015c150 2023-11-08T08:20:00.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.886+0000 7f7e15fdc700 1 -- 172.21.15.40:0/1490546252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e100d2730 con 0x7f7e1015c150 2023-11-08T08:20:00.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.890+0000 7f7e0d7fa700 1 -- 172.21.15.40:0/1490546252 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7e08010050 con 0x7f7e1015c150 2023-11-08T08:20:00.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.890+0000 7f7e0d7fa700 1 --2- 172.21.15.40:0/1490546252 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7df4064ee0 0x7f7df4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:00.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.890+0000 7f7e0d7fa700 1 -- 172.21.15.40:0/1490546252 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7e08082170 con 0x7f7e1015c150 2023-11-08T08:20:00.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.890+0000 7f7e14fda700 1 --2- 172.21.15.40:0/1490546252 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7df4064ee0 0x7f7df4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:00.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.894+0000 7f7e14fda700 1 --2- 172.21.15.40:0/1490546252 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7df4064ee0 0x7f7df4067390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7e040090b0 tx=0x7f7e0401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:00.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:00.898+0000 7f7e0d7fa700 1 -- 172.21.15.40:0/1490546252 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7e0804fef0 con 0x7f7e1015c150 2023-11-08T08:20:01.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.070+0000 7f7e15fdc700 1 -- 172.21.15.40:0/1490546252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7e10073d20 con 0x7f7e1015c150 2023-11-08T08:20:01.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.074+0000 7f7e0d7fa700 1 -- 172.21.15.40:0/1490546252 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7e08053510 con 0x7f7e1015c150 2023-11-08T08:20:01.076 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:20:01.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.074+0000 7f7e15fdc700 1 -- 172.21.15.40:0/1490546252 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7df4064ee0 msgr2=0x7f7df4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:01.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.074+0000 7f7e15fdc700 1 --2- 172.21.15.40:0/1490546252 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7df4064ee0 0x7f7df4067390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7e040090b0 tx=0x7f7e0401c040 comp rx=0 tx=0).stop 2023-11-08T08:20:01.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.074+0000 7f7e15fdc700 1 -- 172.21.15.40:0/1490546252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e1015c150 msgr2=0x7f7e101449d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:01.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.074+0000 7f7e15fdc700 1 --2- 172.21.15.40:0/1490546252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e1015c150 0x7f7e101449d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800cc50 tx=0x7f7e0800cf60 comp rx=0 tx=0).stop 2023-11-08T08:20:01.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.074+0000 7f7e15fdc700 1 -- 172.21.15.40:0/1490546252 shutdown_connections 2023-11-08T08:20:01.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.074+0000 7f7e15fdc700 1 --2- 172.21.15.40:0/1490546252 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7df4064ee0 0x7f7df4067390 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:01.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.074+0000 7f7e15fdc700 1 --2- 172.21.15.40:0/1490546252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e1015c150 0x7f7e101449d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:01.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.074+0000 7f7e15fdc700 1 --2- 172.21.15.40:0/1490546252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e10070360 0x7f7e10141fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:01.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.074+0000 7f7e15fdc700 1 --2- 172.21.15.40:0/1490546252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e1006ff10 0x7f7e101f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:01.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.074+0000 7f7e15fdc700 1 -- 172.21.15.40:0/1490546252 >> 172.21.15.40:0/1490546252 conn(0x7f7e100d3680 msgr2=0x7f7e101565f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:01.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.074+0000 7f7e15fdc700 1 -- 172.21.15.40:0/1490546252 shutdown_connections 2023-11-08T08:20:01.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.074+0000 7f7e15fdc700 1 -- 172.21.15.40:0/1490546252 wait complete. 2023-11-08T08:20:01.109 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-08T08:20:01.110 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-08T08:20:01.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.286+0000 7fc7cd145700 1 Processor -- start 2023-11-08T08:20:01.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.286+0000 7fc7cd145700 1 -- start start 2023-11-08T08:20:01.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.286+0000 7fc7cd145700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc7c80b43a0 0x7fc7c815e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:01.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.286+0000 7fc7cd145700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc7c8070790 0x7fc7c806bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:01.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7cd145700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c815a550 con 0x7fc7c80b1640 2023-11-08T08:20:01.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7cd145700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c806b0a0 con 0x7fc7c80b43a0 2023-11-08T08:20:01.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7cd145700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c80caac0 con 0x7fc7c8070790 2023-11-08T08:20:01.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7c7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc7c80b43a0 0x7fc7c815e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:01.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7c7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc7c80b43a0 0x7fc7c815e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36056/0 (socket says 172.21.15.40:36056) 2023-11-08T08:20:01.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7c7fff700 1 -- 172.21.15.40:0/2487676173 learned_addr learned my addr 172.21.15.40:0/2487676173 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:01.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7cc944700 1 --2- 172.21.15.40:0/2487676173 >> v2:172.21.15.169:3300/0 conn(0x7fc7c8070790 0x7fc7c806bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:01.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7c7fff700 1 -- 172.21.15.40:0/2487676173 >> v2:172.21.15.169:3300/0 conn(0x7fc7c8070790 msgr2=0x7fc7c806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:01.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7c7fff700 1 --2- 172.21.15.40:0/2487676173 >> v2:172.21.15.169:3300/0 conn(0x7fc7c8070790 0x7fc7c806bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:01.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7c7fff700 1 -- 172.21.15.40:0/2487676173 >> v1:172.21.15.40:6789/0 conn(0x7fc7c80b1640 legacy=0x7fc7c8070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:01.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7c7fff700 1 -- 172.21.15.40:0/2487676173 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80cada0 con 0x7fc7c80b43a0 2023-11-08T08:20:01.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7c7fff700 1 --2- 172.21.15.40:0/2487676173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c80b43a0 0x7fc7c815e5d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8070010 tx=0x7fc7b8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518fdb4b30ce1ef0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:01.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7c6ffd700 1 -- 172.21.15.40:0/2487676173 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8011830 con 0x7fc7c80b43a0 2023-11-08T08:20:01.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7c6ffd700 1 -- 172.21.15.40:0/2487676173 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b8011990 con 0x7fc7c80b43a0 2023-11-08T08:20:01.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7c6ffd700 1 -- 172.21.15.40:0/2487676173 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8011d20 con 0x7fc7c80b43a0 2023-11-08T08:20:01.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7cd145700 1 -- 172.21.15.40:0/2487676173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c80b43a0 msgr2=0x7fc7c815e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:01.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7cd145700 1 --2- 172.21.15.40:0/2487676173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c80b43a0 0x7fc7c815e5d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8070010 tx=0x7fc7b8009bf0 comp rx=0 tx=0).stop 2023-11-08T08:20:01.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7cd145700 1 -- 172.21.15.40:0/2487676173 shutdown_connections 2023-11-08T08:20:01.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7cd145700 1 --2- 172.21.15.40:0/2487676173 >> v2:172.21.15.169:3300/0 conn(0x7fc7c8070790 0x7fc7c806bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:01.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7cd145700 1 --2- 172.21.15.40:0/2487676173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c80b43a0 0x7fc7c815e5d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:01.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7cd145700 1 -- 172.21.15.40:0/2487676173 >> 172.21.15.40:0/2487676173 conn(0x7fc7c80b0f80 msgr2=0x7fc7c80ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:01.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7cd145700 1 -- 172.21.15.40:0/2487676173 shutdown_connections 2023-11-08T08:20:01.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.290+0000 7fc7cd145700 1 -- 172.21.15.40:0/2487676173 wait complete. 2023-11-08T08:20:01.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.294+0000 7fc7cd145700 1 Processor -- start 2023-11-08T08:20:01.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.298+0000 7fc7cd145700 1 -- start start 2023-11-08T08:20:01.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.298+0000 7fc7cd145700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7c8070790 0x7fc7c80b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:01.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.298+0000 7fc7cd145700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c80b1640 0x7fc7c814f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:01.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.298+0000 7fc7cd145700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7c80b43a0 0x7fc7c806a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:01.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.298+0000 7fc7cd145700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c815c8c0 con 0x7fc7c80b1640 2023-11-08T08:20:01.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.298+0000 7fc7cd145700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80c2700 con 0x7fc7c80b43a0 2023-11-08T08:20:01.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.298+0000 7fc7cd145700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80c29e0 con 0x7fc7c8070790 2023-11-08T08:20:01.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.298+0000 7fc7c77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c80b1640 0x7fc7c814f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:01.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.298+0000 7fc7c77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c80b1640 0x7fc7c814f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36060/0 (socket says 172.21.15.40:36060) 2023-11-08T08:20:01.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.298+0000 7fc7c77fe700 1 -- 172.21.15.40:0/1466603100 learned_addr learned my addr 172.21.15.40:0/1466603100 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:01.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.298+0000 7fc7c7fff700 1 --2- 172.21.15.40:0/1466603100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7c8070790 0x7fc7c80b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:01.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.298+0000 7fc7cc944700 1 --2- 172.21.15.40:0/1466603100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7c80b43a0 0x7fc7c806a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:01.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.302+0000 7fc7c77fe700 1 -- 172.21.15.40:0/1466603100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7c8070790 msgr2=0x7fc7c80b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:01.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.302+0000 7fc7c77fe700 1 --2- 172.21.15.40:0/1466603100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7c8070790 0x7fc7c80b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:01.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.302+0000 7fc7c77fe700 1 -- 172.21.15.40:0/1466603100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7c80b43a0 msgr2=0x7fc7c806a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:01.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.302+0000 7fc7c77fe700 1 --2- 172.21.15.40:0/1466603100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7c80b43a0 0x7fc7c806a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:01.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.302+0000 7fc7c77fe700 1 -- 172.21.15.40:0/1466603100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b8009570 con 0x7fc7c80b1640 2023-11-08T08:20:01.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.302+0000 7fc7c77fe700 1 --2- 172.21.15.40:0/1466603100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c80b1640 0x7fc7c814f310 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00c500 tx=0x7fc7bc00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:01.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.302+0000 7fc7c4ff9700 1 -- 172.21.15.40:0/1466603100 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc0052d0 con 0x7fc7c80b1640 2023-11-08T08:20:01.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.302+0000 7fc7c4ff9700 1 -- 172.21.15.40:0/1466603100 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7bc005430 con 0x7fc7c80b1640 2023-11-08T08:20:01.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.302+0000 7fc7cd145700 1 -- 172.21.15.40:0/1466603100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c806b5d0 con 0x7fc7c80b1640 2023-11-08T08:20:01.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.302+0000 7fc7c4ff9700 1 -- 172.21.15.40:0/1466603100 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc0055a0 con 0x7fc7c80b1640 2023-11-08T08:20:01.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.302+0000 7fc7cd145700 1 -- 172.21.15.40:0/1466603100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c806b2f0 con 0x7fc7c80b1640 2023-11-08T08:20:01.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.302+0000 7fc7cd145700 1 -- 172.21.15.40:0/1466603100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c815eb10 con 0x7fc7c80b1640 2023-11-08T08:20:01.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.306+0000 7fc7c4ff9700 1 -- 172.21.15.40:0/1466603100 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc7bc019030 con 0x7fc7c80b1640 2023-11-08T08:20:01.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.306+0000 7fc7c4ff9700 1 --2- 172.21.15.40:0/1466603100 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc7b4064ee0 0x7fc7b4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:01.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.306+0000 7fc7c4ff9700 1 -- 172.21.15.40:0/1466603100 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc7bc081330 con 0x7fc7c80b1640 2023-11-08T08:20:01.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.306+0000 7fc7c7fff700 1 --2- 172.21.15.40:0/1466603100 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc7b4064ee0 0x7fc7b4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:01.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.310+0000 7fc7c7fff700 1 --2- 172.21.15.40:0/1466603100 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc7b4064ee0 0x7fc7b4067390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8000c00 tx=0x7fc7b8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:01.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.314+0000 7fc7c4ff9700 1 -- 172.21.15.40:0/1466603100 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc7bc04f1a0 con 0x7fc7c80b1640 2023-11-08T08:20:01.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:01.546+0000 7fc7cd145700 1 -- 172.21.15.40:0/1466603100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7fc7c815a550 con 0x7fc7c80b1640 2023-11-08T08:20:02.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.018+0000 7fc7c4ff9700 1 -- 172.21.15.40:0/1466603100 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v78) v1 ==== 145+0+0 (secure 0 0 0) 0x7fc7c815a550 con 0x7fc7c80b1640 2023-11-08T08:20:02.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.022+0000 7fc7cd145700 1 -- 172.21.15.40:0/1466603100 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc7b4064ee0 msgr2=0x7fc7b4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:02.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.022+0000 7fc7cd145700 1 --2- 172.21.15.40:0/1466603100 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc7b4064ee0 0x7fc7b4067390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8000c00 tx=0x7fc7b8008030 comp rx=0 tx=0).stop 2023-11-08T08:20:02.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.022+0000 7fc7cd145700 1 -- 172.21.15.40:0/1466603100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c80b1640 msgr2=0x7fc7c814f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:02.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.022+0000 7fc7cd145700 1 --2- 172.21.15.40:0/1466603100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c80b1640 0x7fc7c814f310 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00c500 tx=0x7fc7bc00c810 comp rx=0 tx=0).stop 2023-11-08T08:20:02.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.022+0000 7fc7cd145700 1 -- 172.21.15.40:0/1466603100 shutdown_connections 2023-11-08T08:20:02.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.022+0000 7fc7cd145700 1 --2- 172.21.15.40:0/1466603100 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc7b4064ee0 0x7fc7b4067390 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.022+0000 7fc7cd145700 1 --2- 172.21.15.40:0/1466603100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7c80b43a0 0x7fc7c806a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.022+0000 7fc7cd145700 1 --2- 172.21.15.40:0/1466603100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c80b1640 0x7fc7c814f310 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.022+0000 7fc7cd145700 1 --2- 172.21.15.40:0/1466603100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7c8070790 0x7fc7c80b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.022+0000 7fc7cd145700 1 -- 172.21.15.40:0/1466603100 >> 172.21.15.40:0/1466603100 conn(0x7fc7c80b0f80 msgr2=0x7fc7c80ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:02.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.022+0000 7fc7cd145700 1 -- 172.21.15.40:0/1466603100 shutdown_connections 2023-11-08T08:20:02.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.022+0000 7fc7cd145700 1 -- 172.21.15.40:0/1466603100 wait complete. 2023-11-08T08:20:02.050 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-08T08:20:02.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.230+0000 7f011b59e700 1 Processor -- start 2023-11-08T08:20:02.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.230+0000 7f011b59e700 1 -- start start 2023-11-08T08:20:02.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011b59e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f011c0cdc50 0x7f011c069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:02.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011b59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f011c15e560 con 0x7f011c15e190 2023-11-08T08:20:02.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011b59e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f011c06be80 con 0x7f011c06ff10 2023-11-08T08:20:02.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011b59e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f011c0ce0a0 con 0x7f011c0cdc50 2023-11-08T08:20:02.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011a59c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f011c15e190 0x7f011c161560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41910/0 (socket says 172.21.15.40:41910) 2023-11-08T08:20:02.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011a59c700 1 -- 172.21.15.40:0/3197103478 learned_addr learned my addr 172.21.15.40:0/3197103478 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:02.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f0119d9b700 1 --2- 172.21.15.40:0/3197103478 >> v2:172.21.15.169:3300/0 conn(0x7f011c0cdc50 0x7f011c069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:02.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011959a700 1 -- 172.21.15.40:0/3197103478 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4059035459 0 0) 0x7f010c001110 con 0x7f011c15e190 2023-11-08T08:20:02.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011959a700 1 -- 172.21.15.40:0/3197103478 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01040035d0 con 0x7f011c15e190 2023-11-08T08:20:02.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011959a700 1 -- 172.21.15.40:0/3197103478 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3394250849 0 0) 0x7f01040035d0 con 0x7f011c15e190 2023-11-08T08:20:02.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011959a700 1 -- 172.21.15.40:0/3197103478 >> v2:172.21.15.169:3300/0 conn(0x7f011c0cdc50 msgr2=0x7f011c069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:02.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011959a700 1 --2- 172.21.15.40:0/3197103478 >> v2:172.21.15.169:3300/0 conn(0x7f011c0cdc50 0x7f011c069bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011959a700 1 -- 172.21.15.40:0/3197103478 >> v1:172.21.15.169:6789/0 conn(0x7f011c06ff10 legacy=0x7f011c06c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:02.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011959a700 1 -- 172.21.15.40:0/3197103478 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c0ce330 con 0x7f011c15e190 2023-11-08T08:20:02.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011959a700 1 -- 172.21.15.40:0/3197103478 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f010c001800 con 0x7f011c15e190 2023-11-08T08:20:02.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011959a700 1 -- 172.21.15.40:0/3197103478 >> v1:172.21.15.40:6789/0 conn(0x7f011c15e190 legacy=0x7f011c161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:02.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011959a700 1 --2- 172.21.15.40:0/3197103478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0104003a20 0x7f0104003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:02.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011959a700 1 --2- 172.21.15.40:0/3197103478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0104004870 0x7f0104004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:02.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011959a700 1 --2- 172.21.15.40:0/3197103478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0104005220 0x7f010400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:02.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011959a700 1 -- 172.21.15.40:0/3197103478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c0ce0a0 con 0x7f0104005220 2023-11-08T08:20:02.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011959a700 1 -- 172.21.15.40:0/3197103478 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010400c0a0 con 0x7f0104003a20 2023-11-08T08:20:02.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011959a700 1 -- 172.21.15.40:0/3197103478 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010400c260 con 0x7f0104004870 2023-11-08T08:20:02.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.234+0000 7f011ad9d700 1 --2- 172.21.15.40:0/3197103478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0104005220 0x7f010400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:02.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.238+0000 7f011a59c700 1 --2- 172.21.15.40:0/3197103478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0104003a20 0x7f0104003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:02.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.238+0000 7f0119d9b700 1 --2- 172.21.15.40:0/3197103478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0104004870 0x7f0104004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:02.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.238+0000 7f011ad9d700 1 -- 172.21.15.40:0/3197103478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0104004870 msgr2=0x7f0104004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:02.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.238+0000 7f011ad9d700 1 --2- 172.21.15.40:0/3197103478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0104004870 0x7f0104004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.238+0000 7f011ad9d700 1 -- 172.21.15.40:0/3197103478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0104003a20 msgr2=0x7f0104003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:02.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.238+0000 7f011ad9d700 1 --2- 172.21.15.40:0/3197103478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0104003a20 0x7f0104003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.238+0000 7f011ad9d700 1 -- 172.21.15.40:0/3197103478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01040035d0 con 0x7f0104005220 2023-11-08T08:20:02.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.238+0000 7f011ad9d700 1 -- 172.21.15.40:0/3197103478 reap_dead start 2023-11-08T08:20:02.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.238+0000 7f011ad9d700 1 --2- 172.21.15.40:0/3197103478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0104005220 0x7f010400ba30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f011400c190 tx=0x7f01040058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81ffb0876b7aa32 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:02.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.238+0000 7f011959a700 1 -- 172.21.15.40:0/3197103478 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0114001db0 con 0x7f0104005220 2023-11-08T08:20:02.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.238+0000 7f011959a700 1 -- 172.21.15.40:0/3197103478 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f011400ccf0 con 0x7f0104005220 2023-11-08T08:20:02.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.238+0000 7f011959a700 1 -- 172.21.15.40:0/3197103478 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0114003c60 con 0x7f0104005220 2023-11-08T08:20:02.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.238+0000 7f011b59e700 1 -- 172.21.15.40:0/3197103478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0104005220 msgr2=0x7f010400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:02.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.238+0000 7f011b59e700 1 --2- 172.21.15.40:0/3197103478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0104005220 0x7f010400ba30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f011400c190 tx=0x7f01040058a0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.238+0000 7f011b59e700 1 -- 172.21.15.40:0/3197103478 shutdown_connections 2023-11-08T08:20:02.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.238+0000 7f011b59e700 1 --2- 172.21.15.40:0/3197103478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0104005220 0x7f010400ba30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.238+0000 7f011b59e700 1 -- 172.21.15.40:0/3197103478 >> 172.21.15.40:0/3197103478 conn(0x7f011c0cf280 msgr2=0x7f011c0cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:02.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.238+0000 7f011b59e700 1 -- 172.21.15.40:0/3197103478 shutdown_connections 2023-11-08T08:20:02.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.238+0000 7f011b59e700 1 -- 172.21.15.40:0/3197103478 wait complete. 2023-11-08T08:20:02.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.242+0000 7f011b59e700 1 Processor -- start 2023-11-08T08:20:02.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.242+0000 7f011b59e700 1 -- start start 2023-11-08T08:20:02.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.242+0000 7f011b59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0104005220 0x7f011c1578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:02.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.242+0000 7f011b59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f011c0ca2d0 0x7f011c14a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:02.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.242+0000 7f011b59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f011c158510 0x7f011c1589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:02.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.242+0000 7f011b59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c158ef0 con 0x7f011c0ca2d0 2023-11-08T08:20:02.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.242+0000 7f011b59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c0c9570 con 0x7f0104005220 2023-11-08T08:20:02.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.242+0000 7f011b59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011c153c50 con 0x7f011c158510 2023-11-08T08:20:02.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.242+0000 7f0119d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f011c0ca2d0 0x7f011c14a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:02.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.246+0000 7f0119d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f011c0ca2d0 0x7f011c14a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36064/0 (socket says 172.21.15.40:36064) 2023-11-08T08:20:02.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.246+0000 7f0119d9b700 1 -- 172.21.15.40:0/2209756613 learned_addr learned my addr 172.21.15.40:0/2209756613 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:02.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.246+0000 7f011ad9d700 1 --2- 172.21.15.40:0/2209756613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f011c158510 0x7f011c1589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:02.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.246+0000 7f011a59c700 1 --2- 172.21.15.40:0/2209756613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0104005220 0x7f011c1578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:02.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.246+0000 7f0119d9b700 1 -- 172.21.15.40:0/2209756613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f011c158510 msgr2=0x7f011c1589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:02.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.246+0000 7f0119d9b700 1 --2- 172.21.15.40:0/2209756613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f011c158510 0x7f011c1589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.246+0000 7f0119d9b700 1 -- 172.21.15.40:0/2209756613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0104005220 msgr2=0x7f011c1578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:02.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.246+0000 7f0119d9b700 1 --2- 172.21.15.40:0/2209756613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0104005220 0x7f011c1578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.246+0000 7f0119d9b700 1 -- 172.21.15.40:0/2209756613 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c0afc70 con 0x7f011c0ca2d0 2023-11-08T08:20:02.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.246+0000 7f0119d9b700 1 --2- 172.21.15.40:0/2209756613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f011c0ca2d0 0x7f011c14a7f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f01100108f0 tx=0x7f0110010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:02.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.246+0000 7f010affd700 1 -- 172.21.15.40:0/2209756613 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110016670 con 0x7f011c0ca2d0 2023-11-08T08:20:02.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.246+0000 7f010affd700 1 -- 172.21.15.40:0/2209756613 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01100167d0 con 0x7f011c0ca2d0 2023-11-08T08:20:02.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.246+0000 7f010affd700 1 -- 172.21.15.40:0/2209756613 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110016b60 con 0x7f011c0ca2d0 2023-11-08T08:20:02.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.246+0000 7f011b59e700 1 -- 172.21.15.40:0/2209756613 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f011c0af990 con 0x7f011c0ca2d0 2023-11-08T08:20:02.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.246+0000 7f011b59e700 1 -- 172.21.15.40:0/2209756613 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f011c0aff50 con 0x7f011c0ca2d0 2023-11-08T08:20:02.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.246+0000 7f011b59e700 1 -- 172.21.15.40:0/2209756613 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011c073d20 con 0x7f011c0ca2d0 2023-11-08T08:20:02.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.250+0000 7f010affd700 1 -- 172.21.15.40:0/2209756613 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f011001c030 con 0x7f011c0ca2d0 2023-11-08T08:20:02.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.250+0000 7f010affd700 1 --2- 172.21.15.40:0/2209756613 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f00fc064ee0 0x7f00fc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:02.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.250+0000 7f010affd700 1 -- 172.21.15.40:0/2209756613 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0110082330 con 0x7f011c0ca2d0 2023-11-08T08:20:02.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.250+0000 7f011a59c700 1 --2- 172.21.15.40:0/2209756613 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f00fc064ee0 0x7f00fc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:02.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.254+0000 7f011a59c700 1 --2- 172.21.15.40:0/2209756613 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f00fc064ee0 0x7f00fc067390 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f010c001050 tx=0x7f010c007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:02.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.258+0000 7f010affd700 1 -- 172.21.15.40:0/2209756613 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f01100500b0 con 0x7f011c0ca2d0 2023-11-08T08:20:02.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.430+0000 7f011b59e700 1 -- 172.21.15.40:0/2209756613 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f011c0cbec0 con 0x7f011c0ca2d0 2023-11-08T08:20:02.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.438+0000 7f010affd700 1 -- 172.21.15.40:0/2209756613 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f01100536d0 con 0x7f011c0ca2d0 2023-11-08T08:20:02.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.442+0000 7f011b59e700 1 -- 172.21.15.40:0/2209756613 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f00fc064ee0 msgr2=0x7f00fc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:02.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.442+0000 7f011b59e700 1 --2- 172.21.15.40:0/2209756613 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f00fc064ee0 0x7f00fc067390 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f010c001050 tx=0x7f010c007040 comp rx=0 tx=0).stop 2023-11-08T08:20:02.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.442+0000 7f011b59e700 1 -- 172.21.15.40:0/2209756613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f011c0ca2d0 msgr2=0x7f011c14a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:02.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.442+0000 7f011b59e700 1 --2- 172.21.15.40:0/2209756613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f011c0ca2d0 0x7f011c14a7f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f01100108f0 tx=0x7f0110010c00 comp rx=0 tx=0).stop 2023-11-08T08:20:02.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.442+0000 7f011b59e700 1 -- 172.21.15.40:0/2209756613 shutdown_connections 2023-11-08T08:20:02.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.442+0000 7f011b59e700 1 --2- 172.21.15.40:0/2209756613 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f00fc064ee0 0x7f00fc067390 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.442+0000 7f011b59e700 1 --2- 172.21.15.40:0/2209756613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f011c158510 0x7f011c1589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.442+0000 7f011b59e700 1 --2- 172.21.15.40:0/2209756613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f011c0ca2d0 0x7f011c14a7f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.442+0000 7f011b59e700 1 --2- 172.21.15.40:0/2209756613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0104005220 0x7f011c1578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.442+0000 7f011b59e700 1 -- 172.21.15.40:0/2209756613 >> 172.21.15.40:0/2209756613 conn(0x7f011c0cf280 msgr2=0x7f011c0c34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:02.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.442+0000 7f011b59e700 1 -- 172.21.15.40:0/2209756613 shutdown_connections 2023-11-08T08:20:02.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.442+0000 7f011b59e700 1 -- 172.21.15.40:0/2209756613 wait complete. 2023-11-08T08:20:02.470 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T08:20:02.471 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T08:20:02.514 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T08:20:02.562 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T08:20:02.563 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:20:02.563 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:20:02.583 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:20:02.584 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:20:02.645 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:20:02.645 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:20:02.707 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:20:02.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.934+0000 7f6c26fb8700 1 Processor -- start 2023-11-08T08:20:02.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.934+0000 7f6c26fb8700 1 -- start start 2023-11-08T08:20:02.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.934+0000 7f6c26fb8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c2015c3b0 0x7f6c200d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:02.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.934+0000 7f6c26fb8700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6c2006c040 0x7f6c2006c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:02.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.934+0000 7f6c26fb8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6c200cafd0 0x7f6c200cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:02.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.934+0000 7f6c26fb8700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c200cb990 con 0x7f6c200cafd0 2023-11-08T08:20:02.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.934+0000 7f6c26fb8700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c200cebb0 con 0x7f6c2006c040 2023-11-08T08:20:02.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.934+0000 7f6c26fb8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c200ce5f0 con 0x7f6c2015c3b0 2023-11-08T08:20:02.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.934+0000 7f6c25555700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6c200cafd0 0x7f6c200cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:02.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.934+0000 7f6c1ffff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6c2006c040 0x7f6c2006c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:02.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.934+0000 7f6c24d54700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c2015c3b0 0x7f6c200d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:02.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.934+0000 7f6c25555700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6c200cafd0 0x7f6c200cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36070/0 (socket says 172.21.15.40:36070) 2023-11-08T08:20:02.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.934+0000 7f6c25555700 1 -- 172.21.15.40:0/2846433264 learned_addr learned my addr 172.21.15.40:0/2846433264 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:02.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.938+0000 7f6c1ffff700 1 -- 172.21.15.40:0/2846433264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c2015c3b0 msgr2=0x7f6c200d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:02.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.938+0000 7f6c1ffff700 1 --2- 172.21.15.40:0/2846433264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c2015c3b0 0x7f6c200d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.938+0000 7f6c1ffff700 1 -- 172.21.15.40:0/2846433264 >> v2:172.21.15.40:3300/0 conn(0x7f6c200cafd0 msgr2=0x7f6c200cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:02.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.938+0000 7f6c1ffff700 1 --2- 172.21.15.40:0/2846433264 >> v2:172.21.15.40:3300/0 conn(0x7f6c200cafd0 0x7f6c200cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.938+0000 7f6c1ffff700 1 -- 172.21.15.40:0/2846433264 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c200ce8d0 con 0x7f6c2006c040 2023-11-08T08:20:02.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.938+0000 7f6c1ffff700 1 --2- 172.21.15.40:0/2846433264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c2006c040 0x7f6c2006c490 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6c200b3540 tx=0x7f6c0c009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30a61c409079e158 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:02.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.938+0000 7f6c1f7fe700 1 -- 172.21.15.40:0/2846433264 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c011830 con 0x7f6c2006c040 2023-11-08T08:20:02.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.938+0000 7f6c1f7fe700 1 -- 172.21.15.40:0/2846433264 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0c011990 con 0x7f6c2006c040 2023-11-08T08:20:02.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.938+0000 7f6c1f7fe700 1 -- 172.21.15.40:0/2846433264 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c011d20 con 0x7f6c2006c040 2023-11-08T08:20:02.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.938+0000 7f6c26fb8700 1 -- 172.21.15.40:0/2846433264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c2006c040 msgr2=0x7f6c2006c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:02.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.938+0000 7f6c26fb8700 1 --2- 172.21.15.40:0/2846433264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c2006c040 0x7f6c2006c490 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6c200b3540 tx=0x7f6c0c009b30 comp rx=0 tx=0).stop 2023-11-08T08:20:02.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.938+0000 7f6c26fb8700 1 -- 172.21.15.40:0/2846433264 shutdown_connections 2023-11-08T08:20:02.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.938+0000 7f6c26fb8700 1 --2- 172.21.15.40:0/2846433264 >> v2:172.21.15.40:3300/0 conn(0x7f6c200cafd0 0x7f6c200cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.938+0000 7f6c26fb8700 1 --2- 172.21.15.40:0/2846433264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c2006c040 0x7f6c2006c490 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.938+0000 7f6c26fb8700 1 --2- 172.21.15.40:0/2846433264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c2015c3b0 0x7f6c200d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.938+0000 7f6c26fb8700 1 -- 172.21.15.40:0/2846433264 >> 172.21.15.40:0/2846433264 conn(0x7f6c2006e520 msgr2=0x7f6c2006e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:02.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.938+0000 7f6c26fb8700 1 -- 172.21.15.40:0/2846433264 shutdown_connections 2023-11-08T08:20:02.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.938+0000 7f6c26fb8700 1 -- 172.21.15.40:0/2846433264 wait complete. 2023-11-08T08:20:02.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.942+0000 7f6c26fb8700 1 Processor -- start 2023-11-08T08:20:02.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.942+0000 7f6c26fb8700 1 -- start start 2023-11-08T08:20:02.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.942+0000 7f6c26fb8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c2006c040 0x7f6c200b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:02.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.942+0000 7f6c26fb8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c200cafd0 0x7f6c200ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:02.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.942+0000 7f6c26fb8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c2015c3b0 0x7f6c20156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:02.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.942+0000 7f6c26fb8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c200cabb0 con 0x7f6c200cafd0 2023-11-08T08:20:02.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.942+0000 7f6c26fb8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c20157350 con 0x7f6c2015c3b0 2023-11-08T08:20:02.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.942+0000 7f6c26fb8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c20157d40 con 0x7f6c2006c040 2023-11-08T08:20:02.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.942+0000 7f6c1ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c200cafd0 0x7f6c200ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:02.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.942+0000 7f6c1ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c200cafd0 0x7f6c200ca670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36078/0 (socket says 172.21.15.40:36078) 2023-11-08T08:20:02.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.942+0000 7f6c1ffff700 1 -- 172.21.15.40:0/2614132374 learned_addr learned my addr 172.21.15.40:0/2614132374 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:02.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.942+0000 7f6c24d54700 1 --2- 172.21.15.40:0/2614132374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c2006c040 0x7f6c200b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:02.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.942+0000 7f6c25555700 1 --2- 172.21.15.40:0/2614132374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c2015c3b0 0x7f6c20156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:02.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.942+0000 7f6c1ffff700 1 -- 172.21.15.40:0/2614132374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c2006c040 msgr2=0x7f6c200b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:02.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.942+0000 7f6c1ffff700 1 --2- 172.21.15.40:0/2614132374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c2006c040 0x7f6c200b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.942+0000 7f6c1ffff700 1 -- 172.21.15.40:0/2614132374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c2015c3b0 msgr2=0x7f6c20156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:02.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.942+0000 7f6c1ffff700 1 --2- 172.21.15.40:0/2614132374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c2015c3b0 0x7f6c20156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:02.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.946+0000 7f6c1ffff700 1 -- 172.21.15.40:0/2614132374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c0094b0 con 0x7f6c200cafd0 2023-11-08T08:20:02.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.946+0000 7f6c1ffff700 1 --2- 172.21.15.40:0/2614132374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c200cafd0 0x7f6c200ca670 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0090b0 tx=0x7f6c0c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:02.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.946+0000 7f6c1d7fa700 1 -- 172.21.15.40:0/2614132374 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c004450 con 0x7f6c200cafd0 2023-11-08T08:20:02.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.946+0000 7f6c1d7fa700 1 -- 172.21.15.40:0/2614132374 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0c01e070 con 0x7f6c200cafd0 2023-11-08T08:20:02.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.946+0000 7f6c1d7fa700 1 -- 172.21.15.40:0/2614132374 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c0038e0 con 0x7f6c200cafd0 2023-11-08T08:20:02.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.946+0000 7f6c26fb8700 1 -- 172.21.15.40:0/2614132374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2006bb20 con 0x7f6c200cafd0 2023-11-08T08:20:02.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.946+0000 7f6c26fb8700 1 -- 172.21.15.40:0/2614132374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2006b840 con 0x7f6c200cafd0 2023-11-08T08:20:02.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.946+0000 7f6c26fb8700 1 -- 172.21.15.40:0/2614132374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c200d05f0 con 0x7f6c200cafd0 2023-11-08T08:20:02.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.950+0000 7f6c1d7fa700 1 -- 172.21.15.40:0/2614132374 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f6c0c00f050 con 0x7f6c200cafd0 2023-11-08T08:20:02.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.950+0000 7f6c1d7fa700 1 --2- 172.21.15.40:0/2614132374 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6c04064ee0 0x7f6c04067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:02.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.950+0000 7f6c1d7fa700 1 -- 172.21.15.40:0/2614132374 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6c0c083070 con 0x7f6c200cafd0 2023-11-08T08:20:02.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.950+0000 7f6c24d54700 1 --2- 172.21.15.40:0/2614132374 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6c04064ee0 0x7f6c04067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:02.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.950+0000 7f6c24d54700 1 --2- 172.21.15.40:0/2614132374 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6c04064ee0 0x7f6c04067390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6c14011340 tx=0x7f6c1400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:02.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:02.958+0000 7f6c1d7fa700 1 -- 172.21.15.40:0/2614132374 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f6c0c0b3400 con 0x7f6c200cafd0 2023-11-08T08:20:03.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.142+0000 7f6c26fb8700 1 -- 172.21.15.40:0/2614132374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c20072710 con 0x7f6c200cafd0 2023-11-08T08:20:03.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.146+0000 7f6c1d7fa700 1 -- 172.21.15.40:0/2614132374 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19748 (secure 0 0 0) 0x7f6c0c020090 con 0x7f6c200cafd0 2023-11-08T08:20:03.151 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:03.152 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":30,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:45.885827+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:20:03.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.150+0000 7f6c26fb8700 1 -- 172.21.15.40:0/2614132374 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6c04064ee0 msgr2=0x7f6c04067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:03.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.150+0000 7f6c26fb8700 1 --2- 172.21.15.40:0/2614132374 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6c04064ee0 0x7f6c04067390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6c14011340 tx=0x7f6c1400b040 comp rx=0 tx=0).stop 2023-11-08T08:20:03.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.150+0000 7f6c26fb8700 1 -- 172.21.15.40:0/2614132374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c200cafd0 msgr2=0x7f6c200ca670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:03.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.150+0000 7f6c26fb8700 1 --2- 172.21.15.40:0/2614132374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c200cafd0 0x7f6c200ca670 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0090b0 tx=0x7f6c0c004770 comp rx=0 tx=0).stop 2023-11-08T08:20:03.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.150+0000 7f6c26fb8700 1 -- 172.21.15.40:0/2614132374 shutdown_connections 2023-11-08T08:20:03.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.150+0000 7f6c26fb8700 1 --2- 172.21.15.40:0/2614132374 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6c04064ee0 0x7f6c04067390 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.150+0000 7f6c26fb8700 1 --2- 172.21.15.40:0/2614132374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c2015c3b0 0x7f6c20156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.150+0000 7f6c26fb8700 1 --2- 172.21.15.40:0/2614132374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c200cafd0 0x7f6c200ca670 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.150+0000 7f6c26fb8700 1 --2- 172.21.15.40:0/2614132374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c2006c040 0x7f6c200b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.150+0000 7f6c26fb8700 1 -- 172.21.15.40:0/2614132374 >> 172.21.15.40:0/2614132374 conn(0x7f6c2006e520 msgr2=0x7f6c200b4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:03.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.150+0000 7f6c26fb8700 1 -- 172.21.15.40:0/2614132374 shutdown_connections 2023-11-08T08:20:03.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.150+0000 7f6c26fb8700 1 -- 172.21.15.40:0/2614132374 wait complete. 2023-11-08T08:20:03.178 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T08:19:02.627691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T08:19:03.473821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T08:19:06.539942+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T08:20:03.179 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:20:03.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.358+0000 7f42f9530700 1 Processor -- start 2023-11-08T08:20:03.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.358+0000 7f42f9530700 1 -- start start 2023-11-08T08:20:03.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.362+0000 7f42f9530700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42f415c150 0x7f42f40b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:03.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.362+0000 7f42f9530700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f415c520 con 0x7f42f4070360 2023-11-08T08:20:03.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.362+0000 7f42f9530700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f406ac00 con 0x7f42f406ff10 2023-11-08T08:20:03.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.362+0000 7f42f9530700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f40b4240 con 0x7f42f415c150 2023-11-08T08:20:03.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.362+0000 7f42f3fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42f415c150 0x7f42f40b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:03.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.362+0000 7f42f37fe700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f42f406ff10 0x7f42f415fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:43822/0 (socket says 172.21.15.40:43822) 2023-11-08T08:20:03.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.362+0000 7f42f37fe700 1 -- 172.21.15.40:0/1273713788 learned_addr learned my addr 172.21.15.40:0/1273713788 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:03.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.362+0000 7f42f8d2f700 1 --1- 172.21.15.40:0/1273713788 >> v1:172.21.15.40:6789/0 conn(0x7f42f4070360 0x7f42f4160340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41926/0 (socket says 172.21.15.40:41926) 2023-11-08T08:20:03.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.362+0000 7f42f3fff700 1 -- 172.21.15.40:0/1273713788 >> v1:172.21.15.169:6789/0 conn(0x7f42f406ff10 legacy=0x7f42f415fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:03.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.362+0000 7f42f3fff700 1 -- 172.21.15.40:0/1273713788 >> v1:172.21.15.40:6789/0 conn(0x7f42f4070360 legacy=0x7f42f4160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:03.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.362+0000 7f42f2ffd700 1 -- 172.21.15.40:0/1273713788 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004797479 0 0) 0x7f42f406ac00 con 0x7f42f406ff10 2023-11-08T08:20:03.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.362+0000 7f42f3fff700 1 -- 172.21.15.40:0/1273713788 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f4160a50 con 0x7f42f415c150 2023-11-08T08:20:03.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.362+0000 7f42f3fff700 1 --2- 172.21.15.40:0/1273713788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42f415c150 0x7f42f40b3d00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009880 tx=0x7f42e4009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6680efe09c06e9c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:03.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.362+0000 7f42f2ffd700 1 -- 172.21.15.40:0/1273713788 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4004230 con 0x7f42f415c150 2023-11-08T08:20:03.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.362+0000 7f42f2ffd700 1 -- 172.21.15.40:0/1273713788 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e4004390 con 0x7f42f415c150 2023-11-08T08:20:03.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.362+0000 7f42f2ffd700 1 -- 172.21.15.40:0/1273713788 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e40036c0 con 0x7f42f415c150 2023-11-08T08:20:03.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.366+0000 7f42f9530700 1 -- 172.21.15.40:0/1273713788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42f415c150 msgr2=0x7f42f40b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:03.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.366+0000 7f42f9530700 1 --2- 172.21.15.40:0/1273713788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42f415c150 0x7f42f40b3d00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009880 tx=0x7f42e4009b90 comp rx=0 tx=0).stop 2023-11-08T08:20:03.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.366+0000 7f42f9530700 1 -- 172.21.15.40:0/1273713788 shutdown_connections 2023-11-08T08:20:03.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.366+0000 7f42f9530700 1 --2- 172.21.15.40:0/1273713788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42f415c150 0x7f42f40b3d00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.366+0000 7f42f9530700 1 -- 172.21.15.40:0/1273713788 >> 172.21.15.40:0/1273713788 conn(0x7f42f40d3680 msgr2=0x7f42f40cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:03.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.366+0000 7f42f9530700 1 -- 172.21.15.40:0/1273713788 shutdown_connections 2023-11-08T08:20:03.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.366+0000 7f42f9530700 1 -- 172.21.15.40:0/1273713788 wait complete. 2023-11-08T08:20:03.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.370+0000 7f42f9530700 1 Processor -- start 2023-11-08T08:20:03.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.370+0000 7f42f9530700 1 -- start start 2023-11-08T08:20:03.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.370+0000 7f42f9530700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42f406ff10 0x7f42f40ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:03.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.370+0000 7f42f9530700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42f4070360 0x7f42f4150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:03.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.370+0000 7f42f9530700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42f415c150 0x7f42f4151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:03.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.370+0000 7f42f9530700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f40ca870 con 0x7f42f406ff10 2023-11-08T08:20:03.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.370+0000 7f42f9530700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f40cab50 con 0x7f42f415c150 2023-11-08T08:20:03.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.370+0000 7f42f9530700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f40cae30 con 0x7f42f4070360 2023-11-08T08:20:03.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.370+0000 7f42f3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42f406ff10 0x7f42f40ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:03.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.370+0000 7f42f3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42f406ff10 0x7f42f40ca190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36082/0 (socket says 172.21.15.40:36082) 2023-11-08T08:20:03.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.370+0000 7f42f3fff700 1 -- 172.21.15.40:0/907769674 learned_addr learned my addr 172.21.15.40:0/907769674 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:03.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.370+0000 7f42f37fe700 1 --2- 172.21.15.40:0/907769674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42f4070360 0x7f42f4150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:03.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.370+0000 7f42f8d2f700 1 --2- 172.21.15.40:0/907769674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42f415c150 0x7f42f4151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:03.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.374+0000 7f42f3fff700 1 -- 172.21.15.40:0/907769674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42f4070360 msgr2=0x7f42f4150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:03.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.374+0000 7f42f3fff700 1 --2- 172.21.15.40:0/907769674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42f4070360 0x7f42f4150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.374+0000 7f42f3fff700 1 -- 172.21.15.40:0/907769674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42f415c150 msgr2=0x7f42f4151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:03.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.374+0000 7f42f3fff700 1 --2- 172.21.15.40:0/907769674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42f415c150 0x7f42f4151280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.374+0000 7f42f3fff700 1 -- 172.21.15.40:0/907769674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e40094b0 con 0x7f42f406ff10 2023-11-08T08:20:03.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.374+0000 7f42f8d2f700 1 --2- 172.21.15.40:0/907769674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42f415c150 0x7f42f4151280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:03.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.374+0000 7f42f3fff700 1 --2- 172.21.15.40:0/907769674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42f406ff10 0x7f42f40ca190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f42e4000c00 tx=0x7f42e4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:03.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.374+0000 7f42f0ff9700 1 -- 172.21.15.40:0/907769674 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4003db0 con 0x7f42f406ff10 2023-11-08T08:20:03.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.374+0000 7f42f0ff9700 1 -- 172.21.15.40:0/907769674 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e401f070 con 0x7f42f406ff10 2023-11-08T08:20:03.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.374+0000 7f42f9530700 1 -- 172.21.15.40:0/907769674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f40d2450 con 0x7f42f406ff10 2023-11-08T08:20:03.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.374+0000 7f42f0ff9700 1 -- 172.21.15.40:0/907769674 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4011870 con 0x7f42f406ff10 2023-11-08T08:20:03.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.374+0000 7f42f9530700 1 -- 172.21.15.40:0/907769674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f406cc50 con 0x7f42f406ff10 2023-11-08T08:20:03.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.374+0000 7f42f9530700 1 -- 172.21.15.40:0/907769674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f40d2730 con 0x7f42f406ff10 2023-11-08T08:20:03.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.378+0000 7f42f0ff9700 1 -- 172.21.15.40:0/907769674 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f42e402a020 con 0x7f42f406ff10 2023-11-08T08:20:03.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.378+0000 7f42f0ff9700 1 --2- 172.21.15.40:0/907769674 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f42e0064ee0 0x7f42e0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:03.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.378+0000 7f42f0ff9700 1 -- 172.21.15.40:0/907769674 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f42e4083990 con 0x7f42f406ff10 2023-11-08T08:20:03.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.382+0000 7f42f37fe700 1 --2- 172.21.15.40:0/907769674 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f42e0064ee0 0x7f42e0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:03.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.382+0000 7f42f37fe700 1 --2- 172.21.15.40:0/907769674 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f42e0064ee0 0x7f42e0067390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f42f4147530 tx=0x7f42e8004a80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:03.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.386+0000 7f42f0ff9700 1 -- 172.21.15.40:0/907769674 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f42e4051710 con 0x7f42f406ff10 2023-11-08T08:20:03.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.566+0000 7f42f9530700 1 -- 172.21.15.40:0/907769674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f42f40d2730 con 0x7f42f406ff10 2023-11-08T08:20:03.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.566+0000 7f42f0ff9700 1 -- 172.21.15.40:0/907769674 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f42e4054d30 con 0x7f42f406ff10 2023-11-08T08:20:03.570 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:20:03.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.570+0000 7f42f9530700 1 -- 172.21.15.40:0/907769674 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f42e0064ee0 msgr2=0x7f42e0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:03.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.570+0000 7f42f9530700 1 --2- 172.21.15.40:0/907769674 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f42e0064ee0 0x7f42e0067390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f42f4147530 tx=0x7f42e8004a80 comp rx=0 tx=0).stop 2023-11-08T08:20:03.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.570+0000 7f42f9530700 1 -- 172.21.15.40:0/907769674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42f406ff10 msgr2=0x7f42f40ca190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:03.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.570+0000 7f42f9530700 1 --2- 172.21.15.40:0/907769674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42f406ff10 0x7f42f40ca190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f42e4000c00 tx=0x7f42e4004770 comp rx=0 tx=0).stop 2023-11-08T08:20:03.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.570+0000 7f42f9530700 1 -- 172.21.15.40:0/907769674 shutdown_connections 2023-11-08T08:20:03.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.570+0000 7f42f9530700 1 --2- 172.21.15.40:0/907769674 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f42e0064ee0 0x7f42e0067390 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.570+0000 7f42f9530700 1 --2- 172.21.15.40:0/907769674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42f415c150 0x7f42f4151280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.570+0000 7f42f9530700 1 --2- 172.21.15.40:0/907769674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42f4070360 0x7f42f4150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.570+0000 7f42f9530700 1 --2- 172.21.15.40:0/907769674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42f406ff10 0x7f42f40ca190 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.570+0000 7f42f9530700 1 -- 172.21.15.40:0/907769674 >> 172.21.15.40:0/907769674 conn(0x7f42f40d3680 msgr2=0x7f42f40b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:03.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.570+0000 7f42f9530700 1 -- 172.21.15.40:0/907769674 shutdown_connections 2023-11-08T08:20:03.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.570+0000 7f42f9530700 1 -- 172.21.15.40:0/907769674 wait complete. 2023-11-08T08:20:03.597 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:20:03.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.774+0000 7f7e49d24700 1 Processor -- start 2023-11-08T08:20:03.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.774+0000 7f7e49d24700 1 -- start start 2023-11-08T08:20:03.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.778+0000 7f7e49d24700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7e4415c150 0x7f7e440b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:03.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.778+0000 7f7e49d24700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7e44070360 0x7f7e44160340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:03.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.778+0000 7f7e49d24700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e4415c520 con 0x7f7e4406ff10 2023-11-08T08:20:03.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.778+0000 7f7e49d24700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e440b4240 con 0x7f7e4415c150 2023-11-08T08:20:03.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.778+0000 7f7e49d24700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e4406ac00 con 0x7f7e44070360 2023-11-08T08:20:03.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.778+0000 7f7e48d22700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7e4415c150 0x7f7e440b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:03.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.778+0000 7f7e48d22700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7e4415c150 0x7f7e440b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36096/0 (socket says 172.21.15.40:36096) 2023-11-08T08:20:03.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.778+0000 7f7e48d22700 1 -- 172.21.15.40:0/508949553 learned_addr learned my addr 172.21.15.40:0/508949553 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:03.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.778+0000 7f7e49523700 1 --2- 172.21.15.40:0/508949553 >> v2:172.21.15.169:3300/0 conn(0x7f7e44070360 0x7f7e44160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:03.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.778+0000 7f7e48d22700 1 -- 172.21.15.40:0/508949553 >> v2:172.21.15.169:3300/0 conn(0x7f7e44070360 msgr2=0x7f7e44160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:03.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.778+0000 7f7e48d22700 1 --2- 172.21.15.40:0/508949553 >> v2:172.21.15.169:3300/0 conn(0x7f7e44070360 0x7f7e44160340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.778+0000 7f7e48d22700 1 -- 172.21.15.40:0/508949553 >> v1:172.21.15.169:6789/0 conn(0x7f7e4406ff10 legacy=0x7f7e4415fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:03.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.778+0000 7f7e48d22700 1 -- 172.21.15.40:0/508949553 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e4406cc50 con 0x7f7e4415c150 2023-11-08T08:20:03.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.778+0000 7f7e48d22700 1 --2- 172.21.15.40:0/508949553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e4415c150 0x7f7e440b3d00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7e4406de90 tx=0x7f7e38009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3ffbfb7c71f079 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:03.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.782+0000 7f7e437fe700 1 -- 172.21.15.40:0/508949553 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38011830 con 0x7f7e4415c150 2023-11-08T08:20:03.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.782+0000 7f7e437fe700 1 -- 172.21.15.40:0/508949553 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e38011990 con 0x7f7e4415c150 2023-11-08T08:20:03.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.782+0000 7f7e437fe700 1 -- 172.21.15.40:0/508949553 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38011d20 con 0x7f7e4415c150 2023-11-08T08:20:03.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.782+0000 7f7e49d24700 1 -- 172.21.15.40:0/508949553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e4415c150 msgr2=0x7f7e440b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:03.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.782+0000 7f7e49d24700 1 --2- 172.21.15.40:0/508949553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e4415c150 0x7f7e440b3d00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7e4406de90 tx=0x7f7e38009bf0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.782+0000 7f7e49d24700 1 -- 172.21.15.40:0/508949553 shutdown_connections 2023-11-08T08:20:03.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.782+0000 7f7e49d24700 1 --2- 172.21.15.40:0/508949553 >> v2:172.21.15.169:3300/0 conn(0x7f7e44070360 0x7f7e44160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.782+0000 7f7e49d24700 1 --2- 172.21.15.40:0/508949553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e4415c150 0x7f7e440b3d00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.782+0000 7f7e49d24700 1 -- 172.21.15.40:0/508949553 >> 172.21.15.40:0/508949553 conn(0x7f7e440d3680 msgr2=0x7f7e440cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:03.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.782+0000 7f7e49d24700 1 -- 172.21.15.40:0/508949553 shutdown_connections 2023-11-08T08:20:03.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.782+0000 7f7e49d24700 1 -- 172.21.15.40:0/508949553 wait complete. 2023-11-08T08:20:03.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.786+0000 7f7e49d24700 1 Processor -- start 2023-11-08T08:20:03.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.786+0000 7f7e49d24700 1 -- start start 2023-11-08T08:20:03.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.786+0000 7f7e49d24700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e4406ff10 0x7f7e440c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:03.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.786+0000 7f7e49d24700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e44070360 0x7f7e441f2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:03.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.786+0000 7f7e49d24700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e4415c150 0x7f7e440ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:03.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.786+0000 7f7e49d24700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e441f27c0 con 0x7f7e4406ff10 2023-11-08T08:20:03.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.786+0000 7f7e49d24700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e440ca350 con 0x7f7e4415c150 2023-11-08T08:20:03.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.786+0000 7f7e49d24700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e440ca630 con 0x7f7e44070360 2023-11-08T08:20:03.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.786+0000 7f7e48d22700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e4406ff10 0x7f7e440c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:03.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.786+0000 7f7e48d22700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e4406ff10 0x7f7e440c9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36100/0 (socket says 172.21.15.40:36100) 2023-11-08T08:20:03.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.786+0000 7f7e48d22700 1 -- 172.21.15.40:0/3000267937 learned_addr learned my addr 172.21.15.40:0/3000267937 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:03.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.786+0000 7f7e49523700 1 --2- 172.21.15.40:0/3000267937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e4415c150 0x7f7e440ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:03.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.786+0000 7f7e43fff700 1 --2- 172.21.15.40:0/3000267937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e44070360 0x7f7e441f2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:03.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.786+0000 7f7e48d22700 1 -- 172.21.15.40:0/3000267937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e44070360 msgr2=0x7f7e441f2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:03.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.790+0000 7f7e48d22700 1 --2- 172.21.15.40:0/3000267937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e44070360 0x7f7e441f2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.790+0000 7f7e48d22700 1 -- 172.21.15.40:0/3000267937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e4415c150 msgr2=0x7f7e440ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:03.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.790+0000 7f7e48d22700 1 --2- 172.21.15.40:0/3000267937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e4415c150 0x7f7e440ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.790+0000 7f7e48d22700 1 -- 172.21.15.40:0/3000267937 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e38009570 con 0x7f7e4406ff10 2023-11-08T08:20:03.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.790+0000 7f7e49523700 1 --2- 172.21.15.40:0/3000267937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e4415c150 0x7f7e440ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:03.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.790+0000 7f7e43fff700 1 --2- 172.21.15.40:0/3000267937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e44070360 0x7f7e441f2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:20:03.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.790+0000 7f7e48d22700 1 --2- 172.21.15.40:0/3000267937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e4406ff10 0x7f7e440c9c50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7e38000c00 tx=0x7f7e38004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:03.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.790+0000 7f7e417fa700 1 -- 172.21.15.40:0/3000267937 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38004610 con 0x7f7e4406ff10 2023-11-08T08:20:03.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.790+0000 7f7e49d24700 1 -- 172.21.15.40:0/3000267937 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e440d2730 con 0x7f7e4406ff10 2023-11-08T08:20:03.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.790+0000 7f7e417fa700 1 -- 172.21.15.40:0/3000267937 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3801e070 con 0x7f7e4406ff10 2023-11-08T08:20:03.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.790+0000 7f7e417fa700 1 -- 172.21.15.40:0/3000267937 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e380038e0 con 0x7f7e4406ff10 2023-11-08T08:20:03.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.790+0000 7f7e49d24700 1 -- 172.21.15.40:0/3000267937 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e440d2450 con 0x7f7e4406ff10 2023-11-08T08:20:03.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.790+0000 7f7e49d24700 1 -- 172.21.15.40:0/3000267937 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e440d05e0 con 0x7f7e4406ff10 2023-11-08T08:20:03.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.794+0000 7f7e417fa700 1 -- 172.21.15.40:0/3000267937 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7e3800f050 con 0x7f7e4406ff10 2023-11-08T08:20:03.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.794+0000 7f7e417fa700 1 --2- 172.21.15.40:0/3000267937 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e28064ee0 0x7f7e28067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:03.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.794+0000 7f7e417fa700 1 -- 172.21.15.40:0/3000267937 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7e38082fd0 con 0x7f7e4406ff10 2023-11-08T08:20:03.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.794+0000 7f7e43fff700 1 --2- 172.21.15.40:0/3000267937 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e28064ee0 0x7f7e28067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:03.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.798+0000 7f7e43fff700 1 --2- 172.21.15.40:0/3000267937 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e28064ee0 0x7f7e28067390 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7e440b5a50 tx=0x7f7e30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:03.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.798+0000 7f7e417fa700 1 -- 172.21.15.40:0/3000267937 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7e38050cd0 con 0x7f7e4406ff10 2023-11-08T08:20:03.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.966+0000 7f7e49d24700 1 -- 172.21.15.40:0/3000267937 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7e44073d20 con 0x7f7e4406ff10 2023-11-08T08:20:03.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.970+0000 7f7e417fa700 1 -- 172.21.15.40:0/3000267937 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7e380200a0 con 0x7f7e4406ff10 2023-11-08T08:20:03.973 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:20:03.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.970+0000 7f7e49d24700 1 -- 172.21.15.40:0/3000267937 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e28064ee0 msgr2=0x7f7e28067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:03.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.970+0000 7f7e49d24700 1 --2- 172.21.15.40:0/3000267937 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e28064ee0 0x7f7e28067390 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7e440b5a50 tx=0x7f7e30009040 comp rx=0 tx=0).stop 2023-11-08T08:20:03.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.970+0000 7f7e49d24700 1 -- 172.21.15.40:0/3000267937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e4406ff10 msgr2=0x7f7e440c9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:03.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.970+0000 7f7e49d24700 1 --2- 172.21.15.40:0/3000267937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e4406ff10 0x7f7e440c9c50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7e38000c00 tx=0x7f7e38004770 comp rx=0 tx=0).stop 2023-11-08T08:20:03.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.970+0000 7f7e49d24700 1 -- 172.21.15.40:0/3000267937 shutdown_connections 2023-11-08T08:20:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.970+0000 7f7e49d24700 1 --2- 172.21.15.40:0/3000267937 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e28064ee0 0x7f7e28067390 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.970+0000 7f7e49d24700 1 --2- 172.21.15.40:0/3000267937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e4415c150 0x7f7e440ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.970+0000 7f7e49d24700 1 --2- 172.21.15.40:0/3000267937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e44070360 0x7f7e441f2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.970+0000 7f7e49d24700 1 --2- 172.21.15.40:0/3000267937 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e4406ff10 0x7f7e440c9c50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:03.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.970+0000 7f7e49d24700 1 -- 172.21.15.40:0/3000267937 >> 172.21.15.40:0/3000267937 conn(0x7f7e440d3680 msgr2=0x7f7e440b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:03.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.974+0000 7f7e49d24700 1 -- 172.21.15.40:0/3000267937 shutdown_connections 2023-11-08T08:20:03.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:03.974+0000 7f7e49d24700 1 -- 172.21.15.40:0/3000267937 wait complete. 2023-11-08T08:20:03.997 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T08:20:04.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.174+0000 7f9bf759e700 1 Processor -- start 2023-11-08T08:20:04.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.178+0000 7f9bf759e700 1 -- start start 2023-11-08T08:20:04.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.178+0000 7f9bf759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf00e0750 0x7f9bf00514c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:04.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.178+0000 7f9bf759e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9bf000c410 0x7f9bf000c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:04.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.178+0000 7f9bf759e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf0053470 con 0x7f9bf0053020 2023-11-08T08:20:04.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.178+0000 7f9bf759e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf000cdd0 con 0x7f9bf000c410 2023-11-08T08:20:04.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.178+0000 7f9bf759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf003a460 con 0x7f9bf00e0750 2023-11-08T08:20:04.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.182+0000 7f9bf5d9b700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9bf0053020 0x7f9bf0039d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41936/0 (socket says 172.21.15.40:41936) 2023-11-08T08:20:04.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.182+0000 7f9bf5d9b700 1 -- 172.21.15.40:0/1333907965 learned_addr learned my addr 172.21.15.40:0/1333907965 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:04.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.182+0000 7f9bf6d9d700 1 --2- 172.21.15.40:0/1333907965 >> v2:172.21.15.169:3300/0 conn(0x7f9bf000c410 0x7f9bf000c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:04.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.182+0000 7f9bf659c700 1 --2- 172.21.15.40:0/1333907965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf00e0750 0x7f9bf00514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:04.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.182+0000 7f9bf559a700 1 -- 172.21.15.40:0/1333907965 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197729281 0 0) 0x7f9bf0053470 con 0x7f9bf0053020 2023-11-08T08:20:04.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.182+0000 7f9bf559a700 1 -- 172.21.15.40:0/1333907965 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9be00035d0 con 0x7f9bf0053020 2023-11-08T08:20:04.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.182+0000 7f9bf6d9d700 1 -- 172.21.15.40:0/1333907965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf00e0750 msgr2=0x7f9bf00514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:04.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.182+0000 7f9bf6d9d700 1 --2- 172.21.15.40:0/1333907965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf00e0750 0x7f9bf00514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:04.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.182+0000 7f9bf6d9d700 1 -- 172.21.15.40:0/1333907965 >> v1:172.21.15.40:6789/0 conn(0x7f9bf0053020 legacy=0x7f9bf0039d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:04.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.182+0000 7f9bf6d9d700 1 -- 172.21.15.40:0/1333907965 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf00554f0 con 0x7f9bf000c410 2023-11-08T08:20:04.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.182+0000 7f9bf6d9d700 1 --2- 172.21.15.40:0/1333907965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf000c410 0x7f9bf000c890 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8066a20 tx=0x7f9bf804e8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74d8a3b9a2690038 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:04.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.182+0000 7f9bf559a700 1 -- 172.21.15.40:0/1333907965 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8072c00 con 0x7f9bf000c410 2023-11-08T08:20:04.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.182+0000 7f9bf559a700 1 -- 172.21.15.40:0/1333907965 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf8072d80 con 0x7f9bf000c410 2023-11-08T08:20:04.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.182+0000 7f9bf559a700 1 -- 172.21.15.40:0/1333907965 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8066d80 con 0x7f9bf000c410 2023-11-08T08:20:04.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.182+0000 7f9bf759e700 1 -- 172.21.15.40:0/1333907965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf000c410 msgr2=0x7f9bf000c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:04.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.182+0000 7f9bf759e700 1 --2- 172.21.15.40:0/1333907965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf000c410 0x7f9bf000c890 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8066a20 tx=0x7f9bf804e8b0 comp rx=0 tx=0).stop 2023-11-08T08:20:04.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.186+0000 7f9bf759e700 1 -- 172.21.15.40:0/1333907965 shutdown_connections 2023-11-08T08:20:04.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.186+0000 7f9bf759e700 1 --2- 172.21.15.40:0/1333907965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf000c410 0x7f9bf000c890 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:04.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.186+0000 7f9bf759e700 1 --2- 172.21.15.40:0/1333907965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf00e0750 0x7f9bf00514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:04.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.186+0000 7f9bf759e700 1 -- 172.21.15.40:0/1333907965 >> 172.21.15.40:0/1333907965 conn(0x7f9bf0056f00 msgr2=0x7f9bf0057300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:04.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.186+0000 7f9bf759e700 1 -- 172.21.15.40:0/1333907965 shutdown_connections 2023-11-08T08:20:04.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.186+0000 7f9bf759e700 1 -- 172.21.15.40:0/1333907965 wait complete. 2023-11-08T08:20:04.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.190+0000 7f9bf759e700 1 Processor -- start 2023-11-08T08:20:04.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.190+0000 7f9bf759e700 1 -- start start 2023-11-08T08:20:04.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.190+0000 7f9bf759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf000c410 0x7f9bf016dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:04.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.190+0000 7f9bf759e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf0053020 0x7f9bf00336f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:04.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.190+0000 7f9bf759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf00e0750 0x7f9bf00dc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:04.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.190+0000 7f9bf759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf0034710 con 0x7f9bf000c410 2023-11-08T08:20:04.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.190+0000 7f9bf759e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf0037c00 con 0x7f9bf0053020 2023-11-08T08:20:04.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.190+0000 7f9bf759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf0034150 con 0x7f9bf00e0750 2023-11-08T08:20:04.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.190+0000 7f9bf659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf000c410 0x7f9bf016dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:04.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.190+0000 7f9bf659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf000c410 0x7f9bf016dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36112/0 (socket says 172.21.15.40:36112) 2023-11-08T08:20:04.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.190+0000 7f9bf659c700 1 -- 172.21.15.40:0/1872387140 learned_addr learned my addr 172.21.15.40:0/1872387140 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:04.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.190+0000 7f9bf5d9b700 1 --2- 172.21.15.40:0/1872387140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf0053020 0x7f9bf00336f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:04.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.190+0000 7f9bf6d9d700 1 --2- 172.21.15.40:0/1872387140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf00e0750 0x7f9bf00dc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:04.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.194+0000 7f9bf659c700 1 -- 172.21.15.40:0/1872387140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf00e0750 msgr2=0x7f9bf00dc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:04.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.194+0000 7f9bf659c700 1 --2- 172.21.15.40:0/1872387140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf00e0750 0x7f9bf00dc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:04.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.194+0000 7f9bf659c700 1 -- 172.21.15.40:0/1872387140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf0053020 msgr2=0x7f9bf00336f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:04.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.194+0000 7f9bf659c700 1 --2- 172.21.15.40:0/1872387140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf0053020 0x7f9bf00336f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:04.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.194+0000 7f9bf659c700 1 -- 172.21.15.40:0/1872387140 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf804e720 con 0x7f9bf000c410 2023-11-08T08:20:04.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.194+0000 7f9bf659c700 1 --2- 172.21.15.40:0/1872387140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf000c410 0x7f9bf016dd30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9be800e870 tx=0x7f9be800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:04.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.194+0000 7f9be6ffd700 1 -- 172.21.15.40:0/1872387140 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be80095a0 con 0x7f9bf000c410 2023-11-08T08:20:04.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.194+0000 7f9be6ffd700 1 -- 172.21.15.40:0/1872387140 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be8013680 con 0x7f9bf000c410 2023-11-08T08:20:04.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.194+0000 7f9bf759e700 1 -- 172.21.15.40:0/1872387140 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf0055790 con 0x7f9bf000c410 2023-11-08T08:20:04.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.194+0000 7f9be6ffd700 1 -- 172.21.15.40:0/1872387140 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be80139f0 con 0x7f9bf000c410 2023-11-08T08:20:04.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.194+0000 7f9bf759e700 1 -- 172.21.15.40:0/1872387140 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf00e0b20 con 0x7f9bf000c410 2023-11-08T08:20:04.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.194+0000 7f9bf759e700 1 -- 172.21.15.40:0/1872387140 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf0055a70 con 0x7f9bf000c410 2023-11-08T08:20:04.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.198+0000 7f9be6ffd700 1 -- 172.21.15.40:0/1872387140 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f9be8019030 con 0x7f9bf000c410 2023-11-08T08:20:04.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.198+0000 7f9be6ffd700 1 --2- 172.21.15.40:0/1872387140 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9bd8064ee0 0x7f9bd8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:04.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.198+0000 7f9be6ffd700 1 -- 172.21.15.40:0/1872387140 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9be8081f70 con 0x7f9bf000c410 2023-11-08T08:20:04.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.198+0000 7f9bf5d9b700 1 --2- 172.21.15.40:0/1872387140 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9bd8064ee0 0x7f9bd8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:04.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.202+0000 7f9bf5d9b700 1 --2- 172.21.15.40:0/1872387140 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9bd8064ee0 0x7f9bd8067390 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00370d0 tx=0x7f9bec007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:04.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.206+0000 7f9be6ffd700 1 -- 172.21.15.40:0/1872387140 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f9be804fcf0 con 0x7f9bf000c410 2023-11-08T08:20:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.382+0000 7f9bf759e700 1 -- 172.21.15.40:0/1872387140 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9bf0005570 con 0x7f9bf000c410 2023-11-08T08:20:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.382+0000 7f9be6ffd700 1 -- 172.21.15.40:0/1872387140 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9be8053310 con 0x7f9bf000c410 2023-11-08T08:20:04.386 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:20:04.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.386+0000 7f9bf759e700 1 -- 172.21.15.40:0/1872387140 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9bd8064ee0 msgr2=0x7f9bd8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:04.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.386+0000 7f9bf759e700 1 --2- 172.21.15.40:0/1872387140 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9bd8064ee0 0x7f9bd8067390 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00370d0 tx=0x7f9bec007de0 comp rx=0 tx=0).stop 2023-11-08T08:20:04.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.386+0000 7f9bf759e700 1 -- 172.21.15.40:0/1872387140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf000c410 msgr2=0x7f9bf016dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:04.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.386+0000 7f9bf759e700 1 --2- 172.21.15.40:0/1872387140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf000c410 0x7f9bf016dd30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9be800e870 tx=0x7f9be800eb80 comp rx=0 tx=0).stop 2023-11-08T08:20:04.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.386+0000 7f9bf759e700 1 -- 172.21.15.40:0/1872387140 shutdown_connections 2023-11-08T08:20:04.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.386+0000 7f9bf759e700 1 --2- 172.21.15.40:0/1872387140 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9bd8064ee0 0x7f9bd8067390 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:04.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.386+0000 7f9bf759e700 1 --2- 172.21.15.40:0/1872387140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9bf00e0750 0x7f9bf00dc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:04.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.386+0000 7f9bf759e700 1 --2- 172.21.15.40:0/1872387140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bf0053020 0x7f9bf00336f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:04.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.386+0000 7f9bf759e700 1 --2- 172.21.15.40:0/1872387140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bf000c410 0x7f9bf016dd30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:04.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.386+0000 7f9bf759e700 1 -- 172.21.15.40:0/1872387140 >> 172.21.15.40:0/1872387140 conn(0x7f9bf0056f00 msgr2=0x7f9bf0054a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:04.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.386+0000 7f9bf759e700 1 -- 172.21.15.40:0/1872387140 shutdown_connections 2023-11-08T08:20:04.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.386+0000 7f9bf759e700 1 -- 172.21.15.40:0/1872387140 wait complete. 2023-11-08T08:20:04.411 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T08:20:04.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.610+0000 7f1b7f2bc700 1 Processor -- start 2023-11-08T08:20:04.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.610+0000 7f1b7f2bc700 1 -- start start 2023-11-08T08:20:04.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.614+0000 7f1b7f2bc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b7815d5f0 0x7f1b780ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:04.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.614+0000 7f1b7f2bc700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b780cecb0 con 0x7f1b7815dd50 2023-11-08T08:20:04.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.614+0000 7f1b7f2bc700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b780ce9e0 con 0x7f1b780b2350 2023-11-08T08:20:04.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.614+0000 7f1b7f2bc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7815c900 con 0x7f1b7815d5f0 2023-11-08T08:20:04.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.614+0000 7f1b7e2ba700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f1b780b2350 0x7f1b780b2720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:43840/0 (socket says 172.21.15.40:43840) 2023-11-08T08:20:04.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.614+0000 7f1b7e2ba700 1 -- 172.21.15.40:0/1117074830 learned_addr learned my addr 172.21.15.40:0/1117074830 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:04.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.614+0000 7f1b7dab9700 1 --2- 172.21.15.40:0/1117074830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b7815d5f0 0x7f1b780ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:04.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.614+0000 7f1b7d2b8700 1 -- 172.21.15.40:0/1117074830 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3079977957 0 0) 0x7f1b780ce9e0 con 0x7f1b780b2350 2023-11-08T08:20:04.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.614+0000 7f1b7d2b8700 1 -- 172.21.15.40:0/1117074830 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b640035d0 con 0x7f1b780b2350 2023-11-08T08:20:04.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.614+0000 7f1b7d2b8700 1 -- 172.21.15.40:0/1117074830 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679344505 0 0) 0x7f1b780cecb0 con 0x7f1b7815dd50 2023-11-08T08:20:04.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.614+0000 7f1b7d2b8700 1 -- 172.21.15.40:0/1117074830 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b780ce9e0 con 0x7f1b7815dd50 2023-11-08T08:20:04.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.614+0000 7f1b7dab9700 1 -- 172.21.15.40:0/1117074830 >> v1:172.21.15.169:6789/0 conn(0x7f1b780b2350 legacy=0x7f1b780b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:04.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.614+0000 7f1b7dab9700 1 -- 172.21.15.40:0/1117074830 >> v1:172.21.15.40:6789/0 conn(0x7f1b7815dd50 legacy=0x7f1b7815e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:04.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.614+0000 7f1b7dab9700 1 -- 172.21.15.40:0/1117074830 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7806e770 con 0x7f1b7815d5f0 2023-11-08T08:20:04.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.614+0000 7f1b7dab9700 1 --2- 172.21.15.40:0/1117074830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b7815d5f0 0x7f1b780ce4a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009880 tx=0x7f1b68009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebaab5dc7a7b8f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:04.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.614+0000 7f1b7d2b8700 1 -- 172.21.15.40:0/1117074830 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b68011830 con 0x7f1b7815d5f0 2023-11-08T08:20:04.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.614+0000 7f1b7d2b8700 1 -- 172.21.15.40:0/1117074830 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b68011990 con 0x7f1b7815d5f0 2023-11-08T08:20:04.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.614+0000 7f1b7d2b8700 1 -- 172.21.15.40:0/1117074830 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b68011d20 con 0x7f1b7815d5f0 2023-11-08T08:20:04.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.618+0000 7f1b7f2bc700 1 -- 172.21.15.40:0/1117074830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b7815d5f0 msgr2=0x7f1b780ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:04.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.618+0000 7f1b7f2bc700 1 --2- 172.21.15.40:0/1117074830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b7815d5f0 0x7f1b780ce4a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009880 tx=0x7f1b68009b90 comp rx=0 tx=0).stop 2023-11-08T08:20:04.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.618+0000 7f1b7f2bc700 1 -- 172.21.15.40:0/1117074830 shutdown_connections 2023-11-08T08:20:04.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.618+0000 7f1b7f2bc700 1 --2- 172.21.15.40:0/1117074830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b7815d5f0 0x7f1b780ce4a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:04.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.618+0000 7f1b7f2bc700 1 -- 172.21.15.40:0/1117074830 >> 172.21.15.40:0/1117074830 conn(0x7f1b7806a840 msgr2=0x7f1b780cd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:04.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.618+0000 7f1b7f2bc700 1 -- 172.21.15.40:0/1117074830 shutdown_connections 2023-11-08T08:20:04.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.618+0000 7f1b7f2bc700 1 -- 172.21.15.40:0/1117074830 wait complete. 2023-11-08T08:20:04.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.618+0000 7f1b7f2bc700 1 Processor -- start 2023-11-08T08:20:04.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.618+0000 7f1b7f2bc700 1 -- start start 2023-11-08T08:20:04.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b7f2bc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b7815d5f0 0x7f1b780c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:04.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b7f2bc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b7815dd50 0x7f1b780b1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:04.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b7f2bc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b780b4750 0x7f1b780b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:04.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b7f2bc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b780c49e0 con 0x7f1b7815dd50 2023-11-08T08:20:04.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b7f2bc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b780c4cc0 con 0x7f1b7815d5f0 2023-11-08T08:20:04.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b7f2bc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b780c4fa0 con 0x7f1b780b4750 2023-11-08T08:20:04.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b7dab9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b7815dd50 0x7f1b780b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:04.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b7dab9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b7815dd50 0x7f1b780b1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36120/0 (socket says 172.21.15.40:36120) 2023-11-08T08:20:04.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b7dab9700 1 -- 172.21.15.40:0/702015262 learned_addr learned my addr 172.21.15.40:0/702015262 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b7e2ba700 1 --2- 172.21.15.40:0/702015262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b7815d5f0 0x7f1b780c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b7eabb700 1 --2- 172.21.15.40:0/702015262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b780b4750 0x7f1b780b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b7dab9700 1 -- 172.21.15.40:0/702015262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b780b4750 msgr2=0x7f1b780b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b7dab9700 1 --2- 172.21.15.40:0/702015262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b780b4750 0x7f1b780b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:04.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b7dab9700 1 -- 172.21.15.40:0/702015262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b7815d5f0 msgr2=0x7f1b780c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:04.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b7dab9700 1 --2- 172.21.15.40:0/702015262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b7815d5f0 0x7f1b780c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:04.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b7dab9700 1 -- 172.21.15.40:0/702015262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b680094b0 con 0x7f1b7815dd50 2023-11-08T08:20:04.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b7e2ba700 1 --2- 172.21.15.40:0/702015262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b7815d5f0 0x7f1b780c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:04.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b7dab9700 1 --2- 172.21.15.40:0/702015262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b7815dd50 0x7f1b780b1620 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1b68013040 tx=0x7f1b68004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b6effd700 1 -- 172.21.15.40:0/702015262 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b680198d0 con 0x7f1b7815dd50 2023-11-08T08:20:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b6effd700 1 -- 172.21.15.40:0/702015262 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6801e070 con 0x7f1b7815dd50 2023-11-08T08:20:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b7f2bc700 1 -- 172.21.15.40:0/702015262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b7806ed30 con 0x7f1b7815dd50 2023-11-08T08:20:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b6effd700 1 -- 172.21.15.40:0/702015262 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b68019c50 con 0x7f1b7815dd50 2023-11-08T08:20:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.622+0000 7f1b7f2bc700 1 -- 172.21.15.40:0/702015262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7806ea50 con 0x7f1b7815dd50 2023-11-08T08:20:04.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.626+0000 7f1b7f2bc700 1 -- 172.21.15.40:0/702015262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b7806f010 con 0x7f1b7815dd50 2023-11-08T08:20:04.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.626+0000 7f1b6effd700 1 -- 172.21.15.40:0/702015262 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1b6800f050 con 0x7f1b7815dd50 2023-11-08T08:20:04.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.626+0000 7f1b6effd700 1 --2- 172.21.15.40:0/702015262 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1b5c064ee0 0x7f1b5c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:04.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.626+0000 7f1b6effd700 1 -- 172.21.15.40:0/702015262 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1b68083100 con 0x7f1b7815dd50 2023-11-08T08:20:04.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.626+0000 7f1b7e2ba700 1 --2- 172.21.15.40:0/702015262 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1b5c064ee0 0x7f1b5c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:04.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.630+0000 7f1b7e2ba700 1 --2- 172.21.15.40:0/702015262 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1b5c064ee0 0x7f1b5c067390 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1b74000f60 tx=0x7f1b74009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:04.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.634+0000 7f1b6effd700 1 -- 172.21.15.40:0/702015262 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1b68050f70 con 0x7f1b7815dd50 2023-11-08T08:20:04.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.806+0000 7f1b7f2bc700 1 -- 172.21.15.40:0/702015262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b78073d20 con 0x7f1b7815dd50 2023-11-08T08:20:04.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.806+0000 7f1b6effd700 1 -- 172.21.15.40:0/702015262 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1b680200a0 con 0x7f1b7815dd50 2023-11-08T08:20:04.810 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-11-08T08:20:04.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.810+0000 7f1b7f2bc700 1 -- 172.21.15.40:0/702015262 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1b5c064ee0 msgr2=0x7f1b5c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:04.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.810+0000 7f1b7f2bc700 1 --2- 172.21.15.40:0/702015262 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1b5c064ee0 0x7f1b5c067390 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1b74000f60 tx=0x7f1b74009040 comp rx=0 tx=0).stop 2023-11-08T08:20:04.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.810+0000 7f1b7f2bc700 1 -- 172.21.15.40:0/702015262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b7815dd50 msgr2=0x7f1b780b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:04.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.810+0000 7f1b7f2bc700 1 --2- 172.21.15.40:0/702015262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b7815dd50 0x7f1b780b1620 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1b68013040 tx=0x7f1b68004770 comp rx=0 tx=0).stop 2023-11-08T08:20:04.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.810+0000 7f1b7f2bc700 1 -- 172.21.15.40:0/702015262 shutdown_connections 2023-11-08T08:20:04.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.810+0000 7f1b7f2bc700 1 --2- 172.21.15.40:0/702015262 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1b5c064ee0 0x7f1b5c067390 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:04.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.810+0000 7f1b7f2bc700 1 --2- 172.21.15.40:0/702015262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b780b4750 0x7f1b780b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:04.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.810+0000 7f1b7f2bc700 1 --2- 172.21.15.40:0/702015262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b7815dd50 0x7f1b780b1620 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:04.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.810+0000 7f1b7f2bc700 1 --2- 172.21.15.40:0/702015262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b7815d5f0 0x7f1b780c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:04.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.810+0000 7f1b7f2bc700 1 -- 172.21.15.40:0/702015262 >> 172.21.15.40:0/702015262 conn(0x7f1b780b2350 msgr2=0x7f1b7806b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:04.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.810+0000 7f1b7f2bc700 1 -- 172.21.15.40:0/702015262 shutdown_connections 2023-11-08T08:20:04.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:04.810+0000 7f1b7f2bc700 1 -- 172.21.15.40:0/702015262 wait complete. 2023-11-08T08:20:04.837 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T08:20:05.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.022+0000 7fa8a629f700 1 Processor -- start 2023-11-08T08:20:05.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.022+0000 7fa8a629f700 1 -- start start 2023-11-08T08:20:05.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.026+0000 7fa8a629f700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa8a0070620 0x7fa8a00b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:05.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.026+0000 7fa8a629f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8a0069bf0 0x7fa8a006a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:05.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.026+0000 7fa8a629f700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a015c520 con 0x7fa8a015c150 2023-11-08T08:20:05.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.026+0000 7fa8a629f700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a00b4240 con 0x7fa8a0070620 2023-11-08T08:20:05.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.026+0000 7fa8a629f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a006cc50 con 0x7fa8a0069bf0 2023-11-08T08:20:05.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.026+0000 7fa8a529d700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fa8a015c150 0x7fa8a006ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:43844/0 (socket says 172.21.15.40:43844) 2023-11-08T08:20:05.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.026+0000 7fa8a529d700 1 -- 172.21.15.40:0/3714208494 learned_addr learned my addr 172.21.15.40:0/3714208494 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:05.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.026+0000 7fa8a4a9c700 1 --2- 172.21.15.40:0/3714208494 >> v2:172.21.15.169:3300/0 conn(0x7fa8a0070620 0x7fa8a00b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:05.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.026+0000 7fa8a5a9e700 1 --2- 172.21.15.40:0/3714208494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8a0069bf0 0x7fa8a006a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:05.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.026+0000 7fa897fff700 1 -- 172.21.15.40:0/3714208494 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2935809321 0 0) 0x7fa8a015c520 con 0x7fa8a015c150 2023-11-08T08:20:05.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.026+0000 7fa897fff700 1 -- 172.21.15.40:0/3714208494 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa88c0035d0 con 0x7fa8a015c150 2023-11-08T08:20:05.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.026+0000 7fa8a4a9c700 1 -- 172.21.15.40:0/3714208494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8a0069bf0 msgr2=0x7fa8a006a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:05.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.026+0000 7fa8a4a9c700 1 --2- 172.21.15.40:0/3714208494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8a0069bf0 0x7fa8a006a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:05.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.026+0000 7fa8a4a9c700 1 -- 172.21.15.40:0/3714208494 >> v1:172.21.15.169:6789/0 conn(0x7fa8a015c150 legacy=0x7fa8a006ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:05.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.026+0000 7fa8a4a9c700 1 -- 172.21.15.40:0/3714208494 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a00d2450 con 0x7fa8a0070620 2023-11-08T08:20:05.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.026+0000 7fa8a4a9c700 1 --2- 172.21.15.40:0/3714208494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8a0070620 0x7fa8a00b3d00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa890009840 tx=0x7fa890009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f14341df28424db server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:05.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.026+0000 7fa897fff700 1 -- 172.21.15.40:0/3714208494 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa890005e50 con 0x7fa8a0070620 2023-11-08T08:20:05.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.030+0000 7fa897fff700 1 -- 172.21.15.40:0/3714208494 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa890006030 con 0x7fa8a0070620 2023-11-08T08:20:05.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.030+0000 7fa897fff700 1 -- 172.21.15.40:0/3714208494 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8900063c0 con 0x7fa8a0070620 2023-11-08T08:20:05.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.030+0000 7fa8a629f700 1 -- 172.21.15.40:0/3714208494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8a0070620 msgr2=0x7fa8a00b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:05.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.030+0000 7fa8a629f700 1 --2- 172.21.15.40:0/3714208494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8a0070620 0x7fa8a00b3d00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa890009840 tx=0x7fa890009b50 comp rx=0 tx=0).stop 2023-11-08T08:20:05.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.030+0000 7fa8a629f700 1 -- 172.21.15.40:0/3714208494 shutdown_connections 2023-11-08T08:20:05.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.030+0000 7fa8a629f700 1 --2- 172.21.15.40:0/3714208494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8a0069bf0 0x7fa8a006a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:05.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.030+0000 7fa8a629f700 1 --2- 172.21.15.40:0/3714208494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8a0070620 0x7fa8a00b3d00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:05.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.030+0000 7fa8a629f700 1 -- 172.21.15.40:0/3714208494 >> 172.21.15.40:0/3714208494 conn(0x7fa8a00d3680 msgr2=0x7fa8a00cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:05.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.030+0000 7fa8a629f700 1 -- 172.21.15.40:0/3714208494 shutdown_connections 2023-11-08T08:20:05.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.030+0000 7fa8a629f700 1 -- 172.21.15.40:0/3714208494 wait complete. 2023-11-08T08:20:05.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.034+0000 7fa8a629f700 1 Processor -- start 2023-11-08T08:20:05.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.034+0000 7fa8a629f700 1 -- start start 2023-11-08T08:20:05.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.034+0000 7fa8a629f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8a0069bf0 0x7fa8a0146080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:05.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.034+0000 7fa8a629f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8a0070620 0x7fa8a0148760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:05.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.034+0000 7fa8a629f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8a015c150 0x7fa8a00c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:05.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.034+0000 7fa8a629f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a01490c0 con 0x7fa8a0070620 2023-11-08T08:20:05.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.034+0000 7fa8a629f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a01493a0 con 0x7fa8a015c150 2023-11-08T08:20:05.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.034+0000 7fa8a629f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a0149680 con 0x7fa8a0069bf0 2023-11-08T08:20:05.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.034+0000 7fa8a4a9c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8a0070620 0x7fa8a0148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:05.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.034+0000 7fa8a529d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8a0069bf0 0x7fa8a0146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:05.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.034+0000 7fa8a4a9c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8a0070620 0x7fa8a0148760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36124/0 (socket says 172.21.15.40:36124) 2023-11-08T08:20:05.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.034+0000 7fa8a4a9c700 1 -- 172.21.15.40:0/2785691087 learned_addr learned my addr 172.21.15.40:0/2785691087 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:05.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.034+0000 7fa8a5a9e700 1 --2- 172.21.15.40:0/2785691087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8a015c150 0x7fa8a00c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:05.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.038+0000 7fa8a4a9c700 1 -- 172.21.15.40:0/2785691087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8a0069bf0 msgr2=0x7fa8a0146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:05.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.038+0000 7fa8a4a9c700 1 --2- 172.21.15.40:0/2785691087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8a0069bf0 0x7fa8a0146080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:05.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.038+0000 7fa8a4a9c700 1 -- 172.21.15.40:0/2785691087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8a015c150 msgr2=0x7fa8a00c78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:05.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.038+0000 7fa8a4a9c700 1 --2- 172.21.15.40:0/2785691087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8a015c150 0x7fa8a00c78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:05.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.038+0000 7fa8a4a9c700 1 -- 172.21.15.40:0/2785691087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa890009450 con 0x7fa8a0070620 2023-11-08T08:20:05.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.038+0000 7fa8a529d700 1 --2- 172.21.15.40:0/2785691087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8a0069bf0 0x7fa8a0146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:05.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.038+0000 7fa8a4a9c700 1 --2- 172.21.15.40:0/2785691087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8a0070620 0x7fa8a0148760 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa890000c00 tx=0x7fa890004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:05.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.038+0000 7fa895ffb700 1 -- 172.21.15.40:0/2785691087 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8900038c0 con 0x7fa8a0070620 2023-11-08T08:20:05.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.038+0000 7fa895ffb700 1 -- 172.21.15.40:0/2785691087 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa890003a40 con 0x7fa8a0070620 2023-11-08T08:20:05.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.038+0000 7fa895ffb700 1 -- 172.21.15.40:0/2785691087 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa890003df0 con 0x7fa8a0070620 2023-11-08T08:20:05.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.038+0000 7fa8a629f700 1 -- 172.21.15.40:0/2785691087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a00d05e0 con 0x7fa8a0070620 2023-11-08T08:20:05.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.038+0000 7fa8a629f700 1 -- 172.21.15.40:0/2785691087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a00d2730 con 0x7fa8a0070620 2023-11-08T08:20:05.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.038+0000 7fa8a629f700 1 -- 172.21.15.40:0/2785691087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a00d2140 con 0x7fa8a0070620 2023-11-08T08:20:05.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.042+0000 7fa895ffb700 1 -- 172.21.15.40:0/2785691087 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa89000f050 con 0x7fa8a0070620 2023-11-08T08:20:05.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.042+0000 7fa895ffb700 1 --2- 172.21.15.40:0/2785691087 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa884064ee0 0x7fa884067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:05.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.042+0000 7fa895ffb700 1 -- 172.21.15.40:0/2785691087 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa890083b80 con 0x7fa8a0070620 2023-11-08T08:20:05.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.042+0000 7fa8a529d700 1 --2- 172.21.15.40:0/2785691087 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa884064ee0 0x7fa884067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:05.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.042+0000 7fa8a529d700 1 --2- 172.21.15.40:0/2785691087 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa884064ee0 0x7fa884067390 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa89c001560 tx=0x7fa89c0019f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:05.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.050+0000 7fa895ffb700 1 -- 172.21.15.40:0/2785691087 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa890051900 con 0x7fa8a0070620 2023-11-08T08:20:05.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.218+0000 7fa8a629f700 1 -- 172.21.15.40:0/2785691087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa8a0073d20 con 0x7fa8a0070620 2023-11-08T08:20:05.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.218+0000 7fa895ffb700 1 -- 172.21.15.40:0/2785691087 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa890083e60 con 0x7fa8a0070620 2023-11-08T08:20:05.222 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:20:05.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.222+0000 7fa8a629f700 1 -- 172.21.15.40:0/2785691087 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa884064ee0 msgr2=0x7fa884067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:05.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.222+0000 7fa8a629f700 1 --2- 172.21.15.40:0/2785691087 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa884064ee0 0x7fa884067390 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa89c001560 tx=0x7fa89c0019f0 comp rx=0 tx=0).stop 2023-11-08T08:20:05.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.222+0000 7fa8a629f700 1 -- 172.21.15.40:0/2785691087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8a0070620 msgr2=0x7fa8a0148760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:05.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.222+0000 7fa8a629f700 1 --2- 172.21.15.40:0/2785691087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8a0070620 0x7fa8a0148760 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa890000c00 tx=0x7fa890004770 comp rx=0 tx=0).stop 2023-11-08T08:20:05.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.222+0000 7fa8a629f700 1 -- 172.21.15.40:0/2785691087 shutdown_connections 2023-11-08T08:20:05.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.222+0000 7fa8a629f700 1 --2- 172.21.15.40:0/2785691087 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa884064ee0 0x7fa884067390 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:05.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.222+0000 7fa8a629f700 1 --2- 172.21.15.40:0/2785691087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8a015c150 0x7fa8a00c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:05.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.222+0000 7fa8a629f700 1 --2- 172.21.15.40:0/2785691087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8a0070620 0x7fa8a0148760 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:05.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.222+0000 7fa8a629f700 1 --2- 172.21.15.40:0/2785691087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8a0069bf0 0x7fa8a0146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:05.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.222+0000 7fa8a629f700 1 -- 172.21.15.40:0/2785691087 >> 172.21.15.40:0/2785691087 conn(0x7fa8a00d3680 msgr2=0x7fa8a00b6610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:05.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.222+0000 7fa8a629f700 1 -- 172.21.15.40:0/2785691087 shutdown_connections 2023-11-08T08:20:05.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.222+0000 7fa8a629f700 1 -- 172.21.15.40:0/2785691087 wait complete. 2023-11-08T08:20:05.249 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T08:20:05.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.438+0000 7fd3c42cf700 1 Processor -- start 2023-11-08T08:20:05.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.438+0000 7fd3c42cf700 1 -- start start 2023-11-08T08:20:05.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.442+0000 7fd3c42cf700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd3bc1534a0 0x7fd3bc15d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:05.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.442+0000 7fd3c42cf700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3bc0b6200 0x7fd3bc070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:05.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.442+0000 7fd3c42cf700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3bc15d080 con 0x7fd3bc160920 2023-11-08T08:20:05.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.442+0000 7fd3c42cf700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3bc153870 con 0x7fd3bc1534a0 2023-11-08T08:20:05.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.442+0000 7fd3c42cf700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc0b53c0 con 0x7fd3bc0b6200 2023-11-08T08:20:05.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.442+0000 7fd3c286c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3bc0b6200 0x7fd3bc070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:05.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.442+0000 7fd3c206b700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd3bc1534a0 0x7fd3bc15d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:05.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.442+0000 7fd3c186a700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fd3bc160920 0x7fd3bc0b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:43854/0 (socket says 172.21.15.40:43854) 2023-11-08T08:20:05.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.442+0000 7fd3c186a700 1 -- 172.21.15.40:0/1982747535 learned_addr learned my addr 172.21.15.40:0/1982747535 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:05.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.442+0000 7fd3c1069700 1 -- 172.21.15.40:0/1982747535 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3275462288 0 0) 0x7fd3bc15d080 con 0x7fd3bc160920 2023-11-08T08:20:05.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.442+0000 7fd3c286c700 1 -- 172.21.15.40:0/1982747535 >> v2:172.21.15.169:3300/0 conn(0x7fd3bc1534a0 msgr2=0x7fd3bc15d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:05.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.442+0000 7fd3c286c700 1 --2- 172.21.15.40:0/1982747535 >> v2:172.21.15.169:3300/0 conn(0x7fd3bc1534a0 0x7fd3bc15d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:05.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.442+0000 7fd3c286c700 1 -- 172.21.15.40:0/1982747535 >> v1:172.21.15.169:6789/0 conn(0x7fd3bc160920 legacy=0x7fd3bc0b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:05.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.442+0000 7fd3c286c700 1 -- 172.21.15.40:0/1982747535 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3bc0b56a0 con 0x7fd3bc0b6200 2023-11-08T08:20:05.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.442+0000 7fd3c286c700 1 --2- 172.21.15.40:0/1982747535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3bc0b6200 0x7fd3bc070630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009d90 tx=0x7fd3b400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caa67a6144c26365 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:05.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.446+0000 7fd3c1069700 1 -- 172.21.15.40:0/1982747535 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b400e070 con 0x7fd3bc0b6200 2023-11-08T08:20:05.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.446+0000 7fd3c1069700 1 -- 172.21.15.40:0/1982747535 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b400bb20 con 0x7fd3bc0b6200 2023-11-08T08:20:05.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.446+0000 7fd3c1069700 1 -- 172.21.15.40:0/1982747535 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4012430 con 0x7fd3bc0b6200 2023-11-08T08:20:05.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.446+0000 7fd3c42cf700 1 -- 172.21.15.40:0/1982747535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3bc0b6200 msgr2=0x7fd3bc070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:05.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.446+0000 7fd3c42cf700 1 --2- 172.21.15.40:0/1982747535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3bc0b6200 0x7fd3bc070630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009d90 tx=0x7fd3b400b330 comp rx=0 tx=0).stop 2023-11-08T08:20:05.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.446+0000 7fd3c42cf700 1 -- 172.21.15.40:0/1982747535 shutdown_connections 2023-11-08T08:20:05.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.446+0000 7fd3c42cf700 1 --2- 172.21.15.40:0/1982747535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3bc0b6200 0x7fd3bc070630 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:05.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.446+0000 7fd3c42cf700 1 --2- 172.21.15.40:0/1982747535 >> v2:172.21.15.169:3300/0 conn(0x7fd3bc1534a0 0x7fd3bc15d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:05.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.446+0000 7fd3c42cf700 1 -- 172.21.15.40:0/1982747535 >> 172.21.15.40:0/1982747535 conn(0x7fd3bc06bfc0 msgr2=0x7fd3bc06c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:05.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.446+0000 7fd3c42cf700 1 -- 172.21.15.40:0/1982747535 shutdown_connections 2023-11-08T08:20:05.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.446+0000 7fd3c42cf700 1 -- 172.21.15.40:0/1982747535 wait complete. 2023-11-08T08:20:05.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.446+0000 7fd3c42cf700 1 Processor -- start 2023-11-08T08:20:05.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3c42cf700 1 -- start start 2023-11-08T08:20:05.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3c42cf700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3bc0b6200 0x7fd3bc159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:05.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3c42cf700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3bc1534a0 0x7fd3bc1f2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:05.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3c42cf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3bc160920 0x7fd3bc147680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:05.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3c42cf700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc158030 con 0x7fd3bc160920 2023-11-08T08:20:05.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3c42cf700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc147bc0 con 0x7fd3bc1534a0 2023-11-08T08:20:05.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3c42cf700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc158240 con 0x7fd3bc0b6200 2023-11-08T08:20:05.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3c286c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3bc160920 0x7fd3bc147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:05.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3c286c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3bc160920 0x7fd3bc147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36132/0 (socket says 172.21.15.40:36132) 2023-11-08T08:20:05.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3c286c700 1 -- 172.21.15.40:0/1510592351 learned_addr learned my addr 172.21.15.40:0/1510592351 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:05.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3c186a700 1 --2- 172.21.15.40:0/1510592351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3bc1534a0 0x7fd3bc1f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:05.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3c206b700 1 --2- 172.21.15.40:0/1510592351 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3bc0b6200 0x7fd3bc159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:05.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3c286c700 1 -- 172.21.15.40:0/1510592351 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3bc0b6200 msgr2=0x7fd3bc159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:05.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3c286c700 1 --2- 172.21.15.40:0/1510592351 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3bc0b6200 0x7fd3bc159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:05.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3c286c700 1 -- 172.21.15.40:0/1510592351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3bc1534a0 msgr2=0x7fd3bc1f2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:05.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3c286c700 1 --2- 172.21.15.40:0/1510592351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3bc1534a0 0x7fd3bc1f2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:05.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3c286c700 1 -- 172.21.15.40:0/1510592351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b4009a70 con 0x7fd3bc160920 2023-11-08T08:20:05.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3c286c700 1 --2- 172.21.15.40:0/1510592351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3bc160920 0x7fd3bc147680 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0b5500 tx=0x7fd3b4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:05.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3b2ffd700 1 -- 172.21.15.40:0/1510592351 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4012430 con 0x7fd3bc160920 2023-11-08T08:20:05.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3b2ffd700 1 -- 172.21.15.40:0/1510592351 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b4020070 con 0x7fd3bc160920 2023-11-08T08:20:05.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3c42cf700 1 -- 172.21.15.40:0/1510592351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3bc15cd10 con 0x7fd3bc160920 2023-11-08T08:20:05.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3b2ffd700 1 -- 172.21.15.40:0/1510592351 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4004280 con 0x7fd3bc160920 2023-11-08T08:20:05.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.450+0000 7fd3c42cf700 1 -- 172.21.15.40:0/1510592351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3bc15ca30 con 0x7fd3bc160920 2023-11-08T08:20:05.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.454+0000 7fd3c42cf700 1 -- 172.21.15.40:0/1510592351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3bc0b7f10 con 0x7fd3bc160920 2023-11-08T08:20:05.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.454+0000 7fd3b2ffd700 1 -- 172.21.15.40:0/1510592351 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd3b402e070 con 0x7fd3bc160920 2023-11-08T08:20:05.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.454+0000 7fd3b2ffd700 1 --2- 172.21.15.40:0/1510592351 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd3a0064f10 0x7fd3a00673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:05.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.458+0000 7fd3b2ffd700 1 -- 172.21.15.40:0/1510592351 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd3b4022070 con 0x7fd3bc160920 2023-11-08T08:20:05.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.458+0000 7fd3c206b700 1 --2- 172.21.15.40:0/1510592351 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd3a0064f10 0x7fd3a00673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:05.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.458+0000 7fd3c206b700 1 --2- 172.21.15.40:0/1510592351 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd3a0064f10 0x7fd3a00673c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc1521b0 tx=0x7fd3b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:05.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.462+0000 7fd3b2ffd700 1 -- 172.21.15.40:0/1510592351 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd3b4051d50 con 0x7fd3bc160920 2023-11-08T08:20:05.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.646+0000 7fd3c42cf700 1 -- 172.21.15.40:0/1510592351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd3bc0726c0 con 0x7fd3bc160920 2023-11-08T08:20:05.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.646+0000 7fd3b2ffd700 1 -- 172.21.15.40:0/1510592351 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd3b4055370 con 0x7fd3bc160920 2023-11-08T08:20:05.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.650+0000 7fd3c42cf700 1 -- 172.21.15.40:0/1510592351 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd3a0064f10 msgr2=0x7fd3a00673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:05.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.650+0000 7fd3c42cf700 1 --2- 172.21.15.40:0/1510592351 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd3a0064f10 0x7fd3a00673c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc1521b0 tx=0x7fd3b8008040 comp rx=0 tx=0).stop 2023-11-08T08:20:05.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.650+0000 7fd3c42cf700 1 -- 172.21.15.40:0/1510592351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3bc160920 msgr2=0x7fd3bc147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:05.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.650+0000 7fd3c42cf700 1 --2- 172.21.15.40:0/1510592351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3bc160920 0x7fd3bc147680 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0b5500 tx=0x7fd3b4004af0 comp rx=0 tx=0).stop 2023-11-08T08:20:05.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.650+0000 7fd3c42cf700 1 -- 172.21.15.40:0/1510592351 shutdown_connections 2023-11-08T08:20:05.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.650+0000 7fd3c42cf700 1 --2- 172.21.15.40:0/1510592351 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd3a0064f10 0x7fd3a00673c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:05.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.650+0000 7fd3c42cf700 1 --2- 172.21.15.40:0/1510592351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3bc160920 0x7fd3bc147680 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:05.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.650+0000 7fd3c42cf700 1 --2- 172.21.15.40:0/1510592351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3bc1534a0 0x7fd3bc1f2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:05.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.650+0000 7fd3c42cf700 1 --2- 172.21.15.40:0/1510592351 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3bc0b6200 0x7fd3bc159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:05.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.650+0000 7fd3c42cf700 1 -- 172.21.15.40:0/1510592351 >> 172.21.15.40:0/1510592351 conn(0x7fd3bc06bfc0 msgr2=0x7fd3bc15d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:05.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.650+0000 7fd3c42cf700 1 -- 172.21.15.40:0/1510592351 shutdown_connections 2023-11-08T08:20:05.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:05.650+0000 7fd3c42cf700 1 -- 172.21.15.40:0/1510592351 wait complete. 2023-11-08T08:20:05.657 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-11-08T08:20:05.679 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-08T08:20:05.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:05.680 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-08T08:20:05.732 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-11-08T08:20:05.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:05.733 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T08:20:05.795 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:20:05.796 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:20:05.815 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:20:05.815 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:20:05.876 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:20:05.876 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:20:05.938 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:20:06.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.118+0000 7f0eb6ce4700 1 Processor -- start 2023-11-08T08:20:06.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.118+0000 7f0eb6ce4700 1 -- start start 2023-11-08T08:20:06.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.118+0000 7f0eb6ce4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0eb0160710 0x7f0eb00cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:06.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.118+0000 7f0eb6ce4700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0eb00cc860 0x7f0eb0070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:06.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.118+0000 7f0eb6ce4700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb00ad6c0 con 0x7f0eb00705c0 2023-11-08T08:20:06.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.118+0000 7f0eb6ce4700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb006b1c0 con 0x7f0eb0160710 2023-11-08T08:20:06.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.118+0000 7f0eb6ce4700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb00adc60 con 0x7f0eb00cc860 2023-11-08T08:20:06.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.122+0000 7f0eb5ce2700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0eb0160710 0x7f0eb00cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:06.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.122+0000 7f0eb5ce2700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0eb0160710 0x7f0eb00cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36140/0 (socket says 172.21.15.40:36140) 2023-11-08T08:20:06.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.122+0000 7f0eb5ce2700 1 -- 172.21.15.40:0/1289123658 learned_addr learned my addr 172.21.15.40:0/1289123658 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:06.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.122+0000 7f0eb54e1700 1 --2- 172.21.15.40:0/1289123658 >> v2:172.21.15.169:3300/0 conn(0x7f0eb00cc860 0x7f0eb0070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:06.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.122+0000 7f0eb5ce2700 1 -- 172.21.15.40:0/1289123658 >> v2:172.21.15.169:3300/0 conn(0x7f0eb00cc860 msgr2=0x7f0eb0070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.122+0000 7f0eb5ce2700 1 --2- 172.21.15.40:0/1289123658 >> v2:172.21.15.169:3300/0 conn(0x7f0eb00cc860 0x7f0eb0070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.122+0000 7f0eb5ce2700 1 -- 172.21.15.40:0/1289123658 >> v1:172.21.15.169:6789/0 conn(0x7f0eb00705c0 legacy=0x7f0eb0069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.122+0000 7f0eb5ce2700 1 -- 172.21.15.40:0/1289123658 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb0070a20 con 0x7f0eb0160710 2023-11-08T08:20:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.122+0000 7f0eb5ce2700 1 --2- 172.21.15.40:0/1289123658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb0160710 0x7f0eb00cc320 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0eb01641b0 tx=0x7f0eac009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a3778b8f6f59f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:06.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.122+0000 7f0eb4ce0700 1 -- 172.21.15.40:0/1289123658 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac011830 con 0x7f0eb0160710 2023-11-08T08:20:06.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.122+0000 7f0eb4ce0700 1 -- 172.21.15.40:0/1289123658 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eac011990 con 0x7f0eb0160710 2023-11-08T08:20:06.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.122+0000 7f0eb4ce0700 1 -- 172.21.15.40:0/1289123658 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac011d20 con 0x7f0eb0160710 2023-11-08T08:20:06.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.122+0000 7f0eb6ce4700 1 -- 172.21.15.40:0/1289123658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb0160710 msgr2=0x7f0eb00cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:06.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.122+0000 7f0eb6ce4700 1 --2- 172.21.15.40:0/1289123658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb0160710 0x7f0eb00cc320 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0eb01641b0 tx=0x7f0eac009bf0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.122+0000 7f0eb6ce4700 1 -- 172.21.15.40:0/1289123658 shutdown_connections 2023-11-08T08:20:06.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.122+0000 7f0eb6ce4700 1 --2- 172.21.15.40:0/1289123658 >> v2:172.21.15.169:3300/0 conn(0x7f0eb00cc860 0x7f0eb0070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.122+0000 7f0eb6ce4700 1 --2- 172.21.15.40:0/1289123658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb0160710 0x7f0eb00cc320 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.122+0000 7f0eb6ce4700 1 -- 172.21.15.40:0/1289123658 >> 172.21.15.40:0/1289123658 conn(0x7f0eb00b0f80 msgr2=0x7f0eb00ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:06.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.126+0000 7f0eb6ce4700 1 -- 172.21.15.40:0/1289123658 shutdown_connections 2023-11-08T08:20:06.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.126+0000 7f0eb6ce4700 1 -- 172.21.15.40:0/1289123658 wait complete. 2023-11-08T08:20:06.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.126+0000 7f0eb6ce4700 1 Processor -- start 2023-11-08T08:20:06.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.126+0000 7f0eb6ce4700 1 -- start start 2023-11-08T08:20:06.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.126+0000 7f0eb6ce4700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0eb00705c0 0x7f0eb00b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:06.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.126+0000 7f0eb6ce4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0eb00cc860 0x7f0eb00c32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:06.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.130+0000 7f0eb6ce4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb0160710 0x7f0eb0069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:06.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.130+0000 7f0eb6ce4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb01562d0 con 0x7f0eb0160710 2023-11-08T08:20:06.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.130+0000 7f0eb6ce4700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb00c3800 con 0x7f0eb00cc860 2023-11-08T08:20:06.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.130+0000 7f0eb6ce4700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb00c2240 con 0x7f0eb00705c0 2023-11-08T08:20:06.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.130+0000 7f0eb64e3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb0160710 0x7f0eb0069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:06.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.130+0000 7f0eb54e1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0eb00cc860 0x7f0eb00c32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:06.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.130+0000 7f0eb54e1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0eb00cc860 0x7f0eb00c32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:53350/0 (socket says 172.21.15.40:53350) 2023-11-08T08:20:06.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.130+0000 7f0eb54e1700 1 -- 172.21.15.40:0/802848181 learned_addr learned my addr 172.21.15.40:0/802848181 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:06.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.130+0000 7f0eb5ce2700 1 --2- 172.21.15.40:0/802848181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0eb00705c0 0x7f0eb00b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:06.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.130+0000 7f0eb54e1700 1 -- 172.21.15.40:0/802848181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0eb00705c0 msgr2=0x7f0eb00b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:06.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.130+0000 7f0eb54e1700 1 --2- 172.21.15.40:0/802848181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0eb00705c0 0x7f0eb00b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.130+0000 7f0eb54e1700 1 -- 172.21.15.40:0/802848181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb0160710 msgr2=0x7f0eb0069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:06.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.130+0000 7f0eb54e1700 1 --2- 172.21.15.40:0/802848181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb0160710 0x7f0eb0069bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.130+0000 7f0eb54e1700 1 -- 172.21.15.40:0/802848181 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac009570 con 0x7f0eb00cc860 2023-11-08T08:20:06.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.130+0000 7f0eb54e1700 1 --2- 172.21.15.40:0/802848181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0eb00cc860 0x7f0eb00c32c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000e9d0 tx=0x7f0ea000ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:06.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.130+0000 7f0ea67fc700 1 -- 172.21.15.40:0/802848181 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0009720 con 0x7f0eb00cc860 2023-11-08T08:20:06.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.130+0000 7f0ea67fc700 1 -- 172.21.15.40:0/802848181 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea000ee70 con 0x7f0eb00cc860 2023-11-08T08:20:06.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.130+0000 7f0ea67fc700 1 -- 172.21.15.40:0/802848181 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea000ca50 con 0x7f0eb00cc860 2023-11-08T08:20:06.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.130+0000 7f0eb6ce4700 1 -- 172.21.15.40:0/802848181 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb006b6b0 con 0x7f0eb00cc860 2023-11-08T08:20:06.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.130+0000 7f0eb6ce4700 1 -- 172.21.15.40:0/802848181 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb006b3d0 con 0x7f0eb00cc860 2023-11-08T08:20:06.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.134+0000 7f0eb6ce4700 1 -- 172.21.15.40:0/802848181 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb00ad8f0 con 0x7f0eb00cc860 2023-11-08T08:20:06.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.134+0000 7f0ea67fc700 1 -- 172.21.15.40:0/802848181 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f0ea0023070 con 0x7f0eb00cc860 2023-11-08T08:20:06.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.138+0000 7f0ea67fc700 1 --2- 172.21.15.40:0/802848181 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0e94064ee0 0x7f0e94067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:06.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.138+0000 7f0ea67fc700 1 -- 172.21.15.40:0/802848181 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0ea0082550 con 0x7f0eb00cc860 2023-11-08T08:20:06.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.138+0000 7f0eb5ce2700 1 --2- 172.21.15.40:0/802848181 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0e94064ee0 0x7f0e94067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:06.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.138+0000 7f0eb5ce2700 1 --2- 172.21.15.40:0/802848181 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0e94064ee0 0x7f0e94067390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009860 tx=0x7f0eac008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:06.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.146+0000 7f0ea67fc700 1 -- 172.21.15.40:0/802848181 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f0ea00501e0 con 0x7f0eb00cc860 2023-11-08T08:20:06.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.322+0000 7f0eb6ce4700 1 -- 172.21.15.40:0/802848181 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0eb0073cb0 con 0x7f0eb00cc860 2023-11-08T08:20:06.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.326+0000 7f0ea67fc700 1 -- 172.21.15.40:0/802848181 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19748 (secure 0 0 0) 0x7f0ea0017090 con 0x7f0eb00cc860 2023-11-08T08:20:06.331 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:06.331 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":30,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:45.885827+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:20:06.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.330+0000 7f0eb6ce4700 1 -- 172.21.15.40:0/802848181 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0e94064ee0 msgr2=0x7f0e94067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:06.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.330+0000 7f0eb6ce4700 1 --2- 172.21.15.40:0/802848181 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0e94064ee0 0x7f0e94067390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009860 tx=0x7f0eac008030 comp rx=0 tx=0).stop 2023-11-08T08:20:06.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.330+0000 7f0eb6ce4700 1 -- 172.21.15.40:0/802848181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0eb00cc860 msgr2=0x7f0eb00c32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:06.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.330+0000 7f0eb6ce4700 1 --2- 172.21.15.40:0/802848181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0eb00cc860 0x7f0eb00c32c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000e9d0 tx=0x7f0ea000ece0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.330+0000 7f0eb6ce4700 1 -- 172.21.15.40:0/802848181 shutdown_connections 2023-11-08T08:20:06.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.330+0000 7f0eb6ce4700 1 --2- 172.21.15.40:0/802848181 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0e94064ee0 0x7f0e94067390 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.330+0000 7f0eb6ce4700 1 --2- 172.21.15.40:0/802848181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb0160710 0x7f0eb0069bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.330+0000 7f0eb6ce4700 1 --2- 172.21.15.40:0/802848181 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0eb00cc860 0x7f0eb00c32c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.330+0000 7f0eb6ce4700 1 --2- 172.21.15.40:0/802848181 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0eb00705c0 0x7f0eb00b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.330+0000 7f0eb6ce4700 1 -- 172.21.15.40:0/802848181 >> 172.21.15.40:0/802848181 conn(0x7f0eb00b0f80 msgr2=0x7f0eb00afb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:06.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.330+0000 7f0eb6ce4700 1 -- 172.21.15.40:0/802848181 shutdown_connections 2023-11-08T08:20:06.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.330+0000 7f0eb6ce4700 1 -- 172.21.15.40:0/802848181 wait complete. 2023-11-08T08:20:06.359 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T08:19:02.627691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T08:19:03.473821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T08:19:06.539942+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T08:20:06.360 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:20:06.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.550+0000 7f2692485700 1 Processor -- start 2023-11-08T08:20:06.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.550+0000 7f2692485700 1 -- start start 2023-11-08T08:20:06.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.554+0000 7f2692485700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f268c15c150 0x7f268c0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:06.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.554+0000 7f2692485700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f268c06ff10 0x7f268c070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:06.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.554+0000 7f2692485700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f268c15c520 con 0x7f268c15fc30 2023-11-08T08:20:06.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.554+0000 7f2692485700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268c0b4240 con 0x7f268c15c150 2023-11-08T08:20:06.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.554+0000 7f2692485700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268c0708a0 con 0x7f268c06ff10 2023-11-08T08:20:06.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.554+0000 7f2691483700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f268c15c150 0x7f268c0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:06.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.554+0000 7f2691483700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f268c15c150 0x7f268c0b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36156/0 (socket says 172.21.15.40:36156) 2023-11-08T08:20:06.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.554+0000 7f2691483700 1 -- 172.21.15.40:0/329101932 learned_addr learned my addr 172.21.15.40:0/329101932 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:06.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.554+0000 7f2690c82700 1 --2- 172.21.15.40:0/329101932 >> v2:172.21.15.169:3300/0 conn(0x7f268c06ff10 0x7f268c070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:06.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.554+0000 7f2691483700 1 -- 172.21.15.40:0/329101932 >> v2:172.21.15.169:3300/0 conn(0x7f268c06ff10 msgr2=0x7f268c070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:06.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.554+0000 7f2691483700 1 --2- 172.21.15.40:0/329101932 >> v2:172.21.15.169:3300/0 conn(0x7f268c06ff10 0x7f268c070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.554+0000 7f2691483700 1 -- 172.21.15.40:0/329101932 >> v1:172.21.15.169:6789/0 conn(0x7f268c15fc30 legacy=0x7f268c160050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:06.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.554+0000 7f2691483700 1 -- 172.21.15.40:0/329101932 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c06ac00 con 0x7f268c15c150 2023-11-08T08:20:06.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.554+0000 7f2691483700 1 --2- 172.21.15.40:0/329101932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f268c15c150 0x7f268c0b3d00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f268c066d30 tx=0x7f268800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ba903175b759b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:06.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.554+0000 7f2683fff700 1 -- 172.21.15.40:0/329101932 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268800e070 con 0x7f268c15c150 2023-11-08T08:20:06.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.558+0000 7f2683fff700 1 -- 172.21.15.40:0/329101932 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268800b9c0 con 0x7f268c15c150 2023-11-08T08:20:06.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.558+0000 7f2683fff700 1 -- 172.21.15.40:0/329101932 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268800bd70 con 0x7f268c15c150 2023-11-08T08:20:06.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.558+0000 7f2692485700 1 -- 172.21.15.40:0/329101932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f268c15c150 msgr2=0x7f268c0b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:06.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.558+0000 7f2692485700 1 --2- 172.21.15.40:0/329101932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f268c15c150 0x7f268c0b3d00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f268c066d30 tx=0x7f268800b330 comp rx=0 tx=0).stop 2023-11-08T08:20:06.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.558+0000 7f2692485700 1 -- 172.21.15.40:0/329101932 shutdown_connections 2023-11-08T08:20:06.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.558+0000 7f2692485700 1 --2- 172.21.15.40:0/329101932 >> v2:172.21.15.169:3300/0 conn(0x7f268c06ff10 0x7f268c070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.558+0000 7f2692485700 1 --2- 172.21.15.40:0/329101932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f268c15c150 0x7f268c0b3d00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.558+0000 7f2692485700 1 -- 172.21.15.40:0/329101932 >> 172.21.15.40:0/329101932 conn(0x7f268c0d3680 msgr2=0x7f268c0cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:06.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.558+0000 7f2692485700 1 -- 172.21.15.40:0/329101932 shutdown_connections 2023-11-08T08:20:06.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.558+0000 7f2692485700 1 -- 172.21.15.40:0/329101932 wait complete. 2023-11-08T08:20:06.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.562+0000 7f2692485700 1 Processor -- start 2023-11-08T08:20:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.562+0000 7f2692485700 1 -- start start 2023-11-08T08:20:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.562+0000 7f2692485700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f268c06ff10 0x7f268c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.562+0000 7f2692485700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f268c15c150 0x7f268c157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:06.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.562+0000 7f2692485700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f268c15fc30 0x7f268c14a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:06.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.562+0000 7f2692485700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c1585b0 con 0x7f268c15c150 2023-11-08T08:20:06.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.562+0000 7f2692485700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c158380 con 0x7f268c06ff10 2023-11-08T08:20:06.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.562+0000 7f2692485700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c15a810 con 0x7f268c15fc30 2023-11-08T08:20:06.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.562+0000 7f2690c82700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f268c15c150 0x7f268c157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.562+0000 7f2690c82700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f268c15c150 0x7f268c157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36158/0 (socket says 172.21.15.40:36158) 2023-11-08T08:20:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.562+0000 7f2690c82700 1 -- 172.21.15.40:0/3448601763 learned_addr learned my addr 172.21.15.40:0/3448601763 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.562+0000 7f2691483700 1 --2- 172.21.15.40:0/3448601763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f268c06ff10 0x7f268c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.562+0000 7f2691c84700 1 --2- 172.21.15.40:0/3448601763 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f268c15fc30 0x7f268c14a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.566+0000 7f2690c82700 1 -- 172.21.15.40:0/3448601763 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f268c15fc30 msgr2=0x7f268c14a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.566+0000 7f2690c82700 1 --2- 172.21.15.40:0/3448601763 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f268c15fc30 0x7f268c14a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.566+0000 7f2690c82700 1 -- 172.21.15.40:0/3448601763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f268c06ff10 msgr2=0x7f268c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.566+0000 7f2690c82700 1 --2- 172.21.15.40:0/3448601763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f268c06ff10 0x7f268c0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.566+0000 7f2690c82700 1 -- 172.21.15.40:0/3448601763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2688009b30 con 0x7f268c15c150 2023-11-08T08:20:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.566+0000 7f2690c82700 1 --2- 172.21.15.40:0/3448601763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f268c15c150 0x7f268c157e40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f267c00e840 tx=0x7f267c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.566+0000 7f2681ffb700 1 -- 172.21.15.40:0/3448601763 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c005c40 con 0x7f268c15c150 2023-11-08T08:20:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.566+0000 7f2681ffb700 1 -- 172.21.15.40:0/3448601763 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c005da0 con 0x7f268c15c150 2023-11-08T08:20:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.566+0000 7f2681ffb700 1 -- 172.21.15.40:0/3448601763 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c006260 con 0x7f268c15c150 2023-11-08T08:20:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.566+0000 7f2692485700 1 -- 172.21.15.40:0/3448601763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268c0d2450 con 0x7f268c15c150 2023-11-08T08:20:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.566+0000 7f2692485700 1 -- 172.21.15.40:0/3448601763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268c06cc50 con 0x7f268c15c150 2023-11-08T08:20:06.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.566+0000 7f2692485700 1 -- 172.21.15.40:0/3448601763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f268c0d2730 con 0x7f268c15c150 2023-11-08T08:20:06.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.570+0000 7f2681ffb700 1 -- 172.21.15.40:0/3448601763 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f267c022070 con 0x7f268c15c150 2023-11-08T08:20:06.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.570+0000 7f2681ffb700 1 --2- 172.21.15.40:0/3448601763 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2670064ee0 0x7f2670067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:06.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.570+0000 7f2681ffb700 1 -- 172.21.15.40:0/3448601763 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f267c081ae0 con 0x7f268c15c150 2023-11-08T08:20:06.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.570+0000 7f2691483700 1 --2- 172.21.15.40:0/3448601763 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2670064ee0 0x7f2670067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:06.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.570+0000 7f2691483700 1 --2- 172.21.15.40:0/3448601763 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2670064ee0 0x7f2670067390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2688000f80 tx=0x7f2688009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:06.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.578+0000 7f2681ffb700 1 -- 172.21.15.40:0/3448601763 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f267c04f860 con 0x7f268c15c150 2023-11-08T08:20:06.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.766+0000 7f2692485700 1 -- 172.21.15.40:0/3448601763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f268c073d20 con 0x7f268c15c150 2023-11-08T08:20:06.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.770+0000 7f2681ffb700 1 -- 172.21.15.40:0/3448601763 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f267c020020 con 0x7f268c15c150 2023-11-08T08:20:06.773 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:20:06.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.770+0000 7f2692485700 1 -- 172.21.15.40:0/3448601763 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2670064ee0 msgr2=0x7f2670067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:06.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.770+0000 7f2692485700 1 --2- 172.21.15.40:0/3448601763 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2670064ee0 0x7f2670067390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2688000f80 tx=0x7f2688009000 comp rx=0 tx=0).stop 2023-11-08T08:20:06.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.770+0000 7f2692485700 1 -- 172.21.15.40:0/3448601763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f268c15c150 msgr2=0x7f268c157e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:06.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.770+0000 7f2692485700 1 --2- 172.21.15.40:0/3448601763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f268c15c150 0x7f268c157e40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f267c00e840 tx=0x7f267c00eb50 comp rx=0 tx=0).stop 2023-11-08T08:20:06.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.770+0000 7f2692485700 1 -- 172.21.15.40:0/3448601763 shutdown_connections 2023-11-08T08:20:06.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.770+0000 7f2692485700 1 --2- 172.21.15.40:0/3448601763 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2670064ee0 0x7f2670067390 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.770+0000 7f2692485700 1 --2- 172.21.15.40:0/3448601763 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f268c15fc30 0x7f268c14a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.770+0000 7f2692485700 1 --2- 172.21.15.40:0/3448601763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f268c15c150 0x7f268c157e40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.770+0000 7f2692485700 1 --2- 172.21.15.40:0/3448601763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f268c06ff10 0x7f268c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.770+0000 7f2692485700 1 -- 172.21.15.40:0/3448601763 >> 172.21.15.40:0/3448601763 conn(0x7f268c0d3680 msgr2=0x7f268c0c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:06.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.770+0000 7f2692485700 1 -- 172.21.15.40:0/3448601763 shutdown_connections 2023-11-08T08:20:06.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.774+0000 7f2692485700 1 -- 172.21.15.40:0/3448601763 wait complete. 2023-11-08T08:20:06.802 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:20:06.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.982+0000 7f9ed4a66700 1 Processor -- start 2023-11-08T08:20:06.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.982+0000 7f9ed4a66700 1 -- start start 2023-11-08T08:20:06.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.986+0000 7f9ed4a66700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9ed015c150 0x7f9ed00b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:06.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.986+0000 7f9ed4a66700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ed0070360 0x7f9ed0160340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:06.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.986+0000 7f9ed4a66700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed015c520 con 0x7f9ed006ff10 2023-11-08T08:20:06.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.986+0000 7f9ed4a66700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed00b4240 con 0x7f9ed015c150 2023-11-08T08:20:06.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.986+0000 7f9ed4a66700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed006ac00 con 0x7f9ed0070360 2023-11-08T08:20:06.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.986+0000 7f9ecf7fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9ed015c150 0x7f9ed00b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:06.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.986+0000 7f9eceffd700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f9ed006ff10 0x7f9ed015fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:43890/0 (socket says 172.21.15.40:43890) 2023-11-08T08:20:06.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.986+0000 7f9eceffd700 1 -- 172.21.15.40:0/2720043830 learned_addr learned my addr 172.21.15.40:0/2720043830 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:06.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.986+0000 7f9ecffff700 1 --2- 172.21.15.40:0/2720043830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ed0070360 0x7f9ed0160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:06.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.986+0000 7f9ece7fc700 1 -- 172.21.15.40:0/2720043830 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 814965429 0 0) 0x7f9ed015c520 con 0x7f9ed006ff10 2023-11-08T08:20:06.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.986+0000 7f9ece7fc700 1 -- 172.21.15.40:0/2720043830 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9eb80035d0 con 0x7f9ed006ff10 2023-11-08T08:20:06.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.986+0000 7f9ecf7fe700 1 -- 172.21.15.40:0/2720043830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ed0070360 msgr2=0x7f9ed0160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.986+0000 7f9ecf7fe700 1 --2- 172.21.15.40:0/2720043830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ed0070360 0x7f9ed0160340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.986+0000 7f9ecf7fe700 1 -- 172.21.15.40:0/2720043830 >> v1:172.21.15.169:6789/0 conn(0x7f9ed006ff10 legacy=0x7f9ed015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.986+0000 7f9ecf7fe700 1 -- 172.21.15.40:0/2720043830 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed006cc50 con 0x7f9ed015c150 2023-11-08T08:20:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.986+0000 7f9ecffff700 1 --2- 172.21.15.40:0/2720043830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ed0070360 0x7f9ed0160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.990+0000 7f9ecf7fe700 1 --2- 172.21.15.40:0/2720043830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed015c150 0x7f9ed00b3d00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0009850 tx=0x7f9ec0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b48b1e1760d30c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.990+0000 7f9ece7fc700 1 -- 172.21.15.40:0/2720043830 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0011830 con 0x7f9ed015c150 2023-11-08T08:20:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.990+0000 7f9ece7fc700 1 -- 172.21.15.40:0/2720043830 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec0011990 con 0x7f9ed015c150 2023-11-08T08:20:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.990+0000 7f9ece7fc700 1 -- 172.21.15.40:0/2720043830 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0011d40 con 0x7f9ed015c150 2023-11-08T08:20:06.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.990+0000 7f9ed4a66700 1 -- 172.21.15.40:0/2720043830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed015c150 msgr2=0x7f9ed00b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.990+0000 7f9ed4a66700 1 --2- 172.21.15.40:0/2720043830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed015c150 0x7f9ed00b3d00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0009850 tx=0x7f9ec0009b60 comp rx=0 tx=0).stop 2023-11-08T08:20:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.990+0000 7f9ed4a66700 1 -- 172.21.15.40:0/2720043830 shutdown_connections 2023-11-08T08:20:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.990+0000 7f9ed4a66700 1 --2- 172.21.15.40:0/2720043830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ed0070360 0x7f9ed0160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.990+0000 7f9ed4a66700 1 --2- 172.21.15.40:0/2720043830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed015c150 0x7f9ed00b3d00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.990+0000 7f9ed4a66700 1 -- 172.21.15.40:0/2720043830 >> 172.21.15.40:0/2720043830 conn(0x7f9ed00d3680 msgr2=0x7f9ed00cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.990+0000 7f9ed4a66700 1 -- 172.21.15.40:0/2720043830 shutdown_connections 2023-11-08T08:20:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.990+0000 7f9ed4a66700 1 -- 172.21.15.40:0/2720043830 wait complete. 2023-11-08T08:20:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.994+0000 7f9ed4a66700 1 Processor -- start 2023-11-08T08:20:07.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.994+0000 7f9ed4a66700 1 -- start start 2023-11-08T08:20:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.994+0000 7f9ed4a66700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ed006ff10 0x7f9ed00cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.994+0000 7f9ed4a66700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ed0070360 0x7f9ed01f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.994+0000 7f9ed4a66700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed015c150 0x7f9ed0159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.994+0000 7f9ed4a66700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed01f6690 con 0x7f9ed0070360 2023-11-08T08:20:07.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.994+0000 7f9ed4a66700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed01f6080 con 0x7f9ed015c150 2023-11-08T08:20:07.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.994+0000 7f9ed4a66700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed0159da0 con 0x7f9ed006ff10 2023-11-08T08:20:07.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.994+0000 7f9eceffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ed0070360 0x7f9ed01f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:07.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.994+0000 7f9eceffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ed0070360 0x7f9ed01f6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36172/0 (socket says 172.21.15.40:36172) 2023-11-08T08:20:07.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.994+0000 7f9eceffd700 1 -- 172.21.15.40:0/4285752377 learned_addr learned my addr 172.21.15.40:0/4285752377 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:07.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.994+0000 7f9ecffff700 1 --2- 172.21.15.40:0/4285752377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed015c150 0x7f9ed0159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:07.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.994+0000 7f9ecf7fe700 1 --2- 172.21.15.40:0/4285752377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ed006ff10 0x7f9ed00cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:07.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.994+0000 7f9eceffd700 1 -- 172.21.15.40:0/4285752377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ed006ff10 msgr2=0x7f9ed00cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:07.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.994+0000 7f9eceffd700 1 --2- 172.21.15.40:0/4285752377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ed006ff10 0x7f9ed00cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.994+0000 7f9eceffd700 1 -- 172.21.15.40:0/4285752377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed015c150 msgr2=0x7f9ed0159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:07.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.994+0000 7f9eceffd700 1 --2- 172.21.15.40:0/4285752377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed015c150 0x7f9ed0159570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.994+0000 7f9eceffd700 1 -- 172.21.15.40:0/4285752377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec8009a70 con 0x7f9ed0070360 2023-11-08T08:20:07.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.998+0000 7f9eceffd700 1 --2- 172.21.15.40:0/4285752377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ed0070360 0x7f9ed01f6b20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4001130 tx=0x7f9ec400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:07.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.998+0000 7f9eb7fff700 1 -- 172.21.15.40:0/4285752377 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4005de0 con 0x7f9ed0070360 2023-11-08T08:20:07.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.998+0000 7f9eb7fff700 1 -- 172.21.15.40:0/4285752377 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec40070d0 con 0x7f9ed0070360 2023-11-08T08:20:07.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.998+0000 7f9ed4a66700 1 -- 172.21.15.40:0/4285752377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec00094b0 con 0x7f9ed0070360 2023-11-08T08:20:07.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.998+0000 7f9eb7fff700 1 -- 172.21.15.40:0/4285752377 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4007460 con 0x7f9ed0070360 2023-11-08T08:20:07.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.998+0000 7f9ed4a66700 1 -- 172.21.15.40:0/4285752377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed00d2730 con 0x7f9ed0070360 2023-11-08T08:20:07.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:06.998+0000 7f9ed4a66700 1 -- 172.21.15.40:0/4285752377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed00d2450 con 0x7f9ed0070360 2023-11-08T08:20:07.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.002+0000 7f9eb7fff700 1 -- 172.21.15.40:0/4285752377 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f9ec4019030 con 0x7f9ed0070360 2023-11-08T08:20:07.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.002+0000 7f9eb7fff700 1 --2- 172.21.15.40:0/4285752377 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9ebc064ee0 0x7f9ebc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:07.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.002+0000 7f9eb7fff700 1 -- 172.21.15.40:0/4285752377 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9ec4081540 con 0x7f9ed0070360 2023-11-08T08:20:07.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.002+0000 7f9ecf7fe700 1 --2- 172.21.15.40:0/4285752377 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9ebc064ee0 0x7f9ebc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:07.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.002+0000 7f9ecf7fe700 1 --2- 172.21.15.40:0/4285752377 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9ebc064ee0 0x7f9ebc067390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9ec00090b0 tx=0x7f9ec001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:07.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.010+0000 7f9eb7fff700 1 -- 172.21.15.40:0/4285752377 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f9ec404f2c0 con 0x7f9ed0070360 2023-11-08T08:20:07.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.182+0000 7f9ed4a66700 1 -- 172.21.15.40:0/4285752377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9ed00d05e0 con 0x7f9ed0070360 2023-11-08T08:20:07.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.182+0000 7f9eb7fff700 1 -- 172.21.15.40:0/4285752377 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9ec40170a0 con 0x7f9ed0070360 2023-11-08T08:20:07.187 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:20:07.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.190+0000 7f9ed4a66700 1 -- 172.21.15.40:0/4285752377 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9ebc064ee0 msgr2=0x7f9ebc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:07.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.190+0000 7f9ed4a66700 1 --2- 172.21.15.40:0/4285752377 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9ebc064ee0 0x7f9ebc067390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9ec00090b0 tx=0x7f9ec001c040 comp rx=0 tx=0).stop 2023-11-08T08:20:07.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.190+0000 7f9ed4a66700 1 -- 172.21.15.40:0/4285752377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ed0070360 msgr2=0x7f9ed01f6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:07.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.190+0000 7f9ed4a66700 1 --2- 172.21.15.40:0/4285752377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ed0070360 0x7f9ed01f6b20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4001130 tx=0x7f9ec400c9e0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.190+0000 7f9ed4a66700 1 -- 172.21.15.40:0/4285752377 shutdown_connections 2023-11-08T08:20:07.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.190+0000 7f9ed4a66700 1 --2- 172.21.15.40:0/4285752377 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9ebc064ee0 0x7f9ebc067390 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.190+0000 7f9ed4a66700 1 --2- 172.21.15.40:0/4285752377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed015c150 0x7f9ed0159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.190+0000 7f9ed4a66700 1 --2- 172.21.15.40:0/4285752377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ed0070360 0x7f9ed01f6b20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.190+0000 7f9ed4a66700 1 --2- 172.21.15.40:0/4285752377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ed006ff10 0x7f9ed00cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.190+0000 7f9ed4a66700 1 -- 172.21.15.40:0/4285752377 >> 172.21.15.40:0/4285752377 conn(0x7f9ed00d3680 msgr2=0x7f9ed00b74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:07.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.190+0000 7f9ed4a66700 1 -- 172.21.15.40:0/4285752377 shutdown_connections 2023-11-08T08:20:07.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.190+0000 7f9ed4a66700 1 -- 172.21.15.40:0/4285752377 wait complete. 2023-11-08T08:20:07.217 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T08:20:07.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.402+0000 7f63cf59e700 1 Processor -- start 2023-11-08T08:20:07.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.402+0000 7f63cf59e700 1 -- start start 2023-11-08T08:20:07.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.406+0000 7f63cf59e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f63d015c150 0x7f63d00b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:07.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.406+0000 7f63cf59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63d0070360 0x7f63d0160340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:07.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.406+0000 7f63cf59e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d015c520 con 0x7f63d006ff10 2023-11-08T08:20:07.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.406+0000 7f63cf59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d00b4240 con 0x7f63d0070360 2023-11-08T08:20:07.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.406+0000 7f63cf59e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d006ac00 con 0x7f63d015c150 2023-11-08T08:20:07.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.406+0000 7f63ced9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63d0070360 0x7f63d0160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:07.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.406+0000 7f63ced9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63d0070360 0x7f63d0160340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39986/0 (socket says 172.21.15.40:39986) 2023-11-08T08:20:07.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.406+0000 7f63ced9d700 1 -- 172.21.15.40:0/1769120878 learned_addr learned my addr 172.21.15.40:0/1769120878 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:07.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.406+0000 7f63ce59c700 1 --2- 172.21.15.40:0/1769120878 >> v2:172.21.15.169:3300/0 conn(0x7f63d015c150 0x7f63d00b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:07.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.406+0000 7f63ced9d700 1 -- 172.21.15.40:0/1769120878 >> v2:172.21.15.169:3300/0 conn(0x7f63d015c150 msgr2=0x7f63d00b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:07.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.406+0000 7f63ced9d700 1 --2- 172.21.15.40:0/1769120878 >> v2:172.21.15.169:3300/0 conn(0x7f63d015c150 0x7f63d00b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.406+0000 7f63ced9d700 1 -- 172.21.15.40:0/1769120878 >> v1:172.21.15.169:6789/0 conn(0x7f63d006ff10 legacy=0x7f63d015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:07.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.406+0000 7f63ced9d700 1 -- 172.21.15.40:0/1769120878 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d006cc50 con 0x7f63d0070360 2023-11-08T08:20:07.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.406+0000 7f63ce59c700 1 --2- 172.21.15.40:0/1769120878 >> v2:172.21.15.169:3300/0 conn(0x7f63d015c150 0x7f63d00b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:07.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.406+0000 7f63ced9d700 1 --2- 172.21.15.40:0/1769120878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d0070360 0x7f63d0160340 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f63d0071900 tx=0x7f63c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10e18d0c4c996c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:07.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.406+0000 7f63cd59a700 1 -- 172.21.15.40:0/1769120878 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c800e070 con 0x7f63d0070360 2023-11-08T08:20:07.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.406+0000 7f63cd59a700 1 -- 172.21.15.40:0/1769120878 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c800b9c0 con 0x7f63d0070360 2023-11-08T08:20:07.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.406+0000 7f63cd59a700 1 -- 172.21.15.40:0/1769120878 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c800bd70 con 0x7f63d0070360 2023-11-08T08:20:07.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.406+0000 7f63cf59e700 1 -- 172.21.15.40:0/1769120878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d0070360 msgr2=0x7f63d0160340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:07.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.406+0000 7f63cf59e700 1 --2- 172.21.15.40:0/1769120878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d0070360 0x7f63d0160340 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f63d0071900 tx=0x7f63c800b330 comp rx=0 tx=0).stop 2023-11-08T08:20:07.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.410+0000 7f63cf59e700 1 -- 172.21.15.40:0/1769120878 shutdown_connections 2023-11-08T08:20:07.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.410+0000 7f63cf59e700 1 --2- 172.21.15.40:0/1769120878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d0070360 0x7f63d0160340 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.410+0000 7f63cf59e700 1 --2- 172.21.15.40:0/1769120878 >> v2:172.21.15.169:3300/0 conn(0x7f63d015c150 0x7f63d00b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.410+0000 7f63cf59e700 1 -- 172.21.15.40:0/1769120878 >> 172.21.15.40:0/1769120878 conn(0x7f63d00d3680 msgr2=0x7f63d00cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:07.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.410+0000 7f63cf59e700 1 -- 172.21.15.40:0/1769120878 shutdown_connections 2023-11-08T08:20:07.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.410+0000 7f63cf59e700 1 -- 172.21.15.40:0/1769120878 wait complete. 2023-11-08T08:20:07.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.410+0000 7f63cf59e700 1 Processor -- start 2023-11-08T08:20:07.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.410+0000 7f63cf59e700 1 -- start start 2023-11-08T08:20:07.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63cf59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d006ff10 0x7f63d00b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:07.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63cf59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63d0070360 0x7f63d00cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:07.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63cf59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63d015c150 0x7f63d00cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:07.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63cf59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d00ce4d0 con 0x7f63d006ff10 2023-11-08T08:20:07.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63cf59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d00c9a40 con 0x7f63d0070360 2023-11-08T08:20:07.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63cf59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d00b7080 con 0x7f63d015c150 2023-11-08T08:20:07.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63ce59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d006ff10 0x7f63d00b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:07.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63ce59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d006ff10 0x7f63d00b69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39994/0 (socket says 172.21.15.40:39994) 2023-11-08T08:20:07.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63ce59c700 1 -- 172.21.15.40:0/1527916418 learned_addr learned my addr 172.21.15.40:0/1527916418 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:07.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63cdd9b700 1 --2- 172.21.15.40:0/1527916418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63d0070360 0x7f63d00cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:07.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63ce59c700 1 -- 172.21.15.40:0/1527916418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63d015c150 msgr2=0x7f63d00cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:07.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63ce59c700 1 --2- 172.21.15.40:0/1527916418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63d015c150 0x7f63d00cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63ce59c700 1 -- 172.21.15.40:0/1527916418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63d0070360 msgr2=0x7f63d00cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:07.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63ce59c700 1 --2- 172.21.15.40:0/1527916418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63d0070360 0x7f63d00cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63ce59c700 1 -- 172.21.15.40:0/1527916418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c8009b30 con 0x7f63d006ff10 2023-11-08T08:20:07.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63cdd9b700 1 --2- 172.21.15.40:0/1527916418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63d0070360 0x7f63d00cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:20:07.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63ce59c700 1 --2- 172.21.15.40:0/1527916418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d006ff10 0x7f63d00b69a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f63c000e810 tx=0x7f63c000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:07.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63beffd700 1 -- 172.21.15.40:0/1527916418 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0004230 con 0x7f63d006ff10 2023-11-08T08:20:07.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63beffd700 1 -- 172.21.15.40:0/1527916418 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c0004390 con 0x7f63d006ff10 2023-11-08T08:20:07.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63beffd700 1 -- 172.21.15.40:0/1527916418 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c00036c0 con 0x7f63d006ff10 2023-11-08T08:20:07.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63cf59e700 1 -- 172.21.15.40:0/1527916418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d00d2730 con 0x7f63d006ff10 2023-11-08T08:20:07.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.414+0000 7f63cf59e700 1 -- 172.21.15.40:0/1527916418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d00d2450 con 0x7f63d006ff10 2023-11-08T08:20:07.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.418+0000 7f63cf59e700 1 -- 172.21.15.40:0/1527916418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d00d05e0 con 0x7f63d006ff10 2023-11-08T08:20:07.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.418+0000 7f63beffd700 1 -- 172.21.15.40:0/1527916418 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f63c0022070 con 0x7f63d006ff10 2023-11-08T08:20:07.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.422+0000 7f63beffd700 1 --2- 172.21.15.40:0/1527916418 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f63b0064f10 0x7f63b00673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:07.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.422+0000 7f63beffd700 1 -- 172.21.15.40:0/1527916418 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f63c0081cd0 con 0x7f63d006ff10 2023-11-08T08:20:07.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.422+0000 7f63cdd9b700 1 --2- 172.21.15.40:0/1527916418 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f63b0064f10 0x7f63b00673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:07.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.422+0000 7f63cdd9b700 1 --2- 172.21.15.40:0/1527916418 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f63b0064f10 0x7f63b00673c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f63d00c9680 tx=0x7f63c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:07.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.426+0000 7f63beffd700 1 -- 172.21.15.40:0/1527916418 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f63c004f9f0 con 0x7f63d006ff10 2023-11-08T08:20:07.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.598+0000 7f63cf59e700 1 -- 172.21.15.40:0/1527916418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f63d0073d20 con 0x7f63d006ff10 2023-11-08T08:20:07.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.598+0000 7f63beffd700 1 -- 172.21.15.40:0/1527916418 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f63c0053010 con 0x7f63d006ff10 2023-11-08T08:20:07.604 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:20:07.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.602+0000 7f63cf59e700 1 -- 172.21.15.40:0/1527916418 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f63b0064f10 msgr2=0x7f63b00673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:07.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.602+0000 7f63cf59e700 1 --2- 172.21.15.40:0/1527916418 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f63b0064f10 0x7f63b00673c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f63d00c9680 tx=0x7f63c4009040 comp rx=0 tx=0).stop 2023-11-08T08:20:07.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.602+0000 7f63cf59e700 1 -- 172.21.15.40:0/1527916418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d006ff10 msgr2=0x7f63d00b69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:07.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.602+0000 7f63cf59e700 1 --2- 172.21.15.40:0/1527916418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d006ff10 0x7f63d00b69a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f63c000e810 tx=0x7f63c000ebd0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.602+0000 7f63cf59e700 1 -- 172.21.15.40:0/1527916418 shutdown_connections 2023-11-08T08:20:07.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.602+0000 7f63cf59e700 1 --2- 172.21.15.40:0/1527916418 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f63b0064f10 0x7f63b00673c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.602+0000 7f63cf59e700 1 --2- 172.21.15.40:0/1527916418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63d015c150 0x7f63d00cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.602+0000 7f63cf59e700 1 --2- 172.21.15.40:0/1527916418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63d0070360 0x7f63d00cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.602+0000 7f63cf59e700 1 --2- 172.21.15.40:0/1527916418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d006ff10 0x7f63d00b69a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.602+0000 7f63cf59e700 1 -- 172.21.15.40:0/1527916418 >> 172.21.15.40:0/1527916418 conn(0x7f63d00d3680 msgr2=0x7f63d0158420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:07.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.602+0000 7f63cf59e700 1 -- 172.21.15.40:0/1527916418 shutdown_connections 2023-11-08T08:20:07.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.602+0000 7f63cf59e700 1 -- 172.21.15.40:0/1527916418 wait complete. 2023-11-08T08:20:07.630 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T08:20:07.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.810+0000 7fdd2d6b5700 1 Processor -- start 2023-11-08T08:20:07.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.810+0000 7fdd2d6b5700 1 -- start start 2023-11-08T08:20:07.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.814+0000 7fdd2d6b5700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdd2815c150 0x7fdd280b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:07.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.814+0000 7fdd2d6b5700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd28070360 0x7fdd28160340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:07.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.814+0000 7fdd2d6b5700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd2815c520 con 0x7fdd2806ff10 2023-11-08T08:20:07.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.814+0000 7fdd2d6b5700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd280b4240 con 0x7fdd2815c150 2023-11-08T08:20:07.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.814+0000 7fdd2d6b5700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2806ac00 con 0x7fdd28070360 2023-11-08T08:20:07.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.814+0000 7fdd267fc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdd2806ff10 0x7fdd2815fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59404/0 (socket says 172.21.15.40:59404) 2023-11-08T08:20:07.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.814+0000 7fdd267fc700 1 -- 172.21.15.40:0/1959747595 learned_addr learned my addr 172.21.15.40:0/1959747595 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:07.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.814+0000 7fdd26ffd700 1 --2- 172.21.15.40:0/1959747595 >> v2:172.21.15.40:3300/0 conn(0x7fdd2815c150 0x7fdd280b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:07.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.814+0000 7fdd277fe700 1 --2- 172.21.15.40:0/1959747595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd28070360 0x7fdd28160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:07.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.814+0000 7fdd25ffb700 1 -- 172.21.15.40:0/1959747595 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628516660 0 0) 0x7fdd2815c520 con 0x7fdd2806ff10 2023-11-08T08:20:07.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.814+0000 7fdd25ffb700 1 -- 172.21.15.40:0/1959747595 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd100035d0 con 0x7fdd2806ff10 2023-11-08T08:20:07.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.814+0000 7fdd26ffd700 1 -- 172.21.15.40:0/1959747595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd28070360 msgr2=0x7fdd28160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:07.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.814+0000 7fdd26ffd700 1 --2- 172.21.15.40:0/1959747595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd28070360 0x7fdd28160340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.814+0000 7fdd26ffd700 1 -- 172.21.15.40:0/1959747595 >> v1:172.21.15.40:6789/0 conn(0x7fdd2806ff10 legacy=0x7fdd2815fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:07.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.814+0000 7fdd26ffd700 1 -- 172.21.15.40:0/1959747595 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2806cc50 con 0x7fdd2815c150 2023-11-08T08:20:07.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.814+0000 7fdd26ffd700 1 --2- 172.21.15.40:0/1959747595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd2815c150 0x7fdd280b3d00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009850 tx=0x7fdd18009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113145842ed0f771 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:07.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.818+0000 7fdd25ffb700 1 -- 172.21.15.40:0/1959747595 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd18011830 con 0x7fdd2815c150 2023-11-08T08:20:07.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.818+0000 7fdd25ffb700 1 -- 172.21.15.40:0/1959747595 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd18011990 con 0x7fdd2815c150 2023-11-08T08:20:07.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.818+0000 7fdd25ffb700 1 -- 172.21.15.40:0/1959747595 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd18011d40 con 0x7fdd2815c150 2023-11-08T08:20:07.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.818+0000 7fdd2d6b5700 1 -- 172.21.15.40:0/1959747595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd2815c150 msgr2=0x7fdd280b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:07.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.818+0000 7fdd2d6b5700 1 --2- 172.21.15.40:0/1959747595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd2815c150 0x7fdd280b3d00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009850 tx=0x7fdd18009b60 comp rx=0 tx=0).stop 2023-11-08T08:20:07.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.818+0000 7fdd2d6b5700 1 -- 172.21.15.40:0/1959747595 shutdown_connections 2023-11-08T08:20:07.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.818+0000 7fdd2d6b5700 1 --2- 172.21.15.40:0/1959747595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd28070360 0x7fdd28160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.818+0000 7fdd2d6b5700 1 --2- 172.21.15.40:0/1959747595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd2815c150 0x7fdd280b3d00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.818+0000 7fdd2d6b5700 1 -- 172.21.15.40:0/1959747595 >> 172.21.15.40:0/1959747595 conn(0x7fdd280d3680 msgr2=0x7fdd280cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:07.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.818+0000 7fdd2d6b5700 1 -- 172.21.15.40:0/1959747595 shutdown_connections 2023-11-08T08:20:07.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.818+0000 7fdd2d6b5700 1 -- 172.21.15.40:0/1959747595 wait complete. 2023-11-08T08:20:07.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.822+0000 7fdd2d6b5700 1 Processor -- start 2023-11-08T08:20:07.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.822+0000 7fdd2d6b5700 1 -- start start 2023-11-08T08:20:07.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.822+0000 7fdd2d6b5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2806ff10 0x7fdd28155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:07.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.822+0000 7fdd2d6b5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd28070360 0x7fdd281434a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:07.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.822+0000 7fdd2d6b5700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd2815c150 0x7fdd281f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:07.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.822+0000 7fdd2d6b5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd280ce4c0 con 0x7fdd28070360 2023-11-08T08:20:07.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.822+0000 7fdd2d6b5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd281439e0 con 0x7fdd2806ff10 2023-11-08T08:20:07.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.822+0000 7fdd2d6b5700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd28156010 con 0x7fdd2815c150 2023-11-08T08:20:07.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.822+0000 7fdd267fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd28070360 0x7fdd281434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:07.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.822+0000 7fdd267fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd28070360 0x7fdd281434a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40000/0 (socket says 172.21.15.40:40000) 2023-11-08T08:20:07.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.822+0000 7fdd267fc700 1 -- 172.21.15.40:0/1399470496 learned_addr learned my addr 172.21.15.40:0/1399470496 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:07.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.822+0000 7fdd277fe700 1 --2- 172.21.15.40:0/1399470496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd2815c150 0x7fdd281f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:07.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.822+0000 7fdd26ffd700 1 --2- 172.21.15.40:0/1399470496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2806ff10 0x7fdd28155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:07.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.822+0000 7fdd267fc700 1 -- 172.21.15.40:0/1399470496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd2815c150 msgr2=0x7fdd281f50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:07.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.822+0000 7fdd267fc700 1 --2- 172.21.15.40:0/1399470496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd2815c150 0x7fdd281f50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.822+0000 7fdd267fc700 1 -- 172.21.15.40:0/1399470496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2806ff10 msgr2=0x7fdd28155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:07.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.822+0000 7fdd267fc700 1 --2- 172.21.15.40:0/1399470496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2806ff10 0x7fdd28155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:07.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.822+0000 7fdd267fc700 1 -- 172.21.15.40:0/1399470496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd180094b0 con 0x7fdd28070360 2023-11-08T08:20:07.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.826+0000 7fdd267fc700 1 --2- 172.21.15.40:0/1399470496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd28070360 0x7fdd281434a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c00c700 tx=0x7fdd1c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:07.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.826+0000 7fdd0f7fe700 1 -- 172.21.15.40:0/1399470496 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c0074e0 con 0x7fdd28070360 2023-11-08T08:20:07.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.826+0000 7fdd0f7fe700 1 -- 172.21.15.40:0/1399470496 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c007640 con 0x7fdd28070360 2023-11-08T08:20:07.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.826+0000 7fdd2d6b5700 1 -- 172.21.15.40:0/1399470496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd280d2730 con 0x7fdd28070360 2023-11-08T08:20:07.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.826+0000 7fdd0f7fe700 1 -- 172.21.15.40:0/1399470496 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c005a50 con 0x7fdd28070360 2023-11-08T08:20:07.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.826+0000 7fdd2d6b5700 1 -- 172.21.15.40:0/1399470496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd280d2450 con 0x7fdd28070360 2023-11-08T08:20:07.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.826+0000 7fdd2d6b5700 1 -- 172.21.15.40:0/1399470496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd280d05e0 con 0x7fdd28070360 2023-11-08T08:20:07.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.830+0000 7fdd0f7fe700 1 -- 172.21.15.40:0/1399470496 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fdd1c019030 con 0x7fdd28070360 2023-11-08T08:20:07.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.830+0000 7fdd0f7fe700 1 --2- 172.21.15.40:0/1399470496 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdd14064ee0 0x7fdd14067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:07.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.830+0000 7fdd0f7fe700 1 -- 172.21.15.40:0/1399470496 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdd1c081490 con 0x7fdd28070360 2023-11-08T08:20:07.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.830+0000 7fdd26ffd700 1 --2- 172.21.15.40:0/1399470496 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdd14064ee0 0x7fdd14067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:07.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.830+0000 7fdd26ffd700 1 --2- 172.21.15.40:0/1399470496 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdd14064ee0 0x7fdd14067390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdd180090b0 tx=0x7fdd1801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:07.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:07.838+0000 7fdd0f7fe700 1 -- 172.21.15.40:0/1399470496 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fdd1c04a400 con 0x7fdd28070360 2023-11-08T08:20:08.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.018+0000 7fdd2d6b5700 1 -- 172.21.15.40:0/1399470496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdd280ce600 con 0x7fdd28070360 2023-11-08T08:20:08.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.018+0000 7fdd0f7fe700 1 -- 172.21.15.40:0/1399470496 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd1c017020 con 0x7fdd28070360 2023-11-08T08:20:08.022 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-11-08T08:20:08.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.022+0000 7fdd2d6b5700 1 -- 172.21.15.40:0/1399470496 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdd14064ee0 msgr2=0x7fdd14067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:08.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.022+0000 7fdd2d6b5700 1 --2- 172.21.15.40:0/1399470496 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdd14064ee0 0x7fdd14067390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdd180090b0 tx=0x7fdd1801c040 comp rx=0 tx=0).stop 2023-11-08T08:20:08.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.022+0000 7fdd2d6b5700 1 -- 172.21.15.40:0/1399470496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd28070360 msgr2=0x7fdd281434a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:08.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.022+0000 7fdd2d6b5700 1 --2- 172.21.15.40:0/1399470496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd28070360 0x7fdd281434a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c00c700 tx=0x7fdd1c00ca10 comp rx=0 tx=0).stop 2023-11-08T08:20:08.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.022+0000 7fdd2d6b5700 1 -- 172.21.15.40:0/1399470496 shutdown_connections 2023-11-08T08:20:08.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.022+0000 7fdd2d6b5700 1 --2- 172.21.15.40:0/1399470496 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdd14064ee0 0x7fdd14067390 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.022+0000 7fdd2d6b5700 1 --2- 172.21.15.40:0/1399470496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd2815c150 0x7fdd281f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.022+0000 7fdd2d6b5700 1 --2- 172.21.15.40:0/1399470496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd28070360 0x7fdd281434a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.022+0000 7fdd2d6b5700 1 --2- 172.21.15.40:0/1399470496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2806ff10 0x7fdd28155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.022+0000 7fdd2d6b5700 1 -- 172.21.15.40:0/1399470496 >> 172.21.15.40:0/1399470496 conn(0x7fdd280d3680 msgr2=0x7fdd280b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:08.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.022+0000 7fdd2d6b5700 1 -- 172.21.15.40:0/1399470496 shutdown_connections 2023-11-08T08:20:08.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.022+0000 7fdd2d6b5700 1 -- 172.21.15.40:0/1399470496 wait complete. 2023-11-08T08:20:08.048 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T08:20:08.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.230+0000 7fc8295d5700 1 Processor -- start 2023-11-08T08:20:08.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.230+0000 7fc8295d5700 1 -- start start 2023-11-08T08:20:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc8295d5700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc82415c150 0x7fc8240b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc8295d5700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc82406ff10 0x7fc824070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:08.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc8295d5700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc82415fc30 0x7fc824160090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:08.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc8295d5700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8240b4240 con 0x7fc82406ff10 2023-11-08T08:20:08.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc8295d5700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc82415c520 con 0x7fc82415c150 2023-11-08T08:20:08.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc8295d5700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8240708a0 con 0x7fc82415fc30 2023-11-08T08:20:08.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc81bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc82406ff10 0x7fc824070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:08.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc81bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc82406ff10 0x7fc824070360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40006/0 (socket says 172.21.15.40:40006) 2023-11-08T08:20:08.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc823fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc82415c150 0x7fc8240b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:08.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc81bfff700 1 -- 172.21.15.40:0/2545117157 learned_addr learned my addr 172.21.15.40:0/2545117157 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:08.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc828dd4700 1 --2- 172.21.15.40:0/2545117157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc82415fc30 0x7fc824160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:08.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc823fff700 1 -- 172.21.15.40:0/2545117157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc82415fc30 msgr2=0x7fc824160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:08.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc823fff700 1 --2- 172.21.15.40:0/2545117157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc82415fc30 0x7fc824160090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc823fff700 1 -- 172.21.15.40:0/2545117157 >> v2:172.21.15.40:3300/0 conn(0x7fc82406ff10 msgr2=0x7fc824070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:08.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc823fff700 1 --2- 172.21.15.40:0/2545117157 >> v2:172.21.15.40:3300/0 conn(0x7fc82406ff10 0x7fc824070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc823fff700 1 -- 172.21.15.40:0/2545117157 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82406cc50 con 0x7fc82415c150 2023-11-08T08:20:08.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc81bfff700 1 --2- 172.21.15.40:0/2545117157 >> v2:172.21.15.40:3300/0 conn(0x7fc82406ff10 0x7fc824070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:08.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc823fff700 1 --2- 172.21.15.40:0/2545117157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc82415c150 0x7fc8240b3d00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc824071900 tx=0x7fc814009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90ba2ca99aae057e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:08.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc8237fe700 1 -- 172.21.15.40:0/2545117157 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814011830 con 0x7fc82415c150 2023-11-08T08:20:08.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc8237fe700 1 -- 172.21.15.40:0/2545117157 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc814011990 con 0x7fc82415c150 2023-11-08T08:20:08.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.234+0000 7fc8237fe700 1 -- 172.21.15.40:0/2545117157 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814011d20 con 0x7fc82415c150 2023-11-08T08:20:08.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.238+0000 7fc8295d5700 1 -- 172.21.15.40:0/2545117157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc82415c150 msgr2=0x7fc8240b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:08.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.238+0000 7fc8295d5700 1 --2- 172.21.15.40:0/2545117157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc82415c150 0x7fc8240b3d00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc824071900 tx=0x7fc814009b30 comp rx=0 tx=0).stop 2023-11-08T08:20:08.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.238+0000 7fc8295d5700 1 -- 172.21.15.40:0/2545117157 shutdown_connections 2023-11-08T08:20:08.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.238+0000 7fc8295d5700 1 --2- 172.21.15.40:0/2545117157 >> v2:172.21.15.40:3300/0 conn(0x7fc82406ff10 0x7fc824070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.238+0000 7fc8295d5700 1 --2- 172.21.15.40:0/2545117157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc82415fc30 0x7fc824160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.238+0000 7fc8295d5700 1 --2- 172.21.15.40:0/2545117157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc82415c150 0x7fc8240b3d00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.238+0000 7fc8295d5700 1 -- 172.21.15.40:0/2545117157 >> 172.21.15.40:0/2545117157 conn(0x7fc8240d3680 msgr2=0x7fc8240cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:08.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.238+0000 7fc8295d5700 1 -- 172.21.15.40:0/2545117157 shutdown_connections 2023-11-08T08:20:08.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.238+0000 7fc8295d5700 1 -- 172.21.15.40:0/2545117157 wait complete. 2023-11-08T08:20:08.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.238+0000 7fc8295d5700 1 Processor -- start 2023-11-08T08:20:08.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.238+0000 7fc8295d5700 1 -- start start 2023-11-08T08:20:08.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc8295d5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc82406ff10 0x7fc824150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:08.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc8295d5700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc82415c150 0x7fc824159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:08.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc8295d5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc82415fc30 0x7fc8240c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:08.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc8295d5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8240ccfa0 con 0x7fc82415fc30 2023-11-08T08:20:08.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc8295d5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8241f51e0 con 0x7fc82406ff10 2023-11-08T08:20:08.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc8295d5700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8241f54c0 con 0x7fc82415c150 2023-11-08T08:20:08.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc828dd4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc82415fc30 0x7fc8240c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:08.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc828dd4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc82415fc30 0x7fc8240c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40020/0 (socket says 172.21.15.40:40020) 2023-11-08T08:20:08.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc828dd4700 1 -- 172.21.15.40:0/1896311271 learned_addr learned my addr 172.21.15.40:0/1896311271 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:08.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc823fff700 1 --2- 172.21.15.40:0/1896311271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc82406ff10 0x7fc824150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:08.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc81bfff700 1 --2- 172.21.15.40:0/1896311271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc82415c150 0x7fc824159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:08.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc828dd4700 1 -- 172.21.15.40:0/1896311271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc82415c150 msgr2=0x7fc824159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:08.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc828dd4700 1 --2- 172.21.15.40:0/1896311271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc82415c150 0x7fc824159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc828dd4700 1 -- 172.21.15.40:0/1896311271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc82406ff10 msgr2=0x7fc824150490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:08.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc828dd4700 1 --2- 172.21.15.40:0/1896311271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc82406ff10 0x7fc824150490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc828dd4700 1 -- 172.21.15.40:0/1896311271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80c009520 con 0x7fc82415fc30 2023-11-08T08:20:08.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc823fff700 1 --2- 172.21.15.40:0/1896311271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc82406ff10 0x7fc824150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:08.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc81bfff700 1 --2- 172.21.15.40:0/1896311271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc82415c150 0x7fc824159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:08.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc828dd4700 1 --2- 172.21.15.40:0/1896311271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc82415fc30 0x7fc8240c8d00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00efc0 tx=0x7fc81c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:08.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc8217fa700 1 -- 172.21.15.40:0/1896311271 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c009ba0 con 0x7fc82415fc30 2023-11-08T08:20:08.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc8217fa700 1 -- 172.21.15.40:0/1896311271 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81c00f040 con 0x7fc82415fc30 2023-11-08T08:20:08.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc8295d5700 1 -- 172.21.15.40:0/1896311271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8140094b0 con 0x7fc82415fc30 2023-11-08T08:20:08.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc8217fa700 1 -- 172.21.15.40:0/1896311271 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c014480 con 0x7fc82415fc30 2023-11-08T08:20:08.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.242+0000 7fc8295d5700 1 -- 172.21.15.40:0/1896311271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8240d2730 con 0x7fc82415fc30 2023-11-08T08:20:08.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.246+0000 7fc8295d5700 1 -- 172.21.15.40:0/1896311271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8240d2450 con 0x7fc82415fc30 2023-11-08T08:20:08.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.246+0000 7fc8217fa700 1 -- 172.21.15.40:0/1896311271 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc81c0145e0 con 0x7fc82415fc30 2023-11-08T08:20:08.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.250+0000 7fc8217fa700 1 --2- 172.21.15.40:0/1896311271 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc804064ee0 0x7fc804067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:08.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.250+0000 7fc8217fa700 1 -- 172.21.15.40:0/1896311271 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc81c082c70 con 0x7fc82415fc30 2023-11-08T08:20:08.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.250+0000 7fc823fff700 1 --2- 172.21.15.40:0/1896311271 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc804064ee0 0x7fc804067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:08.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.250+0000 7fc823fff700 1 --2- 172.21.15.40:0/1896311271 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc804064ee0 0x7fc804067390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc814009fd0 tx=0x7fc814008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:08.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.254+0000 7fc8217fa700 1 -- 172.21.15.40:0/1896311271 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc81c0509f0 con 0x7fc82415fc30 2023-11-08T08:20:08.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.426+0000 7fc8295d5700 1 -- 172.21.15.40:0/1896311271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc8240d05e0 con 0x7fc82415fc30 2023-11-08T08:20:08.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.430+0000 7fc8217fa700 1 -- 172.21.15.40:0/1896311271 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc81c054010 con 0x7fc82415fc30 2023-11-08T08:20:08.433 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:20:08.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.430+0000 7fc8295d5700 1 -- 172.21.15.40:0/1896311271 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc804064ee0 msgr2=0x7fc804067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:08.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.430+0000 7fc8295d5700 1 --2- 172.21.15.40:0/1896311271 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc804064ee0 0x7fc804067390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc814009fd0 tx=0x7fc814008030 comp rx=0 tx=0).stop 2023-11-08T08:20:08.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.430+0000 7fc8295d5700 1 -- 172.21.15.40:0/1896311271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc82415fc30 msgr2=0x7fc8240c8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:08.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.430+0000 7fc8295d5700 1 --2- 172.21.15.40:0/1896311271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc82415fc30 0x7fc8240c8d00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00efc0 tx=0x7fc81c00c3a0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.430+0000 7fc8295d5700 1 -- 172.21.15.40:0/1896311271 shutdown_connections 2023-11-08T08:20:08.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.430+0000 7fc8295d5700 1 --2- 172.21.15.40:0/1896311271 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc804064ee0 0x7fc804067390 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.430+0000 7fc8295d5700 1 --2- 172.21.15.40:0/1896311271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc82415fc30 0x7fc8240c8d00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.430+0000 7fc8295d5700 1 --2- 172.21.15.40:0/1896311271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc82415c150 0x7fc824159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.430+0000 7fc8295d5700 1 --2- 172.21.15.40:0/1896311271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc82406ff10 0x7fc824150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.430+0000 7fc8295d5700 1 -- 172.21.15.40:0/1896311271 >> 172.21.15.40:0/1896311271 conn(0x7fc8240d3680 msgr2=0x7fc8241571b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:08.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.430+0000 7fc8295d5700 1 -- 172.21.15.40:0/1896311271 shutdown_connections 2023-11-08T08:20:08.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.430+0000 7fc8295d5700 1 -- 172.21.15.40:0/1896311271 wait complete. 2023-11-08T08:20:08.457 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T08:20:08.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.634+0000 7f29e65f9700 1 Processor -- start 2023-11-08T08:20:08.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.638+0000 7f29e65f9700 1 -- start start 2023-11-08T08:20:08.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.638+0000 7f29e65f9700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f29e00c8d10 0x7f29e00cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:08.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.638+0000 7f29e65f9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f29e0069d60 0x7f29e006a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:08.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.638+0000 7f29e65f9700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e015a850 con 0x7f29e015a480 2023-11-08T08:20:08.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.638+0000 7f29e65f9700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e00ce060 con 0x7f29e0069d60 2023-11-08T08:20:08.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.638+0000 7f29e65f9700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e00ce300 con 0x7f29e00c8d10 2023-11-08T08:20:08.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.638+0000 7f29e4b96700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f29e0069d60 0x7f29e006a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:08.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.638+0000 7f29dffff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f29e015a480 0x7f29e00cc290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59406/0 (socket says 172.21.15.40:59406) 2023-11-08T08:20:08.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.638+0000 7f29dffff700 1 -- 172.21.15.40:0/611925661 learned_addr learned my addr 172.21.15.40:0/611925661 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:08.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.638+0000 7f29df7fe700 1 --2- 172.21.15.40:0/611925661 >> v2:172.21.15.169:3300/0 conn(0x7f29e00c8d10 0x7f29e00cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:08.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.638+0000 7f29deffd700 1 -- 172.21.15.40:0/611925661 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2567550576 0 0) 0x7f29e015a850 con 0x7f29e015a480 2023-11-08T08:20:08.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.638+0000 7f29deffd700 1 -- 172.21.15.40:0/611925661 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29c80035d0 con 0x7f29e015a480 2023-11-08T08:20:08.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.638+0000 7f29e4b96700 1 -- 172.21.15.40:0/611925661 >> v2:172.21.15.169:3300/0 conn(0x7f29e00c8d10 msgr2=0x7f29e00cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:08.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.638+0000 7f29e4b96700 1 --2- 172.21.15.40:0/611925661 >> v2:172.21.15.169:3300/0 conn(0x7f29e00c8d10 0x7f29e00cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.638+0000 7f29e4b96700 1 -- 172.21.15.40:0/611925661 >> v1:172.21.15.40:6789/0 conn(0x7f29e015a480 legacy=0x7f29e00cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:08.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.638+0000 7f29e4b96700 1 -- 172.21.15.40:0/611925661 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e00cada0 con 0x7f29e0069d60 2023-11-08T08:20:08.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.642+0000 7f29e4b96700 1 --2- 172.21.15.40:0/611925661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29e0069d60 0x7f29e006a1e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f29e00b1230 tx=0x7f29d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898d07496d811062 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:08.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.642+0000 7f29deffd700 1 -- 172.21.15.40:0/611925661 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d800e070 con 0x7f29e0069d60 2023-11-08T08:20:08.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.642+0000 7f29deffd700 1 -- 172.21.15.40:0/611925661 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d800bb20 con 0x7f29e0069d60 2023-11-08T08:20:08.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.642+0000 7f29deffd700 1 -- 172.21.15.40:0/611925661 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d8012430 con 0x7f29e0069d60 2023-11-08T08:20:08.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.642+0000 7f29e65f9700 1 -- 172.21.15.40:0/611925661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29e0069d60 msgr2=0x7f29e006a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:08.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.642+0000 7f29e65f9700 1 --2- 172.21.15.40:0/611925661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29e0069d60 0x7f29e006a1e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f29e00b1230 tx=0x7f29d800b330 comp rx=0 tx=0).stop 2023-11-08T08:20:08.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.642+0000 7f29e65f9700 1 -- 172.21.15.40:0/611925661 shutdown_connections 2023-11-08T08:20:08.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.642+0000 7f29e65f9700 1 --2- 172.21.15.40:0/611925661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29e0069d60 0x7f29e006a1e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.642+0000 7f29e65f9700 1 --2- 172.21.15.40:0/611925661 >> v2:172.21.15.169:3300/0 conn(0x7f29e00c8d10 0x7f29e00cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.642+0000 7f29e65f9700 1 -- 172.21.15.40:0/611925661 >> 172.21.15.40:0/611925661 conn(0x7f29e00cf350 msgr2=0x7f29e00b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:08.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.642+0000 7f29e65f9700 1 -- 172.21.15.40:0/611925661 shutdown_connections 2023-11-08T08:20:08.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.642+0000 7f29e65f9700 1 -- 172.21.15.40:0/611925661 wait complete. 2023-11-08T08:20:08.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.646+0000 7f29e65f9700 1 Processor -- start 2023-11-08T08:20:08.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.646+0000 7f29e65f9700 1 -- start start 2023-11-08T08:20:08.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.646+0000 7f29e65f9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29e0069d60 0x7f29e01551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:08.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.646+0000 7f29e65f9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e00c8d10 0x7f29e00b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:08.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.646+0000 7f29e65f9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29e015a480 0x7f29e00b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:08.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.646+0000 7f29e65f9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e00b43b0 con 0x7f29e015a480 2023-11-08T08:20:08.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.646+0000 7f29e65f9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e00c5b10 con 0x7f29e00c8d10 2023-11-08T08:20:08.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.646+0000 7f29e65f9700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e00b3b10 con 0x7f29e0069d60 2023-11-08T08:20:08.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.646+0000 7f29e4b96700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29e015a480 0x7f29e00b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:08.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.650+0000 7f29e4b96700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29e015a480 0x7f29e00b3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40040/0 (socket says 172.21.15.40:40040) 2023-11-08T08:20:08.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.650+0000 7f29e4b96700 1 -- 172.21.15.40:0/3186239444 learned_addr learned my addr 172.21.15.40:0/3186239444 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:08.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.650+0000 7f29dffff700 1 --2- 172.21.15.40:0/3186239444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29e0069d60 0x7f29e01551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:08.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.650+0000 7f29df7fe700 1 --2- 172.21.15.40:0/3186239444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e00c8d10 0x7f29e00b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:08.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.650+0000 7f29e4b96700 1 -- 172.21.15.40:0/3186239444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29e0069d60 msgr2=0x7f29e01551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:08.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.650+0000 7f29e4b96700 1 --2- 172.21.15.40:0/3186239444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29e0069d60 0x7f29e01551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.650+0000 7f29e4b96700 1 -- 172.21.15.40:0/3186239444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e00c8d10 msgr2=0x7f29e00b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:08.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.650+0000 7f29e4b96700 1 --2- 172.21.15.40:0/3186239444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e00c8d10 0x7f29e00b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.650+0000 7f29e4b96700 1 -- 172.21.15.40:0/3186239444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d8009a70 con 0x7f29e015a480 2023-11-08T08:20:08.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.650+0000 7f29df7fe700 1 --2- 172.21.15.40:0/3186239444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e00c8d10 0x7f29e00b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:20:08.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.650+0000 7f29e4b96700 1 --2- 172.21.15.40:0/3186239444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29e015a480 0x7f29e00b3460 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f29d8014040 tx=0x7f29d8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:08.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.650+0000 7f29dcff9700 1 -- 172.21.15.40:0/3186239444 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d8012430 con 0x7f29e015a480 2023-11-08T08:20:08.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.650+0000 7f29dcff9700 1 -- 172.21.15.40:0/3186239444 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d8020070 con 0x7f29e015a480 2023-11-08T08:20:08.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.650+0000 7f29e65f9700 1 -- 172.21.15.40:0/3186239444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e00add50 con 0x7f29e015a480 2023-11-08T08:20:08.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.650+0000 7f29dcff9700 1 -- 172.21.15.40:0/3186239444 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d8004280 con 0x7f29e015a480 2023-11-08T08:20:08.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.650+0000 7f29e65f9700 1 -- 172.21.15.40:0/3186239444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e00caa30 con 0x7f29e015a480 2023-11-08T08:20:08.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.650+0000 7f29e65f9700 1 -- 172.21.15.40:0/3186239444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e00ca6e0 con 0x7f29e015a480 2023-11-08T08:20:08.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.654+0000 7f29dcff9700 1 -- 172.21.15.40:0/3186239444 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f29d80043e0 con 0x7f29e015a480 2023-11-08T08:20:08.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.654+0000 7f29dcff9700 1 --2- 172.21.15.40:0/3186239444 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f29cc064ee0 0x7f29cc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:08.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.654+0000 7f29dcff9700 1 -- 172.21.15.40:0/3186239444 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f29d8022070 con 0x7f29e015a480 2023-11-08T08:20:08.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.654+0000 7f29dffff700 1 --2- 172.21.15.40:0/3186239444 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f29cc064ee0 0x7f29cc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:08.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.658+0000 7f29dffff700 1 --2- 172.21.15.40:0/3186239444 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f29cc064ee0 0x7f29cc067390 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f29d00011e0 tx=0x7f29d0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:08.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.662+0000 7f29dcff9700 1 -- 172.21.15.40:0/3186239444 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f29d8052380 con 0x7f29e015a480 2023-11-08T08:20:08.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.854+0000 7f29e65f9700 1 -- 172.21.15.40:0/3186239444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f29e00c58e0 con 0x7f29e015a480 2023-11-08T08:20:08.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.854+0000 7f29dcff9700 1 -- 172.21.15.40:0/3186239444 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f29e00c58e0 con 0x7f29e015a480 2023-11-08T08:20:08.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.862+0000 7f29e65f9700 1 -- 172.21.15.40:0/3186239444 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f29cc064ee0 msgr2=0x7f29cc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:08.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.862+0000 7f29e65f9700 1 --2- 172.21.15.40:0/3186239444 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f29cc064ee0 0x7f29cc067390 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f29d00011e0 tx=0x7f29d0009040 comp rx=0 tx=0).stop 2023-11-08T08:20:08.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.862+0000 7f29e65f9700 1 -- 172.21.15.40:0/3186239444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29e015a480 msgr2=0x7f29e00b3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:08.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.862+0000 7f29e65f9700 1 --2- 172.21.15.40:0/3186239444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29e015a480 0x7f29e00b3460 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f29d8014040 tx=0x7f29d8004af0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.862+0000 7f29e65f9700 1 -- 172.21.15.40:0/3186239444 shutdown_connections 2023-11-08T08:20:08.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.862+0000 7f29e65f9700 1 --2- 172.21.15.40:0/3186239444 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f29cc064ee0 0x7f29cc067390 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.862+0000 7f29e65f9700 1 --2- 172.21.15.40:0/3186239444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29e015a480 0x7f29e00b3460 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.862+0000 7f29e65f9700 1 --2- 172.21.15.40:0/3186239444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29e00c8d10 0x7f29e00b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.862+0000 7f29e65f9700 1 --2- 172.21.15.40:0/3186239444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29e0069d60 0x7f29e01551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:08.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.862+0000 7f29e65f9700 1 -- 172.21.15.40:0/3186239444 >> 172.21.15.40:0/3186239444 conn(0x7f29e00cf350 msgr2=0x7f29e00cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:08.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.862+0000 7f29e65f9700 1 -- 172.21.15.40:0/3186239444 shutdown_connections 2023-11-08T08:20:08.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:08.862+0000 7f29e65f9700 1 -- 172.21.15.40:0/3186239444 wait complete. 2023-11-08T08:20:08.867 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-11-08T08:20:08.887 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-08T08:20:08.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:08.888 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-08T08:20:08.947 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2023-11-08T08:20:08.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:08.948 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-08T08:20:08.974 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:20:08.974 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:20:08.995 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:20:08.995 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:20:09.053 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:20:09.054 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:20:09.113 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:20:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.334+0000 7f745a412700 1 Processor -- start 2023-11-08T08:20:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.338+0000 7f745a412700 1 -- start start 2023-11-08T08:20:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.338+0000 7f745a412700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f745415c640 0x7f74540d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.338+0000 7f745a412700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7454071aa0 0x7f745406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:09.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.338+0000 7f745a412700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7454069e00 0x7f745406a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:09.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.338+0000 7f745a412700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f745406cef0 con 0x7f745415c640 2023-11-08T08:20:09.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.338+0000 7f745a412700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f745406a7c0 con 0x7f7454071aa0 2023-11-08T08:20:09.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.338+0000 7f745a412700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745406c600 con 0x7f7454069e00 2023-11-08T08:20:09.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.338+0000 7f7459410700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f745415c640 0x7f74540d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:09.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f7459410700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f745415c640 0x7f74540d31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40048/0 (socket says 172.21.15.40:40048) 2023-11-08T08:20:09.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f7458c0f700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7454071aa0 0x7f745406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:09.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f7459410700 1 -- 172.21.15.40:0/1079503068 learned_addr learned my addr 172.21.15.40:0/1079503068 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:09.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f7459c11700 1 --2- 172.21.15.40:0/1079503068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7454069e00 0x7f745406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:09.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f7459410700 1 -- 172.21.15.40:0/1079503068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7454069e00 msgr2=0x7f745406a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:09.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f7459410700 1 --2- 172.21.15.40:0/1079503068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7454069e00 0x7f745406a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f7459410700 1 -- 172.21.15.40:0/1079503068 >> v2:172.21.15.169:3300/0 conn(0x7f7454071aa0 msgr2=0x7f745406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:09.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f7459410700 1 --2- 172.21.15.40:0/1079503068 >> v2:172.21.15.169:3300/0 conn(0x7f7454071aa0 0x7f745406d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f7459410700 1 -- 172.21.15.40:0/1079503068 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74540722f0 con 0x7f745415c640 2023-11-08T08:20:09.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f7458c0f700 1 --2- 172.21.15.40:0/1079503068 >> v2:172.21.15.169:3300/0 conn(0x7f7454071aa0 0x7f745406d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:09.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f7459410700 1 --2- 172.21.15.40:0/1079503068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745415c640 0x7f74540d31e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7454070e70 tx=0x7f7450009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79f5e03a12fc10d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f744bfff700 1 -- 172.21.15.40:0/1079503068 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450011830 con 0x7f745415c640 2023-11-08T08:20:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f744bfff700 1 -- 172.21.15.40:0/1079503068 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7450011990 con 0x7f745415c640 2023-11-08T08:20:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f744bfff700 1 -- 172.21.15.40:0/1079503068 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450011d20 con 0x7f745415c640 2023-11-08T08:20:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f745a412700 1 -- 172.21.15.40:0/1079503068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745415c640 msgr2=0x7f74540d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:09.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f745a412700 1 --2- 172.21.15.40:0/1079503068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745415c640 0x7f74540d31e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7454070e70 tx=0x7f7450009bf0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f745a412700 1 -- 172.21.15.40:0/1079503068 shutdown_connections 2023-11-08T08:20:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f745a412700 1 --2- 172.21.15.40:0/1079503068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7454069e00 0x7f745406a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f745a412700 1 --2- 172.21.15.40:0/1079503068 >> v2:172.21.15.169:3300/0 conn(0x7f7454071aa0 0x7f745406d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f745a412700 1 --2- 172.21.15.40:0/1079503068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745415c640 0x7f74540d31e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f745a412700 1 -- 172.21.15.40:0/1079503068 >> 172.21.15.40:0/1079503068 conn(0x7f74540b6520 msgr2=0x7f7454071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:09.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f745a412700 1 -- 172.21.15.40:0/1079503068 shutdown_connections 2023-11-08T08:20:09.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.342+0000 7f745a412700 1 -- 172.21.15.40:0/1079503068 wait complete. 2023-11-08T08:20:09.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.346+0000 7f745a412700 1 Processor -- start 2023-11-08T08:20:09.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.346+0000 7f745a412700 1 -- start start 2023-11-08T08:20:09.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.346+0000 7f745a412700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7454069e00 0x7f74540b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:09.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.346+0000 7f745a412700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7454071aa0 0x7f74540cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:09.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.346+0000 7f745a412700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745415c640 0x7f74540c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:09.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.346+0000 7f745a412700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74540cdd30 con 0x7f745415c640 2023-11-08T08:20:09.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.346+0000 7f745a412700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74540cdfb0 con 0x7f7454069e00 2023-11-08T08:20:09.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.346+0000 7f745a412700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74540ce290 con 0x7f7454071aa0 2023-11-08T08:20:09.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.346+0000 7f7459c11700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745415c640 0x7f74540c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:09.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.350+0000 7f7459c11700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745415c640 0x7f74540c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40058/0 (socket says 172.21.15.40:40058) 2023-11-08T08:20:09.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.350+0000 7f7459c11700 1 -- 172.21.15.40:0/4237060250 learned_addr learned my addr 172.21.15.40:0/4237060250 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:09.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.350+0000 7f7458c0f700 1 --2- 172.21.15.40:0/4237060250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7454071aa0 0x7f74540cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:09.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.350+0000 7f7459410700 1 --2- 172.21.15.40:0/4237060250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7454069e00 0x7f74540b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:09.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.350+0000 7f7459c11700 1 -- 172.21.15.40:0/4237060250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7454071aa0 msgr2=0x7f74540cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:09.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.350+0000 7f7459c11700 1 --2- 172.21.15.40:0/4237060250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7454071aa0 0x7f74540cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.350+0000 7f7459c11700 1 -- 172.21.15.40:0/4237060250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7454069e00 msgr2=0x7f74540b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:09.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.350+0000 7f7459c11700 1 --2- 172.21.15.40:0/4237060250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7454069e00 0x7f74540b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.350+0000 7f7459c11700 1 -- 172.21.15.40:0/4237060250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74440094b0 con 0x7f745415c640 2023-11-08T08:20:09.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.350+0000 7f7458c0f700 1 --2- 172.21.15.40:0/4237060250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7454071aa0 0x7f74540cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:09.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.350+0000 7f7459410700 1 --2- 172.21.15.40:0/4237060250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7454069e00 0x7f74540b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:09.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.350+0000 7f7459c11700 1 --2- 172.21.15.40:0/4237060250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745415c640 0x7f74540c8ce0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f744c00eed0 tx=0x7f744c00b0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:09.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.350+0000 7f7449ffb700 1 -- 172.21.15.40:0/4237060250 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c010040 con 0x7f745415c640 2023-11-08T08:20:09.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.350+0000 7f7449ffb700 1 -- 172.21.15.40:0/4237060250 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744c00bb70 con 0x7f745415c640 2023-11-08T08:20:09.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.350+0000 7f7449ffb700 1 -- 172.21.15.40:0/4237060250 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c014400 con 0x7f745415c640 2023-11-08T08:20:09.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.350+0000 7f745a412700 1 -- 172.21.15.40:0/4237060250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7450009570 con 0x7f745415c640 2023-11-08T08:20:09.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.350+0000 7f745a412700 1 -- 172.21.15.40:0/4237060250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7454072010 con 0x7f745415c640 2023-11-08T08:20:09.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.354+0000 7f745a412700 1 -- 172.21.15.40:0/4237060250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745415ca10 con 0x7f745415c640 2023-11-08T08:20:09.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.354+0000 7f7449ffb700 1 -- 172.21.15.40:0/4237060250 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f744c014560 con 0x7f745415c640 2023-11-08T08:20:09.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.354+0000 7f7449ffb700 1 --2- 172.21.15.40:0/4237060250 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7438064f30 0x7f74380673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:09.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.358+0000 7f7449ffb700 1 -- 172.21.15.40:0/4237060250 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f744c082780 con 0x7f745415c640 2023-11-08T08:20:09.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.358+0000 7f7459410700 1 --2- 172.21.15.40:0/4237060250 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7438064f30 0x7f74380673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:09.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.358+0000 7f7459410700 1 --2- 172.21.15.40:0/4237060250 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7438064f30 0x7f74380673e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7450009860 tx=0x7f7450008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:09.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.362+0000 7f7449ffb700 1 -- 172.21.15.40:0/4237060250 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f744c050520 con 0x7f745415c640 2023-11-08T08:20:09.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.538+0000 7f745a412700 1 -- 172.21.15.40:0/4237060250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f745406c830 con 0x7f745415c640 2023-11-08T08:20:09.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.542+0000 7f7449ffb700 1 -- 172.21.15.40:0/4237060250 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19748 (secure 0 0 0) 0x7f744c053b40 con 0x7f745415c640 2023-11-08T08:20:09.546 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:09.547 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":30,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:45.885827+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:20:09.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.546+0000 7f745a412700 1 -- 172.21.15.40:0/4237060250 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7438064f30 msgr2=0x7f74380673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:09.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.546+0000 7f745a412700 1 --2- 172.21.15.40:0/4237060250 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7438064f30 0x7f74380673e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7450009860 tx=0x7f7450008030 comp rx=0 tx=0).stop 2023-11-08T08:20:09.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.546+0000 7f745a412700 1 -- 172.21.15.40:0/4237060250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745415c640 msgr2=0x7f74540c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:09.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.546+0000 7f745a412700 1 --2- 172.21.15.40:0/4237060250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745415c640 0x7f74540c8ce0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f744c00eed0 tx=0x7f744c00b0b0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.546+0000 7f745a412700 1 -- 172.21.15.40:0/4237060250 shutdown_connections 2023-11-08T08:20:09.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.546+0000 7f745a412700 1 --2- 172.21.15.40:0/4237060250 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7438064f30 0x7f74380673e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.546+0000 7f745a412700 1 --2- 172.21.15.40:0/4237060250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745415c640 0x7f74540c8ce0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.546+0000 7f745a412700 1 --2- 172.21.15.40:0/4237060250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7454071aa0 0x7f74540cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.546+0000 7f745a412700 1 --2- 172.21.15.40:0/4237060250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7454069e00 0x7f74540b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.546+0000 7f745a412700 1 -- 172.21.15.40:0/4237060250 >> 172.21.15.40:0/4237060250 conn(0x7f74540b6520 msgr2=0x7f74540b03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:09.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.546+0000 7f745a412700 1 -- 172.21.15.40:0/4237060250 shutdown_connections 2023-11-08T08:20:09.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.546+0000 7f745a412700 1 -- 172.21.15.40:0/4237060250 wait complete. 2023-11-08T08:20:09.570 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T08:19:02.627691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T08:19:03.473821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T08:19:06.539942+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T08:20:09.572 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:20:09.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.754+0000 7f9d156cb700 1 Processor -- start 2023-11-08T08:20:09.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.754+0000 7f9d156cb700 1 -- start start 2023-11-08T08:20:09.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.758+0000 7f9d156cb700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1015c150 0x7f9d100b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:09.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.758+0000 7f9d156cb700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9d1006ff10 0x7f9d10070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:09.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.758+0000 7f9d156cb700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9d1015fc30 0x7f9d101600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:09.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.758+0000 7f9d156cb700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d100b4240 con 0x7f9d1015fc30 2023-11-08T08:20:09.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.758+0000 7f9d156cb700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1015c520 con 0x7f9d1006ff10 2023-11-08T08:20:09.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.758+0000 7f9d156cb700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d100708a0 con 0x7f9d1015c150 2023-11-08T08:20:09.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.758+0000 7f9d14eca700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9d1015fc30 0x7f9d101600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.758+0000 7f9d077fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9d1006ff10 0x7f9d10070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.758+0000 7f9d14eca700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9d1015fc30 0x7f9d101600b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40068/0 (socket says 172.21.15.40:40068) 2023-11-08T08:20:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.758+0000 7f9d14eca700 1 -- 172.21.15.40:0/2966717828 learned_addr learned my addr 172.21.15.40:0/2966717828 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.758+0000 7f9d0ffff700 1 --2- 172.21.15.40:0/2966717828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1015c150 0x7f9d100b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.758+0000 7f9d14eca700 1 -- 172.21.15.40:0/2966717828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1015c150 msgr2=0x7f9d100b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:09.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.758+0000 7f9d14eca700 1 --2- 172.21.15.40:0/2966717828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1015c150 0x7f9d100b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.758+0000 7f9d14eca700 1 -- 172.21.15.40:0/2966717828 >> v2:172.21.15.169:3300/0 conn(0x7f9d1006ff10 msgr2=0x7f9d10070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:09.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.758+0000 7f9d14eca700 1 --2- 172.21.15.40:0/2966717828 >> v2:172.21.15.169:3300/0 conn(0x7f9d1006ff10 0x7f9d10070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.758+0000 7f9d14eca700 1 -- 172.21.15.40:0/2966717828 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1006cc50 con 0x7f9d1015fc30 2023-11-08T08:20:09.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.758+0000 7f9d077fe700 1 --2- 172.21.15.40:0/2966717828 >> v2:172.21.15.169:3300/0 conn(0x7f9d1006ff10 0x7f9d10070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:09.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.762+0000 7f9d14eca700 1 --2- 172.21.15.40:0/2966717828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d1015fc30 0x7f9d101600b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9d1006bd50 tx=0x7f9d0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f9f90e9cb4bdc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:09.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.762+0000 7f9d0f7fe700 1 -- 172.21.15.40:0/2966717828 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800e070 con 0x7f9d1015fc30 2023-11-08T08:20:09.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.762+0000 7f9d0f7fe700 1 -- 172.21.15.40:0/2966717828 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0800b8d0 con 0x7f9d1015fc30 2023-11-08T08:20:09.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.762+0000 7f9d0f7fe700 1 -- 172.21.15.40:0/2966717828 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800bc80 con 0x7f9d1015fc30 2023-11-08T08:20:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.762+0000 7f9d156cb700 1 -- 172.21.15.40:0/2966717828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d1015fc30 msgr2=0x7f9d101600b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.762+0000 7f9d156cb700 1 --2- 172.21.15.40:0/2966717828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d1015fc30 0x7f9d101600b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9d1006bd50 tx=0x7f9d0800b330 comp rx=0 tx=0).stop 2023-11-08T08:20:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.762+0000 7f9d156cb700 1 -- 172.21.15.40:0/2966717828 shutdown_connections 2023-11-08T08:20:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.762+0000 7f9d156cb700 1 --2- 172.21.15.40:0/2966717828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d1015fc30 0x7f9d101600b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.762+0000 7f9d156cb700 1 --2- 172.21.15.40:0/2966717828 >> v2:172.21.15.169:3300/0 conn(0x7f9d1006ff10 0x7f9d10070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.762+0000 7f9d156cb700 1 --2- 172.21.15.40:0/2966717828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1015c150 0x7f9d100b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.762+0000 7f9d156cb700 1 -- 172.21.15.40:0/2966717828 >> 172.21.15.40:0/2966717828 conn(0x7f9d100d3680 msgr2=0x7f9d100cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:09.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.762+0000 7f9d156cb700 1 -- 172.21.15.40:0/2966717828 shutdown_connections 2023-11-08T08:20:09.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.762+0000 7f9d156cb700 1 -- 172.21.15.40:0/2966717828 wait complete. 2023-11-08T08:20:09.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.766+0000 7f9d156cb700 1 Processor -- start 2023-11-08T08:20:09.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.766+0000 7f9d156cb700 1 -- start start 2023-11-08T08:20:09.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.766+0000 7f9d156cb700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1006ff10 0x7f9d101515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:09.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.766+0000 7f9d156cb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d1015c150 0x7f9d100b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:09.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.766+0000 7f9d156cb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d1015fc30 0x7f9d100b7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:09.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.766+0000 7f9d156cb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1015d360 con 0x7f9d1015fc30 2023-11-08T08:20:09.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.766+0000 7f9d156cb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d10151ae0 con 0x7f9d1015c150 2023-11-08T08:20:09.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.766+0000 7f9d156cb700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1015c750 con 0x7f9d1006ff10 2023-11-08T08:20:09.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.766+0000 7f9d14eca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d1015fc30 0x7f9d100b7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:09.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.766+0000 7f9d14eca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d1015fc30 0x7f9d100b7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40078/0 (socket says 172.21.15.40:40078) 2023-11-08T08:20:09.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.766+0000 7f9d14eca700 1 -- 172.21.15.40:0/4195344108 learned_addr learned my addr 172.21.15.40:0/4195344108 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:09.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.766+0000 7f9d077fe700 1 --2- 172.21.15.40:0/4195344108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d1015c150 0x7f9d100b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:09.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.766+0000 7f9d0ffff700 1 --2- 172.21.15.40:0/4195344108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1006ff10 0x7f9d101515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:09.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.766+0000 7f9d14eca700 1 -- 172.21.15.40:0/4195344108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1006ff10 msgr2=0x7f9d101515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:09.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.766+0000 7f9d14eca700 1 --2- 172.21.15.40:0/4195344108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1006ff10 0x7f9d101515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.766+0000 7f9d14eca700 1 -- 172.21.15.40:0/4195344108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d1015c150 msgr2=0x7f9d100b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:09.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.766+0000 7f9d14eca700 1 --2- 172.21.15.40:0/4195344108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d1015c150 0x7f9d100b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.766+0000 7f9d14eca700 1 -- 172.21.15.40:0/4195344108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf8009450 con 0x7f9d1015fc30 2023-11-08T08:20:09.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.766+0000 7f9d0ffff700 1 --2- 172.21.15.40:0/4195344108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1006ff10 0x7f9d101515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:09.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.770+0000 7f9d14eca700 1 --2- 172.21.15.40:0/4195344108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d1015fc30 0x7f9d100b7480 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9d10073200 tx=0x7f9d08004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:09.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.770+0000 7f9d0d7fa700 1 -- 172.21.15.40:0/4195344108 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d08003e30 con 0x7f9d1015fc30 2023-11-08T08:20:09.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.770+0000 7f9d0d7fa700 1 -- 172.21.15.40:0/4195344108 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d08004250 con 0x7f9d1015fc30 2023-11-08T08:20:09.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.770+0000 7f9d0d7fa700 1 -- 172.21.15.40:0/4195344108 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d08004600 con 0x7f9d1015fc30 2023-11-08T08:20:09.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.770+0000 7f9d156cb700 1 -- 172.21.15.40:0/4195344108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d08009b30 con 0x7f9d1015fc30 2023-11-08T08:20:09.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.770+0000 7f9d156cb700 1 -- 172.21.15.40:0/4195344108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d100d2730 con 0x7f9d1015fc30 2023-11-08T08:20:09.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.770+0000 7f9d156cb700 1 -- 172.21.15.40:0/4195344108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d100d2450 con 0x7f9d1015fc30 2023-11-08T08:20:09.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.774+0000 7f9d0d7fa700 1 -- 172.21.15.40:0/4195344108 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f9d0802b070 con 0x7f9d1015fc30 2023-11-08T08:20:09.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.774+0000 7f9d0d7fa700 1 --2- 172.21.15.40:0/4195344108 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9cf0064ee0 0x7f9cf0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:09.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.774+0000 7f9d0d7fa700 1 -- 172.21.15.40:0/4195344108 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9d08083120 con 0x7f9d1015fc30 2023-11-08T08:20:09.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.774+0000 7f9d0ffff700 1 --2- 172.21.15.40:0/4195344108 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9cf0064ee0 0x7f9cf0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:09.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.778+0000 7f9d0ffff700 1 --2- 172.21.15.40:0/4195344108 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9cf0064ee0 0x7f9cf0067390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009c40 tx=0x7f9d00008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:09.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.778+0000 7f9d0d7fa700 1 -- 172.21.15.40:0/4195344108 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f9d08050ea0 con 0x7f9d1015fc30 2023-11-08T08:20:09.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.954+0000 7f9d156cb700 1 -- 172.21.15.40:0/4195344108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9d100d05e0 con 0x7f9d1015fc30 2023-11-08T08:20:09.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.954+0000 7f9d0d7fa700 1 -- 172.21.15.40:0/4195344108 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9d080544c0 con 0x7f9d1015fc30 2023-11-08T08:20:09.957 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:20:09.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.954+0000 7f9d156cb700 1 -- 172.21.15.40:0/4195344108 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9cf0064ee0 msgr2=0x7f9cf0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:09.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.954+0000 7f9d156cb700 1 --2- 172.21.15.40:0/4195344108 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9cf0064ee0 0x7f9cf0067390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009c40 tx=0x7f9d00008040 comp rx=0 tx=0).stop 2023-11-08T08:20:09.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.954+0000 7f9d156cb700 1 -- 172.21.15.40:0/4195344108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d1015fc30 msgr2=0x7f9d100b7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:09.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.954+0000 7f9d156cb700 1 --2- 172.21.15.40:0/4195344108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d1015fc30 0x7f9d100b7480 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9d10073200 tx=0x7f9d08004af0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.954+0000 7f9d156cb700 1 -- 172.21.15.40:0/4195344108 shutdown_connections 2023-11-08T08:20:09.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.954+0000 7f9d156cb700 1 --2- 172.21.15.40:0/4195344108 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9cf0064ee0 0x7f9cf0067390 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.954+0000 7f9d156cb700 1 --2- 172.21.15.40:0/4195344108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d1015fc30 0x7f9d100b7480 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.954+0000 7f9d156cb700 1 --2- 172.21.15.40:0/4195344108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d1015c150 0x7f9d100b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.954+0000 7f9d156cb700 1 --2- 172.21.15.40:0/4195344108 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d1006ff10 0x7f9d101515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:09.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.954+0000 7f9d156cb700 1 -- 172.21.15.40:0/4195344108 >> 172.21.15.40:0/4195344108 conn(0x7f9d100d3680 msgr2=0x7f9d10160030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:09.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.958+0000 7f9d156cb700 1 -- 172.21.15.40:0/4195344108 shutdown_connections 2023-11-08T08:20:09.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:09.958+0000 7f9d156cb700 1 -- 172.21.15.40:0/4195344108 wait complete. 2023-11-08T08:20:09.985 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:20:10.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.178+0000 7fac72126700 1 Processor -- start 2023-11-08T08:20:10.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.178+0000 7fac72126700 1 -- start start 2023-11-08T08:20:10.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.178+0000 7fac72126700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fac6c070a00 0x7fac6c069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:10.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.178+0000 7fac72126700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac6c0b2130 0x7fac6c06a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:10.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.178+0000 7fac72126700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac6c0cd360 con 0x7fac6c15c700 2023-11-08T08:20:10.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.178+0000 7fac72126700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac6c160190 con 0x7fac6c070a00 2023-11-08T08:20:10.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.178+0000 7fac72126700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c160470 con 0x7fac6c0b2130 2023-11-08T08:20:10.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.178+0000 7fac63fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fac6c070a00 0x7fac6c069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:10.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.178+0000 7fac63fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fac6c070a00 0x7fac6c069bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40082/0 (socket says 172.21.15.40:40082) 2023-11-08T08:20:10.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.178+0000 7fac63fff700 1 -- 172.21.15.40:0/988430932 learned_addr learned my addr 172.21.15.40:0/988430932 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:10.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.178+0000 7fac6bfff700 1 --2- 172.21.15.40:0/988430932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac6c0b2130 0x7fac6c06a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:10.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.178+0000 7fac63fff700 1 -- 172.21.15.40:0/988430932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac6c0b2130 msgr2=0x7fac6c06a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:10.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.178+0000 7fac63fff700 1 --2- 172.21.15.40:0/988430932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac6c0b2130 0x7fac6c06a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:10.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.178+0000 7fac63fff700 1 -- 172.21.15.40:0/988430932 >> v1:172.21.15.169:6789/0 conn(0x7fac6c15c700 legacy=0x7fac6c0702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:10.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.178+0000 7fac63fff700 1 -- 172.21.15.40:0/988430932 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c160750 con 0x7fac6c070a00 2023-11-08T08:20:10.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.178+0000 7fac63fff700 1 --2- 172.21.15.40:0/988430932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac6c070a00 0x7fac6c069bf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fac54009910 tx=0x7fac54009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534b443097a7e38e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:10.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.182+0000 7fac6affd700 1 -- 172.21.15.40:0/988430932 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5400ba60 con 0x7fac6c070a00 2023-11-08T08:20:10.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.182+0000 7fac6affd700 1 -- 172.21.15.40:0/988430932 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5400bbc0 con 0x7fac6c070a00 2023-11-08T08:20:10.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.182+0000 7fac6affd700 1 -- 172.21.15.40:0/988430932 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac54009e60 con 0x7fac6c070a00 2023-11-08T08:20:10.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.182+0000 7fac72126700 1 -- 172.21.15.40:0/988430932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac6c070a00 msgr2=0x7fac6c069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.182+0000 7fac72126700 1 --2- 172.21.15.40:0/988430932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac6c070a00 0x7fac6c069bf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fac54009910 tx=0x7fac54009c20 comp rx=0 tx=0).stop 2023-11-08T08:20:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.182+0000 7fac72126700 1 -- 172.21.15.40:0/988430932 shutdown_connections 2023-11-08T08:20:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.182+0000 7fac72126700 1 --2- 172.21.15.40:0/988430932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac6c0b2130 0x7fac6c06a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.182+0000 7fac72126700 1 --2- 172.21.15.40:0/988430932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac6c070a00 0x7fac6c069bf0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.182+0000 7fac72126700 1 -- 172.21.15.40:0/988430932 >> 172.21.15.40:0/988430932 conn(0x7fac6c0d2d10 msgr2=0x7fac6c0d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.182+0000 7fac72126700 1 -- 172.21.15.40:0/988430932 shutdown_connections 2023-11-08T08:20:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.182+0000 7fac72126700 1 -- 172.21.15.40:0/988430932 wait complete. 2023-11-08T08:20:10.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac72126700 1 Processor -- start 2023-11-08T08:20:10.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac72126700 1 -- start start 2023-11-08T08:20:10.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac72126700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac6c070a00 0x7fac6c15d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:10.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac72126700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac6c0b2130 0x7fac6c15ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:10.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac72126700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac6c15c700 0x7fac6c1f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:10.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac72126700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c15f170 con 0x7fac6c0b2130 2023-11-08T08:20:10.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac72126700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c15db70 con 0x7fac6c15c700 2023-11-08T08:20:10.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac72126700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c15de50 con 0x7fac6c070a00 2023-11-08T08:20:10.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac63fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac6c0b2130 0x7fac6c15ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:10.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac63fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac6c0b2130 0x7fac6c15ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40090/0 (socket says 172.21.15.40:40090) 2023-11-08T08:20:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac63fff700 1 -- 172.21.15.40:0/3340699374 learned_addr learned my addr 172.21.15.40:0/3340699374 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac6b7fe700 1 --2- 172.21.15.40:0/3340699374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac6c070a00 0x7fac6c15d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac6bfff700 1 --2- 172.21.15.40:0/3340699374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac6c15c700 0x7fac6c1f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac63fff700 1 -- 172.21.15.40:0/3340699374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac6c070a00 msgr2=0x7fac6c15d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac63fff700 1 --2- 172.21.15.40:0/3340699374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac6c070a00 0x7fac6c15d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:10.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac63fff700 1 -- 172.21.15.40:0/3340699374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac6c15c700 msgr2=0x7fac6c1f15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:10.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac63fff700 1 --2- 172.21.15.40:0/3340699374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac6c15c700 0x7fac6c1f15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:10.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac63fff700 1 -- 172.21.15.40:0/3340699374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac54009570 con 0x7fac6c0b2130 2023-11-08T08:20:10.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac6bfff700 1 --2- 172.21.15.40:0/3340699374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac6c15c700 0x7fac6c1f15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:20:10.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac63fff700 1 --2- 172.21.15.40:0/3340699374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac6c0b2130 0x7fac6c15ec30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fac54013000 tx=0x7fac54004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:10.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac68ff9700 1 -- 172.21.15.40:0/3340699374 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac54004610 con 0x7fac6c0b2130 2023-11-08T08:20:10.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac68ff9700 1 -- 172.21.15.40:0/3340699374 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5401f070 con 0x7fac6c0b2130 2023-11-08T08:20:10.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac68ff9700 1 -- 172.21.15.40:0/3340699374 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac54003900 con 0x7fac6c0b2130 2023-11-08T08:20:10.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac72126700 1 -- 172.21.15.40:0/3340699374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac6c06b530 con 0x7fac6c0b2130 2023-11-08T08:20:10.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.186+0000 7fac72126700 1 -- 172.21.15.40:0/3340699374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6c160a30 con 0x7fac6c0b2130 2023-11-08T08:20:10.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.190+0000 7fac72126700 1 -- 172.21.15.40:0/3340699374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6c0b1be0 con 0x7fac6c0b2130 2023-11-08T08:20:10.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.190+0000 7fac68ff9700 1 -- 172.21.15.40:0/3340699374 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fac5400f050 con 0x7fac6c0b2130 2023-11-08T08:20:10.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.190+0000 7fac68ff9700 1 --2- 172.21.15.40:0/3340699374 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fac4c064ee0 0x7fac4c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:10.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.194+0000 7fac68ff9700 1 -- 172.21.15.40:0/3340699374 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fac54021070 con 0x7fac6c0b2130 2023-11-08T08:20:10.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.194+0000 7fac6b7fe700 1 --2- 172.21.15.40:0/3340699374 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fac4c064ee0 0x7fac4c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:10.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.194+0000 7fac6b7fe700 1 --2- 172.21.15.40:0/3340699374 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fac4c064ee0 0x7fac4c067390 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fac5c000f60 tx=0x7fac5c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:10.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.198+0000 7fac68ff9700 1 -- 172.21.15.40:0/3340699374 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fac540518d0 con 0x7fac6c0b2130 2023-11-08T08:20:10.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.378+0000 7fac72126700 1 -- 172.21.15.40:0/3340699374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fac6c0726c0 con 0x7fac6c0b2130 2023-11-08T08:20:10.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.378+0000 7fac68ff9700 1 -- 172.21.15.40:0/3340699374 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fac54054ef0 con 0x7fac6c0b2130 2023-11-08T08:20:10.381 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:20:10.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.378+0000 7fac72126700 1 -- 172.21.15.40:0/3340699374 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fac4c064ee0 msgr2=0x7fac4c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:10.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.378+0000 7fac72126700 1 --2- 172.21.15.40:0/3340699374 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fac4c064ee0 0x7fac4c067390 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fac5c000f60 tx=0x7fac5c009040 comp rx=0 tx=0).stop 2023-11-08T08:20:10.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.378+0000 7fac72126700 1 -- 172.21.15.40:0/3340699374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac6c0b2130 msgr2=0x7fac6c15ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:10.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.378+0000 7fac72126700 1 --2- 172.21.15.40:0/3340699374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac6c0b2130 0x7fac6c15ec30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fac54013000 tx=0x7fac54004770 comp rx=0 tx=0).stop 2023-11-08T08:20:10.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.378+0000 7fac72126700 1 -- 172.21.15.40:0/3340699374 shutdown_connections 2023-11-08T08:20:10.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.378+0000 7fac72126700 1 --2- 172.21.15.40:0/3340699374 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fac4c064ee0 0x7fac4c067390 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:10.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.378+0000 7fac72126700 1 --2- 172.21.15.40:0/3340699374 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac6c15c700 0x7fac6c1f15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:10.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.378+0000 7fac72126700 1 --2- 172.21.15.40:0/3340699374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac6c0b2130 0x7fac6c15ec30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:10.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.378+0000 7fac72126700 1 --2- 172.21.15.40:0/3340699374 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac6c070a00 0x7fac6c15d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:10.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.378+0000 7fac72126700 1 -- 172.21.15.40:0/3340699374 >> 172.21.15.40:0/3340699374 conn(0x7fac6c0d2d10 msgr2=0x7fac6c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:10.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.382+0000 7fac72126700 1 -- 172.21.15.40:0/3340699374 shutdown_connections 2023-11-08T08:20:10.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.382+0000 7fac72126700 1 -- 172.21.15.40:0/3340699374 wait complete. 2023-11-08T08:20:10.409 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T08:20:10.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.594+0000 7feb74990700 1 Processor -- start 2023-11-08T08:20:10.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.594+0000 7feb74990700 1 -- start start 2023-11-08T08:20:10.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.598+0000 7feb74990700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb7015c150 0x7feb700b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:10.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.598+0000 7feb74990700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feb7006ff10 0x7feb70070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:10.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.598+0000 7feb74990700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb7015c520 con 0x7feb7015fc30 2023-11-08T08:20:10.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.598+0000 7feb74990700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb700b4240 con 0x7feb7006ff10 2023-11-08T08:20:10.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.598+0000 7feb74990700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb700708a0 con 0x7feb7015c150 2023-11-08T08:20:10.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.598+0000 7feb6ffff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7feb7015fc30 0x7feb701600b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59412/0 (socket says 172.21.15.40:59412) 2023-11-08T08:20:10.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.598+0000 7feb6ffff700 1 -- 172.21.15.40:0/3323667426 learned_addr learned my addr 172.21.15.40:0/3323667426 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:10.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.598+0000 7feb6f7fe700 1 --2- 172.21.15.40:0/3323667426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb7015c150 0x7feb700b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:10.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.598+0000 7feb6effd700 1 --2- 172.21.15.40:0/3323667426 >> v2:172.21.15.40:3300/0 conn(0x7feb7006ff10 0x7feb70070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:10.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.598+0000 7feb6e7fc700 1 -- 172.21.15.40:0/3323667426 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713004380 0 0) 0x7feb7015c520 con 0x7feb7015fc30 2023-11-08T08:20:10.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.598+0000 7feb6e7fc700 1 -- 172.21.15.40:0/3323667426 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb580035d0 con 0x7feb7015fc30 2023-11-08T08:20:10.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.598+0000 7feb6e7fc700 1 -- 172.21.15.40:0/3323667426 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3211581947 0 0) 0x7feb580035d0 con 0x7feb7015fc30 2023-11-08T08:20:10.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.598+0000 7feb6f7fe700 1 -- 172.21.15.40:0/3323667426 >> v2:172.21.15.40:3300/0 conn(0x7feb7006ff10 msgr2=0x7feb70070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:10.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.598+0000 7feb6f7fe700 1 --2- 172.21.15.40:0/3323667426 >> v2:172.21.15.40:3300/0 conn(0x7feb7006ff10 0x7feb70070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:10.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.598+0000 7feb6f7fe700 1 -- 172.21.15.40:0/3323667426 >> v1:172.21.15.40:6789/0 conn(0x7feb7015fc30 legacy=0x7feb701600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:10.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.598+0000 7feb6f7fe700 1 -- 172.21.15.40:0/3323667426 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7006ac00 con 0x7feb7015c150 2023-11-08T08:20:10.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.602+0000 7feb6f7fe700 1 --2- 172.21.15.40:0/3323667426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb7015c150 0x7feb700b3d00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feb60009850 tx=0x7feb60009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=700cc1136a7e8551 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:10.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.602+0000 7feb6e7fc700 1 -- 172.21.15.40:0/3323667426 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb680019f0 con 0x7feb7015c150 2023-11-08T08:20:10.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.602+0000 7feb6e7fc700 1 -- 172.21.15.40:0/3323667426 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb60004030 con 0x7feb7015c150 2023-11-08T08:20:10.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.602+0000 7feb6e7fc700 1 -- 172.21.15.40:0/3323667426 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb600043e0 con 0x7feb7015c150 2023-11-08T08:20:10.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.602+0000 7feb74990700 1 -- 172.21.15.40:0/3323667426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb7015c150 msgr2=0x7feb700b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:10.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.602+0000 7feb74990700 1 --2- 172.21.15.40:0/3323667426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb7015c150 0x7feb700b3d00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feb60009850 tx=0x7feb60009b60 comp rx=0 tx=0).stop 2023-11-08T08:20:10.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.602+0000 7feb74990700 1 -- 172.21.15.40:0/3323667426 shutdown_connections 2023-11-08T08:20:10.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.602+0000 7feb74990700 1 --2- 172.21.15.40:0/3323667426 >> v2:172.21.15.40:3300/0 conn(0x7feb7006ff10 0x7feb70070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:10.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.602+0000 7feb74990700 1 --2- 172.21.15.40:0/3323667426 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb7015c150 0x7feb700b3d00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:10.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.602+0000 7feb74990700 1 -- 172.21.15.40:0/3323667426 >> 172.21.15.40:0/3323667426 conn(0x7feb700d3680 msgr2=0x7feb700cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:10.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.602+0000 7feb74990700 1 -- 172.21.15.40:0/3323667426 shutdown_connections 2023-11-08T08:20:10.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.602+0000 7feb74990700 1 -- 172.21.15.40:0/3323667426 wait complete. 2023-11-08T08:20:10.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.606+0000 7feb74990700 1 Processor -- start 2023-11-08T08:20:10.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.606+0000 7feb74990700 1 -- start start 2023-11-08T08:20:10.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb74990700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb7006ff10 0x7feb700cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:10.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb74990700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb7015c150 0x7feb700ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:10.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb74990700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb7015fc30 0x7feb701f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:10.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb74990700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb700cd780 con 0x7feb7015c150 2023-11-08T08:20:10.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb74990700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb700cd4e0 con 0x7feb7015fc30 2023-11-08T08:20:10.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb74990700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb700cf850 con 0x7feb7006ff10 2023-11-08T08:20:10.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb6effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb7015c150 0x7feb700ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:10.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb6effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb7015c150 0x7feb700ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40102/0 (socket says 172.21.15.40:40102) 2023-11-08T08:20:10.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb6effd700 1 -- 172.21.15.40:0/2056831769 learned_addr learned my addr 172.21.15.40:0/2056831769 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:10.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb6f7fe700 1 --2- 172.21.15.40:0/2056831769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb7006ff10 0x7feb700cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:10.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb6ffff700 1 --2- 172.21.15.40:0/2056831769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb7015fc30 0x7feb701f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:10.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb6effd700 1 -- 172.21.15.40:0/2056831769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb7006ff10 msgr2=0x7feb700cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:10.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb6effd700 1 --2- 172.21.15.40:0/2056831769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb7006ff10 0x7feb700cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:10.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb6effd700 1 -- 172.21.15.40:0/2056831769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb7015fc30 msgr2=0x7feb701f4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:10.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb6effd700 1 --2- 172.21.15.40:0/2056831769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb7015fc30 0x7feb701f4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:10.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb6effd700 1 -- 172.21.15.40:0/2056831769 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb600094b0 con 0x7feb7015c150 2023-11-08T08:20:10.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb6effd700 1 --2- 172.21.15.40:0/2056831769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb7015c150 0x7feb700ccfa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7feb6400e9b0 tx=0x7feb6400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:10.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb57fff700 1 -- 172.21.15.40:0/2056831769 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb64009710 con 0x7feb7015c150 2023-11-08T08:20:10.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb57fff700 1 -- 172.21.15.40:0/2056831769 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6400c850 con 0x7feb7015c150 2023-11-08T08:20:10.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb74990700 1 -- 172.21.15.40:0/2056831769 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb700d2450 con 0x7feb7015c150 2023-11-08T08:20:10.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.610+0000 7feb57fff700 1 -- 172.21.15.40:0/2056831769 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6400cbc0 con 0x7feb7015c150 2023-11-08T08:20:10.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.614+0000 7feb74990700 1 -- 172.21.15.40:0/2056831769 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7006cc50 con 0x7feb7015c150 2023-11-08T08:20:10.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.614+0000 7feb74990700 1 -- 172.21.15.40:0/2056831769 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb700d2730 con 0x7feb7015c150 2023-11-08T08:20:10.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.614+0000 7feb57fff700 1 -- 172.21.15.40:0/2056831769 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7feb64019030 con 0x7feb7015c150 2023-11-08T08:20:10.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.618+0000 7feb57fff700 1 --2- 172.21.15.40:0/2056831769 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feb5c064ee0 0x7feb5c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:10.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.618+0000 7feb57fff700 1 -- 172.21.15.40:0/2056831769 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7feb64082290 con 0x7feb7015c150 2023-11-08T08:20:10.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.618+0000 7feb6f7fe700 1 --2- 172.21.15.40:0/2056831769 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feb5c064ee0 0x7feb5c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:10.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.618+0000 7feb6f7fe700 1 --2- 172.21.15.40:0/2056831769 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feb5c064ee0 0x7feb5c067390 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7feb600090b0 tx=0x7feb60008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:10.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.622+0000 7feb57fff700 1 -- 172.21.15.40:0/2056831769 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7feb6404ff90 con 0x7feb7015c150 2023-11-08T08:20:10.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.798+0000 7feb74990700 1 -- 172.21.15.40:0/2056831769 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feb70073d20 con 0x7feb7015c150 2023-11-08T08:20:10.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.798+0000 7feb57fff700 1 -- 172.21.15.40:0/2056831769 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7feb640535b0 con 0x7feb7015c150 2023-11-08T08:20:10.804 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:20:10.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.806+0000 7feb74990700 1 -- 172.21.15.40:0/2056831769 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feb5c064ee0 msgr2=0x7feb5c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:10.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.806+0000 7feb74990700 1 --2- 172.21.15.40:0/2056831769 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feb5c064ee0 0x7feb5c067390 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7feb600090b0 tx=0x7feb60008030 comp rx=0 tx=0).stop 2023-11-08T08:20:10.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.806+0000 7feb74990700 1 -- 172.21.15.40:0/2056831769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb7015c150 msgr2=0x7feb700ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:10.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.806+0000 7feb74990700 1 --2- 172.21.15.40:0/2056831769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb7015c150 0x7feb700ccfa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7feb6400e9b0 tx=0x7feb6400ecc0 comp rx=0 tx=0).stop 2023-11-08T08:20:10.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.806+0000 7feb74990700 1 -- 172.21.15.40:0/2056831769 shutdown_connections 2023-11-08T08:20:10.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.806+0000 7feb74990700 1 --2- 172.21.15.40:0/2056831769 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feb5c064ee0 0x7feb5c067390 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:10.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.806+0000 7feb74990700 1 --2- 172.21.15.40:0/2056831769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb7015fc30 0x7feb701f4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:10.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.806+0000 7feb74990700 1 --2- 172.21.15.40:0/2056831769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb7015c150 0x7feb700ccfa0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:10.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.806+0000 7feb74990700 1 --2- 172.21.15.40:0/2056831769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feb7006ff10 0x7feb700cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:10.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.806+0000 7feb74990700 1 -- 172.21.15.40:0/2056831769 >> 172.21.15.40:0/2056831769 conn(0x7feb700d3680 msgr2=0x7feb700c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:10.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.806+0000 7feb74990700 1 -- 172.21.15.40:0/2056831769 shutdown_connections 2023-11-08T08:20:10.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:10.806+0000 7feb74990700 1 -- 172.21.15.40:0/2056831769 wait complete. 2023-11-08T08:20:10.830 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T08:20:11.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.014+0000 7fb6bd107700 1 Processor -- start 2023-11-08T08:20:11.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.014+0000 7fb6bd107700 1 -- start start 2023-11-08T08:20:11.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.018+0000 7fb6bd107700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb6b815c150 0x7fb6b80b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:11.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.018+0000 7fb6bd107700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806ff10 0x7fb6b8070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:11.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.018+0000 7fb6bd107700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b815c520 con 0x7fb6b815fc30 2023-11-08T08:20:11.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.018+0000 7fb6bd107700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b80b4240 con 0x7fb6b815c150 2023-11-08T08:20:11.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.018+0000 7fb6bd107700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80708a0 con 0x7fb6b806ff10 2023-11-08T08:20:11.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.018+0000 7fb6b7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb6b815c150 0x7fb6b80b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:11.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.018+0000 7fb6b77fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806ff10 0x7fb6b8070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:11.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.018+0000 7fb6b7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb6b815c150 0x7fb6b80b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40118/0 (socket says 172.21.15.40:40118) 2023-11-08T08:20:11.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.018+0000 7fb6b7fff700 1 -- 172.21.15.40:0/1060657114 learned_addr learned my addr 172.21.15.40:0/1060657114 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:11.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.018+0000 7fb6b6ffd700 1 -- 172.21.15.40:0/1060657114 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3092927444 0 0) 0x7fb6b815c520 con 0x7fb6b815fc30 2023-11-08T08:20:11.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.018+0000 7fb6b7fff700 1 -- 172.21.15.40:0/1060657114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806ff10 msgr2=0x7fb6b8070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:11.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.018+0000 7fb6b7fff700 1 --2- 172.21.15.40:0/1060657114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806ff10 0x7fb6b8070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.018+0000 7fb6b7fff700 1 -- 172.21.15.40:0/1060657114 >> v1:172.21.15.169:6789/0 conn(0x7fb6b815fc30 legacy=0x7fb6b81600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:11.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.018+0000 7fb6b7fff700 1 -- 172.21.15.40:0/1060657114 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b806ac00 con 0x7fb6b815c150 2023-11-08T08:20:11.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.018+0000 7fb6b7fff700 1 --2- 172.21.15.40:0/1060657114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b815c150 0x7fb6b80b3d00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009910 tx=0x7fb6a8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b58e472249960c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:11.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.018+0000 7fb6b6ffd700 1 -- 172.21.15.40:0/1060657114 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8011830 con 0x7fb6b815c150 2023-11-08T08:20:11.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.018+0000 7fb6b6ffd700 1 -- 172.21.15.40:0/1060657114 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a8011990 con 0x7fb6b815c150 2023-11-08T08:20:11.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.018+0000 7fb6b6ffd700 1 -- 172.21.15.40:0/1060657114 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8011d40 con 0x7fb6b815c150 2023-11-08T08:20:11.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.022+0000 7fb6bd107700 1 -- 172.21.15.40:0/1060657114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b815c150 msgr2=0x7fb6b80b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:11.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.022+0000 7fb6bd107700 1 --2- 172.21.15.40:0/1060657114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b815c150 0x7fb6b80b3d00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009910 tx=0x7fb6a8009c20 comp rx=0 tx=0).stop 2023-11-08T08:20:11.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.022+0000 7fb6bd107700 1 -- 172.21.15.40:0/1060657114 shutdown_connections 2023-11-08T08:20:11.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.022+0000 7fb6bd107700 1 --2- 172.21.15.40:0/1060657114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806ff10 0x7fb6b8070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.022+0000 7fb6bd107700 1 --2- 172.21.15.40:0/1060657114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b815c150 0x7fb6b80b3d00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.022+0000 7fb6bd107700 1 -- 172.21.15.40:0/1060657114 >> 172.21.15.40:0/1060657114 conn(0x7fb6b80d3680 msgr2=0x7fb6b80cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:11.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.022+0000 7fb6bd107700 1 -- 172.21.15.40:0/1060657114 shutdown_connections 2023-11-08T08:20:11.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.022+0000 7fb6bd107700 1 -- 172.21.15.40:0/1060657114 wait complete. 2023-11-08T08:20:11.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.026+0000 7fb6bd107700 1 Processor -- start 2023-11-08T08:20:11.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.026+0000 7fb6bd107700 1 -- start start 2023-11-08T08:20:11.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6bd107700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806ff10 0x7fb6b80ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:11.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6bd107700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6b815c150 0x7fb6b81566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:11.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6bd107700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b815fc30 0x7fb6b80ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:11.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6bd107700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b815b920 con 0x7fb6b815fc30 2023-11-08T08:20:11.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6bd107700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80cd4e0 con 0x7fb6b815c150 2023-11-08T08:20:11.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6bd107700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b8156c30 con 0x7fb6b806ff10 2023-11-08T08:20:11.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6bc906700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b815fc30 0x7fb6b80ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:11.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6b7fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806ff10 0x7fb6b80ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:11.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6b77fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6b815c150 0x7fb6b81566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:11.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6b7fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806ff10 0x7fb6b80ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:54874/0 (socket says 172.21.15.40:54874) 2023-11-08T08:20:11.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6b7fff700 1 -- 172.21.15.40:0/3927301261 learned_addr learned my addr 172.21.15.40:0/3927301261 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:11.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6b7fff700 1 -- 172.21.15.40:0/3927301261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6b815c150 msgr2=0x7fb6b81566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:11.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6b7fff700 1 --2- 172.21.15.40:0/3927301261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6b815c150 0x7fb6b81566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6b7fff700 1 -- 172.21.15.40:0/3927301261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b815fc30 msgr2=0x7fb6b80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:11.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6b7fff700 1 --2- 172.21.15.40:0/3927301261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b815fc30 0x7fb6b80ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6b7fff700 1 -- 172.21.15.40:0/3927301261 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a8009570 con 0x7fb6b806ff10 2023-11-08T08:20:11.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6b77fe700 1 --2- 172.21.15.40:0/3927301261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6b815c150 0x7fb6b81566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:11.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6bc906700 1 --2- 172.21.15.40:0/3927301261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b815fc30 0x7fb6b80ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:11.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6b7fff700 1 --2- 172.21.15.40:0/3927301261 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806ff10 0x7fb6b80ccfa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009f70 tx=0x7fb6a8004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:11.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6b4ff9700 1 -- 172.21.15.40:0/3927301261 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a801a900 con 0x7fb6b806ff10 2023-11-08T08:20:11.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6b4ff9700 1 -- 172.21.15.40:0/3927301261 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a801f070 con 0x7fb6b806ff10 2023-11-08T08:20:11.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6bd107700 1 -- 172.21.15.40:0/3927301261 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b80d2450 con 0x7fb6b806ff10 2023-11-08T08:20:11.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6b4ff9700 1 -- 172.21.15.40:0/3927301261 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a801acc0 con 0x7fb6b806ff10 2023-11-08T08:20:11.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.030+0000 7fb6bd107700 1 -- 172.21.15.40:0/3927301261 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b806cc50 con 0x7fb6b806ff10 2023-11-08T08:20:11.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.034+0000 7fb6bd107700 1 -- 172.21.15.40:0/3927301261 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b80d2730 con 0x7fb6b806ff10 2023-11-08T08:20:11.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.034+0000 7fb6b4ff9700 1 -- 172.21.15.40:0/3927301261 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fb6a800f050 con 0x7fb6b806ff10 2023-11-08T08:20:11.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.034+0000 7fb6b4ff9700 1 --2- 172.21.15.40:0/3927301261 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb6a4064f10 0x7fb6a40673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:11.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.038+0000 7fb6b4ff9700 1 -- 172.21.15.40:0/3927301261 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb6a8021070 con 0x7fb6b806ff10 2023-11-08T08:20:11.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.038+0000 7fb6b77fe700 1 --2- 172.21.15.40:0/3927301261 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb6a4064f10 0x7fb6a40673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:11.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.038+0000 7fb6b77fe700 1 --2- 172.21.15.40:0/3927301261 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb6a4064f10 0x7fb6a40673c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80ccf20 tx=0x7fb6ac0091c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:11.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.042+0000 7fb6b4ff9700 1 -- 172.21.15.40:0/3927301261 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fb6a8051a70 con 0x7fb6b806ff10 2023-11-08T08:20:11.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.218+0000 7fb6bd107700 1 -- 172.21.15.40:0/3927301261 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb6b8073d20 con 0x7fb6b806ff10 2023-11-08T08:20:11.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.218+0000 7fb6b4ff9700 1 -- 172.21.15.40:0/3927301261 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb6a80150a0 con 0x7fb6b806ff10 2023-11-08T08:20:11.223 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-11-08T08:20:11.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.222+0000 7fb6bd107700 1 -- 172.21.15.40:0/3927301261 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb6a4064f10 msgr2=0x7fb6a40673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:11.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.222+0000 7fb6bd107700 1 --2- 172.21.15.40:0/3927301261 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb6a4064f10 0x7fb6a40673c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80ccf20 tx=0x7fb6ac0091c0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.222+0000 7fb6bd107700 1 -- 172.21.15.40:0/3927301261 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806ff10 msgr2=0x7fb6b80ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:11.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.222+0000 7fb6bd107700 1 --2- 172.21.15.40:0/3927301261 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806ff10 0x7fb6b80ccfa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009f70 tx=0x7fb6a8004770 comp rx=0 tx=0).stop 2023-11-08T08:20:11.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.222+0000 7fb6bd107700 1 -- 172.21.15.40:0/3927301261 shutdown_connections 2023-11-08T08:20:11.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.222+0000 7fb6bd107700 1 --2- 172.21.15.40:0/3927301261 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb6a4064f10 0x7fb6a40673c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.222+0000 7fb6bd107700 1 --2- 172.21.15.40:0/3927301261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b815fc30 0x7fb6b80ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.222+0000 7fb6bd107700 1 --2- 172.21.15.40:0/3927301261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6b815c150 0x7fb6b81566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.222+0000 7fb6bd107700 1 --2- 172.21.15.40:0/3927301261 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6b806ff10 0x7fb6b80ccfa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.222+0000 7fb6bd107700 1 -- 172.21.15.40:0/3927301261 >> 172.21.15.40:0/3927301261 conn(0x7fb6b80d3680 msgr2=0x7fb6b80c8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:11.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.222+0000 7fb6bd107700 1 -- 172.21.15.40:0/3927301261 shutdown_connections 2023-11-08T08:20:11.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.222+0000 7fb6bd107700 1 -- 172.21.15.40:0/3927301261 wait complete. 2023-11-08T08:20:11.248 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T08:20:11.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.434+0000 7f2be9466700 1 Processor -- start 2023-11-08T08:20:11.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.438+0000 7f2be9466700 1 -- start start 2023-11-08T08:20:11.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.438+0000 7f2be9466700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2be4160b30 0x7f2be40b1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:11.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.438+0000 7f2be9466700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2be40caba0 0x7f2be40cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:11.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.438+0000 7f2be9466700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be4160f00 con 0x7f2be40b1780 2023-11-08T08:20:11.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.438+0000 7f2be9466700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be40b1c00 con 0x7f2be4160b30 2023-11-08T08:20:11.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.438+0000 7f2be9466700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be41646d0 con 0x7f2be40caba0 2023-11-08T08:20:11.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2be3fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2be4160b30 0x7f2be40b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:11.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2bdb7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2be40b1780 0x7f2be40ca490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59416/0 (socket says 172.21.15.40:59416) 2023-11-08T08:20:11.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2bdb7fe700 1 -- 172.21.15.40:0/932730987 learned_addr learned my addr 172.21.15.40:0/932730987 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:11.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2be8c65700 1 --2- 172.21.15.40:0/932730987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2be40caba0 0x7f2be40cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:11.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2be3fff700 1 -- 172.21.15.40:0/932730987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2be40caba0 msgr2=0x7f2be40cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:11.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2be3fff700 1 --2- 172.21.15.40:0/932730987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2be40caba0 0x7f2be40cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2be3fff700 1 -- 172.21.15.40:0/932730987 >> v1:172.21.15.40:6789/0 conn(0x7f2be40b1780 legacy=0x7f2be40ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:11.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2be3fff700 1 -- 172.21.15.40:0/932730987 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be41649b0 con 0x7f2be4160b30 2023-11-08T08:20:11.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2be8c65700 1 --2- 172.21.15.40:0/932730987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2be40caba0 0x7f2be40cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:20:11.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2be37fe700 1 -- 172.21.15.40:0/932730987 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1384810750 0 0) 0x7f2bcc001110 con 0x7f2be40b1780 2023-11-08T08:20:11.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2be3fff700 1 --2- 172.21.15.40:0/932730987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2be4160b30 0x7f2be40b1240 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009850 tx=0x7f2bd4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf62b1f0d3ab8e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:11.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2be37fe700 1 -- 172.21.15.40:0/932730987 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4011830 con 0x7f2be4160b30 2023-11-08T08:20:11.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2be37fe700 1 -- 172.21.15.40:0/932730987 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd4011990 con 0x7f2be4160b30 2023-11-08T08:20:11.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2be37fe700 1 -- 172.21.15.40:0/932730987 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4011d20 con 0x7f2be4160b30 2023-11-08T08:20:11.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2be9466700 1 -- 172.21.15.40:0/932730987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2be4160b30 msgr2=0x7f2be40b1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:11.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2be9466700 1 --2- 172.21.15.40:0/932730987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2be4160b30 0x7f2be40b1240 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009850 tx=0x7f2bd4009b60 comp rx=0 tx=0).stop 2023-11-08T08:20:11.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2be9466700 1 -- 172.21.15.40:0/932730987 shutdown_connections 2023-11-08T08:20:11.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2be9466700 1 --2- 172.21.15.40:0/932730987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2be40caba0 0x7f2be40cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2be9466700 1 --2- 172.21.15.40:0/932730987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2be4160b30 0x7f2be40b1240 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2be9466700 1 -- 172.21.15.40:0/932730987 >> 172.21.15.40:0/932730987 conn(0x7f2be40cc240 msgr2=0x7f2be40cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:11.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2be9466700 1 -- 172.21.15.40:0/932730987 shutdown_connections 2023-11-08T08:20:11.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.442+0000 7f2be9466700 1 -- 172.21.15.40:0/932730987 wait complete. 2023-11-08T08:20:11.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.446+0000 7f2be9466700 1 Processor -- start 2023-11-08T08:20:11.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.446+0000 7f2be9466700 1 -- start start 2023-11-08T08:20:11.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be9466700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2be40b1780 0x7f2be41fa010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:11.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be9466700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2be40caba0 0x7f2be40c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:11.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be9466700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2be4160b30 0x7f2be41529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:11.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be9466700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be41f9b80 con 0x7f2be4160b30 2023-11-08T08:20:11.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be9466700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be40c62a0 con 0x7f2be40caba0 2023-11-08T08:20:11.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be9466700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be41f9530 con 0x7f2be40b1780 2023-11-08T08:20:11.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be8c65700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2be4160b30 0x7f2be41529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:11.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be8c65700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2be4160b30 0x7f2be41529f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40148/0 (socket says 172.21.15.40:40148) 2023-11-08T08:20:11.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be8c65700 1 -- 172.21.15.40:0/3477520168 learned_addr learned my addr 172.21.15.40:0/3477520168 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:11.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be3fff700 1 --2- 172.21.15.40:0/3477520168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2be40b1780 0x7f2be41fa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:11.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be8c65700 1 -- 172.21.15.40:0/3477520168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2be40b1780 msgr2=0x7f2be41fa010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:11.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be8c65700 1 --2- 172.21.15.40:0/3477520168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2be40b1780 0x7f2be41fa010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be8c65700 1 -- 172.21.15.40:0/3477520168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2be40caba0 msgr2=0x7f2be40c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:11.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2bdb7fe700 1 --2- 172.21.15.40:0/3477520168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2be40caba0 0x7f2be40c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:11.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be8c65700 1 --2- 172.21.15.40:0/3477520168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2be40caba0 0x7f2be40c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be8c65700 1 -- 172.21.15.40:0/3477520168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd40094b0 con 0x7f2be4160b30 2023-11-08T08:20:11.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2bdb7fe700 1 --2- 172.21.15.40:0/3477520168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2be40caba0 0x7f2be40c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:20:11.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be8c65700 1 --2- 172.21.15.40:0/3477520168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2be4160b30 0x7f2be41529f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00cd80 tx=0x7f2bdc00a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:11.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be17fa700 1 -- 172.21.15.40:0/3477520168 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc003a40 con 0x7f2be4160b30 2023-11-08T08:20:11.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be17fa700 1 -- 172.21.15.40:0/3477520168 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc003bc0 con 0x7f2be4160b30 2023-11-08T08:20:11.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be17fa700 1 -- 172.21.15.40:0/3477520168 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc004020 con 0x7f2be4160b30 2023-11-08T08:20:11.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be9466700 1 -- 172.21.15.40:0/3477520168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be4164f70 con 0x7f2be4160b30 2023-11-08T08:20:11.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.450+0000 7f2be9466700 1 -- 172.21.15.40:0/3477520168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be4164c90 con 0x7f2be4160b30 2023-11-08T08:20:11.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.454+0000 7f2be9466700 1 -- 172.21.15.40:0/3477520168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be4165250 con 0x7f2be4160b30 2023-11-08T08:20:11.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.454+0000 7f2be17fa700 1 -- 172.21.15.40:0/3477520168 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f2bdc01f070 con 0x7f2be4160b30 2023-11-08T08:20:11.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.458+0000 7f2be17fa700 1 --2- 172.21.15.40:0/3477520168 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2bc4064f10 0x7f2bc40673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:11.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.458+0000 7f2be17fa700 1 -- 172.21.15.40:0/3477520168 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2bdc082810 con 0x7f2be4160b30 2023-11-08T08:20:11.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.458+0000 7f2be3fff700 1 --2- 172.21.15.40:0/3477520168 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2bc4064f10 0x7f2bc40673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:11.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.458+0000 7f2be3fff700 1 --2- 172.21.15.40:0/3477520168 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2bc4064f10 0x7f2bc40673c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009fd0 tx=0x7f2bd4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:11.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.462+0000 7f2be17fa700 1 -- 172.21.15.40:0/3477520168 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f2bdc050590 con 0x7f2be4160b30 2023-11-08T08:20:11.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.638+0000 7f2be9466700 1 -- 172.21.15.40:0/3477520168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2be4073d20 con 0x7f2be4160b30 2023-11-08T08:20:11.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.638+0000 7f2be17fa700 1 -- 172.21.15.40:0/3477520168 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2bdc01d0a0 con 0x7f2be4160b30 2023-11-08T08:20:11.643 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:20:11.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.642+0000 7f2be9466700 1 -- 172.21.15.40:0/3477520168 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2bc4064f10 msgr2=0x7f2bc40673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:11.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.642+0000 7f2be9466700 1 --2- 172.21.15.40:0/3477520168 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2bc4064f10 0x7f2bc40673c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009fd0 tx=0x7f2bd4008030 comp rx=0 tx=0).stop 2023-11-08T08:20:11.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.642+0000 7f2be9466700 1 -- 172.21.15.40:0/3477520168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2be4160b30 msgr2=0x7f2be41529f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:11.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.642+0000 7f2be9466700 1 --2- 172.21.15.40:0/3477520168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2be4160b30 0x7f2be41529f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00cd80 tx=0x7f2bdc00a560 comp rx=0 tx=0).stop 2023-11-08T08:20:11.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.642+0000 7f2be9466700 1 -- 172.21.15.40:0/3477520168 shutdown_connections 2023-11-08T08:20:11.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.642+0000 7f2be9466700 1 --2- 172.21.15.40:0/3477520168 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2bc4064f10 0x7f2bc40673c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.642+0000 7f2be9466700 1 --2- 172.21.15.40:0/3477520168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2be4160b30 0x7f2be41529f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.642+0000 7f2be9466700 1 --2- 172.21.15.40:0/3477520168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2be40caba0 0x7f2be40c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.642+0000 7f2be9466700 1 --2- 172.21.15.40:0/3477520168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2be40b1780 0x7f2be41fa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.642+0000 7f2be9466700 1 -- 172.21.15.40:0/3477520168 >> 172.21.15.40:0/3477520168 conn(0x7f2be40cc240 msgr2=0x7f2be4069c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:11.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.642+0000 7f2be9466700 1 -- 172.21.15.40:0/3477520168 shutdown_connections 2023-11-08T08:20:11.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.642+0000 7f2be9466700 1 -- 172.21.15.40:0/3477520168 wait complete. 2023-11-08T08:20:11.669 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T08:20:11.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.858+0000 7f4c51ebc700 1 Processor -- start 2023-11-08T08:20:11.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.858+0000 7f4c51ebc700 1 -- start start 2023-11-08T08:20:11.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.862+0000 7f4c51ebc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c4c15a210 0x7f4c4c06ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:11.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.862+0000 7f4c51ebc700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4c4c06c0e0 0x7f4c4c0b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:11.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.862+0000 7f4c51ebc700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4c15d3e0 con 0x7f4c4c06bd10 2023-11-08T08:20:11.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.862+0000 7f4c51ebc700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c4c15d1f0 con 0x7f4c4c06c0e0 2023-11-08T08:20:11.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.862+0000 7f4c51ebc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c15d6c0 con 0x7f4c4c15a210 2023-11-08T08:20:11.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.862+0000 7f4c4b7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4c4c06bd10 0x7f4c4c15cae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59432/0 (socket says 172.21.15.40:59432) 2023-11-08T08:20:11.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.862+0000 7f4c4b7fe700 1 -- 172.21.15.40:0/427021041 learned_addr learned my addr 172.21.15.40:0/427021041 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:11.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.862+0000 7f4c4affd700 1 --2- 172.21.15.40:0/427021041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c4c15a210 0x7f4c4c06ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:11.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.862+0000 7f4c4bfff700 1 --2- 172.21.15.40:0/427021041 >> v2:172.21.15.169:3300/0 conn(0x7f4c4c06c0e0 0x7f4c4c0b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:11.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.862+0000 7f4c4a7fc700 1 -- 172.21.15.40:0/427021041 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479958320 0 0) 0x7f4c4c15d3e0 con 0x7f4c4c06bd10 2023-11-08T08:20:11.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.862+0000 7f4c4a7fc700 1 -- 172.21.15.40:0/427021041 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c340035d0 con 0x7f4c4c06bd10 2023-11-08T08:20:11.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.862+0000 7f4c4affd700 1 -- 172.21.15.40:0/427021041 >> v2:172.21.15.169:3300/0 conn(0x7f4c4c06c0e0 msgr2=0x7f4c4c0b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:11.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.862+0000 7f4c4affd700 1 --2- 172.21.15.40:0/427021041 >> v2:172.21.15.169:3300/0 conn(0x7f4c4c06c0e0 0x7f4c4c0b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.862+0000 7f4c4affd700 1 -- 172.21.15.40:0/427021041 >> v1:172.21.15.40:6789/0 conn(0x7f4c4c06bd10 legacy=0x7f4c4c15cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:11.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.862+0000 7f4c4a7fc700 1 -- 172.21.15.40:0/427021041 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3910526935 0 0) 0x7f4c340035d0 con 0x7f4c4c06bd10 2023-11-08T08:20:11.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.862+0000 7f4c4affd700 1 -- 172.21.15.40:0/427021041 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c15d9a0 con 0x7f4c4c15a210 2023-11-08T08:20:11.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.862+0000 7f4c4affd700 1 --2- 172.21.15.40:0/427021041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c4c15a210 0x7f4c4c06ad00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009850 tx=0x7f4c40009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=627c25307eec3ec9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.862+0000 7f4c4a7fc700 1 -- 172.21.15.40:0/427021041 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c001670 con 0x7f4c4c15a210 2023-11-08T08:20:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.862+0000 7f4c4a7fc700 1 -- 172.21.15.40:0/427021041 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c40004030 con 0x7f4c4c15a210 2023-11-08T08:20:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.862+0000 7f4c4a7fc700 1 -- 172.21.15.40:0/427021041 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c400043e0 con 0x7f4c4c15a210 2023-11-08T08:20:11.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.866+0000 7f4c51ebc700 1 -- 172.21.15.40:0/427021041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c4c15a210 msgr2=0x7f4c4c06ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:11.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.866+0000 7f4c51ebc700 1 --2- 172.21.15.40:0/427021041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c4c15a210 0x7f4c4c06ad00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009850 tx=0x7f4c40009b60 comp rx=0 tx=0).stop 2023-11-08T08:20:11.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.866+0000 7f4c51ebc700 1 -- 172.21.15.40:0/427021041 shutdown_connections 2023-11-08T08:20:11.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.866+0000 7f4c51ebc700 1 --2- 172.21.15.40:0/427021041 >> v2:172.21.15.169:3300/0 conn(0x7f4c4c06c0e0 0x7f4c4c0b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.866+0000 7f4c51ebc700 1 --2- 172.21.15.40:0/427021041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c4c15a210 0x7f4c4c06ad00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.866+0000 7f4c51ebc700 1 -- 172.21.15.40:0/427021041 >> 172.21.15.40:0/427021041 conn(0x7f4c4c0cc290 msgr2=0x7f4c4c0cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:11.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.866+0000 7f4c51ebc700 1 -- 172.21.15.40:0/427021041 shutdown_connections 2023-11-08T08:20:11.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.866+0000 7f4c51ebc700 1 -- 172.21.15.40:0/427021041 wait complete. 2023-11-08T08:20:11.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.866+0000 7f4c51ebc700 1 Processor -- start 2023-11-08T08:20:11.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.866+0000 7f4c51ebc700 1 -- start start 2023-11-08T08:20:11.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c51ebc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c4c06bd10 0x7f4c4c06a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:11.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c51ebc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c4c06c0e0 0x7f4c4c0b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:11.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c51ebc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c4c15a210 0x7f4c4c0b0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:11.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c51ebc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c1f96e0 con 0x7f4c4c15a210 2023-11-08T08:20:11.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c51ebc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c1f9f80 con 0x7f4c4c06c0e0 2023-11-08T08:20:11.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c51ebc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c06ab60 con 0x7f4c4c06bd10 2023-11-08T08:20:11.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c4bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c4c15a210 0x7f4c4c0b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:11.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c4bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c4c15a210 0x7f4c4c0b0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40160/0 (socket says 172.21.15.40:40160) 2023-11-08T08:20:11.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c4bfff700 1 -- 172.21.15.40:0/4019439401 learned_addr learned my addr 172.21.15.40:0/4019439401 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:11.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c4b7fe700 1 --2- 172.21.15.40:0/4019439401 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c4c06bd10 0x7f4c4c06a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:11.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c4affd700 1 --2- 172.21.15.40:0/4019439401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c4c06c0e0 0x7f4c4c0b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:11.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c4bfff700 1 -- 172.21.15.40:0/4019439401 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c4c06bd10 msgr2=0x7f4c4c06a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:11.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c4bfff700 1 --2- 172.21.15.40:0/4019439401 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c4c06bd10 0x7f4c4c06a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c4bfff700 1 -- 172.21.15.40:0/4019439401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c4c06c0e0 msgr2=0x7f4c4c0b0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:11.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c4bfff700 1 --2- 172.21.15.40:0/4019439401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c4c06c0e0 0x7f4c4c0b0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:11.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c4bfff700 1 -- 172.21.15.40:0/4019439401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c400094b0 con 0x7f4c4c15a210 2023-11-08T08:20:11.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c4affd700 1 --2- 172.21.15.40:0/4019439401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c4c06c0e0 0x7f4c4c0b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:11.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c4bfff700 1 --2- 172.21.15.40:0/4019439401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c4c15a210 0x7f4c4c0b0850 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400efc0 tx=0x7f4c4400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:11.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c33fff700 1 -- 172.21.15.40:0/4019439401 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4400cd40 con 0x7f4c4c15a210 2023-11-08T08:20:11.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c33fff700 1 -- 172.21.15.40:0/4019439401 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4400f040 con 0x7f4c4c15a210 2023-11-08T08:20:11.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c33fff700 1 -- 172.21.15.40:0/4019439401 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c440146c0 con 0x7f4c4c15a210 2023-11-08T08:20:11.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c51ebc700 1 -- 172.21.15.40:0/4019439401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4c0cad60 con 0x7f4c4c15a210 2023-11-08T08:20:11.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.870+0000 7f4c51ebc700 1 -- 172.21.15.40:0/4019439401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4c0caa80 con 0x7f4c4c15a210 2023-11-08T08:20:11.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.874+0000 7f4c51ebc700 1 -- 172.21.15.40:0/4019439401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c4c06b240 con 0x7f4c4c15a210 2023-11-08T08:20:11.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.878+0000 7f4c33fff700 1 -- 172.21.15.40:0/4019439401 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4c44014820 con 0x7f4c4c15a210 2023-11-08T08:20:11.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.878+0000 7f4c33fff700 1 --2- 172.21.15.40:0/4019439401 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4c38064ee0 0x7f4c38067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:11.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.878+0000 7f4c33fff700 1 -- 172.21.15.40:0/4019439401 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4c44083550 con 0x7f4c4c15a210 2023-11-08T08:20:11.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.878+0000 7f4c4b7fe700 1 --2- 172.21.15.40:0/4019439401 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4c38064ee0 0x7f4c38067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:11.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.882+0000 7f4c4b7fe700 1 --2- 172.21.15.40:0/4019439401 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4c38064ee0 0x7f4c38067390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0011e0 tx=0x7f4c3c001550 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:11.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:11.882+0000 7f4c33fff700 1 -- 172.21.15.40:0/4019439401 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4c44051350 con 0x7f4c4c15a210 2023-11-08T08:20:12.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.070+0000 7f4c51ebc700 1 -- 172.21.15.40:0/4019439401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4c4c1fa0c0 con 0x7f4c4c15a210 2023-11-08T08:20:12.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.070+0000 7f4c33fff700 1 -- 172.21.15.40:0/4019439401 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4c4c1fa0c0 con 0x7f4c4c15a210 2023-11-08T08:20:12.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.074+0000 7f4c51ebc700 1 -- 172.21.15.40:0/4019439401 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4c38064ee0 msgr2=0x7f4c38067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:12.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.074+0000 7f4c51ebc700 1 --2- 172.21.15.40:0/4019439401 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4c38064ee0 0x7f4c38067390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0011e0 tx=0x7f4c3c001550 comp rx=0 tx=0).stop 2023-11-08T08:20:12.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.074+0000 7f4c51ebc700 1 -- 172.21.15.40:0/4019439401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c4c15a210 msgr2=0x7f4c4c0b0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:12.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.074+0000 7f4c51ebc700 1 --2- 172.21.15.40:0/4019439401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c4c15a210 0x7f4c4c0b0850 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400efc0 tx=0x7f4c4400c3a0 comp rx=0 tx=0).stop 2023-11-08T08:20:12.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.074+0000 7f4c51ebc700 1 -- 172.21.15.40:0/4019439401 shutdown_connections 2023-11-08T08:20:12.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.074+0000 7f4c51ebc700 1 --2- 172.21.15.40:0/4019439401 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4c38064ee0 0x7f4c38067390 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:12.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.074+0000 7f4c51ebc700 1 --2- 172.21.15.40:0/4019439401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c4c15a210 0x7f4c4c0b0850 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:12.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.074+0000 7f4c51ebc700 1 --2- 172.21.15.40:0/4019439401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c4c06c0e0 0x7f4c4c0b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:12.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.074+0000 7f4c51ebc700 1 --2- 172.21.15.40:0/4019439401 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c4c06bd10 0x7f4c4c06a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:12.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.074+0000 7f4c51ebc700 1 -- 172.21.15.40:0/4019439401 >> 172.21.15.40:0/4019439401 conn(0x7f4c4c0cc290 msgr2=0x7f4c4c155fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:12.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.074+0000 7f4c51ebc700 1 -- 172.21.15.40:0/4019439401 shutdown_connections 2023-11-08T08:20:12.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.074+0000 7f4c51ebc700 1 -- 172.21.15.40:0/4019439401 wait complete. 2023-11-08T08:20:12.081 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-11-08T08:20:12.099 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-08T08:20:12.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:12.100 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-08T08:20:12.142 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-11-08T08:20:12.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:12.143 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-08T08:20:12.205 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:20:12.205 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:20:12.225 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:20:12.226 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:20:12.285 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:20:12.285 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:20:12.350 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:20:12.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.542+0000 7fc3a8833700 1 Processor -- start 2023-11-08T08:20:12.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.542+0000 7fc3a8833700 1 -- start start 2023-11-08T08:20:12.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.542+0000 7fc3a8833700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc3a415c640 0x7fc3a40d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:12.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.542+0000 7fc3a8833700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3a4071aa0 0x7fc3a406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:12.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.542+0000 7fc3a8833700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a406c600 con 0x7fc3a4069e00 2023-11-08T08:20:12.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.542+0000 7fc3a8833700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a406cef0 con 0x7fc3a415c640 2023-11-08T08:20:12.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.542+0000 7fc3a8833700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40722f0 con 0x7fc3a4071aa0 2023-11-08T08:20:12.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.542+0000 7fc3a2d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc3a415c640 0x7fc3a40d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:12.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.542+0000 7fc3a2d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc3a415c640 0x7fc3a40d31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40162/0 (socket says 172.21.15.40:40162) 2023-11-08T08:20:12.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.542+0000 7fc3a2d9d700 1 -- 172.21.15.40:0/3105405044 learned_addr learned my addr 172.21.15.40:0/3105405044 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:12.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.542+0000 7fc3a259c700 1 --2- 172.21.15.40:0/3105405044 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3a4071aa0 0x7fc3a406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:12.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.546+0000 7fc3a2d9d700 1 -- 172.21.15.40:0/3105405044 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3a4071aa0 msgr2=0x7fc3a406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:12.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.546+0000 7fc3a2d9d700 1 --2- 172.21.15.40:0/3105405044 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3a4071aa0 0x7fc3a406d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:12.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.546+0000 7fc3a2d9d700 1 -- 172.21.15.40:0/3105405044 >> v1:172.21.15.169:6789/0 conn(0x7fc3a4069e00 legacy=0x7fc3a40d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:12.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.546+0000 7fc3a2d9d700 1 -- 172.21.15.40:0/3105405044 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a415ca10 con 0x7fc3a415c640 2023-11-08T08:20:12.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.546+0000 7fc3a2d9d700 1 --2- 172.21.15.40:0/3105405044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a415c640 0x7fc3a40d31e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc394009910 tx=0x7fc394009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef247663c9232c58 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:12.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.546+0000 7fc3a1d9b700 1 -- 172.21.15.40:0/3105405044 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394011830 con 0x7fc3a415c640 2023-11-08T08:20:12.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.546+0000 7fc3a1d9b700 1 -- 172.21.15.40:0/3105405044 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc394011990 con 0x7fc3a415c640 2023-11-08T08:20:12.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.546+0000 7fc3a1d9b700 1 -- 172.21.15.40:0/3105405044 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394011d40 con 0x7fc3a415c640 2023-11-08T08:20:12.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.546+0000 7fc3a8833700 1 -- 172.21.15.40:0/3105405044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a415c640 msgr2=0x7fc3a40d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:12.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.546+0000 7fc3a8833700 1 --2- 172.21.15.40:0/3105405044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a415c640 0x7fc3a40d31e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc394009910 tx=0x7fc394009c20 comp rx=0 tx=0).stop 2023-11-08T08:20:12.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.546+0000 7fc3a8833700 1 -- 172.21.15.40:0/3105405044 shutdown_connections 2023-11-08T08:20:12.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.546+0000 7fc3a8833700 1 --2- 172.21.15.40:0/3105405044 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3a4071aa0 0x7fc3a406d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:12.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.546+0000 7fc3a8833700 1 --2- 172.21.15.40:0/3105405044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a415c640 0x7fc3a40d31e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:12.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.546+0000 7fc3a8833700 1 -- 172.21.15.40:0/3105405044 >> 172.21.15.40:0/3105405044 conn(0x7fc3a40b6520 msgr2=0x7fc3a4071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:12.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.546+0000 7fc3a8833700 1 -- 172.21.15.40:0/3105405044 shutdown_connections 2023-11-08T08:20:12.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.546+0000 7fc3a8833700 1 -- 172.21.15.40:0/3105405044 wait complete. 2023-11-08T08:20:12.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.550+0000 7fc3a8833700 1 Processor -- start 2023-11-08T08:20:12.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.550+0000 7fc3a8833700 1 -- start start 2023-11-08T08:20:12.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.550+0000 7fc3a8833700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3a4069e00 0x7fc3a415e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:12.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.550+0000 7fc3a8833700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3a4071aa0 0x7fc3a415e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:12.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.550+0000 7fc3a8833700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a415c640 0x7fc3a41f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:12.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.550+0000 7fc3a8833700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a415dfa0 con 0x7fc3a415c640 2023-11-08T08:20:12.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.550+0000 7fc3a8833700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a415d9e0 con 0x7fc3a4071aa0 2023-11-08T08:20:12.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.550+0000 7fc3a8833700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a415dcc0 con 0x7fc3a4069e00 2023-11-08T08:20:12.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.554+0000 7fc3a359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a415c640 0x7fc3a41f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:12.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.554+0000 7fc3a2d9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3a4069e00 0x7fc3a415e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:12.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.554+0000 7fc3a359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a415c640 0x7fc3a41f5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40168/0 (socket says 172.21.15.40:40168) 2023-11-08T08:20:12.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.554+0000 7fc3a359e700 1 -- 172.21.15.40:0/2757518045 learned_addr learned my addr 172.21.15.40:0/2757518045 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:12.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.554+0000 7fc3a259c700 1 --2- 172.21.15.40:0/2757518045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3a4071aa0 0x7fc3a415e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:12.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.554+0000 7fc3a2d9d700 1 -- 172.21.15.40:0/2757518045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3a4071aa0 msgr2=0x7fc3a415e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:12.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.554+0000 7fc3a2d9d700 1 --2- 172.21.15.40:0/2757518045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3a4071aa0 0x7fc3a415e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:12.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.554+0000 7fc3a2d9d700 1 -- 172.21.15.40:0/2757518045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a415c640 msgr2=0x7fc3a41f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:12.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.554+0000 7fc3a2d9d700 1 --2- 172.21.15.40:0/2757518045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a415c640 0x7fc3a41f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:12.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.554+0000 7fc3a2d9d700 1 -- 172.21.15.40:0/2757518045 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc394009570 con 0x7fc3a4069e00 2023-11-08T08:20:12.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.554+0000 7fc3a359e700 1 --2- 172.21.15.40:0/2757518045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a415c640 0x7fc3a41f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:12.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.554+0000 7fc3a2d9d700 1 --2- 172.21.15.40:0/2757518045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3a4069e00 0x7fc3a415e1d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc394016000 tx=0x7fc394004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:12.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.554+0000 7fc38b7fe700 1 -- 172.21.15.40:0/2757518045 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394004450 con 0x7fc3a4069e00 2023-11-08T08:20:12.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.554+0000 7fc38b7fe700 1 -- 172.21.15.40:0/2757518045 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3940045d0 con 0x7fc3a4069e00 2023-11-08T08:20:12.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.554+0000 7fc38b7fe700 1 -- 172.21.15.40:0/2757518045 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394003900 con 0x7fc3a4069e00 2023-11-08T08:20:12.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.554+0000 7fc3a8833700 1 -- 172.21.15.40:0/2757518045 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a406c830 con 0x7fc3a4069e00 2023-11-08T08:20:12.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.554+0000 7fc3a8833700 1 -- 172.21.15.40:0/2757518045 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a4072010 con 0x7fc3a4069e00 2023-11-08T08:20:12.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.558+0000 7fc3a8833700 1 -- 172.21.15.40:0/2757518045 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a406cb10 con 0x7fc3a4069e00 2023-11-08T08:20:12.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.558+0000 7fc38b7fe700 1 -- 172.21.15.40:0/2757518045 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc39402d070 con 0x7fc3a4069e00 2023-11-08T08:20:12.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.558+0000 7fc38b7fe700 1 --2- 172.21.15.40:0/2757518045 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc390064f60 0x7fc390067410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:12.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.562+0000 7fc38b7fe700 1 -- 172.21.15.40:0/2757518045 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc394021030 con 0x7fc3a4069e00 2023-11-08T08:20:12.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.562+0000 7fc3a259c700 1 --2- 172.21.15.40:0/2757518045 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc390064f60 0x7fc390067410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:12.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.562+0000 7fc3a259c700 1 --2- 172.21.15.40:0/2757518045 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc390064f60 0x7fc390067410 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40d1f10 tx=0x7fc398009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:12.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.566+0000 7fc38b7fe700 1 -- 172.21.15.40:0/2757518045 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc394051ad0 con 0x7fc3a4069e00 2023-11-08T08:20:12.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.742+0000 7fc3a8833700 1 -- 172.21.15.40:0/2757518045 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3a415de00 con 0x7fc3a4069e00 2023-11-08T08:20:12.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.746+0000 7fc38b7fe700 1 -- 172.21.15.40:0/2757518045 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19748 (secure 0 0 0) 0x7fc3a415de00 con 0x7fc3a4069e00 2023-11-08T08:20:12.750 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:12.750 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":30,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:45.885827+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:20:12.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.750+0000 7fc3a8833700 1 -- 172.21.15.40:0/2757518045 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc390064f60 msgr2=0x7fc390067410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:12.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.750+0000 7fc3a8833700 1 --2- 172.21.15.40:0/2757518045 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc390064f60 0x7fc390067410 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40d1f10 tx=0x7fc398009240 comp rx=0 tx=0).stop 2023-11-08T08:20:12.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.750+0000 7fc3a8833700 1 -- 172.21.15.40:0/2757518045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3a4069e00 msgr2=0x7fc3a415e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:12.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.750+0000 7fc3a8833700 1 --2- 172.21.15.40:0/2757518045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3a4069e00 0x7fc3a415e1d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc394016000 tx=0x7fc394004770 comp rx=0 tx=0).stop 2023-11-08T08:20:12.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.750+0000 7fc3a8833700 1 -- 172.21.15.40:0/2757518045 shutdown_connections 2023-11-08T08:20:12.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.750+0000 7fc3a8833700 1 --2- 172.21.15.40:0/2757518045 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc390064f60 0x7fc390067410 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:12.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.750+0000 7fc3a8833700 1 --2- 172.21.15.40:0/2757518045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a415c640 0x7fc3a41f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:12.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.750+0000 7fc3a8833700 1 --2- 172.21.15.40:0/2757518045 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3a4071aa0 0x7fc3a415e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:12.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.750+0000 7fc3a8833700 1 --2- 172.21.15.40:0/2757518045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3a4069e00 0x7fc3a415e1d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:12.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.750+0000 7fc3a8833700 1 -- 172.21.15.40:0/2757518045 >> 172.21.15.40:0/2757518045 conn(0x7fc3a40b6520 msgr2=0x7fc3a40af580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:12.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.750+0000 7fc3a8833700 1 -- 172.21.15.40:0/2757518045 shutdown_connections 2023-11-08T08:20:12.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.750+0000 7fc3a8833700 1 -- 172.21.15.40:0/2757518045 wait complete. 2023-11-08T08:20:12.778 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T08:19:02.627691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T08:19:03.473821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T08:19:06.539942+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T08:20:12.779 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:20:12.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.974+0000 7fad0482f700 1 Processor -- start 2023-11-08T08:20:12.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.974+0000 7fad0482f700 1 -- start start 2023-11-08T08:20:12.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.978+0000 7fad0482f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad000b3d00 0x7fad0006a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:12.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.978+0000 7fad0482f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0015c520 con 0x7fad0015c150 2023-11-08T08:20:12.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.978+0000 7fad0482f700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0006cc50 con 0x7fad00070620 2023-11-08T08:20:12.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.978+0000 7fad0482f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad000d2450 con 0x7fad000b3d00 2023-11-08T08:20:12.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.978+0000 7facfed9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fad0015c150 0x7fad0006ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59438/0 (socket says 172.21.15.40:59438) 2023-11-08T08:20:12.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.978+0000 7facfed9d700 1 -- 172.21.15.40:0/469587208 learned_addr learned my addr 172.21.15.40:0/469587208 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:12.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.978+0000 7facff59e700 1 --2- 172.21.15.40:0/469587208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad000b3d00 0x7fad0006a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:12.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.978+0000 7facfdd9b700 1 -- 172.21.15.40:0/469587208 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1464121723 0 0) 0x7fad0015c520 con 0x7fad0015c150 2023-11-08T08:20:12.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.978+0000 7facfdd9b700 1 -- 172.21.15.40:0/469587208 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7face80035d0 con 0x7fad0015c150 2023-11-08T08:20:12.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.978+0000 7facfdd9b700 1 -- 172.21.15.40:0/469587208 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3999345319 0 0) 0x7fad0006cc50 con 0x7fad00070620 2023-11-08T08:20:12.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.978+0000 7facfdd9b700 1 -- 172.21.15.40:0/469587208 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad0015c520 con 0x7fad00070620 2023-11-08T08:20:12.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.978+0000 7facfdd9b700 1 -- 172.21.15.40:0/469587208 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 631666593 0 0) 0x7face80035d0 con 0x7fad0015c150 2023-11-08T08:20:12.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.978+0000 7facff59e700 1 -- 172.21.15.40:0/469587208 >> v1:172.21.15.169:6789/0 conn(0x7fad00070620 legacy=0x7fad00069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:12.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.978+0000 7facff59e700 1 -- 172.21.15.40:0/469587208 >> v1:172.21.15.40:6789/0 conn(0x7fad0015c150 legacy=0x7fad0006ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:12.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.978+0000 7facff59e700 1 -- 172.21.15.40:0/469587208 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad000d2730 con 0x7fad000b3d00 2023-11-08T08:20:12.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.982+0000 7facff59e700 1 --2- 172.21.15.40:0/469587208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad000b3d00 0x7fad0006a300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7facf8009dc0 tx=0x7facf800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd46159c13583e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:12.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.982+0000 7facfdd9b700 1 -- 172.21.15.40:0/469587208 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf800e070 con 0x7fad000b3d00 2023-11-08T08:20:12.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.982+0000 7facfdd9b700 1 -- 172.21.15.40:0/469587208 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf8003a40 con 0x7fad000b3d00 2023-11-08T08:20:12.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.982+0000 7facfdd9b700 1 -- 172.21.15.40:0/469587208 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf0001670 con 0x7fad000b3d00 2023-11-08T08:20:12.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.982+0000 7fad0482f700 1 -- 172.21.15.40:0/469587208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad000b3d00 msgr2=0x7fad0006a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:12.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.982+0000 7fad0482f700 1 --2- 172.21.15.40:0/469587208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad000b3d00 0x7fad0006a300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7facf8009dc0 tx=0x7facf800b330 comp rx=0 tx=0).stop 2023-11-08T08:20:12.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.982+0000 7fad0482f700 1 -- 172.21.15.40:0/469587208 shutdown_connections 2023-11-08T08:20:12.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.982+0000 7fad0482f700 1 --2- 172.21.15.40:0/469587208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad000b3d00 0x7fad0006a300 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:12.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.982+0000 7fad0482f700 1 -- 172.21.15.40:0/469587208 >> 172.21.15.40:0/469587208 conn(0x7fad000d3680 msgr2=0x7fad000cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:12.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.982+0000 7fad0482f700 1 -- 172.21.15.40:0/469587208 shutdown_connections 2023-11-08T08:20:12.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.982+0000 7fad0482f700 1 -- 172.21.15.40:0/469587208 wait complete. 2023-11-08T08:20:12.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.986+0000 7fad0482f700 1 Processor -- start 2023-11-08T08:20:12.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.986+0000 7fad0482f700 1 -- start start 2023-11-08T08:20:12.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.986+0000 7fad0482f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad00070620 0x7fad000cea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:12.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.986+0000 7fad0482f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad000b3d00 0x7fad0013f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:12.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.986+0000 7fad0482f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad0015c150 0x7fad000b7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:12.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.986+0000 7fad0482f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0013fdf0 con 0x7fad00070620 2023-11-08T08:20:12.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.986+0000 7fad0482f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad000cf130 con 0x7fad000b3d00 2023-11-08T08:20:12.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.986+0000 7fad0482f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad000cf410 con 0x7fad0015c150 2023-11-08T08:20:12.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.986+0000 7facfed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad00070620 0x7fad000cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:12.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.986+0000 7facfed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad00070620 0x7fad000cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40172/0 (socket says 172.21.15.40:40172) 2023-11-08T08:20:12.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.986+0000 7facfed9d700 1 -- 172.21.15.40:0/2288947312 learned_addr learned my addr 172.21.15.40:0/2288947312 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:12.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.986+0000 7facfe59c700 1 --2- 172.21.15.40:0/2288947312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad000b3d00 0x7fad0013f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:12.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.986+0000 7facff59e700 1 --2- 172.21.15.40:0/2288947312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad0015c150 0x7fad000b7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:12.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.986+0000 7facfed9d700 1 -- 172.21.15.40:0/2288947312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad0015c150 msgr2=0x7fad000b7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:12.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.986+0000 7facfed9d700 1 --2- 172.21.15.40:0/2288947312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad0015c150 0x7fad000b7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:12.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.986+0000 7facfed9d700 1 -- 172.21.15.40:0/2288947312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad000b3d00 msgr2=0x7fad0013f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:12.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.986+0000 7facfed9d700 1 --2- 172.21.15.40:0/2288947312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad000b3d00 0x7fad0013f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:12.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.986+0000 7facfed9d700 1 -- 172.21.15.40:0/2288947312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf8009a70 con 0x7fad00070620 2023-11-08T08:20:12.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.986+0000 7facff59e700 1 --2- 172.21.15.40:0/2288947312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad0015c150 0x7fad000b7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:12.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.986+0000 7facfed9d700 1 --2- 172.21.15.40:0/2288947312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad00070620 0x7fad000cea70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7facf000c750 tx=0x7facf000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:12.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.990+0000 7face77fe700 1 -- 172.21.15.40:0/2288947312 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf0004ca0 con 0x7fad00070620 2023-11-08T08:20:12.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.990+0000 7face77fe700 1 -- 172.21.15.40:0/2288947312 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf0004e20 con 0x7fad00070620 2023-11-08T08:20:12.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.990+0000 7fad0482f700 1 -- 172.21.15.40:0/2288947312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad000d2140 con 0x7fad00070620 2023-11-08T08:20:12.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.990+0000 7face77fe700 1 -- 172.21.15.40:0/2288947312 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf0007310 con 0x7fad00070620 2023-11-08T08:20:12.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.990+0000 7fad0482f700 1 -- 172.21.15.40:0/2288947312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad000d05e0 con 0x7fad00070620 2023-11-08T08:20:12.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.990+0000 7fad0482f700 1 -- 172.21.15.40:0/2288947312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad00070cd0 con 0x7fad00070620 2023-11-08T08:20:12.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.990+0000 7face77fe700 1 -- 172.21.15.40:0/2288947312 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7facf001a020 con 0x7fad00070620 2023-11-08T08:20:12.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.994+0000 7face77fe700 1 --2- 172.21.15.40:0/2288947312 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7facec064ee0 0x7facec067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:12.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.994+0000 7face77fe700 1 -- 172.21.15.40:0/2288947312 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7facf0081ed0 con 0x7fad00070620 2023-11-08T08:20:12.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.994+0000 7facfe59c700 1 --2- 172.21.15.40:0/2288947312 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7facec064ee0 0x7facec067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:12.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.994+0000 7facfe59c700 1 --2- 172.21.15.40:0/2288947312 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7facec064ee0 0x7facec067390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fad0014dea0 tx=0x7facf4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:13.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:12.998+0000 7face77fe700 1 -- 172.21.15.40:0/2288947312 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7facf004fc50 con 0x7fad00070620 2023-11-08T08:20:13.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.174+0000 7fad0482f700 1 -- 172.21.15.40:0/2288947312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fad00073d20 con 0x7fad00070620 2023-11-08T08:20:13.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.174+0000 7face77fe700 1 -- 172.21.15.40:0/2288947312 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7facf0053270 con 0x7fad00070620 2023-11-08T08:20:13.179 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:20:13.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.178+0000 7fad0482f700 1 -- 172.21.15.40:0/2288947312 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7facec064ee0 msgr2=0x7facec067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:13.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.178+0000 7fad0482f700 1 --2- 172.21.15.40:0/2288947312 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7facec064ee0 0x7facec067390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fad0014dea0 tx=0x7facf4009040 comp rx=0 tx=0).stop 2023-11-08T08:20:13.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.178+0000 7fad0482f700 1 -- 172.21.15.40:0/2288947312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad00070620 msgr2=0x7fad000cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:13.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.178+0000 7fad0482f700 1 --2- 172.21.15.40:0/2288947312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad00070620 0x7fad000cea70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7facf000c750 tx=0x7facf000ca60 comp rx=0 tx=0).stop 2023-11-08T08:20:13.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.178+0000 7fad0482f700 1 -- 172.21.15.40:0/2288947312 shutdown_connections 2023-11-08T08:20:13.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.178+0000 7fad0482f700 1 --2- 172.21.15.40:0/2288947312 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7facec064ee0 0x7facec067390 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:13.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.178+0000 7fad0482f700 1 --2- 172.21.15.40:0/2288947312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad0015c150 0x7fad000b7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:13.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.178+0000 7fad0482f700 1 --2- 172.21.15.40:0/2288947312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad000b3d00 0x7fad0013f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:13.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.178+0000 7fad0482f700 1 --2- 172.21.15.40:0/2288947312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad00070620 0x7fad000cea70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:13.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.178+0000 7fad0482f700 1 -- 172.21.15.40:0/2288947312 >> 172.21.15.40:0/2288947312 conn(0x7fad000d3680 msgr2=0x7fad00158d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:13.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.178+0000 7fad0482f700 1 -- 172.21.15.40:0/2288947312 shutdown_connections 2023-11-08T08:20:13.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.178+0000 7fad0482f700 1 -- 172.21.15.40:0/2288947312 wait complete. 2023-11-08T08:20:13.204 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:20:13.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.382+0000 7f16bffff700 1 Processor -- start 2023-11-08T08:20:13.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.382+0000 7f16bffff700 1 -- start start 2023-11-08T08:20:13.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bffff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f16c00b3d00 0x7f16c006a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:13.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bffff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c015c520 con 0x7f16c0070620 2023-11-08T08:20:13.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bffff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c006cc50 con 0x7f16c015c150 2023-11-08T08:20:13.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bffff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c00d2450 con 0x7f16c00b3d00 2023-11-08T08:20:13.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16be7fc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f16c0070620 0x7f16c0069bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59454/0 (socket says 172.21.15.40:59454) 2023-11-08T08:20:13.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16be7fc700 1 -- 172.21.15.40:0/3900377811 learned_addr learned my addr 172.21.15.40:0/3900377811 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:13.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16beffd700 1 --1- 172.21.15.40:0/3900377811 >> v1:172.21.15.169:6789/0 conn(0x7f16c015c150 0x7f16c006ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:40696/0 (socket says 172.21.15.40:40696) 2023-11-08T08:20:13.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bf7fe700 1 --2- 172.21.15.40:0/3900377811 >> v2:172.21.15.169:3300/0 conn(0x7f16c00b3d00 0x7f16c006a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:13.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bdffb700 1 -- 172.21.15.40:0/3900377811 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1075788197 0 0) 0x7f16c006cc50 con 0x7f16c015c150 2023-11-08T08:20:13.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bdffb700 1 -- 172.21.15.40:0/3900377811 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16a80035d0 con 0x7f16c015c150 2023-11-08T08:20:13.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bdffb700 1 -- 172.21.15.40:0/3900377811 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1646474038 0 0) 0x7f16c015c520 con 0x7f16c0070620 2023-11-08T08:20:13.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bdffb700 1 -- 172.21.15.40:0/3900377811 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16c006cc50 con 0x7f16c0070620 2023-11-08T08:20:13.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bdffb700 1 -- 172.21.15.40:0/3900377811 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 915737578 0 0) 0x7f16a80035d0 con 0x7f16c015c150 2023-11-08T08:20:13.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bdffb700 1 -- 172.21.15.40:0/3900377811 >> v2:172.21.15.169:3300/0 conn(0x7f16c00b3d00 msgr2=0x7f16c006a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:13.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bdffb700 1 --2- 172.21.15.40:0/3900377811 >> v2:172.21.15.169:3300/0 conn(0x7f16c00b3d00 0x7f16c006a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:13.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bdffb700 1 -- 172.21.15.40:0/3900377811 >> v1:172.21.15.40:6789/0 conn(0x7f16c0070620 legacy=0x7f16c0069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:13.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bdffb700 1 -- 172.21.15.40:0/3900377811 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c00d2730 con 0x7f16c015c150 2023-11-08T08:20:13.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bdffb700 1 -- 172.21.15.40:0/3900377811 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f16b0001670 con 0x7f16c015c150 2023-11-08T08:20:13.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bdffb700 1 -- 172.21.15.40:0/3900377811 >> v1:172.21.15.169:6789/0 conn(0x7f16c015c150 legacy=0x7f16c006ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:13.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bdffb700 1 --2- 172.21.15.40:0/3900377811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a8003bb0 0x7f16a8003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:13.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bdffb700 1 --2- 172.21.15.40:0/3900377811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a8004870 0x7f16a8004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:13.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bdffb700 1 --2- 172.21.15.40:0/3900377811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a8005220 0x7f16a800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:13.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bdffb700 1 -- 172.21.15.40:0/3900377811 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b4004d00 con 0x7f16a8004870 2023-11-08T08:20:13.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bdffb700 1 -- 172.21.15.40:0/3900377811 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00d2450 con 0x7f16a8005220 2023-11-08T08:20:13.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.386+0000 7f16bdffb700 1 -- 172.21.15.40:0/3900377811 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a800c0a0 con 0x7f16a8003bb0 2023-11-08T08:20:13.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.390+0000 7f16be7fc700 1 --2- 172.21.15.40:0/3900377811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a8004870 0x7f16a8004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:13.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.390+0000 7f16beffd700 1 --2- 172.21.15.40:0/3900377811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a8003bb0 0x7f16a8003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:13.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.390+0000 7f16bf7fe700 1 --2- 172.21.15.40:0/3900377811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a8005220 0x7f16a800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:13.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.390+0000 7f16be7fc700 1 -- 172.21.15.40:0/3900377811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a8003bb0 msgr2=0x7f16a8003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:13.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.390+0000 7f16be7fc700 1 --2- 172.21.15.40:0/3900377811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16a8003bb0 0x7f16a8003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:13.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.390+0000 7f16be7fc700 1 -- 172.21.15.40:0/3900377811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a8005220 msgr2=0x7f16a800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:13.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.390+0000 7f16be7fc700 1 --2- 172.21.15.40:0/3900377811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16a8005220 0x7f16a800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:13.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.390+0000 7f16be7fc700 1 -- 172.21.15.40:0/3900377811 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a80035d0 con 0x7f16a8004870 2023-11-08T08:20:13.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.390+0000 7f16bf7fe700 1 -- 172.21.15.40:0/3900377811 reap_dead start 2023-11-08T08:20:13.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.390+0000 7f16be7fc700 1 --2- 172.21.15.40:0/3900377811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a8004870 0x7f16a8004ce0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f16b4007fd0 tx=0x7f16a80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6e62b1886d7493 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:13.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.390+0000 7f16bdffb700 1 -- 172.21.15.40:0/3900377811 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b400cc90 con 0x7f16a8004870 2023-11-08T08:20:13.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.390+0000 7f16bdffb700 1 -- 172.21.15.40:0/3900377811 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b400cdf0 con 0x7f16a8004870 2023-11-08T08:20:13.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.390+0000 7f16bdffb700 1 -- 172.21.15.40:0/3900377811 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b4005ba0 con 0x7f16a8004870 2023-11-08T08:20:13.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.390+0000 7f16bffff700 1 -- 172.21.15.40:0/3900377811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a8004870 msgr2=0x7f16a8004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:13.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.390+0000 7f16bffff700 1 --2- 172.21.15.40:0/3900377811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a8004870 0x7f16a8004ce0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f16b4007fd0 tx=0x7f16a80058a0 comp rx=0 tx=0).stop 2023-11-08T08:20:13.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.390+0000 7f16bffff700 1 -- 172.21.15.40:0/3900377811 shutdown_connections 2023-11-08T08:20:13.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.390+0000 7f16bffff700 1 --2- 172.21.15.40:0/3900377811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a8004870 0x7f16a8004ce0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:13.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.390+0000 7f16bffff700 1 -- 172.21.15.40:0/3900377811 >> 172.21.15.40:0/3900377811 conn(0x7f16c00d3680 msgr2=0x7f16c00cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:13.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.390+0000 7f16bffff700 1 -- 172.21.15.40:0/3900377811 shutdown_connections 2023-11-08T08:20:13.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.390+0000 7f16bffff700 1 -- 172.21.15.40:0/3900377811 wait complete. 2023-11-08T08:20:13.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.394+0000 7f16bffff700 1 Processor -- start 2023-11-08T08:20:13.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.394+0000 7f16bffff700 1 -- start start 2023-11-08T08:20:13.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16bffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a8004870 0x7f16c00cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:13.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16bffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16c014a7b0 0x7f16c01474e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16bffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16c00b7960 0x7f16c00b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16bffff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c0147c90 con 0x7f16a8004870 2023-11-08T08:20:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16bffff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00cffc0 con 0x7f16c014a7b0 2023-11-08T08:20:13.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16bffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c0147a20 con 0x7f16c00b7960 2023-11-08T08:20:13.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16beffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a8004870 0x7f16c00cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:13.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16beffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a8004870 0x7f16c00cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40204/0 (socket says 172.21.15.40:40204) 2023-11-08T08:20:13.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16beffd700 1 -- 172.21.15.40:0/1016464706 learned_addr learned my addr 172.21.15.40:0/1016464706 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:13.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16be7fc700 1 --2- 172.21.15.40:0/1016464706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16c014a7b0 0x7f16c01474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:13.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16bf7fe700 1 --2- 172.21.15.40:0/1016464706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16c00b7960 0x7f16c00b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:13.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16beffd700 1 -- 172.21.15.40:0/1016464706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16c00b7960 msgr2=0x7f16c00b7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:13.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16beffd700 1 --2- 172.21.15.40:0/1016464706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16c00b7960 0x7f16c00b7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:13.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16beffd700 1 -- 172.21.15.40:0/1016464706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16c014a7b0 msgr2=0x7f16c01474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:13.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16beffd700 1 --2- 172.21.15.40:0/1016464706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16c014a7b0 0x7f16c01474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:13.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16beffd700 1 -- 172.21.15.40:0/1016464706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c00d2140 con 0x7f16a8004870 2023-11-08T08:20:13.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16beffd700 1 --2- 172.21.15.40:0/1016464706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a8004870 0x7f16c00cf8c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f16b0004fd0 tx=0x7f16b0007340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:13.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16a77fe700 1 -- 172.21.15.40:0/1016464706 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0011580 con 0x7f16a8004870 2023-11-08T08:20:13.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16a77fe700 1 -- 172.21.15.40:0/1016464706 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b0011700 con 0x7f16a8004870 2023-11-08T08:20:13.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16a77fe700 1 -- 172.21.15.40:0/1016464706 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0011ab0 con 0x7f16a8004870 2023-11-08T08:20:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16bffff700 1 -- 172.21.15.40:0/1016464706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c00d05e0 con 0x7f16a8004870 2023-11-08T08:20:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.398+0000 7f16bffff700 1 -- 172.21.15.40:0/1016464706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c0070cd0 con 0x7f16a8004870 2023-11-08T08:20:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.402+0000 7f16bffff700 1 -- 172.21.15.40:0/1016464706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c0073d20 con 0x7f16a8004870 2023-11-08T08:20:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.402+0000 7f16a77fe700 1 -- 172.21.15.40:0/1016464706 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f16b0011c10 con 0x7f16a8004870 2023-11-08T08:20:13.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.406+0000 7f16a77fe700 1 --2- 172.21.15.40:0/1016464706 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f16ac064ee0 0x7f16ac067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:13.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.406+0000 7f16a77fe700 1 -- 172.21.15.40:0/1016464706 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f16b00822b0 con 0x7f16a8004870 2023-11-08T08:20:13.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.406+0000 7f16be7fc700 1 --2- 172.21.15.40:0/1016464706 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f16ac064ee0 0x7f16ac067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:13.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.406+0000 7f16be7fc700 1 --2- 172.21.15.40:0/1016464706 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f16ac064ee0 0x7f16ac067390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f16b4007fd0 tx=0x7f16b4007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:13.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.410+0000 7f16a77fe700 1 -- 172.21.15.40:0/1016464706 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f16b0050120 con 0x7f16a8004870 2023-11-08T08:20:13.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.582+0000 7f16bffff700 1 -- 172.21.15.40:0/1016464706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f16c00d0150 con 0x7f16a8004870 2023-11-08T08:20:13.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.582+0000 7f16a77fe700 1 -- 172.21.15.40:0/1016464706 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f16b0053740 con 0x7f16a8004870 2023-11-08T08:20:13.588 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:20:13.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.586+0000 7f16bffff700 1 -- 172.21.15.40:0/1016464706 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f16ac064ee0 msgr2=0x7f16ac067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:13.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.586+0000 7f16bffff700 1 --2- 172.21.15.40:0/1016464706 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f16ac064ee0 0x7f16ac067390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f16b4007fd0 tx=0x7f16b4007000 comp rx=0 tx=0).stop 2023-11-08T08:20:13.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.586+0000 7f16bffff700 1 -- 172.21.15.40:0/1016464706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a8004870 msgr2=0x7f16c00cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:13.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.586+0000 7f16bffff700 1 --2- 172.21.15.40:0/1016464706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a8004870 0x7f16c00cf8c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f16b0004fd0 tx=0x7f16b0007340 comp rx=0 tx=0).stop 2023-11-08T08:20:13.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.586+0000 7f16bffff700 1 -- 172.21.15.40:0/1016464706 shutdown_connections 2023-11-08T08:20:13.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.586+0000 7f16bffff700 1 --2- 172.21.15.40:0/1016464706 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f16ac064ee0 0x7f16ac067390 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:13.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.586+0000 7f16bffff700 1 --2- 172.21.15.40:0/1016464706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16c00b7960 0x7f16c00b7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:13.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.586+0000 7f16bffff700 1 --2- 172.21.15.40:0/1016464706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16c014a7b0 0x7f16c01474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.586+0000 7f16bffff700 1 --2- 172.21.15.40:0/1016464706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a8004870 0x7f16c00cf8c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.586+0000 7f16bffff700 1 -- 172.21.15.40:0/1016464706 >> 172.21.15.40:0/1016464706 conn(0x7f16c00d3680 msgr2=0x7f16c0156b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.590+0000 7f16bffff700 1 -- 172.21.15.40:0/1016464706 shutdown_connections 2023-11-08T08:20:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.590+0000 7f16bffff700 1 -- 172.21.15.40:0/1016464706 wait complete. 2023-11-08T08:20:13.615 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T08:20:13.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.794+0000 7f1c9b769700 1 Processor -- start 2023-11-08T08:20:13.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.794+0000 7f1c9b769700 1 -- start start 2023-11-08T08:20:13.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.794+0000 7f1c9b769700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1c9415c700 0x7f1c940702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:13.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.794+0000 7f1c9b769700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c940b1be0 0x7f1c94070830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:13.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.794+0000 7f1c9b769700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c9406ac00 con 0x7f1c940b2130 2023-11-08T08:20:13.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.794+0000 7f1c9b769700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c940cd360 con 0x7f1c9415c700 2023-11-08T08:20:13.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.794+0000 7f1c9b769700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c94160190 con 0x7f1c940b1be0 2023-11-08T08:20:13.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.794+0000 7f1c99505700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1c9415c700 0x7f1c940702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:13.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.794+0000 7f1c99505700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1c9415c700 0x7f1c940702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40212/0 (socket says 172.21.15.40:40212) 2023-11-08T08:20:13.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.798+0000 7f1c99505700 1 -- 172.21.15.40:0/1862665960 learned_addr learned my addr 172.21.15.40:0/1862665960 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:13.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.798+0000 7f1c99d06700 1 --2- 172.21.15.40:0/1862665960 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c940b1be0 0x7f1c94070830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:13.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.798+0000 7f1c99505700 1 -- 172.21.15.40:0/1862665960 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c940b1be0 msgr2=0x7f1c94070830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:13.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.798+0000 7f1c99505700 1 --2- 172.21.15.40:0/1862665960 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c940b1be0 0x7f1c94070830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:13.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.798+0000 7f1c99505700 1 -- 172.21.15.40:0/1862665960 >> v1:172.21.15.169:6789/0 conn(0x7f1c940b2130 legacy=0x7f1c9406c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:13.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.798+0000 7f1c99505700 1 -- 172.21.15.40:0/1862665960 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c94160470 con 0x7f1c9415c700 2023-11-08T08:20:13.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.798+0000 7f1c8bfff700 1 -- 172.21.15.40:0/1862665960 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445413334 0 0) 0x7f1c9406ac00 con 0x7f1c940b2130 2023-11-08T08:20:13.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.798+0000 7f1c99505700 1 --2- 172.21.15.40:0/1862665960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c9415c700 0x7f1c940702f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009e50 tx=0x7f1c9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a15d07db425508 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:13.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.798+0000 7f1c8bfff700 1 -- 172.21.15.40:0/1862665960 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9000e070 con 0x7f1c9415c700 2023-11-08T08:20:13.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.798+0000 7f1c8bfff700 1 -- 172.21.15.40:0/1862665960 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9000b910 con 0x7f1c9415c700 2023-11-08T08:20:13.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.798+0000 7f1c8bfff700 1 -- 172.21.15.40:0/1862665960 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9000bcc0 con 0x7f1c9415c700 2023-11-08T08:20:13.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.798+0000 7f1c9b769700 1 -- 172.21.15.40:0/1862665960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c9415c700 msgr2=0x7f1c940702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:13.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.798+0000 7f1c9b769700 1 --2- 172.21.15.40:0/1862665960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c9415c700 0x7f1c940702f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009e50 tx=0x7f1c9000b330 comp rx=0 tx=0).stop 2023-11-08T08:20:13.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.798+0000 7f1c9b769700 1 -- 172.21.15.40:0/1862665960 shutdown_connections 2023-11-08T08:20:13.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.798+0000 7f1c9b769700 1 --2- 172.21.15.40:0/1862665960 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c940b1be0 0x7f1c94070830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:13.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.798+0000 7f1c9b769700 1 --2- 172.21.15.40:0/1862665960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c9415c700 0x7f1c940702f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:13.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.798+0000 7f1c9b769700 1 -- 172.21.15.40:0/1862665960 >> 172.21.15.40:0/1862665960 conn(0x7f1c940d2d10 msgr2=0x7f1c940d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:13.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.798+0000 7f1c9b769700 1 -- 172.21.15.40:0/1862665960 shutdown_connections 2023-11-08T08:20:13.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.798+0000 7f1c9b769700 1 -- 172.21.15.40:0/1862665960 wait complete. 2023-11-08T08:20:13.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.802+0000 7f1c9b769700 1 Processor -- start 2023-11-08T08:20:13.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.802+0000 7f1c9b769700 1 -- start start 2023-11-08T08:20:13.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c9b769700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c940b1be0 0x7f1c9415dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:13.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c9b769700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c940b2130 0x7f1c940b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:13.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c9b769700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c9415c700 0x7f1c94155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:13.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c9b769700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9415d920 con 0x7f1c9415c700 2023-11-08T08:20:13.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c9b769700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c941f2ab0 con 0x7f1c940b2130 2023-11-08T08:20:13.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c9b769700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c941f2d90 con 0x7f1c940b1be0 2023-11-08T08:20:13.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c99d06700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c9415c700 0x7f1c94155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:13.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c99d06700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c9415c700 0x7f1c94155e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40224/0 (socket says 172.21.15.40:40224) 2023-11-08T08:20:13.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c99d06700 1 -- 172.21.15.40:0/1247054298 learned_addr learned my addr 172.21.15.40:0/1247054298 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:13.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c99505700 1 --2- 172.21.15.40:0/1247054298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c940b1be0 0x7f1c9415dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:13.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c98d04700 1 --2- 172.21.15.40:0/1247054298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c940b2130 0x7f1c940b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:13.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c99d06700 1 -- 172.21.15.40:0/1247054298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c940b1be0 msgr2=0x7f1c9415dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:13.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c99d06700 1 --2- 172.21.15.40:0/1247054298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c940b1be0 0x7f1c9415dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:13.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c99d06700 1 -- 172.21.15.40:0/1247054298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c940b2130 msgr2=0x7f1c940b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:13.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c99d06700 1 --2- 172.21.15.40:0/1247054298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c940b2130 0x7f1c940b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:13.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c99d06700 1 -- 172.21.15.40:0/1247054298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c90009b30 con 0x7f1c9415c700 2023-11-08T08:20:13.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c98d04700 1 --2- 172.21.15.40:0/1247054298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c940b2130 0x7f1c940b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:13.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c99d06700 1 --2- 172.21.15.40:0/1247054298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c9415c700 0x7f1c94155e40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00e7f0 tx=0x7f1c8c00eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:13.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c89ffb700 1 -- 172.21.15.40:0/1247054298 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c0095c0 con 0x7f1c9415c700 2023-11-08T08:20:13.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c89ffb700 1 -- 172.21.15.40:0/1247054298 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8c013680 con 0x7f1c9415c700 2023-11-08T08:20:13.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c9b769700 1 -- 172.21.15.40:0/1247054298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c94160a30 con 0x7f1c9415c700 2023-11-08T08:20:13.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c89ffb700 1 -- 172.21.15.40:0/1247054298 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c013a10 con 0x7f1c9415c700 2023-11-08T08:20:13.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.806+0000 7f1c9b769700 1 -- 172.21.15.40:0/1247054298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c94160750 con 0x7f1c9415c700 2023-11-08T08:20:13.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.810+0000 7f1c9b769700 1 -- 172.21.15.40:0/1247054298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c9406b530 con 0x7f1c9415c700 2023-11-08T08:20:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.810+0000 7f1c89ffb700 1 -- 172.21.15.40:0/1247054298 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1c8c019030 con 0x7f1c9415c700 2023-11-08T08:20:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.810+0000 7f1c89ffb700 1 --2- 172.21.15.40:0/1247054298 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1c78064ee0 0x7f1c78067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.814+0000 7f1c89ffb700 1 -- 172.21.15.40:0/1247054298 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1c8c081ff0 con 0x7f1c9415c700 2023-11-08T08:20:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.814+0000 7f1c99505700 1 --2- 172.21.15.40:0/1247054298 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1c78064ee0 0x7f1c78067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:13.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.814+0000 7f1c99505700 1 --2- 172.21.15.40:0/1247054298 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1c78064ee0 0x7f1c78067390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000b370 tx=0x7f1c90009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:13.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.818+0000 7f1c89ffb700 1 -- 172.21.15.40:0/1247054298 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1c8c04fd70 con 0x7f1c9415c700 2023-11-08T08:20:13.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.990+0000 7f1c9b769700 1 -- 172.21.15.40:0/1247054298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c9415da60 con 0x7f1c9415c700 2023-11-08T08:20:13.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.994+0000 7f1c89ffb700 1 -- 172.21.15.40:0/1247054298 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1c9415da60 con 0x7f1c9415c700 2023-11-08T08:20:13.997 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:20:14.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.994+0000 7f1c9b769700 1 -- 172.21.15.40:0/1247054298 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1c78064ee0 msgr2=0x7f1c78067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:14.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.994+0000 7f1c9b769700 1 --2- 172.21.15.40:0/1247054298 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1c78064ee0 0x7f1c78067390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000b370 tx=0x7f1c90009000 comp rx=0 tx=0).stop 2023-11-08T08:20:14.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.998+0000 7f1c9b769700 1 -- 172.21.15.40:0/1247054298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c9415c700 msgr2=0x7f1c94155e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:14.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.998+0000 7f1c9b769700 1 --2- 172.21.15.40:0/1247054298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c9415c700 0x7f1c94155e40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00e7f0 tx=0x7f1c8c00eb00 comp rx=0 tx=0).stop 2023-11-08T08:20:14.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.998+0000 7f1c9b769700 1 -- 172.21.15.40:0/1247054298 shutdown_connections 2023-11-08T08:20:14.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.998+0000 7f1c9b769700 1 --2- 172.21.15.40:0/1247054298 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1c78064ee0 0x7f1c78067390 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.998+0000 7f1c9b769700 1 --2- 172.21.15.40:0/1247054298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c9415c700 0x7f1c94155e40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.998+0000 7f1c9b769700 1 --2- 172.21.15.40:0/1247054298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c940b2130 0x7f1c940b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.998+0000 7f1c9b769700 1 --2- 172.21.15.40:0/1247054298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c940b1be0 0x7f1c9415dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.998+0000 7f1c9b769700 1 -- 172.21.15.40:0/1247054298 >> 172.21.15.40:0/1247054298 conn(0x7f1c940d2d10 msgr2=0x7f1c940b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.998+0000 7f1c9b769700 1 -- 172.21.15.40:0/1247054298 shutdown_connections 2023-11-08T08:20:14.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:13.998+0000 7f1c9b769700 1 -- 172.21.15.40:0/1247054298 wait complete. 2023-11-08T08:20:14.024 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T08:20:14.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.206+0000 7f16caadd700 1 Processor -- start 2023-11-08T08:20:14.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.206+0000 7f16caadd700 1 -- start start 2023-11-08T08:20:14.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.210+0000 7f16caadd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16c4070a90 0x7f16c40c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:14.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.210+0000 7f16caadd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f16c415a9f0 0x7f16c406b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:14.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.210+0000 7f16caadd700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c4070600 con 0x7f16c4159f10 2023-11-08T08:20:14.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.210+0000 7f16caadd700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c4070f00 con 0x7f16c415a9f0 2023-11-08T08:20:14.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.210+0000 7f16caadd700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c406b840 con 0x7f16c4070a90 2023-11-08T08:20:14.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.210+0000 7f16c92da700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16c4070a90 0x7f16c40c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:14.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.210+0000 7f16c92da700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16c4070a90 0x7f16c40c8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:55004/0 (socket says 172.21.15.40:55004) 2023-11-08T08:20:14.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.210+0000 7f16c92da700 1 -- 172.21.15.40:0/3065971848 learned_addr learned my addr 172.21.15.40:0/3065971848 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:14.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.210+0000 7f16ca2dc700 1 --2- 172.21.15.40:0/3065971848 >> v2:172.21.15.169:3300/0 conn(0x7f16c415a9f0 0x7f16c406b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:14.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.210+0000 7f16c92da700 1 -- 172.21.15.40:0/3065971848 >> v2:172.21.15.169:3300/0 conn(0x7f16c415a9f0 msgr2=0x7f16c406b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:14.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.210+0000 7f16c92da700 1 --2- 172.21.15.40:0/3065971848 >> v2:172.21.15.169:3300/0 conn(0x7f16c415a9f0 0x7f16c406b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.210+0000 7f16c92da700 1 -- 172.21.15.40:0/3065971848 >> v1:172.21.15.169:6789/0 conn(0x7f16c4159f10 legacy=0x7f16c415a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:14.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.210+0000 7f16c92da700 1 -- 172.21.15.40:0/3065971848 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c415ae30 con 0x7f16c4070a90 2023-11-08T08:20:14.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.210+0000 7f16ca2dc700 1 --2- 172.21.15.40:0/3065971848 >> v2:172.21.15.169:3300/0 conn(0x7f16c415a9f0 0x7f16c406b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:14.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.210+0000 7f16c92da700 1 --2- 172.21.15.40:0/3065971848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16c4070a90 0x7f16c40c8c90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009910 tx=0x7f16b4009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2f6f3a49a136668 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:14.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.210+0000 7f16c8ad9700 1 -- 172.21.15.40:0/3065971848 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b400ba60 con 0x7f16c4070a90 2023-11-08T08:20:14.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.210+0000 7f16c8ad9700 1 -- 172.21.15.40:0/3065971848 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b400bbc0 con 0x7f16c4070a90 2023-11-08T08:20:14.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.210+0000 7f16c8ad9700 1 -- 172.21.15.40:0/3065971848 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b4009e60 con 0x7f16c4070a90 2023-11-08T08:20:14.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.214+0000 7f16caadd700 1 -- 172.21.15.40:0/3065971848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16c4070a90 msgr2=0x7f16c40c8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:14.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.214+0000 7f16caadd700 1 --2- 172.21.15.40:0/3065971848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16c4070a90 0x7f16c40c8c90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009910 tx=0x7f16b4009c20 comp rx=0 tx=0).stop 2023-11-08T08:20:14.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.214+0000 7f16caadd700 1 -- 172.21.15.40:0/3065971848 shutdown_connections 2023-11-08T08:20:14.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.214+0000 7f16caadd700 1 --2- 172.21.15.40:0/3065971848 >> v2:172.21.15.169:3300/0 conn(0x7f16c415a9f0 0x7f16c406b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.214+0000 7f16caadd700 1 --2- 172.21.15.40:0/3065971848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16c4070a90 0x7f16c40c8c90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.214+0000 7f16caadd700 1 -- 172.21.15.40:0/3065971848 >> 172.21.15.40:0/3065971848 conn(0x7f16c40cf280 msgr2=0x7f16c40cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:14.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.214+0000 7f16caadd700 1 -- 172.21.15.40:0/3065971848 shutdown_connections 2023-11-08T08:20:14.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.214+0000 7f16caadd700 1 -- 172.21.15.40:0/3065971848 wait complete. 2023-11-08T08:20:14.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.214+0000 7f16caadd700 1 Processor -- start 2023-11-08T08:20:14.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.214+0000 7f16caadd700 1 -- start start 2023-11-08T08:20:14.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16caadd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16c4070a90 0x7f16c415b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:14.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16caadd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16c4159f10 0x7f16c41551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:14.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16caadd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16c415a9f0 0x7f16c40b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:14.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16caadd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c4155730 con 0x7f16c4159f10 2023-11-08T08:20:14.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16caadd700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c40ad2f0 con 0x7f16c4070a90 2023-11-08T08:20:14.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16caadd700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c40b2500 con 0x7f16c415a9f0 2023-11-08T08:20:14.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16c92da700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16c4159f10 0x7f16c41551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:14.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16c92da700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16c4159f10 0x7f16c41551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40226/0 (socket says 172.21.15.40:40226) 2023-11-08T08:20:14.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16c92da700 1 -- 172.21.15.40:0/968374779 learned_addr learned my addr 172.21.15.40:0/968374779 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:14.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16c9adb700 1 --2- 172.21.15.40:0/968374779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16c4070a90 0x7f16c415b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:14.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16ca2dc700 1 --2- 172.21.15.40:0/968374779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16c415a9f0 0x7f16c40b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:14.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16c92da700 1 -- 172.21.15.40:0/968374779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16c415a9f0 msgr2=0x7f16c40b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:14.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16c92da700 1 --2- 172.21.15.40:0/968374779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16c415a9f0 0x7f16c40b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16c92da700 1 -- 172.21.15.40:0/968374779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16c4070a90 msgr2=0x7f16c415b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:14.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16c92da700 1 --2- 172.21.15.40:0/968374779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16c4070a90 0x7f16c415b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16c92da700 1 -- 172.21.15.40:0/968374779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b4009570 con 0x7f16c4159f10 2023-11-08T08:20:14.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16c9adb700 1 --2- 172.21.15.40:0/968374779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16c4070a90 0x7f16c415b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:14.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16c92da700 1 --2- 172.21.15.40:0/968374779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16c4159f10 0x7f16c41551f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f16b4000c00 tx=0x7f16b4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:14.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16ba7fc700 1 -- 172.21.15.40:0/968374779 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b4004510 con 0x7f16c4159f10 2023-11-08T08:20:14.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16ba7fc700 1 -- 172.21.15.40:0/968374779 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b40036c0 con 0x7f16c4159f10 2023-11-08T08:20:14.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16caadd700 1 -- 172.21.15.40:0/968374779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c406ffb0 con 0x7f16c4159f10 2023-11-08T08:20:14.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16ba7fc700 1 -- 172.21.15.40:0/968374779 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b4003a70 con 0x7f16c4159f10 2023-11-08T08:20:14.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.218+0000 7f16caadd700 1 -- 172.21.15.40:0/968374779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c40afcc0 con 0x7f16c4159f10 2023-11-08T08:20:14.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.222+0000 7f16caadd700 1 -- 172.21.15.40:0/968374779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c4070290 con 0x7f16c4159f10 2023-11-08T08:20:14.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.222+0000 7f16ba7fc700 1 -- 172.21.15.40:0/968374779 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f16b402a030 con 0x7f16c4159f10 2023-11-08T08:20:14.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.226+0000 7f16ba7fc700 1 --2- 172.21.15.40:0/968374779 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f16a8064f60 0x7f16a8067410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:14.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.226+0000 7f16ba7fc700 1 -- 172.21.15.40:0/968374779 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f16b4083300 con 0x7f16c4159f10 2023-11-08T08:20:14.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.226+0000 7f16c9adb700 1 --2- 172.21.15.40:0/968374779 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f16a8064f60 0x7f16a8067410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:14.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.226+0000 7f16c9adb700 1 --2- 172.21.15.40:0/968374779 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f16a8064f60 0x7f16a8067410 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f16c0000f60 tx=0x7f16c0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:14.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.230+0000 7f16ba7fc700 1 -- 172.21.15.40:0/968374779 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f16b4051100 con 0x7f16c4159f10 2023-11-08T08:20:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.402+0000 7f16caadd700 1 -- 172.21.15.40:0/968374779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f16c406cf30 con 0x7f16c4159f10 2023-11-08T08:20:14.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.406+0000 7f16ba7fc700 1 -- 172.21.15.40:0/968374779 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f16b4054720 con 0x7f16c4159f10 2023-11-08T08:20:14.408 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-11-08T08:20:14.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.406+0000 7f16caadd700 1 -- 172.21.15.40:0/968374779 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f16a8064f60 msgr2=0x7f16a8067410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:14.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.406+0000 7f16caadd700 1 --2- 172.21.15.40:0/968374779 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f16a8064f60 0x7f16a8067410 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f16c0000f60 tx=0x7f16c0009040 comp rx=0 tx=0).stop 2023-11-08T08:20:14.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.406+0000 7f16caadd700 1 -- 172.21.15.40:0/968374779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16c4159f10 msgr2=0x7f16c41551f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.406+0000 7f16caadd700 1 --2- 172.21.15.40:0/968374779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16c4159f10 0x7f16c41551f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f16b4000c00 tx=0x7f16b4004770 comp rx=0 tx=0).stop 2023-11-08T08:20:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.406+0000 7f16caadd700 1 -- 172.21.15.40:0/968374779 shutdown_connections 2023-11-08T08:20:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.406+0000 7f16caadd700 1 --2- 172.21.15.40:0/968374779 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f16a8064f60 0x7f16a8067410 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.406+0000 7f16caadd700 1 --2- 172.21.15.40:0/968374779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16c415a9f0 0x7f16c40b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.406+0000 7f16caadd700 1 --2- 172.21.15.40:0/968374779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16c4159f10 0x7f16c41551f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.406+0000 7f16caadd700 1 --2- 172.21.15.40:0/968374779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16c4070a90 0x7f16c415b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.406+0000 7f16caadd700 1 -- 172.21.15.40:0/968374779 >> 172.21.15.40:0/968374779 conn(0x7f16c40cf280 msgr2=0x7f16c40c3550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.406+0000 7f16caadd700 1 -- 172.21.15.40:0/968374779 shutdown_connections 2023-11-08T08:20:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.406+0000 7f16caadd700 1 -- 172.21.15.40:0/968374779 wait complete. 2023-11-08T08:20:14.436 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T08:20:14.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.626+0000 7f90ab158700 1 Processor -- start 2023-11-08T08:20:14.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.626+0000 7f90ab158700 1 -- start start 2023-11-08T08:20:14.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.626+0000 7f90ab158700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90a4070620 0x7f90a40b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:14.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.630+0000 7f90ab158700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f90a4069bf0 0x7f90a406a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:14.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.630+0000 7f90ab158700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a415c520 con 0x7f90a415c150 2023-11-08T08:20:14.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.630+0000 7f90ab158700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a40b4240 con 0x7f90a4069bf0 2023-11-08T08:20:14.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.630+0000 7f90ab158700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a406cc50 con 0x7f90a4070620 2023-11-08T08:20:14.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.630+0000 7f90a9955700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90a4070620 0x7f90a40b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:14.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.630+0000 7f90a9955700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90a4070620 0x7f90a40b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:55020/0 (socket says 172.21.15.40:55020) 2023-11-08T08:20:14.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.630+0000 7f90a9955700 1 -- 172.21.15.40:0/932421519 learned_addr learned my addr 172.21.15.40:0/932421519 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:14.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.630+0000 7f90aa957700 1 --2- 172.21.15.40:0/932421519 >> v2:172.21.15.169:3300/0 conn(0x7f90a4069bf0 0x7f90a406a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:14.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.630+0000 7f90a9955700 1 -- 172.21.15.40:0/932421519 >> v2:172.21.15.169:3300/0 conn(0x7f90a4069bf0 msgr2=0x7f90a406a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:14.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.630+0000 7f90a9955700 1 --2- 172.21.15.40:0/932421519 >> v2:172.21.15.169:3300/0 conn(0x7f90a4069bf0 0x7f90a406a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.630+0000 7f90a9955700 1 -- 172.21.15.40:0/932421519 >> v1:172.21.15.169:6789/0 conn(0x7f90a415c150 legacy=0x7f90a406ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:14.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.630+0000 7f90a9955700 1 -- 172.21.15.40:0/932421519 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a40d2450 con 0x7f90a4070620 2023-11-08T08:20:14.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.630+0000 7f90a9955700 1 --2- 172.21.15.40:0/932421519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90a4070620 0x7f90a40b3d00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9094009910 tx=0x7f9094009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c048242c7f7034b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:14.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.630+0000 7f90a9154700 1 -- 172.21.15.40:0/932421519 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094005e50 con 0x7f90a4070620 2023-11-08T08:20:14.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.630+0000 7f90a9154700 1 -- 172.21.15.40:0/932421519 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9094006030 con 0x7f90a4070620 2023-11-08T08:20:14.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.630+0000 7f90a9154700 1 -- 172.21.15.40:0/932421519 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90940063e0 con 0x7f90a4070620 2023-11-08T08:20:14.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.634+0000 7f90ab158700 1 -- 172.21.15.40:0/932421519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90a4070620 msgr2=0x7f90a40b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:14.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.634+0000 7f90ab158700 1 --2- 172.21.15.40:0/932421519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90a4070620 0x7f90a40b3d00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9094009910 tx=0x7f9094009c20 comp rx=0 tx=0).stop 2023-11-08T08:20:14.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.634+0000 7f90ab158700 1 -- 172.21.15.40:0/932421519 shutdown_connections 2023-11-08T08:20:14.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.634+0000 7f90ab158700 1 --2- 172.21.15.40:0/932421519 >> v2:172.21.15.169:3300/0 conn(0x7f90a4069bf0 0x7f90a406a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.634+0000 7f90ab158700 1 --2- 172.21.15.40:0/932421519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90a4070620 0x7f90a40b3d00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.634+0000 7f90ab158700 1 -- 172.21.15.40:0/932421519 >> 172.21.15.40:0/932421519 conn(0x7f90a40d3680 msgr2=0x7f90a40cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:14.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.634+0000 7f90ab158700 1 -- 172.21.15.40:0/932421519 shutdown_connections 2023-11-08T08:20:14.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.634+0000 7f90ab158700 1 -- 172.21.15.40:0/932421519 wait complete. 2023-11-08T08:20:14.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.634+0000 7f90ab158700 1 Processor -- start 2023-11-08T08:20:14.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.634+0000 7f90ab158700 1 -- start start 2023-11-08T08:20:14.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f90ab158700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90a4069bf0 0x7f90a40cf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:14.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f90ab158700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90a4070620 0x7f90a40cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:14.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f90ab158700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90a415c150 0x7f90a40c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:14.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f90ab158700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40cfec0 con 0x7f90a4069bf0 2023-11-08T08:20:14.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f90ab158700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40b6210 con 0x7f90a4070620 2023-11-08T08:20:14.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f90ab158700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a4140140 con 0x7f90a415c150 2023-11-08T08:20:14.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f90aa156700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90a4069bf0 0x7f90a40cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:14.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f90aa156700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90a4069bf0 0x7f90a40cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40234/0 (socket says 172.21.15.40:40234) 2023-11-08T08:20:14.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f90aa156700 1 -- 172.21.15.40:0/3237440459 learned_addr learned my addr 172.21.15.40:0/3237440459 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:14.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f90a9955700 1 --2- 172.21.15.40:0/3237440459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90a4070620 0x7f90a40cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:14.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f90aa957700 1 --2- 172.21.15.40:0/3237440459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90a415c150 0x7f90a40c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:14.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f90aa156700 1 -- 172.21.15.40:0/3237440459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90a415c150 msgr2=0x7f90a40c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:14.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f90aa156700 1 --2- 172.21.15.40:0/3237440459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90a415c150 0x7f90a40c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f90aa156700 1 -- 172.21.15.40:0/3237440459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90a4070620 msgr2=0x7f90a40cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:14.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f90aa156700 1 --2- 172.21.15.40:0/3237440459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90a4070620 0x7f90a40cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f90aa156700 1 -- 172.21.15.40:0/3237440459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9094009520 con 0x7f90a4069bf0 2023-11-08T08:20:14.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f90aa156700 1 --2- 172.21.15.40:0/3237440459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90a4069bf0 0x7f90a40cf980 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f90a000c500 tx=0x7f90a000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:14.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f909affd700 1 -- 172.21.15.40:0/3237440459 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a00052d0 con 0x7f90a4069bf0 2023-11-08T08:20:14.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f909affd700 1 -- 172.21.15.40:0/3237440459 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a0005430 con 0x7f90a4069bf0 2023-11-08T08:20:14.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f909affd700 1 -- 172.21.15.40:0/3237440459 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a00055a0 con 0x7f90a4069bf0 2023-11-08T08:20:14.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f90ab158700 1 -- 172.21.15.40:0/3237440459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a40d05e0 con 0x7f90a4069bf0 2023-11-08T08:20:14.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.638+0000 7f90ab158700 1 -- 172.21.15.40:0/3237440459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a40d2730 con 0x7f90a4069bf0 2023-11-08T08:20:14.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.642+0000 7f90ab158700 1 -- 172.21.15.40:0/3237440459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a40d2140 con 0x7f90a4069bf0 2023-11-08T08:20:14.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.642+0000 7f909affd700 1 -- 172.21.15.40:0/3237440459 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f90a0019030 con 0x7f90a4069bf0 2023-11-08T08:20:14.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.642+0000 7f909affd700 1 --2- 172.21.15.40:0/3237440459 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9088064ee0 0x7f9088067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:14.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.646+0000 7f909affd700 1 -- 172.21.15.40:0/3237440459 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f90a0081370 con 0x7f90a4069bf0 2023-11-08T08:20:14.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.646+0000 7f90a9955700 1 --2- 172.21.15.40:0/3237440459 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9088064ee0 0x7f9088067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:14.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.646+0000 7f90a9955700 1 --2- 172.21.15.40:0/3237440459 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9088064ee0 0x7f9088067390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f90a4070a10 tx=0x7f9094008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:14.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.650+0000 7f909affd700 1 -- 172.21.15.40:0/3237440459 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f90a004f1e0 con 0x7f90a4069bf0 2023-11-08T08:20:14.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.818+0000 7f90ab158700 1 -- 172.21.15.40:0/3237440459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f90a4073d20 con 0x7f90a4069bf0 2023-11-08T08:20:14.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.822+0000 7f909affd700 1 -- 172.21.15.40:0/3237440459 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f90a0052800 con 0x7f90a4069bf0 2023-11-08T08:20:14.825 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:20:14.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.826+0000 7f90ab158700 1 -- 172.21.15.40:0/3237440459 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9088064ee0 msgr2=0x7f9088067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:14.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.826+0000 7f90ab158700 1 --2- 172.21.15.40:0/3237440459 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9088064ee0 0x7f9088067390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f90a4070a10 tx=0x7f9094008070 comp rx=0 tx=0).stop 2023-11-08T08:20:14.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.826+0000 7f90ab158700 1 -- 172.21.15.40:0/3237440459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90a4069bf0 msgr2=0x7f90a40cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:14.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.826+0000 7f90ab158700 1 --2- 172.21.15.40:0/3237440459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90a4069bf0 0x7f90a40cf980 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f90a000c500 tx=0x7f90a000c810 comp rx=0 tx=0).stop 2023-11-08T08:20:14.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.826+0000 7f90ab158700 1 -- 172.21.15.40:0/3237440459 shutdown_connections 2023-11-08T08:20:14.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.826+0000 7f90ab158700 1 --2- 172.21.15.40:0/3237440459 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9088064ee0 0x7f9088067390 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.826+0000 7f90ab158700 1 --2- 172.21.15.40:0/3237440459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90a415c150 0x7f90a40c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.826+0000 7f90ab158700 1 --2- 172.21.15.40:0/3237440459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90a4070620 0x7f90a40cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.826+0000 7f90ab158700 1 --2- 172.21.15.40:0/3237440459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90a4069bf0 0x7f90a40cf980 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:14.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.826+0000 7f90ab158700 1 -- 172.21.15.40:0/3237440459 >> 172.21.15.40:0/3237440459 conn(0x7f90a40d3680 msgr2=0x7f90a406ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:14.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.826+0000 7f90ab158700 1 -- 172.21.15.40:0/3237440459 shutdown_connections 2023-11-08T08:20:14.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:14.826+0000 7f90ab158700 1 -- 172.21.15.40:0/3237440459 wait complete. 2023-11-08T08:20:14.852 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T08:20:15.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.030+0000 7fbd160cb700 1 Processor -- start 2023-11-08T08:20:15.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.030+0000 7fbd160cb700 1 -- start start 2023-11-08T08:20:15.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.034+0000 7fbd160cb700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbd1015c1a0 0x7fbd1006c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:15.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.034+0000 7fbd160cb700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd100adf90 con 0x7fbd100cdeb0 2023-11-08T08:20:15.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.034+0000 7fbd160cb700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd100ce370 con 0x7fbd10070790 2023-11-08T08:20:15.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.034+0000 7fbd160cb700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd10069bf0 con 0x7fbd1015c1a0 2023-11-08T08:20:15.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.034+0000 7fbd148c8700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbd100cdeb0 0x7fbd10070080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59456/0 (socket says 172.21.15.40:59456) 2023-11-08T08:20:15.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.034+0000 7fbd148c8700 1 -- 172.21.15.40:0/2103917828 learned_addr learned my addr 172.21.15.40:0/2103917828 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:15.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.034+0000 7fbd150c9700 1 --2- 172.21.15.40:0/2103917828 >> v2:172.21.15.40:3300/0 conn(0x7fbd1015c1a0 0x7fbd1006c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:15.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.034+0000 7fbd07fff700 1 -- 172.21.15.40:0/2103917828 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648420249 0 0) 0x7fbd100adf90 con 0x7fbd100cdeb0 2023-11-08T08:20:15.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.034+0000 7fbd07fff700 1 -- 172.21.15.40:0/2103917828 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcfc0035d0 con 0x7fbd100cdeb0 2023-11-08T08:20:15.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.034+0000 7fbd150c9700 1 -- 172.21.15.40:0/2103917828 >> v1:172.21.15.169:6789/0 conn(0x7fbd10070790 legacy=0x7fbd1006a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:15.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.034+0000 7fbd150c9700 1 -- 172.21.15.40:0/2103917828 >> v1:172.21.15.40:6789/0 conn(0x7fbd100cdeb0 legacy=0x7fbd10070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:15.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.034+0000 7fbd07fff700 1 -- 172.21.15.40:0/2103917828 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350765136 0 0) 0x7fbd100ce370 con 0x7fbd10070790 2023-11-08T08:20:15.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.034+0000 7fbd150c9700 1 -- 172.21.15.40:0/2103917828 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd100cc4a0 con 0x7fbd1015c1a0 2023-11-08T08:20:15.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.034+0000 7fbd150c9700 1 --2- 172.21.15.40:0/2103917828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd1015c1a0 0x7fbd1006c8b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009850 tx=0x7fbd0c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57db6b8e2d786369 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:15.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.034+0000 7fbd07fff700 1 -- 172.21.15.40:0/2103917828 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c011830 con 0x7fbd1015c1a0 2023-11-08T08:20:15.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.038+0000 7fbd07fff700 1 -- 172.21.15.40:0/2103917828 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0c011990 con 0x7fbd1015c1a0 2023-11-08T08:20:15.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.038+0000 7fbd07fff700 1 -- 172.21.15.40:0/2103917828 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c011d40 con 0x7fbd1015c1a0 2023-11-08T08:20:15.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.038+0000 7fbd160cb700 1 -- 172.21.15.40:0/2103917828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd1015c1a0 msgr2=0x7fbd1006c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:15.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.038+0000 7fbd160cb700 1 --2- 172.21.15.40:0/2103917828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd1015c1a0 0x7fbd1006c8b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009850 tx=0x7fbd0c009b60 comp rx=0 tx=0).stop 2023-11-08T08:20:15.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.038+0000 7fbd160cb700 1 -- 172.21.15.40:0/2103917828 shutdown_connections 2023-11-08T08:20:15.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.038+0000 7fbd160cb700 1 --2- 172.21.15.40:0/2103917828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd1015c1a0 0x7fbd1006c8b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:15.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.038+0000 7fbd160cb700 1 -- 172.21.15.40:0/2103917828 >> 172.21.15.40:0/2103917828 conn(0x7fbd100b0f80 msgr2=0x7fbd100ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:15.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.038+0000 7fbd160cb700 1 -- 172.21.15.40:0/2103917828 shutdown_connections 2023-11-08T08:20:15.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.038+0000 7fbd160cb700 1 -- 172.21.15.40:0/2103917828 wait complete. 2023-11-08T08:20:15.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.038+0000 7fbd160cb700 1 Processor -- start 2023-11-08T08:20:15.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.042+0000 7fbd160cb700 1 -- start start 2023-11-08T08:20:15.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.042+0000 7fbd160cb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd10070790 0x7fbd100c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:15.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.042+0000 7fbd160cb700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd100cdeb0 0x7fbd100b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:15.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.042+0000 7fbd160cb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd1015c1a0 0x7fbd100ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:15.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.042+0000 7fbd160cb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd100c87e0 con 0x7fbd1015c1a0 2023-11-08T08:20:15.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.042+0000 7fbd160cb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd100b1b80 con 0x7fbd10070790 2023-11-08T08:20:15.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.042+0000 7fbd160cb700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1015cbc0 con 0x7fbd100cdeb0 2023-11-08T08:20:15.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.042+0000 7fbd158ca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd1015c1a0 0x7fbd100ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.042+0000 7fbd148c8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd100cdeb0 0x7fbd100b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.042+0000 7fbd158ca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd1015c1a0 0x7fbd100ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40266/0 (socket says 172.21.15.40:40266) 2023-11-08T08:20:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.042+0000 7fbd158ca700 1 -- 172.21.15.40:0/4265237438 learned_addr learned my addr 172.21.15.40:0/4265237438 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:15.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.042+0000 7fbd150c9700 1 --2- 172.21.15.40:0/4265237438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd10070790 0x7fbd100c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:15.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.042+0000 7fbd158ca700 1 -- 172.21.15.40:0/4265237438 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd100cdeb0 msgr2=0x7fbd100b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:15.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.042+0000 7fbd158ca700 1 --2- 172.21.15.40:0/4265237438 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd100cdeb0 0x7fbd100b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:15.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.042+0000 7fbd158ca700 1 -- 172.21.15.40:0/4265237438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd10070790 msgr2=0x7fbd100c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:15.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.042+0000 7fbd158ca700 1 --2- 172.21.15.40:0/4265237438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd10070790 0x7fbd100c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:15.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.042+0000 7fbd158ca700 1 -- 172.21.15.40:0/4265237438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c0094b0 con 0x7fbd1015c1a0 2023-11-08T08:20:15.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.042+0000 7fbd158ca700 1 --2- 172.21.15.40:0/4265237438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd1015c1a0 0x7fbd100ac8b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800cc70 tx=0x7fbd0800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:15.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.046+0000 7fbd05ffb700 1 -- 172.21.15.40:0/4265237438 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd08003e50 con 0x7fbd1015c1a0 2023-11-08T08:20:15.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.046+0000 7fbd05ffb700 1 -- 172.21.15.40:0/4265237438 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0800a6b0 con 0x7fbd1015c1a0 2023-11-08T08:20:15.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.046+0000 7fbd160cb700 1 -- 172.21.15.40:0/4265237438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd1006cdf0 con 0x7fbd1015c1a0 2023-11-08T08:20:15.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.046+0000 7fbd05ffb700 1 -- 172.21.15.40:0/4265237438 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0800aa60 con 0x7fbd1015c1a0 2023-11-08T08:20:15.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.046+0000 7fbd160cb700 1 -- 172.21.15.40:0/4265237438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd100cc780 con 0x7fbd1015c1a0 2023-11-08T08:20:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.046+0000 7fbd160cb700 1 -- 172.21.15.40:0/4265237438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd1015c570 con 0x7fbd1015c1a0 2023-11-08T08:20:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.046+0000 7fbd05ffb700 1 -- 172.21.15.40:0/4265237438 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fbd08019030 con 0x7fbd1015c1a0 2023-11-08T08:20:15.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.050+0000 7fbd05ffb700 1 --2- 172.21.15.40:0/4265237438 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbcf4064ee0 0x7fbcf4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:15.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.050+0000 7fbd05ffb700 1 -- 172.21.15.40:0/4265237438 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbd08081bd0 con 0x7fbd1015c1a0 2023-11-08T08:20:15.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.050+0000 7fbd150c9700 1 --2- 172.21.15.40:0/4265237438 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbcf4064ee0 0x7fbcf4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:15.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.050+0000 7fbd150c9700 1 --2- 172.21.15.40:0/4265237438 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbcf4064ee0 0x7fbcf4067390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0090b0 tx=0x7fbd0c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:15.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.058+0000 7fbd05ffb700 1 -- 172.21.15.40:0/4265237438 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fbd0804f950 con 0x7fbd1015c1a0 2023-11-08T08:20:15.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.250+0000 7fbd160cb700 1 -- 172.21.15.40:0/4265237438 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbd10073cb0 con 0x7fbd1015c1a0 2023-11-08T08:20:15.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.250+0000 7fbd05ffb700 1 -- 172.21.15.40:0/4265237438 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbd08017080 con 0x7fbd1015c1a0 2023-11-08T08:20:15.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.254+0000 7fbd160cb700 1 -- 172.21.15.40:0/4265237438 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbcf4064ee0 msgr2=0x7fbcf4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:15.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.254+0000 7fbd160cb700 1 --2- 172.21.15.40:0/4265237438 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbcf4064ee0 0x7fbcf4067390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0090b0 tx=0x7fbd0c01c040 comp rx=0 tx=0).stop 2023-11-08T08:20:15.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.254+0000 7fbd160cb700 1 -- 172.21.15.40:0/4265237438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd1015c1a0 msgr2=0x7fbd100ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:15.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.254+0000 7fbd160cb700 1 --2- 172.21.15.40:0/4265237438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd1015c1a0 0x7fbd100ac8b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800cc70 tx=0x7fbd0800cf80 comp rx=0 tx=0).stop 2023-11-08T08:20:15.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.254+0000 7fbd160cb700 1 -- 172.21.15.40:0/4265237438 shutdown_connections 2023-11-08T08:20:15.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.254+0000 7fbd160cb700 1 --2- 172.21.15.40:0/4265237438 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbcf4064ee0 0x7fbcf4067390 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:15.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.254+0000 7fbd160cb700 1 --2- 172.21.15.40:0/4265237438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbd1015c1a0 0x7fbd100ac8b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:15.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.254+0000 7fbd160cb700 1 --2- 172.21.15.40:0/4265237438 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd100cdeb0 0x7fbd100b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.254+0000 7fbd160cb700 1 --2- 172.21.15.40:0/4265237438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd10070790 0x7fbd100c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.254+0000 7fbd160cb700 1 -- 172.21.15.40:0/4265237438 >> 172.21.15.40:0/4265237438 conn(0x7fbd100b0f80 msgr2=0x7fbd1015fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.254+0000 7fbd160cb700 1 -- 172.21.15.40:0/4265237438 shutdown_connections 2023-11-08T08:20:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.254+0000 7fbd160cb700 1 -- 172.21.15.40:0/4265237438 wait complete. 2023-11-08T08:20:15.261 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-11-08T08:20:15.280 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-08T08:20:15.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:15.281 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-08T08:20:15.325 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2023-11-08T08:20:15.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:15.325 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-08T08:20:15.387 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-08T08:20:15.388 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:20:15.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.562+0000 7faf62e82700 1 Processor -- start 2023-11-08T08:20:15.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.562+0000 7faf62e82700 1 -- start start 2023-11-08T08:20:15.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.562+0000 7faf62e82700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf5c15d510 0x7faf5c0b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:15.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.562+0000 7faf62e82700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faf5c0b6030 0x7faf5c070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:15.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.562+0000 7faf62e82700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5c15d080 con 0x7faf5c1534a0 2023-11-08T08:20:15.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.562+0000 7faf62e82700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5c15d940 con 0x7faf5c0b6030 2023-11-08T08:20:15.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.562+0000 7faf62e82700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c153870 con 0x7faf5c15d510 2023-11-08T08:20:15.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.562+0000 7faf60c1e700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7faf5c1534a0 0x7faf5c160920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59462/0 (socket says 172.21.15.40:59462) 2023-11-08T08:20:15.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.566+0000 7faf60c1e700 1 -- 172.21.15.40:0/3022466321 learned_addr learned my addr 172.21.15.40:0/3022466321 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:15.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.566+0000 7faf6141f700 1 --2- 172.21.15.40:0/3022466321 >> v2:172.21.15.40:3300/0 conn(0x7faf5c0b6030 0x7faf5c070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:15.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.566+0000 7faf5bfff700 1 --2- 172.21.15.40:0/3022466321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf5c15d510 0x7faf5c0b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:15.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.566+0000 7faf5b7fe700 1 -- 172.21.15.40:0/3022466321 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3676978040 0 0) 0x7faf5c15d080 con 0x7faf5c1534a0 2023-11-08T08:20:15.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.566+0000 7faf5b7fe700 1 -- 172.21.15.40:0/3022466321 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf4c0035d0 con 0x7faf5c1534a0 2023-11-08T08:20:15.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.566+0000 7faf5bfff700 1 -- 172.21.15.40:0/3022466321 >> v2:172.21.15.40:3300/0 conn(0x7faf5c0b6030 msgr2=0x7faf5c070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:15.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.566+0000 7faf5bfff700 1 --2- 172.21.15.40:0/3022466321 >> v2:172.21.15.40:3300/0 conn(0x7faf5c0b6030 0x7faf5c070630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:15.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.566+0000 7faf5bfff700 1 -- 172.21.15.40:0/3022466321 >> v1:172.21.15.40:6789/0 conn(0x7faf5c1534a0 legacy=0x7faf5c160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:15.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.566+0000 7faf5bfff700 1 -- 172.21.15.40:0/3022466321 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5c0b6450 con 0x7faf5c15d510 2023-11-08T08:20:15.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.566+0000 7faf6141f700 1 --2- 172.21.15.40:0/3022466321 >> v2:172.21.15.40:3300/0 conn(0x7faf5c0b6030 0x7faf5c070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:15.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.566+0000 7faf5bfff700 1 --2- 172.21.15.40:0/3022466321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf5c15d510 0x7faf5c0b5af0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faf48009850 tx=0x7faf48009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4b68af105e032cf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:15.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.566+0000 7faf5b7fe700 1 -- 172.21.15.40:0/3022466321 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48011830 con 0x7faf5c15d510 2023-11-08T08:20:15.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.566+0000 7faf5b7fe700 1 -- 172.21.15.40:0/3022466321 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf48011990 con 0x7faf5c15d510 2023-11-08T08:20:15.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.566+0000 7faf5b7fe700 1 -- 172.21.15.40:0/3022466321 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48011d40 con 0x7faf5c15d510 2023-11-08T08:20:15.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.566+0000 7faf62e82700 1 -- 172.21.15.40:0/3022466321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf5c15d510 msgr2=0x7faf5c0b5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:15.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.566+0000 7faf62e82700 1 --2- 172.21.15.40:0/3022466321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf5c15d510 0x7faf5c0b5af0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faf48009850 tx=0x7faf48009b60 comp rx=0 tx=0).stop 2023-11-08T08:20:15.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.566+0000 7faf62e82700 1 -- 172.21.15.40:0/3022466321 shutdown_connections 2023-11-08T08:20:15.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.566+0000 7faf62e82700 1 --2- 172.21.15.40:0/3022466321 >> v2:172.21.15.40:3300/0 conn(0x7faf5c0b6030 0x7faf5c070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:15.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.566+0000 7faf62e82700 1 --2- 172.21.15.40:0/3022466321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf5c15d510 0x7faf5c0b5af0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:15.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.570+0000 7faf62e82700 1 -- 172.21.15.40:0/3022466321 >> 172.21.15.40:0/3022466321 conn(0x7faf5c06bfc0 msgr2=0x7faf5c06c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:15.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.570+0000 7faf62e82700 1 -- 172.21.15.40:0/3022466321 shutdown_connections 2023-11-08T08:20:15.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.570+0000 7faf62e82700 1 -- 172.21.15.40:0/3022466321 wait complete. 2023-11-08T08:20:15.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf62e82700 1 Processor -- start 2023-11-08T08:20:15.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf62e82700 1 -- start start 2023-11-08T08:20:15.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf62e82700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf5c0b6030 0x7faf5c1f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:15.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf62e82700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf5c1534a0 0x7faf5c158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:15.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf62e82700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf5c15d510 0x7faf5c1579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:15.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf62e82700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c1f21e0 con 0x7faf5c1534a0 2023-11-08T08:20:15.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf62e82700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c153bc0 con 0x7faf5c15d510 2023-11-08T08:20:15.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf62e82700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c153ea0 con 0x7faf5c0b6030 2023-11-08T08:20:15.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf5bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf5c1534a0 0x7faf5c158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:15.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf5bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf5c1534a0 0x7faf5c158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40284/0 (socket says 172.21.15.40:40284) 2023-11-08T08:20:15.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf5bfff700 1 -- 172.21.15.40:0/1603716095 learned_addr learned my addr 172.21.15.40:0/1603716095 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:15.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf60c1e700 1 --2- 172.21.15.40:0/1603716095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf5c0b6030 0x7faf5c1f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:15.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf6141f700 1 --2- 172.21.15.40:0/1603716095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf5c15d510 0x7faf5c1579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:15.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf5bfff700 1 -- 172.21.15.40:0/1603716095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf5c0b6030 msgr2=0x7faf5c1f1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:15.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf5bfff700 1 --2- 172.21.15.40:0/1603716095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf5c0b6030 0x7faf5c1f1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:15.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf5bfff700 1 -- 172.21.15.40:0/1603716095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf5c15d510 msgr2=0x7faf5c1579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:15.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf5bfff700 1 --2- 172.21.15.40:0/1603716095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf5c15d510 0x7faf5c1579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:15.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf5bfff700 1 -- 172.21.15.40:0/1603716095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf54009a70 con 0x7faf5c1534a0 2023-11-08T08:20:15.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf6141f700 1 --2- 172.21.15.40:0/1603716095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf5c15d510 0x7faf5c1579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:15.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf5bfff700 1 --2- 172.21.15.40:0/1603716095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf5c1534a0 0x7faf5c158e00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7faf480090b0 tx=0x7faf48004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:15.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.574+0000 7faf597fa700 1 -- 172.21.15.40:0/1603716095 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48004520 con 0x7faf5c1534a0 2023-11-08T08:20:15.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.578+0000 7faf597fa700 1 -- 172.21.15.40:0/1603716095 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4801f070 con 0x7faf5c1534a0 2023-11-08T08:20:15.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.578+0000 7faf597fa700 1 -- 172.21.15.40:0/1603716095 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48003900 con 0x7faf5c1534a0 2023-11-08T08:20:15.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.578+0000 7faf62e82700 1 -- 172.21.15.40:0/1603716095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf480094b0 con 0x7faf5c1534a0 2023-11-08T08:20:15.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.578+0000 7faf62e82700 1 -- 172.21.15.40:0/1603716095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5c0b56a0 con 0x7faf5c1534a0 2023-11-08T08:20:15.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.578+0000 7faf62e82700 1 -- 172.21.15.40:0/1603716095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5c0b53c0 con 0x7faf5c1534a0 2023-11-08T08:20:15.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.578+0000 7faf597fa700 1 -- 172.21.15.40:0/1603716095 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7faf4800f050 con 0x7faf5c1534a0 2023-11-08T08:20:15.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.582+0000 7faf597fa700 1 --2- 172.21.15.40:0/1603716095 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7faf40064ee0 0x7faf40067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:15.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.582+0000 7faf597fa700 1 -- 172.21.15.40:0/1603716095 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7faf48021070 con 0x7faf5c1534a0 2023-11-08T08:20:15.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.582+0000 7faf60c1e700 1 --2- 172.21.15.40:0/1603716095 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7faf40064ee0 0x7faf40067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:15.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.586+0000 7faf60c1e700 1 --2- 172.21.15.40:0/1603716095 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7faf40064ee0 0x7faf40067390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faf50007fd0 tx=0x7faf50007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:15.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.590+0000 7faf597fa700 1 -- 172.21.15.40:0/1603716095 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7faf480514d0 con 0x7faf5c1534a0 2023-11-08T08:20:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.802+0000 7faf62e82700 1 -- 172.21.15.40:0/1603716095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf5c15ca30 con 0x7faf5c1534a0 2023-11-08T08:20:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.802+0000 7faf597fa700 1 -- 172.21.15.40:0/1603716095 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4641 (secure 0 0 0) 0x7faf48015090 con 0x7faf5c1534a0 2023-11-08T08:20:15.806 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:15.807 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/196211087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":196211087},{"type":"v1","addr":"172.21.15.40:6834","nonce":196211087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/67127106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":67127106},{"type":"v1","addr":"172.21.15.40:6835","nonce":67127106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1231240563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1231240563},{"type":"v1","addr":"172.21.15.169:6835","nonce":1231240563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:19:04.456508+0000","modified":"2023-11-08T08:19:31.023792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6837/1989495204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1989495204},{"type":"v1","addr":"172.21.15.169:6837","nonce":1989495204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T08:20:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.806+0000 7faf62e82700 1 -- 172.21.15.40:0/1603716095 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7faf40064ee0 msgr2=0x7faf40067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.806+0000 7faf62e82700 1 --2- 172.21.15.40:0/1603716095 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7faf40064ee0 0x7faf40067390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faf50007fd0 tx=0x7faf50007de0 comp rx=0 tx=0).stop 2023-11-08T08:20:15.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.806+0000 7faf62e82700 1 -- 172.21.15.40:0/1603716095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf5c1534a0 msgr2=0x7faf5c158e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:15.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.806+0000 7faf62e82700 1 --2- 172.21.15.40:0/1603716095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf5c1534a0 0x7faf5c158e00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7faf480090b0 tx=0x7faf48004770 comp rx=0 tx=0).stop 2023-11-08T08:20:15.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.806+0000 7faf62e82700 1 -- 172.21.15.40:0/1603716095 shutdown_connections 2023-11-08T08:20:15.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.806+0000 7faf62e82700 1 --2- 172.21.15.40:0/1603716095 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7faf40064ee0 0x7faf40067390 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:15.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.806+0000 7faf62e82700 1 --2- 172.21.15.40:0/1603716095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf5c15d510 0x7faf5c1579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:15.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.806+0000 7faf62e82700 1 --2- 172.21.15.40:0/1603716095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf5c1534a0 0x7faf5c158e00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:15.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.806+0000 7faf62e82700 1 --2- 172.21.15.40:0/1603716095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf5c0b6030 0x7faf5c1f1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.806+0000 7faf62e82700 1 -- 172.21.15.40:0/1603716095 >> 172.21.15.40:0/1603716095 conn(0x7faf5c06bfc0 msgr2=0x7faf5c06a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.806+0000 7faf62e82700 1 -- 172.21.15.40:0/1603716095 shutdown_connections 2023-11-08T08:20:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:15.806+0000 7faf62e82700 1 -- 172.21.15.40:0/1603716095 wait complete. 2023-11-08T08:20:15.811 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 9 2023-11-08T08:20:15.832 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:20:16.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.006+0000 7f769f298700 1 Processor -- start 2023-11-08T08:20:16.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.006+0000 7f769f298700 1 -- start start 2023-11-08T08:20:16.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.006+0000 7f769f298700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7698071aa0 0x7f769806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:16.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.006+0000 7f769f298700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7698069e00 0x7f769806a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:16.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.006+0000 7f769f298700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f769806c600 con 0x7f769815c640 2023-11-08T08:20:16.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.006+0000 7f769f298700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769806cef0 con 0x7f7698071aa0 2023-11-08T08:20:16.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.006+0000 7f769f298700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76980d38f0 con 0x7f7698069e00 2023-11-08T08:20:16.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.006+0000 7f769da95700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7698071aa0 0x7f769806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:16.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.006+0000 7f769e296700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f769815c640 0x7f76980d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59470/0 (socket says 172.21.15.40:59470) 2023-11-08T08:20:16.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.006+0000 7f769e296700 1 -- 172.21.15.40:0/2979315986 learned_addr learned my addr 172.21.15.40:0/2979315986 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:16.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.006+0000 7f769ea97700 1 --2- 172.21.15.40:0/2979315986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7698069e00 0x7f769806a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:16.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.006+0000 7f769da95700 1 -- 172.21.15.40:0/2979315986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7698069e00 msgr2=0x7f769806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:16.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.006+0000 7f769da95700 1 --2- 172.21.15.40:0/2979315986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7698069e00 0x7f769806a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.006+0000 7f769da95700 1 -- 172.21.15.40:0/2979315986 >> v1:172.21.15.40:6789/0 conn(0x7f769815c640 legacy=0x7f76980d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:16.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.006+0000 7f769da95700 1 -- 172.21.15.40:0/2979315986 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76980722f0 con 0x7f7698071aa0 2023-11-08T08:20:16.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.006+0000 7f769d294700 1 -- 172.21.15.40:0/2979315986 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1318628760 0 0) 0x7f769806c600 con 0x7f769815c640 2023-11-08T08:20:16.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.006+0000 7f769ea97700 1 --2- 172.21.15.40:0/2979315986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7698069e00 0x7f769806a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:16.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.010+0000 7f769da95700 1 --2- 172.21.15.40:0/2979315986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7698071aa0 0x7f769806d310 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7688009850 tx=0x7f7688009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1238ff946f7c20c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:16.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.010+0000 7f769d294700 1 -- 172.21.15.40:0/2979315986 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7688011830 con 0x7f7698071aa0 2023-11-08T08:20:16.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.010+0000 7f769d294700 1 -- 172.21.15.40:0/2979315986 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7688011990 con 0x7f7698071aa0 2023-11-08T08:20:16.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.010+0000 7f769d294700 1 -- 172.21.15.40:0/2979315986 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7688011d20 con 0x7f7698071aa0 2023-11-08T08:20:16.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.010+0000 7f769f298700 1 -- 172.21.15.40:0/2979315986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7698071aa0 msgr2=0x7f769806d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:16.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.010+0000 7f769f298700 1 --2- 172.21.15.40:0/2979315986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7698071aa0 0x7f769806d310 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7688009850 tx=0x7f7688009b60 comp rx=0 tx=0).stop 2023-11-08T08:20:16.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.010+0000 7f769f298700 1 -- 172.21.15.40:0/2979315986 shutdown_connections 2023-11-08T08:20:16.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.010+0000 7f769f298700 1 --2- 172.21.15.40:0/2979315986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7698069e00 0x7f769806a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.010+0000 7f769f298700 1 --2- 172.21.15.40:0/2979315986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7698071aa0 0x7f769806d310 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.010+0000 7f769f298700 1 -- 172.21.15.40:0/2979315986 >> 172.21.15.40:0/2979315986 conn(0x7f76980b6520 msgr2=0x7f7698071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:16.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.010+0000 7f769f298700 1 -- 172.21.15.40:0/2979315986 shutdown_connections 2023-11-08T08:20:16.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.010+0000 7f769f298700 1 -- 172.21.15.40:0/2979315986 wait complete. 2023-11-08T08:20:16.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.014+0000 7f769f298700 1 Processor -- start 2023-11-08T08:20:16.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.014+0000 7f769f298700 1 -- start start 2023-11-08T08:20:16.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.014+0000 7f769f298700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7698069e00 0x7f769809bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:16.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.014+0000 7f769f298700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7698071aa0 0x7f769815cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:16.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.014+0000 7f769f298700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f769815c640 0x7f769806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:16.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.014+0000 7f769f298700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769809c2f0 con 0x7f7698071aa0 2023-11-08T08:20:16.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.014+0000 7f769f298700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7698148e50 con 0x7f769815c640 2023-11-08T08:20:16.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.014+0000 7f769f298700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76980c64e0 con 0x7f7698069e00 2023-11-08T08:20:16.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.018+0000 7f769da95700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7698071aa0 0x7f769815cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:16.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.018+0000 7f769da95700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7698071aa0 0x7f769815cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40300/0 (socket says 172.21.15.40:40300) 2023-11-08T08:20:16.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.018+0000 7f769da95700 1 -- 172.21.15.40:0/2171390432 learned_addr learned my addr 172.21.15.40:0/2171390432 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:16.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.018+0000 7f769e296700 1 --2- 172.21.15.40:0/2171390432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7698069e00 0x7f769809bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:16.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.018+0000 7f769ea97700 1 --2- 172.21.15.40:0/2171390432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f769815c640 0x7f769806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:16.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.018+0000 7f769da95700 1 -- 172.21.15.40:0/2171390432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7698069e00 msgr2=0x7f769809bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:16.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.018+0000 7f769da95700 1 --2- 172.21.15.40:0/2171390432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7698069e00 0x7f769809bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.018+0000 7f769da95700 1 -- 172.21.15.40:0/2171390432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f769815c640 msgr2=0x7f769806d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:16.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.018+0000 7f769da95700 1 --2- 172.21.15.40:0/2171390432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f769815c640 0x7f769806d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.018+0000 7f769da95700 1 -- 172.21.15.40:0/2171390432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76880094b0 con 0x7f7698071aa0 2023-11-08T08:20:16.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.018+0000 7f769da95700 1 --2- 172.21.15.40:0/2171390432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7698071aa0 0x7f769815cba0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7688000c00 tx=0x7f7688004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:16.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.018+0000 7f768effd700 1 -- 172.21.15.40:0/2171390432 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7688004510 con 0x7f7698071aa0 2023-11-08T08:20:16.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.018+0000 7f768effd700 1 -- 172.21.15.40:0/2171390432 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76880036c0 con 0x7f7698071aa0 2023-11-08T08:20:16.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.018+0000 7f768effd700 1 -- 172.21.15.40:0/2171390432 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7688003a70 con 0x7f7698071aa0 2023-11-08T08:20:16.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.018+0000 7f769f298700 1 -- 172.21.15.40:0/2171390432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7698072010 con 0x7f7698071aa0 2023-11-08T08:20:16.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.018+0000 7f769f298700 1 -- 172.21.15.40:0/2171390432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769815ca10 con 0x7f7698071aa0 2023-11-08T08:20:16.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.022+0000 7f769f298700 1 -- 172.21.15.40:0/2171390432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f769806c830 con 0x7f7698071aa0 2023-11-08T08:20:16.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.022+0000 7f768effd700 1 -- 172.21.15.40:0/2171390432 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f768802d070 con 0x7f7698071aa0 2023-11-08T08:20:16.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.022+0000 7f768effd700 1 --2- 172.21.15.40:0/2171390432 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f767c064ee0 0x7f767c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:16.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.022+0000 7f768effd700 1 -- 172.21.15.40:0/2171390432 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7688021030 con 0x7f7698071aa0 2023-11-08T08:20:16.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.022+0000 7f769e296700 1 --2- 172.21.15.40:0/2171390432 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f767c064ee0 0x7f767c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:16.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.026+0000 7f769e296700 1 --2- 172.21.15.40:0/2171390432 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f767c064ee0 0x7f767c067390 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7694000f60 tx=0x7f7694007e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:16.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.030+0000 7f768effd700 1 -- 172.21.15.40:0/2171390432 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7688051570 con 0x7f7698071aa0 2023-11-08T08:20:16.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.214+0000 7f769f298700 1 -- 172.21.15.40:0/2171390432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7698073cb0 con 0x7f7698071aa0 2023-11-08T08:20:16.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.218+0000 7f768effd700 1 -- 172.21.15.40:0/2171390432 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19748 (secure 0 0 0) 0x7f7688015090 con 0x7f7698071aa0 2023-11-08T08:20:16.222 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:16.222 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":30,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:45.885827+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:20:16.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.222+0000 7f769f298700 1 -- 172.21.15.40:0/2171390432 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f767c064ee0 msgr2=0x7f767c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:16.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.222+0000 7f769f298700 1 --2- 172.21.15.40:0/2171390432 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f767c064ee0 0x7f767c067390 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7694000f60 tx=0x7f7694007e00 comp rx=0 tx=0).stop 2023-11-08T08:20:16.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.222+0000 7f769f298700 1 -- 172.21.15.40:0/2171390432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7698071aa0 msgr2=0x7f769815cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:16.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.222+0000 7f769f298700 1 --2- 172.21.15.40:0/2171390432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7698071aa0 0x7f769815cba0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7688000c00 tx=0x7f7688004770 comp rx=0 tx=0).stop 2023-11-08T08:20:16.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.222+0000 7f769f298700 1 -- 172.21.15.40:0/2171390432 shutdown_connections 2023-11-08T08:20:16.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.222+0000 7f769f298700 1 --2- 172.21.15.40:0/2171390432 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f767c064ee0 0x7f767c067390 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.222+0000 7f769f298700 1 --2- 172.21.15.40:0/2171390432 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f769815c640 0x7f769806d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.222+0000 7f769f298700 1 --2- 172.21.15.40:0/2171390432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7698071aa0 0x7f769815cba0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.222+0000 7f769f298700 1 --2- 172.21.15.40:0/2171390432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7698069e00 0x7f769809bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.222+0000 7f769f298700 1 -- 172.21.15.40:0/2171390432 >> 172.21.15.40:0/2171390432 conn(0x7f76980b6520 msgr2=0x7f769815b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:16.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.222+0000 7f769f298700 1 -- 172.21.15.40:0/2171390432 shutdown_connections 2023-11-08T08:20:16.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.222+0000 7f769f298700 1 -- 172.21.15.40:0/2171390432 wait complete. 2023-11-08T08:20:16.252 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T08:19:02.627691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T08:19:03.473821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T08:19:06.539942+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T08:20:16.254 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:20:16.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.438+0000 7f62aadbc700 1 Processor -- start 2023-11-08T08:20:16.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.438+0000 7f62aadbc700 1 -- start start 2023-11-08T08:20:16.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.442+0000 7f62aadbc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f62a415c310 0x7f62a40d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:16.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.442+0000 7f62aadbc700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f62a406b830 0x7f62a40700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.442+0000 7f62aadbc700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a40cd080 con 0x7f62a4070600 2023-11-08T08:20:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.442+0000 7f62aadbc700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a406d240 con 0x7f62a415c310 2023-11-08T08:20:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.442+0000 7f62aadbc700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a40d21a0 con 0x7f62a406b830 2023-11-08T08:20:16.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.442+0000 7f62a9dba700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f62a415c310 0x7f62a40d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:16.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.442+0000 7f62aa5bb700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f62a4070600 0x7f62a415fdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59482/0 (socket says 172.21.15.40:59482) 2023-11-08T08:20:16.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.442+0000 7f62aa5bb700 1 -- 172.21.15.40:0/1981349431 learned_addr learned my addr 172.21.15.40:0/1981349431 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:16.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.442+0000 7f62a95b9700 1 --2- 172.21.15.40:0/1981349431 >> v2:172.21.15.169:3300/0 conn(0x7f62a406b830 0x7f62a40700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.442+0000 7f62a9dba700 1 -- 172.21.15.40:0/1981349431 >> v2:172.21.15.169:3300/0 conn(0x7f62a406b830 msgr2=0x7f62a40700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.442+0000 7f62a9dba700 1 --2- 172.21.15.40:0/1981349431 >> v2:172.21.15.169:3300/0 conn(0x7f62a406b830 0x7f62a40700c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.442+0000 7f62a9dba700 1 -- 172.21.15.40:0/1981349431 >> v1:172.21.15.40:6789/0 conn(0x7f62a4070600 legacy=0x7f62a415fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.442+0000 7f62a9dba700 1 -- 172.21.15.40:0/1981349431 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a4070a60 con 0x7f62a415c310 2023-11-08T08:20:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.442+0000 7f62a8db8700 1 -- 172.21.15.40:0/1981349431 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009021449 0 0) 0x7f62a40cd080 con 0x7f62a4070600 2023-11-08T08:20:16.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.442+0000 7f62a9dba700 1 --2- 172.21.15.40:0/1981349431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62a415c310 0x7f62a40d1310 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f62a40b5620 tx=0x7f62a0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d475d4e5b97d79 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:16.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.442+0000 7f62a8db8700 1 -- 172.21.15.40:0/1981349431 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a0011830 con 0x7f62a415c310 2023-11-08T08:20:16.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.442+0000 7f62a8db8700 1 -- 172.21.15.40:0/1981349431 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a0011990 con 0x7f62a415c310 2023-11-08T08:20:16.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.442+0000 7f62a8db8700 1 -- 172.21.15.40:0/1981349431 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a0011d20 con 0x7f62a415c310 2023-11-08T08:20:16.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.446+0000 7f62aadbc700 1 -- 172.21.15.40:0/1981349431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62a415c310 msgr2=0x7f62a40d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:16.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.446+0000 7f62aadbc700 1 --2- 172.21.15.40:0/1981349431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62a415c310 0x7f62a40d1310 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f62a40b5620 tx=0x7f62a0009b30 comp rx=0 tx=0).stop 2023-11-08T08:20:16.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.446+0000 7f62aadbc700 1 -- 172.21.15.40:0/1981349431 shutdown_connections 2023-11-08T08:20:16.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.446+0000 7f62aadbc700 1 --2- 172.21.15.40:0/1981349431 >> v2:172.21.15.169:3300/0 conn(0x7f62a406b830 0x7f62a40700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.446+0000 7f62aadbc700 1 --2- 172.21.15.40:0/1981349431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62a415c310 0x7f62a40d1310 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.446+0000 7f62aadbc700 1 -- 172.21.15.40:0/1981349431 >> 172.21.15.40:0/1981349431 conn(0x7f62a40d0140 msgr2=0x7f62a40d2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:16.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.446+0000 7f62aadbc700 1 -- 172.21.15.40:0/1981349431 shutdown_connections 2023-11-08T08:20:16.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.446+0000 7f62aadbc700 1 -- 172.21.15.40:0/1981349431 wait complete. 2023-11-08T08:20:16.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.450+0000 7f62aadbc700 1 Processor -- start 2023-11-08T08:20:16.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.450+0000 7f62aadbc700 1 -- start start 2023-11-08T08:20:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.450+0000 7f62aadbc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62a406b830 0x7f62a40b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.450+0000 7f62aadbc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62a4070600 0x7f62a415b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.450+0000 7f62aadbc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62a415c310 0x7f62a41f1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.450+0000 7f62aadbc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a41585f0 con 0x7f62a406b830 2023-11-08T08:20:16.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.450+0000 7f62aadbc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a40b8160 con 0x7f62a415c310 2023-11-08T08:20:16.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.450+0000 7f62aadbc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a40b6d70 con 0x7f62a4070600 2023-11-08T08:20:16.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.450+0000 7f62a9dba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62a406b830 0x7f62a40b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:16.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.450+0000 7f62a9dba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62a406b830 0x7f62a40b7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40318/0 (socket says 172.21.15.40:40318) 2023-11-08T08:20:16.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.450+0000 7f62a9dba700 1 -- 172.21.15.40:0/4037359913 learned_addr learned my addr 172.21.15.40:0/4037359913 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:16.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.450+0000 7f62a95b9700 1 --2- 172.21.15.40:0/4037359913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62a4070600 0x7f62a415b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:16.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.450+0000 7f62aa5bb700 1 --2- 172.21.15.40:0/4037359913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62a415c310 0x7f62a41f1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:16.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.450+0000 7f62a9dba700 1 -- 172.21.15.40:0/4037359913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62a4070600 msgr2=0x7f62a415b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:16.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.450+0000 7f62a9dba700 1 --2- 172.21.15.40:0/4037359913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62a4070600 0x7f62a415b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.450+0000 7f62a9dba700 1 -- 172.21.15.40:0/4037359913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62a415c310 msgr2=0x7f62a41f1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:16.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.450+0000 7f62a9dba700 1 --2- 172.21.15.40:0/4037359913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62a415c310 0x7f62a41f1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.450+0000 7f62a9dba700 1 -- 172.21.15.40:0/4037359913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a00094b0 con 0x7f62a406b830 2023-11-08T08:20:16.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.454+0000 7f62aa5bb700 1 --2- 172.21.15.40:0/4037359913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62a415c310 0x7f62a41f1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:20:16.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.454+0000 7f62a9dba700 1 --2- 172.21.15.40:0/4037359913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62a406b830 0x7f62a40b7c20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f62a00090b0 tx=0x7f62a0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:16.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.454+0000 7f629a7fc700 1 -- 172.21.15.40:0/4037359913 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a0004570 con 0x7f62a406b830 2023-11-08T08:20:16.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.454+0000 7f629a7fc700 1 -- 172.21.15.40:0/4037359913 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a001f070 con 0x7f62a406b830 2023-11-08T08:20:16.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.454+0000 7f629a7fc700 1 -- 172.21.15.40:0/4037359913 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a00038e0 con 0x7f62a406b830 2023-11-08T08:20:16.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.454+0000 7f62aadbc700 1 -- 172.21.15.40:0/4037359913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a415c6e0 con 0x7f62a406b830 2023-11-08T08:20:16.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.454+0000 7f62aadbc700 1 -- 172.21.15.40:0/4037359913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a40d1900 con 0x7f62a406b830 2023-11-08T08:20:16.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.454+0000 7f62aadbc700 1 -- 172.21.15.40:0/4037359913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a406d450 con 0x7f62a406b830 2023-11-08T08:20:16.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.458+0000 7f629a7fc700 1 -- 172.21.15.40:0/4037359913 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f62a002a020 con 0x7f62a406b830 2023-11-08T08:20:16.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.458+0000 7f629a7fc700 1 --2- 172.21.15.40:0/4037359913 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6288064f10 0x7f62880673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:16.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.458+0000 7f629a7fc700 1 -- 172.21.15.40:0/4037359913 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f62a0083820 con 0x7f62a406b830 2023-11-08T08:20:16.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.462+0000 7f62a95b9700 1 --2- 172.21.15.40:0/4037359913 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6288064f10 0x7f62880673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:16.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.462+0000 7f62a95b9700 1 --2- 172.21.15.40:0/4037359913 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6288064f10 0x7f62880673c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f62a4156660 tx=0x7f6294008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:16.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.466+0000 7f629a7fc700 1 -- 172.21.15.40:0/4037359913 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f62a00515a0 con 0x7f62a406b830 2023-11-08T08:20:16.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.646+0000 7f62aadbc700 1 -- 172.21.15.40:0/4037359913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f62a4073d20 con 0x7f62a406b830 2023-11-08T08:20:16.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.650+0000 7f629a7fc700 1 -- 172.21.15.40:0/4037359913 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f62a0054bc0 con 0x7f62a406b830 2023-11-08T08:20:16.652 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:20:16.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.654+0000 7f62aadbc700 1 -- 172.21.15.40:0/4037359913 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6288064f10 msgr2=0x7f62880673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:16.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.654+0000 7f62aadbc700 1 --2- 172.21.15.40:0/4037359913 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6288064f10 0x7f62880673c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f62a4156660 tx=0x7f6294008040 comp rx=0 tx=0).stop 2023-11-08T08:20:16.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.654+0000 7f62aadbc700 1 -- 172.21.15.40:0/4037359913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62a406b830 msgr2=0x7f62a40b7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:16.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.654+0000 7f62aadbc700 1 --2- 172.21.15.40:0/4037359913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62a406b830 0x7f62a40b7c20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f62a00090b0 tx=0x7f62a0004770 comp rx=0 tx=0).stop 2023-11-08T08:20:16.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.654+0000 7f62aadbc700 1 -- 172.21.15.40:0/4037359913 shutdown_connections 2023-11-08T08:20:16.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.654+0000 7f62aadbc700 1 --2- 172.21.15.40:0/4037359913 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6288064f10 0x7f62880673c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.654+0000 7f62aadbc700 1 --2- 172.21.15.40:0/4037359913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62a415c310 0x7f62a41f1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.654+0000 7f62aadbc700 1 --2- 172.21.15.40:0/4037359913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62a4070600 0x7f62a415b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.654+0000 7f62aadbc700 1 --2- 172.21.15.40:0/4037359913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62a406b830 0x7f62a40b7c20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.654+0000 7f62aadbc700 1 -- 172.21.15.40:0/4037359913 >> 172.21.15.40:0/4037359913 conn(0x7f62a40d0140 msgr2=0x7f62a40c8570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:16.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.654+0000 7f62aadbc700 1 -- 172.21.15.40:0/4037359913 shutdown_connections 2023-11-08T08:20:16.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.654+0000 7f62aadbc700 1 -- 172.21.15.40:0/4037359913 wait complete. 2023-11-08T08:20:16.680 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:20:16.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.866+0000 7f2c949dc700 1 Processor -- start 2023-11-08T08:20:16.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.866+0000 7f2c949dc700 1 -- start start 2023-11-08T08:20:16.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c949dc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c900d1310 0x7f2c900700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c949dc700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c900cd080 con 0x7f2c90070600 2023-11-08T08:20:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c949dc700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c900d21a0 con 0x7f2c9015c310 2023-11-08T08:20:16.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c949dc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9006d240 con 0x7f2c900d1310 2023-11-08T08:20:16.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8ffff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2c90070600 0x7f2c90069bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59494/0 (socket says 172.21.15.40:59494) 2023-11-08T08:20:16.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8ffff700 1 -- 172.21.15.40:0/26827828 learned_addr learned my addr 172.21.15.40:0/26827828 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:16.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8f7fe700 1 --1- 172.21.15.40:0/26827828 >> v1:172.21.15.169:6789/0 conn(0x7f2c9015c310 0x7f2c9006b830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:40746/0 (socket says 172.21.15.40:40746) 2023-11-08T08:20:16.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8effd700 1 --2- 172.21.15.40:0/26827828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c900d1310 0x7f2c900700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:16.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8e7fc700 1 -- 172.21.15.40:0/26827828 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26538833 0 0) 0x7f2c900cd080 con 0x7f2c90070600 2023-11-08T08:20:16.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8e7fc700 1 -- 172.21.15.40:0/26827828 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c780035d0 con 0x7f2c90070600 2023-11-08T08:20:16.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8e7fc700 1 -- 172.21.15.40:0/26827828 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4170330535 0 0) 0x7f2c900d21a0 con 0x7f2c9015c310 2023-11-08T08:20:16.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8e7fc700 1 -- 172.21.15.40:0/26827828 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c900cd080 con 0x7f2c9015c310 2023-11-08T08:20:16.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8e7fc700 1 -- 172.21.15.40:0/26827828 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1765790145 0 0) 0x7f2c780035d0 con 0x7f2c90070600 2023-11-08T08:20:16.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8e7fc700 1 -- 172.21.15.40:0/26827828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c900d1310 msgr2=0x7f2c900700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:16.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8e7fc700 1 --2- 172.21.15.40:0/26827828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c900d1310 0x7f2c900700c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8e7fc700 1 -- 172.21.15.40:0/26827828 >> v1:172.21.15.169:6789/0 conn(0x7f2c9015c310 legacy=0x7f2c9006b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:16.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8e7fc700 1 -- 172.21.15.40:0/26827828 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c900d1900 con 0x7f2c90070600 2023-11-08T08:20:16.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8e7fc700 1 -- 172.21.15.40:0/26827828 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f2c880019f0 con 0x7f2c90070600 2023-11-08T08:20:16.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8e7fc700 1 -- 172.21.15.40:0/26827828 >> v1:172.21.15.40:6789/0 conn(0x7f2c90070600 legacy=0x7f2c90069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:16.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8e7fc700 1 --2- 172.21.15.40:0/26827828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c78003be0 0x7f2c78003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:16.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8e7fc700 1 --2- 172.21.15.40:0/26827828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c900d1310 0x7f2c78004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:16.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8e7fc700 1 --2- 172.21.15.40:0/26827828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c78004db0 0x7f2c78005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:16.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8e7fc700 1 -- 172.21.15.40:0/26827828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9006d240 con 0x7f2c78003be0 2023-11-08T08:20:16.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8e7fc700 1 -- 172.21.15.40:0/26827828 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7800bae0 con 0x7f2c78004db0 2023-11-08T08:20:16.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8e7fc700 1 -- 172.21.15.40:0/26827828 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7800bc70 con 0x7f2c900d1310 2023-11-08T08:20:16.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8f7fe700 1 --2- 172.21.15.40:0/26827828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c78003be0 0x7f2c78003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:16.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8ffff700 1 --2- 172.21.15.40:0/26827828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c78004db0 0x7f2c78005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:16.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.870+0000 7f2c8effd700 1 --2- 172.21.15.40:0/26827828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c900d1310 0x7f2c78004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:16.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.874+0000 7f2c8f7fe700 1 -- 172.21.15.40:0/26827828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c900d1310 msgr2=0x7f2c78004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:16.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.874+0000 7f2c8f7fe700 1 --2- 172.21.15.40:0/26827828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c900d1310 0x7f2c78004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.874+0000 7f2c8f7fe700 1 -- 172.21.15.40:0/26827828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c78004db0 msgr2=0x7f2c78005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:16.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.874+0000 7f2c8f7fe700 1 --2- 172.21.15.40:0/26827828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c78004db0 0x7f2c78005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.874+0000 7f2c8f7fe700 1 -- 172.21.15.40:0/26827828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c780035d0 con 0x7f2c78003be0 2023-11-08T08:20:16.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.874+0000 7f2c8f7fe700 1 --2- 172.21.15.40:0/26827828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c78003be0 0x7f2c78003fb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2c800019a0 tx=0x7f2c780058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb321f54c2aed21 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:16.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.874+0000 7f2c8e7fc700 1 -- 172.21.15.40:0/26827828 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80001460 con 0x7f2c78003be0 2023-11-08T08:20:16.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.874+0000 7f2c8e7fc700 1 -- 172.21.15.40:0/26827828 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c80004a90 con 0x7f2c78003be0 2023-11-08T08:20:16.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.874+0000 7f2c8e7fc700 1 -- 172.21.15.40:0/26827828 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80004e40 con 0x7f2c78003be0 2023-11-08T08:20:16.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.874+0000 7f2c949dc700 1 -- 172.21.15.40:0/26827828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c78003be0 msgr2=0x7f2c78003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:16.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.874+0000 7f2c949dc700 1 --2- 172.21.15.40:0/26827828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c78003be0 0x7f2c78003fb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2c800019a0 tx=0x7f2c780058a0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.874+0000 7f2c8ffff700 1 -- 172.21.15.40:0/26827828 reap_dead start 2023-11-08T08:20:16.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.874+0000 7f2c949dc700 1 -- 172.21.15.40:0/26827828 shutdown_connections 2023-11-08T08:20:16.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.874+0000 7f2c949dc700 1 -- 172.21.15.40:0/26827828 >> 172.21.15.40:0/26827828 conn(0x7f2c900d0140 msgr2=0x7f2c900d2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:16.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.874+0000 7f2c949dc700 1 -- 172.21.15.40:0/26827828 shutdown_connections 2023-11-08T08:20:16.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.874+0000 7f2c949dc700 1 -- 172.21.15.40:0/26827828 wait complete. 2023-11-08T08:20:16.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.878+0000 7f2c949dc700 1 Processor -- start 2023-11-08T08:20:16.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.878+0000 7f2c949dc700 1 -- start start 2023-11-08T08:20:16.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.878+0000 7f2c949dc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c901f6290 0x7f2c900c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:16.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.878+0000 7f2c949dc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c900c7b60 0x7f2c900c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:16.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.878+0000 7f2c949dc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c90152de0 0x7f2c901f9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:16.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.878+0000 7f2c949dc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c900c7ff0 con 0x7f2c900c7b60 2023-11-08T08:20:16.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.878+0000 7f2c949dc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c900c82d0 con 0x7f2c901f6290 2023-11-08T08:20:16.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.878+0000 7f2c949dc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c900c85b0 con 0x7f2c90152de0 2023-11-08T08:20:16.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.878+0000 7f2c8effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c900c7b60 0x7f2c900c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:16.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.878+0000 7f2c8effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c900c7b60 0x7f2c900c6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40338/0 (socket says 172.21.15.40:40338) 2023-11-08T08:20:16.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.878+0000 7f2c8effd700 1 -- 172.21.15.40:0/4003183574 learned_addr learned my addr 172.21.15.40:0/4003183574 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:16.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.878+0000 7f2c8f7fe700 1 --2- 172.21.15.40:0/4003183574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c901f6290 0x7f2c900c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:16.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.878+0000 7f2c8ffff700 1 --2- 172.21.15.40:0/4003183574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c90152de0 0x7f2c901f9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:16.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.878+0000 7f2c8effd700 1 -- 172.21.15.40:0/4003183574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c90152de0 msgr2=0x7f2c901f9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:16.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.878+0000 7f2c8effd700 1 --2- 172.21.15.40:0/4003183574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c90152de0 0x7f2c901f9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.878+0000 7f2c8effd700 1 -- 172.21.15.40:0/4003183574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c901f6290 msgr2=0x7f2c900c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:16.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.878+0000 7f2c8effd700 1 --2- 172.21.15.40:0/4003183574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c901f6290 0x7f2c900c6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:16.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.878+0000 7f2c8effd700 1 -- 172.21.15.40:0/4003183574 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9006d450 con 0x7f2c900c7b60 2023-11-08T08:20:16.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.882+0000 7f2c8effd700 1 --2- 172.21.15.40:0/4003183574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c900c7b60 0x7f2c900c6aa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2c840116d0 tx=0x7f2c840119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:16.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.882+0000 7f2c77fff700 1 -- 172.21.15.40:0/4003183574 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c84015870 con 0x7f2c900c7b60 2023-11-08T08:20:16.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.882+0000 7f2c77fff700 1 -- 172.21.15.40:0/4003183574 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c840159d0 con 0x7f2c900c7b60 2023-11-08T08:20:16.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.882+0000 7f2c77fff700 1 -- 172.21.15.40:0/4003183574 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c84015d40 con 0x7f2c900c7b60 2023-11-08T08:20:16.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.882+0000 7f2c949dc700 1 -- 172.21.15.40:0/4003183574 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9015c6e0 con 0x7f2c900c7b60 2023-11-08T08:20:16.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.882+0000 7f2c949dc700 1 -- 172.21.15.40:0/4003183574 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9006d730 con 0x7f2c900c7b60 2023-11-08T08:20:16.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.882+0000 7f2c949dc700 1 -- 172.21.15.40:0/4003183574 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c90073d20 con 0x7f2c900c7b60 2023-11-08T08:20:16.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.886+0000 7f2c77fff700 1 -- 172.21.15.40:0/4003183574 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f2c8401b030 con 0x7f2c900c7b60 2023-11-08T08:20:16.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.886+0000 7f2c77fff700 1 --2- 172.21.15.40:0/4003183574 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2c7c064f30 0x7f2c7c0673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:16.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.886+0000 7f2c77fff700 1 -- 172.21.15.40:0/4003183574 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2c84082730 con 0x7f2c900c7b60 2023-11-08T08:20:16.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.886+0000 7f2c8f7fe700 1 --2- 172.21.15.40:0/4003183574 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2c7c064f30 0x7f2c7c0673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:16.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.886+0000 7f2c8f7fe700 1 --2- 172.21.15.40:0/4003183574 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2c7c064f30 0x7f2c7c0673e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2c9006fb70 tx=0x7f2c80007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:16.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:16.894+0000 7f2c77fff700 1 -- 172.21.15.40:0/4003183574 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f2c84054070 con 0x7f2c900c7b60 2023-11-08T08:20:17.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.066+0000 7f2c949dc700 1 -- 172.21.15.40:0/4003183574 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2c900c5eb0 con 0x7f2c900c7b60 2023-11-08T08:20:17.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.066+0000 7f2c77fff700 1 -- 172.21.15.40:0/4003183574 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2c84053c60 con 0x7f2c900c7b60 2023-11-08T08:20:17.072 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:20:17.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.070+0000 7f2c949dc700 1 -- 172.21.15.40:0/4003183574 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2c7c064f30 msgr2=0x7f2c7c0673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:17.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.070+0000 7f2c949dc700 1 --2- 172.21.15.40:0/4003183574 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2c7c064f30 0x7f2c7c0673e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2c9006fb70 tx=0x7f2c80007000 comp rx=0 tx=0).stop 2023-11-08T08:20:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.070+0000 7f2c949dc700 1 -- 172.21.15.40:0/4003183574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c900c7b60 msgr2=0x7f2c900c6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.070+0000 7f2c949dc700 1 --2- 172.21.15.40:0/4003183574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c900c7b60 0x7f2c900c6aa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2c840116d0 tx=0x7f2c840119e0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.070+0000 7f2c949dc700 1 -- 172.21.15.40:0/4003183574 shutdown_connections 2023-11-08T08:20:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.070+0000 7f2c949dc700 1 --2- 172.21.15.40:0/4003183574 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2c7c064f30 0x7f2c7c0673e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.070+0000 7f2c949dc700 1 --2- 172.21.15.40:0/4003183574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c90152de0 0x7f2c901f9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.070+0000 7f2c949dc700 1 --2- 172.21.15.40:0/4003183574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c900c7b60 0x7f2c900c6aa0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.070+0000 7f2c949dc700 1 --2- 172.21.15.40:0/4003183574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c901f6290 0x7f2c900c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.070+0000 7f2c949dc700 1 -- 172.21.15.40:0/4003183574 >> 172.21.15.40:0/4003183574 conn(0x7f2c900d0140 msgr2=0x7f2c9006b830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:17.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.070+0000 7f2c949dc700 1 -- 172.21.15.40:0/4003183574 shutdown_connections 2023-11-08T08:20:17.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.070+0000 7f2c949dc700 1 -- 172.21.15.40:0/4003183574 wait complete. 2023-11-08T08:20:17.097 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T08:20:17.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.274+0000 7f58ff118700 1 Processor -- start 2023-11-08T08:20:17.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.274+0000 7f58ff118700 1 -- start start 2023-11-08T08:20:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.278+0000 7f58ff118700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58f815c150 0x7f58f80b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.278+0000 7f58ff118700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f58f806ff10 0x7f58f8070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.278+0000 7f58ff118700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f815c520 con 0x7f58f815fc30 2023-11-08T08:20:17.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.278+0000 7f58ff118700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f80b4240 con 0x7f58f815c150 2023-11-08T08:20:17.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.278+0000 7f58ff118700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f80708a0 con 0x7f58f806ff10 2023-11-08T08:20:17.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.278+0000 7f58fe116700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58f815c150 0x7f58f80b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.278+0000 7f58fe116700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58f815c150 0x7f58f80b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40342/0 (socket says 172.21.15.40:40342) 2023-11-08T08:20:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.278+0000 7f58fe116700 1 -- 172.21.15.40:0/2782163759 learned_addr learned my addr 172.21.15.40:0/2782163759 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:17.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.278+0000 7f58fd915700 1 --2- 172.21.15.40:0/2782163759 >> v2:172.21.15.169:3300/0 conn(0x7f58f806ff10 0x7f58f8070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:17.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.278+0000 7f58fe116700 1 -- 172.21.15.40:0/2782163759 >> v2:172.21.15.169:3300/0 conn(0x7f58f806ff10 msgr2=0x7f58f8070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:17.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.278+0000 7f58fe116700 1 --2- 172.21.15.40:0/2782163759 >> v2:172.21.15.169:3300/0 conn(0x7f58f806ff10 0x7f58f8070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.278+0000 7f58fe116700 1 -- 172.21.15.40:0/2782163759 >> v1:172.21.15.169:6789/0 conn(0x7f58f815fc30 legacy=0x7f58f8160050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:17.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.278+0000 7f58fe116700 1 -- 172.21.15.40:0/2782163759 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f806ac00 con 0x7f58f815c150 2023-11-08T08:20:17.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.278+0000 7f58fe116700 1 --2- 172.21.15.40:0/2782163759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f815c150 0x7f58f80b3d00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f58f8066d30 tx=0x7f58f4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5420367ccefb07 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:17.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.282+0000 7f58fd114700 1 -- 172.21.15.40:0/2782163759 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f4004230 con 0x7f58f815c150 2023-11-08T08:20:17.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.282+0000 7f58fd114700 1 -- 172.21.15.40:0/2782163759 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f4004390 con 0x7f58f815c150 2023-11-08T08:20:17.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.282+0000 7f58fd114700 1 -- 172.21.15.40:0/2782163759 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f40036c0 con 0x7f58f815c150 2023-11-08T08:20:17.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.282+0000 7f58ff118700 1 -- 172.21.15.40:0/2782163759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f815c150 msgr2=0x7f58f80b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:17.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.282+0000 7f58ff118700 1 --2- 172.21.15.40:0/2782163759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f815c150 0x7f58f80b3d00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f58f8066d30 tx=0x7f58f4009bf0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.282+0000 7f58ff118700 1 -- 172.21.15.40:0/2782163759 shutdown_connections 2023-11-08T08:20:17.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.282+0000 7f58ff118700 1 --2- 172.21.15.40:0/2782163759 >> v2:172.21.15.169:3300/0 conn(0x7f58f806ff10 0x7f58f8070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.282+0000 7f58ff118700 1 --2- 172.21.15.40:0/2782163759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f815c150 0x7f58f80b3d00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.282+0000 7f58ff118700 1 -- 172.21.15.40:0/2782163759 >> 172.21.15.40:0/2782163759 conn(0x7f58f80d3680 msgr2=0x7f58f80cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:17.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.282+0000 7f58ff118700 1 -- 172.21.15.40:0/2782163759 shutdown_connections 2023-11-08T08:20:17.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.282+0000 7f58ff118700 1 -- 172.21.15.40:0/2782163759 wait complete. 2023-11-08T08:20:17.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.282+0000 7f58ff118700 1 Processor -- start 2023-11-08T08:20:17.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58ff118700 1 -- start start 2023-11-08T08:20:17.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58ff118700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f806ff10 0x7f58f8146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:17.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58ff118700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58f815c150 0x7f58f8159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:17.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58ff118700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58f815fc30 0x7f58f8147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:17.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58ff118700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f8146650 con 0x7f58f806ff10 2023-11-08T08:20:17.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58ff118700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f8146dd0 con 0x7f58f815c150 2023-11-08T08:20:17.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58ff118700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f81470b0 con 0x7f58f815fc30 2023-11-08T08:20:17.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58fe116700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f806ff10 0x7f58f8146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:17.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58fe116700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f806ff10 0x7f58f8146110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40352/0 (socket says 172.21.15.40:40352) 2023-11-08T08:20:17.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58fe116700 1 -- 172.21.15.40:0/4262557146 learned_addr learned my addr 172.21.15.40:0/4262557146 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:17.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58fd915700 1 --2- 172.21.15.40:0/4262557146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58f815c150 0x7f58f8159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:17.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58fe917700 1 --2- 172.21.15.40:0/4262557146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58f815fc30 0x7f58f8147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:17.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58fe116700 1 -- 172.21.15.40:0/4262557146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58f815fc30 msgr2=0x7f58f8147e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:17.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58fe116700 1 --2- 172.21.15.40:0/4262557146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58f815fc30 0x7f58f8147e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58fe116700 1 -- 172.21.15.40:0/4262557146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58f815c150 msgr2=0x7f58f8159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:17.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58fe116700 1 --2- 172.21.15.40:0/4262557146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58f815c150 0x7f58f8159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58fe116700 1 -- 172.21.15.40:0/4262557146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f4009570 con 0x7f58f806ff10 2023-11-08T08:20:17.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58fe116700 1 --2- 172.21.15.40:0/4262557146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f806ff10 0x7f58f8146110 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f58f4004000 tx=0x7f58f4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:17.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58eeffd700 1 -- 172.21.15.40:0/4262557146 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f40036e0 con 0x7f58f806ff10 2023-11-08T08:20:17.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58eeffd700 1 -- 172.21.15.40:0/4262557146 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f401f070 con 0x7f58f806ff10 2023-11-08T08:20:17.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58eeffd700 1 -- 172.21.15.40:0/4262557146 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f400b660 con 0x7f58f806ff10 2023-11-08T08:20:17.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58ff118700 1 -- 172.21.15.40:0/4262557146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f80d2450 con 0x7f58f806ff10 2023-11-08T08:20:17.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.286+0000 7f58ff118700 1 -- 172.21.15.40:0/4262557146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f806cc50 con 0x7f58f806ff10 2023-11-08T08:20:17.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.290+0000 7f58ff118700 1 -- 172.21.15.40:0/4262557146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f80d2730 con 0x7f58f806ff10 2023-11-08T08:20:17.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.290+0000 7f58eeffd700 1 -- 172.21.15.40:0/4262557146 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f58f402c070 con 0x7f58f806ff10 2023-11-08T08:20:17.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.294+0000 7f58eeffd700 1 --2- 172.21.15.40:0/4262557146 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f58dc064ee0 0x7f58dc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:17.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.294+0000 7f58eeffd700 1 -- 172.21.15.40:0/4262557146 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f58f40833e0 con 0x7f58f806ff10 2023-11-08T08:20:17.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.294+0000 7f58fd915700 1 --2- 172.21.15.40:0/4262557146 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f58dc064ee0 0x7f58dc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:17.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.294+0000 7f58fd915700 1 --2- 172.21.15.40:0/4262557146 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f58dc064ee0 0x7f58dc067390 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f58f8156a00 tx=0x7f58e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:17.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.298+0000 7f58eeffd700 1 -- 172.21.15.40:0/4262557146 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f58f4051250 con 0x7f58f806ff10 2023-11-08T08:20:17.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.470+0000 7f58ff118700 1 -- 172.21.15.40:0/4262557146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58f8073d20 con 0x7f58f806ff10 2023-11-08T08:20:17.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.470+0000 7f58eeffd700 1 -- 172.21.15.40:0/4262557146 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f58f4054870 con 0x7f58f806ff10 2023-11-08T08:20:17.476 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:20:17.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.474+0000 7f58ff118700 1 -- 172.21.15.40:0/4262557146 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f58dc064ee0 msgr2=0x7f58dc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:17.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.474+0000 7f58ff118700 1 --2- 172.21.15.40:0/4262557146 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f58dc064ee0 0x7f58dc067390 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f58f8156a00 tx=0x7f58e8008040 comp rx=0 tx=0).stop 2023-11-08T08:20:17.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.474+0000 7f58ff118700 1 -- 172.21.15.40:0/4262557146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f806ff10 msgr2=0x7f58f8146110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:17.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.474+0000 7f58ff118700 1 --2- 172.21.15.40:0/4262557146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f806ff10 0x7f58f8146110 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f58f4004000 tx=0x7f58f4004770 comp rx=0 tx=0).stop 2023-11-08T08:20:17.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.474+0000 7f58ff118700 1 -- 172.21.15.40:0/4262557146 shutdown_connections 2023-11-08T08:20:17.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.474+0000 7f58ff118700 1 --2- 172.21.15.40:0/4262557146 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f58dc064ee0 0x7f58dc067390 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.474+0000 7f58ff118700 1 --2- 172.21.15.40:0/4262557146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58f815fc30 0x7f58f8147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.474+0000 7f58ff118700 1 --2- 172.21.15.40:0/4262557146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58f815c150 0x7f58f8159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.474+0000 7f58ff118700 1 --2- 172.21.15.40:0/4262557146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58f806ff10 0x7f58f8146110 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.474+0000 7f58ff118700 1 -- 172.21.15.40:0/4262557146 >> 172.21.15.40:0/4262557146 conn(0x7f58f80d3680 msgr2=0x7f58f80c8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:17.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.474+0000 7f58ff118700 1 -- 172.21.15.40:0/4262557146 shutdown_connections 2023-11-08T08:20:17.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.474+0000 7f58ff118700 1 -- 172.21.15.40:0/4262557146 wait complete. 2023-11-08T08:20:17.499 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T08:20:17.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.690+0000 7f31b359e700 1 Processor -- start 2023-11-08T08:20:17.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.690+0000 7f31b359e700 1 -- start start 2023-11-08T08:20:17.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.690+0000 7f31b359e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f31b4156a30 0x7f31b40d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:17.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.690+0000 7f31b359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31b40afd40 0x7f31b40d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:17.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.690+0000 7f31b359e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b40cffd0 con 0x7f31b406c9c0 2023-11-08T08:20:17.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.690+0000 7f31b359e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b4160d70 con 0x7f31b4156a30 2023-11-08T08:20:17.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.690+0000 7f31b359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b40ce5e0 con 0x7f31b40afd40 2023-11-08T08:20:17.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.694+0000 7f31b259c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f31b4156a30 0x7f31b40d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:17.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.694+0000 7f31b1d9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31b40afd40 0x7f31b40d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:17.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.694+0000 7f31b1d9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31b40afd40 0x7f31b40d0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:33754/0 (socket says 172.21.15.40:33754) 2023-11-08T08:20:17.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.694+0000 7f31b1d9b700 1 -- 172.21.15.40:0/2443967112 learned_addr learned my addr 172.21.15.40:0/2443967112 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:17.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.694+0000 7f31b259c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f31b4156a30 0x7f31b40d0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45390/0 (socket says 172.21.15.40:45390) 2023-11-08T08:20:17.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.694+0000 7f31b1d9b700 1 -- 172.21.15.40:0/2443967112 >> v2:172.21.15.40:3300/0 conn(0x7f31b4156a30 msgr2=0x7f31b40d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:17.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.694+0000 7f31b1d9b700 1 --2- 172.21.15.40:0/2443967112 >> v2:172.21.15.40:3300/0 conn(0x7f31b4156a30 0x7f31b40d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.694+0000 7f31b1d9b700 1 -- 172.21.15.40:0/2443967112 >> v1:172.21.15.169:6789/0 conn(0x7f31b406c9c0 legacy=0x7f31b40cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:17.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.694+0000 7f31b1d9b700 1 -- 172.21.15.40:0/2443967112 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b41607b0 con 0x7f31b40afd40 2023-11-08T08:20:17.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.694+0000 7f31b1d9b700 1 --2- 172.21.15.40:0/2443967112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31b40afd40 0x7f31b40d0db0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009910 tx=0x7f31a8009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=677a64150a995f75 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:17.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.694+0000 7f31b159a700 1 -- 172.21.15.40:0/2443967112 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a8011830 con 0x7f31b40afd40 2023-11-08T08:20:17.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.694+0000 7f31b159a700 1 -- 172.21.15.40:0/2443967112 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a8011990 con 0x7f31b40afd40 2023-11-08T08:20:17.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.694+0000 7f31b159a700 1 -- 172.21.15.40:0/2443967112 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a8011d40 con 0x7f31b40afd40 2023-11-08T08:20:17.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.694+0000 7f31b359e700 1 -- 172.21.15.40:0/2443967112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31b40afd40 msgr2=0x7f31b40d0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:17.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.694+0000 7f31b359e700 1 --2- 172.21.15.40:0/2443967112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31b40afd40 0x7f31b40d0db0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009910 tx=0x7f31a8009c20 comp rx=0 tx=0).stop 2023-11-08T08:20:17.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.698+0000 7f31b359e700 1 -- 172.21.15.40:0/2443967112 shutdown_connections 2023-11-08T08:20:17.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.698+0000 7f31b359e700 1 --2- 172.21.15.40:0/2443967112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31b40afd40 0x7f31b40d0db0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.698+0000 7f31b359e700 1 --2- 172.21.15.40:0/2443967112 >> v2:172.21.15.40:3300/0 conn(0x7f31b4156a30 0x7f31b40d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.698+0000 7f31b359e700 1 -- 172.21.15.40:0/2443967112 >> 172.21.15.40:0/2443967112 conn(0x7f31b4069bf0 msgr2=0x7f31b4069ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:17.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.698+0000 7f31b359e700 1 -- 172.21.15.40:0/2443967112 shutdown_connections 2023-11-08T08:20:17.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.698+0000 7f31b359e700 1 -- 172.21.15.40:0/2443967112 wait complete. 2023-11-08T08:20:17.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.702+0000 7f31b359e700 1 Processor -- start 2023-11-08T08:20:17.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.702+0000 7f31b359e700 1 -- start start 2023-11-08T08:20:17.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.702+0000 7f31b359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31b406c9c0 0x7f31b4138340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:17.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.702+0000 7f31b359e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31b40afd40 0x7f31b415f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:17.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.702+0000 7f31b359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31b4156a30 0x7f31b40c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:17.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.702+0000 7f31b359e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b41505e0 con 0x7f31b4156a30 2023-11-08T08:20:17.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.702+0000 7f31b359e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b40b0220 con 0x7f31b40afd40 2023-11-08T08:20:17.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.702+0000 7f31b359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b40b0470 con 0x7f31b406c9c0 2023-11-08T08:20:17.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.702+0000 7f31b2d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31b4156a30 0x7f31b40c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:17.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.702+0000 7f31b2d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31b4156a30 0x7f31b40c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45398/0 (socket says 172.21.15.40:45398) 2023-11-08T08:20:17.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.702+0000 7f31b2d9d700 1 -- 172.21.15.40:0/3260632554 learned_addr learned my addr 172.21.15.40:0/3260632554 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:17.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.702+0000 7f31b259c700 1 --2- 172.21.15.40:0/3260632554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31b406c9c0 0x7f31b4138340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:17.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.702+0000 7f31b1d9b700 1 --2- 172.21.15.40:0/3260632554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31b40afd40 0x7f31b415f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:17.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.702+0000 7f31b2d9d700 1 -- 172.21.15.40:0/3260632554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31b406c9c0 msgr2=0x7f31b4138340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:17.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.702+0000 7f31b2d9d700 1 --2- 172.21.15.40:0/3260632554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31b406c9c0 0x7f31b4138340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.702+0000 7f31b2d9d700 1 -- 172.21.15.40:0/3260632554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31b40afd40 msgr2=0x7f31b415f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:17.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.702+0000 7f31b2d9d700 1 --2- 172.21.15.40:0/3260632554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31b40afd40 0x7f31b415f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.702+0000 7f31b2d9d700 1 -- 172.21.15.40:0/3260632554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a8009570 con 0x7f31b4156a30 2023-11-08T08:20:17.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.706+0000 7f31b2d9d700 1 --2- 172.21.15.40:0/3260632554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31b4156a30 0x7f31b40c65e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00cb50 tx=0x7f31ac00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:17.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.706+0000 7f31a2ffd700 1 -- 172.21.15.40:0/3260632554 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac003c20 con 0x7f31b4156a30 2023-11-08T08:20:17.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.706+0000 7f31a2ffd700 1 -- 172.21.15.40:0/3260632554 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ac003d80 con 0x7f31b4156a30 2023-11-08T08:20:17.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.706+0000 7f31b359e700 1 -- 172.21.15.40:0/3260632554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b40d0250 con 0x7f31b4156a30 2023-11-08T08:20:17.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.706+0000 7f31a2ffd700 1 -- 172.21.15.40:0/3260632554 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac00aa20 con 0x7f31b4156a30 2023-11-08T08:20:17.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.706+0000 7f31b359e700 1 -- 172.21.15.40:0/3260632554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b4160a90 con 0x7f31b4156a30 2023-11-08T08:20:17.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.706+0000 7f31b359e700 1 -- 172.21.15.40:0/3260632554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b40d0530 con 0x7f31b4156a30 2023-11-08T08:20:17.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.710+0000 7f31a2ffd700 1 -- 172.21.15.40:0/3260632554 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f31ac010050 con 0x7f31b4156a30 2023-11-08T08:20:17.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.710+0000 7f31a2ffd700 1 --2- 172.21.15.40:0/3260632554 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3194064ee0 0x7f3194067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:17.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.710+0000 7f31a2ffd700 1 -- 172.21.15.40:0/3260632554 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f31ac0821d0 con 0x7f31b4156a30 2023-11-08T08:20:17.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.710+0000 7f31b259c700 1 --2- 172.21.15.40:0/3260632554 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3194064ee0 0x7f3194067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:17.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.714+0000 7f31b259c700 1 --2- 172.21.15.40:0/3260632554 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3194064ee0 0x7f3194067390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009b50 tx=0x7f31a4009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:17.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.717+0000 7f31a2ffd700 1 -- 172.21.15.40:0/3260632554 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f31ac04fec0 con 0x7f31b4156a30 2023-11-08T08:20:17.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.893+0000 7f31b359e700 1 -- 172.21.15.40:0/3260632554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f31b40d16a0 con 0x7f31b4156a30 2023-11-08T08:20:17.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.893+0000 7f31a2ffd700 1 -- 172.21.15.40:0/3260632554 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f31ac0534e0 con 0x7f31b4156a30 2023-11-08T08:20:17.897 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-11-08T08:20:17.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.893+0000 7f31b359e700 1 -- 172.21.15.40:0/3260632554 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3194064ee0 msgr2=0x7f3194067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:17.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.893+0000 7f31b359e700 1 --2- 172.21.15.40:0/3260632554 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3194064ee0 0x7f3194067390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009b50 tx=0x7f31a4009240 comp rx=0 tx=0).stop 2023-11-08T08:20:17.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.893+0000 7f31b359e700 1 -- 172.21.15.40:0/3260632554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31b4156a30 msgr2=0x7f31b40c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:17.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.893+0000 7f31b359e700 1 --2- 172.21.15.40:0/3260632554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31b4156a30 0x7f31b40c65e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00cb50 tx=0x7f31ac00ce60 comp rx=0 tx=0).stop 2023-11-08T08:20:17.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.893+0000 7f31b359e700 1 -- 172.21.15.40:0/3260632554 shutdown_connections 2023-11-08T08:20:17.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.893+0000 7f31b359e700 1 --2- 172.21.15.40:0/3260632554 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3194064ee0 0x7f3194067390 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.893+0000 7f31b359e700 1 --2- 172.21.15.40:0/3260632554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31b4156a30 0x7f31b40c65e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.893+0000 7f31b359e700 1 --2- 172.21.15.40:0/3260632554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31b40afd40 0x7f31b415f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.893+0000 7f31b359e700 1 --2- 172.21.15.40:0/3260632554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31b406c9c0 0x7f31b4138340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:17.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.893+0000 7f31b359e700 1 -- 172.21.15.40:0/3260632554 >> 172.21.15.40:0/3260632554 conn(0x7f31b4069bf0 msgr2=0x7f31b4069fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:17.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.897+0000 7f31b359e700 1 -- 172.21.15.40:0/3260632554 shutdown_connections 2023-11-08T08:20:17.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:17.897+0000 7f31b359e700 1 -- 172.21.15.40:0/3260632554 wait complete. 2023-11-08T08:20:17.923 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T08:20:18.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.097+0000 7fb0ef184700 1 Processor -- start 2023-11-08T08:20:18.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.097+0000 7fb0ef184700 1 -- start start 2023-11-08T08:20:18.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.101+0000 7fb0ef184700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb0e815c150 0x7fb0e80b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:18.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.101+0000 7fb0ef184700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e815c520 con 0x7fb0e806ff10 2023-11-08T08:20:18.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.101+0000 7fb0ef184700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e806ac00 con 0x7fb0e8070360 2023-11-08T08:20:18.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.101+0000 7fb0ef184700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e80b4240 con 0x7fb0e815c150 2023-11-08T08:20:18.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.101+0000 7fb0ed981700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb0e806ff10 0x7fb0e815fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37948/0 (socket says 172.21.15.40:37948) 2023-11-08T08:20:18.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.101+0000 7fb0ed981700 1 -- 172.21.15.40:0/3178308703 learned_addr learned my addr 172.21.15.40:0/3178308703 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:18.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.101+0000 7fb0ee182700 1 --2- 172.21.15.40:0/3178308703 >> v2:172.21.15.40:3300/0 conn(0x7fb0e815c150 0x7fb0e80b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:18.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.101+0000 7fb0ed180700 1 -- 172.21.15.40:0/3178308703 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318028289 0 0) 0x7fb0e815c520 con 0x7fb0e806ff10 2023-11-08T08:20:18.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.101+0000 7fb0ed180700 1 -- 172.21.15.40:0/3178308703 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0d40035d0 con 0x7fb0e806ff10 2023-11-08T08:20:18.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.101+0000 7fb0ee182700 1 -- 172.21.15.40:0/3178308703 >> v1:172.21.15.169:6789/0 conn(0x7fb0e8070360 legacy=0x7fb0e8160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:18.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.101+0000 7fb0ee182700 1 -- 172.21.15.40:0/3178308703 >> v1:172.21.15.40:6789/0 conn(0x7fb0e806ff10 legacy=0x7fb0e815fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:18.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.101+0000 7fb0ee182700 1 -- 172.21.15.40:0/3178308703 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e8160a50 con 0x7fb0e815c150 2023-11-08T08:20:18.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.101+0000 7fb0ee182700 1 --2- 172.21.15.40:0/3178308703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e815c150 0x7fb0e80b3d00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009850 tx=0x7fb0e4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f4a6934bf70d3d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:18.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.105+0000 7fb0ed180700 1 -- 172.21.15.40:0/3178308703 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4011830 con 0x7fb0e815c150 2023-11-08T08:20:18.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.105+0000 7fb0ed180700 1 -- 172.21.15.40:0/3178308703 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e4011990 con 0x7fb0e815c150 2023-11-08T08:20:18.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.105+0000 7fb0ed180700 1 -- 172.21.15.40:0/3178308703 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4011d40 con 0x7fb0e815c150 2023-11-08T08:20:18.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.105+0000 7fb0ef184700 1 -- 172.21.15.40:0/3178308703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e815c150 msgr2=0x7fb0e80b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:18.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.105+0000 7fb0ef184700 1 --2- 172.21.15.40:0/3178308703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e815c150 0x7fb0e80b3d00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009850 tx=0x7fb0e4009b60 comp rx=0 tx=0).stop 2023-11-08T08:20:18.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.105+0000 7fb0ef184700 1 -- 172.21.15.40:0/3178308703 shutdown_connections 2023-11-08T08:20:18.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.105+0000 7fb0ef184700 1 --2- 172.21.15.40:0/3178308703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e815c150 0x7fb0e80b3d00 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.105+0000 7fb0ef184700 1 -- 172.21.15.40:0/3178308703 >> 172.21.15.40:0/3178308703 conn(0x7fb0e80d3680 msgr2=0x7fb0e80cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:18.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.105+0000 7fb0ef184700 1 -- 172.21.15.40:0/3178308703 shutdown_connections 2023-11-08T08:20:18.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.105+0000 7fb0ef184700 1 -- 172.21.15.40:0/3178308703 wait complete. 2023-11-08T08:20:18.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.105+0000 7fb0ef184700 1 Processor -- start 2023-11-08T08:20:18.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.105+0000 7fb0ef184700 1 -- start start 2023-11-08T08:20:18.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0ef184700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e806ff10 0x7fb0e80cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0ef184700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0e8070360 0x7fb0e8150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0ef184700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0e815c150 0x7fb0e814fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0ef184700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8152d00 con 0x7fb0e806ff10 2023-11-08T08:20:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0ef184700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8152fe0 con 0x7fb0e8070360 2023-11-08T08:20:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0ef184700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e81532c0 con 0x7fb0e815c150 2023-11-08T08:20:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0ee182700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e806ff10 0x7fb0e80cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0ee182700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e806ff10 0x7fb0e80cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45424/0 (socket says 172.21.15.40:45424) 2023-11-08T08:20:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0ee182700 1 -- 172.21.15.40:0/68636233 learned_addr learned my addr 172.21.15.40:0/68636233 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0ed981700 1 --2- 172.21.15.40:0/68636233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0e8070360 0x7fb0e8150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0ee983700 1 --2- 172.21.15.40:0/68636233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0e815c150 0x7fb0e814fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0ee182700 1 -- 172.21.15.40:0/68636233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0e815c150 msgr2=0x7fb0e814fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0ee182700 1 --2- 172.21.15.40:0/68636233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0e815c150 0x7fb0e814fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0ee182700 1 -- 172.21.15.40:0/68636233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0e8070360 msgr2=0x7fb0e8150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0ee182700 1 --2- 172.21.15.40:0/68636233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0e8070360 0x7fb0e8150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0ee182700 1 -- 172.21.15.40:0/68636233 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e40094b0 con 0x7fb0e806ff10 2023-11-08T08:20:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0ee182700 1 --2- 172.21.15.40:0/68636233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e806ff10 0x7fb0e80cc530 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4014040 tx=0x7fb0e4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0deffd700 1 -- 172.21.15.40:0/68636233 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4019980 con 0x7fb0e806ff10 2023-11-08T08:20:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0deffd700 1 -- 172.21.15.40:0/68636233 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e401e070 con 0x7fb0e806ff10 2023-11-08T08:20:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0deffd700 1 -- 172.21.15.40:0/68636233 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4019d20 con 0x7fb0e806ff10 2023-11-08T08:20:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0ef184700 1 -- 172.21.15.40:0/68636233 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e80d2450 con 0x7fb0e806ff10 2023-11-08T08:20:18.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.109+0000 7fb0ef184700 1 -- 172.21.15.40:0/68636233 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e806cc50 con 0x7fb0e806ff10 2023-11-08T08:20:18.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.113+0000 7fb0ef184700 1 -- 172.21.15.40:0/68636233 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e80d2730 con 0x7fb0e806ff10 2023-11-08T08:20:18.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.113+0000 7fb0deffd700 1 -- 172.21.15.40:0/68636233 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fb0e400f050 con 0x7fb0e806ff10 2023-11-08T08:20:18.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.117+0000 7fb0deffd700 1 --2- 172.21.15.40:0/68636233 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb0cc064ee0 0x7fb0cc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:18.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.117+0000 7fb0deffd700 1 -- 172.21.15.40:0/68636233 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb0e40831d0 con 0x7fb0e806ff10 2023-11-08T08:20:18.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.117+0000 7fb0ed981700 1 --2- 172.21.15.40:0/68636233 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb0cc064ee0 0x7fb0cc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:18.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.117+0000 7fb0ed981700 1 --2- 172.21.15.40:0/68636233 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb0cc064ee0 0x7fb0cc067390 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80c9a60 tx=0x7fb0d8007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.121+0000 7fb0deffd700 1 -- 172.21.15.40:0/68636233 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fb0e4050f50 con 0x7fb0e806ff10 2023-11-08T08:20:18.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.297+0000 7fb0ef184700 1 -- 172.21.15.40:0/68636233 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb0e80cf4a0 con 0x7fb0e806ff10 2023-11-08T08:20:18.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.297+0000 7fb0deffd700 1 -- 172.21.15.40:0/68636233 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb0e40200a0 con 0x7fb0e806ff10 2023-11-08T08:20:18.302 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:20:18.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.297+0000 7fb0ef184700 1 -- 172.21.15.40:0/68636233 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb0cc064ee0 msgr2=0x7fb0cc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:18.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.297+0000 7fb0ef184700 1 --2- 172.21.15.40:0/68636233 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb0cc064ee0 0x7fb0cc067390 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80c9a60 tx=0x7fb0d8007c90 comp rx=0 tx=0).stop 2023-11-08T08:20:18.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.297+0000 7fb0ef184700 1 -- 172.21.15.40:0/68636233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e806ff10 msgr2=0x7fb0e80cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:18.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.297+0000 7fb0ef184700 1 --2- 172.21.15.40:0/68636233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e806ff10 0x7fb0e80cc530 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4014040 tx=0x7fb0e4004770 comp rx=0 tx=0).stop 2023-11-08T08:20:18.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.301+0000 7fb0ef184700 1 -- 172.21.15.40:0/68636233 shutdown_connections 2023-11-08T08:20:18.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.301+0000 7fb0ef184700 1 --2- 172.21.15.40:0/68636233 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb0cc064ee0 0x7fb0cc067390 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.301+0000 7fb0ef184700 1 --2- 172.21.15.40:0/68636233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0e815c150 0x7fb0e814fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.301+0000 7fb0ef184700 1 --2- 172.21.15.40:0/68636233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0e8070360 0x7fb0e8150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.301+0000 7fb0ef184700 1 --2- 172.21.15.40:0/68636233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0e806ff10 0x7fb0e80cc530 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.301+0000 7fb0ef184700 1 -- 172.21.15.40:0/68636233 >> 172.21.15.40:0/68636233 conn(0x7fb0e80d3680 msgr2=0x7fb0e8156710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:18.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.301+0000 7fb0ef184700 1 -- 172.21.15.40:0/68636233 shutdown_connections 2023-11-08T08:20:18.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.301+0000 7fb0ef184700 1 -- 172.21.15.40:0/68636233 wait complete. 2023-11-08T08:20:18.324 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:20:18.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.501+0000 7f5a04bce700 1 Processor -- start 2023-11-08T08:20:18.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.501+0000 7f5a04bce700 1 -- start start 2023-11-08T08:20:18.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.505+0000 7f5a04bce700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a0015c640 0x7f5a000d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:18.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.505+0000 7f5a04bce700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5a00071aa0 0x7f5a0006d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:18.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.505+0000 7f5a04bce700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5a00069e00 0x7f5a0006a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:18.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.505+0000 7f5a04bce700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a0006cef0 con 0x7f5a00069e00 2023-11-08T08:20:18.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.505+0000 7f5a04bce700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a0006a7c0 con 0x7f5a00071aa0 2023-11-08T08:20:18.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.505+0000 7f5a04bce700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0006c600 con 0x7f5a0015c640 2023-11-08T08:20:18.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.505+0000 7f59fffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5a00069e00 0x7f5a0006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:18.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.505+0000 7f59ff7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a0015c640 0x7f5a000d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:18.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.505+0000 7f59fffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5a00069e00 0x7f5a0006a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45436/0 (socket says 172.21.15.40:45436) 2023-11-08T08:20:18.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.505+0000 7f59feffd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5a00071aa0 0x7f5a0006d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:18.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.505+0000 7f59fffff700 1 -- 172.21.15.40:0/1184323603 learned_addr learned my addr 172.21.15.40:0/1184323603 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:18.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.505+0000 7f59fffff700 1 -- 172.21.15.40:0/1184323603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a0015c640 msgr2=0x7f5a000d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:18.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.505+0000 7f59fffff700 1 --2- 172.21.15.40:0/1184323603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a0015c640 0x7f5a000d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.505+0000 7f59fffff700 1 -- 172.21.15.40:0/1184323603 >> v2:172.21.15.169:3300/0 conn(0x7f5a00071aa0 msgr2=0x7f5a0006d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:18.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.505+0000 7f59fffff700 1 --2- 172.21.15.40:0/1184323603 >> v2:172.21.15.169:3300/0 conn(0x7f5a00071aa0 0x7f5a0006d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.505+0000 7f59fffff700 1 -- 172.21.15.40:0/1184323603 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a000722f0 con 0x7f5a00069e00 2023-11-08T08:20:18.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.505+0000 7f59fffff700 1 --2- 172.21.15.40:0/1184323603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a00069e00 0x7f5a0006a280 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5a0006fc90 tx=0x7f59f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5f5d461e4edb5f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:18.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.505+0000 7f59fe7fc700 1 -- 172.21.15.40:0/1184323603 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f800e070 con 0x7f5a00069e00 2023-11-08T08:20:18.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.505+0000 7f59fe7fc700 1 -- 172.21.15.40:0/1184323603 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f800b910 con 0x7f5a00069e00 2023-11-08T08:20:18.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.509+0000 7f59fe7fc700 1 -- 172.21.15.40:0/1184323603 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f800bcc0 con 0x7f5a00069e00 2023-11-08T08:20:18.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.509+0000 7f5a04bce700 1 -- 172.21.15.40:0/1184323603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a00069e00 msgr2=0x7f5a0006a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:18.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.509+0000 7f5a04bce700 1 --2- 172.21.15.40:0/1184323603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a00069e00 0x7f5a0006a280 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5a0006fc90 tx=0x7f59f800b330 comp rx=0 tx=0).stop 2023-11-08T08:20:18.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.509+0000 7f5a04bce700 1 -- 172.21.15.40:0/1184323603 shutdown_connections 2023-11-08T08:20:18.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.509+0000 7f5a04bce700 1 --2- 172.21.15.40:0/1184323603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a00069e00 0x7f5a0006a280 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.509+0000 7f5a04bce700 1 --2- 172.21.15.40:0/1184323603 >> v2:172.21.15.169:3300/0 conn(0x7f5a00071aa0 0x7f5a0006d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.509+0000 7f5a04bce700 1 --2- 172.21.15.40:0/1184323603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a0015c640 0x7f5a000d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.509+0000 7f5a04bce700 1 -- 172.21.15.40:0/1184323603 >> 172.21.15.40:0/1184323603 conn(0x7f5a000b6520 msgr2=0x7f5a00071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:18.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.509+0000 7f5a04bce700 1 -- 172.21.15.40:0/1184323603 shutdown_connections 2023-11-08T08:20:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.509+0000 7f5a04bce700 1 -- 172.21.15.40:0/1184323603 wait complete. 2023-11-08T08:20:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.513+0000 7f5a04bce700 1 Processor -- start 2023-11-08T08:20:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.513+0000 7f5a04bce700 1 -- start start 2023-11-08T08:20:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.513+0000 7f5a04bce700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a00069e00 0x7f5a000c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:18.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.513+0000 7f5a04bce700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a00071aa0 0x7f5a000ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:18.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.513+0000 7f5a04bce700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a0015c640 0x7f5a001588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:18.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.513+0000 7f5a04bce700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a000cb880 con 0x7f5a0015c640 2023-11-08T08:20:18.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.513+0000 7f5a04bce700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a000ca9d0 con 0x7f5a00069e00 2023-11-08T08:20:18.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.513+0000 7f5a04bce700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a000cac70 con 0x7f5a00071aa0 2023-11-08T08:20:18.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.513+0000 7f59fffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a0015c640 0x7f5a001588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:18.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.513+0000 7f59ff7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a00069e00 0x7f5a000c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:18.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.513+0000 7f59fffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a0015c640 0x7f5a001588f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45442/0 (socket says 172.21.15.40:45442) 2023-11-08T08:20:18.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.513+0000 7f59ff7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a00069e00 0x7f5a000c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:37254/0 (socket says 172.21.15.40:37254) 2023-11-08T08:20:18.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.513+0000 7f59fffff700 1 -- 172.21.15.40:0/2301983077 learned_addr learned my addr 172.21.15.40:0/2301983077 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:18.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.513+0000 7f59feffd700 1 --2- 172.21.15.40:0/2301983077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a00071aa0 0x7f5a000ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:18.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.517+0000 7f59fffff700 1 -- 172.21.15.40:0/2301983077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a00071aa0 msgr2=0x7f5a000ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:18.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.517+0000 7f59fffff700 1 --2- 172.21.15.40:0/2301983077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a00071aa0 0x7f5a000ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.517+0000 7f59fffff700 1 -- 172.21.15.40:0/2301983077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a00069e00 msgr2=0x7f5a000c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:18.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.517+0000 7f59fffff700 1 --2- 172.21.15.40:0/2301983077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a00069e00 0x7f5a000c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.517+0000 7f59fffff700 1 -- 172.21.15.40:0/2301983077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f8009b30 con 0x7f5a0015c640 2023-11-08T08:20:18.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.517+0000 7f59ff7fe700 1 --2- 172.21.15.40:0/2301983077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a00069e00 0x7f5a000c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:18.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.517+0000 7f59fffff700 1 --2- 172.21.15.40:0/2301983077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a0015c640 0x7f5a001588f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5a0006f090 tx=0x7f59f8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.517+0000 7f59f7fff700 1 -- 172.21.15.40:0/2301983077 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f80099b0 con 0x7f5a0015c640 2023-11-08T08:20:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.517+0000 7f59f7fff700 1 -- 172.21.15.40:0/2301983077 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f800ce80 con 0x7f5a0015c640 2023-11-08T08:20:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.517+0000 7f59f7fff700 1 -- 172.21.15.40:0/2301983077 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8004470 con 0x7f5a0015c640 2023-11-08T08:20:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.517+0000 7f5a04bce700 1 -- 172.21.15.40:0/2301983077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a00072010 con 0x7f5a0015c640 2023-11-08T08:20:18.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.517+0000 7f5a04bce700 1 -- 172.21.15.40:0/2301983077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0015ca10 con 0x7f5a0015c640 2023-11-08T08:20:18.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.517+0000 7f5a04bce700 1 -- 172.21.15.40:0/2301983077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a0006c830 con 0x7f5a0015c640 2023-11-08T08:20:18.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.521+0000 7f59f7fff700 1 -- 172.21.15.40:0/2301983077 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f59f802c070 con 0x7f5a0015c640 2023-11-08T08:20:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.521+0000 7f59f7fff700 1 --2- 172.21.15.40:0/2301983077 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f59e0064ee0 0x7f59e0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.521+0000 7f59f7fff700 1 -- 172.21.15.40:0/2301983077 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f59f80833d0 con 0x7f5a0015c640 2023-11-08T08:20:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.521+0000 7f59ff7fe700 1 --2- 172.21.15.40:0/2301983077 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f59e0064ee0 0x7f59e0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:18.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.521+0000 7f59ff7fe700 1 --2- 172.21.15.40:0/2301983077 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f59e0064ee0 0x7f59e0067390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f59f00112c0 tx=0x7f59f0009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.529+0000 7f59f7fff700 1 -- 172.21.15.40:0/2301983077 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f59f8051150 con 0x7f5a0015c640 2023-11-08T08:20:18.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.737+0000 7f5a04bce700 1 -- 172.21.15.40:0/2301983077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a00073cb0 con 0x7f5a0015c640 2023-11-08T08:20:18.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.737+0000 7f59f7fff700 1 -- 172.21.15.40:0/2301983077 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4641 (secure 0 0 0) 0x7f59f8054770 con 0x7f5a0015c640 2023-11-08T08:20:18.742 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:18.743 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/196211087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":196211087},{"type":"v1","addr":"172.21.15.40:6834","nonce":196211087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/67127106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":67127106},{"type":"v1","addr":"172.21.15.40:6835","nonce":67127106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1231240563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1231240563},{"type":"v1","addr":"172.21.15.169:6835","nonce":1231240563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:19:04.456508+0000","modified":"2023-11-08T08:19:31.023792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6837/1989495204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1989495204},{"type":"v1","addr":"172.21.15.169:6837","nonce":1989495204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T08:20:18.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.741+0000 7f5a04bce700 1 -- 172.21.15.40:0/2301983077 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f59e0064ee0 msgr2=0x7f59e0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:18.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.741+0000 7f5a04bce700 1 --2- 172.21.15.40:0/2301983077 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f59e0064ee0 0x7f59e0067390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f59f00112c0 tx=0x7f59f0009240 comp rx=0 tx=0).stop 2023-11-08T08:20:18.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.741+0000 7f5a04bce700 1 -- 172.21.15.40:0/2301983077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a0015c640 msgr2=0x7f5a001588f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:18.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.741+0000 7f5a04bce700 1 --2- 172.21.15.40:0/2301983077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a0015c640 0x7f5a001588f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5a0006f090 tx=0x7f59f8004af0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.741+0000 7f5a04bce700 1 -- 172.21.15.40:0/2301983077 shutdown_connections 2023-11-08T08:20:18.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.741+0000 7f5a04bce700 1 --2- 172.21.15.40:0/2301983077 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f59e0064ee0 0x7f59e0067390 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.741+0000 7f5a04bce700 1 --2- 172.21.15.40:0/2301983077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a0015c640 0x7f5a001588f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.741+0000 7f5a04bce700 1 --2- 172.21.15.40:0/2301983077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a00071aa0 0x7f5a000ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.741+0000 7f5a04bce700 1 --2- 172.21.15.40:0/2301983077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a00069e00 0x7f5a000c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.741+0000 7f5a04bce700 1 -- 172.21.15.40:0/2301983077 >> 172.21.15.40:0/2301983077 conn(0x7f5a000b6520 msgr2=0x7f5a000b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:18.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.741+0000 7f5a04bce700 1 -- 172.21.15.40:0/2301983077 shutdown_connections 2023-11-08T08:20:18.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.741+0000 7f5a04bce700 1 -- 172.21.15.40:0/2301983077 wait complete. 2023-11-08T08:20:18.750 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 9 2023-11-08T08:20:18.769 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:20:18.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.937+0000 7f6306d77700 1 Processor -- start 2023-11-08T08:20:18.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.941+0000 7f6306d77700 1 -- start start 2023-11-08T08:20:18.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.941+0000 7f6306d77700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6300070080 0x7f63000704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:18.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.941+0000 7f6306d77700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6300164250 0x7f63001646d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:18.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.941+0000 7f6306d77700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63000ad6c0 con 0x7f6300160710 2023-11-08T08:20:18.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.941+0000 7f6306d77700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f630006b1c0 con 0x7f6300164250 2023-11-08T08:20:18.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.941+0000 7f6306d77700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63000adc60 con 0x7f6300070080 2023-11-08T08:20:18.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.941+0000 7f6305d75700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6300160710 0x7f63000cc320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37950/0 (socket says 172.21.15.40:37950) 2023-11-08T08:20:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.941+0000 7f6305d75700 1 -- 172.21.15.40:0/1494530903 learned_addr learned my addr 172.21.15.40:0/1494530903 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.941+0000 7f6305574700 1 --2- 172.21.15.40:0/1494530903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6300070080 0x7f63000704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.941+0000 7f6306576700 1 --2- 172.21.15.40:0/1494530903 >> v2:172.21.15.169:3300/0 conn(0x7f6300164250 0x7f63001646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.945+0000 7f6306576700 1 -- 172.21.15.40:0/1494530903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6300070080 msgr2=0x7f63000704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.945+0000 7f6306576700 1 --2- 172.21.15.40:0/1494530903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6300070080 0x7f63000704b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.945+0000 7f6306576700 1 -- 172.21.15.40:0/1494530903 >> v1:172.21.15.40:6789/0 conn(0x7f6300160710 legacy=0x7f63000cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:18.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.945+0000 7f6304d73700 1 -- 172.21.15.40:0/1494530903 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074803416 0 0) 0x7f63000ad6c0 con 0x7f6300160710 2023-11-08T08:20:18.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.945+0000 7f6306576700 1 -- 172.21.15.40:0/1494530903 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63000709f0 con 0x7f6300164250 2023-11-08T08:20:18.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.945+0000 7f6305574700 1 --2- 172.21.15.40:0/1494530903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6300070080 0x7f63000704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:18.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.945+0000 7f6306576700 1 --2- 172.21.15.40:0/1494530903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6300164250 0x7f63001646d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009d90 tx=0x7f62f800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d2010f9d3f742fb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:18.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.945+0000 7f6304d73700 1 -- 172.21.15.40:0/1494530903 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f800e070 con 0x7f6300164250 2023-11-08T08:20:18.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.945+0000 7f6304d73700 1 -- 172.21.15.40:0/1494530903 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f800bb20 con 0x7f6300164250 2023-11-08T08:20:18.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.945+0000 7f6304d73700 1 -- 172.21.15.40:0/1494530903 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f8012430 con 0x7f6300164250 2023-11-08T08:20:18.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.945+0000 7f6306d77700 1 -- 172.21.15.40:0/1494530903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6300164250 msgr2=0x7f63001646d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:18.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.945+0000 7f6306d77700 1 --2- 172.21.15.40:0/1494530903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6300164250 0x7f63001646d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009d90 tx=0x7f62f800b330 comp rx=0 tx=0).stop 2023-11-08T08:20:18.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.945+0000 7f6306d77700 1 -- 172.21.15.40:0/1494530903 shutdown_connections 2023-11-08T08:20:18.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.945+0000 7f6306d77700 1 --2- 172.21.15.40:0/1494530903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6300164250 0x7f63001646d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.945+0000 7f6306d77700 1 --2- 172.21.15.40:0/1494530903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6300070080 0x7f63000704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.945+0000 7f6306d77700 1 -- 172.21.15.40:0/1494530903 >> 172.21.15.40:0/1494530903 conn(0x7f63000b0f80 msgr2=0x7f63000ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:18.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.945+0000 7f6306d77700 1 -- 172.21.15.40:0/1494530903 shutdown_connections 2023-11-08T08:20:18.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.945+0000 7f6306d77700 1 -- 172.21.15.40:0/1494530903 wait complete. 2023-11-08T08:20:18.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.949+0000 7f6306d77700 1 Processor -- start 2023-11-08T08:20:18.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.949+0000 7f6306d77700 1 -- start start 2023-11-08T08:20:18.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f6306d77700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6300070080 0x7f63000c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:18.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f6306d77700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6300160710 0x7f6300151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:18.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f6306d77700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6300164250 0x7f630015f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:18.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f6306d77700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63001fa0c0 con 0x7f6300160710 2023-11-08T08:20:18.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f6306d77700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63001f9230 con 0x7f6300070080 2023-11-08T08:20:18.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f6306d77700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63001f9440 con 0x7f6300164250 2023-11-08T08:20:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f6305574700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6300160710 0x7f6300151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f6305d75700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6300070080 0x7f63000c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f6305574700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6300160710 0x7f6300151cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45452/0 (socket says 172.21.15.40:45452) 2023-11-08T08:20:18.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f6305574700 1 -- 172.21.15.40:0/1580856419 learned_addr learned my addr 172.21.15.40:0/1580856419 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:18.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f6306576700 1 --2- 172.21.15.40:0/1580856419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6300164250 0x7f630015f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:18.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f6305574700 1 -- 172.21.15.40:0/1580856419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6300164250 msgr2=0x7f630015f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:18.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f6305574700 1 --2- 172.21.15.40:0/1580856419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6300164250 0x7f630015f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f6305574700 1 -- 172.21.15.40:0/1580856419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6300070080 msgr2=0x7f63000c2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f6305574700 1 --2- 172.21.15.40:0/1580856419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6300070080 0x7f63000c2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f6305574700 1 -- 172.21.15.40:0/1580856419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f00094b0 con 0x7f6300160710 2023-11-08T08:20:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f6305574700 1 --2- 172.21.15.40:0/1580856419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6300160710 0x7f6300151cb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f62f000ebe0 tx=0x7f62f000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f62f67fc700 1 -- 172.21.15.40:0/1580856419 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f000c850 con 0x7f6300160710 2023-11-08T08:20:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f62f67fc700 1 -- 172.21.15.40:0/1580856419 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f000c9b0 con 0x7f6300160710 2023-11-08T08:20:18.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f6306d77700 1 -- 172.21.15.40:0/1580856419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f8009a70 con 0x7f6300160710 2023-11-08T08:20:18.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f62f67fc700 1 -- 172.21.15.40:0/1580856419 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f000cd40 con 0x7f6300160710 2023-11-08T08:20:18.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.953+0000 7f6306d77700 1 -- 172.21.15.40:0/1580856419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f630006b3d0 con 0x7f6300160710 2023-11-08T08:20:18.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.957+0000 7f6306d77700 1 -- 172.21.15.40:0/1580856419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630006b6b0 con 0x7f6300160710 2023-11-08T08:20:18.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.957+0000 7f62f67fc700 1 -- 172.21.15.40:0/1580856419 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f62f0019030 con 0x7f6300160710 2023-11-08T08:20:18.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.961+0000 7f62f67fc700 1 --2- 172.21.15.40:0/1580856419 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f62e4064ee0 0x7f62e4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:18.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.961+0000 7f62f67fc700 1 -- 172.21.15.40:0/1580856419 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f62f0082230 con 0x7f6300160710 2023-11-08T08:20:18.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.961+0000 7f6305d75700 1 --2- 172.21.15.40:0/1580856419 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f62e4064ee0 0x7f62e4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:18.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.961+0000 7f6305d75700 1 --2- 172.21.15.40:0/1580856419 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f62e4064ee0 0x7f62e4067390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f62fc000f60 tx=0x7f62fc007e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:18.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:18.965+0000 7f62f67fc700 1 -- 172.21.15.40:0/1580856419 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f62f004ffb0 con 0x7f6300160710 2023-11-08T08:20:19.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.145+0000 7f6306d77700 1 -- 172.21.15.40:0/1580856419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6300073cb0 con 0x7f6300160710 2023-11-08T08:20:19.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.145+0000 7f62f67fc700 1 -- 172.21.15.40:0/1580856419 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19748 (secure 0 0 0) 0x7f62f000cea0 con 0x7f6300160710 2023-11-08T08:20:19.152 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:19.152 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":30,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:45.885827+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:20:19.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.149+0000 7f6306d77700 1 -- 172.21.15.40:0/1580856419 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f62e4064ee0 msgr2=0x7f62e4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:19.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.149+0000 7f6306d77700 1 --2- 172.21.15.40:0/1580856419 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f62e4064ee0 0x7f62e4067390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f62fc000f60 tx=0x7f62fc007e00 comp rx=0 tx=0).stop 2023-11-08T08:20:19.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.149+0000 7f6306d77700 1 -- 172.21.15.40:0/1580856419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6300160710 msgr2=0x7f6300151cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:19.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.149+0000 7f6306d77700 1 --2- 172.21.15.40:0/1580856419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6300160710 0x7f6300151cb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f62f000ebe0 tx=0x7f62f000eef0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.149+0000 7f6306d77700 1 -- 172.21.15.40:0/1580856419 shutdown_connections 2023-11-08T08:20:19.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.149+0000 7f6306d77700 1 --2- 172.21.15.40:0/1580856419 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f62e4064ee0 0x7f62e4067390 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.149+0000 7f6306d77700 1 --2- 172.21.15.40:0/1580856419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6300164250 0x7f630015f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.149+0000 7f6306d77700 1 --2- 172.21.15.40:0/1580856419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6300160710 0x7f6300151cb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.149+0000 7f6306d77700 1 --2- 172.21.15.40:0/1580856419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6300070080 0x7f63000c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.149+0000 7f6306d77700 1 -- 172.21.15.40:0/1580856419 >> 172.21.15.40:0/1580856419 conn(0x7f63000b0f80 msgr2=0x7f63000b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:19.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.149+0000 7f6306d77700 1 -- 172.21.15.40:0/1580856419 shutdown_connections 2023-11-08T08:20:19.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.149+0000 7f6306d77700 1 -- 172.21.15.40:0/1580856419 wait complete. 2023-11-08T08:20:19.182 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T08:20:19.182 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:20:19.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.349+0000 7fa7cf530700 1 Processor -- start 2023-11-08T08:20:19.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.349+0000 7fa7cf530700 1 -- start start 2023-11-08T08:20:19.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.349+0000 7fa7cf530700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa7c80cdeb0 0x7fa7c8070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:19.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.349+0000 7fa7cf530700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa7c80705c0 0x7fa7c806a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:19.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.349+0000 7fa7cf530700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7c80709d0 con 0x7fa7c815c1a0 2023-11-08T08:20:19.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.349+0000 7fa7cf530700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c8069bf0 con 0x7fa7c80705c0 2023-11-08T08:20:19.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.349+0000 7fa7cf530700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c80adf90 con 0x7fa7c80cdeb0 2023-11-08T08:20:19.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.353+0000 7fa7cdd2d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa7c80cdeb0 0x7fa7c8070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:19.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.353+0000 7fa7ce52e700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa7c815c1a0 0x7fa7c806c8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37958/0 (socket says 172.21.15.40:37958) 2023-11-08T08:20:19.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.353+0000 7fa7ce52e700 1 -- 172.21.15.40:0/2585735845 learned_addr learned my addr 172.21.15.40:0/2585735845 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:19.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.353+0000 7fa7ced2f700 1 --2- 172.21.15.40:0/2585735845 >> v2:172.21.15.40:3300/0 conn(0x7fa7c80705c0 0x7fa7c806a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:19.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.353+0000 7fa7cd52c700 1 -- 172.21.15.40:0/2585735845 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2350222570 0 0) 0x7fa7c80709d0 con 0x7fa7c815c1a0 2023-11-08T08:20:19.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.353+0000 7fa7cd52c700 1 -- 172.21.15.40:0/2585735845 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7b40035d0 con 0x7fa7c815c1a0 2023-11-08T08:20:19.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.353+0000 7fa7cdd2d700 1 -- 172.21.15.40:0/2585735845 >> v2:172.21.15.40:3300/0 conn(0x7fa7c80705c0 msgr2=0x7fa7c806a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:19.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.353+0000 7fa7cdd2d700 1 --2- 172.21.15.40:0/2585735845 >> v2:172.21.15.40:3300/0 conn(0x7fa7c80705c0 0x7fa7c806a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.353+0000 7fa7cdd2d700 1 -- 172.21.15.40:0/2585735845 >> v1:172.21.15.40:6789/0 conn(0x7fa7c815c1a0 legacy=0x7fa7c806c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:19.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.353+0000 7fa7cdd2d700 1 -- 172.21.15.40:0/2585735845 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c80ce370 con 0x7fa7c80cdeb0 2023-11-08T08:20:19.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.353+0000 7fa7ced2f700 1 --2- 172.21.15.40:0/2585735845 >> v2:172.21.15.40:3300/0 conn(0x7fa7c80705c0 0x7fa7c806a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:19.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.353+0000 7fa7cdd2d700 1 --2- 172.21.15.40:0/2585735845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7c80cdeb0 0x7fa7c8070080 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa7c806d2b0 tx=0x7fa7b8009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fec21278f396ff6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:19.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.353+0000 7fa7cd52c700 1 -- 172.21.15.40:0/2585735845 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8011830 con 0x7fa7c80cdeb0 2023-11-08T08:20:19.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.353+0000 7fa7cd52c700 1 -- 172.21.15.40:0/2585735845 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b8011990 con 0x7fa7c80cdeb0 2023-11-08T08:20:19.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.353+0000 7fa7cd52c700 1 -- 172.21.15.40:0/2585735845 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8011d20 con 0x7fa7c80cdeb0 2023-11-08T08:20:19.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.357+0000 7fa7cf530700 1 -- 172.21.15.40:0/2585735845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7c80cdeb0 msgr2=0x7fa7c8070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:19.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.357+0000 7fa7cf530700 1 --2- 172.21.15.40:0/2585735845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7c80cdeb0 0x7fa7c8070080 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa7c806d2b0 tx=0x7fa7b8009b30 comp rx=0 tx=0).stop 2023-11-08T08:20:19.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.357+0000 7fa7cf530700 1 -- 172.21.15.40:0/2585735845 shutdown_connections 2023-11-08T08:20:19.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.357+0000 7fa7cf530700 1 --2- 172.21.15.40:0/2585735845 >> v2:172.21.15.40:3300/0 conn(0x7fa7c80705c0 0x7fa7c806a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.357+0000 7fa7cf530700 1 --2- 172.21.15.40:0/2585735845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7c80cdeb0 0x7fa7c8070080 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.357+0000 7fa7cf530700 1 -- 172.21.15.40:0/2585735845 >> 172.21.15.40:0/2585735845 conn(0x7fa7c80b0f80 msgr2=0x7fa7c80ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:19.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.357+0000 7fa7cf530700 1 -- 172.21.15.40:0/2585735845 shutdown_connections 2023-11-08T08:20:19.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.357+0000 7fa7cf530700 1 -- 172.21.15.40:0/2585735845 wait complete. 2023-11-08T08:20:19.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7cf530700 1 Processor -- start 2023-11-08T08:20:19.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7cf530700 1 -- start start 2023-11-08T08:20:19.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7cf530700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7c80705c0 0x7fa7c80b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:19.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7cf530700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7c80cdeb0 0x7fa7c80b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:19.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7cf530700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c815c1a0 0x7fa7c81f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:19.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7cf530700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c80b1b80 con 0x7fa7c815c1a0 2023-11-08T08:20:19.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7cf530700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c81f4a90 con 0x7fa7c80705c0 2023-11-08T08:20:19.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7cf530700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c81f1700 con 0x7fa7c80cdeb0 2023-11-08T08:20:19.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7ced2f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c815c1a0 0x7fa7c81f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:19.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7ced2f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c815c1a0 0x7fa7c81f4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45474/0 (socket says 172.21.15.40:45474) 2023-11-08T08:20:19.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7ced2f700 1 -- 172.21.15.40:0/2111479465 learned_addr learned my addr 172.21.15.40:0/2111479465 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:19.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7ce52e700 1 --2- 172.21.15.40:0/2111479465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7c80705c0 0x7fa7c80b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:19.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7cdd2d700 1 --2- 172.21.15.40:0/2111479465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7c80cdeb0 0x7fa7c80b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:19.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7ced2f700 1 -- 172.21.15.40:0/2111479465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7c80cdeb0 msgr2=0x7fa7c80b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:19.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7ced2f700 1 --2- 172.21.15.40:0/2111479465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7c80cdeb0 0x7fa7c80b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7ced2f700 1 -- 172.21.15.40:0/2111479465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7c80705c0 msgr2=0x7fa7c80b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:19.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7ced2f700 1 --2- 172.21.15.40:0/2111479465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7c80705c0 0x7fa7c80b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7ced2f700 1 -- 172.21.15.40:0/2111479465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c00094b0 con 0x7fa7c815c1a0 2023-11-08T08:20:19.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7cdd2d700 1 --2- 172.21.15.40:0/2111479465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7c80cdeb0 0x7fa7c80b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:20:19.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7ced2f700 1 --2- 172.21.15.40:0/2111479465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c815c1a0 0x7fa7c81f4550 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000eb60 tx=0x7fa7c000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:19.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7beffd700 1 -- 172.21.15.40:0/2111479465 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0004020 con 0x7fa7c815c1a0 2023-11-08T08:20:19.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7beffd700 1 -- 172.21.15.40:0/2111479465 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c0004180 con 0x7fa7c815c1a0 2023-11-08T08:20:19.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7cf530700 1 -- 172.21.15.40:0/2111479465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b80094b0 con 0x7fa7c815c1a0 2023-11-08T08:20:19.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7beffd700 1 -- 172.21.15.40:0/2111479465 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0004530 con 0x7fa7c815c1a0 2023-11-08T08:20:19.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.361+0000 7fa7cf530700 1 -- 172.21.15.40:0/2111479465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c80cc780 con 0x7fa7c815c1a0 2023-11-08T08:20:19.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.365+0000 7fa7cf530700 1 -- 172.21.15.40:0/2111479465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c80cc4a0 con 0x7fa7c815c1a0 2023-11-08T08:20:19.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.365+0000 7fa7beffd700 1 -- 172.21.15.40:0/2111479465 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa7c000c850 con 0x7fa7c815c1a0 2023-11-08T08:20:19.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.369+0000 7fa7beffd700 1 --2- 172.21.15.40:0/2111479465 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa7ac064ee0 0x7fa7ac067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:19.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.369+0000 7fa7beffd700 1 -- 172.21.15.40:0/2111479465 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa7c0082280 con 0x7fa7c815c1a0 2023-11-08T08:20:19.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.369+0000 7fa7ce52e700 1 --2- 172.21.15.40:0/2111479465 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa7ac064ee0 0x7fa7ac067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:19.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.373+0000 7fa7ce52e700 1 --2- 172.21.15.40:0/2111479465 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa7ac064ee0 0x7fa7ac067390 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4001560 tx=0x7fa7c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:19.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.373+0000 7fa7beffd700 1 -- 172.21.15.40:0/2111479465 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa7c0050000 con 0x7fa7c815c1a0 2023-11-08T08:20:19.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.585+0000 7fa7cf530700 1 -- 172.21.15.40:0/2111479465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7c815c570 con 0x7fa7c815c1a0 2023-11-08T08:20:19.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.585+0000 7fa7beffd700 1 -- 172.21.15.40:0/2111479465 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4641 (secure 0 0 0) 0x7fa7c0053620 con 0x7fa7c815c1a0 2023-11-08T08:20:19.590 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:19.591 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/196211087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":196211087},{"type":"v1","addr":"172.21.15.40:6834","nonce":196211087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/67127106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":67127106},{"type":"v1","addr":"172.21.15.40:6835","nonce":67127106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1231240563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1231240563},{"type":"v1","addr":"172.21.15.169:6835","nonce":1231240563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:19:04.456508+0000","modified":"2023-11-08T08:19:31.023792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6837/1989495204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1989495204},{"type":"v1","addr":"172.21.15.169:6837","nonce":1989495204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T08:20:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.589+0000 7fa7cf530700 1 -- 172.21.15.40:0/2111479465 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa7ac064ee0 msgr2=0x7fa7ac067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.589+0000 7fa7cf530700 1 --2- 172.21.15.40:0/2111479465 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa7ac064ee0 0x7fa7ac067390 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4001560 tx=0x7fa7c4009040 comp rx=0 tx=0).stop 2023-11-08T08:20:19.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.589+0000 7fa7cf530700 1 -- 172.21.15.40:0/2111479465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c815c1a0 msgr2=0x7fa7c81f4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:19.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.589+0000 7fa7cf530700 1 --2- 172.21.15.40:0/2111479465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c815c1a0 0x7fa7c81f4550 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000eb60 tx=0x7fa7c000ef20 comp rx=0 tx=0).stop 2023-11-08T08:20:19.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.589+0000 7fa7cf530700 1 -- 172.21.15.40:0/2111479465 shutdown_connections 2023-11-08T08:20:19.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.589+0000 7fa7cf530700 1 --2- 172.21.15.40:0/2111479465 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa7ac064ee0 0x7fa7ac067390 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.589+0000 7fa7cf530700 1 --2- 172.21.15.40:0/2111479465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c815c1a0 0x7fa7c81f4550 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.589+0000 7fa7cf530700 1 --2- 172.21.15.40:0/2111479465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7c80cdeb0 0x7fa7c80b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.589+0000 7fa7cf530700 1 --2- 172.21.15.40:0/2111479465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7c80705c0 0x7fa7c80b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.589+0000 7fa7cf530700 1 -- 172.21.15.40:0/2111479465 >> 172.21.15.40:0/2111479465 conn(0x7fa7c80b0f80 msgr2=0x7fa7c80abaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.589+0000 7fa7cf530700 1 -- 172.21.15.40:0/2111479465 shutdown_connections 2023-11-08T08:20:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.589+0000 7fa7cf530700 1 -- 172.21.15.40:0/2111479465 wait complete. 2023-11-08T08:20:19.596 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 9 2023-11-08T08:20:19.614 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:20:19.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.793+0000 7f975ad70700 1 Processor -- start 2023-11-08T08:20:19.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.793+0000 7f975ad70700 1 -- start start 2023-11-08T08:20:19.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.793+0000 7f975ad70700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f975415c640 0x7f97540d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:19.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.793+0000 7f975ad70700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9754071aa0 0x7f975406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:19.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.793+0000 7f975ad70700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9754069e00 0x7f975406a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:19.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.793+0000 7f975ad70700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f975406cef0 con 0x7f975415c640 2023-11-08T08:20:19.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.793+0000 7f975ad70700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f975406a7c0 con 0x7f9754069e00 2023-11-08T08:20:19.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f975ad70700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f975406c600 con 0x7f9754071aa0 2023-11-08T08:20:19.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f9759d6e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f975415c640 0x7f97540d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:19.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f9759d6e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f975415c640 0x7f97540d31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45486/0 (socket says 172.21.15.40:45486) 2023-11-08T08:20:19.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f9759d6e700 1 -- 172.21.15.40:0/3006795728 learned_addr learned my addr 172.21.15.40:0/3006795728 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:19.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f9759d6e700 1 -- 172.21.15.40:0/3006795728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9754071aa0 msgr2=0x7f975406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:19.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f9759d6e700 1 --2- 172.21.15.40:0/3006795728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9754071aa0 0x7f975406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f975a56f700 1 --2- 172.21.15.40:0/3006795728 >> v2:172.21.15.169:3300/0 conn(0x7f9754069e00 0x7f975406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:19.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f9759d6e700 1 -- 172.21.15.40:0/3006795728 >> v2:172.21.15.169:3300/0 conn(0x7f9754069e00 msgr2=0x7f975406a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:19.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f9759d6e700 1 --2- 172.21.15.40:0/3006795728 >> v2:172.21.15.169:3300/0 conn(0x7f9754069e00 0x7f975406a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f9759d6e700 1 -- 172.21.15.40:0/3006795728 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97540722f0 con 0x7f975415c640 2023-11-08T08:20:19.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f975a56f700 1 --2- 172.21.15.40:0/3006795728 >> v2:172.21.15.169:3300/0 conn(0x7f9754069e00 0x7f975406a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:20:19.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f9759d6e700 1 --2- 172.21.15.40:0/3006795728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f975415c640 0x7f97540d31e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f975406efb0 tx=0x7f975000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5349770cae868d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:19.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f9758d6c700 1 -- 172.21.15.40:0/3006795728 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975000e070 con 0x7f975415c640 2023-11-08T08:20:19.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f9758d6c700 1 -- 172.21.15.40:0/3006795728 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975000b9c0 con 0x7f975415c640 2023-11-08T08:20:19.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f9758d6c700 1 -- 172.21.15.40:0/3006795728 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975000bd70 con 0x7f975415c640 2023-11-08T08:20:19.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f975ad70700 1 -- 172.21.15.40:0/3006795728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f975415c640 msgr2=0x7f97540d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:19.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f975ad70700 1 --2- 172.21.15.40:0/3006795728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f975415c640 0x7f97540d31e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f975406efb0 tx=0x7f975000b330 comp rx=0 tx=0).stop 2023-11-08T08:20:19.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f975ad70700 1 -- 172.21.15.40:0/3006795728 shutdown_connections 2023-11-08T08:20:19.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f975ad70700 1 --2- 172.21.15.40:0/3006795728 >> v2:172.21.15.169:3300/0 conn(0x7f9754069e00 0x7f975406a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f975ad70700 1 --2- 172.21.15.40:0/3006795728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9754071aa0 0x7f975406d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f975ad70700 1 --2- 172.21.15.40:0/3006795728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f975415c640 0x7f97540d31e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f975ad70700 1 -- 172.21.15.40:0/3006795728 >> 172.21.15.40:0/3006795728 conn(0x7f97540b6520 msgr2=0x7f9754071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:19.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f975ad70700 1 -- 172.21.15.40:0/3006795728 shutdown_connections 2023-11-08T08:20:19.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.797+0000 7f975ad70700 1 -- 172.21.15.40:0/3006795728 wait complete. 2023-11-08T08:20:19.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.805+0000 7f975ad70700 1 Processor -- start 2023-11-08T08:20:19.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.805+0000 7f975ad70700 1 -- start start 2023-11-08T08:20:19.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.805+0000 7f975ad70700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9754069e00 0x7f97540ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:19.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.805+0000 7f975ad70700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9754071aa0 0x7f97540cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:19.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.805+0000 7f975ad70700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f975415c640 0x7f97541f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:19.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.805+0000 7f975ad70700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97541f61b0 con 0x7f975415c640 2023-11-08T08:20:19.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.805+0000 7f975ad70700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97540ced40 con 0x7f9754069e00 2023-11-08T08:20:19.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.805+0000 7f975ad70700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97540ce4a0 con 0x7f9754071aa0 2023-11-08T08:20:19.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.805+0000 7f975a56f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f975415c640 0x7f97541f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:19.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.805+0000 7f975a56f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f975415c640 0x7f97541f5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45502/0 (socket says 172.21.15.40:45502) 2023-11-08T08:20:19.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.805+0000 7f975a56f700 1 -- 172.21.15.40:0/684787114 learned_addr learned my addr 172.21.15.40:0/684787114 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:19.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.805+0000 7f9759d6e700 1 --2- 172.21.15.40:0/684787114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9754069e00 0x7f97540ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:19.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.805+0000 7f975956d700 1 --2- 172.21.15.40:0/684787114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9754071aa0 0x7f97540cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:19.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.805+0000 7f975a56f700 1 -- 172.21.15.40:0/684787114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9754071aa0 msgr2=0x7f97540cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:19.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.805+0000 7f975a56f700 1 --2- 172.21.15.40:0/684787114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9754071aa0 0x7f97540cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.805+0000 7f975a56f700 1 -- 172.21.15.40:0/684787114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9754069e00 msgr2=0x7f97540ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:19.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.805+0000 7f975a56f700 1 --2- 172.21.15.40:0/684787114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9754069e00 0x7f97540ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:19.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.805+0000 7f975a56f700 1 -- 172.21.15.40:0/684787114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9750009b30 con 0x7f975415c640 2023-11-08T08:20:19.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.805+0000 7f975a56f700 1 --2- 172.21.15.40:0/684787114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f975415c640 0x7f97541f5c70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f974c00c7f0 tx=0x7f974c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:19.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.809+0000 7f974a7fc700 1 -- 172.21.15.40:0/684787114 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c011800 con 0x7f975415c640 2023-11-08T08:20:19.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.809+0000 7f974a7fc700 1 -- 172.21.15.40:0/684787114 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974c011980 con 0x7f975415c640 2023-11-08T08:20:19.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.809+0000 7f975ad70700 1 -- 172.21.15.40:0/684787114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9754072010 con 0x7f975415c640 2023-11-08T08:20:19.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.809+0000 7f974a7fc700 1 -- 172.21.15.40:0/684787114 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c011d30 con 0x7f975415c640 2023-11-08T08:20:19.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.809+0000 7f975ad70700 1 -- 172.21.15.40:0/684787114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975415ca10 con 0x7f975415c640 2023-11-08T08:20:19.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.809+0000 7f975ad70700 1 -- 172.21.15.40:0/684787114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f975406c830 con 0x7f975415c640 2023-11-08T08:20:19.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.809+0000 7f974a7fc700 1 -- 172.21.15.40:0/684787114 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f974c021070 con 0x7f975415c640 2023-11-08T08:20:19.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.813+0000 7f974a7fc700 1 --2- 172.21.15.40:0/684787114 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9738064ee0 0x7f9738067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:19.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.813+0000 7f974a7fc700 1 -- 172.21.15.40:0/684787114 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f974c0820b0 con 0x7f975415c640 2023-11-08T08:20:19.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.813+0000 7f9759d6e700 1 --2- 172.21.15.40:0/684787114 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9738064ee0 0x7f9738067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:19.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.813+0000 7f9759d6e700 1 --2- 172.21.15.40:0/684787114 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9738064ee0 0x7f9738067390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f975000b370 tx=0x7f9750009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:19.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:19.821+0000 7f974a7fc700 1 -- 172.21.15.40:0/684787114 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f974c04fdb0 con 0x7f975415c640 2023-11-08T08:20:20.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.029+0000 7f975ad70700 1 -- 172.21.15.40:0/684787114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9754072c10 con 0x7f975415c640 2023-11-08T08:20:20.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.029+0000 7f974a7fc700 1 -- 172.21.15.40:0/684787114 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4641 (secure 0 0 0) 0x7f974c0533d0 con 0x7f975415c640 2023-11-08T08:20:20.034 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:20.034 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/196211087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":196211087},{"type":"v1","addr":"172.21.15.40:6834","nonce":196211087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/67127106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":67127106},{"type":"v1","addr":"172.21.15.40:6835","nonce":67127106}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1231240563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1231240563},{"type":"v1","addr":"172.21.15.169:6835","nonce":1231240563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:19:04.456508+0000","modified":"2023-11-08T08:19:31.023792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6837/1989495204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1989495204},{"type":"v1","addr":"172.21.15.169:6837","nonce":1989495204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T08:20:20.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.033+0000 7f975ad70700 1 -- 172.21.15.40:0/684787114 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9738064ee0 msgr2=0x7f9738067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:20.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.033+0000 7f975ad70700 1 --2- 172.21.15.40:0/684787114 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9738064ee0 0x7f9738067390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f975000b370 tx=0x7f9750009000 comp rx=0 tx=0).stop 2023-11-08T08:20:20.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.033+0000 7f975ad70700 1 -- 172.21.15.40:0/684787114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f975415c640 msgr2=0x7f97541f5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:20.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.033+0000 7f975ad70700 1 --2- 172.21.15.40:0/684787114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f975415c640 0x7f97541f5c70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f974c00c7f0 tx=0x7f974c00cb00 comp rx=0 tx=0).stop 2023-11-08T08:20:20.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.033+0000 7f975ad70700 1 -- 172.21.15.40:0/684787114 shutdown_connections 2023-11-08T08:20:20.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.033+0000 7f975ad70700 1 --2- 172.21.15.40:0/684787114 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9738064ee0 0x7f9738067390 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.033+0000 7f975ad70700 1 --2- 172.21.15.40:0/684787114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f975415c640 0x7f97541f5c70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.033+0000 7f975ad70700 1 --2- 172.21.15.40:0/684787114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9754071aa0 0x7f97540cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.033+0000 7f975ad70700 1 --2- 172.21.15.40:0/684787114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9754069e00 0x7f97540ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.033+0000 7f975ad70700 1 -- 172.21.15.40:0/684787114 >> 172.21.15.40:0/684787114 conn(0x7f97540b6520 msgr2=0x7f97540b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:20.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.033+0000 7f975ad70700 1 -- 172.21.15.40:0/684787114 shutdown_connections 2023-11-08T08:20:20.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.033+0000 7f975ad70700 1 -- 172.21.15.40:0/684787114 wait complete. 2023-11-08T08:20:20.042 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 9 2023-11-08T08:20:20.061 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:20:20.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.245+0000 7f20b61e1700 1 Processor -- start 2023-11-08T08:20:20.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.245+0000 7f20b61e1700 1 -- start start 2023-11-08T08:20:20.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.245+0000 7f20b61e1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f20b015c640 0x7f20b00d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:20.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.245+0000 7f20b61e1700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f20b0071aa0 0x7f20b006d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:20.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b61e1700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b006c600 con 0x7f20b0069e00 2023-11-08T08:20:20.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b61e1700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b006cef0 con 0x7f20b015c640 2023-11-08T08:20:20.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b61e1700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b00722f0 con 0x7f20b0071aa0 2023-11-08T08:20:20.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b51df700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f20b015c640 0x7f20b00d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:20.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b51df700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f20b015c640 0x7f20b00d31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45510/0 (socket says 172.21.15.40:45510) 2023-11-08T08:20:20.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b51df700 1 -- 172.21.15.40:0/2268758281 learned_addr learned my addr 172.21.15.40:0/2268758281 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:20.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b49de700 1 --2- 172.21.15.40:0/2268758281 >> v2:172.21.15.169:3300/0 conn(0x7f20b0071aa0 0x7f20b006d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:20.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b51df700 1 -- 172.21.15.40:0/2268758281 >> v2:172.21.15.169:3300/0 conn(0x7f20b0071aa0 msgr2=0x7f20b006d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:20.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b51df700 1 --2- 172.21.15.40:0/2268758281 >> v2:172.21.15.169:3300/0 conn(0x7f20b0071aa0 0x7f20b006d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b51df700 1 -- 172.21.15.40:0/2268758281 >> v1:172.21.15.169:6789/0 conn(0x7f20b0069e00 legacy=0x7f20b00d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:20.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b51df700 1 -- 172.21.15.40:0/2268758281 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b015ca10 con 0x7f20b015c640 2023-11-08T08:20:20.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b49de700 1 --2- 172.21.15.40:0/2268758281 >> v2:172.21.15.169:3300/0 conn(0x7f20b0071aa0 0x7f20b006d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:20:20.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b51df700 1 --2- 172.21.15.40:0/2268758281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b015c640 0x7f20b00d31e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f20b0070e70 tx=0x7f20ac009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e5977110de5ad7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:20.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20a7fff700 1 -- 172.21.15.40:0/2268758281 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac004230 con 0x7f20b015c640 2023-11-08T08:20:20.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20a7fff700 1 -- 172.21.15.40:0/2268758281 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ac004390 con 0x7f20b015c640 2023-11-08T08:20:20.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20a7fff700 1 -- 172.21.15.40:0/2268758281 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac0036c0 con 0x7f20b015c640 2023-11-08T08:20:20.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b61e1700 1 -- 172.21.15.40:0/2268758281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b015c640 msgr2=0x7f20b00d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:20.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b61e1700 1 --2- 172.21.15.40:0/2268758281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b015c640 0x7f20b00d31e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f20b0070e70 tx=0x7f20ac009bf0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b61e1700 1 -- 172.21.15.40:0/2268758281 shutdown_connections 2023-11-08T08:20:20.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b61e1700 1 --2- 172.21.15.40:0/2268758281 >> v2:172.21.15.169:3300/0 conn(0x7f20b0071aa0 0x7f20b006d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b61e1700 1 --2- 172.21.15.40:0/2268758281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b015c640 0x7f20b00d31e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b61e1700 1 -- 172.21.15.40:0/2268758281 >> 172.21.15.40:0/2268758281 conn(0x7f20b00b6520 msgr2=0x7f20b0071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:20.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b61e1700 1 -- 172.21.15.40:0/2268758281 shutdown_connections 2023-11-08T08:20:20.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.249+0000 7f20b61e1700 1 -- 172.21.15.40:0/2268758281 wait complete. 2023-11-08T08:20:20.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.253+0000 7f20b61e1700 1 Processor -- start 2023-11-08T08:20:20.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.253+0000 7f20b61e1700 1 -- start start 2023-11-08T08:20:20.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.253+0000 7f20b61e1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b0069e00 0x7f20b01e3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:20.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.253+0000 7f20b61e1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b0071aa0 0x7f20b01e42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:20.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.253+0000 7f20b61e1700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b015c640 0x7f20b01e4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:20.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.253+0000 7f20b61e1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b00cdb00 con 0x7f20b0071aa0 2023-11-08T08:20:20.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.253+0000 7f20b61e1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b00cce80 con 0x7f20b0069e00 2023-11-08T08:20:20.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.253+0000 7f20b61e1700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b00cd160 con 0x7f20b015c640 2023-11-08T08:20:20.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.253+0000 7f20b49de700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b0071aa0 0x7f20b01e42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:20.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.253+0000 7f20b49de700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b0071aa0 0x7f20b01e42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45516/0 (socket says 172.21.15.40:45516) 2023-11-08T08:20:20.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.253+0000 7f20b49de700 1 -- 172.21.15.40:0/733583661 learned_addr learned my addr 172.21.15.40:0/733583661 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:20.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.253+0000 7f20b59e0700 1 --2- 172.21.15.40:0/733583661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b015c640 0x7f20b01e4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:20.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.253+0000 7f20b51df700 1 --2- 172.21.15.40:0/733583661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b0069e00 0x7f20b01e3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:20.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.257+0000 7f20b49de700 1 -- 172.21.15.40:0/733583661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b015c640 msgr2=0x7f20b01e4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:20.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.257+0000 7f20b49de700 1 --2- 172.21.15.40:0/733583661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b015c640 0x7f20b01e4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.257+0000 7f20b49de700 1 -- 172.21.15.40:0/733583661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b0069e00 msgr2=0x7f20b01e3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:20.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.257+0000 7f20b49de700 1 --2- 172.21.15.40:0/733583661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b0069e00 0x7f20b01e3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.257+0000 7f20b49de700 1 -- 172.21.15.40:0/733583661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac009570 con 0x7f20b0071aa0 2023-11-08T08:20:20.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.257+0000 7f20b59e0700 1 --2- 172.21.15.40:0/733583661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b015c640 0x7f20b01e4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:20.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.257+0000 7f20b49de700 1 --2- 172.21.15.40:0/733583661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b0071aa0 0x7f20b01e42e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f20a000c770 tx=0x7f20a000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:20.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.257+0000 7f20a5ffb700 1 -- 172.21.15.40:0/733583661 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a0011800 con 0x7f20b0071aa0 2023-11-08T08:20:20.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.257+0000 7f20b61e1700 1 -- 172.21.15.40:0/733583661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b006c830 con 0x7f20b0071aa0 2023-11-08T08:20:20.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.257+0000 7f20a5ffb700 1 -- 172.21.15.40:0/733583661 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a0011960 con 0x7f20b0071aa0 2023-11-08T08:20:20.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.257+0000 7f20a5ffb700 1 -- 172.21.15.40:0/733583661 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a0011d10 con 0x7f20b0071aa0 2023-11-08T08:20:20.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.257+0000 7f20b61e1700 1 -- 172.21.15.40:0/733583661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b0072010 con 0x7f20b0071aa0 2023-11-08T08:20:20.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.257+0000 7f20b61e1700 1 -- 172.21.15.40:0/733583661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b006cb10 con 0x7f20b0071aa0 2023-11-08T08:20:20.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.261+0000 7f20a5ffb700 1 -- 172.21.15.40:0/733583661 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f20a0021070 con 0x7f20b0071aa0 2023-11-08T08:20:20.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.261+0000 7f20a5ffb700 1 --2- 172.21.15.40:0/733583661 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2094064ee0 0x7f2094067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:20.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.261+0000 7f20a5ffb700 1 -- 172.21.15.40:0/733583661 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f20a0082280 con 0x7f20b0071aa0 2023-11-08T08:20:20.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.261+0000 7f20b51df700 1 --2- 172.21.15.40:0/733583661 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2094064ee0 0x7f2094067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:20.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.261+0000 7f20b51df700 1 --2- 172.21.15.40:0/733583661 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2094064ee0 0x7f2094067390 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f20b0070e70 tx=0x7f20ac008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:20.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.269+0000 7f20a5ffb700 1 -- 172.21.15.40:0/733583661 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f20a004ff20 con 0x7f20b0071aa0 2023-11-08T08:20:20.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.441+0000 7f20b61e1700 1 -- 172.21.15.40:0/733583661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20b0073cb0 con 0x7f20b0071aa0 2023-11-08T08:20:20.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.445+0000 7f20a5ffb700 1 -- 172.21.15.40:0/733583661 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19748 (secure 0 0 0) 0x7f20a0053540 con 0x7f20b0071aa0 2023-11-08T08:20:20.450 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:20.450 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":30,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:19:45.885827+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:19:02.627691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-08T08:19:03.473821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T08:19:06.539942+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.37","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:20:20.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.449+0000 7f20b61e1700 1 -- 172.21.15.40:0/733583661 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2094064ee0 msgr2=0x7f2094067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:20.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.449+0000 7f20b61e1700 1 --2- 172.21.15.40:0/733583661 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2094064ee0 0x7f2094067390 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f20b0070e70 tx=0x7f20ac008030 comp rx=0 tx=0).stop 2023-11-08T08:20:20.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.449+0000 7f20b61e1700 1 -- 172.21.15.40:0/733583661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b0071aa0 msgr2=0x7f20b01e42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:20.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.449+0000 7f20b61e1700 1 --2- 172.21.15.40:0/733583661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b0071aa0 0x7f20b01e42e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f20a000c770 tx=0x7f20a000cb30 comp rx=0 tx=0).stop 2023-11-08T08:20:20.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.449+0000 7f20b61e1700 1 -- 172.21.15.40:0/733583661 shutdown_connections 2023-11-08T08:20:20.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.449+0000 7f20b61e1700 1 --2- 172.21.15.40:0/733583661 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2094064ee0 0x7f2094067390 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.449+0000 7f20b61e1700 1 --2- 172.21.15.40:0/733583661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b015c640 0x7f20b01e4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.449+0000 7f20b61e1700 1 --2- 172.21.15.40:0/733583661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b0071aa0 0x7f20b01e42e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.449+0000 7f20b61e1700 1 --2- 172.21.15.40:0/733583661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b0069e00 0x7f20b01e3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.449+0000 7f20b61e1700 1 -- 172.21.15.40:0/733583661 >> 172.21.15.40:0/733583661 conn(0x7f20b00b6520 msgr2=0x7f20b00b3670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:20.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.449+0000 7f20b61e1700 1 -- 172.21.15.40:0/733583661 shutdown_connections 2023-11-08T08:20:20.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.449+0000 7f20b61e1700 1 -- 172.21.15.40:0/733583661 wait complete. 2023-11-08T08:20:20.475 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T08:20:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.673+0000 7fc68934f700 1 Processor -- start 2023-11-08T08:20:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.673+0000 7fc68934f700 1 -- start start 2023-11-08T08:20:20.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.673+0000 7fc68934f700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc68415c640 0x7fc6840d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:20.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.673+0000 7fc68934f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc684071aa0 0x7fc68406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:20.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.673+0000 7fc68934f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc684069e00 0x7fc68406a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:20.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.673+0000 7fc68934f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc68406cef0 con 0x7fc684069e00 2023-11-08T08:20:20.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.673+0000 7fc68934f700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc68406a7c0 con 0x7fc68415c640 2023-11-08T08:20:20.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.673+0000 7fc68934f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68406c600 con 0x7fc684071aa0 2023-11-08T08:20:20.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.673+0000 7fc6827fc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc684071aa0 0x7fc68406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:20.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.673+0000 7fc6837fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc684069e00 0x7fc68406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:20.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.673+0000 7fc6827fc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc684071aa0 0x7fc68406d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:33848/0 (socket says 172.21.15.40:33848) 2023-11-08T08:20:20.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.673+0000 7fc682ffd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc68415c640 0x7fc6840d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:20.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.673+0000 7fc6827fc700 1 -- 172.21.15.40:0/552747283 learned_addr learned my addr 172.21.15.40:0/552747283 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:20.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.677+0000 7fc6827fc700 1 -- 172.21.15.40:0/552747283 >> v2:172.21.15.169:3300/0 conn(0x7fc68415c640 msgr2=0x7fc6840d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:20.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.677+0000 7fc6827fc700 1 --2- 172.21.15.40:0/552747283 >> v2:172.21.15.169:3300/0 conn(0x7fc68415c640 0x7fc6840d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.677+0000 7fc6827fc700 1 -- 172.21.15.40:0/552747283 >> v2:172.21.15.40:3300/0 conn(0x7fc684069e00 msgr2=0x7fc68406a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:20.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.677+0000 7fc6827fc700 1 --2- 172.21.15.40:0/552747283 >> v2:172.21.15.40:3300/0 conn(0x7fc684069e00 0x7fc68406a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.677+0000 7fc6827fc700 1 -- 172.21.15.40:0/552747283 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6840722f0 con 0x7fc684071aa0 2023-11-08T08:20:20.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.677+0000 7fc6827fc700 1 --2- 172.21.15.40:0/552747283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc684071aa0 0x7fc68406d310 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc68406e290 tx=0x7fc678009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=333e9299551ba4bc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:20.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.677+0000 7fc681ffb700 1 -- 172.21.15.40:0/552747283 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678011830 con 0x7fc684071aa0 2023-11-08T08:20:20.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.677+0000 7fc681ffb700 1 -- 172.21.15.40:0/552747283 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc678011990 con 0x7fc684071aa0 2023-11-08T08:20:20.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.677+0000 7fc681ffb700 1 -- 172.21.15.40:0/552747283 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678011d20 con 0x7fc684071aa0 2023-11-08T08:20:20.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.677+0000 7fc68934f700 1 -- 172.21.15.40:0/552747283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc684071aa0 msgr2=0x7fc68406d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:20.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.677+0000 7fc68934f700 1 --2- 172.21.15.40:0/552747283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc684071aa0 0x7fc68406d310 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc68406e290 tx=0x7fc678009bf0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.677+0000 7fc68934f700 1 -- 172.21.15.40:0/552747283 shutdown_connections 2023-11-08T08:20:20.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.677+0000 7fc68934f700 1 --2- 172.21.15.40:0/552747283 >> v2:172.21.15.40:3300/0 conn(0x7fc684069e00 0x7fc68406a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.677+0000 7fc68934f700 1 --2- 172.21.15.40:0/552747283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc684071aa0 0x7fc68406d310 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.677+0000 7fc68934f700 1 --2- 172.21.15.40:0/552747283 >> v2:172.21.15.169:3300/0 conn(0x7fc68415c640 0x7fc6840d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.677+0000 7fc68934f700 1 -- 172.21.15.40:0/552747283 >> 172.21.15.40:0/552747283 conn(0x7fc6840b6520 msgr2=0x7fc684071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:20.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.677+0000 7fc68934f700 1 -- 172.21.15.40:0/552747283 shutdown_connections 2023-11-08T08:20:20.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.677+0000 7fc68934f700 1 -- 172.21.15.40:0/552747283 wait complete. 2023-11-08T08:20:20.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.681+0000 7fc68934f700 1 Processor -- start 2023-11-08T08:20:20.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.681+0000 7fc68934f700 1 -- start start 2023-11-08T08:20:20.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc68934f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc684069e00 0x7fc6840cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:20.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc68934f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc684071aa0 0x7fc6840ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:20.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc68934f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc68415c640 0x7fc6841f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:20.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc68934f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840cd320 con 0x7fc684071aa0 2023-11-08T08:20:20.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc68934f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840cd530 con 0x7fc68415c640 2023-11-08T08:20:20.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc68934f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840cd810 con 0x7fc684069e00 2023-11-08T08:20:20.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc6827fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc684071aa0 0x7fc6840ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:20.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc6827fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc684071aa0 0x7fc6840ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45536/0 (socket says 172.21.15.40:45536) 2023-11-08T08:20:20.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc6827fc700 1 -- 172.21.15.40:0/1620038184 learned_addr learned my addr 172.21.15.40:0/1620038184 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:20.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc682ffd700 1 --2- 172.21.15.40:0/1620038184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc684069e00 0x7fc6840cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:20.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc6837fe700 1 --2- 172.21.15.40:0/1620038184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc68415c640 0x7fc6841f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:20.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc6827fc700 1 -- 172.21.15.40:0/1620038184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc684069e00 msgr2=0x7fc6840cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:20.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc6827fc700 1 --2- 172.21.15.40:0/1620038184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc684069e00 0x7fc6840cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc6827fc700 1 -- 172.21.15.40:0/1620038184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc68415c640 msgr2=0x7fc6841f5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:20.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc6827fc700 1 --2- 172.21.15.40:0/1620038184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc68415c640 0x7fc6841f5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:20.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc6827fc700 1 -- 172.21.15.40:0/1620038184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc678009570 con 0x7fc684071aa0 2023-11-08T08:20:20.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc6827fc700 1 --2- 172.21.15.40:0/1620038184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc684071aa0 0x7fc6840ccde0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc678014040 tx=0x7fc678004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:20.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc66b7fe700 1 -- 172.21.15.40:0/1620038184 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678004450 con 0x7fc684071aa0 2023-11-08T08:20:20.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc66b7fe700 1 -- 172.21.15.40:0/1620038184 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67801e070 con 0x7fc684071aa0 2023-11-08T08:20:20.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc68934f700 1 -- 172.21.15.40:0/1620038184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc684072010 con 0x7fc684071aa0 2023-11-08T08:20:20.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.685+0000 7fc66b7fe700 1 -- 172.21.15.40:0/1620038184 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678003900 con 0x7fc684071aa0 2023-11-08T08:20:20.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.689+0000 7fc68934f700 1 -- 172.21.15.40:0/1620038184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68415ca10 con 0x7fc684071aa0 2023-11-08T08:20:20.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.689+0000 7fc68934f700 1 -- 172.21.15.40:0/1620038184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc68406c830 con 0x7fc684071aa0 2023-11-08T08:20:20.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.689+0000 7fc66b7fe700 1 -- 172.21.15.40:0/1620038184 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc67800f050 con 0x7fc684071aa0 2023-11-08T08:20:20.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.693+0000 7fc66b7fe700 1 --2- 172.21.15.40:0/1620038184 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc670064ee0 0x7fc670067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:20.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.693+0000 7fc66b7fe700 1 -- 172.21.15.40:0/1620038184 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc678082ef0 con 0x7fc684071aa0 2023-11-08T08:20:20.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.693+0000 7fc682ffd700 1 --2- 172.21.15.40:0/1620038184 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc670064ee0 0x7fc670067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:20.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.697+0000 7fc682ffd700 1 --2- 172.21.15.40:0/1620038184 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc670064ee0 0x7fc670067390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc674009bf0 tx=0x7fc674009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:20.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.701+0000 7fc66b7fe700 1 -- 172.21.15.40:0/1620038184 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc678050cb0 con 0x7fc684071aa0 2023-11-08T08:20:20.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:20.909+0000 7fc68934f700 1 -- 172.21.15.40:0/1620038184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc684073cb0 con 0x7fc684071aa0 2023-11-08T08:20:21.150 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-11-08T08:20:21.145+0000 7f513abf9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T08:20:21.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.157+0000 7fc66b7fe700 1 -- 172.21.15.40:0/1620038184 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc678020090 con 0x7fc684071aa0 2023-11-08T08:20:21.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.161+0000 7fc68934f700 1 -- 172.21.15.40:0/1620038184 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc670064ee0 msgr2=0x7fc670067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:21.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.161+0000 7fc68934f700 1 --2- 172.21.15.40:0/1620038184 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc670064ee0 0x7fc670067390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc674009bf0 tx=0x7fc674009240 comp rx=0 tx=0).stop 2023-11-08T08:20:21.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.161+0000 7fc68934f700 1 -- 172.21.15.40:0/1620038184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc684071aa0 msgr2=0x7fc6840ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:21.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.161+0000 7fc68934f700 1 --2- 172.21.15.40:0/1620038184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc684071aa0 0x7fc6840ccde0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc678014040 tx=0x7fc678004770 comp rx=0 tx=0).stop 2023-11-08T08:20:21.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.161+0000 7fc68934f700 1 -- 172.21.15.40:0/1620038184 shutdown_connections 2023-11-08T08:20:21.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.161+0000 7fc68934f700 1 --2- 172.21.15.40:0/1620038184 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc670064ee0 0x7fc670067390 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:21.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.161+0000 7fc68934f700 1 --2- 172.21.15.40:0/1620038184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc68415c640 0x7fc6841f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:21.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.161+0000 7fc68934f700 1 --2- 172.21.15.40:0/1620038184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc684071aa0 0x7fc6840ccde0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:21.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.161+0000 7fc68934f700 1 --2- 172.21.15.40:0/1620038184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc684069e00 0x7fc6840cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:21.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.161+0000 7fc68934f700 1 -- 172.21.15.40:0/1620038184 >> 172.21.15.40:0/1620038184 conn(0x7fc6840b6520 msgr2=0x7fc68406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:21.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.161+0000 7fc68934f700 1 -- 172.21.15.40:0/1620038184 shutdown_connections 2023-11-08T08:20:21.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.161+0000 7fc68934f700 1 -- 172.21.15.40:0/1620038184 wait complete. 2023-11-08T08:20:21.171 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T08:20:21.180 INFO:tasks.ceph.mds.d.smithi169.stderr: -5153> 2023-11-08T08:19:09.960+0000 7fabd785f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T08:20:21.192 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T08:20:21.242 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2023-11-08T08:20:21.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.393+0000 7f622d098700 1 Processor -- start 2023-11-08T08:20:21.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.393+0000 7f622d098700 1 -- start start 2023-11-08T08:20:21.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.393+0000 7f622d098700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f62280cccf0 0x7f62280cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:21.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.393+0000 7f622d098700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6228159c20 con 0x7f6228071780 2023-11-08T08:20:21.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.393+0000 7f622d098700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f622806d1f0 con 0x7f622806cd70 2023-11-08T08:20:21.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.393+0000 7f622d098700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62280cd6b0 con 0x7f62280cccf0 2023-11-08T08:20:21.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.393+0000 7f6227fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6228071780 0x7f6228159e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37970/0 (socket says 172.21.15.40:37970) 2023-11-08T08:20:21.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.393+0000 7f6227fff700 1 -- 172.21.15.40:0/964943542 learned_addr learned my addr 172.21.15.40:0/964943542 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:21.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.393+0000 7f622c897700 1 --2- 172.21.15.40:0/964943542 >> v2:172.21.15.40:3300/0 conn(0x7f62280cccf0 0x7f62280cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.397+0000 7f6226ffd700 1 -- 172.21.15.40:0/964943542 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1560757974 0 0) 0x7f6228159c20 con 0x7f6228071780 2023-11-08T08:20:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.397+0000 7f6226ffd700 1 -- 172.21.15.40:0/964943542 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62100035d0 con 0x7f6228071780 2023-11-08T08:20:21.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.397+0000 7f622c897700 1 -- 172.21.15.40:0/964943542 >> v1:172.21.15.169:6789/0 conn(0x7f622806cd70 legacy=0x7f6228060ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:21.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.397+0000 7f622c897700 1 -- 172.21.15.40:0/964943542 >> v1:172.21.15.40:6789/0 conn(0x7f6228071780 legacy=0x7f6228159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:21.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.397+0000 7f622c897700 1 -- 172.21.15.40:0/964943542 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62281590a0 con 0x7f62280cccf0 2023-11-08T08:20:21.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.397+0000 7f622c897700 1 --2- 172.21.15.40:0/964943542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62280cccf0 0x7f62280cd170 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6220009d90 tx=0x7f622000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245e4df4c5274ec7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:21.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.397+0000 7f6226ffd700 1 -- 172.21.15.40:0/964943542 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622000e070 con 0x7f62280cccf0 2023-11-08T08:20:21.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.397+0000 7f6226ffd700 1 -- 172.21.15.40:0/964943542 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622000bb20 con 0x7f62280cccf0 2023-11-08T08:20:21.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.397+0000 7f6226ffd700 1 -- 172.21.15.40:0/964943542 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6220012430 con 0x7f62280cccf0 2023-11-08T08:20:21.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.397+0000 7f622d098700 1 -- 172.21.15.40:0/964943542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62280cccf0 msgr2=0x7f62280cd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:21.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.397+0000 7f622d098700 1 --2- 172.21.15.40:0/964943542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62280cccf0 0x7f62280cd170 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6220009d90 tx=0x7f622000b330 comp rx=0 tx=0).stop 2023-11-08T08:20:21.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.397+0000 7f622d098700 1 -- 172.21.15.40:0/964943542 shutdown_connections 2023-11-08T08:20:21.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.397+0000 7f622d098700 1 --2- 172.21.15.40:0/964943542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62280cccf0 0x7f62280cd170 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:21.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.397+0000 7f622d098700 1 -- 172.21.15.40:0/964943542 >> 172.21.15.40:0/964943542 conn(0x7f62280b0a20 msgr2=0x7f62280b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:21.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.397+0000 7f622d098700 1 -- 172.21.15.40:0/964943542 shutdown_connections 2023-11-08T08:20:21.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.397+0000 7f622d098700 1 -- 172.21.15.40:0/964943542 wait complete. 2023-11-08T08:20:21.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.401+0000 7f622d098700 1 Processor -- start 2023-11-08T08:20:21.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.401+0000 7f622d098700 1 -- start start 2023-11-08T08:20:21.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.401+0000 7f622d098700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f622806cd70 0x7f62280cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:21.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.401+0000 7f622d098700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6228071780 0x7f62280ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:21.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.401+0000 7f622d098700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62280cccf0 0x7f62281f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:21.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.401+0000 7f622d098700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6228144ad0 con 0x7f6228071780 2023-11-08T08:20:21.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.401+0000 7f622d098700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6228143e50 con 0x7f62280cccf0 2023-11-08T08:20:21.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.401+0000 7f622d098700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6228144130 con 0x7f622806cd70 2023-11-08T08:20:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.405+0000 7f62277fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6228071780 0x7f62280ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.405+0000 7f62277fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6228071780 0x7f62280ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45556/0 (socket says 172.21.15.40:45556) 2023-11-08T08:20:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.405+0000 7f62277fe700 1 -- 172.21.15.40:0/3047912046 learned_addr learned my addr 172.21.15.40:0/3047912046 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.405+0000 7f622c897700 1 --2- 172.21.15.40:0/3047912046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62280cccf0 0x7f62281f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.405+0000 7f6227fff700 1 --2- 172.21.15.40:0/3047912046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f622806cd70 0x7f62280cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.405+0000 7f62277fe700 1 -- 172.21.15.40:0/3047912046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f622806cd70 msgr2=0x7f62280cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.405+0000 7f62277fe700 1 --2- 172.21.15.40:0/3047912046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f622806cd70 0x7f62280cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.405+0000 7f62277fe700 1 -- 172.21.15.40:0/3047912046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62280cccf0 msgr2=0x7f62281f16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.405+0000 7f62277fe700 1 --2- 172.21.15.40:0/3047912046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62280cccf0 0x7f62281f16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.405+0000 7f62277fe700 1 -- 172.21.15.40:0/3047912046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6220009a70 con 0x7f6228071780 2023-11-08T08:20:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.405+0000 7f622c897700 1 --2- 172.21.15.40:0/3047912046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62280cccf0 0x7f62281f16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.405+0000 7f62277fe700 1 --2- 172.21.15.40:0/3047912046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6228071780 0x7f62280ca660 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f621c00c500 tx=0x7f621c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:21.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.405+0000 7f6224ff9700 1 -- 172.21.15.40:0/3047912046 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c0052d0 con 0x7f6228071780 2023-11-08T08:20:21.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.405+0000 7f6224ff9700 1 -- 172.21.15.40:0/3047912046 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621c005430 con 0x7f6228071780 2023-11-08T08:20:21.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.405+0000 7f622d098700 1 -- 172.21.15.40:0/3047912046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6228159660 con 0x7f6228071780 2023-11-08T08:20:21.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.405+0000 7f6224ff9700 1 -- 172.21.15.40:0/3047912046 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c0055a0 con 0x7f6228071780 2023-11-08T08:20:21.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.405+0000 7f622d098700 1 -- 172.21.15.40:0/3047912046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6228159380 con 0x7f6228071780 2023-11-08T08:20:21.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.409+0000 7f622d098700 1 -- 172.21.15.40:0/3047912046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6228159940 con 0x7f6228071780 2023-11-08T08:20:21.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.409+0000 7f6224ff9700 1 -- 172.21.15.40:0/3047912046 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f621c019030 con 0x7f6228071780 2023-11-08T08:20:21.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.409+0000 7f6224ff9700 1 --2- 172.21.15.40:0/3047912046 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6214064ee0 0x7f6214067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:21.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.409+0000 7f6224ff9700 1 -- 172.21.15.40:0/3047912046 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f621c0814d0 con 0x7f6228071780 2023-11-08T08:20:21.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.409+0000 7f6227fff700 1 --2- 172.21.15.40:0/3047912046 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6214064ee0 0x7f6214067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:21.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.413+0000 7f6227fff700 1 --2- 172.21.15.40:0/3047912046 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6214064ee0 0x7f6214067390 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6218007fd0 tx=0x7f6218007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:21.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.417+0000 7f6224ff9700 1 -- 172.21.15.40:0/3047912046 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f621c04f200 con 0x7f6228071780 2023-11-08T08:20:21.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:21.625+0000 7f622d098700 1 -- 172.21.15.40:0/3047912046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6228144c10 con 0x7f6228071780 2023-11-08T08:20:22.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.161+0000 7f6224ff9700 1 -- 172.21.15.40:0/3047912046 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6228144c10 con 0x7f6228071780 2023-11-08T08:20:22.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.165+0000 7f622d098700 1 -- 172.21.15.40:0/3047912046 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6214064ee0 msgr2=0x7f6214067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.165+0000 7f622d098700 1 --2- 172.21.15.40:0/3047912046 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6214064ee0 0x7f6214067390 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6218007fd0 tx=0x7f6218007de0 comp rx=0 tx=0).stop 2023-11-08T08:20:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.165+0000 7f622d098700 1 -- 172.21.15.40:0/3047912046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6228071780 msgr2=0x7f62280ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.165+0000 7f622d098700 1 --2- 172.21.15.40:0/3047912046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6228071780 0x7f62280ca660 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f621c00c500 tx=0x7f621c00c810 comp rx=0 tx=0).stop 2023-11-08T08:20:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.165+0000 7f622d098700 1 -- 172.21.15.40:0/3047912046 shutdown_connections 2023-11-08T08:20:22.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.165+0000 7f622d098700 1 --2- 172.21.15.40:0/3047912046 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6214064ee0 0x7f6214067390 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:22.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.165+0000 7f622d098700 1 --2- 172.21.15.40:0/3047912046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62280cccf0 0x7f62281f16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:22.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.165+0000 7f622d098700 1 --2- 172.21.15.40:0/3047912046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6228071780 0x7f62280ca660 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:22.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.165+0000 7f622d098700 1 --2- 172.21.15.40:0/3047912046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f622806cd70 0x7f62280cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:22.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.165+0000 7f622d098700 1 -- 172.21.15.40:0/3047912046 >> 172.21.15.40:0/3047912046 conn(0x7f62280b0a20 msgr2=0x7f6228155170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.165+0000 7f622d098700 1 -- 172.21.15.40:0/3047912046 shutdown_connections 2023-11-08T08:20:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.165+0000 7f622d098700 1 -- 172.21.15.40:0/3047912046 wait complete. 2023-11-08T08:20:22.199 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T08:20:22.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.377+0000 7f7cd211e700 1 Processor -- start 2023-11-08T08:20:22.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.377+0000 7f7cd211e700 1 -- start start 2023-11-08T08:20:22.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.377+0000 7f7cd211e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7ccc0b0d30 0x7f7ccc140ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:22.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.377+0000 7f7cd211e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7ccc15e7c0 0x7f7ccc0cac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:22.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.377+0000 7f7cd211e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ccc141d60 con 0x7f7ccc141220 2023-11-08T08:20:22.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.377+0000 7f7cd211e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ccc15ec40 con 0x7f7ccc15e7c0 2023-11-08T08:20:22.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.377+0000 7f7cd211e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ccc141670 con 0x7f7ccc0b0d30 2023-11-08T08:20:22.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.377+0000 7f7cc3fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7ccc141220 0x7f7ccc15e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37984/0 (socket says 172.21.15.40:37984) 2023-11-08T08:20:22.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.377+0000 7f7cc3fff700 1 -- 172.21.15.40:0/572518847 learned_addr learned my addr 172.21.15.40:0/572518847 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:22.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.377+0000 7f7ccbfff700 1 --2- 172.21.15.40:0/572518847 >> v2:172.21.15.40:3300/0 conn(0x7f7ccc15e7c0 0x7f7ccc0cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:22.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.377+0000 7f7ccb7fe700 1 --2- 172.21.15.40:0/572518847 >> v2:172.21.15.169:3300/0 conn(0x7f7ccc0b0d30 0x7f7ccc140ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:22.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.381+0000 7f7ccaffd700 1 -- 172.21.15.40:0/572518847 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3653111701 0 0) 0x7f7ccc141d60 con 0x7f7ccc141220 2023-11-08T08:20:22.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.381+0000 7f7ccaffd700 1 -- 172.21.15.40:0/572518847 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cb80035d0 con 0x7f7ccc141220 2023-11-08T08:20:22.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.381+0000 7f7ccbfff700 1 -- 172.21.15.40:0/572518847 >> v2:172.21.15.169:3300/0 conn(0x7f7ccc0b0d30 msgr2=0x7f7ccc140ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:22.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.381+0000 7f7ccbfff700 1 --2- 172.21.15.40:0/572518847 >> v2:172.21.15.169:3300/0 conn(0x7f7ccc0b0d30 0x7f7ccc140ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:22.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.381+0000 7f7ccbfff700 1 -- 172.21.15.40:0/572518847 >> v1:172.21.15.40:6789/0 conn(0x7f7ccc141220 legacy=0x7f7ccc15e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:22.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.381+0000 7f7ccbfff700 1 -- 172.21.15.40:0/572518847 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc15a790 con 0x7f7ccc15e7c0 2023-11-08T08:20:22.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.381+0000 7f7ccbfff700 1 --2- 172.21.15.40:0/572518847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ccc15e7c0 0x7f7ccc0cac00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc06cfa0 tx=0x7f7cc400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1cad4cb6fafc5f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:22.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.381+0000 7f7ccaffd700 1 -- 172.21.15.40:0/572518847 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc400e070 con 0x7f7ccc15e7c0 2023-11-08T08:20:22.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.381+0000 7f7ccaffd700 1 -- 172.21.15.40:0/572518847 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc400bb20 con 0x7f7ccc15e7c0 2023-11-08T08:20:22.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.381+0000 7f7ccaffd700 1 -- 172.21.15.40:0/572518847 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc400be90 con 0x7f7ccc15e7c0 2023-11-08T08:20:22.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.381+0000 7f7cd211e700 1 -- 172.21.15.40:0/572518847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ccc15e7c0 msgr2=0x7f7ccc0cac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:22.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.381+0000 7f7cd211e700 1 --2- 172.21.15.40:0/572518847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ccc15e7c0 0x7f7ccc0cac00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc06cfa0 tx=0x7f7cc400b330 comp rx=0 tx=0).stop 2023-11-08T08:20:22.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.381+0000 7f7cd211e700 1 -- 172.21.15.40:0/572518847 shutdown_connections 2023-11-08T08:20:22.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.381+0000 7f7cd211e700 1 --2- 172.21.15.40:0/572518847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ccc15e7c0 0x7f7ccc0cac00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:22.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.381+0000 7f7cd211e700 1 --2- 172.21.15.40:0/572518847 >> v2:172.21.15.169:3300/0 conn(0x7f7ccc0b0d30 0x7f7ccc140ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:22.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.381+0000 7f7cd211e700 1 -- 172.21.15.40:0/572518847 >> 172.21.15.40:0/572518847 conn(0x7f7ccc06bfc0 msgr2=0x7f7ccc06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:22.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.381+0000 7f7cd211e700 1 -- 172.21.15.40:0/572518847 shutdown_connections 2023-11-08T08:20:22.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.381+0000 7f7cd211e700 1 -- 172.21.15.40:0/572518847 wait complete. 2023-11-08T08:20:22.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.385+0000 7f7cd211e700 1 Processor -- start 2023-11-08T08:20:22.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.385+0000 7f7cd211e700 1 -- start start 2023-11-08T08:20:22.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.385+0000 7f7cd211e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ccc0b0d30 0x7f7ccc0c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:22.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.385+0000 7f7cd211e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ccc141220 0x7f7ccc158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:22.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.385+0000 7f7cd211e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ccc15e7c0 0x7f7ccc158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:22.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.385+0000 7f7cd211e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0ce4b0 con 0x7f7ccc15e7c0 2023-11-08T08:20:22.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.385+0000 7f7cd211e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0ce790 con 0x7f7ccc0b0d30 2023-11-08T08:20:22.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.385+0000 7f7cd211e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0cea70 con 0x7f7ccc141220 2023-11-08T08:20:22.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.389+0000 7f7ccbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ccc15e7c0 0x7f7ccc158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:22.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.389+0000 7f7ccbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ccc15e7c0 0x7f7ccc158680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45580/0 (socket says 172.21.15.40:45580) 2023-11-08T08:20:22.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.389+0000 7f7ccbfff700 1 -- 172.21.15.40:0/3352064528 learned_addr learned my addr 172.21.15.40:0/3352064528 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:22.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.389+0000 7f7ccb7fe700 1 --2- 172.21.15.40:0/3352064528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ccc0b0d30 0x7f7ccc0c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:22.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.389+0000 7f7cc3fff700 1 --2- 172.21.15.40:0/3352064528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ccc141220 0x7f7ccc158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:22.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.389+0000 7f7ccbfff700 1 -- 172.21.15.40:0/3352064528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ccc141220 msgr2=0x7f7ccc158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:22.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.389+0000 7f7ccbfff700 1 --2- 172.21.15.40:0/3352064528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ccc141220 0x7f7ccc158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:22.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.389+0000 7f7ccbfff700 1 -- 172.21.15.40:0/3352064528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ccc0b0d30 msgr2=0x7f7ccc0c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:22.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.389+0000 7f7ccbfff700 1 --2- 172.21.15.40:0/3352064528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ccc0b0d30 0x7f7ccc0c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:22.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.389+0000 7f7ccbfff700 1 -- 172.21.15.40:0/3352064528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc4009a70 con 0x7f7ccc15e7c0 2023-11-08T08:20:22.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.389+0000 7f7ccbfff700 1 --2- 172.21.15.40:0/3352064528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ccc15e7c0 0x7f7ccc158680 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7cc400b370 tx=0x7f7cc4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:22.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.389+0000 7f7cc8ff9700 1 -- 172.21.15.40:0/3352064528 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4004880 con 0x7f7ccc15e7c0 2023-11-08T08:20:22.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.389+0000 7f7cc8ff9700 1 -- 172.21.15.40:0/3352064528 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc4020070 con 0x7f7ccc15e7c0 2023-11-08T08:20:22.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.389+0000 7f7cc8ff9700 1 -- 172.21.15.40:0/3352064528 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc40276c0 con 0x7f7ccc15e7c0 2023-11-08T08:20:22.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.389+0000 7f7cd211e700 1 -- 172.21.15.40:0/3352064528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ccc15ad30 con 0x7f7ccc15e7c0 2023-11-08T08:20:22.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.389+0000 7f7cd211e700 1 -- 172.21.15.40:0/3352064528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ccc15aa50 con 0x7f7ccc15e7c0 2023-11-08T08:20:22.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.393+0000 7f7cd211e700 1 -- 172.21.15.40:0/3352064528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ccc15b010 con 0x7f7ccc15e7c0 2023-11-08T08:20:22.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.393+0000 7f7cc8ff9700 1 -- 172.21.15.40:0/3352064528 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7cc402e070 con 0x7f7ccc15e7c0 2023-11-08T08:20:22.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.393+0000 7f7cc8ff9700 1 --2- 172.21.15.40:0/3352064528 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7cac064ee0 0x7f7cac067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:22.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.393+0000 7f7cc8ff9700 1 -- 172.21.15.40:0/3352064528 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f7cc4022070 con 0x7f7ccc15e7c0 2023-11-08T08:20:22.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.393+0000 7f7ccb7fe700 1 --2- 172.21.15.40:0/3352064528 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7cac064ee0 0x7f7cac067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:22.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.397+0000 7f7ccb7fe700 1 --2- 172.21.15.40:0/3352064528 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7cac064ee0 0x7f7cac067390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc009bf0 tx=0x7f7cbc009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:22.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.401+0000 7f7cc8ff9700 1 -- 172.21.15.40:0/3352064528 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7cc4051bd0 con 0x7f7ccc15e7c0 2023-11-08T08:20:22.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:22.573+0000 7f7cd211e700 1 -- 172.21.15.40:0/3352064528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7ccc0726c0 con 0x7f7ccc15e7c0 2023-11-08T08:20:23.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.169+0000 7f7cc8ff9700 1 -- 172.21.15.40:0/3352064528 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v32) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7cc402c0d0 con 0x7f7ccc15e7c0 2023-11-08T08:20:23.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.177+0000 7f7cd211e700 1 -- 172.21.15.40:0/3352064528 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7cac064ee0 msgr2=0x7f7cac067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:23.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.177+0000 7f7cd211e700 1 --2- 172.21.15.40:0/3352064528 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7cac064ee0 0x7f7cac067390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc009bf0 tx=0x7f7cbc009240 comp rx=0 tx=0).stop 2023-11-08T08:20:23.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.177+0000 7f7cd211e700 1 -- 172.21.15.40:0/3352064528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ccc15e7c0 msgr2=0x7f7ccc158680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:23.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.177+0000 7f7cd211e700 1 --2- 172.21.15.40:0/3352064528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ccc15e7c0 0x7f7ccc158680 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7cc400b370 tx=0x7f7cc4004af0 comp rx=0 tx=0).stop 2023-11-08T08:20:23.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.177+0000 7f7cd211e700 1 -- 172.21.15.40:0/3352064528 shutdown_connections 2023-11-08T08:20:23.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.177+0000 7f7cd211e700 1 --2- 172.21.15.40:0/3352064528 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7cac064ee0 0x7f7cac067390 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:23.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.177+0000 7f7cd211e700 1 --2- 172.21.15.40:0/3352064528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ccc15e7c0 0x7f7ccc158680 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:23.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.177+0000 7f7cd211e700 1 --2- 172.21.15.40:0/3352064528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ccc141220 0x7f7ccc158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:23.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.177+0000 7f7cd211e700 1 --2- 172.21.15.40:0/3352064528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ccc0b0d30 0x7f7ccc0c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:23.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.177+0000 7f7cd211e700 1 -- 172.21.15.40:0/3352064528 >> 172.21.15.40:0/3352064528 conn(0x7f7ccc06bfc0 msgr2=0x7f7ccc06a970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:23.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.181+0000 7f7cd211e700 1 -- 172.21.15.40:0/3352064528 shutdown_connections 2023-11-08T08:20:23.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.181+0000 7f7cd211e700 1 -- 172.21.15.40:0/3352064528 wait complete. 2023-11-08T08:20:23.192 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2023-11-08T08:20:23.207 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T08:20:23.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.381+0000 7f1def59e700 1 Processor -- start 2023-11-08T08:20:23.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.385+0000 7f1def59e700 1 -- start start 2023-11-08T08:20:23.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.385+0000 7f1def59e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1df015c6e0 0x7f1df0071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:23.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.385+0000 7f1def59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df0071fe0 0x7f1df0069e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.385+0000 7f1def59e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df006a790 con 0x7f1df006a340 2023-11-08T08:20:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.385+0000 7f1def59e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df00d2800 con 0x7f1df015c6e0 2023-11-08T08:20:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.385+0000 7f1def59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df015cab0 con 0x7f1df0071fe0 2023-11-08T08:20:23.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.389+0000 7f1dee59c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1df015c6e0 0x7f1df0071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:23.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.389+0000 7f1dedd9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df0071fe0 0x7f1df0069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:23.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.389+0000 7f1dee59c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1df015c6e0 0x7f1df0071aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:37384/0 (socket says 172.21.15.40:37384) 2023-11-08T08:20:23.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.389+0000 7f1dedd9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df0071fe0 0x7f1df0069e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:33890/0 (socket says 172.21.15.40:33890) 2023-11-08T08:20:23.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.389+0000 7f1dee59c700 1 -- 172.21.15.40:0/103670168 learned_addr learned my addr 172.21.15.40:0/103670168 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:23.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.389+0000 7f1deed9d700 1 --1- 172.21.15.40:0/103670168 >> v1:172.21.15.169:6789/0 conn(0x7f1df006a340 0x7f1df00b79a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:52674/0 (socket says 172.21.15.40:52674) 2023-11-08T08:20:23.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.389+0000 7f1dee59c700 1 -- 172.21.15.40:0/103670168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df0071fe0 msgr2=0x7f1df0069e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:23.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.389+0000 7f1dee59c700 1 --2- 172.21.15.40:0/103670168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df0071fe0 0x7f1df0069e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:23.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.389+0000 7f1dee59c700 1 -- 172.21.15.40:0/103670168 >> v1:172.21.15.169:6789/0 conn(0x7f1df006a340 legacy=0x7f1df00b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.389+0000 7f1dee59c700 1 -- 172.21.15.40:0/103670168 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df00702d0 con 0x7f1df015c6e0 2023-11-08T08:20:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.389+0000 7f1dedd9b700 1 --2- 172.21.15.40:0/103670168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df0071fe0 0x7f1df0069e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.389+0000 7f1dee59c700 1 --2- 172.21.15.40:0/103670168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df015c6e0 0x7f1df0071aa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009910 tx=0x7f1de0009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6f2a9556ecf723f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.389+0000 7f1ded59a700 1 -- 172.21.15.40:0/103670168 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0011830 con 0x7f1df015c6e0 2023-11-08T08:20:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.389+0000 7f1ded59a700 1 -- 172.21.15.40:0/103670168 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de0011990 con 0x7f1df015c6e0 2023-11-08T08:20:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.389+0000 7f1ded59a700 1 -- 172.21.15.40:0/103670168 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0011d40 con 0x7f1df015c6e0 2023-11-08T08:20:23.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.393+0000 7f1def59e700 1 -- 172.21.15.40:0/103670168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df015c6e0 msgr2=0x7f1df0071aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:23.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.393+0000 7f1def59e700 1 --2- 172.21.15.40:0/103670168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df015c6e0 0x7f1df0071aa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009910 tx=0x7f1de0009c20 comp rx=0 tx=0).stop 2023-11-08T08:20:23.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.393+0000 7f1def59e700 1 -- 172.21.15.40:0/103670168 shutdown_connections 2023-11-08T08:20:23.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.393+0000 7f1def59e700 1 --2- 172.21.15.40:0/103670168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df0071fe0 0x7f1df0069e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:23.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.393+0000 7f1def59e700 1 --2- 172.21.15.40:0/103670168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df015c6e0 0x7f1df0071aa0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:23.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.393+0000 7f1def59e700 1 -- 172.21.15.40:0/103670168 >> 172.21.15.40:0/103670168 conn(0x7f1df00b65b0 msgr2=0x7f1df0071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:23.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.393+0000 7f1def59e700 1 -- 172.21.15.40:0/103670168 shutdown_connections 2023-11-08T08:20:23.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.393+0000 7f1def59e700 1 -- 172.21.15.40:0/103670168 wait complete. 2023-11-08T08:20:23.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.397+0000 7f1def59e700 1 Processor -- start 2023-11-08T08:20:23.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.397+0000 7f1def59e700 1 -- start start 2023-11-08T08:20:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.397+0000 7f1def59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df006a340 0x7f1df0153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1def59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1df0071fe0 0x7f1df00b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1def59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df015c6e0 0x7f1df00c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:23.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1def59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df01526a0 con 0x7f1df0071fe0 2023-11-08T08:20:23.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1def59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df009c2b0 con 0x7f1df015c6e0 2023-11-08T08:20:23.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1def59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df0151a20 con 0x7f1df006a340 2023-11-08T08:20:23.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1dedd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1df0071fe0 0x7f1df00b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:23.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1dedd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1df0071fe0 0x7f1df00b22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45592/0 (socket says 172.21.15.40:45592) 2023-11-08T08:20:23.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1dedd9b700 1 -- 172.21.15.40:0/4113776831 learned_addr learned my addr 172.21.15.40:0/4113776831 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:23.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1dee59c700 1 --2- 172.21.15.40:0/4113776831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df006a340 0x7f1df0153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:23.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1deed9d700 1 --2- 172.21.15.40:0/4113776831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df015c6e0 0x7f1df00c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:23.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1dedd9b700 1 -- 172.21.15.40:0/4113776831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df006a340 msgr2=0x7f1df0153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:23.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1dedd9b700 1 --2- 172.21.15.40:0/4113776831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df006a340 0x7f1df0153a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:23.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1dedd9b700 1 -- 172.21.15.40:0/4113776831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df015c6e0 msgr2=0x7f1df00c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:23.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1dedd9b700 1 --2- 172.21.15.40:0/4113776831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df015c6e0 0x7f1df00c73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:23.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1dedd9b700 1 -- 172.21.15.40:0/4113776831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de40094b0 con 0x7f1df0071fe0 2023-11-08T08:20:23.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1dedd9b700 1 --2- 172.21.15.40:0/4113776831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1df0071fe0 0x7f1df00b22f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1de400ebe0 tx=0x7f1de400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:23.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1ddeffd700 1 -- 172.21.15.40:0/4113776831 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de400c850 con 0x7f1df0071fe0 2023-11-08T08:20:23.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1ddeffd700 1 -- 172.21.15.40:0/4113776831 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de400c9b0 con 0x7f1df0071fe0 2023-11-08T08:20:23.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1def59e700 1 -- 172.21.15.40:0/4113776831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de0009570 con 0x7f1df0071fe0 2023-11-08T08:20:23.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1ddeffd700 1 -- 172.21.15.40:0/4113776831 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de400cd40 con 0x7f1df0071fe0 2023-11-08T08:20:23.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.401+0000 7f1def59e700 1 -- 172.21.15.40:0/4113776831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df00d2cf0 con 0x7f1df0071fe0 2023-11-08T08:20:23.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.405+0000 7f1def59e700 1 -- 172.21.15.40:0/4113776831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df00d2a10 con 0x7f1df0071fe0 2023-11-08T08:20:23.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.405+0000 7f1ddeffd700 1 -- 172.21.15.40:0/4113776831 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1de4019030 con 0x7f1df0071fe0 2023-11-08T08:20:23.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.409+0000 7f1ddeffd700 1 --2- 172.21.15.40:0/4113776831 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1dd0064ee0 0x7f1dd0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:23.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.409+0000 7f1ddeffd700 1 -- 172.21.15.40:0/4113776831 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f1de40820f0 con 0x7f1df0071fe0 2023-11-08T08:20:23.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.409+0000 7f1dee59c700 1 --2- 172.21.15.40:0/4113776831 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1dd0064ee0 0x7f1dd0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:23.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.409+0000 7f1dee59c700 1 --2- 172.21.15.40:0/4113776831 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1dd0064ee0 0x7f1dd0067390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1de0000c00 tx=0x7f1de0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:23.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.413+0000 7f1ddeffd700 1 -- 172.21.15.40:0/4113776831 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1de404ffd0 con 0x7f1df0071fe0 2023-11-08T08:20:23.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:23.593+0000 7f1def59e700 1 -- 172.21.15.40:0/4113776831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1df00d2fd0 con 0x7f1df0071fe0 2023-11-08T08:20:24.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.177+0000 7f1ddeffd700 1 -- 172.21.15.40:0/4113776831 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v33) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1de400cea0 con 0x7f1df0071fe0 2023-11-08T08:20:24.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.189+0000 7f1def59e700 1 -- 172.21.15.40:0/4113776831 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1dd0064ee0 msgr2=0x7f1dd0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:24.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.189+0000 7f1def59e700 1 --2- 172.21.15.40:0/4113776831 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1dd0064ee0 0x7f1dd0067390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1de0000c00 tx=0x7f1de0008030 comp rx=0 tx=0).stop 2023-11-08T08:20:24.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.189+0000 7f1def59e700 1 -- 172.21.15.40:0/4113776831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1df0071fe0 msgr2=0x7f1df00b22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:24.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.189+0000 7f1def59e700 1 --2- 172.21.15.40:0/4113776831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1df0071fe0 0x7f1df00b22f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1de400ebe0 tx=0x7f1de400eef0 comp rx=0 tx=0).stop 2023-11-08T08:20:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.189+0000 7f1def59e700 1 -- 172.21.15.40:0/4113776831 shutdown_connections 2023-11-08T08:20:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.189+0000 7f1def59e700 1 --2- 172.21.15.40:0/4113776831 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1dd0064ee0 0x7f1dd0067390 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.189+0000 7f1def59e700 1 --2- 172.21.15.40:0/4113776831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df015c6e0 0x7f1df00c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.189+0000 7f1def59e700 1 --2- 172.21.15.40:0/4113776831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1df0071fe0 0x7f1df00b22f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.189+0000 7f1def59e700 1 --2- 172.21.15.40:0/4113776831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df006a340 0x7f1df0153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.189+0000 7f1def59e700 1 -- 172.21.15.40:0/4113776831 >> 172.21.15.40:0/4113776831 conn(0x7f1df00b65b0 msgr2=0x7f1df0159610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.189+0000 7f1def59e700 1 -- 172.21.15.40:0/4113776831 shutdown_connections 2023-11-08T08:20:24.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.189+0000 7f1def59e700 1 -- 172.21.15.40:0/4113776831 wait complete. 2023-11-08T08:20:24.199 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2023-11-08T08:20:24.226 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-11-08T08:20:24.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.405+0000 7fe384a0a700 1 Processor -- start 2023-11-08T08:20:24.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.405+0000 7fe384a0a700 1 -- start start 2023-11-08T08:20:24.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.409+0000 7fe384a0a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe3800b6110 0x7fe380142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:24.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.409+0000 7fe384a0a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe38006aab0 0x7fe38006af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:24.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.409+0000 7fe384a0a700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe38006b8d0 con 0x7fe380142650 2023-11-08T08:20:24.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.409+0000 7fe384a0a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe38006bb70 con 0x7fe3800b6110 2023-11-08T08:20:24.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.409+0000 7fe384a0a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38006be50 con 0x7fe38006aab0 2023-11-08T08:20:24.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.409+0000 7fe37dd9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe3800b6110 0x7fe380142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:24.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.409+0000 7fe37dd9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe3800b6110 0x7fe380142fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45594/0 (socket says 172.21.15.40:45594) 2023-11-08T08:20:24.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.409+0000 7fe37dd9b700 1 -- 172.21.15.40:0/3435894735 learned_addr learned my addr 172.21.15.40:0/3435894735 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.409+0000 7fe37ed9d700 1 --2- 172.21.15.40:0/3435894735 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe38006aab0 0x7fe38006af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.409+0000 7fe37dd9b700 1 -- 172.21.15.40:0/3435894735 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe38006aab0 msgr2=0x7fe38006af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.409+0000 7fe37dd9b700 1 --2- 172.21.15.40:0/3435894735 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe38006aab0 0x7fe38006af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.409+0000 7fe37dd9b700 1 -- 172.21.15.40:0/3435894735 >> v1:172.21.15.169:6789/0 conn(0x7fe380142650 legacy=0x7fe3800b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.413+0000 7fe37dd9b700 1 -- 172.21.15.40:0/3435894735 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38006c130 con 0x7fe3800b6110 2023-11-08T08:20:24.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.413+0000 7fe37dd9b700 1 --2- 172.21.15.40:0/3435894735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3800b6110 0x7fe380142fa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe374009910 tx=0x7fe374009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecd8bd495f9d26a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:24.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.413+0000 7fe37d59a700 1 -- 172.21.15.40:0/3435894735 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37400ba60 con 0x7fe3800b6110 2023-11-08T08:20:24.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.413+0000 7fe37d59a700 1 -- 172.21.15.40:0/3435894735 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37400bbc0 con 0x7fe3800b6110 2023-11-08T08:20:24.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.413+0000 7fe37d59a700 1 -- 172.21.15.40:0/3435894735 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374009e60 con 0x7fe3800b6110 2023-11-08T08:20:24.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.413+0000 7fe384a0a700 1 -- 172.21.15.40:0/3435894735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3800b6110 msgr2=0x7fe380142fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:24.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.413+0000 7fe384a0a700 1 --2- 172.21.15.40:0/3435894735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3800b6110 0x7fe380142fa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe374009910 tx=0x7fe374009c20 comp rx=0 tx=0).stop 2023-11-08T08:20:24.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.413+0000 7fe384a0a700 1 -- 172.21.15.40:0/3435894735 shutdown_connections 2023-11-08T08:20:24.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.413+0000 7fe384a0a700 1 --2- 172.21.15.40:0/3435894735 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe38006aab0 0x7fe38006af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:24.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.413+0000 7fe384a0a700 1 --2- 172.21.15.40:0/3435894735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3800b6110 0x7fe380142fa0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:24.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.413+0000 7fe384a0a700 1 -- 172.21.15.40:0/3435894735 >> 172.21.15.40:0/3435894735 conn(0x7fe3800b5390 msgr2=0x7fe3800d2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:24.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.413+0000 7fe384a0a700 1 -- 172.21.15.40:0/3435894735 shutdown_connections 2023-11-08T08:20:24.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.413+0000 7fe384a0a700 1 -- 172.21.15.40:0/3435894735 wait complete. 2023-11-08T08:20:24.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.417+0000 7fe384a0a700 1 Processor -- start 2023-11-08T08:20:24.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.417+0000 7fe384a0a700 1 -- start start 2023-11-08T08:20:24.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe384a0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe38006aab0 0x7fe38015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:24.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe384a0a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3800b6110 0x7fe38015f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:24.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe384a0a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe380142650 0x7fe38015fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:24.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe384a0a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe380137050 con 0x7fe38006aab0 2023-11-08T08:20:24.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe384a0a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38015a6e0 con 0x7fe380142650 2023-11-08T08:20:24.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe384a0a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3801f5e10 con 0x7fe3800b6110 2023-11-08T08:20:24.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe37e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe38006aab0 0x7fe38015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:24.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe37e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe38006aab0 0x7fe38015a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45600/0 (socket says 172.21.15.40:45600) 2023-11-08T08:20:24.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe37e59c700 1 -- 172.21.15.40:0/1480515605 learned_addr learned my addr 172.21.15.40:0/1480515605 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:24.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe37dd9b700 1 --2- 172.21.15.40:0/1480515605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3800b6110 0x7fe38015f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:24.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe37ed9d700 1 --2- 172.21.15.40:0/1480515605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe380142650 0x7fe38015fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:24.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe37e59c700 1 -- 172.21.15.40:0/1480515605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3800b6110 msgr2=0x7fe38015f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:24.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe37e59c700 1 --2- 172.21.15.40:0/1480515605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3800b6110 0x7fe38015f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:24.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe37e59c700 1 -- 172.21.15.40:0/1480515605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe380142650 msgr2=0x7fe38015fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:24.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe37e59c700 1 --2- 172.21.15.40:0/1480515605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe380142650 0x7fe38015fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:24.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe37e59c700 1 -- 172.21.15.40:0/1480515605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe374009570 con 0x7fe38006aab0 2023-11-08T08:20:24.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe37ed9d700 1 --2- 172.21.15.40:0/1480515605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe380142650 0x7fe38015fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:20:24.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe37e59c700 1 --2- 172.21.15.40:0/1480515605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe38006aab0 0x7fe38015a1a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe37000c480 tx=0x7fe37000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:24.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe36effd700 1 -- 172.21.15.40:0/1480515605 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3700052d0 con 0x7fe38006aab0 2023-11-08T08:20:24.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe36effd700 1 -- 172.21.15.40:0/1480515605 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe370005430 con 0x7fe38006aab0 2023-11-08T08:20:24.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe384a0a700 1 -- 172.21.15.40:0/1480515605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe380142a20 con 0x7fe38006aab0 2023-11-08T08:20:24.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe36effd700 1 -- 172.21.15.40:0/1480515605 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3700055a0 con 0x7fe38006aab0 2023-11-08T08:20:24.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.421+0000 7fe384a0a700 1 -- 172.21.15.40:0/1480515605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38006c410 con 0x7fe38006aab0 2023-11-08T08:20:24.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.425+0000 7fe384a0a700 1 -- 172.21.15.40:0/1480515605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe38006b450 con 0x7fe38006aab0 2023-11-08T08:20:24.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.425+0000 7fe36effd700 1 -- 172.21.15.40:0/1480515605 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fe370019030 con 0x7fe38006aab0 2023-11-08T08:20:24.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.425+0000 7fe36effd700 1 --2- 172.21.15.40:0/1480515605 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe360064ee0 0x7fe360067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:24.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.429+0000 7fe36effd700 1 -- 172.21.15.40:0/1480515605 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6047+0+0 (secure 0 0 0) 0x7fe3700810f0 con 0x7fe38006aab0 2023-11-08T08:20:24.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.429+0000 7fe37dd9b700 1 --2- 172.21.15.40:0/1480515605 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe360064ee0 0x7fe360067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:24.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.429+0000 7fe37dd9b700 1 --2- 172.21.15.40:0/1480515605 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe360064ee0 0x7fe360067390 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe374009910 tx=0x7fe374008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:24.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.433+0000 7fe36effd700 1 -- 172.21.15.40:0/1480515605 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fe37004f2a0 con 0x7fe38006aab0 2023-11-08T08:20:24.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:24.613+0000 7fe384a0a700 1 -- 172.21.15.40:0/1480515605 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe3800726c0 con 0x7fe38006aab0 2023-11-08T08:20:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.197+0000 7fe36effd700 1 -- 172.21.15.40:0/1480515605 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v34) v1 ==== 178+0+0 (secure 0 0 0) 0x7fe3700528c0 con 0x7fe38006aab0 2023-11-08T08:20:25.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.205+0000 7fe384a0a700 1 -- 172.21.15.40:0/1480515605 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe360064ee0 msgr2=0x7fe360067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:25.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.205+0000 7fe384a0a700 1 --2- 172.21.15.40:0/1480515605 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe360064ee0 0x7fe360067390 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe374009910 tx=0x7fe374008030 comp rx=0 tx=0).stop 2023-11-08T08:20:25.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.205+0000 7fe384a0a700 1 -- 172.21.15.40:0/1480515605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe38006aab0 msgr2=0x7fe38015a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:25.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.205+0000 7fe384a0a700 1 --2- 172.21.15.40:0/1480515605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe38006aab0 0x7fe38015a1a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe37000c480 tx=0x7fe37000c790 comp rx=0 tx=0).stop 2023-11-08T08:20:25.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.205+0000 7fe384a0a700 1 -- 172.21.15.40:0/1480515605 shutdown_connections 2023-11-08T08:20:25.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.205+0000 7fe384a0a700 1 --2- 172.21.15.40:0/1480515605 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe360064ee0 0x7fe360067390 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:25.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.205+0000 7fe384a0a700 1 --2- 172.21.15.40:0/1480515605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe380142650 0x7fe38015fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:25.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.205+0000 7fe384a0a700 1 --2- 172.21.15.40:0/1480515605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3800b6110 0x7fe38015f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:25.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.205+0000 7fe384a0a700 1 --2- 172.21.15.40:0/1480515605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe38006aab0 0x7fe38015a1a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:25.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.205+0000 7fe384a0a700 1 -- 172.21.15.40:0/1480515605 >> 172.21.15.40:0/1480515605 conn(0x7fe3800b5390 msgr2=0x7fe3800b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:25.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.205+0000 7fe384a0a700 1 -- 172.21.15.40:0/1480515605 shutdown_connections 2023-11-08T08:20:25.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.205+0000 7fe384a0a700 1 -- 172.21.15.40:0/1480515605 wait complete. 2023-11-08T08:20:25.213 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data_ec' removed 2023-11-08T08:20:25.232 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T08:20:25.232 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T08:20:25.233 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T08:20:25.233 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:20:25.233 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T08:20:25.234 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T08:20:25.234 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T08:20:25.234 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:20:25.235 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T08:20:25.235 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T08:20:25.236 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T08:20:25.236 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:20:25.236 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T08:20:25.237 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T08:20:25.237 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T08:20:25.238 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:20:25.254 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T08:20:25.255 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T08:20:25.256 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T08:20:25.257 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T08:20:25.258 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T08:20:25.258 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T08:20:25.259 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T08:20:25.259 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T08:20:25.303 INFO:tasks.ceph.mds.b:Started 2023-11-08T08:20:25.303 DEBUG:teuthology.parallel:result is None 2023-11-08T08:20:25.306 INFO:tasks.ceph.mds.d:Started 2023-11-08T08:20:25.306 DEBUG:teuthology.parallel:result is None 2023-11-08T08:20:25.307 INFO:tasks.ceph.mds.c:Started 2023-11-08T08:20:25.307 DEBUG:teuthology.parallel:result is None 2023-11-08T08:20:25.310 INFO:tasks.ceph.mds.a:Started 2023-11-08T08:20:25.311 DEBUG:teuthology.parallel:result is None 2023-11-08T08:20:25.311 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-08T08:20:25.417 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2023-11-08T08:20:25.417 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2023-11-08T08:20:25.424 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-11-08T08:20:25.424 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2023-11-08T08:20:25.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.485+0000 7fbb2f085700 1 Processor -- start 2023-11-08T08:20:25.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.485+0000 7fbb2f085700 1 -- start start 2023-11-08T08:20:25.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.489+0000 7fbb2f085700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbb2806a300 0x7fbb28070a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:25.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.489+0000 7fbb2f085700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb280ce7a0 con 0x7fbb2815fbd0 2023-11-08T08:20:25.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.489+0000 7fbb2f085700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb280ccae0 con 0x7fbb280cbe70 2023-11-08T08:20:25.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.489+0000 7fbb2f085700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb28165160 con 0x7fbb2806a300 2023-11-08T08:20:25.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.489+0000 7fbb2e884700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbb2815fbd0 0x7fbb28160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38014/0 (socket says 172.21.15.40:38014) 2023-11-08T08:20:25.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.489+0000 7fbb2e884700 1 -- 172.21.15.40:0/671013350 learned_addr learned my addr 172.21.15.40:0/671013350 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:25.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.489+0000 7fbb2d882700 1 --2- 172.21.15.40:0/671013350 >> v2:172.21.15.169:3300/0 conn(0x7fbb2806a300 0x7fbb28070a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:25.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.489+0000 7fbb2d882700 1 -- 172.21.15.40:0/671013350 >> v1:172.21.15.169:6789/0 conn(0x7fbb280cbe70 legacy=0x7fbb28069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:25.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.489+0000 7fbb2d882700 1 -- 172.21.15.40:0/671013350 >> v1:172.21.15.40:6789/0 conn(0x7fbb2815fbd0 legacy=0x7fbb28160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:25.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.489+0000 7fbb2d882700 1 -- 172.21.15.40:0/671013350 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb280cec80 con 0x7fbb2806a300 2023-11-08T08:20:25.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.489+0000 7fbb2d081700 1 -- 172.21.15.40:0/671013350 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4041801456 0 0) 0x7fbb280ce7a0 con 0x7fbb2815fbd0 2023-11-08T08:20:25.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.489+0000 7fbb2d882700 1 --2- 172.21.15.40:0/671013350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb2806a300 0x7fbb28070a90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbb180097d0 tx=0x7fbb18009ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fca89f34d239f99f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:25.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.493+0000 7fbb2d081700 1 -- 172.21.15.40:0/671013350 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18004230 con 0x7fbb2806a300 2023-11-08T08:20:25.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.493+0000 7fbb2d081700 1 -- 172.21.15.40:0/671013350 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb18004390 con 0x7fbb2806a300 2023-11-08T08:20:25.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.493+0000 7fbb2d081700 1 -- 172.21.15.40:0/671013350 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb180036c0 con 0x7fbb2806a300 2023-11-08T08:20:25.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.493+0000 7fbb2f085700 1 -- 172.21.15.40:0/671013350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb2806a300 msgr2=0x7fbb28070a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:25.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.493+0000 7fbb2f085700 1 --2- 172.21.15.40:0/671013350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb2806a300 0x7fbb28070a90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbb180097d0 tx=0x7fbb18009ae0 comp rx=0 tx=0).stop 2023-11-08T08:20:25.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.493+0000 7fbb2f085700 1 -- 172.21.15.40:0/671013350 shutdown_connections 2023-11-08T08:20:25.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.493+0000 7fbb2f085700 1 --2- 172.21.15.40:0/671013350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb2806a300 0x7fbb28070a90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:25.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.493+0000 7fbb2f085700 1 -- 172.21.15.40:0/671013350 >> 172.21.15.40:0/671013350 conn(0x7fbb280b0a20 msgr2=0x7fbb280b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:25.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.493+0000 7fbb2f085700 1 -- 172.21.15.40:0/671013350 shutdown_connections 2023-11-08T08:20:25.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.493+0000 7fbb2f085700 1 -- 172.21.15.40:0/671013350 wait complete. 2023-11-08T08:20:25.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.493+0000 7fbb2f085700 1 Processor -- start 2023-11-08T08:20:25.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.493+0000 7fbb2f085700 1 -- start start 2023-11-08T08:20:25.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.497+0000 7fbb2f085700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb2806a300 0x7fbb281f5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:25.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.497+0000 7fbb2f085700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb280cbe70 0x7fbb28151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:25.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.497+0000 7fbb2f085700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2815fbd0 0x7fbb281604c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:25.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.497+0000 7fbb2f085700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb280c65d0 con 0x7fbb2815fbd0 2023-11-08T08:20:25.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.497+0000 7fbb2f085700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb280c7430 con 0x7fbb2806a300 2023-11-08T08:20:25.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.497+0000 7fbb2f085700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb280c6820 con 0x7fbb280cbe70 2023-11-08T08:20:25.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.497+0000 7fbb2e083700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb2806a300 0x7fbb281f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:25.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.497+0000 7fbb2e083700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb2806a300 0x7fbb281f5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:37444/0 (socket says 172.21.15.40:37444) 2023-11-08T08:20:25.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.497+0000 7fbb2e083700 1 -- 172.21.15.40:0/1553641502 learned_addr learned my addr 172.21.15.40:0/1553641502 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:25.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.497+0000 7fbb2e884700 1 --2- 172.21.15.40:0/1553641502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2815fbd0 0x7fbb281604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:25.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.497+0000 7fbb2d882700 1 --2- 172.21.15.40:0/1553641502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb280cbe70 0x7fbb28151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:25.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.497+0000 7fbb2d882700 1 -- 172.21.15.40:0/1553641502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb2806a300 msgr2=0x7fbb281f5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:25.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.497+0000 7fbb2d882700 1 --2- 172.21.15.40:0/1553641502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb2806a300 0x7fbb281f5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:25.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.497+0000 7fbb2d882700 1 -- 172.21.15.40:0/1553641502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2815fbd0 msgr2=0x7fbb281604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:25.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.497+0000 7fbb2d882700 1 --2- 172.21.15.40:0/1553641502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2815fbd0 0x7fbb281604c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:25.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.497+0000 7fbb2d882700 1 -- 172.21.15.40:0/1553641502 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb180094b0 con 0x7fbb280cbe70 2023-11-08T08:20:25.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.497+0000 7fbb2e083700 1 --2- 172.21.15.40:0/1553641502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb2806a300 0x7fbb281f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:25.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.497+0000 7fbb2d882700 1 --2- 172.21.15.40:0/1553641502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb280cbe70 0x7fbb28151fa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbb18013000 tx=0x7fbb18004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:25.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.497+0000 7fbb1effd700 1 -- 172.21.15.40:0/1553641502 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18011630 con 0x7fbb280cbe70 2023-11-08T08:20:25.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.501+0000 7fbb1effd700 1 -- 172.21.15.40:0/1553641502 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb180117b0 con 0x7fbb280cbe70 2023-11-08T08:20:25.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.501+0000 7fbb2f085700 1 -- 172.21.15.40:0/1553641502 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb280ccda0 con 0x7fbb280cbe70 2023-11-08T08:20:25.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.501+0000 7fbb1effd700 1 -- 172.21.15.40:0/1553641502 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18011b60 con 0x7fbb280cbe70 2023-11-08T08:20:25.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.501+0000 7fbb2f085700 1 -- 172.21.15.40:0/1553641502 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb28161a00 con 0x7fbb280cbe70 2023-11-08T08:20:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.501+0000 7fbb2f085700 1 -- 172.21.15.40:0/1553641502 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb280cc240 con 0x7fbb280cbe70 2023-11-08T08:20:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.505+0000 7fbb1effd700 1 -- 172.21.15.40:0/1553641502 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fbb1802d070 con 0x7fbb280cbe70 2023-11-08T08:20:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.505+0000 7fbb1effd700 1 --2- 172.21.15.40:0/1553641502 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbb0c064ee0 0x7fbb0c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.505+0000 7fbb1effd700 1 -- 172.21.15.40:0/1553641502 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fbb18082e10 con 0x7fbb280cbe70 2023-11-08T08:20:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.505+0000 7fbb2e083700 1 --2- 172.21.15.40:0/1553641502 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbb0c064ee0 0x7fbb0c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.505+0000 7fbb2e083700 1 --2- 172.21.15.40:0/1553641502 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbb0c064ee0 0x7fbb0c067390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400f220 tx=0x7fbb24009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:25.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.513+0000 7fbb1effd700 1 -- 172.21.15.40:0/1553641502 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fbb18051120 con 0x7fbb280cbe70 2023-11-08T08:20:25.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:25.713+0000 7fbb2f085700 1 -- 172.21.15.40:0/1553641502 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbb28073cb0 con 0x7fbb280cbe70 2023-11-08T08:20:26.421 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:20:26.430 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:20:26.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.477+0000 7fbb1effd700 1 -- 172.21.15.40:0/1553641502 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v39) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbb18054740 con 0x7fbb280cbe70 2023-11-08T08:20:26.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.481+0000 7fbb2f085700 1 -- 172.21.15.40:0/1553641502 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbb0c064ee0 msgr2=0x7fbb0c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:26.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.481+0000 7fbb2f085700 1 --2- 172.21.15.40:0/1553641502 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbb0c064ee0 0x7fbb0c067390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400f220 tx=0x7fbb24009040 comp rx=0 tx=0).stop 2023-11-08T08:20:26.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.481+0000 7fbb2f085700 1 -- 172.21.15.40:0/1553641502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb280cbe70 msgr2=0x7fbb28151fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:26.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.481+0000 7fbb2f085700 1 --2- 172.21.15.40:0/1553641502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb280cbe70 0x7fbb28151fa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbb18013000 tx=0x7fbb18004770 comp rx=0 tx=0).stop 2023-11-08T08:20:26.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.481+0000 7fbb2f085700 1 -- 172.21.15.40:0/1553641502 shutdown_connections 2023-11-08T08:20:26.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.481+0000 7fbb2f085700 1 --2- 172.21.15.40:0/1553641502 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbb0c064ee0 0x7fbb0c067390 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:26.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.481+0000 7fbb2f085700 1 --2- 172.21.15.40:0/1553641502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb2815fbd0 0x7fbb281604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:26.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.481+0000 7fbb2f085700 1 --2- 172.21.15.40:0/1553641502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb280cbe70 0x7fbb28151fa0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:26.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.481+0000 7fbb2f085700 1 --2- 172.21.15.40:0/1553641502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb2806a300 0x7fbb281f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:26.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.481+0000 7fbb2f085700 1 -- 172.21.15.40:0/1553641502 >> 172.21.15.40:0/1553641502 conn(0x7fbb280b0a20 msgr2=0x7fbb280b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:26.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.481+0000 7fbb2f085700 1 -- 172.21.15.40:0/1553641502 shutdown_connections 2023-11-08T08:20:26.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.481+0000 7fbb2f085700 1 -- 172.21.15.40:0/1553641502 wait complete. 2023-11-08T08:20:26.488 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2023-11-08T08:20:26.511 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-08T08:20:26.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.741+0000 7fda8b40a700 1 Processor -- start 2023-11-08T08:20:26.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.741+0000 7fda8b40a700 1 -- start start 2023-11-08T08:20:26.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.745+0000 7fda8b40a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda841534a0 0x7fda8415d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:26.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.745+0000 7fda8b40a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fda84160920 0x7fda840b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:26.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.745+0000 7fda8b40a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fda840b6030 0x7fda84070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:26.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.745+0000 7fda8b40a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda8415d080 con 0x7fda84160920 2023-11-08T08:20:26.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.745+0000 7fda8b40a700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda84153870 con 0x7fda840b6030 2023-11-08T08:20:26.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.745+0000 7fda8b40a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda840b6450 con 0x7fda841534a0 2023-11-08T08:20:26.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.745+0000 7fda889a5700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fda84160920 0x7fda840b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:26.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.745+0000 7fda889a5700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fda84160920 0x7fda840b5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45652/0 (socket says 172.21.15.40:45652) 2023-11-08T08:20:26.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.745+0000 7fda889a5700 1 -- 172.21.15.40:0/3149387296 learned_addr learned my addr 172.21.15.40:0/3149387296 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:26.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.745+0000 7fda891a6700 1 --2- 172.21.15.40:0/3149387296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda841534a0 0x7fda8415d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:26.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.745+0000 7fda899a7700 1 --2- 172.21.15.40:0/3149387296 >> v2:172.21.15.169:3300/0 conn(0x7fda840b6030 0x7fda84070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:26.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.745+0000 7fda889a5700 1 -- 172.21.15.40:0/3149387296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda841534a0 msgr2=0x7fda8415d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:26.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.745+0000 7fda889a5700 1 --2- 172.21.15.40:0/3149387296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda841534a0 0x7fda8415d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:26.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.745+0000 7fda889a5700 1 -- 172.21.15.40:0/3149387296 >> v2:172.21.15.169:3300/0 conn(0x7fda840b6030 msgr2=0x7fda84070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:26.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.745+0000 7fda889a5700 1 --2- 172.21.15.40:0/3149387296 >> v2:172.21.15.169:3300/0 conn(0x7fda840b6030 0x7fda84070630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:26.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.745+0000 7fda889a5700 1 -- 172.21.15.40:0/3149387296 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda840b53c0 con 0x7fda84160920 2023-11-08T08:20:26.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.745+0000 7fda889a5700 1 --2- 172.21.15.40:0/3149387296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda84160920 0x7fda840b5af0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fda840d0790 tx=0x7fda74009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544c0e76c9647077 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:26.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.745+0000 7fda7bfff700 1 -- 172.21.15.40:0/3149387296 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74011830 con 0x7fda84160920 2023-11-08T08:20:26.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.745+0000 7fda7bfff700 1 -- 172.21.15.40:0/3149387296 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda74011990 con 0x7fda84160920 2023-11-08T08:20:26.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.745+0000 7fda7bfff700 1 -- 172.21.15.40:0/3149387296 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74011d20 con 0x7fda84160920 2023-11-08T08:20:26.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.749+0000 7fda8b40a700 1 -- 172.21.15.40:0/3149387296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda84160920 msgr2=0x7fda840b5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:26.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.749+0000 7fda8b40a700 1 --2- 172.21.15.40:0/3149387296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda84160920 0x7fda840b5af0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fda840d0790 tx=0x7fda74009bf0 comp rx=0 tx=0).stop 2023-11-08T08:20:26.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.749+0000 7fda8b40a700 1 -- 172.21.15.40:0/3149387296 shutdown_connections 2023-11-08T08:20:26.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.749+0000 7fda8b40a700 1 --2- 172.21.15.40:0/3149387296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda84160920 0x7fda840b5af0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:26.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.749+0000 7fda8b40a700 1 --2- 172.21.15.40:0/3149387296 >> v2:172.21.15.169:3300/0 conn(0x7fda840b6030 0x7fda84070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:26.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.749+0000 7fda8b40a700 1 --2- 172.21.15.40:0/3149387296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda841534a0 0x7fda8415d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:26.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.749+0000 7fda8b40a700 1 -- 172.21.15.40:0/3149387296 >> 172.21.15.40:0/3149387296 conn(0x7fda8406bfc0 msgr2=0x7fda8406c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:26.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.749+0000 7fda8b40a700 1 -- 172.21.15.40:0/3149387296 shutdown_connections 2023-11-08T08:20:26.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.749+0000 7fda8b40a700 1 -- 172.21.15.40:0/3149387296 wait complete. 2023-11-08T08:20:26.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.753+0000 7fda8b40a700 1 Processor -- start 2023-11-08T08:20:26.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.753+0000 7fda8b40a700 1 -- start start 2023-11-08T08:20:26.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.753+0000 7fda8b40a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda840b6030 0x7fda8408bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:26.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.753+0000 7fda8b40a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda841534a0 0x7fda8408d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:26.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.753+0000 7fda8b40a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda84160920 0x7fda84149000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:26.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.753+0000 7fda8b40a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda841f6600 con 0x7fda84160920 2023-11-08T08:20:26.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.753+0000 7fda8b40a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda841e7350 con 0x7fda840b6030 2023-11-08T08:20:26.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.753+0000 7fda8b40a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda841e67d0 con 0x7fda841534a0 2023-11-08T08:20:26.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.753+0000 7fda899a7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda84160920 0x7fda84149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:26.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.753+0000 7fda899a7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda84160920 0x7fda84149000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45664/0 (socket says 172.21.15.40:45664) 2023-11-08T08:20:26.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.753+0000 7fda899a7700 1 -- 172.21.15.40:0/3148601674 learned_addr learned my addr 172.21.15.40:0/3148601674 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:26.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.753+0000 7fda899a7700 1 -- 172.21.15.40:0/3148601674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda841534a0 msgr2=0x7fda8408d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:26.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.753+0000 7fda899a7700 1 --2- 172.21.15.40:0/3148601674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda841534a0 0x7fda8408d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:26.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.753+0000 7fda899a7700 1 -- 172.21.15.40:0/3148601674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda840b6030 msgr2=0x7fda8408bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:26.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.753+0000 7fda899a7700 1 --2- 172.21.15.40:0/3148601674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda840b6030 0x7fda8408bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:26.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.757+0000 7fda899a7700 1 -- 172.21.15.40:0/3148601674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda74009570 con 0x7fda84160920 2023-11-08T08:20:26.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.757+0000 7fda899a7700 1 --2- 172.21.15.40:0/3148601674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda84160920 0x7fda84149000 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00efc0 tx=0x7fda7c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:26.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.757+0000 7fda79ffb700 1 -- 172.21.15.40:0/3148601674 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c009ba0 con 0x7fda84160920 2023-11-08T08:20:26.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.757+0000 7fda79ffb700 1 -- 172.21.15.40:0/3148601674 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7c00f040 con 0x7fda84160920 2023-11-08T08:20:26.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.757+0000 7fda79ffb700 1 -- 172.21.15.40:0/3148601674 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c014480 con 0x7fda84160920 2023-11-08T08:20:26.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.757+0000 7fda8b40a700 1 -- 172.21.15.40:0/3148601674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda8415ca30 con 0x7fda84160920 2023-11-08T08:20:26.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.757+0000 7fda8b40a700 1 -- 172.21.15.40:0/3148601674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda840b56a0 con 0x7fda84160920 2023-11-08T08:20:26.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.757+0000 7fda8b40a700 1 -- 172.21.15.40:0/3148601674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda8415cd10 con 0x7fda84160920 2023-11-08T08:20:26.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.761+0000 7fda79ffb700 1 -- 172.21.15.40:0/3148601674 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fda7c0145e0 con 0x7fda84160920 2023-11-08T08:20:26.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.761+0000 7fda79ffb700 1 --2- 172.21.15.40:0/3148601674 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fda68064f30 0x7fda680673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:26.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.761+0000 7fda79ffb700 1 -- 172.21.15.40:0/3148601674 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fda7c082740 con 0x7fda84160920 2023-11-08T08:20:26.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.761+0000 7fda891a6700 1 --2- 172.21.15.40:0/3148601674 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fda68064f30 0x7fda680673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:26.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.765+0000 7fda891a6700 1 --2- 172.21.15.40:0/3148601674 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fda68064f30 0x7fda680673e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fda80009af0 tx=0x7fda800095b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:26.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:26.769+0000 7fda79ffb700 1 -- 172.21.15.40:0/3148601674 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fda7c0509f0 con 0x7fda84160920 2023-11-08T08:20:27.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.009+0000 7fda8b40a700 1 -- 172.21.15.40:0/3148601674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fda8406c940 con 0x7fda84160920 2023-11-08T08:20:27.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.009+0000 7fda79ffb700 1 -- 172.21.15.40:0/3148601674 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fda7c018090 con 0x7fda84160920 2023-11-08T08:20:27.015 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:27.015 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:20:27.015 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2023-11-08T08:20:27.015 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.016 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2023-11-08T08:20:27.016 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA3REtlj6YZGRAABMVq4pKyrp6U3tbqShtmuw==", 2023-11-08T08:20:27.016 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.016 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:20:27.017 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-11-08T08:20:27.017 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-11-08T08:20:27.017 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:20:27.017 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.017 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.018 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.018 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2023-11-08T08:20:27.018 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA3REtlAfqKLRAAmNg22Rw+FWmzeRDBKQtjfw==", 2023-11-08T08:20:27.018 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.018 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:20:27.019 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-11-08T08:20:27.019 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-11-08T08:20:27.019 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:20:27.019 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.019 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.020 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.020 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2023-11-08T08:20:27.020 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA3REtlRseiJBAAZFNQO+PK+kioVBn+S/54Hw==", 2023-11-08T08:20:27.020 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.020 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:20:27.021 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-11-08T08:20:27.021 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-11-08T08:20:27.021 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:20:27.021 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.022 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.022 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.022 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2023-11-08T08:20:27.022 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA3REtlZHCiOBAAW8bJst1bqTZ2jPOr+wXcpg==", 2023-11-08T08:20:27.022 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.023 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:20:27.023 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-11-08T08:20:27.023 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-11-08T08:20:27.023 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:20:27.024 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.024 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.024 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.024 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2023-11-08T08:20:27.024 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA7REtlvWsaLBAAws4XZXHoYl9KnhBp6LEh0w==", 2023-11-08T08:20:27.025 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.025 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:20:27.025 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:20:27.025 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:20:27.026 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.026 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.026 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.026 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2023-11-08T08:20:27.026 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA+REtl8FYFOhAAOY5q9LJAl6PS+essHcF8cw==", 2023-11-08T08:20:27.027 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.027 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:20:27.027 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:20:27.027 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:20:27.027 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.028 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.028 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.028 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2023-11-08T08:20:27.028 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBCREtljU1ODBAA1gVE4sX2QNdGHIjdRue/rA==", 2023-11-08T08:20:27.028 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.029 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:20:27.029 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:20:27.029 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:20:27.029 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.029 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.030 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.030 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2023-11-08T08:20:27.030 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBFREtlRZsSGxAAh7v5f7TVZNQozESV6k641g==", 2023-11-08T08:20:27.030 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.030 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:20:27.031 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:20:27.031 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:20:27.031 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.031 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.031 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.031 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2023-11-08T08:20:27.032 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBLREtla+qaNRAA0jaCFarD/RgWGp1ltYXQdA==", 2023-11-08T08:20:27.032 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.032 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:20:27.032 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:20:27.032 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:20:27.033 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.033 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.033 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.033 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2023-11-08T08:20:27.033 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPREtltHKNCRAA/TKgyxOGRkN5y/RHWr8vSQ==", 2023-11-08T08:20:27.034 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.034 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:20:27.034 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:20:27.034 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:20:27.034 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.035 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.035 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.035 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2023-11-08T08:20:27.035 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBSREtlpXC4GRAAZl6YWPVnbX3o1tyfEkR7CA==", 2023-11-08T08:20:27.036 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.036 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:20:27.036 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:20:27.036 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:20:27.036 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.036 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.037 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.037 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2023-11-08T08:20:27.037 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBVREtlvL/4JxAAZtGdthYoFzdMRs3DMcRXNw==", 2023-11-08T08:20:27.037 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.037 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:20:27.038 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:20:27.038 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:20:27.038 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.038 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.038 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.039 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2023-11-08T08:20:27.039 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA4REtl0YCpBBAAtap/irtK+GkSmR8g37HRVQ==", 2023-11-08T08:20:27.039 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.039 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:20:27.039 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-11-08T08:20:27.040 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-11-08T08:20:27.040 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-11-08T08:20:27.040 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.040 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.040 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.041 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2023-11-08T08:20:27.041 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA4REtlgapFERAAxg8GS7iiH06t2mCF5ImFXA==", 2023-11-08T08:20:27.041 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.041 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:20:27.041 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-11-08T08:20:27.042 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-11-08T08:20:27.042 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-11-08T08:20:27.042 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.042 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.043 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.043 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2023-11-08T08:20:27.043 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA4REtldlrMChAAC9+syjsFWWFwJGXPw82zwQ==", 2023-11-08T08:20:27.043 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.043 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:20:27.043 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-11-08T08:20:27.044 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-11-08T08:20:27.044 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-11-08T08:20:27.044 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.044 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.045 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.045 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2023-11-08T08:20:27.045 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA4REtllHEDFxAACJSjTZQaOTnyA0K3X/igXQ==", 2023-11-08T08:20:27.045 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.045 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:20:27.045 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-11-08T08:20:27.046 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-11-08T08:20:27.046 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-11-08T08:20:27.046 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.046 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.047 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.047 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2023-11-08T08:20:27.047 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA2REtlqqakKRAAzyr9EbqU+Pj0m2weeru80g==", 2023-11-08T08:20:27.047 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.047 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-11-08T08:20:27.048 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-11-08T08:20:27.048 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-11-08T08:20:27.048 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:20:27.048 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.048 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.049 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.049 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-mds", 2023-11-08T08:20:27.049 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBcREtlMMIiOhAA4/3yJWZPTJl7F6x522Jrcg==", 2023-11-08T08:20:27.049 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.049 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-mds" 2023-11-08T08:20:27.050 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.050 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.050 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.050 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-mgr", 2023-11-08T08:20:27.050 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBcREtlgdkiOhAAhwGmTRYAoj5PKE5VpH0SxQ==", 2023-11-08T08:20:27.051 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.051 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-08T08:20:27.051 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.051 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.051 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.052 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-osd", 2023-11-08T08:20:27.052 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBcREtlLOwiOhAAlS4o/gt50hUWJhyu9sxYwg==", 2023-11-08T08:20:27.052 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.052 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-osd" 2023-11-08T08:20:27.052 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.053 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.053 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.053 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-rbd", 2023-11-08T08:20:27.053 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBcREtlNf0iOhAAEAAxy/ypSIiyeW+Xhro0Yw==", 2023-11-08T08:20:27.053 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.054 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-08T08:20:27.054 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.054 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.054 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.054 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-08T08:20:27.055 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBcREtlEw0jOhAAjaSWla2DUB7htsILzDcrEQ==", 2023-11-08T08:20:27.055 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.055 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-08T08:20:27.055 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.055 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.056 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.056 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-rgw", 2023-11-08T08:20:27.056 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBcREtl3RkjOhAAMWYDdyqZpmBo4HBoS/pjRQ==", 2023-11-08T08:20:27.056 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.056 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-08T08:20:27.057 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.057 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.057 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.057 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2023-11-08T08:20:27.057 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA3REtl6YeRERAAKW6fA2vM7HOpZGCulFFLzQ==", 2023-11-08T08:20:27.058 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.058 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-11-08T08:20:27.058 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-11-08T08:20:27.058 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:20:27.058 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.059 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:27.059 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:27.059 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2023-11-08T08:20:27.059 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA3REtlmuaiChAAsI2681k5qbe8I0hdIXHJGg==", 2023-11-08T08:20:27.059 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:20:27.060 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-11-08T08:20:27.060 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-11-08T08:20:27.060 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:20:27.060 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.060 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:27.061 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-11-08T08:20:27.061 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:20:27.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.013+0000 7fda8b40a700 1 -- 172.21.15.40:0/3148601674 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fda68064f30 msgr2=0x7fda680673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:27.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.013+0000 7fda8b40a700 1 --2- 172.21.15.40:0/3148601674 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fda68064f30 0x7fda680673e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fda80009af0 tx=0x7fda800095b0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.013+0000 7fda8b40a700 1 -- 172.21.15.40:0/3148601674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda84160920 msgr2=0x7fda84149000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:27.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.013+0000 7fda8b40a700 1 --2- 172.21.15.40:0/3148601674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda84160920 0x7fda84149000 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00efc0 tx=0x7fda7c00c3a0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.013+0000 7fda8b40a700 1 -- 172.21.15.40:0/3148601674 shutdown_connections 2023-11-08T08:20:27.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.013+0000 7fda8b40a700 1 --2- 172.21.15.40:0/3148601674 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fda68064f30 0x7fda680673e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.013+0000 7fda8b40a700 1 --2- 172.21.15.40:0/3148601674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda84160920 0x7fda84149000 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.013+0000 7fda8b40a700 1 --2- 172.21.15.40:0/3148601674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda841534a0 0x7fda8408d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.013+0000 7fda8b40a700 1 --2- 172.21.15.40:0/3148601674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda840b6030 0x7fda8408bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.013+0000 7fda8b40a700 1 -- 172.21.15.40:0/3148601674 >> 172.21.15.40:0/3148601674 conn(0x7fda8406bfc0 msgr2=0x7fda8415daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:27.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.013+0000 7fda8b40a700 1 -- 172.21.15.40:0/3148601674 shutdown_connections 2023-11-08T08:20:27.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.013+0000 7fda8b40a700 1 -- 172.21.15.40:0/3148601674 wait complete. 2023-11-08T08:20:27.065 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-08T08:20:27.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.257+0000 7f52b5fa1700 1 Processor -- start 2023-11-08T08:20:27.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.257+0000 7f52b5fa1700 1 -- start start 2023-11-08T08:20:27.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.261+0000 7f52b5fa1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f52b015c640 0x7f52b00d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:27.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.261+0000 7f52b5fa1700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b0071aa0 0x7f52b006d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:27.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.261+0000 7f52b5fa1700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b006c600 con 0x7f52b0069e00 2023-11-08T08:20:27.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.261+0000 7f52b5fa1700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b006cef0 con 0x7f52b015c640 2023-11-08T08:20:27.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.261+0000 7f52b5fa1700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b00722f0 con 0x7f52b0071aa0 2023-11-08T08:20:27.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.261+0000 7f52b4f9f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f52b015c640 0x7f52b00d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:27.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.261+0000 7f52affff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b0071aa0 0x7f52b006d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:27.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.261+0000 7f52b4f9f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f52b015c640 0x7f52b00d31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45666/0 (socket says 172.21.15.40:45666) 2023-11-08T08:20:27.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.261+0000 7f52b4f9f700 1 -- 172.21.15.40:0/4017319314 learned_addr learned my addr 172.21.15.40:0/4017319314 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:27.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.261+0000 7f52b4f9f700 1 -- 172.21.15.40:0/4017319314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b0071aa0 msgr2=0x7f52b006d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:27.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.261+0000 7f52b4f9f700 1 --2- 172.21.15.40:0/4017319314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b0071aa0 0x7f52b006d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.261+0000 7f52b4f9f700 1 -- 172.21.15.40:0/4017319314 >> v1:172.21.15.169:6789/0 conn(0x7f52b0069e00 legacy=0x7f52b00d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:27.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.261+0000 7f52b4f9f700 1 -- 172.21.15.40:0/4017319314 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b015ca10 con 0x7f52b015c640 2023-11-08T08:20:27.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.261+0000 7f52b4f9f700 1 --2- 172.21.15.40:0/4017319314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52b015c640 0x7f52b00d31e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009910 tx=0x7f52a4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5948f958288b64 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:27.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.261+0000 7f52af7fe700 1 -- 172.21.15.40:0/4017319314 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a4011830 con 0x7f52b015c640 2023-11-08T08:20:27.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.261+0000 7f52af7fe700 1 -- 172.21.15.40:0/4017319314 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a4011990 con 0x7f52b015c640 2023-11-08T08:20:27.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.261+0000 7f52af7fe700 1 -- 172.21.15.40:0/4017319314 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a4011d40 con 0x7f52b015c640 2023-11-08T08:20:27.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.265+0000 7f52b5fa1700 1 -- 172.21.15.40:0/4017319314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52b015c640 msgr2=0x7f52b00d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:27.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.265+0000 7f52b5fa1700 1 --2- 172.21.15.40:0/4017319314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52b015c640 0x7f52b00d31e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009910 tx=0x7f52a4009c20 comp rx=0 tx=0).stop 2023-11-08T08:20:27.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.265+0000 7f52b5fa1700 1 -- 172.21.15.40:0/4017319314 shutdown_connections 2023-11-08T08:20:27.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.265+0000 7f52b5fa1700 1 --2- 172.21.15.40:0/4017319314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b0071aa0 0x7f52b006d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.265+0000 7f52b5fa1700 1 --2- 172.21.15.40:0/4017319314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52b015c640 0x7f52b00d31e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.265+0000 7f52b5fa1700 1 -- 172.21.15.40:0/4017319314 >> 172.21.15.40:0/4017319314 conn(0x7f52b00b6520 msgr2=0x7f52b0071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:27.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.265+0000 7f52b5fa1700 1 -- 172.21.15.40:0/4017319314 shutdown_connections 2023-11-08T08:20:27.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.265+0000 7f52b5fa1700 1 -- 172.21.15.40:0/4017319314 wait complete. 2023-11-08T08:20:27.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.265+0000 7f52b5fa1700 1 Processor -- start 2023-11-08T08:20:27.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.265+0000 7f52b5fa1700 1 -- start start 2023-11-08T08:20:27.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52b5fa1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52b0069e00 0x7f52b015e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:27.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52b5fa1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52b0071aa0 0x7f52b015e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:27.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52b5fa1700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b015c640 0x7f52b01f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:27.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52b5fa1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b015dfa0 con 0x7f52b0071aa0 2023-11-08T08:20:27.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52b5fa1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b015d9e0 con 0x7f52b0069e00 2023-11-08T08:20:27.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52b5fa1700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b015dcc0 con 0x7f52b015c640 2023-11-08T08:20:27.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52affff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52b0071aa0 0x7f52b015e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:27.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52affff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52b0071aa0 0x7f52b015e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45680/0 (socket says 172.21.15.40:45680) 2023-11-08T08:20:27.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52affff700 1 -- 172.21.15.40:0/748363469 learned_addr learned my addr 172.21.15.40:0/748363469 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:27.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52b57a0700 1 --2- 172.21.15.40:0/748363469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b015c640 0x7f52b01f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:27.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52b4f9f700 1 --2- 172.21.15.40:0/748363469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52b0069e00 0x7f52b015e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:27.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52affff700 1 -- 172.21.15.40:0/748363469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b015c640 msgr2=0x7f52b01f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:27.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52affff700 1 --2- 172.21.15.40:0/748363469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b015c640 0x7f52b01f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52affff700 1 -- 172.21.15.40:0/748363469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52b0069e00 msgr2=0x7f52b015e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:27.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52affff700 1 --2- 172.21.15.40:0/748363469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52b0069e00 0x7f52b015e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52affff700 1 -- 172.21.15.40:0/748363469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a4009570 con 0x7f52b0071aa0 2023-11-08T08:20:27.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52affff700 1 --2- 172.21.15.40:0/748363469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52b0071aa0 0x7f52b015e710 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f529c00e840 tx=0x7f529c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:27.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52ad7fa700 1 -- 172.21.15.40:0/748363469 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c0095a0 con 0x7f52b0071aa0 2023-11-08T08:20:27.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52ad7fa700 1 -- 172.21.15.40:0/748363469 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529c013680 con 0x7f52b0071aa0 2023-11-08T08:20:27.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52b5fa1700 1 -- 172.21.15.40:0/748363469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b006c830 con 0x7f52b0071aa0 2023-11-08T08:20:27.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52ad7fa700 1 -- 172.21.15.40:0/748363469 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c0139f0 con 0x7f52b0071aa0 2023-11-08T08:20:27.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.269+0000 7f52b5fa1700 1 -- 172.21.15.40:0/748363469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b0072010 con 0x7f52b0071aa0 2023-11-08T08:20:27.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.273+0000 7f52b5fa1700 1 -- 172.21.15.40:0/748363469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b006cb10 con 0x7f52b0071aa0 2023-11-08T08:20:27.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.273+0000 7f52ad7fa700 1 -- 172.21.15.40:0/748363469 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f529c019030 con 0x7f52b0071aa0 2023-11-08T08:20:27.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.277+0000 7f52ad7fa700 1 --2- 172.21.15.40:0/748363469 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5294064f10 0x7f52940673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:27.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.277+0000 7f52ad7fa700 1 -- 172.21.15.40:0/748363469 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f529c081a20 con 0x7f52b0071aa0 2023-11-08T08:20:27.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.277+0000 7f52b4f9f700 1 --2- 172.21.15.40:0/748363469 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5294064f10 0x7f52940673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:27.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.277+0000 7f52b4f9f700 1 --2- 172.21.15.40:0/748363469 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5294064f10 0x7f52940673c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009860 tx=0x7f52a4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:27.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.281+0000 7f52ad7fa700 1 -- 172.21.15.40:0/748363469 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f529c04fce0 con 0x7f52b0071aa0 2023-11-08T08:20:27.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.513+0000 7f52b5fa1700 1 -- 172.21.15.40:0/748363469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f52b0073cb0 con 0x7f52b0071aa0 2023-11-08T08:20:27.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.521+0000 7f52ad7fa700 1 -- 172.21.15.40:0/748363469 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f529c053300 con 0x7f52b0071aa0 2023-11-08T08:20:27.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.525+0000 7f52b5fa1700 1 -- 172.21.15.40:0/748363469 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5294064f10 msgr2=0x7f52940673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:27.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.525+0000 7f52b5fa1700 1 --2- 172.21.15.40:0/748363469 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5294064f10 0x7f52940673c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009860 tx=0x7f52a4008030 comp rx=0 tx=0).stop 2023-11-08T08:20:27.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.525+0000 7f52b5fa1700 1 -- 172.21.15.40:0/748363469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52b0071aa0 msgr2=0x7f52b015e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:27.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.525+0000 7f52b5fa1700 1 --2- 172.21.15.40:0/748363469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52b0071aa0 0x7f52b015e710 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f529c00e840 tx=0x7f529c00eb50 comp rx=0 tx=0).stop 2023-11-08T08:20:27.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.525+0000 7f52b5fa1700 1 -- 172.21.15.40:0/748363469 shutdown_connections 2023-11-08T08:20:27.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.525+0000 7f52b5fa1700 1 --2- 172.21.15.40:0/748363469 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5294064f10 0x7f52940673c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.525+0000 7f52b5fa1700 1 --2- 172.21.15.40:0/748363469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52b015c640 0x7f52b01f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.525+0000 7f52b5fa1700 1 --2- 172.21.15.40:0/748363469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52b0071aa0 0x7f52b015e710 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.525+0000 7f52b5fa1700 1 --2- 172.21.15.40:0/748363469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52b0069e00 0x7f52b015e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.525+0000 7f52b5fa1700 1 -- 172.21.15.40:0/748363469 >> 172.21.15.40:0/748363469 conn(0x7f52b00b6520 msgr2=0x7f52b00af580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:27.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.525+0000 7f52b5fa1700 1 -- 172.21.15.40:0/748363469 shutdown_connections 2023-11-08T08:20:27.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.525+0000 7f52b5fa1700 1 -- 172.21.15.40:0/748363469 wait complete. 2023-11-08T08:20:27.554 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-08T08:20:27.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.737+0000 7f88e5b56700 1 Processor -- start 2023-11-08T08:20:27.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.737+0000 7f88e5b56700 1 -- start start 2023-11-08T08:20:27.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.737+0000 7f88e5b56700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f88e00cdeb0 0x7f88e0070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:27.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.737+0000 7f88e5b56700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88e00705c0 0x7f88e006a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:27.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.737+0000 7f88e5b56700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e0070a20 con 0x7f88e015c1a0 2023-11-08T08:20:27.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.737+0000 7f88e5b56700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e0069bf0 con 0x7f88e00cdeb0 2023-11-08T08:20:27.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.737+0000 7f88e5b56700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e00adf90 con 0x7f88e00705c0 2023-11-08T08:20:27.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.737+0000 7f88e4b54700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f88e015c1a0 0x7f88e006c8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57168/0 (socket says 172.21.15.40:57168) 2023-11-08T08:20:27.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.737+0000 7f88e4b54700 1 -- 172.21.15.40:0/3088381392 learned_addr learned my addr 172.21.15.40:0/3088381392 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:27.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.737+0000 7f88dffff700 1 --2- 172.21.15.40:0/3088381392 >> v2:172.21.15.169:3300/0 conn(0x7f88e00cdeb0 0x7f88e0070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:27.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.737+0000 7f88e5355700 1 --2- 172.21.15.40:0/3088381392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88e00705c0 0x7f88e006a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:27.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.737+0000 7f88df7fe700 1 -- 172.21.15.40:0/3088381392 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2098007793 0 0) 0x7f88e0070a20 con 0x7f88e015c1a0 2023-11-08T08:20:27.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88df7fe700 1 -- 172.21.15.40:0/3088381392 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88d00035d0 con 0x7f88e015c1a0 2023-11-08T08:20:27.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88df7fe700 1 -- 172.21.15.40:0/3088381392 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3138371230 0 0) 0x7f88d00035d0 con 0x7f88e015c1a0 2023-11-08T08:20:27.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88df7fe700 1 -- 172.21.15.40:0/3088381392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88e00705c0 msgr2=0x7f88e006a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:27.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88df7fe700 1 --2- 172.21.15.40:0/3088381392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88e00705c0 0x7f88e006a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88df7fe700 1 -- 172.21.15.40:0/3088381392 >> v2:172.21.15.169:3300/0 conn(0x7f88e00cdeb0 msgr2=0x7f88e0070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:27.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88df7fe700 1 --2- 172.21.15.40:0/3088381392 >> v2:172.21.15.169:3300/0 conn(0x7f88e00cdeb0 0x7f88e0070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88df7fe700 1 -- 172.21.15.40:0/3088381392 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e00ce370 con 0x7f88e015c1a0 2023-11-08T08:20:27.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88df7fe700 1 -- 172.21.15.40:0/3088381392 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f88d4001690 con 0x7f88e015c1a0 2023-11-08T08:20:27.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88df7fe700 1 -- 172.21.15.40:0/3088381392 >> v1:172.21.15.40:6789/0 conn(0x7f88e015c1a0 legacy=0x7f88e006c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:27.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88df7fe700 1 --2- 172.21.15.40:0/3088381392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88e00705c0 0x7f88d0003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:27.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88df7fe700 1 --2- 172.21.15.40:0/3088381392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88d0003fa0 0x7f88d0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:27.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88df7fe700 1 --2- 172.21.15.40:0/3088381392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88d0004db0 0x7f88d0005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:27.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88df7fe700 1 -- 172.21.15.40:0/3088381392 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e0069bf0 con 0x7f88d0004db0 2023-11-08T08:20:27.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88df7fe700 1 -- 172.21.15.40:0/3088381392 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e00adf90 con 0x7f88d0003fa0 2023-11-08T08:20:27.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88df7fe700 1 -- 172.21.15.40:0/3088381392 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d000ba60 con 0x7f88e00705c0 2023-11-08T08:20:27.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88dffff700 1 --2- 172.21.15.40:0/3088381392 >> v2:172.21.15.169:3300/0 conn(0x7f88e00cdeb0 0x7f88e0070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:27.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88e5355700 1 --2- 172.21.15.40:0/3088381392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88d0004db0 0x7f88d0005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:27.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88e4b54700 1 --2- 172.21.15.40:0/3088381392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88e00705c0 0x7f88d0003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:27.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88e5355700 1 -- 172.21.15.40:0/3088381392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88e00705c0 msgr2=0x7f88d0003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:27.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88e5355700 1 --2- 172.21.15.40:0/3088381392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88e00705c0 0x7f88d0003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88e5355700 1 -- 172.21.15.40:0/3088381392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88d0003fa0 msgr2=0x7f88d0004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:27.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88e5355700 1 --2- 172.21.15.40:0/3088381392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88d0003fa0 0x7f88d0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88e5355700 1 -- 172.21.15.40:0/3088381392 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d00035d0 con 0x7f88d0004db0 2023-11-08T08:20:27.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88e5355700 1 --2- 172.21.15.40:0/3088381392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88d0004db0 0x7f88d0005220 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009fc0 tx=0x7f88d00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3328880a89ba0d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:27.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88df7fe700 1 -- 172.21.15.40:0/3088381392 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d800ebd0 con 0x7f88d0004db0 2023-11-08T08:20:27.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88df7fe700 1 -- 172.21.15.40:0/3088381392 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d800f040 con 0x7f88d0004db0 2023-11-08T08:20:27.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88df7fe700 1 -- 172.21.15.40:0/3088381392 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8014480 con 0x7f88d0004db0 2023-11-08T08:20:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88e5b56700 1 -- 172.21.15.40:0/3088381392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88d0004db0 msgr2=0x7f88d0005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88e5b56700 1 --2- 172.21.15.40:0/3088381392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88d0004db0 0x7f88d0005220 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009fc0 tx=0x7f88d00058a0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88e5355700 1 -- 172.21.15.40:0/3088381392 reap_dead start 2023-11-08T08:20:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88e5b56700 1 -- 172.21.15.40:0/3088381392 shutdown_connections 2023-11-08T08:20:27.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.741+0000 7f88e5b56700 1 -- 172.21.15.40:0/3088381392 >> 172.21.15.40:0/3088381392 conn(0x7f88e00b0f80 msgr2=0x7f88e00ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:27.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.745+0000 7f88e5b56700 1 -- 172.21.15.40:0/3088381392 shutdown_connections 2023-11-08T08:20:27.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.745+0000 7f88e5b56700 1 -- 172.21.15.40:0/3088381392 wait complete. 2023-11-08T08:20:27.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.745+0000 7f88e5b56700 1 Processor -- start 2023-11-08T08:20:27.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.745+0000 7f88e5b56700 1 -- start start 2023-11-08T08:20:27.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.749+0000 7f88e5b56700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88e0155dd0 0x7f88e015b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:27.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.749+0000 7f88e5b56700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88e00c5820 0x7f88e015cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:27.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.749+0000 7f88e5b56700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88e015d1e0 0x7f88e00aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.749+0000 7f88e5b56700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e015d650 con 0x7f88e00c5820 2023-11-08T08:20:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.749+0000 7f88e5b56700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e00c5d90 con 0x7f88e0155dd0 2023-11-08T08:20:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.749+0000 7f88e5b56700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e00c6070 con 0x7f88e015d1e0 2023-11-08T08:20:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.749+0000 7f88dffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88e00c5820 0x7f88e015cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.749+0000 7f88dffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88e00c5820 0x7f88e015cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34028/0 (socket says 172.21.15.40:34028) 2023-11-08T08:20:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.749+0000 7f88dffff700 1 -- 172.21.15.40:0/2640703269 learned_addr learned my addr 172.21.15.40:0/2640703269 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.749+0000 7f88e4b54700 1 --2- 172.21.15.40:0/2640703269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88e0155dd0 0x7f88e015b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.749+0000 7f88e5355700 1 --2- 172.21.15.40:0/2640703269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88e015d1e0 0x7f88e00aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.749+0000 7f88dffff700 1 -- 172.21.15.40:0/2640703269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88e015d1e0 msgr2=0x7f88e00aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.749+0000 7f88dffff700 1 --2- 172.21.15.40:0/2640703269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88e015d1e0 0x7f88e00aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.749+0000 7f88dffff700 1 -- 172.21.15.40:0/2640703269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88e0155dd0 msgr2=0x7f88e015b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:27.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.749+0000 7f88dffff700 1 --2- 172.21.15.40:0/2640703269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88e0155dd0 0x7f88e015b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:27.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.749+0000 7f88dffff700 1 -- 172.21.15.40:0/2640703269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e00cc780 con 0x7f88e00c5820 2023-11-08T08:20:27.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.749+0000 7f88dffff700 1 --2- 172.21.15.40:0/2640703269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88e00c5820 0x7f88e015cca0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f88cc004840 tx=0x7f88cc00ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:27.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.753+0000 7f88dd7fa700 1 -- 172.21.15.40:0/2640703269 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc00ee80 con 0x7f88e00c5820 2023-11-08T08:20:27.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.753+0000 7f88e5b56700 1 -- 172.21.15.40:0/2640703269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e00cc4a0 con 0x7f88e00c5820 2023-11-08T08:20:27.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.753+0000 7f88dd7fa700 1 -- 172.21.15.40:0/2640703269 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88cc004030 con 0x7f88e00c5820 2023-11-08T08:20:27.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.753+0000 7f88dd7fa700 1 -- 172.21.15.40:0/2640703269 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc0043e0 con 0x7f88e00c5820 2023-11-08T08:20:27.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.753+0000 7f88e5b56700 1 -- 172.21.15.40:0/2640703269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e015c570 con 0x7f88e00c5820 2023-11-08T08:20:27.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.753+0000 7f88e5b56700 1 -- 172.21.15.40:0/2640703269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e0073cb0 con 0x7f88e00c5820 2023-11-08T08:20:27.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.753+0000 7f88dd7fa700 1 -- 172.21.15.40:0/2640703269 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f88cc019070 con 0x7f88e00c5820 2023-11-08T08:20:27.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.753+0000 7f88dd7fa700 1 --2- 172.21.15.40:0/2640703269 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f88c4064ee0 0x7f88c4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:27.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.753+0000 7f88dd7fa700 1 -- 172.21.15.40:0/2640703269 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f88cc081a20 con 0x7f88e00c5820 2023-11-08T08:20:27.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.757+0000 7f88e4b54700 1 --2- 172.21.15.40:0/2640703269 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f88c4064ee0 0x7f88c4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:27.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.757+0000 7f88e4b54700 1 --2- 172.21.15.40:0/2640703269 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f88c4064ee0 0x7f88c4067390 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f88d40017c0 tx=0x7f88d4007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:27.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.761+0000 7f88dd7fa700 1 -- 172.21.15.40:0/2640703269 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f88cc04fd50 con 0x7f88e00c5820 2023-11-08T08:20:27.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:27.993+0000 7f88e5b56700 1 -- 172.21.15.40:0/2640703269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f88e00711c0 con 0x7f88e00c5820 2023-11-08T08:20:28.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.005+0000 7f88dd7fa700 1 -- 172.21.15.40:0/2640703269 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f88cc053370 con 0x7f88e00c5820 2023-11-08T08:20:28.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.009+0000 7f88e5b56700 1 -- 172.21.15.40:0/2640703269 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f88c4064ee0 msgr2=0x7f88c4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:28.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.009+0000 7f88e5b56700 1 --2- 172.21.15.40:0/2640703269 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f88c4064ee0 0x7f88c4067390 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f88d40017c0 tx=0x7f88d4007040 comp rx=0 tx=0).stop 2023-11-08T08:20:28.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.009+0000 7f88e5b56700 1 -- 172.21.15.40:0/2640703269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88e00c5820 msgr2=0x7f88e015cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:28.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.009+0000 7f88e5b56700 1 --2- 172.21.15.40:0/2640703269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88e00c5820 0x7f88e015cca0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f88cc004840 tx=0x7f88cc00ee10 comp rx=0 tx=0).stop 2023-11-08T08:20:28.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.009+0000 7f88e5b56700 1 -- 172.21.15.40:0/2640703269 shutdown_connections 2023-11-08T08:20:28.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.009+0000 7f88e5b56700 1 --2- 172.21.15.40:0/2640703269 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f88c4064ee0 0x7f88c4067390 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.009+0000 7f88e5b56700 1 --2- 172.21.15.40:0/2640703269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88e015d1e0 0x7f88e00aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.009+0000 7f88e5b56700 1 --2- 172.21.15.40:0/2640703269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88e00c5820 0x7f88e015cca0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.009+0000 7f88e5b56700 1 --2- 172.21.15.40:0/2640703269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88e0155dd0 0x7f88e015b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.009+0000 7f88e5b56700 1 -- 172.21.15.40:0/2640703269 >> 172.21.15.40:0/2640703269 conn(0x7f88e00b0f80 msgr2=0x7f88e00c8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:28.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.009+0000 7f88e5b56700 1 -- 172.21.15.40:0/2640703269 shutdown_connections 2023-11-08T08:20:28.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.009+0000 7f88e5b56700 1 -- 172.21.15.40:0/2640703269 wait complete. 2023-11-08T08:20:28.039 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-08T08:20:28.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.213+0000 7f6f230ed700 1 Processor -- start 2023-11-08T08:20:28.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.213+0000 7f6f230ed700 1 -- start start 2023-11-08T08:20:28.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.217+0000 7f6f230ed700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f1c070080 0x7f6f1c0704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:28.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.217+0000 7f6f230ed700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1c0ad6c0 con 0x7f6f1c160710 2023-11-08T08:20:28.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.217+0000 7f6f230ed700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1c0adc60 con 0x7f6f1c164250 2023-11-08T08:20:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.217+0000 7f6f230ed700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c06b1c0 con 0x7f6f1c070080 2023-11-08T08:20:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.217+0000 7f6f220eb700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6f1c160710 0x7f6f1c0cc320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57172/0 (socket says 172.21.15.40:57172) 2023-11-08T08:20:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.217+0000 7f6f220eb700 1 -- 172.21.15.40:0/962805872 learned_addr learned my addr 172.21.15.40:0/962805872 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.217+0000 7f6f218ea700 1 --2- 172.21.15.40:0/962805872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f1c070080 0x7f6f1c0704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.217+0000 7f6f210e9700 1 -- 172.21.15.40:0/962805872 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672306230 0 0) 0x7f6f1c0ad6c0 con 0x7f6f1c160710 2023-11-08T08:20:28.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.217+0000 7f6f210e9700 1 -- 172.21.15.40:0/962805872 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f080035d0 con 0x7f6f1c160710 2023-11-08T08:20:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.217+0000 7f6f218ea700 1 -- 172.21.15.40:0/962805872 >> v1:172.21.15.169:6789/0 conn(0x7f6f1c164250 legacy=0x7f6f1c1646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.217+0000 7f6f218ea700 1 -- 172.21.15.40:0/962805872 >> v1:172.21.15.40:6789/0 conn(0x7f6f1c160710 legacy=0x7f6f1c0cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:28.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.217+0000 7f6f218ea700 1 -- 172.21.15.40:0/962805872 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1c0709f0 con 0x7f6f1c070080 2023-11-08T08:20:28.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.221+0000 7f6f218ea700 1 --2- 172.21.15.40:0/962805872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f1c070080 0x7f6f1c0704b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009880 tx=0x7f6f0c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb1f95b340bff41d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:28.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.221+0000 7f6f210e9700 1 -- 172.21.15.40:0/962805872 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c011830 con 0x7f6f1c070080 2023-11-08T08:20:28.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.221+0000 7f6f210e9700 1 -- 172.21.15.40:0/962805872 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0c011990 con 0x7f6f1c070080 2023-11-08T08:20:28.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.221+0000 7f6f210e9700 1 -- 172.21.15.40:0/962805872 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c011d20 con 0x7f6f1c070080 2023-11-08T08:20:28.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.221+0000 7f6f230ed700 1 -- 172.21.15.40:0/962805872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f1c070080 msgr2=0x7f6f1c0704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:28.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.221+0000 7f6f230ed700 1 --2- 172.21.15.40:0/962805872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f1c070080 0x7f6f1c0704b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009880 tx=0x7f6f0c009b90 comp rx=0 tx=0).stop 2023-11-08T08:20:28.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.221+0000 7f6f230ed700 1 -- 172.21.15.40:0/962805872 shutdown_connections 2023-11-08T08:20:28.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.221+0000 7f6f230ed700 1 --2- 172.21.15.40:0/962805872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f1c070080 0x7f6f1c0704b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.221+0000 7f6f230ed700 1 -- 172.21.15.40:0/962805872 >> 172.21.15.40:0/962805872 conn(0x7f6f1c0b0f80 msgr2=0x7f6f1c0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:28.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.221+0000 7f6f230ed700 1 -- 172.21.15.40:0/962805872 shutdown_connections 2023-11-08T08:20:28.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.221+0000 7f6f230ed700 1 -- 172.21.15.40:0/962805872 wait complete. 2023-11-08T08:20:28.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.225+0000 7f6f230ed700 1 Processor -- start 2023-11-08T08:20:28.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.225+0000 7f6f230ed700 1 -- start start 2023-11-08T08:20:28.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.225+0000 7f6f230ed700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f1c070080 0x7f6f1c0c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:28.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.225+0000 7f6f230ed700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f1c160710 0x7f6f1c15b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:28.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.225+0000 7f6f230ed700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f1c164250 0x7f6f1c15b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:28.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.225+0000 7f6f230ed700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c1617d0 con 0x7f6f1c070080 2023-11-08T08:20:28.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.225+0000 7f6f230ed700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c161ab0 con 0x7f6f1c160710 2023-11-08T08:20:28.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.225+0000 7f6f230ed700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c161d90 con 0x7f6f1c164250 2023-11-08T08:20:28.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.225+0000 7f6f220eb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f1c070080 0x7f6f1c0c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.225+0000 7f6f220eb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f1c070080 0x7f6f1c0c3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34038/0 (socket says 172.21.15.40:34038) 2023-11-08T08:20:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.225+0000 7f6f220eb700 1 -- 172.21.15.40:0/2890090602 learned_addr learned my addr 172.21.15.40:0/2890090602 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.225+0000 7f6f218ea700 1 --2- 172.21.15.40:0/2890090602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f1c160710 0x7f6f1c15b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:28.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.225+0000 7f6f228ec700 1 --2- 172.21.15.40:0/2890090602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f1c164250 0x7f6f1c15b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.229+0000 7f6f220eb700 1 -- 172.21.15.40:0/2890090602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f1c164250 msgr2=0x7f6f1c15b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.229+0000 7f6f220eb700 1 --2- 172.21.15.40:0/2890090602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f1c164250 0x7f6f1c15b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.229+0000 7f6f220eb700 1 -- 172.21.15.40:0/2890090602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f1c160710 msgr2=0x7f6f1c15b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.229+0000 7f6f220eb700 1 --2- 172.21.15.40:0/2890090602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f1c160710 0x7f6f1c15b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.229+0000 7f6f220eb700 1 -- 172.21.15.40:0/2890090602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0c0094b0 con 0x7f6f1c070080 2023-11-08T08:20:28.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.229+0000 7f6f220eb700 1 --2- 172.21.15.40:0/2890090602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f1c070080 0x7f6f1c0c3c20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800c700 tx=0x7f6f1800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:28.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.229+0000 7f6f12ffd700 1 -- 172.21.15.40:0/2890090602 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f18005d90 con 0x7f6f1c070080 2023-11-08T08:20:28.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.229+0000 7f6f12ffd700 1 -- 172.21.15.40:0/2890090602 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f180070d0 con 0x7f6f1c070080 2023-11-08T08:20:28.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.229+0000 7f6f230ed700 1 -- 172.21.15.40:0/2890090602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f1c06b3d0 con 0x7f6f1c070080 2023-11-08T08:20:28.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.229+0000 7f6f12ffd700 1 -- 172.21.15.40:0/2890090602 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f18007480 con 0x7f6f1c070080 2023-11-08T08:20:28.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.229+0000 7f6f230ed700 1 -- 172.21.15.40:0/2890090602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f1c0cca30 con 0x7f6f1c070080 2023-11-08T08:20:28.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.229+0000 7f6f230ed700 1 -- 172.21.15.40:0/2890090602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f1c06b6b0 con 0x7f6f1c070080 2023-11-08T08:20:28.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.233+0000 7f6f12ffd700 1 -- 172.21.15.40:0/2890090602 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f6f18019030 con 0x7f6f1c070080 2023-11-08T08:20:28.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.233+0000 7f6f12ffd700 1 --2- 172.21.15.40:0/2890090602 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6f00064ee0 0x7f6f00067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:28.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.233+0000 7f6f12ffd700 1 -- 172.21.15.40:0/2890090602 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f6f180811f0 con 0x7f6f1c070080 2023-11-08T08:20:28.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.233+0000 7f6f218ea700 1 --2- 172.21.15.40:0/2890090602 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6f00064ee0 0x7f6f00067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:28.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.233+0000 7f6f218ea700 1 --2- 172.21.15.40:0/2890090602 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6f00064ee0 0x7f6f00067390 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0090b0 tx=0x7f6f0c010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.237+0000 7f6f12ffd700 1 -- 172.21.15.40:0/2890090602 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f6f1804f420 con 0x7f6f1c070080 2023-11-08T08:20:28.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.469+0000 7f6f230ed700 1 -- 172.21.15.40:0/2890090602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f6f1c073cb0 con 0x7f6f1c070080 2023-11-08T08:20:28.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.477+0000 7f6f12ffd700 1 -- 172.21.15.40:0/2890090602 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6f180170a0 con 0x7f6f1c070080 2023-11-08T08:20:28.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.481+0000 7f6f230ed700 1 -- 172.21.15.40:0/2890090602 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6f00064ee0 msgr2=0x7f6f00067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:28.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.481+0000 7f6f230ed700 1 --2- 172.21.15.40:0/2890090602 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6f00064ee0 0x7f6f00067390 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0090b0 tx=0x7f6f0c010000 comp rx=0 tx=0).stop 2023-11-08T08:20:28.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.481+0000 7f6f230ed700 1 -- 172.21.15.40:0/2890090602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f1c070080 msgr2=0x7f6f1c0c3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:28.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.481+0000 7f6f230ed700 1 --2- 172.21.15.40:0/2890090602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f1c070080 0x7f6f1c0c3c20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800c700 tx=0x7f6f1800ca10 comp rx=0 tx=0).stop 2023-11-08T08:20:28.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.481+0000 7f6f230ed700 1 -- 172.21.15.40:0/2890090602 shutdown_connections 2023-11-08T08:20:28.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.481+0000 7f6f230ed700 1 --2- 172.21.15.40:0/2890090602 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6f00064ee0 0x7f6f00067390 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.481+0000 7f6f230ed700 1 --2- 172.21.15.40:0/2890090602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f1c164250 0x7f6f1c15b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.481+0000 7f6f230ed700 1 --2- 172.21.15.40:0/2890090602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f1c160710 0x7f6f1c15b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.481+0000 7f6f230ed700 1 --2- 172.21.15.40:0/2890090602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f1c070080 0x7f6f1c0c3c20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.481+0000 7f6f230ed700 1 -- 172.21.15.40:0/2890090602 >> 172.21.15.40:0/2890090602 conn(0x7f6f1c0b0f80 msgr2=0x7f6f1c0af800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:28.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.481+0000 7f6f230ed700 1 -- 172.21.15.40:0/2890090602 shutdown_connections 2023-11-08T08:20:28.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.481+0000 7f6f230ed700 1 -- 172.21.15.40:0/2890090602 wait complete. 2023-11-08T08:20:28.510 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-08T08:20:28.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.701+0000 7f1b3a4ee700 1 Processor -- start 2023-11-08T08:20:28.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.701+0000 7f1b3a4ee700 1 -- start start 2023-11-08T08:20:28.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.701+0000 7f1b3a4ee700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1b3415c680 0x7f1b340b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.701+0000 7f1b3a4ee700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1b340b5040 0x7f1b340ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.701+0000 7f1b3a4ee700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b340ceaa0 con 0x7f1b340ce640 2023-11-08T08:20:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.701+0000 7f1b3a4ee700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b340b6700 con 0x7f1b340b5040 2023-11-08T08:20:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.701+0000 7f1b3a4ee700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b34160bc0 con 0x7f1b3415c680 2023-11-08T08:20:28.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.701+0000 7f1b337fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1b340b5040 0x7f1b340ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:28.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.701+0000 7f1b337fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1b340b5040 0x7f1b340ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34054/0 (socket says 172.21.15.40:34054) 2023-11-08T08:20:28.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.701+0000 7f1b337fe700 1 -- 172.21.15.40:0/3034765082 learned_addr learned my addr 172.21.15.40:0/3034765082 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:28.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.701+0000 7f1b33fff700 1 --2- 172.21.15.40:0/3034765082 >> v2:172.21.15.169:3300/0 conn(0x7f1b3415c680 0x7f1b340b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:28.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.701+0000 7f1b337fe700 1 -- 172.21.15.40:0/3034765082 >> v2:172.21.15.169:3300/0 conn(0x7f1b3415c680 msgr2=0x7f1b340b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:28.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.701+0000 7f1b337fe700 1 --2- 172.21.15.40:0/3034765082 >> v2:172.21.15.169:3300/0 conn(0x7f1b3415c680 0x7f1b340b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.701+0000 7f1b337fe700 1 -- 172.21.15.40:0/3034765082 >> v1:172.21.15.169:6789/0 conn(0x7f1b340ce640 legacy=0x7f1b3406c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:28.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.701+0000 7f1b337fe700 1 -- 172.21.15.40:0/3034765082 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b340cd4f0 con 0x7f1b340b5040 2023-11-08T08:20:28.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.705+0000 7f1b337fe700 1 --2- 172.21.15.40:0/3034765082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b340b5040 0x7f1b340ce100 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1b340cd200 tx=0x7f1b28009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a440921e8586d04 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:28.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.705+0000 7f1b32ffd700 1 -- 172.21.15.40:0/3034765082 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b28011830 con 0x7f1b340b5040 2023-11-08T08:20:28.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.705+0000 7f1b32ffd700 1 -- 172.21.15.40:0/3034765082 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b28011990 con 0x7f1b340b5040 2023-11-08T08:20:28.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.705+0000 7f1b32ffd700 1 -- 172.21.15.40:0/3034765082 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b28011d20 con 0x7f1b340b5040 2023-11-08T08:20:28.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.705+0000 7f1b3a4ee700 1 -- 172.21.15.40:0/3034765082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b340b5040 msgr2=0x7f1b340ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:28.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.705+0000 7f1b3a4ee700 1 --2- 172.21.15.40:0/3034765082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b340b5040 0x7f1b340ce100 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1b340cd200 tx=0x7f1b28009bf0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.705+0000 7f1b3a4ee700 1 -- 172.21.15.40:0/3034765082 shutdown_connections 2023-11-08T08:20:28.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.705+0000 7f1b3a4ee700 1 --2- 172.21.15.40:0/3034765082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b340b5040 0x7f1b340ce100 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.705+0000 7f1b3a4ee700 1 --2- 172.21.15.40:0/3034765082 >> v2:172.21.15.169:3300/0 conn(0x7f1b3415c680 0x7f1b340b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.705+0000 7f1b3a4ee700 1 -- 172.21.15.40:0/3034765082 >> 172.21.15.40:0/3034765082 conn(0x7f1b340d2c00 msgr2=0x7f1b340d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:28.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.705+0000 7f1b3a4ee700 1 -- 172.21.15.40:0/3034765082 shutdown_connections 2023-11-08T08:20:28.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.705+0000 7f1b3a4ee700 1 -- 172.21.15.40:0/3034765082 wait complete. 2023-11-08T08:20:28.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.709+0000 7f1b3a4ee700 1 Processor -- start 2023-11-08T08:20:28.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.709+0000 7f1b3a4ee700 1 -- start start 2023-11-08T08:20:28.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b3a4ee700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b340b5040 0x7f1b340cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:28.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b3a4ee700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b340ce640 0x7f1b3415f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:28.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b3a4ee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b3415c680 0x7f1b340c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:28.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b3a4ee700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3415f9d0 con 0x7f1b3415c680 2023-11-08T08:20:28.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b3a4ee700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b34158a60 con 0x7f1b340b5040 2023-11-08T08:20:28.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b3a4ee700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b340cc690 con 0x7f1b340ce640 2023-11-08T08:20:28.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b38a8b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b3415c680 0x7f1b340c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:28.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b38a8b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b3415c680 0x7f1b340c8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34060/0 (socket says 172.21.15.40:34060) 2023-11-08T08:20:28.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b38a8b700 1 -- 172.21.15.40:0/2704085412 learned_addr learned my addr 172.21.15.40:0/2704085412 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b33fff700 1 --2- 172.21.15.40:0/2704085412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b340b5040 0x7f1b340cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b337fe700 1 --2- 172.21.15.40:0/2704085412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b340ce640 0x7f1b3415f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:28.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b38a8b700 1 -- 172.21.15.40:0/2704085412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b340ce640 msgr2=0x7f1b3415f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:28.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b38a8b700 1 --2- 172.21.15.40:0/2704085412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b340ce640 0x7f1b3415f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b38a8b700 1 -- 172.21.15.40:0/2704085412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b340b5040 msgr2=0x7f1b340cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:28.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b38a8b700 1 --2- 172.21.15.40:0/2704085412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b340b5040 0x7f1b340cb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b38a8b700 1 -- 172.21.15.40:0/2704085412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b28009570 con 0x7f1b3415c680 2023-11-08T08:20:28.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b38a8b700 1 --2- 172.21.15.40:0/2704085412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b3415c680 0x7f1b340c8a60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00cc50 tx=0x7f1b2c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:28.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b30ff9700 1 -- 172.21.15.40:0/2704085412 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c003c20 con 0x7f1b3415c680 2023-11-08T08:20:28.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b30ff9700 1 -- 172.21.15.40:0/2704085412 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2c003d80 con 0x7f1b3415c680 2023-11-08T08:20:28.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b30ff9700 1 -- 172.21.15.40:0/2704085412 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c00aab0 con 0x7f1b3415c680 2023-11-08T08:20:28.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b3a4ee700 1 -- 172.21.15.40:0/2704085412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b340b5670 con 0x7f1b3415c680 2023-11-08T08:20:28.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.713+0000 7f1b3a4ee700 1 -- 172.21.15.40:0/2704085412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b3406b730 con 0x7f1b3415c680 2023-11-08T08:20:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.717+0000 7f1b3a4ee700 1 -- 172.21.15.40:0/2704085412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b34160290 con 0x7f1b3415c680 2023-11-08T08:20:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.717+0000 7f1b30ff9700 1 -- 172.21.15.40:0/2704085412 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1b2c010050 con 0x7f1b3415c680 2023-11-08T08:20:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.721+0000 7f1b30ff9700 1 --2- 172.21.15.40:0/2704085412 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1b20064f30 0x7f1b200673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.721+0000 7f1b30ff9700 1 -- 172.21.15.40:0/2704085412 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f1b2c081b70 con 0x7f1b3415c680 2023-11-08T08:20:28.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.721+0000 7f1b33fff700 1 --2- 172.21.15.40:0/2704085412 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1b20064f30 0x7f1b200673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:28.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.721+0000 7f1b33fff700 1 --2- 172.21.15.40:0/2704085412 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1b20064f30 0x7f1b200673e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009bf0 tx=0x7f1b24009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:28.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.725+0000 7f1b30ff9700 1 -- 172.21.15.40:0/2704085412 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1b2c04feb0 con 0x7f1b3415c680 2023-11-08T08:20:28.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.957+0000 7f1b3a4ee700 1 -- 172.21.15.40:0/2704085412 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f1b34158bd0 con 0x7f1b3415c680 2023-11-08T08:20:28.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.965+0000 7f1b30ff9700 1 -- 172.21.15.40:0/2704085412 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1b34158bd0 con 0x7f1b3415c680 2023-11-08T08:20:28.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.969+0000 7f1b3a4ee700 1 -- 172.21.15.40:0/2704085412 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1b20064f30 msgr2=0x7f1b200673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:28.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.969+0000 7f1b3a4ee700 1 --2- 172.21.15.40:0/2704085412 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1b20064f30 0x7f1b200673e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009bf0 tx=0x7f1b24009240 comp rx=0 tx=0).stop 2023-11-08T08:20:28.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.969+0000 7f1b3a4ee700 1 -- 172.21.15.40:0/2704085412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b3415c680 msgr2=0x7f1b340c8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:28.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.969+0000 7f1b3a4ee700 1 --2- 172.21.15.40:0/2704085412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b3415c680 0x7f1b340c8a60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00cc50 tx=0x7f1b2c00cf60 comp rx=0 tx=0).stop 2023-11-08T08:20:28.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.969+0000 7f1b3a4ee700 1 -- 172.21.15.40:0/2704085412 shutdown_connections 2023-11-08T08:20:28.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.969+0000 7f1b3a4ee700 1 --2- 172.21.15.40:0/2704085412 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1b20064f30 0x7f1b200673e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.969+0000 7f1b3a4ee700 1 --2- 172.21.15.40:0/2704085412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b3415c680 0x7f1b340c8a60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.969+0000 7f1b3a4ee700 1 --2- 172.21.15.40:0/2704085412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b340ce640 0x7f1b3415f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.969+0000 7f1b3a4ee700 1 --2- 172.21.15.40:0/2704085412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b340b5040 0x7f1b340cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:28.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.969+0000 7f1b3a4ee700 1 -- 172.21.15.40:0/2704085412 >> 172.21.15.40:0/2704085412 conn(0x7f1b340d2c00 msgr2=0x7f1b340d1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:28.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.969+0000 7f1b3a4ee700 1 -- 172.21.15.40:0/2704085412 shutdown_connections 2023-11-08T08:20:28.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:28.969+0000 7f1b3a4ee700 1 -- 172.21.15.40:0/2704085412 wait complete. 2023-11-08T08:20:28.998 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-08T08:20:29.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.185+0000 7f4cad68e700 1 Processor -- start 2023-11-08T08:20:29.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.185+0000 7f4cad68e700 1 -- start start 2023-11-08T08:20:29.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4cad68e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ca8070790 0x7f4ca8164fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:29.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4cad68e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca815a550 con 0x7f4ca80b1640 2023-11-08T08:20:29.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4cad68e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca806b0a0 con 0x7f4ca80b43a0 2023-11-08T08:20:29.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4cad68e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ca80caac0 con 0x7f4ca8070790 2023-11-08T08:20:29.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4c9ffff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4ca80b1640 0x7f4ca8070080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57178/0 (socket says 172.21.15.40:57178) 2023-11-08T08:20:29.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4c9ffff700 1 -- 172.21.15.40:0/813856056 learned_addr learned my addr 172.21.15.40:0/813856056 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4ca77fe700 1 --2- 172.21.15.40:0/813856056 >> v2:172.21.15.40:3300/0 conn(0x7f4ca8070790 0x7f4ca8164fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:29.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4ca67fc700 1 -- 172.21.15.40:0/813856056 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 46914107 0 0) 0x7f4c90001110 con 0x7f4ca80b1640 2023-11-08T08:20:29.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4ca67fc700 1 -- 172.21.15.40:0/813856056 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c940035d0 con 0x7f4ca80b1640 2023-11-08T08:20:29.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4ca67fc700 1 -- 172.21.15.40:0/813856056 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1742389994 0 0) 0x7f4ca806b0a0 con 0x7f4ca80b43a0 2023-11-08T08:20:29.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4ca67fc700 1 -- 172.21.15.40:0/813856056 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c90001110 con 0x7f4ca80b43a0 2023-11-08T08:20:29.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4ca67fc700 1 -- 172.21.15.40:0/813856056 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1445397180 0 0) 0x7f4c940035d0 con 0x7f4ca80b1640 2023-11-08T08:20:29.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4ca67fc700 1 -- 172.21.15.40:0/813856056 >> v2:172.21.15.40:3300/0 conn(0x7f4ca8070790 msgr2=0x7f4ca8164fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:29.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4ca67fc700 1 --2- 172.21.15.40:0/813856056 >> v2:172.21.15.40:3300/0 conn(0x7f4ca8070790 0x7f4ca8164fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4ca67fc700 1 -- 172.21.15.40:0/813856056 >> v1:172.21.15.169:6789/0 conn(0x7f4ca80b43a0 legacy=0x7f4ca815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4ca67fc700 1 -- 172.21.15.40:0/813856056 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca80cada0 con 0x7f4ca80b1640 2023-11-08T08:20:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4ca67fc700 1 -- 172.21.15.40:0/813856056 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f4c90001800 con 0x7f4ca80b1640 2023-11-08T08:20:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4ca67fc700 1 -- 172.21.15.40:0/813856056 >> v1:172.21.15.40:6789/0 conn(0x7f4ca80b1640 legacy=0x7f4ca8070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:29.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4ca67fc700 1 --2- 172.21.15.40:0/813856056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c94003bb0 0x7f4c94003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:29.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4ca67fc700 1 --2- 172.21.15.40:0/813856056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c94004870 0x7f4c94004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:29.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4ca67fc700 1 --2- 172.21.15.40:0/813856056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c94005220 0x7f4c9400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:29.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4ca67fc700 1 -- 172.21.15.40:0/813856056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca80caac0 con 0x7f4c94003bb0 2023-11-08T08:20:29.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4ca67fc700 1 -- 172.21.15.40:0/813856056 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9400c100 con 0x7f4c94005220 2023-11-08T08:20:29.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4ca67fc700 1 -- 172.21.15.40:0/813856056 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9400c2c0 con 0x7f4c94004870 2023-11-08T08:20:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.189+0000 7f4ca6ffd700 1 --2- 172.21.15.40:0/813856056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c94003bb0 0x7f4c94003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.193+0000 7f4c9ffff700 1 --2- 172.21.15.40:0/813856056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c94004870 0x7f4c94004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.193+0000 7f4ca6ffd700 1 -- 172.21.15.40:0/813856056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c94004870 msgr2=0x7f4c94004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.193+0000 7f4ca6ffd700 1 --2- 172.21.15.40:0/813856056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c94004870 0x7f4c94004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.193+0000 7f4ca6ffd700 1 -- 172.21.15.40:0/813856056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c94005220 msgr2=0x7f4c9400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.193+0000 7f4ca77fe700 1 --2- 172.21.15.40:0/813856056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c94005220 0x7f4c9400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.193+0000 7f4ca6ffd700 1 --2- 172.21.15.40:0/813856056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c94005220 0x7f4c9400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.193+0000 7f4ca6ffd700 1 -- 172.21.15.40:0/813856056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c940035d0 con 0x7f4c94003bb0 2023-11-08T08:20:29.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.193+0000 7f4ca77fe700 1 --2- 172.21.15.40:0/813856056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c94005220 0x7f4c9400ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:20:29.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.193+0000 7f4ca77fe700 1 -- 172.21.15.40:0/813856056 reap_dead start 2023-11-08T08:20:29.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.193+0000 7f4ca6ffd700 1 --2- 172.21.15.40:0/813856056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c94003bb0 0x7f4c94003f80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4c98007fd0 tx=0x7f4c940058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf2e128876f2b54 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:29.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.193+0000 7f4ca67fc700 1 -- 172.21.15.40:0/813856056 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c98001450 con 0x7f4c94003bb0 2023-11-08T08:20:29.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.193+0000 7f4ca67fc700 1 -- 172.21.15.40:0/813856056 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c98006a80 con 0x7f4c94003bb0 2023-11-08T08:20:29.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.193+0000 7f4ca67fc700 1 -- 172.21.15.40:0/813856056 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c98006e30 con 0x7f4c94003bb0 2023-11-08T08:20:29.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.193+0000 7f4cad68e700 1 -- 172.21.15.40:0/813856056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c94003bb0 msgr2=0x7f4c94003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:29.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.193+0000 7f4cad68e700 1 --2- 172.21.15.40:0/813856056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c94003bb0 0x7f4c94003f80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4c98007fd0 tx=0x7f4c940058a0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.193+0000 7f4cad68e700 1 -- 172.21.15.40:0/813856056 shutdown_connections 2023-11-08T08:20:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.193+0000 7f4cad68e700 1 --2- 172.21.15.40:0/813856056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c94003bb0 0x7f4c94003f80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.193+0000 7f4cad68e700 1 -- 172.21.15.40:0/813856056 >> 172.21.15.40:0/813856056 conn(0x7f4ca80b0f80 msgr2=0x7f4ca80ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.193+0000 7f4cad68e700 1 -- 172.21.15.40:0/813856056 shutdown_connections 2023-11-08T08:20:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.193+0000 7f4cad68e700 1 -- 172.21.15.40:0/813856056 wait complete. 2023-11-08T08:20:29.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.197+0000 7f4cad68e700 1 Processor -- start 2023-11-08T08:20:29.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.197+0000 7f4cad68e700 1 -- start start 2023-11-08T08:20:29.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4cad68e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c94003bb0 0x7f4ca815a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4cad68e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ca815c990 0x7f4ca80ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4cad68e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ca81f56c0 0x7f4ca81f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4cad68e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca815b5e0 con 0x7f4ca81f56c0 2023-11-08T08:20:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4cad68e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca815ad40 con 0x7f4c94003bb0 2023-11-08T08:20:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4cad68e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca815b020 con 0x7f4ca815c990 2023-11-08T08:20:29.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4ca77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ca81f56c0 0x7f4ca81f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:29.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4ca77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ca81f56c0 0x7f4ca81f5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34076/0 (socket says 172.21.15.40:34076) 2023-11-08T08:20:29.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4ca77fe700 1 -- 172.21.15.40:0/3561931671 learned_addr learned my addr 172.21.15.40:0/3561931671 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:29.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4ca6ffd700 1 --2- 172.21.15.40:0/3561931671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c94003bb0 0x7f4ca815a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:29.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4c9ffff700 1 --2- 172.21.15.40:0/3561931671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ca815c990 0x7f4ca80ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:29.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4ca77fe700 1 -- 172.21.15.40:0/3561931671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ca815c990 msgr2=0x7f4ca80ad660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:29.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4ca77fe700 1 --2- 172.21.15.40:0/3561931671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ca815c990 0x7f4ca80ad660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4ca77fe700 1 -- 172.21.15.40:0/3561931671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c94003bb0 msgr2=0x7f4ca815a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:29.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4ca77fe700 1 --2- 172.21.15.40:0/3561931671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c94003bb0 0x7f4ca815a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4ca77fe700 1 -- 172.21.15.40:0/3561931671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca806b5d0 con 0x7f4ca81f56c0 2023-11-08T08:20:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4ca6ffd700 1 --2- 172.21.15.40:0/3561931671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c94003bb0 0x7f4ca815a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4c9ffff700 1 --2- 172.21.15.40:0/3561931671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ca815c990 0x7f4ca80ad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4ca77fe700 1 --2- 172.21.15.40:0/3561931671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ca81f56c0 0x7f4ca81f5b60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4ca80ceef0 tx=0x7f4ca000f430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4c9f7fe700 1 -- 172.21.15.40:0/3561931671 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca000fe60 con 0x7f4ca81f56c0 2023-11-08T08:20:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4c9f7fe700 1 -- 172.21.15.40:0/3561931671 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca0016420 con 0x7f4ca81f56c0 2023-11-08T08:20:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4c9f7fe700 1 -- 172.21.15.40:0/3561931671 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0016790 con 0x7f4ca81f56c0 2023-11-08T08:20:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4cad68e700 1 -- 172.21.15.40:0/3561931671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca806b2f0 con 0x7f4ca81f56c0 2023-11-08T08:20:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.201+0000 7f4cad68e700 1 -- 172.21.15.40:0/3561931671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca80ca7b0 con 0x7f4ca81f56c0 2023-11-08T08:20:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.205+0000 7f4cad68e700 1 -- 172.21.15.40:0/3561931671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca8073cb0 con 0x7f4ca81f56c0 2023-11-08T08:20:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.205+0000 7f4c9f7fe700 1 -- 172.21.15.40:0/3561931671 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4ca00168f0 con 0x7f4ca81f56c0 2023-11-08T08:20:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.209+0000 7f4c9f7fe700 1 --2- 172.21.15.40:0/3561931671 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4c88064ee0 0x7f4c88067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.209+0000 7f4c9f7fe700 1 -- 172.21.15.40:0/3561931671 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f4ca0082ae0 con 0x7f4ca81f56c0 2023-11-08T08:20:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.209+0000 7f4ca6ffd700 1 --2- 172.21.15.40:0/3561931671 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4c88064ee0 0x7f4c88067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.209+0000 7f4ca6ffd700 1 --2- 172.21.15.40:0/3561931671 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4c88064ee0 0x7f4c88067390 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4c98015040 tx=0x7f4c98005040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:29.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.213+0000 7f4c9f7fe700 1 -- 172.21.15.40:0/3561931671 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4ca0050da0 con 0x7f4ca81f56c0 2023-11-08T08:20:29.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.441+0000 7f4cad68e700 1 -- 172.21.15.40:0/3561931671 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f4ca8073200 con 0x7f4ca81f56c0 2023-11-08T08:20:29.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.449+0000 7f4c9f7fe700 1 -- 172.21.15.40:0/3561931671 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4ca8073200 con 0x7f4ca81f56c0 2023-11-08T08:20:29.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.453+0000 7f4cad68e700 1 -- 172.21.15.40:0/3561931671 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4c88064ee0 msgr2=0x7f4c88067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:29.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.453+0000 7f4cad68e700 1 --2- 172.21.15.40:0/3561931671 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4c88064ee0 0x7f4c88067390 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4c98015040 tx=0x7f4c98005040 comp rx=0 tx=0).stop 2023-11-08T08:20:29.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.453+0000 7f4cad68e700 1 -- 172.21.15.40:0/3561931671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ca81f56c0 msgr2=0x7f4ca81f5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:29.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.453+0000 7f4cad68e700 1 --2- 172.21.15.40:0/3561931671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ca81f56c0 0x7f4ca81f5b60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4ca80ceef0 tx=0x7f4ca000f430 comp rx=0 tx=0).stop 2023-11-08T08:20:29.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.453+0000 7f4cad68e700 1 -- 172.21.15.40:0/3561931671 shutdown_connections 2023-11-08T08:20:29.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.453+0000 7f4cad68e700 1 --2- 172.21.15.40:0/3561931671 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4c88064ee0 0x7f4c88067390 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.453+0000 7f4cad68e700 1 --2- 172.21.15.40:0/3561931671 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ca81f56c0 0x7f4ca81f5b60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.453+0000 7f4cad68e700 1 --2- 172.21.15.40:0/3561931671 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ca815c990 0x7f4ca80ad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.453+0000 7f4cad68e700 1 --2- 172.21.15.40:0/3561931671 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c94003bb0 0x7f4ca815a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.453+0000 7f4cad68e700 1 -- 172.21.15.40:0/3561931671 >> 172.21.15.40:0/3561931671 conn(0x7f4ca80b0f80 msgr2=0x7f4ca8069bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:29.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.453+0000 7f4cad68e700 1 -- 172.21.15.40:0/3561931671 shutdown_connections 2023-11-08T08:20:29.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.453+0000 7f4cad68e700 1 -- 172.21.15.40:0/3561931671 wait complete. 2023-11-08T08:20:29.484 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-08T08:20:29.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.657+0000 7effdd718700 1 Processor -- start 2023-11-08T08:20:29.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.657+0000 7effdd718700 1 -- start start 2023-11-08T08:20:29.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effdd718700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7effd80b43a0 0x7effd815e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:29.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effdd718700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7effd80b1640 0x7effd8070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:29.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effdd718700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd80c9040 0x7effd80705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:29.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effdd718700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd815a550 con 0x7effd80b43a0 2023-11-08T08:20:29.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effdd718700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd806b0a0 con 0x7effd80b1640 2023-11-08T08:20:29.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effdd718700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd80caac0 con 0x7effd80c9040 2023-11-08T08:20:29.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effd7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7effd80b43a0 0x7effd815e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:29.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effd7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7effd80b43a0 0x7effd815e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34086/0 (socket says 172.21.15.40:34086) 2023-11-08T08:20:29.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effd7fff700 1 -- 172.21.15.40:0/9375978 learned_addr learned my addr 172.21.15.40:0/9375978 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:29.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effcffff700 1 --2- 172.21.15.40:0/9375978 >> v2:172.21.15.169:3300/0 conn(0x7effd80b1640 0x7effd8070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:29.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effdcf17700 1 --2- 172.21.15.40:0/9375978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd80c9040 0x7effd80705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:29.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effd7fff700 1 -- 172.21.15.40:0/9375978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd80c9040 msgr2=0x7effd80705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:29.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effd7fff700 1 --2- 172.21.15.40:0/9375978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd80c9040 0x7effd80705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effd7fff700 1 -- 172.21.15.40:0/9375978 >> v2:172.21.15.169:3300/0 conn(0x7effd80b1640 msgr2=0x7effd8070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:29.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effd7fff700 1 --2- 172.21.15.40:0/9375978 >> v2:172.21.15.169:3300/0 conn(0x7effd80b1640 0x7effd8070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effd7fff700 1 -- 172.21.15.40:0/9375978 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd80cada0 con 0x7effd80b43a0 2023-11-08T08:20:29.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effdcf17700 1 --2- 172.21.15.40:0/9375978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd80c9040 0x7effd80705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:29.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effd7fff700 1 --2- 172.21.15.40:0/9375978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effd80b43a0 0x7effd815e5d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7effd8072d40 tx=0x7effc8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17238513dd9c5dc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:29.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effd77fe700 1 -- 172.21.15.40:0/9375978 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc8011830 con 0x7effd80b43a0 2023-11-08T08:20:29.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effd77fe700 1 -- 172.21.15.40:0/9375978 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc8011990 con 0x7effd80b43a0 2023-11-08T08:20:29.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.661+0000 7effd77fe700 1 -- 172.21.15.40:0/9375978 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc8011d20 con 0x7effd80b43a0 2023-11-08T08:20:29.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.665+0000 7effdd718700 1 -- 172.21.15.40:0/9375978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effd80b43a0 msgr2=0x7effd815e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:29.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.665+0000 7effdd718700 1 --2- 172.21.15.40:0/9375978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effd80b43a0 0x7effd815e5d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7effd8072d40 tx=0x7effc8009bf0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.665+0000 7effdd718700 1 -- 172.21.15.40:0/9375978 shutdown_connections 2023-11-08T08:20:29.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.665+0000 7effdd718700 1 --2- 172.21.15.40:0/9375978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd80c9040 0x7effd80705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.665+0000 7effdd718700 1 --2- 172.21.15.40:0/9375978 >> v2:172.21.15.169:3300/0 conn(0x7effd80b1640 0x7effd8070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.665+0000 7effdd718700 1 --2- 172.21.15.40:0/9375978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effd80b43a0 0x7effd815e5d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.665+0000 7effdd718700 1 -- 172.21.15.40:0/9375978 >> 172.21.15.40:0/9375978 conn(0x7effd80b0f80 msgr2=0x7effd80ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:29.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.665+0000 7effdd718700 1 -- 172.21.15.40:0/9375978 shutdown_connections 2023-11-08T08:20:29.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.665+0000 7effdd718700 1 -- 172.21.15.40:0/9375978 wait complete. 2023-11-08T08:20:29.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.669+0000 7effdd718700 1 Processor -- start 2023-11-08T08:20:29.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.669+0000 7effdd718700 1 -- start start 2023-11-08T08:20:29.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.669+0000 7effdd718700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd80b1640 0x7effd81f4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:29.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.669+0000 7effdd718700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effd80b43a0 0x7effd80c34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:29.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.669+0000 7effdd718700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effd80c9040 0x7effd80b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:29.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.669+0000 7effdd718700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd80abf10 con 0x7effd80b43a0 2023-11-08T08:20:29.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.669+0000 7effdd718700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd80b3540 con 0x7effd80c9040 2023-11-08T08:20:29.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.669+0000 7effdd718700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd80b3820 con 0x7effd80b1640 2023-11-08T08:20:29.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.669+0000 7effcffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effd80b43a0 0x7effd80c34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:29.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.669+0000 7effcffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effd80b43a0 0x7effd80c34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34090/0 (socket says 172.21.15.40:34090) 2023-11-08T08:20:29.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.669+0000 7effcffff700 1 -- 172.21.15.40:0/2361280547 learned_addr learned my addr 172.21.15.40:0/2361280547 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:29.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.669+0000 7effd7fff700 1 --2- 172.21.15.40:0/2361280547 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd80b1640 0x7effd81f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:29.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.669+0000 7effdcf17700 1 --2- 172.21.15.40:0/2361280547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effd80c9040 0x7effd80b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:29.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.669+0000 7effcffff700 1 -- 172.21.15.40:0/2361280547 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd80b1640 msgr2=0x7effd81f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.669+0000 7effcffff700 1 --2- 172.21.15.40:0/2361280547 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd80b1640 0x7effd81f4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.669+0000 7effcffff700 1 -- 172.21.15.40:0/2361280547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effd80c9040 msgr2=0x7effd80b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.669+0000 7effcffff700 1 --2- 172.21.15.40:0/2361280547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effd80c9040 0x7effd80b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.669+0000 7effcffff700 1 -- 172.21.15.40:0/2361280547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc8009570 con 0x7effd80b43a0 2023-11-08T08:20:29.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.669+0000 7effdcf17700 1 --2- 172.21.15.40:0/2361280547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effd80c9040 0x7effd80b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:29.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.669+0000 7effcffff700 1 --2- 172.21.15.40:0/2361280547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effd80b43a0 0x7effd80c34c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7effc000e940 tx=0x7effc000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:29.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.673+0000 7effd57fa700 1 -- 172.21.15.40:0/2361280547 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc000ee90 con 0x7effd80b43a0 2023-11-08T08:20:29.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.673+0000 7effd57fa700 1 -- 172.21.15.40:0/2361280547 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc000c850 con 0x7effd80b43a0 2023-11-08T08:20:29.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.673+0000 7effd57fa700 1 -- 172.21.15.40:0/2361280547 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc000cbe0 con 0x7effd80b43a0 2023-11-08T08:20:29.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.673+0000 7effdd718700 1 -- 172.21.15.40:0/2361280547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd806b5d0 con 0x7effd80b43a0 2023-11-08T08:20:29.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.673+0000 7effdd718700 1 -- 172.21.15.40:0/2361280547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd806b2f0 con 0x7effd80b43a0 2023-11-08T08:20:29.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.673+0000 7effdd718700 1 -- 172.21.15.40:0/2361280547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd815eb10 con 0x7effd80b43a0 2023-11-08T08:20:29.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.677+0000 7effd57fa700 1 -- 172.21.15.40:0/2361280547 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7effc0019030 con 0x7effd80b43a0 2023-11-08T08:20:29.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.677+0000 7effd57fa700 1 --2- 172.21.15.40:0/2361280547 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7effb8064ee0 0x7effb8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:29.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.677+0000 7effd57fa700 1 -- 172.21.15.40:0/2361280547 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7effc0081c70 con 0x7effd80b43a0 2023-11-08T08:20:29.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.677+0000 7effd7fff700 1 --2- 172.21.15.40:0/2361280547 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7effb8064ee0 0x7effb8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:29.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.677+0000 7effd7fff700 1 --2- 172.21.15.40:0/2361280547 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7effb8064ee0 0x7effb8067390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7effc8009860 tx=0x7effc8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:29.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.685+0000 7effd57fa700 1 -- 172.21.15.40:0/2361280547 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7effc004ff30 con 0x7effd80b43a0 2023-11-08T08:20:29.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.925+0000 7effdd718700 1 -- 172.21.15.40:0/2361280547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7effd8073cb0 con 0x7effd80b43a0 2023-11-08T08:20:29.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.933+0000 7effd57fa700 1 -- 172.21.15.40:0/2361280547 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7effc0053550 con 0x7effd80b43a0 2023-11-08T08:20:29.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.937+0000 7effdd718700 1 -- 172.21.15.40:0/2361280547 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7effb8064ee0 msgr2=0x7effb8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:29.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.937+0000 7effdd718700 1 --2- 172.21.15.40:0/2361280547 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7effb8064ee0 0x7effb8067390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7effc8009860 tx=0x7effc8008030 comp rx=0 tx=0).stop 2023-11-08T08:20:29.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.937+0000 7effdd718700 1 -- 172.21.15.40:0/2361280547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effd80b43a0 msgr2=0x7effd80c34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:29.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.937+0000 7effdd718700 1 --2- 172.21.15.40:0/2361280547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effd80b43a0 0x7effd80c34c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7effc000e940 tx=0x7effc000ec50 comp rx=0 tx=0).stop 2023-11-08T08:20:29.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.937+0000 7effdd718700 1 -- 172.21.15.40:0/2361280547 shutdown_connections 2023-11-08T08:20:29.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.937+0000 7effdd718700 1 --2- 172.21.15.40:0/2361280547 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7effb8064ee0 0x7effb8067390 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.937+0000 7effdd718700 1 --2- 172.21.15.40:0/2361280547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effd80c9040 0x7effd80b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.937+0000 7effdd718700 1 --2- 172.21.15.40:0/2361280547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effd80b43a0 0x7effd80c34c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.937+0000 7effdd718700 1 --2- 172.21.15.40:0/2361280547 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effd80b1640 0x7effd81f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:29.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.937+0000 7effdd718700 1 -- 172.21.15.40:0/2361280547 >> 172.21.15.40:0/2361280547 conn(0x7effd80b0f80 msgr2=0x7effd8159ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:29.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.937+0000 7effdd718700 1 -- 172.21.15.40:0/2361280547 shutdown_connections 2023-11-08T08:20:29.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:29.937+0000 7effdd718700 1 -- 172.21.15.40:0/2361280547 wait complete. 2023-11-08T08:20:29.967 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:20:30.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.141+0000 7facb648c700 1 Processor -- start 2023-11-08T08:20:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.141+0000 7facb648c700 1 -- start start 2023-11-08T08:20:30.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.145+0000 7facb648c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb015c1a0 0x7facb006c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:30.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.145+0000 7facb648c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7facb00cdeb0 0x7facb0070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.145+0000 7facb648c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb00adf90 con 0x7facb00705c0 2023-11-08T08:20:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.145+0000 7facb648c700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facb0069bf0 con 0x7facb00cdeb0 2023-11-08T08:20:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.145+0000 7facb648c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb0070a40 con 0x7facb015c1a0 2023-11-08T08:20:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.145+0000 7facb4c89700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7facb00cdeb0 0x7facb0070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.145+0000 7facb5c8b700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7facb00705c0 0x7facb015fd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57190/0 (socket says 172.21.15.40:57190) 2023-11-08T08:20:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.145+0000 7facb5c8b700 1 -- 172.21.15.40:0/1922616866 learned_addr learned my addr 172.21.15.40:0/1922616866 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.145+0000 7facb4c89700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7facb00cdeb0 0x7facb0070080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34094/0 (socket says 172.21.15.40:34094) 2023-11-08T08:20:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.145+0000 7facb548a700 1 --2- 172.21.15.40:0/1922616866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb015c1a0 0x7facb006c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.145+0000 7facb4c89700 1 -- 172.21.15.40:0/1922616866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb015c1a0 msgr2=0x7facb006c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.145+0000 7facb4c89700 1 --2- 172.21.15.40:0/1922616866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb015c1a0 0x7facb006c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.145+0000 7facb4c89700 1 -- 172.21.15.40:0/1922616866 >> v1:172.21.15.40:6789/0 conn(0x7facb00705c0 legacy=0x7facb015fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:30.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.145+0000 7facb4c89700 1 -- 172.21.15.40:0/1922616866 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb00ce370 con 0x7facb00cdeb0 2023-11-08T08:20:30.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.149+0000 7facb4c89700 1 --2- 172.21.15.40:0/1922616866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb00cdeb0 0x7facb0070080 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7faca0009850 tx=0x7faca0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fcfbc10bf4adb43 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:30.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.149+0000 7faca7fff700 1 -- 172.21.15.40:0/1922616866 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca0011830 con 0x7facb00cdeb0 2023-11-08T08:20:30.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.149+0000 7faca7fff700 1 -- 172.21.15.40:0/1922616866 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca0011990 con 0x7facb00cdeb0 2023-11-08T08:20:30.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.149+0000 7faca7fff700 1 -- 172.21.15.40:0/1922616866 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca0011d20 con 0x7facb00cdeb0 2023-11-08T08:20:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.149+0000 7facb648c700 1 -- 172.21.15.40:0/1922616866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb00cdeb0 msgr2=0x7facb0070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:30.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.149+0000 7facb648c700 1 --2- 172.21.15.40:0/1922616866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb00cdeb0 0x7facb0070080 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7faca0009850 tx=0x7faca0009b60 comp rx=0 tx=0).stop 2023-11-08T08:20:30.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.149+0000 7facb648c700 1 -- 172.21.15.40:0/1922616866 shutdown_connections 2023-11-08T08:20:30.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.149+0000 7facb648c700 1 --2- 172.21.15.40:0/1922616866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb00cdeb0 0x7facb0070080 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.149+0000 7facb648c700 1 --2- 172.21.15.40:0/1922616866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb015c1a0 0x7facb006c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.149+0000 7facb648c700 1 -- 172.21.15.40:0/1922616866 >> 172.21.15.40:0/1922616866 conn(0x7facb00b0f80 msgr2=0x7facb00ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:30.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.149+0000 7facb648c700 1 -- 172.21.15.40:0/1922616866 shutdown_connections 2023-11-08T08:20:30.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.149+0000 7facb648c700 1 -- 172.21.15.40:0/1922616866 wait complete. 2023-11-08T08:20:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.153+0000 7facb648c700 1 Processor -- start 2023-11-08T08:20:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.153+0000 7facb648c700 1 -- start start 2023-11-08T08:20:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.153+0000 7facb648c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb00705c0 0x7facb01581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.153+0000 7facb648c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facb00cdeb0 0x7facb015b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.153+0000 7facb648c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb015c1a0 0x7facb00b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.153+0000 7facb648c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb015b600 con 0x7facb00705c0 2023-11-08T08:20:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.153+0000 7facb648c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb015ead0 con 0x7facb00cdeb0 2023-11-08T08:20:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.153+0000 7facb648c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb015e1a0 con 0x7facb015c1a0 2023-11-08T08:20:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.153+0000 7facb548a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb00705c0 0x7facb01581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.153+0000 7facb548a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb00705c0 0x7facb01581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34106/0 (socket says 172.21.15.40:34106) 2023-11-08T08:20:30.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.153+0000 7facb548a700 1 -- 172.21.15.40:0/4252714236 learned_addr learned my addr 172.21.15.40:0/4252714236 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:30.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.153+0000 7facb5c8b700 1 --2- 172.21.15.40:0/4252714236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb015c1a0 0x7facb00b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:30.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.157+0000 7facb4c89700 1 --2- 172.21.15.40:0/4252714236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facb00cdeb0 0x7facb015b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:30.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.157+0000 7facb548a700 1 -- 172.21.15.40:0/4252714236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb015c1a0 msgr2=0x7facb00b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:30.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.157+0000 7facb548a700 1 --2- 172.21.15.40:0/4252714236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb015c1a0 0x7facb00b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.157+0000 7facb548a700 1 -- 172.21.15.40:0/4252714236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facb00cdeb0 msgr2=0x7facb015b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:30.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.157+0000 7facb548a700 1 --2- 172.21.15.40:0/4252714236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facb00cdeb0 0x7facb015b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.157+0000 7facb548a700 1 -- 172.21.15.40:0/4252714236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca00094b0 con 0x7facb00705c0 2023-11-08T08:20:30.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.157+0000 7facb4c89700 1 --2- 172.21.15.40:0/4252714236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facb00cdeb0 0x7facb015b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:20:30.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.157+0000 7facb548a700 1 --2- 172.21.15.40:0/4252714236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb00705c0 0x7facb01581f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7facac00e760 tx=0x7facac00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:30.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.157+0000 7faca5ffb700 1 -- 172.21.15.40:0/4252714236 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac013610 con 0x7facb00705c0 2023-11-08T08:20:30.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.157+0000 7faca5ffb700 1 -- 172.21.15.40:0/4252714236 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facac013770 con 0x7facb00705c0 2023-11-08T08:20:30.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.157+0000 7facb648c700 1 -- 172.21.15.40:0/4252714236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facb00cc780 con 0x7facb00705c0 2023-11-08T08:20:30.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.157+0000 7faca5ffb700 1 -- 172.21.15.40:0/4252714236 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac013ae0 con 0x7facb00705c0 2023-11-08T08:20:30.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.157+0000 7facb648c700 1 -- 172.21.15.40:0/4252714236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb00cc4a0 con 0x7facb00705c0 2023-11-08T08:20:30.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.157+0000 7facb648c700 1 -- 172.21.15.40:0/4252714236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facb006cdf0 con 0x7facb00705c0 2023-11-08T08:20:30.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.161+0000 7faca5ffb700 1 -- 172.21.15.40:0/4252714236 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7facac023070 con 0x7facb00705c0 2023-11-08T08:20:30.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.161+0000 7faca5ffb700 1 --2- 172.21.15.40:0/4252714236 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fac94064ee0 0x7fac94067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:30.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.161+0000 7faca5ffb700 1 -- 172.21.15.40:0/4252714236 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7facac081a30 con 0x7facb00705c0 2023-11-08T08:20:30.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.161+0000 7facb4c89700 1 --2- 172.21.15.40:0/4252714236 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fac94064ee0 0x7fac94067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:30.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.165+0000 7facb4c89700 1 --2- 172.21.15.40:0/4252714236 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fac94064ee0 0x7fac94067390 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7facb00ce2a0 tx=0x7faca0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:30.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.165+0000 7faca5ffb700 1 -- 172.21.15.40:0/4252714236 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7facac04fcf0 con 0x7facb00705c0 2023-11-08T08:20:30.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.345+0000 7facb648c700 1 -- 172.21.15.40:0/4252714236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7facb0073cb0 con 0x7facb00705c0 2023-11-08T08:20:30.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.345+0000 7faca5ffb700 1 -- 172.21.15.40:0/4252714236 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+14376 (secure 0 0 0) 0x7facac053310 con 0x7facb00705c0 2023-11-08T08:20:30.352 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:30.353 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":39,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:20:26.463197+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:20:30.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.349+0000 7facb648c700 1 -- 172.21.15.40:0/4252714236 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fac94064ee0 msgr2=0x7fac94067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:30.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.349+0000 7facb648c700 1 --2- 172.21.15.40:0/4252714236 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fac94064ee0 0x7fac94067390 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7facb00ce2a0 tx=0x7faca0008030 comp rx=0 tx=0).stop 2023-11-08T08:20:30.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.349+0000 7facb648c700 1 -- 172.21.15.40:0/4252714236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb00705c0 msgr2=0x7facb01581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:30.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.349+0000 7facb648c700 1 --2- 172.21.15.40:0/4252714236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb00705c0 0x7facb01581f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7facac00e760 tx=0x7facac00eb20 comp rx=0 tx=0).stop 2023-11-08T08:20:30.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.349+0000 7facb648c700 1 -- 172.21.15.40:0/4252714236 shutdown_connections 2023-11-08T08:20:30.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.349+0000 7facb648c700 1 --2- 172.21.15.40:0/4252714236 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fac94064ee0 0x7fac94067390 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.349+0000 7facb648c700 1 --2- 172.21.15.40:0/4252714236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7facb015c1a0 0x7facb00b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.349+0000 7facb648c700 1 --2- 172.21.15.40:0/4252714236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7facb00cdeb0 0x7facb015b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.349+0000 7facb648c700 1 --2- 172.21.15.40:0/4252714236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb00705c0 0x7facb01581f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.349+0000 7facb648c700 1 -- 172.21.15.40:0/4252714236 >> 172.21.15.40:0/4252714236 conn(0x7facb00b0f80 msgr2=0x7facb00acd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:30.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.349+0000 7facb648c700 1 -- 172.21.15.40:0/4252714236 shutdown_connections 2023-11-08T08:20:30.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.353+0000 7facb648c700 1 -- 172.21.15.40:0/4252714236 wait complete. 2023-11-08T08:20:30.379 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T08:20:30.380 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:20:30.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.557+0000 7fb556c19700 1 Processor -- start 2023-11-08T08:20:30.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.557+0000 7fb556c19700 1 -- start start 2023-11-08T08:20:30.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.557+0000 7fb556c19700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb550070a00 0x7fb550069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:30.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.557+0000 7fb556c19700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb5500b2130 0x7fb55006a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:30.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.557+0000 7fb556c19700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5500cd360 con 0x7fb55015c700 2023-11-08T08:20:30.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.557+0000 7fb556c19700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb550160190 con 0x7fb5500b2130 2023-11-08T08:20:30.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.557+0000 7fb556c19700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb550160470 con 0x7fb550070a00 2023-11-08T08:20:30.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.557+0000 7fb5551b6700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb5500b2130 0x7fb55006a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:30.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.557+0000 7fb54ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb550070a00 0x7fb550069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:30.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.557+0000 7fb5551b6700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb5500b2130 0x7fb55006a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:56634/0 (socket says 172.21.15.40:56634) 2023-11-08T08:20:30.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.557+0000 7fb5551b6700 1 -- 172.21.15.40:0/1726469246 learned_addr learned my addr 172.21.15.40:0/1726469246 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:30.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.561+0000 7fb5551b6700 1 -- 172.21.15.40:0/1726469246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb550070a00 msgr2=0x7fb550069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:30.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.561+0000 7fb5551b6700 1 --2- 172.21.15.40:0/1726469246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb550070a00 0x7fb550069bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.561+0000 7fb5551b6700 1 -- 172.21.15.40:0/1726469246 >> v1:172.21.15.169:6789/0 conn(0x7fb55015c700 legacy=0x7fb5500702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:30.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.561+0000 7fb5551b6700 1 -- 172.21.15.40:0/1726469246 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb550160750 con 0x7fb5500b2130 2023-11-08T08:20:30.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.561+0000 7fb54ffff700 1 --2- 172.21.15.40:0/1726469246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb550070a00 0x7fb550069bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:30.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.561+0000 7fb5551b6700 1 --2- 172.21.15.40:0/1726469246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5500b2130 0x7fb55006a130 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb540009e50 tx=0x7fb54000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8624b2898e23d87 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:30.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.561+0000 7fb54f7fe700 1 -- 172.21.15.40:0/1726469246 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54000e070 con 0x7fb5500b2130 2023-11-08T08:20:30.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.561+0000 7fb54f7fe700 1 -- 172.21.15.40:0/1726469246 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54000b910 con 0x7fb5500b2130 2023-11-08T08:20:30.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.561+0000 7fb54f7fe700 1 -- 172.21.15.40:0/1726469246 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54000bcc0 con 0x7fb5500b2130 2023-11-08T08:20:30.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.561+0000 7fb556c19700 1 -- 172.21.15.40:0/1726469246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5500b2130 msgr2=0x7fb55006a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:30.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.561+0000 7fb556c19700 1 --2- 172.21.15.40:0/1726469246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5500b2130 0x7fb55006a130 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb540009e50 tx=0x7fb54000b330 comp rx=0 tx=0).stop 2023-11-08T08:20:30.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.561+0000 7fb556c19700 1 -- 172.21.15.40:0/1726469246 shutdown_connections 2023-11-08T08:20:30.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.561+0000 7fb556c19700 1 --2- 172.21.15.40:0/1726469246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5500b2130 0x7fb55006a130 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.561+0000 7fb556c19700 1 --2- 172.21.15.40:0/1726469246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb550070a00 0x7fb550069bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.561+0000 7fb556c19700 1 -- 172.21.15.40:0/1726469246 >> 172.21.15.40:0/1726469246 conn(0x7fb5500d2d10 msgr2=0x7fb5500d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:30.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.561+0000 7fb556c19700 1 -- 172.21.15.40:0/1726469246 shutdown_connections 2023-11-08T08:20:30.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.561+0000 7fb556c19700 1 -- 172.21.15.40:0/1726469246 wait complete. 2023-11-08T08:20:30.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.565+0000 7fb556c19700 1 Processor -- start 2023-11-08T08:20:30.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.565+0000 7fb556c19700 1 -- start start 2023-11-08T08:20:30.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb556c19700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb550070a00 0x7fb55015ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:30.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb556c19700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5500b2130 0x7fb5500b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:30.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb556c19700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb55015c700 0x7fb5500b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:30.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb556c19700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5500b7a90 con 0x7fb550070a00 2023-11-08T08:20:30.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb556c19700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55015cad0 con 0x7fb55015c700 2023-11-08T08:20:30.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb556c19700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55015d090 con 0x7fb5500b2130 2023-11-08T08:20:30.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb5549b5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb550070a00 0x7fb55015ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:30.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb5549b5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb550070a00 0x7fb55015ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34110/0 (socket says 172.21.15.40:34110) 2023-11-08T08:20:30.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb5549b5700 1 -- 172.21.15.40:0/2802834615 learned_addr learned my addr 172.21.15.40:0/2802834615 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:30.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb54ffff700 1 --2- 172.21.15.40:0/2802834615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5500b2130 0x7fb5500b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:30.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb5551b6700 1 --2- 172.21.15.40:0/2802834615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb55015c700 0x7fb5500b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:30.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb5549b5700 1 -- 172.21.15.40:0/2802834615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5500b2130 msgr2=0x7fb5500b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:30.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb5549b5700 1 --2- 172.21.15.40:0/2802834615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5500b2130 0x7fb5500b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb5549b5700 1 -- 172.21.15.40:0/2802834615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb55015c700 msgr2=0x7fb5500b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:30.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb5549b5700 1 --2- 172.21.15.40:0/2802834615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb55015c700 0x7fb5500b6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb5549b5700 1 -- 172.21.15.40:0/2802834615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5380094b0 con 0x7fb550070a00 2023-11-08T08:20:30.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb5549b5700 1 --2- 172.21.15.40:0/2802834615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb550070a00 0x7fb55015ec70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb54800c530 tx=0x7fb54800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:30.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb54d7fa700 1 -- 172.21.15.40:0/2802834615 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54800cea0 con 0x7fb550070a00 2023-11-08T08:20:30.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb54d7fa700 1 -- 172.21.15.40:0/2802834615 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5480052d0 con 0x7fb550070a00 2023-11-08T08:20:30.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb54d7fa700 1 -- 172.21.15.40:0/2802834615 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5480036a0 con 0x7fb550070a00 2023-11-08T08:20:30.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb556c19700 1 -- 172.21.15.40:0/2802834615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb540009b30 con 0x7fb550070a00 2023-11-08T08:20:30.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.569+0000 7fb556c19700 1 -- 172.21.15.40:0/2802834615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55006b530 con 0x7fb550070a00 2023-11-08T08:20:30.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.573+0000 7fb556c19700 1 -- 172.21.15.40:0/2802834615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb550160a30 con 0x7fb550070a00 2023-11-08T08:20:30.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.573+0000 7fb54d7fa700 1 -- 172.21.15.40:0/2802834615 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fb548019030 con 0x7fb550070a00 2023-11-08T08:20:30.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.573+0000 7fb54d7fa700 1 --2- 172.21.15.40:0/2802834615 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb530064ee0 0x7fb530067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:30.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.577+0000 7fb54d7fa700 1 -- 172.21.15.40:0/2802834615 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fb548080e40 con 0x7fb550070a00 2023-11-08T08:20:30.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.577+0000 7fb54ffff700 1 --2- 172.21.15.40:0/2802834615 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb530064ee0 0x7fb530067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:30.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.577+0000 7fb54ffff700 1 --2- 172.21.15.40:0/2802834615 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb530064ee0 0x7fb530067390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb5501f5150 tx=0x7fb538009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:30.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.581+0000 7fb54d7fa700 1 -- 172.21.15.40:0/2802834615 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fb54804f1f0 con 0x7fb550070a00 2023-11-08T08:20:30.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.753+0000 7fb556c19700 1 -- 172.21.15.40:0/2802834615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5500b1be0 con 0x7fb550070a00 2023-11-08T08:20:30.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.757+0000 7fb54d7fa700 1 -- 172.21.15.40:0/2802834615 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb548052810 con 0x7fb550070a00 2023-11-08T08:20:30.761 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:20:30.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.757+0000 7fb556c19700 1 -- 172.21.15.40:0/2802834615 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb530064ee0 msgr2=0x7fb530067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:30.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.757+0000 7fb556c19700 1 --2- 172.21.15.40:0/2802834615 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb530064ee0 0x7fb530067390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb5501f5150 tx=0x7fb538009240 comp rx=0 tx=0).stop 2023-11-08T08:20:30.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.757+0000 7fb556c19700 1 -- 172.21.15.40:0/2802834615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb550070a00 msgr2=0x7fb55015ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:30.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.757+0000 7fb556c19700 1 --2- 172.21.15.40:0/2802834615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb550070a00 0x7fb55015ec70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb54800c530 tx=0x7fb54800c840 comp rx=0 tx=0).stop 2023-11-08T08:20:30.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.757+0000 7fb556c19700 1 -- 172.21.15.40:0/2802834615 shutdown_connections 2023-11-08T08:20:30.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.757+0000 7fb556c19700 1 --2- 172.21.15.40:0/2802834615 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb530064ee0 0x7fb530067390 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.757+0000 7fb556c19700 1 --2- 172.21.15.40:0/2802834615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb55015c700 0x7fb5500b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.757+0000 7fb556c19700 1 --2- 172.21.15.40:0/2802834615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5500b2130 0x7fb5500b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.757+0000 7fb556c19700 1 --2- 172.21.15.40:0/2802834615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb550070a00 0x7fb55015ec70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.757+0000 7fb556c19700 1 -- 172.21.15.40:0/2802834615 >> 172.21.15.40:0/2802834615 conn(0x7fb5500d2d10 msgr2=0x7fb5500cd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:30.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.761+0000 7fb556c19700 1 -- 172.21.15.40:0/2802834615 shutdown_connections 2023-11-08T08:20:30.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.761+0000 7fb556c19700 1 -- 172.21.15.40:0/2802834615 wait complete. 2023-11-08T08:20:30.784 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:20:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.965+0000 7f385166d700 1 Processor -- start 2023-11-08T08:20:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.965+0000 7f385166d700 1 -- start start 2023-11-08T08:20:30.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.969+0000 7f385166d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f384c159f10 0x7f384c070a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:30.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.969+0000 7f385166d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f384c15a9f0 0x7f384c06b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:30.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.969+0000 7f385166d700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f384c070600 con 0x7f384c0c8c90 2023-11-08T08:20:30.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.969+0000 7f385166d700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384c0c9130 con 0x7f384c159f10 2023-11-08T08:20:30.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.969+0000 7f385166d700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384c06b840 con 0x7f384c15a9f0 2023-11-08T08:20:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.969+0000 7f384bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f384c159f10 0x7f384c070a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.969+0000 7f384bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f384c159f10 0x7f384c070a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34118/0 (socket says 172.21.15.40:34118) 2023-11-08T08:20:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.969+0000 7f384bfff700 1 -- 172.21.15.40:0/1189611878 learned_addr learned my addr 172.21.15.40:0/1189611878 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:30.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.969+0000 7f3850e6c700 1 --2- 172.21.15.40:0/1189611878 >> v2:172.21.15.169:3300/0 conn(0x7f384c15a9f0 0x7f384c06b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:30.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.969+0000 7f384bfff700 1 -- 172.21.15.40:0/1189611878 >> v2:172.21.15.169:3300/0 conn(0x7f384c15a9f0 msgr2=0x7f384c06b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:30.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.969+0000 7f384bfff700 1 --2- 172.21.15.40:0/1189611878 >> v2:172.21.15.169:3300/0 conn(0x7f384c15a9f0 0x7f384c06b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.969+0000 7f384bfff700 1 -- 172.21.15.40:0/1189611878 >> v1:172.21.15.40:6789/0 conn(0x7f384c0c8c90 legacy=0x7f384c15a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:30.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.969+0000 7f384bfff700 1 -- 172.21.15.40:0/1189611878 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384c15adf0 con 0x7f384c159f10 2023-11-08T08:20:30.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.969+0000 7f384bfff700 1 --2- 172.21.15.40:0/1189611878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c159f10 0x7f384c070a90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f384c06bb40 tx=0x7f383c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce50bfb0f5a9c46 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.969+0000 7f384b7fe700 1 -- 172.21.15.40:0/1189611878 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c00b8a0 con 0x7f384c159f10 2023-11-08T08:20:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.969+0000 7f384b7fe700 1 -- 172.21.15.40:0/1189611878 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383c00ba00 con 0x7f384c159f10 2023-11-08T08:20:30.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.969+0000 7f384b7fe700 1 -- 172.21.15.40:0/1189611878 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c00bd90 con 0x7f384c159f10 2023-11-08T08:20:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.969+0000 7f385166d700 1 -- 172.21.15.40:0/1189611878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c159f10 msgr2=0x7f384c070a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.969+0000 7f385166d700 1 --2- 172.21.15.40:0/1189611878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c159f10 0x7f384c070a90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f384c06bb40 tx=0x7f383c009bf0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.973+0000 7f385166d700 1 -- 172.21.15.40:0/1189611878 shutdown_connections 2023-11-08T08:20:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.973+0000 7f385166d700 1 --2- 172.21.15.40:0/1189611878 >> v2:172.21.15.169:3300/0 conn(0x7f384c15a9f0 0x7f384c06b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.973+0000 7f385166d700 1 --2- 172.21.15.40:0/1189611878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c159f10 0x7f384c070a90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.973+0000 7f385166d700 1 -- 172.21.15.40:0/1189611878 >> 172.21.15.40:0/1189611878 conn(0x7f384c0cf280 msgr2=0x7f384c0cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.973+0000 7f385166d700 1 -- 172.21.15.40:0/1189611878 shutdown_connections 2023-11-08T08:20:30.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.973+0000 7f385166d700 1 -- 172.21.15.40:0/1189611878 wait complete. 2023-11-08T08:20:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.973+0000 7f385166d700 1 Processor -- start 2023-11-08T08:20:30.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.973+0000 7f385166d700 1 -- start start 2023-11-08T08:20:30.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f385166d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0c8c90 0x7f384c158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:30.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f385166d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f384c159f10 0x7f384c0c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:30.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f385166d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f384c15a9f0 0x7f384c0af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:30.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f385166d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c0cc820 con 0x7f384c0c8c90 2023-11-08T08:20:30.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f385166d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c0c6930 con 0x7f384c159f10 2023-11-08T08:20:30.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f385166d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384c0c6bd0 con 0x7f384c15a9f0 2023-11-08T08:20:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f384bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0c8c90 0x7f384c158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:30.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f384bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0c8c90 0x7f384c158990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34126/0 (socket says 172.21.15.40:34126) 2023-11-08T08:20:30.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f384bfff700 1 -- 172.21.15.40:0/1208562278 learned_addr learned my addr 172.21.15.40:0/1208562278 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:30.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f3850e6c700 1 --2- 172.21.15.40:0/1208562278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f384c15a9f0 0x7f384c0af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:30.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f38437fe700 1 --2- 172.21.15.40:0/1208562278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f384c159f10 0x7f384c0c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:30.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f384bfff700 1 -- 172.21.15.40:0/1208562278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f384c15a9f0 msgr2=0x7f384c0af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:30.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f384bfff700 1 --2- 172.21.15.40:0/1208562278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f384c15a9f0 0x7f384c0af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f384bfff700 1 -- 172.21.15.40:0/1208562278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f384c159f10 msgr2=0x7f384c0c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f384bfff700 1 --2- 172.21.15.40:0/1208562278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f384c159f10 0x7f384c0c63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f384bfff700 1 -- 172.21.15.40:0/1208562278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383c009570 con 0x7f384c0c8c90 2023-11-08T08:20:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f384bfff700 1 --2- 172.21.15.40:0/1208562278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0c8c90 0x7f384c158990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f384c0717e0 tx=0x7f383c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f38497fa700 1 -- 172.21.15.40:0/1208562278 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c0038c0 con 0x7f384c0c8c90 2023-11-08T08:20:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f38497fa700 1 -- 172.21.15.40:0/1208562278 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383c01f070 con 0x7f384c0c8c90 2023-11-08T08:20:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f385166d700 1 -- 172.21.15.40:0/1208562278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384c06ffb0 con 0x7f384c0c8c90 2023-11-08T08:20:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.977+0000 7f38497fa700 1 -- 172.21.15.40:0/1208562278 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c003c60 con 0x7f384c0c8c90 2023-11-08T08:20:30.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.981+0000 7f385166d700 1 -- 172.21.15.40:0/1208562278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384c0afcc0 con 0x7f384c0c8c90 2023-11-08T08:20:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.981+0000 7f385166d700 1 -- 172.21.15.40:0/1208562278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384c070290 con 0x7f384c0c8c90 2023-11-08T08:20:30.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.981+0000 7f38497fa700 1 -- 172.21.15.40:0/1208562278 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f383c00f050 con 0x7f384c0c8c90 2023-11-08T08:20:30.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.985+0000 7f38497fa700 1 --2- 172.21.15.40:0/1208562278 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f382c064ee0 0x7f382c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:30.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.985+0000 7f38497fa700 1 -- 172.21.15.40:0/1208562278 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f383c021070 con 0x7f384c0c8c90 2023-11-08T08:20:30.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.985+0000 7f38437fe700 1 --2- 172.21.15.40:0/1208562278 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f382c064ee0 0x7f382c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:30.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.985+0000 7f38437fe700 1 --2- 172.21.15.40:0/1208562278 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f382c064ee0 0x7f382c067390 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f384c0c8250 tx=0x7f3834009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:30.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:30.989+0000 7f38497fa700 1 -- 172.21.15.40:0/1208562278 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f383c051b60 con 0x7f384c0c8c90 2023-11-08T08:20:31.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.157+0000 7f385166d700 1 -- 172.21.15.40:0/1208562278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f384c06cf30 con 0x7f384c0c8c90 2023-11-08T08:20:31.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.161+0000 7f38497fa700 1 -- 172.21.15.40:0/1208562278 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v39) v1 ==== 92+0+10 (secure 0 0 0) 0x7f383c055180 con 0x7f384c0c8c90 2023-11-08T08:20:31.164 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:20:31.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.161+0000 7f385166d700 1 -- 172.21.15.40:0/1208562278 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f382c064ee0 msgr2=0x7f382c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:31.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.161+0000 7f385166d700 1 --2- 172.21.15.40:0/1208562278 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f382c064ee0 0x7f382c067390 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f384c0c8250 tx=0x7f3834009040 comp rx=0 tx=0).stop 2023-11-08T08:20:31.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.161+0000 7f385166d700 1 -- 172.21.15.40:0/1208562278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0c8c90 msgr2=0x7f384c158990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:31.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.161+0000 7f385166d700 1 --2- 172.21.15.40:0/1208562278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0c8c90 0x7f384c158990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f384c0717e0 tx=0x7f383c004770 comp rx=0 tx=0).stop 2023-11-08T08:20:31.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.161+0000 7f385166d700 1 -- 172.21.15.40:0/1208562278 shutdown_connections 2023-11-08T08:20:31.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.161+0000 7f385166d700 1 --2- 172.21.15.40:0/1208562278 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f382c064ee0 0x7f382c067390 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:31.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.161+0000 7f385166d700 1 --2- 172.21.15.40:0/1208562278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f384c15a9f0 0x7f384c0af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:31.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.161+0000 7f385166d700 1 --2- 172.21.15.40:0/1208562278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f384c159f10 0x7f384c0c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:31.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.161+0000 7f385166d700 1 --2- 172.21.15.40:0/1208562278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0c8c90 0x7f384c158990 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:31.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.161+0000 7f385166d700 1 -- 172.21.15.40:0/1208562278 >> 172.21.15.40:0/1208562278 conn(0x7f384c0cf280 msgr2=0x7f384c0affa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:31.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.161+0000 7f385166d700 1 -- 172.21.15.40:0/1208562278 shutdown_connections 2023-11-08T08:20:31.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.161+0000 7f385166d700 1 -- 172.21.15.40:0/1208562278 wait complete. 2023-11-08T08:20:31.190 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T08:20:31.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.365+0000 7f220286f700 1 Processor -- start 2023-11-08T08:20:31.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.365+0000 7f220286f700 1 -- start start 2023-11-08T08:20:31.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.365+0000 7f220286f700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f21fc069e00 0x7f21fc06a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:31.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.365+0000 7f220286f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21fc06c600 con 0x7f21fc15c640 2023-11-08T08:20:31.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.365+0000 7f220286f700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21fc0722f0 con 0x7f21fc071aa0 2023-11-08T08:20:31.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.365+0000 7f220286f700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21fc06cef0 con 0x7f21fc069e00 2023-11-08T08:20:31.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.365+0000 7f220186d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f21fc15c640 0x7f21fc0d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57200/0 (socket says 172.21.15.40:57200) 2023-11-08T08:20:31.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.365+0000 7f220186d700 1 -- 172.21.15.40:0/1205528643 learned_addr learned my addr 172.21.15.40:0/1205528643 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:31.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.365+0000 7f220206e700 1 --2- 172.21.15.40:0/1205528643 >> v2:172.21.15.169:3300/0 conn(0x7f21fc069e00 0x7f21fc06a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:31.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.369+0000 7f220086b700 1 -- 172.21.15.40:0/1205528643 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213488490 0 0) 0x7f21fc0722f0 con 0x7f21fc071aa0 2023-11-08T08:20:31.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.369+0000 7f220086b700 1 -- 172.21.15.40:0/1205528643 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21e80035d0 con 0x7f21fc071aa0 2023-11-08T08:20:31.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.369+0000 7f220206e700 1 -- 172.21.15.40:0/1205528643 >> v1:172.21.15.169:6789/0 conn(0x7f21fc071aa0 legacy=0x7f21fc06d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:31.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.369+0000 7f220206e700 1 -- 172.21.15.40:0/1205528643 >> v1:172.21.15.40:6789/0 conn(0x7f21fc15c640 legacy=0x7f21fc0d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:31.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.369+0000 7f220086b700 1 -- 172.21.15.40:0/1205528643 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3454763126 0 0) 0x7f21fc06c600 con 0x7f21fc15c640 2023-11-08T08:20:31.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.369+0000 7f220206e700 1 -- 172.21.15.40:0/1205528643 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21fc06da20 con 0x7f21fc069e00 2023-11-08T08:20:31.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.369+0000 7f220206e700 1 --2- 172.21.15.40:0/1205528643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc069e00 0x7f21fc06a280 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009d90 tx=0x7f21f400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b71cd6d430479ff2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:31.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.369+0000 7f220086b700 1 -- 172.21.15.40:0/1205528643 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f400e070 con 0x7f21fc069e00 2023-11-08T08:20:31.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.369+0000 7f220086b700 1 -- 172.21.15.40:0/1205528643 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f400bb20 con 0x7f21fc069e00 2023-11-08T08:20:31.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.369+0000 7f220086b700 1 -- 172.21.15.40:0/1205528643 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f400bed0 con 0x7f21fc069e00 2023-11-08T08:20:31.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.369+0000 7f220286f700 1 -- 172.21.15.40:0/1205528643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc069e00 msgr2=0x7f21fc06a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:31.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.369+0000 7f220286f700 1 --2- 172.21.15.40:0/1205528643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc069e00 0x7f21fc06a280 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009d90 tx=0x7f21f400b330 comp rx=0 tx=0).stop 2023-11-08T08:20:31.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.369+0000 7f220286f700 1 -- 172.21.15.40:0/1205528643 shutdown_connections 2023-11-08T08:20:31.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.369+0000 7f220286f700 1 --2- 172.21.15.40:0/1205528643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc069e00 0x7f21fc06a280 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:31.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.369+0000 7f220286f700 1 -- 172.21.15.40:0/1205528643 >> 172.21.15.40:0/1205528643 conn(0x7f21fc0b6520 msgr2=0x7f21fc071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:31.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.369+0000 7f220286f700 1 -- 172.21.15.40:0/1205528643 shutdown_connections 2023-11-08T08:20:31.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.369+0000 7f220286f700 1 -- 172.21.15.40:0/1205528643 wait complete. 2023-11-08T08:20:31.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.373+0000 7f220286f700 1 Processor -- start 2023-11-08T08:20:31.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.373+0000 7f220286f700 1 -- start start 2023-11-08T08:20:31.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f220286f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc069e00 0x7f21fc0cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:31.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f220286f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21fc071aa0 0x7f21fc1f1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:31.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f220286f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc15c640 0x7f21fc0c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:31.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f220286f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc0d38f0 con 0x7f21fc15c640 2023-11-08T08:20:31.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f220286f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc0caba0 con 0x7f21fc069e00 2023-11-08T08:20:31.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f220286f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc0cae80 con 0x7f21fc071aa0 2023-11-08T08:20:31.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f220206e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc15c640 0x7f21fc0c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:31.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f220206e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc15c640 0x7f21fc0c87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34128/0 (socket says 172.21.15.40:34128) 2023-11-08T08:20:31.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f220206e700 1 -- 172.21.15.40:0/2953383492 learned_addr learned my addr 172.21.15.40:0/2953383492 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:31.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f220106c700 1 --2- 172.21.15.40:0/2953383492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21fc071aa0 0x7f21fc1f1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:31.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f220186d700 1 --2- 172.21.15.40:0/2953383492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc069e00 0x7f21fc0cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:31.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f220206e700 1 -- 172.21.15.40:0/2953383492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21fc071aa0 msgr2=0x7f21fc1f1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:31.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f220206e700 1 --2- 172.21.15.40:0/2953383492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21fc071aa0 0x7f21fc1f1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:31.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f220206e700 1 -- 172.21.15.40:0/2953383492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc069e00 msgr2=0x7f21fc0cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:31.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f220206e700 1 --2- 172.21.15.40:0/2953383492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc069e00 0x7f21fc0cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:31.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f220206e700 1 -- 172.21.15.40:0/2953383492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f4009a70 con 0x7f21fc15c640 2023-11-08T08:20:31.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f220206e700 1 --2- 172.21.15.40:0/2953383492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc15c640 0x7f21fc0c87a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f21fc06f090 tx=0x7f21f4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:31.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f21f27fc700 1 -- 172.21.15.40:0/2953383492 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f401b810 con 0x7f21fc15c640 2023-11-08T08:20:31.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f21f27fc700 1 -- 172.21.15.40:0/2953383492 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f4020070 con 0x7f21fc15c640 2023-11-08T08:20:31.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f21f27fc700 1 -- 172.21.15.40:0/2953383492 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f401bbb0 con 0x7f21fc15c640 2023-11-08T08:20:31.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f220286f700 1 -- 172.21.15.40:0/2953383492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21fc072010 con 0x7f21fc15c640 2023-11-08T08:20:31.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.377+0000 7f220286f700 1 -- 172.21.15.40:0/2953383492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21fc15ca10 con 0x7f21fc15c640 2023-11-08T08:20:31.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.381+0000 7f220286f700 1 -- 172.21.15.40:0/2953383492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21fc06c810 con 0x7f21fc15c640 2023-11-08T08:20:31.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.381+0000 7f21f27fc700 1 -- 172.21.15.40:0/2953383492 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f21f402e070 con 0x7f21fc15c640 2023-11-08T08:20:31.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.381+0000 7f21f27fc700 1 --2- 172.21.15.40:0/2953383492 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f21e0064ee0 0x7f21e0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:31.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.381+0000 7f21f27fc700 1 -- 172.21.15.40:0/2953383492 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f21f4022070 con 0x7f21fc15c640 2023-11-08T08:20:31.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.385+0000 7f220186d700 1 --2- 172.21.15.40:0/2953383492 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f21e0064ee0 0x7f21e0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:31.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.385+0000 7f220186d700 1 --2- 172.21.15.40:0/2953383492 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f21e0064ee0 0x7f21e0067390 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f21f8000f60 tx=0x7f21f8007e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:31.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.389+0000 7f21f27fc700 1 -- 172.21.15.40:0/2953383492 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f21f4052010 con 0x7f21fc15c640 2023-11-08T08:20:31.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.569+0000 7f220286f700 1 -- 172.21.15.40:0/2953383492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f21fc073cb0 con 0x7f21fc15c640 2023-11-08T08:20:31.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.569+0000 7f21f27fc700 1 -- 172.21.15.40:0/2953383492 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v39) v1 ==== 84+0+126 (secure 0 0 0) 0x7f21f402c090 con 0x7f21fc15c640 2023-11-08T08:20:31.573 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:31.574 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2023-11-08T08:20:31.574 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:31.574 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2023-11-08T08:20:31.574 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2023-11-08T08:20:31.574 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:20:31.575 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:20:31.575 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 2, 2023-11-08T08:20:31.575 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": "rbd" 2023-11-08T08:20:31.575 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:20:31.575 INFO:teuthology.orchestra.run.smithi040.stdout:] 2023-11-08T08:20:31.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.573+0000 7f220286f700 1 -- 172.21.15.40:0/2953383492 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f21e0064ee0 msgr2=0x7f21e0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:31.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.573+0000 7f220286f700 1 --2- 172.21.15.40:0/2953383492 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f21e0064ee0 0x7f21e0067390 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f21f8000f60 tx=0x7f21f8007e00 comp rx=0 tx=0).stop 2023-11-08T08:20:31.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.573+0000 7f220286f700 1 -- 172.21.15.40:0/2953383492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc15c640 msgr2=0x7f21fc0c87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:31.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.573+0000 7f220286f700 1 --2- 172.21.15.40:0/2953383492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc15c640 0x7f21fc0c87a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f21fc06f090 tx=0x7f21f4004af0 comp rx=0 tx=0).stop 2023-11-08T08:20:31.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.573+0000 7f220286f700 1 -- 172.21.15.40:0/2953383492 shutdown_connections 2023-11-08T08:20:31.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.573+0000 7f220286f700 1 --2- 172.21.15.40:0/2953383492 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f21e0064ee0 0x7f21e0067390 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:31.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.573+0000 7f220286f700 1 --2- 172.21.15.40:0/2953383492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc15c640 0x7f21fc0c87a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:31.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.573+0000 7f220286f700 1 --2- 172.21.15.40:0/2953383492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21fc071aa0 0x7f21fc1f1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:31.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.573+0000 7f220286f700 1 --2- 172.21.15.40:0/2953383492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc069e00 0x7f21fc0cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:31.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.573+0000 7f220286f700 1 -- 172.21.15.40:0/2953383492 >> 172.21.15.40:0/2953383492 conn(0x7f21fc0b6520 msgr2=0x7f21fc0d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:31.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.573+0000 7f220286f700 1 -- 172.21.15.40:0/2953383492 shutdown_connections 2023-11-08T08:20:31.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.573+0000 7f220286f700 1 -- 172.21.15.40:0/2953383492 wait complete. 2023-11-08T08:20:31.602 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T08:20:31.602 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T08:20:31.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.785+0000 7f4d07d69700 1 Processor -- start 2023-11-08T08:20:31.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.785+0000 7f4d07d69700 1 -- start start 2023-11-08T08:20:31.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.789+0000 7f4d07d69700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d000ccb50 0x7f4d000b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:31.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.789+0000 7f4d07d69700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4d000ccf80 0x7f4d000b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:31.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.789+0000 7f4d07d69700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d000ce5e0 con 0x7f4d000b60e0 2023-11-08T08:20:31.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.789+0000 7f4d07d69700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d000d07d0 con 0x7f4d000ccf80 2023-11-08T08:20:31.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.789+0000 7f4d07d69700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d000b64b0 con 0x7f4d000ccb50 2023-11-08T08:20:31.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.789+0000 7f4d05b05700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4d000b60e0 0x7f4d0015e820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57204/0 (socket says 172.21.15.40:57204) 2023-11-08T08:20:31.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.789+0000 7f4d05b05700 1 -- 172.21.15.40:0/3458938003 learned_addr learned my addr 172.21.15.40:0/3458938003 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:31.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.789+0000 7f4d05304700 1 --2- 172.21.15.40:0/3458938003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d000ccb50 0x7f4d000b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:31.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.789+0000 7f4d06306700 1 --2- 172.21.15.40:0/3458938003 >> v2:172.21.15.169:3300/0 conn(0x7f4d000ccf80 0x7f4d000b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:31.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.793+0000 7f4d05304700 1 -- 172.21.15.40:0/3458938003 >> v2:172.21.15.169:3300/0 conn(0x7f4d000ccf80 msgr2=0x7f4d000b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:31.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.793+0000 7f4d05304700 1 --2- 172.21.15.40:0/3458938003 >> v2:172.21.15.169:3300/0 conn(0x7f4d000ccf80 0x7f4d000b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:31.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.793+0000 7f4d05304700 1 -- 172.21.15.40:0/3458938003 >> v1:172.21.15.40:6789/0 conn(0x7f4d000b60e0 legacy=0x7f4d0015e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:31.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.793+0000 7f4d05304700 1 -- 172.21.15.40:0/3458938003 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d000d04f0 con 0x7f4d000ccb50 2023-11-08T08:20:31.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.793+0000 7f4d05304700 1 --2- 172.21.15.40:0/3458938003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d000ccb50 0x7f4d000b2490 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009850 tx=0x7f4cf0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b5e84e507555ffc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:31.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.797+0000 7f4d04b03700 1 -- 172.21.15.40:0/3458938003 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0011830 con 0x7f4d000ccb50 2023-11-08T08:20:31.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.797+0000 7f4d04b03700 1 -- 172.21.15.40:0/3458938003 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf0011990 con 0x7f4d000ccb50 2023-11-08T08:20:31.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.797+0000 7f4d04b03700 1 -- 172.21.15.40:0/3458938003 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0011d40 con 0x7f4d000ccb50 2023-11-08T08:20:31.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.797+0000 7f4d07d69700 1 -- 172.21.15.40:0/3458938003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d000ccb50 msgr2=0x7f4d000b2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:31.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.797+0000 7f4d07d69700 1 --2- 172.21.15.40:0/3458938003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d000ccb50 0x7f4d000b2490 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009850 tx=0x7f4cf0009b60 comp rx=0 tx=0).stop 2023-11-08T08:20:31.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.797+0000 7f4d07d69700 1 -- 172.21.15.40:0/3458938003 shutdown_connections 2023-11-08T08:20:31.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.797+0000 7f4d07d69700 1 --2- 172.21.15.40:0/3458938003 >> v2:172.21.15.169:3300/0 conn(0x7f4d000ccf80 0x7f4d000b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:31.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.797+0000 7f4d07d69700 1 --2- 172.21.15.40:0/3458938003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d000ccb50 0x7f4d000b2490 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:31.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.797+0000 7f4d07d69700 1 -- 172.21.15.40:0/3458938003 >> 172.21.15.40:0/3458938003 conn(0x7f4d00069bf0 msgr2=0x7f4d00069ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:31.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.797+0000 7f4d07d69700 1 -- 172.21.15.40:0/3458938003 shutdown_connections 2023-11-08T08:20:31.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.797+0000 7f4d07d69700 1 -- 172.21.15.40:0/3458938003 wait complete. 2023-11-08T08:20:31.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.801+0000 7f4d07d69700 1 Processor -- start 2023-11-08T08:20:31.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.801+0000 7f4d07d69700 1 -- start start 2023-11-08T08:20:31.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.801+0000 7f4d07d69700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d000b60e0 0x7f4d001f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:31.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.801+0000 7f4d07d69700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d000ccb50 0x7f4d000ac570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:31.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.801+0000 7f4d07d69700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d000ccf80 0x7f4d000acab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:31.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.801+0000 7f4d07d69700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d001f5460 con 0x7f4d000ccf80 2023-11-08T08:20:31.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.801+0000 7f4d07d69700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d000c62e0 con 0x7f4d000ccb50 2023-11-08T08:20:31.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.801+0000 7f4d07d69700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d000c65c0 con 0x7f4d000b60e0 2023-11-08T08:20:31.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.801+0000 7f4d06306700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d000ccf80 0x7f4d000acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:31.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.801+0000 7f4d05304700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d000ccb50 0x7f4d000ac570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:31.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.801+0000 7f4d06306700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d000ccf80 0x7f4d000acab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34142/0 (socket says 172.21.15.40:34142) 2023-11-08T08:20:31.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.801+0000 7f4d06306700 1 -- 172.21.15.40:0/1955505305 learned_addr learned my addr 172.21.15.40:0/1955505305 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:31.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.801+0000 7f4d05b05700 1 --2- 172.21.15.40:0/1955505305 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d000b60e0 0x7f4d001f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:31.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.805+0000 7f4d06306700 1 -- 172.21.15.40:0/1955505305 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d000b60e0 msgr2=0x7f4d001f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:31.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.805+0000 7f4d06306700 1 --2- 172.21.15.40:0/1955505305 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d000b60e0 0x7f4d001f4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:31.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.805+0000 7f4d06306700 1 -- 172.21.15.40:0/1955505305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d000ccb50 msgr2=0x7f4d000ac570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:31.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.805+0000 7f4d06306700 1 --2- 172.21.15.40:0/1955505305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d000ccb50 0x7f4d000ac570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:31.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.805+0000 7f4d06306700 1 -- 172.21.15.40:0/1955505305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf00094b0 con 0x7f4d000ccf80 2023-11-08T08:20:31.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.805+0000 7f4d05b05700 1 --2- 172.21.15.40:0/1955505305 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d000b60e0 0x7f4d001f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:31.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.805+0000 7f4d05304700 1 --2- 172.21.15.40:0/1955505305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d000ccb50 0x7f4d000ac570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:31.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.805+0000 7f4d06306700 1 --2- 172.21.15.40:0/1955505305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d000ccf80 0x7f4d000acab0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800efc0 tx=0x7f4cf8009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:31.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.805+0000 7f4cf67fc700 1 -- 172.21.15.40:0/1955505305 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8009b20 con 0x7f4d000ccf80 2023-11-08T08:20:31.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.805+0000 7f4cf67fc700 1 -- 172.21.15.40:0/1955505305 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf800f040 con 0x7f4d000ccf80 2023-11-08T08:20:31.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.805+0000 7f4cf67fc700 1 -- 172.21.15.40:0/1955505305 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf800cc90 con 0x7f4d000ccf80 2023-11-08T08:20:31.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.805+0000 7f4d07d69700 1 -- 172.21.15.40:0/1955505305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d000cb4e0 con 0x7f4d000ccf80 2023-11-08T08:20:31.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.805+0000 7f4d07d69700 1 -- 172.21.15.40:0/1955505305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d000cb200 con 0x7f4d000ccf80 2023-11-08T08:20:31.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.805+0000 7f4d07d69700 1 -- 172.21.15.40:0/1955505305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d000d01a0 con 0x7f4d000ccf80 2023-11-08T08:20:31.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.809+0000 7f4cf67fc700 1 -- 172.21.15.40:0/1955505305 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4cf8014480 con 0x7f4d000ccf80 2023-11-08T08:20:31.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.809+0000 7f4cf67fc700 1 --2- 172.21.15.40:0/1955505305 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4ce4064f10 0x7f4ce40673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:31.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.809+0000 7f4cf67fc700 1 -- 172.21.15.40:0/1955505305 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f4cf8082880 con 0x7f4d000ccf80 2023-11-08T08:20:31.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.809+0000 7f4d05b05700 1 --2- 172.21.15.40:0/1955505305 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4ce4064f10 0x7f4ce40673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:31.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.813+0000 7f4d05b05700 1 --2- 172.21.15.40:0/1955505305 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4ce4064f10 0x7f4ce40673c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc000f60 tx=0x7f4cfc007f30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:31.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.817+0000 7f4cf67fc700 1 -- 172.21.15.40:0/1955505305 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4cf8050ae0 con 0x7f4d000ccf80 2023-11-08T08:20:31.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:31.993+0000 7f4d07d69700 1 -- 172.21.15.40:0/1955505305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4d000726c0 con 0x7f4d000ccf80 2023-11-08T08:20:32.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:32.805+0000 7f4cf67fc700 1 -- 172.21.15.40:0/1955505305 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v40) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4cf80180b0 con 0x7f4d000ccf80 2023-11-08T08:20:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:32.809+0000 7f4d07d69700 1 -- 172.21.15.40:0/1955505305 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4ce4064f10 msgr2=0x7f4ce40673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:32.809+0000 7f4d07d69700 1 --2- 172.21.15.40:0/1955505305 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4ce4064f10 0x7f4ce40673c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc000f60 tx=0x7f4cfc007f30 comp rx=0 tx=0).stop 2023-11-08T08:20:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:32.809+0000 7f4d07d69700 1 -- 172.21.15.40:0/1955505305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d000ccf80 msgr2=0x7f4d000acab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:32.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:32.809+0000 7f4d07d69700 1 --2- 172.21.15.40:0/1955505305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d000ccf80 0x7f4d000acab0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800efc0 tx=0x7f4cf8009f60 comp rx=0 tx=0).stop 2023-11-08T08:20:32.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:32.809+0000 7f4d07d69700 1 -- 172.21.15.40:0/1955505305 shutdown_connections 2023-11-08T08:20:32.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:32.809+0000 7f4d07d69700 1 --2- 172.21.15.40:0/1955505305 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4ce4064f10 0x7f4ce40673c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:32.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:32.813+0000 7f4d07d69700 1 --2- 172.21.15.40:0/1955505305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d000ccf80 0x7f4d000acab0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:32.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:32.813+0000 7f4d07d69700 1 --2- 172.21.15.40:0/1955505305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d000ccb50 0x7f4d000ac570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:32.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:32.813+0000 7f4d07d69700 1 --2- 172.21.15.40:0/1955505305 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d000b60e0 0x7f4d001f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:32.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:32.813+0000 7f4d07d69700 1 -- 172.21.15.40:0/1955505305 >> 172.21.15.40:0/1955505305 conn(0x7f4d00069bf0 msgr2=0x7f4d0015b890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:32.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:32.813+0000 7f4d07d69700 1 -- 172.21.15.40:0/1955505305 shutdown_connections 2023-11-08T08:20:32.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:32.813+0000 7f4d07d69700 1 -- 172.21.15.40:0/1955505305 wait complete. 2023-11-08T08:20:32.819 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2023-11-08T08:20:32.840 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T08:20:33.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.025+0000 7fd69ee98700 1 Processor -- start 2023-11-08T08:20:33.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.025+0000 7fd69ee98700 1 -- start start 2023-11-08T08:20:33.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.029+0000 7fd69ee98700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd6980b4f70 0x7fd698069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:33.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.029+0000 7fd69ee98700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6980b31d0 0x7fd69806cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:33.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.029+0000 7fd69ee98700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6980b3670 con 0x7fd6980b5340 2023-11-08T08:20:33.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.029+0000 7fd69ee98700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd69806a2f0 con 0x7fd6980b4f70 2023-11-08T08:20:33.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.029+0000 7fd69ee98700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69806a5d0 con 0x7fd6980b31d0 2023-11-08T08:20:33.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.029+0000 7fd69d695700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd6980b5340 0x7fd69806c530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57216/0 (socket says 172.21.15.40:57216) 2023-11-08T08:20:33.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.029+0000 7fd69d695700 1 -- 172.21.15.40:0/2807764700 learned_addr learned my addr 172.21.15.40:0/2807764700 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:33.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.029+0000 7fd69de96700 1 --2- 172.21.15.40:0/2807764700 >> v2:172.21.15.40:3300/0 conn(0x7fd6980b4f70 0x7fd698069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:33.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.029+0000 7fd69e697700 1 --2- 172.21.15.40:0/2807764700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6980b31d0 0x7fd69806cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:33.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.029+0000 7fd69ce94700 1 -- 172.21.15.40:0/2807764700 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3444067296 0 0) 0x7fd6980b3670 con 0x7fd6980b5340 2023-11-08T08:20:33.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.029+0000 7fd69ce94700 1 -- 172.21.15.40:0/2807764700 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6840035d0 con 0x7fd6980b5340 2023-11-08T08:20:33.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.029+0000 7fd69de96700 1 -- 172.21.15.40:0/2807764700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6980b31d0 msgr2=0x7fd69806cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:33.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.029+0000 7fd69de96700 1 --2- 172.21.15.40:0/2807764700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6980b31d0 0x7fd69806cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:33.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.029+0000 7fd69de96700 1 -- 172.21.15.40:0/2807764700 >> v1:172.21.15.40:6789/0 conn(0x7fd6980b5340 legacy=0x7fd69806c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:33.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.029+0000 7fd69de96700 1 -- 172.21.15.40:0/2807764700 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69806a8b0 con 0x7fd6980b4f70 2023-11-08T08:20:33.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.033+0000 7fd69de96700 1 --2- 172.21.15.40:0/2807764700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6980b4f70 0x7fd698069bf0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd694009850 tx=0x7fd694009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad9e136613bb31a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:33.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.033+0000 7fd69ce94700 1 -- 172.21.15.40:0/2807764700 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd694011830 con 0x7fd6980b4f70 2023-11-08T08:20:33.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.033+0000 7fd69ce94700 1 -- 172.21.15.40:0/2807764700 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd694011990 con 0x7fd6980b4f70 2023-11-08T08:20:33.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.033+0000 7fd69ce94700 1 -- 172.21.15.40:0/2807764700 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd694011d40 con 0x7fd6980b4f70 2023-11-08T08:20:33.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.033+0000 7fd69ee98700 1 -- 172.21.15.40:0/2807764700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6980b4f70 msgr2=0x7fd698069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:33.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.033+0000 7fd69ee98700 1 --2- 172.21.15.40:0/2807764700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6980b4f70 0x7fd698069bf0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd694009850 tx=0x7fd694009b60 comp rx=0 tx=0).stop 2023-11-08T08:20:33.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.033+0000 7fd69ee98700 1 -- 172.21.15.40:0/2807764700 shutdown_connections 2023-11-08T08:20:33.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.033+0000 7fd69ee98700 1 --2- 172.21.15.40:0/2807764700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6980b31d0 0x7fd69806cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:33.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.033+0000 7fd69ee98700 1 --2- 172.21.15.40:0/2807764700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6980b4f70 0x7fd698069bf0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:33.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.033+0000 7fd69ee98700 1 -- 172.21.15.40:0/2807764700 >> 172.21.15.40:0/2807764700 conn(0x7fd6980d3800 msgr2=0x7fd6980cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:33.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.033+0000 7fd69ee98700 1 -- 172.21.15.40:0/2807764700 shutdown_connections 2023-11-08T08:20:33.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.033+0000 7fd69ee98700 1 -- 172.21.15.40:0/2807764700 wait complete. 2023-11-08T08:20:33.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.037+0000 7fd69ee98700 1 Processor -- start 2023-11-08T08:20:33.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.037+0000 7fd69ee98700 1 -- start start 2023-11-08T08:20:33.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.037+0000 7fd69ee98700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6980b31d0 0x7fd6980c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:33.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.037+0000 7fd69ee98700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6980b4f70 0x7fd6981593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:33.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.037+0000 7fd69ee98700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6980b5340 0x7fd6980d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:33.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.037+0000 7fd69ee98700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd698154d70 con 0x7fd6980b31d0 2023-11-08T08:20:33.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.037+0000 7fd69ee98700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6981587a0 con 0x7fd6980b5340 2023-11-08T08:20:33.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.037+0000 7fd69ee98700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980c7cd0 con 0x7fd6980b4f70 2023-11-08T08:20:33.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.041+0000 7fd69de96700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6980b31d0 0x7fd6980c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:33.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.041+0000 7fd69de96700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6980b31d0 0x7fd6980c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34152/0 (socket says 172.21.15.40:34152) 2023-11-08T08:20:33.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.041+0000 7fd69de96700 1 -- 172.21.15.40:0/3654277708 learned_addr learned my addr 172.21.15.40:0/3654277708 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:33.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.041+0000 7fd69d695700 1 --2- 172.21.15.40:0/3654277708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6980b4f70 0x7fd6981593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:33.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.041+0000 7fd69e697700 1 --2- 172.21.15.40:0/3654277708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6980b5340 0x7fd6980d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:33.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.041+0000 7fd69de96700 1 -- 172.21.15.40:0/3654277708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6980b4f70 msgr2=0x7fd6981593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:33.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.041+0000 7fd69de96700 1 --2- 172.21.15.40:0/3654277708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6980b4f70 0x7fd6981593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:33.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.041+0000 7fd69de96700 1 -- 172.21.15.40:0/3654277708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6980b5340 msgr2=0x7fd6980d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:33.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.041+0000 7fd69de96700 1 --2- 172.21.15.40:0/3654277708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6980b5340 0x7fd6980d1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:33.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.041+0000 7fd69de96700 1 -- 172.21.15.40:0/3654277708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6940094b0 con 0x7fd6980b31d0 2023-11-08T08:20:33.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.041+0000 7fd69de96700 1 --2- 172.21.15.40:0/3654277708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6980b31d0 0x7fd6980c87b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd6940090b0 tx=0x7fd694004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:33.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.041+0000 7fd68e7fc700 1 -- 172.21.15.40:0/3654277708 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd694004380 con 0x7fd6980b31d0 2023-11-08T08:20:33.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.041+0000 7fd68e7fc700 1 -- 172.21.15.40:0/3654277708 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69401f070 con 0x7fd6980b31d0 2023-11-08T08:20:33.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.041+0000 7fd69ee98700 1 -- 172.21.15.40:0/3654277708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69806c010 con 0x7fd6980b31d0 2023-11-08T08:20:33.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.041+0000 7fd68e7fc700 1 -- 172.21.15.40:0/3654277708 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6940036c0 con 0x7fd6980b31d0 2023-11-08T08:20:33.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.041+0000 7fd69ee98700 1 -- 172.21.15.40:0/3654277708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69806ab90 con 0x7fd6980b31d0 2023-11-08T08:20:33.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.041+0000 7fd69ee98700 1 -- 172.21.15.40:0/3654277708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69806d180 con 0x7fd6980b31d0 2023-11-08T08:20:33.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.045+0000 7fd68e7fc700 1 -- 172.21.15.40:0/3654277708 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd69402a020 con 0x7fd6980b31d0 2023-11-08T08:20:33.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.045+0000 7fd68e7fc700 1 --2- 172.21.15.40:0/3654277708 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd67c064ee0 0x7fd67c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:33.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.045+0000 7fd68e7fc700 1 -- 172.21.15.40:0/3654277708 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5900+0+0 (secure 0 0 0) 0x7fd6940836a0 con 0x7fd6980b31d0 2023-11-08T08:20:33.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.049+0000 7fd69d695700 1 --2- 172.21.15.40:0/3654277708 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd67c064ee0 0x7fd67c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:33.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.049+0000 7fd69d695700 1 --2- 172.21.15.40:0/3654277708 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd67c064ee0 0x7fd67c067390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd6980c7580 tx=0x7fd688007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:33.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.053+0000 7fd68e7fc700 1 -- 172.21.15.40:0/3654277708 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd6940518e0 con 0x7fd6980b31d0 2023-11-08T08:20:33.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.229+0000 7fd69ee98700 1 -- 172.21.15.40:0/3654277708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd698073cb0 con 0x7fd6980b31d0 2023-11-08T08:20:33.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.817+0000 7fd68e7fc700 1 -- 172.21.15.40:0/3654277708 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v41) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd694006ce0 con 0x7fd6980b31d0 2023-11-08T08:20:33.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.821+0000 7fd69ee98700 1 -- 172.21.15.40:0/3654277708 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd67c064ee0 msgr2=0x7fd67c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:33.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.821+0000 7fd69ee98700 1 --2- 172.21.15.40:0/3654277708 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd67c064ee0 0x7fd67c067390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd6980c7580 tx=0x7fd688007de0 comp rx=0 tx=0).stop 2023-11-08T08:20:33.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.821+0000 7fd69ee98700 1 -- 172.21.15.40:0/3654277708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6980b31d0 msgr2=0x7fd6980c87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:33.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.821+0000 7fd69ee98700 1 --2- 172.21.15.40:0/3654277708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6980b31d0 0x7fd6980c87b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd6940090b0 tx=0x7fd694004770 comp rx=0 tx=0).stop 2023-11-08T08:20:33.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.821+0000 7fd69ee98700 1 -- 172.21.15.40:0/3654277708 shutdown_connections 2023-11-08T08:20:33.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.821+0000 7fd69ee98700 1 --2- 172.21.15.40:0/3654277708 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd67c064ee0 0x7fd67c067390 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:33.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.821+0000 7fd69ee98700 1 --2- 172.21.15.40:0/3654277708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6980b5340 0x7fd6980d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:33.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.821+0000 7fd69ee98700 1 --2- 172.21.15.40:0/3654277708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6980b4f70 0x7fd6981593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:33.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.821+0000 7fd69ee98700 1 --2- 172.21.15.40:0/3654277708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6980b31d0 0x7fd6980c87b0 secure :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd6940090b0 tx=0x7fd694004770 comp rx=0 tx=0).stop 2023-11-08T08:20:33.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.821+0000 7fd69ee98700 1 -- 172.21.15.40:0/3654277708 >> 172.21.15.40:0/3654277708 conn(0x7fd6980d3800 msgr2=0x7fd6981eb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:33.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.825+0000 7fd69ee98700 1 -- 172.21.15.40:0/3654277708 shutdown_connections 2023-11-08T08:20:33.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:33.825+0000 7fd69ee98700 1 -- 172.21.15.40:0/3654277708 wait complete. 2023-11-08T08:20:33.839 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2023-11-08T08:20:33.863 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T08:20:34.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.045+0000 7efeb759e700 1 Processor -- start 2023-11-08T08:20:34.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.045+0000 7efeb759e700 1 -- start start 2023-11-08T08:20:34.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.049+0000 7efeb759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeb8159cd0 0x7efeb815d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:34.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.049+0000 7efeb759e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb815d650 con 0x7efeb806ff10 2023-11-08T08:20:34.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.049+0000 7efeb759e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb815a0a0 con 0x7efeb80cc1a0 2023-11-08T08:20:34.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.049+0000 7efeb759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb806be80 con 0x7efeb8159cd0 2023-11-08T08:20:34.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.049+0000 7efeb6d9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efeb806ff10 0x7efeb8069bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57224/0 (socket says 172.21.15.40:57224) 2023-11-08T08:20:34.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.049+0000 7efeb6d9d700 1 -- 172.21.15.40:0/1142003842 learned_addr learned my addr 172.21.15.40:0/1142003842 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:34.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.049+0000 7efeb659c700 1 --2- 172.21.15.40:0/1142003842 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeb8159cd0 0x7efeb815d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:34.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.049+0000 7efeb559a700 1 -- 172.21.15.40:0/1142003842 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 776861471 0 0) 0x7efeb815d650 con 0x7efeb806ff10 2023-11-08T08:20:34.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.049+0000 7efeb559a700 1 -- 172.21.15.40:0/1142003842 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efea00035d0 con 0x7efeb806ff10 2023-11-08T08:20:34.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.049+0000 7efeb559a700 1 -- 172.21.15.40:0/1142003842 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208809334 0 0) 0x7efeb815a0a0 con 0x7efeb80cc1a0 2023-11-08T08:20:34.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.049+0000 7efeb559a700 1 -- 172.21.15.40:0/1142003842 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efeb815d650 con 0x7efeb80cc1a0 2023-11-08T08:20:34.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.049+0000 7efeb659c700 1 -- 172.21.15.40:0/1142003842 >> v1:172.21.15.169:6789/0 conn(0x7efeb80cc1a0 legacy=0x7efeb806c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:34.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.049+0000 7efeb659c700 1 -- 172.21.15.40:0/1142003842 >> v1:172.21.15.40:6789/0 conn(0x7efeb806ff10 legacy=0x7efeb8069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:34.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.049+0000 7efeb659c700 1 -- 172.21.15.40:0/1142003842 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb80cdf80 con 0x7efeb8159cd0 2023-11-08T08:20:34.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.049+0000 7efeb659c700 1 --2- 172.21.15.40:0/1142003842 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeb8159cd0 0x7efeb815d110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efea8009880 tx=0x7efea8009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fead791d6322f266 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:34.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.053+0000 7efeb559a700 1 -- 172.21.15.40:0/1142003842 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea8011830 con 0x7efeb8159cd0 2023-11-08T08:20:34.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.053+0000 7efeb559a700 1 -- 172.21.15.40:0/1142003842 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea8011990 con 0x7efeb8159cd0 2023-11-08T08:20:34.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.053+0000 7efeb559a700 1 -- 172.21.15.40:0/1142003842 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea8011d20 con 0x7efeb8159cd0 2023-11-08T08:20:34.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.053+0000 7efeb759e700 1 -- 172.21.15.40:0/1142003842 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeb8159cd0 msgr2=0x7efeb815d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:34.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.053+0000 7efeb759e700 1 --2- 172.21.15.40:0/1142003842 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeb8159cd0 0x7efeb815d110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efea8009880 tx=0x7efea8009b90 comp rx=0 tx=0).stop 2023-11-08T08:20:34.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.053+0000 7efeb759e700 1 -- 172.21.15.40:0/1142003842 shutdown_connections 2023-11-08T08:20:34.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.053+0000 7efeb759e700 1 --2- 172.21.15.40:0/1142003842 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeb8159cd0 0x7efeb815d110 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:34.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.053+0000 7efeb759e700 1 -- 172.21.15.40:0/1142003842 >> 172.21.15.40:0/1142003842 conn(0x7efeb80cf280 msgr2=0x7efeb80cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:34.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.053+0000 7efeb759e700 1 -- 172.21.15.40:0/1142003842 shutdown_connections 2023-11-08T08:20:34.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.053+0000 7efeb759e700 1 -- 172.21.15.40:0/1142003842 wait complete. 2023-11-08T08:20:34.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.057+0000 7efeb759e700 1 Processor -- start 2023-11-08T08:20:34.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.057+0000 7efeb759e700 1 -- start start 2023-11-08T08:20:34.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.057+0000 7efeb759e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeb806ff10 0x7efeb814ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:34.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.057+0000 7efeb759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efeb80cc1a0 0x7efeb81e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:34.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.057+0000 7efeb759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeb8159cd0 0x7efeb81e35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:34.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.057+0000 7efeb759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb81e6a30 con 0x7efeb80cc1a0 2023-11-08T08:20:34.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.057+0000 7efeb759e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb81e6cb0 con 0x7efeb806ff10 2023-11-08T08:20:34.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.057+0000 7efeb759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb81e6f90 con 0x7efeb8159cd0 2023-11-08T08:20:34.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.061+0000 7efeb5d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efeb80cc1a0 0x7efeb81e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:34.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.061+0000 7efeb5d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efeb80cc1a0 0x7efeb81e7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34158/0 (socket says 172.21.15.40:34158) 2023-11-08T08:20:34.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.061+0000 7efeb5d9b700 1 -- 172.21.15.40:0/386742395 learned_addr learned my addr 172.21.15.40:0/386742395 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:34.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.061+0000 7efeb659c700 1 --2- 172.21.15.40:0/386742395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeb806ff10 0x7efeb814ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:34.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.061+0000 7efeb6d9d700 1 --2- 172.21.15.40:0/386742395 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeb8159cd0 0x7efeb81e35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:34.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.061+0000 7efeb5d9b700 1 -- 172.21.15.40:0/386742395 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeb8159cd0 msgr2=0x7efeb81e35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:34.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.061+0000 7efeb5d9b700 1 --2- 172.21.15.40:0/386742395 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeb8159cd0 0x7efeb81e35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:34.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.061+0000 7efeb5d9b700 1 -- 172.21.15.40:0/386742395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeb806ff10 msgr2=0x7efeb814ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:34.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.061+0000 7efeb5d9b700 1 --2- 172.21.15.40:0/386742395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeb806ff10 0x7efeb814ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:34.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.061+0000 7efeb5d9b700 1 -- 172.21.15.40:0/386742395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea80094b0 con 0x7efeb80cc1a0 2023-11-08T08:20:34.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.061+0000 7efeb6d9d700 1 --2- 172.21.15.40:0/386742395 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeb8159cd0 0x7efeb81e35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:20:34.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.061+0000 7efeb5d9b700 1 --2- 172.21.15.40:0/386742395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efeb80cc1a0 0x7efeb81e7d50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efeac00c730 tx=0x7efeac00caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:34.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.061+0000 7efea6ffd700 1 -- 172.21.15.40:0/386742395 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac0052d0 con 0x7efeb80cc1a0 2023-11-08T08:20:34.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.061+0000 7efea6ffd700 1 -- 172.21.15.40:0/386742395 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeac005430 con 0x7efeb80cc1a0 2023-11-08T08:20:34.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.061+0000 7efea6ffd700 1 -- 172.21.15.40:0/386742395 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac0055a0 con 0x7efeb80cc1a0 2023-11-08T08:20:34.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.061+0000 7efeb759e700 1 -- 172.21.15.40:0/386742395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb806c090 con 0x7efeb80cc1a0 2023-11-08T08:20:34.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.061+0000 7efeb759e700 1 -- 172.21.15.40:0/386742395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb80ce260 con 0x7efeb80cc1a0 2023-11-08T08:20:34.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.061+0000 7efeb759e700 1 -- 172.21.15.40:0/386742395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb806c370 con 0x7efeb80cc1a0 2023-11-08T08:20:34.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.065+0000 7efea6ffd700 1 -- 172.21.15.40:0/386742395 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7efeac019030 con 0x7efeb80cc1a0 2023-11-08T08:20:34.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.065+0000 7efea6ffd700 1 --2- 172.21.15.40:0/386742395 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efe98064f10 0x7efe980673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:34.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.065+0000 7efea6ffd700 1 -- 172.21.15.40:0/386742395 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6282+0+0 (secure 0 0 0) 0x7efeac081460 con 0x7efeb80cc1a0 2023-11-08T08:20:34.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.065+0000 7efeb659c700 1 --2- 172.21.15.40:0/386742395 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efe98064f10 0x7efe980673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:34.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.069+0000 7efeb659c700 1 --2- 172.21.15.40:0/386742395 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efe98064f10 0x7efe980673c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efea80090b0 tx=0x7efea801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:34.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.073+0000 7efea6ffd700 1 -- 172.21.15.40:0/386742395 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7efeac04f430 con 0x7efeb80cc1a0 2023-11-08T08:20:34.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.289+0000 7efeb759e700 1 -- 172.21.15.40:0/386742395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7efeb80cca80 con 0x7efeb80cc1a0 2023-11-08T08:20:34.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.301+0000 7efea6ffd700 1 -- 172.21.15.40:0/386742395 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7efeac052a50 con 0x7efeb80cc1a0 2023-11-08T08:20:34.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.313+0000 7efeb759e700 1 -- 172.21.15.40:0/386742395 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efe98064f10 msgr2=0x7efe980673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:34.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.313+0000 7efeb759e700 1 --2- 172.21.15.40:0/386742395 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efe98064f10 0x7efe980673c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efea80090b0 tx=0x7efea801c040 comp rx=0 tx=0).stop 2023-11-08T08:20:34.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.313+0000 7efeb759e700 1 -- 172.21.15.40:0/386742395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efeb80cc1a0 msgr2=0x7efeb81e7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:34.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.313+0000 7efeb759e700 1 --2- 172.21.15.40:0/386742395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efeb80cc1a0 0x7efeb81e7d50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efeac00c730 tx=0x7efeac00caf0 comp rx=0 tx=0).stop 2023-11-08T08:20:34.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.317+0000 7efeb759e700 1 -- 172.21.15.40:0/386742395 shutdown_connections 2023-11-08T08:20:34.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.317+0000 7efeb759e700 1 --2- 172.21.15.40:0/386742395 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efe98064f10 0x7efe980673c0 secure :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efea80090b0 tx=0x7efea801c040 comp rx=0 tx=0).stop 2023-11-08T08:20:34.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.317+0000 7efeb759e700 1 --2- 172.21.15.40:0/386742395 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeb8159cd0 0x7efeb81e35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:34.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.317+0000 7efeb759e700 1 --2- 172.21.15.40:0/386742395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efeb80cc1a0 0x7efeb81e7d50 secure :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efeac00c730 tx=0x7efeac00caf0 comp rx=0 tx=0).stop 2023-11-08T08:20:34.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.317+0000 7efeb759e700 1 --2- 172.21.15.40:0/386742395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeb806ff10 0x7efeb814ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:34.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.317+0000 7efeb759e700 1 -- 172.21.15.40:0/386742395 >> 172.21.15.40:0/386742395 conn(0x7efeb80cf280 msgr2=0x7efeb80afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:34.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.321+0000 7efeb759e700 1 -- 172.21.15.40:0/386742395 shutdown_connections 2023-11-08T08:20:34.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.321+0000 7efeb759e700 1 -- 172.21.15.40:0/386742395 wait complete. 2023-11-08T08:20:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 6 and data pool 7 2023-11-08T08:20:34.363 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:20:34.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.537+0000 7f4ef01d9700 1 Processor -- start 2023-11-08T08:20:34.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.537+0000 7f4ef01d9700 1 -- start start 2023-11-08T08:20:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.541+0000 7f4ef01d9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ee8159e00 0x7f4ee8069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.541+0000 7f4ef01d9700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4ee80cdc60 0x7f4ee80ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.541+0000 7f4ef01d9700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee806c8c0 con 0x7f4ee80cd1e0 2023-11-08T08:20:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.541+0000 7f4ef01d9700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee80ce5c0 con 0x7f4ee80cdc60 2023-11-08T08:20:34.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.541+0000 7f4ef01d9700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee80b1f10 con 0x7f4ee8159e00 2023-11-08T08:20:34.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.541+0000 7f4eed774700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4ee80cd1e0 0x7f4ee815d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57226/0 (socket says 172.21.15.40:57226) 2023-11-08T08:20:34.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.541+0000 7f4eed774700 1 -- 172.21.15.40:0/1254267942 learned_addr learned my addr 172.21.15.40:0/1254267942 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:34.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.541+0000 7f4eedf75700 1 --2- 172.21.15.40:0/1254267942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ee8159e00 0x7f4ee8069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:34.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.541+0000 7f4eee776700 1 --2- 172.21.15.40:0/1254267942 >> v2:172.21.15.169:3300/0 conn(0x7f4ee80cdc60 0x7f4ee80ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:34.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.541+0000 7f4eecf73700 1 -- 172.21.15.40:0/1254267942 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2475418101 0 0) 0x7f4ee806c8c0 con 0x7f4ee80cd1e0 2023-11-08T08:20:34.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.541+0000 7f4eecf73700 1 -- 172.21.15.40:0/1254267942 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ed40035d0 con 0x7f4ee80cd1e0 2023-11-08T08:20:34.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.541+0000 7f4eedf75700 1 -- 172.21.15.40:0/1254267942 >> v2:172.21.15.169:3300/0 conn(0x7f4ee80cdc60 msgr2=0x7f4ee80ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:34.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.541+0000 7f4eedf75700 1 --2- 172.21.15.40:0/1254267942 >> v2:172.21.15.169:3300/0 conn(0x7f4ee80cdc60 0x7f4ee80ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:34.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.541+0000 7f4eedf75700 1 -- 172.21.15.40:0/1254267942 >> v1:172.21.15.40:6789/0 conn(0x7f4ee80cd1e0 legacy=0x7f4ee815d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:34.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.541+0000 7f4eedf75700 1 -- 172.21.15.40:0/1254267942 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee80b21f0 con 0x7f4ee8159e00 2023-11-08T08:20:34.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.541+0000 7f4eee776700 1 --2- 172.21.15.40:0/1254267942 >> v2:172.21.15.169:3300/0 conn(0x7f4ee80cdc60 0x7f4ee80ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:34.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.545+0000 7f4eedf75700 1 --2- 172.21.15.40:0/1254267942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ee8159e00 0x7f4ee8069bf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009850 tx=0x7f4ee4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55c92d14be3a25aa server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:34.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.545+0000 7f4eecf73700 1 -- 172.21.15.40:0/1254267942 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4011830 con 0x7f4ee8159e00 2023-11-08T08:20:34.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.545+0000 7f4eecf73700 1 -- 172.21.15.40:0/1254267942 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee4011990 con 0x7f4ee8159e00 2023-11-08T08:20:34.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.545+0000 7f4eecf73700 1 -- 172.21.15.40:0/1254267942 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4011d40 con 0x7f4ee8159e00 2023-11-08T08:20:34.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.545+0000 7f4ef01d9700 1 -- 172.21.15.40:0/1254267942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ee8159e00 msgr2=0x7f4ee8069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:34.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.545+0000 7f4ef01d9700 1 --2- 172.21.15.40:0/1254267942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ee8159e00 0x7f4ee8069bf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009850 tx=0x7f4ee4009b60 comp rx=0 tx=0).stop 2023-11-08T08:20:34.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.545+0000 7f4ef01d9700 1 -- 172.21.15.40:0/1254267942 shutdown_connections 2023-11-08T08:20:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.545+0000 7f4ef01d9700 1 --2- 172.21.15.40:0/1254267942 >> v2:172.21.15.169:3300/0 conn(0x7f4ee80cdc60 0x7f4ee80ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.545+0000 7f4ef01d9700 1 --2- 172.21.15.40:0/1254267942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ee8159e00 0x7f4ee8069bf0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.545+0000 7f4ef01d9700 1 -- 172.21.15.40:0/1254267942 >> 172.21.15.40:0/1254267942 conn(0x7f4ee80cf6a0 msgr2=0x7f4ee80cfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.545+0000 7f4ef01d9700 1 -- 172.21.15.40:0/1254267942 shutdown_connections 2023-11-08T08:20:34.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.545+0000 7f4ef01d9700 1 -- 172.21.15.40:0/1254267942 wait complete. 2023-11-08T08:20:34.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.549+0000 7f4ef01d9700 1 Processor -- start 2023-11-08T08:20:34.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.549+0000 7f4ef01d9700 1 -- start start 2023-11-08T08:20:34.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.549+0000 7f4ef01d9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ee80cd1e0 0x7f4ee814ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:34.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.549+0000 7f4ef01d9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee80cdc60 0x7f4ee81e3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:34.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.549+0000 7f4ef01d9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ee8159e00 0x7f4ee81e4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:34.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.549+0000 7f4ef01d9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee80b7810 con 0x7f4ee80cdc60 2023-11-08T08:20:34.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.549+0000 7f4ef01d9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee80b6b90 con 0x7f4ee8159e00 2023-11-08T08:20:34.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.549+0000 7f4ef01d9700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee80b6e70 con 0x7f4ee80cd1e0 2023-11-08T08:20:34.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.553+0000 7f4eed774700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee80cdc60 0x7f4ee81e3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:34.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.553+0000 7f4eedf75700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ee80cd1e0 0x7f4ee814ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:34.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.553+0000 7f4eed774700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee80cdc60 0x7f4ee81e3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34168/0 (socket says 172.21.15.40:34168) 2023-11-08T08:20:34.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.553+0000 7f4eed774700 1 -- 172.21.15.40:0/3459471745 learned_addr learned my addr 172.21.15.40:0/3459471745 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.553+0000 7f4eee776700 1 --2- 172.21.15.40:0/3459471745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ee8159e00 0x7f4ee81e4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.553+0000 7f4eedf75700 1 -- 172.21.15.40:0/3459471745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ee8159e00 msgr2=0x7f4ee81e4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.553+0000 7f4eedf75700 1 --2- 172.21.15.40:0/3459471745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ee8159e00 0x7f4ee81e4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.553+0000 7f4eedf75700 1 -- 172.21.15.40:0/3459471745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee80cdc60 msgr2=0x7f4ee81e3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:34.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.553+0000 7f4eedf75700 1 --2- 172.21.15.40:0/3459471745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee80cdc60 0x7f4ee81e3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:34.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.553+0000 7f4eedf75700 1 -- 172.21.15.40:0/3459471745 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee0009a70 con 0x7f4ee80cd1e0 2023-11-08T08:20:34.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.553+0000 7f4eedf75700 1 --2- 172.21.15.40:0/3459471745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ee80cd1e0 0x7f4ee814ea40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4016000 tx=0x7f4ee4004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.553+0000 7f4ede7fc700 1 -- 172.21.15.40:0/3459471745 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4004450 con 0x7f4ee80cd1e0 2023-11-08T08:20:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.553+0000 7f4ede7fc700 1 -- 172.21.15.40:0/3459471745 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee401e070 con 0x7f4ee80cd1e0 2023-11-08T08:20:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.553+0000 7f4ef01d9700 1 -- 172.21.15.40:0/3459471745 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee40094b0 con 0x7f4ee80cd1e0 2023-11-08T08:20:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.553+0000 7f4ede7fc700 1 -- 172.21.15.40:0/3459471745 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40038c0 con 0x7f4ee80cd1e0 2023-11-08T08:20:34.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.553+0000 7f4ef01d9700 1 -- 172.21.15.40:0/3459471745 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee815a1d0 con 0x7f4ee80cd1e0 2023-11-08T08:20:34.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.557+0000 7f4ef01d9700 1 -- 172.21.15.40:0/3459471745 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee806cb30 con 0x7f4ee80cd1e0 2023-11-08T08:20:34.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.557+0000 7f4ede7fc700 1 -- 172.21.15.40:0/3459471745 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4ee402a020 con 0x7f4ee80cd1e0 2023-11-08T08:20:34.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.561+0000 7f4ede7fc700 1 --2- 172.21.15.40:0/3459471745 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4ecc064f30 0x7f4ecc0673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:34.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.561+0000 7f4ede7fc700 1 -- 172.21.15.40:0/3459471745 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4ee4082d70 con 0x7f4ee80cd1e0 2023-11-08T08:20:34.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.561+0000 7f4eed774700 1 --2- 172.21.15.40:0/3459471745 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4ecc064f30 0x7f4ecc0673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:34.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.561+0000 7f4eed774700 1 --2- 172.21.15.40:0/3459471745 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4ecc064f30 0x7f4ecc0673e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4ee81560f0 tx=0x7f4ed8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:34.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.569+0000 7f4ede7fc700 1 -- 172.21.15.40:0/3459471745 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4ee4088020 con 0x7f4ee80cd1e0 2023-11-08T08:20:34.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.749+0000 7f4ef01d9700 1 -- 172.21.15.40:0/3459471745 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ee80b24d0 con 0x7f4ee80cd1e0 2023-11-08T08:20:34.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.749+0000 7f4ede7fc700 1 -- 172.21.15.40:0/3459471745 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18045 (secure 0 0 0) 0x7f4ee4050d60 con 0x7f4ee80cd1e0 2023-11-08T08:20:34.756 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:34.756 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":42,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:20:34.293790+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:20:32.059150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-08T08:20:33.297064+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:20:34.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.753+0000 7f4ef01d9700 1 -- 172.21.15.40:0/3459471745 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4ecc064f30 msgr2=0x7f4ecc0673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:34.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.753+0000 7f4ef01d9700 1 --2- 172.21.15.40:0/3459471745 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4ecc064f30 0x7f4ecc0673e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4ee81560f0 tx=0x7f4ed8009040 comp rx=0 tx=0).stop 2023-11-08T08:20:34.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.753+0000 7f4ef01d9700 1 -- 172.21.15.40:0/3459471745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ee80cd1e0 msgr2=0x7f4ee814ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:34.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.753+0000 7f4ef01d9700 1 --2- 172.21.15.40:0/3459471745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ee80cd1e0 0x7f4ee814ea40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4016000 tx=0x7f4ee4004770 comp rx=0 tx=0).stop 2023-11-08T08:20:34.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.753+0000 7f4ef01d9700 1 -- 172.21.15.40:0/3459471745 shutdown_connections 2023-11-08T08:20:34.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.753+0000 7f4ef01d9700 1 --2- 172.21.15.40:0/3459471745 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4ecc064f30 0x7f4ecc0673e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:34.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.753+0000 7f4ef01d9700 1 --2- 172.21.15.40:0/3459471745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ee8159e00 0x7f4ee81e4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:34.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.753+0000 7f4ef01d9700 1 --2- 172.21.15.40:0/3459471745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee80cdc60 0x7f4ee81e3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:34.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.753+0000 7f4ef01d9700 1 --2- 172.21.15.40:0/3459471745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ee80cd1e0 0x7f4ee814ea40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:34.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.753+0000 7f4ef01d9700 1 -- 172.21.15.40:0/3459471745 >> 172.21.15.40:0/3459471745 conn(0x7f4ee80cf6a0 msgr2=0x7f4ee80b4650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:34.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.753+0000 7f4ef01d9700 1 -- 172.21.15.40:0/3459471745 shutdown_connections 2023-11-08T08:20:34.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.753+0000 7f4ef01d9700 1 -- 172.21.15.40:0/3459471745 wait complete. 2023-11-08T08:20:34.782 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:20:34.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.957+0000 7fcb92fe8700 1 Processor -- start 2023-11-08T08:20:34.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.957+0000 7fcb92fe8700 1 -- start start 2023-11-08T08:20:34.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.957+0000 7fcb92fe8700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcb8c069e00 0x7fcb8c06a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:34.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.957+0000 7fcb92fe8700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8c06c600 con 0x7fcb8c15c640 2023-11-08T08:20:34.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.957+0000 7fcb92fe8700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8c0722f0 con 0x7fcb8c071aa0 2023-11-08T08:20:34.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.957+0000 7fcb92fe8700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb8c06cef0 con 0x7fcb8c069e00 2023-11-08T08:20:34.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.957+0000 7fcb91fe6700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcb8c15c640 0x7fcb8c0d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57228/0 (socket says 172.21.15.40:57228) 2023-11-08T08:20:34.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.957+0000 7fcb91fe6700 1 -- 172.21.15.40:0/2091100984 learned_addr learned my addr 172.21.15.40:0/2091100984 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:34.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.957+0000 7fcb927e7700 1 --2- 172.21.15.40:0/2091100984 >> v2:172.21.15.169:3300/0 conn(0x7fcb8c069e00 0x7fcb8c06a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:34.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.961+0000 7fcb90fe4700 1 -- 172.21.15.40:0/2091100984 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376396956 0 0) 0x7fcb8c06c600 con 0x7fcb8c15c640 2023-11-08T08:20:34.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.961+0000 7fcb90fe4700 1 -- 172.21.15.40:0/2091100984 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb780035d0 con 0x7fcb8c15c640 2023-11-08T08:20:34.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.961+0000 7fcb927e7700 1 -- 172.21.15.40:0/2091100984 >> v1:172.21.15.169:6789/0 conn(0x7fcb8c071aa0 legacy=0x7fcb8c06d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:34.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.961+0000 7fcb927e7700 1 -- 172.21.15.40:0/2091100984 >> v1:172.21.15.40:6789/0 conn(0x7fcb8c15c640 legacy=0x7fcb8c0d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:34.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.961+0000 7fcb927e7700 1 -- 172.21.15.40:0/2091100984 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c06da20 con 0x7fcb8c069e00 2023-11-08T08:20:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.961+0000 7fcb927e7700 1 --2- 172.21.15.40:0/2091100984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb8c069e00 0x7fcb8c06a280 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009d90 tx=0x7fcb8400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acec3411daef01c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.961+0000 7fcb90fe4700 1 -- 172.21.15.40:0/2091100984 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8400e070 con 0x7fcb8c069e00 2023-11-08T08:20:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.961+0000 7fcb90fe4700 1 -- 172.21.15.40:0/2091100984 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8400bb20 con 0x7fcb8c069e00 2023-11-08T08:20:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.961+0000 7fcb90fe4700 1 -- 172.21.15.40:0/2091100984 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84012430 con 0x7fcb8c069e00 2023-11-08T08:20:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.961+0000 7fcb92fe8700 1 -- 172.21.15.40:0/2091100984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb8c069e00 msgr2=0x7fcb8c06a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:34.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.961+0000 7fcb92fe8700 1 --2- 172.21.15.40:0/2091100984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb8c069e00 0x7fcb8c06a280 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009d90 tx=0x7fcb8400b330 comp rx=0 tx=0).stop 2023-11-08T08:20:34.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.961+0000 7fcb92fe8700 1 -- 172.21.15.40:0/2091100984 shutdown_connections 2023-11-08T08:20:34.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.961+0000 7fcb92fe8700 1 --2- 172.21.15.40:0/2091100984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb8c069e00 0x7fcb8c06a280 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:34.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.961+0000 7fcb92fe8700 1 -- 172.21.15.40:0/2091100984 >> 172.21.15.40:0/2091100984 conn(0x7fcb8c0b6520 msgr2=0x7fcb8c071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:34.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.961+0000 7fcb92fe8700 1 -- 172.21.15.40:0/2091100984 shutdown_connections 2023-11-08T08:20:34.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.961+0000 7fcb92fe8700 1 -- 172.21.15.40:0/2091100984 wait complete. 2023-11-08T08:20:34.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.965+0000 7fcb92fe8700 1 Processor -- start 2023-11-08T08:20:34.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.965+0000 7fcb92fe8700 1 -- start start 2023-11-08T08:20:34.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.965+0000 7fcb92fe8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb8c069e00 0x7fcb8c0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:34.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.965+0000 7fcb92fe8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb8c071aa0 0x7fcb8c0c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:34.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.965+0000 7fcb92fe8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb8c15c640 0x7fcb8c1f11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:34.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.965+0000 7fcb92fe8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c0d38f0 con 0x7fcb8c071aa0 2023-11-08T08:20:34.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.965+0000 7fcb92fe8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c0c9f90 con 0x7fcb8c069e00 2023-11-08T08:20:34.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.965+0000 7fcb92fe8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c0c85c0 con 0x7fcb8c15c640 2023-11-08T08:20:34.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.965+0000 7fcb917e5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb8c071aa0 0x7fcb8c0c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:34.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.965+0000 7fcb917e5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb8c071aa0 0x7fcb8c0c6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34170/0 (socket says 172.21.15.40:34170) 2023-11-08T08:20:34.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.965+0000 7fcb917e5700 1 -- 172.21.15.40:0/2293246890 learned_addr learned my addr 172.21.15.40:0/2293246890 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:34.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.965+0000 7fcb91fe6700 1 --2- 172.21.15.40:0/2293246890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb8c069e00 0x7fcb8c0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:34.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.965+0000 7fcb927e7700 1 --2- 172.21.15.40:0/2293246890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb8c15c640 0x7fcb8c1f11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:34.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.969+0000 7fcb917e5700 1 -- 172.21.15.40:0/2293246890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb8c15c640 msgr2=0x7fcb8c1f11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:34.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.969+0000 7fcb917e5700 1 --2- 172.21.15.40:0/2293246890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb8c15c640 0x7fcb8c1f11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:34.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.969+0000 7fcb917e5700 1 -- 172.21.15.40:0/2293246890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb8c069e00 msgr2=0x7fcb8c0b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:34.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.969+0000 7fcb917e5700 1 --2- 172.21.15.40:0/2293246890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb8c069e00 0x7fcb8c0b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:34.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.969+0000 7fcb917e5700 1 -- 172.21.15.40:0/2293246890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb84009a70 con 0x7fcb8c071aa0 2023-11-08T08:20:34.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.969+0000 7fcb917e5700 1 --2- 172.21.15.40:0/2293246890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb8c071aa0 0x7fcb8c0c6ef0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00c500 tx=0x7fcb7c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:34.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.969+0000 7fcb827fc700 1 -- 172.21.15.40:0/2293246890 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c0052d0 con 0x7fcb8c071aa0 2023-11-08T08:20:34.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.969+0000 7fcb827fc700 1 -- 172.21.15.40:0/2293246890 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c005430 con 0x7fcb8c071aa0 2023-11-08T08:20:34.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.969+0000 7fcb827fc700 1 -- 172.21.15.40:0/2293246890 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c0055a0 con 0x7fcb8c071aa0 2023-11-08T08:20:34.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.969+0000 7fcb92fe8700 1 -- 172.21.15.40:0/2293246890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8c072010 con 0x7fcb8c071aa0 2023-11-08T08:20:34.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.969+0000 7fcb92fe8700 1 -- 172.21.15.40:0/2293246890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8c15ca10 con 0x7fcb8c071aa0 2023-11-08T08:20:34.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.969+0000 7fcb92fe8700 1 -- 172.21.15.40:0/2293246890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8c06c830 con 0x7fcb8c071aa0 2023-11-08T08:20:34.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.973+0000 7fcb827fc700 1 -- 172.21.15.40:0/2293246890 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fcb7c019030 con 0x7fcb8c071aa0 2023-11-08T08:20:34.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.973+0000 7fcb827fc700 1 --2- 172.21.15.40:0/2293246890 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcb70064ee0 0x7fcb70067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:34.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.973+0000 7fcb827fc700 1 -- 172.21.15.40:0/2293246890 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fcb7c081240 con 0x7fcb8c071aa0 2023-11-08T08:20:34.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.977+0000 7fcb91fe6700 1 --2- 172.21.15.40:0/2293246890 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcb70064ee0 0x7fcb70067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:34.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.977+0000 7fcb91fe6700 1 --2- 172.21.15.40:0/2293246890 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcb70064ee0 0x7fcb70067390 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcb88007fd0 tx=0x7fcb88007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:34.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:34.981+0000 7fcb827fc700 1 -- 172.21.15.40:0/2293246890 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fcb7c04f1b0 con 0x7fcb8c071aa0 2023-11-08T08:20:35.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.149+0000 7fcb92fe8700 1 -- 172.21.15.40:0/2293246890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb8c073cb0 con 0x7fcb8c071aa0 2023-11-08T08:20:35.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.153+0000 7fcb827fc700 1 -- 172.21.15.40:0/2293246890 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18045 (secure 0 0 0) 0x7fcb7c0527d0 con 0x7fcb8c071aa0 2023-11-08T08:20:35.159 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:35.160 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":42,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:20:34.293790+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:20:32.059150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-08T08:20:33.297064+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:20:35.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.157+0000 7fcb92fe8700 1 -- 172.21.15.40:0/2293246890 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcb70064ee0 msgr2=0x7fcb70067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.157+0000 7fcb92fe8700 1 --2- 172.21.15.40:0/2293246890 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcb70064ee0 0x7fcb70067390 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcb88007fd0 tx=0x7fcb88007de0 comp rx=0 tx=0).stop 2023-11-08T08:20:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.157+0000 7fcb92fe8700 1 -- 172.21.15.40:0/2293246890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb8c071aa0 msgr2=0x7fcb8c0c6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.157+0000 7fcb92fe8700 1 --2- 172.21.15.40:0/2293246890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb8c071aa0 0x7fcb8c0c6ef0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00c500 tx=0x7fcb7c00c810 comp rx=0 tx=0).stop 2023-11-08T08:20:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.157+0000 7fcb92fe8700 1 -- 172.21.15.40:0/2293246890 shutdown_connections 2023-11-08T08:20:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.157+0000 7fcb92fe8700 1 --2- 172.21.15.40:0/2293246890 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcb70064ee0 0x7fcb70067390 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.157+0000 7fcb92fe8700 1 --2- 172.21.15.40:0/2293246890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb8c15c640 0x7fcb8c1f11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.157+0000 7fcb92fe8700 1 --2- 172.21.15.40:0/2293246890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb8c071aa0 0x7fcb8c0c6ef0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.157+0000 7fcb92fe8700 1 --2- 172.21.15.40:0/2293246890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb8c069e00 0x7fcb8c0b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.157+0000 7fcb92fe8700 1 -- 172.21.15.40:0/2293246890 >> 172.21.15.40:0/2293246890 conn(0x7fcb8c0b6520 msgr2=0x7fcb8c0d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:35.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.157+0000 7fcb92fe8700 1 -- 172.21.15.40:0/2293246890 shutdown_connections 2023-11-08T08:20:35.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.157+0000 7fcb92fe8700 1 -- 172.21.15.40:0/2293246890 wait complete. 2023-11-08T08:20:35.185 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T08:20:35.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.369+0000 7fe32580f700 1 Processor -- start 2023-11-08T08:20:35.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.373+0000 7fe32580f700 1 -- start start 2023-11-08T08:20:35.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.373+0000 7fe32580f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe32015c410 0x7fe320160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:35.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.373+0000 7fe32580f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3200ca740 con 0x7fe3200cf2a0 2023-11-08T08:20:35.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.373+0000 7fe32580f700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe320161020 con 0x7fe3200b1240 2023-11-08T08:20:35.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.373+0000 7fe32580f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3200ca9e0 con 0x7fe32015c410 2023-11-08T08:20:35.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.373+0000 7fe32480d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe32015c410 0x7fe320160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:35.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.373+0000 7fe317fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe3200cf2a0 0x7fe320069c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57244/0 (socket says 172.21.15.40:57244) 2023-11-08T08:20:35.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.373+0000 7fe317fff700 1 -- 172.21.15.40:0/1012145958 learned_addr learned my addr 172.21.15.40:0/1012145958 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:35.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.377+0000 7fe32480d700 1 -- 172.21.15.40:0/1012145958 >> v1:172.21.15.169:6789/0 conn(0x7fe3200b1240 legacy=0x7fe3200b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:35.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.377+0000 7fe3177fe700 1 -- 172.21.15.40:0/1012145958 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1351270415 0 0) 0x7fe3200ca740 con 0x7fe3200cf2a0 2023-11-08T08:20:35.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.377+0000 7fe32480d700 1 -- 172.21.15.40:0/1012145958 >> v1:172.21.15.40:6789/0 conn(0x7fe3200cf2a0 legacy=0x7fe320069c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:35.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.377+0000 7fe32480d700 1 -- 172.21.15.40:0/1012145958 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3200cacc0 con 0x7fe32015c410 2023-11-08T08:20:35.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.377+0000 7fe32480d700 1 --2- 172.21.15.40:0/1012145958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe32015c410 0x7fe320160ae0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009d90 tx=0x7fe31c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60646a65bf7a5cf5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:35.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.377+0000 7fe3177fe700 1 -- 172.21.15.40:0/1012145958 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c00e070 con 0x7fe32015c410 2023-11-08T08:20:35.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.377+0000 7fe3177fe700 1 -- 172.21.15.40:0/1012145958 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31c00bad0 con 0x7fe32015c410 2023-11-08T08:20:35.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.377+0000 7fe3177fe700 1 -- 172.21.15.40:0/1012145958 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c00be80 con 0x7fe32015c410 2023-11-08T08:20:35.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.377+0000 7fe32580f700 1 -- 172.21.15.40:0/1012145958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe32015c410 msgr2=0x7fe320160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:35.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.377+0000 7fe32580f700 1 --2- 172.21.15.40:0/1012145958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe32015c410 0x7fe320160ae0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009d90 tx=0x7fe31c00b330 comp rx=0 tx=0).stop 2023-11-08T08:20:35.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.377+0000 7fe32580f700 1 -- 172.21.15.40:0/1012145958 shutdown_connections 2023-11-08T08:20:35.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.377+0000 7fe32580f700 1 --2- 172.21.15.40:0/1012145958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe32015c410 0x7fe320160ae0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:35.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.377+0000 7fe32580f700 1 -- 172.21.15.40:0/1012145958 >> 172.21.15.40:0/1012145958 conn(0x7fe3200cc240 msgr2=0x7fe3200cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:35.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.377+0000 7fe32580f700 1 -- 172.21.15.40:0/1012145958 shutdown_connections 2023-11-08T08:20:35.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.377+0000 7fe32580f700 1 -- 172.21.15.40:0/1012145958 wait complete. 2023-11-08T08:20:35.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.381+0000 7fe32580f700 1 Processor -- start 2023-11-08T08:20:35.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.381+0000 7fe32580f700 1 -- start start 2023-11-08T08:20:35.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe32580f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3200b1240 0x7fe3200c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:35.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe32580f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3200cf2a0 0x7fe3201f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:35.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe32580f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe32015c410 0x7fe3200b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:35.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe32580f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3201ef770 con 0x7fe32015c410 2023-11-08T08:20:35.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe32580f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3201f2cf0 con 0x7fe3200cf2a0 2023-11-08T08:20:35.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe32580f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3201f2fd0 con 0x7fe3200b1240 2023-11-08T08:20:35.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe32500e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe32015c410 0x7fe3200b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:35.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe32500e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe32015c410 0x7fe3200b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34188/0 (socket says 172.21.15.40:34188) 2023-11-08T08:20:35.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe32500e700 1 -- 172.21.15.40:0/2987862785 learned_addr learned my addr 172.21.15.40:0/2987862785 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:35.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe32480d700 1 --2- 172.21.15.40:0/2987862785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3200b1240 0x7fe3200c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:35.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe317fff700 1 --2- 172.21.15.40:0/2987862785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3200cf2a0 0x7fe3201f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:35.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe32500e700 1 -- 172.21.15.40:0/2987862785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3200b1240 msgr2=0x7fe3200c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:35.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe32500e700 1 --2- 172.21.15.40:0/2987862785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3200b1240 0x7fe3200c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:35.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe32500e700 1 -- 172.21.15.40:0/2987862785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3200cf2a0 msgr2=0x7fe3201f2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:35.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe32500e700 1 --2- 172.21.15.40:0/2987862785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3200cf2a0 0x7fe3201f2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:35.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe32500e700 1 -- 172.21.15.40:0/2987862785 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c009a70 con 0x7fe32015c410 2023-11-08T08:20:35.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe32500e700 1 --2- 172.21.15.40:0/2987862785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe32015c410 0x7fe3200b0320 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe31800c750 tx=0x7fe31800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:35.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe3157fa700 1 -- 172.21.15.40:0/2987862785 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3180054e0 con 0x7fe32015c410 2023-11-08T08:20:35.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe3157fa700 1 -- 172.21.15.40:0/2987862785 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3180036a0 con 0x7fe32015c410 2023-11-08T08:20:35.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe3157fa700 1 -- 172.21.15.40:0/2987862785 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe318003a50 con 0x7fe32015c410 2023-11-08T08:20:35.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.385+0000 7fe32580f700 1 -- 172.21.15.40:0/2987862785 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3200cde70 con 0x7fe32015c410 2023-11-08T08:20:35.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.389+0000 7fe32580f700 1 -- 172.21.15.40:0/2987862785 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3200cdb90 con 0x7fe32015c410 2023-11-08T08:20:35.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.389+0000 7fe32580f700 1 -- 172.21.15.40:0/2987862785 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe32006a370 con 0x7fe32015c410 2023-11-08T08:20:35.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.389+0000 7fe3157fa700 1 -- 172.21.15.40:0/2987862785 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fe318010070 con 0x7fe32015c410 2023-11-08T08:20:35.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.393+0000 7fe3157fa700 1 --2- 172.21.15.40:0/2987862785 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe30c064ee0 0x7fe30c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:35.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.393+0000 7fe3157fa700 1 -- 172.21.15.40:0/2987862785 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe318080f40 con 0x7fe32015c410 2023-11-08T08:20:35.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.393+0000 7fe32480d700 1 --2- 172.21.15.40:0/2987862785 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe30c064ee0 0x7fe30c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:35.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.393+0000 7fe32480d700 1 --2- 172.21.15.40:0/2987862785 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe30c064ee0 0x7fe30c067390 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe31c000f80 tx=0x7fe31c011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:35.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.401+0000 7fe3157fa700 1 -- 172.21.15.40:0/2987862785 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fe31804eeb0 con 0x7fe32015c410 2023-11-08T08:20:35.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:35.613+0000 7fe32580f700 1 -- 172.21.15.40:0/2987862785 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe320073d20 con 0x7fe32015c410 2023-11-08T08:20:36.235 INFO:tasks.ceph.mds.c.smithi040.stderr:2023-11-08T08:20:36.229+0000 7fba7688d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T08:20:36.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.317+0000 7fe3157fa700 1 -- 172.21.15.40:0/2987862785 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe3180524d0 con 0x7fe32015c410 2023-11-08T08:20:36.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.321+0000 7fe32580f700 1 -- 172.21.15.40:0/2987862785 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe30c064ee0 msgr2=0x7fe30c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:36.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.321+0000 7fe32580f700 1 --2- 172.21.15.40:0/2987862785 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe30c064ee0 0x7fe30c067390 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe31c000f80 tx=0x7fe31c011000 comp rx=0 tx=0).stop 2023-11-08T08:20:36.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.325+0000 7fe32580f700 1 -- 172.21.15.40:0/2987862785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe32015c410 msgr2=0x7fe3200b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:36.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.325+0000 7fe32580f700 1 --2- 172.21.15.40:0/2987862785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe32015c410 0x7fe3200b0320 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe31800c750 tx=0x7fe31800ca60 comp rx=0 tx=0).stop 2023-11-08T08:20:36.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.325+0000 7fe32580f700 1 -- 172.21.15.40:0/2987862785 shutdown_connections 2023-11-08T08:20:36.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.325+0000 7fe32580f700 1 --2- 172.21.15.40:0/2987862785 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe30c064ee0 0x7fe30c067390 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:36.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.325+0000 7fe32580f700 1 --2- 172.21.15.40:0/2987862785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe32015c410 0x7fe3200b0320 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:36.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.325+0000 7fe32580f700 1 --2- 172.21.15.40:0/2987862785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3200cf2a0 0x7fe3201f2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:36.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.325+0000 7fe32580f700 1 --2- 172.21.15.40:0/2987862785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3200b1240 0x7fe3200c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:36.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.325+0000 7fe32580f700 1 -- 172.21.15.40:0/2987862785 >> 172.21.15.40:0/2987862785 conn(0x7fe3200cc240 msgr2=0x7fe3200710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:36.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.325+0000 7fe32580f700 1 -- 172.21.15.40:0/2987862785 shutdown_connections 2023-11-08T08:20:36.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.325+0000 7fe32580f700 1 -- 172.21.15.40:0/2987862785 wait complete. 2023-11-08T08:20:36.354 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-08T08:20:36.354 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T08:20:36.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.537+0000 7f48399e8700 1 Processor -- start 2023-11-08T08:20:36.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.537+0000 7f48399e8700 1 -- start start 2023-11-08T08:20:36.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.537+0000 7f48399e8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4834070620 0x7f48340b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:36.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.541+0000 7f48399e8700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483415c520 con 0x7f4834069bf0 2023-11-08T08:20:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.541+0000 7f48399e8700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483406cc50 con 0x7f483415c150 2023-11-08T08:20:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.541+0000 7f48399e8700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48340b4240 con 0x7f4834070620 2023-11-08T08:20:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.541+0000 7f4833fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4834070620 0x7f48340b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.541+0000 7f4833fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4834070620 0x7f48340b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34198/0 (socket says 172.21.15.40:34198) 2023-11-08T08:20:36.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.541+0000 7f4833fff700 1 -- 172.21.15.40:0/58170369 learned_addr learned my addr 172.21.15.40:0/58170369 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:36.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.541+0000 7f4833fff700 1 -- 172.21.15.40:0/58170369 >> v1:172.21.15.169:6789/0 conn(0x7f483415c150 legacy=0x7f483406ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:36.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.541+0000 7f48337fe700 1 -- 172.21.15.40:0/58170369 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822992398 0 0) 0x7f483406cc50 con 0x7f483415c150 2023-11-08T08:20:36.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.541+0000 7f4833fff700 1 -- 172.21.15.40:0/58170369 >> v1:172.21.15.40:6789/0 conn(0x7f4834069bf0 legacy=0x7f4834069ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:36.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.541+0000 7f4833fff700 1 -- 172.21.15.40:0/58170369 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48340d2450 con 0x7f4834070620 2023-11-08T08:20:36.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.541+0000 7f4833fff700 1 --2- 172.21.15.40:0/58170369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4834070620 0x7f48340b3d00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4820009900 tx=0x7f4820009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bc9aeeb5ce8611 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:36.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.541+0000 7f48337fe700 1 -- 172.21.15.40:0/58170369 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482000ba60 con 0x7f4834070620 2023-11-08T08:20:36.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.541+0000 7f48337fe700 1 -- 172.21.15.40:0/58170369 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482000bbc0 con 0x7f4834070620 2023-11-08T08:20:36.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.541+0000 7f48337fe700 1 -- 172.21.15.40:0/58170369 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4820009e50 con 0x7f4834070620 2023-11-08T08:20:36.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.545+0000 7f48399e8700 1 -- 172.21.15.40:0/58170369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4834070620 msgr2=0x7f48340b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:36.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.545+0000 7f48399e8700 1 --2- 172.21.15.40:0/58170369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4834070620 0x7f48340b3d00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4820009900 tx=0x7f4820009c10 comp rx=0 tx=0).stop 2023-11-08T08:20:36.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.545+0000 7f48399e8700 1 -- 172.21.15.40:0/58170369 shutdown_connections 2023-11-08T08:20:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.545+0000 7f48399e8700 1 --2- 172.21.15.40:0/58170369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4834070620 0x7f48340b3d00 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.545+0000 7f48399e8700 1 -- 172.21.15.40:0/58170369 >> 172.21.15.40:0/58170369 conn(0x7f48340d3680 msgr2=0x7f48340cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.545+0000 7f48399e8700 1 -- 172.21.15.40:0/58170369 shutdown_connections 2023-11-08T08:20:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.545+0000 7f48399e8700 1 -- 172.21.15.40:0/58170369 wait complete. 2023-11-08T08:20:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.545+0000 7f48399e8700 1 Processor -- start 2023-11-08T08:20:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.545+0000 7f48399e8700 1 -- start start 2023-11-08T08:20:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48399e8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4834069bf0 0x7f48340b82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48399e8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4834070620 0x7f4834153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:36.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48399e8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f483415c150 0x7f48340c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:36.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48399e8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4834153d70 con 0x7f4834069bf0 2023-11-08T08:20:36.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48399e8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48340ccfa0 con 0x7f4834070620 2023-11-08T08:20:36.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48399e8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48340cf5b0 con 0x7f483415c150 2023-11-08T08:20:36.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48389e6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4834069bf0 0x7f48340b82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:36.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48389e6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4834069bf0 0x7f48340b82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34202/0 (socket says 172.21.15.40:34202) 2023-11-08T08:20:36.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48389e6700 1 -- 172.21.15.40:0/3934135202 learned_addr learned my addr 172.21.15.40:0/3934135202 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:36.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f4833fff700 1 --2- 172.21.15.40:0/3934135202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4834070620 0x7f4834153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:36.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48391e7700 1 --2- 172.21.15.40:0/3934135202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f483415c150 0x7f48340c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:36.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48389e6700 1 -- 172.21.15.40:0/3934135202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f483415c150 msgr2=0x7f48340c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:36.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48389e6700 1 --2- 172.21.15.40:0/3934135202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f483415c150 0x7f48340c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:36.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48389e6700 1 -- 172.21.15.40:0/3934135202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4834070620 msgr2=0x7f4834153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:36.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48389e6700 1 --2- 172.21.15.40:0/3934135202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4834070620 0x7f4834153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:36.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48389e6700 1 -- 172.21.15.40:0/3934135202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4820009510 con 0x7f4834069bf0 2023-11-08T08:20:36.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48389e6700 1 --2- 172.21.15.40:0/3934135202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4834069bf0 0x7f48340b82e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f482800c520 tx=0x7f482800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:36.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48317fa700 1 -- 172.21.15.40:0/3934135202 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48280054e0 con 0x7f4834069bf0 2023-11-08T08:20:36.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48317fa700 1 -- 172.21.15.40:0/3934135202 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48280036a0 con 0x7f4834069bf0 2023-11-08T08:20:36.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48399e8700 1 -- 172.21.15.40:0/3934135202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48340d05e0 con 0x7f4834069bf0 2023-11-08T08:20:36.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48317fa700 1 -- 172.21.15.40:0/3934135202 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4828003a50 con 0x7f4834069bf0 2023-11-08T08:20:36.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.549+0000 7f48399e8700 1 -- 172.21.15.40:0/3934135202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48340d2730 con 0x7f4834069bf0 2023-11-08T08:20:36.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.553+0000 7f48399e8700 1 -- 172.21.15.40:0/3934135202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48340d2140 con 0x7f4834069bf0 2023-11-08T08:20:36.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.553+0000 7f48317fa700 1 -- 172.21.15.40:0/3934135202 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4828019030 con 0x7f4834069bf0 2023-11-08T08:20:36.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.553+0000 7f48317fa700 1 --2- 172.21.15.40:0/3934135202 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4818064ee0 0x7f4818067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:36.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.557+0000 7f48317fa700 1 -- 172.21.15.40:0/3934135202 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f48280813f0 con 0x7f4834069bf0 2023-11-08T08:20:36.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.557+0000 7f4833fff700 1 --2- 172.21.15.40:0/3934135202 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4818064ee0 0x7f4818067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:36.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.557+0000 7f4833fff700 1 --2- 172.21.15.40:0/3934135202 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4818064ee0 0x7f4818067390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4834070a10 tx=0x7f4820008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:36.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.561+0000 7f48317fa700 1 -- 172.21.15.40:0/3934135202 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f482804f360 con 0x7f4834069bf0 2023-11-08T08:20:36.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:36.773+0000 7f48399e8700 1 -- 172.21.15.40:0/3934135202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4834073d20 con 0x7f4834069bf0 2023-11-08T08:20:37.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.321+0000 7f48317fa700 1 -- 172.21.15.40:0/3934135202 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4828052980 con 0x7f4834069bf0 2023-11-08T08:20:37.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.325+0000 7f48399e8700 1 -- 172.21.15.40:0/3934135202 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4818064ee0 msgr2=0x7f4818067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:37.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.325+0000 7f48399e8700 1 --2- 172.21.15.40:0/3934135202 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4818064ee0 0x7f4818067390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4834070a10 tx=0x7f4820008030 comp rx=0 tx=0).stop 2023-11-08T08:20:37.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.325+0000 7f48399e8700 1 -- 172.21.15.40:0/3934135202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4834069bf0 msgr2=0x7f48340b82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:37.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.325+0000 7f48399e8700 1 --2- 172.21.15.40:0/3934135202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4834069bf0 0x7f48340b82e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f482800c520 tx=0x7f482800c830 comp rx=0 tx=0).stop 2023-11-08T08:20:37.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.325+0000 7f48399e8700 1 -- 172.21.15.40:0/3934135202 shutdown_connections 2023-11-08T08:20:37.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.325+0000 7f48399e8700 1 --2- 172.21.15.40:0/3934135202 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4818064ee0 0x7f4818067390 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.325+0000 7f48399e8700 1 --2- 172.21.15.40:0/3934135202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f483415c150 0x7f48340c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.325+0000 7f48399e8700 1 --2- 172.21.15.40:0/3934135202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4834070620 0x7f4834153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.325+0000 7f48399e8700 1 --2- 172.21.15.40:0/3934135202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4834069bf0 0x7f48340b82e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.325+0000 7f48399e8700 1 -- 172.21.15.40:0/3934135202 >> 172.21.15.40:0/3934135202 conn(0x7f48340d3680 msgr2=0x7f4834159c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:37.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.325+0000 7f48399e8700 1 -- 172.21.15.40:0/3934135202 shutdown_connections 2023-11-08T08:20:37.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.325+0000 7f48399e8700 1 -- 172.21.15.40:0/3934135202 wait complete. 2023-11-08T08:20:37.356 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T08:20:37.357 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:20:37.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.545+0000 7f8c2e6dd700 1 Processor -- start 2023-11-08T08:20:37.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.545+0000 7f8c2e6dd700 1 -- start start 2023-11-08T08:20:37.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.545+0000 7f8c2e6dd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c28071aa0 0x7f8c2806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:37.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.545+0000 7f8c2e6dd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8c28069e00 0x7f8c2806a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:37.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.545+0000 7f8c2e6dd700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c2806c600 con 0x7f8c2815c640 2023-11-08T08:20:37.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.545+0000 7f8c2e6dd700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c2806cef0 con 0x7f8c28069e00 2023-11-08T08:20:37.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.545+0000 7f8c2e6dd700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c280d38f0 con 0x7f8c28071aa0 2023-11-08T08:20:37.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.545+0000 7f8c2d6db700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8c2815c640 0x7f8c280d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36738/0 (socket says 172.21.15.40:36738) 2023-11-08T08:20:37.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.545+0000 7f8c2d6db700 1 -- 172.21.15.40:0/87983728 learned_addr learned my addr 172.21.15.40:0/87983728 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:37.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.549+0000 7f8c2ceda700 1 --2- 172.21.15.40:0/87983728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c28071aa0 0x7f8c2806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:37.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.549+0000 7f8c2dedc700 1 --2- 172.21.15.40:0/87983728 >> v2:172.21.15.169:3300/0 conn(0x7f8c28069e00 0x7f8c2806a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:37.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.549+0000 7f8c1ffff700 1 -- 172.21.15.40:0/87983728 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1770504101 0 0) 0x7f8c2806c600 con 0x7f8c2815c640 2023-11-08T08:20:37.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.549+0000 7f8c1ffff700 1 -- 172.21.15.40:0/87983728 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c140035d0 con 0x7f8c2815c640 2023-11-08T08:20:37.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.549+0000 7f8c2ceda700 1 -- 172.21.15.40:0/87983728 >> v2:172.21.15.169:3300/0 conn(0x7f8c28069e00 msgr2=0x7f8c2806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:37.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.549+0000 7f8c2ceda700 1 --2- 172.21.15.40:0/87983728 >> v2:172.21.15.169:3300/0 conn(0x7f8c28069e00 0x7f8c2806a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.549+0000 7f8c2ceda700 1 -- 172.21.15.40:0/87983728 >> v1:172.21.15.40:6789/0 conn(0x7f8c2815c640 legacy=0x7f8c280d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:37.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.549+0000 7f8c2ceda700 1 -- 172.21.15.40:0/87983728 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c280722f0 con 0x7f8c28071aa0 2023-11-08T08:20:37.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.549+0000 7f8c2ceda700 1 --2- 172.21.15.40:0/87983728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c28071aa0 0x7f8c2806d310 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8c18009850 tx=0x7f8c18009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f7b2b7840d9acb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:37.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.549+0000 7f8c1ffff700 1 -- 172.21.15.40:0/87983728 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18011830 con 0x7f8c28071aa0 2023-11-08T08:20:37.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.549+0000 7f8c1ffff700 1 -- 172.21.15.40:0/87983728 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c18011990 con 0x7f8c28071aa0 2023-11-08T08:20:37.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.549+0000 7f8c1ffff700 1 -- 172.21.15.40:0/87983728 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18011d40 con 0x7f8c28071aa0 2023-11-08T08:20:37.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.549+0000 7f8c2e6dd700 1 -- 172.21.15.40:0/87983728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c28071aa0 msgr2=0x7f8c2806d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:37.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.549+0000 7f8c2e6dd700 1 --2- 172.21.15.40:0/87983728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c28071aa0 0x7f8c2806d310 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8c18009850 tx=0x7f8c18009b60 comp rx=0 tx=0).stop 2023-11-08T08:20:37.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.549+0000 7f8c2e6dd700 1 -- 172.21.15.40:0/87983728 shutdown_connections 2023-11-08T08:20:37.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.549+0000 7f8c2e6dd700 1 --2- 172.21.15.40:0/87983728 >> v2:172.21.15.169:3300/0 conn(0x7f8c28069e00 0x7f8c2806a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.549+0000 7f8c2e6dd700 1 --2- 172.21.15.40:0/87983728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c28071aa0 0x7f8c2806d310 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.549+0000 7f8c2e6dd700 1 -- 172.21.15.40:0/87983728 >> 172.21.15.40:0/87983728 conn(0x7f8c280b6520 msgr2=0x7f8c28071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:37.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.553+0000 7f8c2e6dd700 1 -- 172.21.15.40:0/87983728 shutdown_connections 2023-11-08T08:20:37.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.553+0000 7f8c2e6dd700 1 -- 172.21.15.40:0/87983728 wait complete. 2023-11-08T08:20:37.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.553+0000 7f8c2e6dd700 1 Processor -- start 2023-11-08T08:20:37.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.553+0000 7f8c2e6dd700 1 -- start start 2023-11-08T08:20:37.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c2e6dd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c28069e00 0x7f8c280c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:37.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c2e6dd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c28071aa0 0x7f8c280c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:37.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c2e6dd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c2815c640 0x7f8c2815e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:37.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c2e6dd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2815e390 con 0x7f8c2815c640 2023-11-08T08:20:37.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c2e6dd700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2815dac0 con 0x7f8c28071aa0 2023-11-08T08:20:37.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c2e6dd700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2815cd20 con 0x7f8c28069e00 2023-11-08T08:20:37.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c2dedc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c2815c640 0x7f8c2815e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:37.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c2dedc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c2815c640 0x7f8c2815e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48728/0 (socket says 172.21.15.40:48728) 2023-11-08T08:20:37.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c2dedc700 1 -- 172.21.15.40:0/1357530775 learned_addr learned my addr 172.21.15.40:0/1357530775 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:37.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c2d6db700 1 --2- 172.21.15.40:0/1357530775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c28069e00 0x7f8c280c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:37.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c2ceda700 1 --2- 172.21.15.40:0/1357530775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c28071aa0 0x7f8c280c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:37.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c2dedc700 1 -- 172.21.15.40:0/1357530775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c28069e00 msgr2=0x7f8c280c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:37.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c2dedc700 1 --2- 172.21.15.40:0/1357530775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c28069e00 0x7f8c280c6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c2dedc700 1 -- 172.21.15.40:0/1357530775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c28071aa0 msgr2=0x7f8c280c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:37.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c2dedc700 1 --2- 172.21.15.40:0/1357530775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c28071aa0 0x7f8c280c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c2dedc700 1 -- 172.21.15.40:0/1357530775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c180094b0 con 0x7f8c2815c640 2023-11-08T08:20:37.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c2d6db700 1 --2- 172.21.15.40:0/1357530775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c28069e00 0x7f8c280c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:37.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c2ceda700 1 --2- 172.21.15.40:0/1357530775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c28071aa0 0x7f8c280c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:37.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c2dedc700 1 --2- 172.21.15.40:0/1357530775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c2815c640 0x7f8c2815e7b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000efc0 tx=0x7f8c2000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:37.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c1dffb700 1 -- 172.21.15.40:0/1357530775 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20009ba0 con 0x7f8c2815c640 2023-11-08T08:20:37.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c1dffb700 1 -- 172.21.15.40:0/1357530775 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2000f040 con 0x7f8c2815c640 2023-11-08T08:20:37.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c1dffb700 1 -- 172.21.15.40:0/1357530775 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20014480 con 0x7f8c2815c640 2023-11-08T08:20:37.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.557+0000 7f8c2e6dd700 1 -- 172.21.15.40:0/1357530775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c28072010 con 0x7f8c2815c640 2023-11-08T08:20:37.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.561+0000 7f8c2e6dd700 1 -- 172.21.15.40:0/1357530775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2815ca10 con 0x7f8c2815c640 2023-11-08T08:20:37.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.561+0000 7f8c2e6dd700 1 -- 172.21.15.40:0/1357530775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c2806c830 con 0x7f8c2815c640 2023-11-08T08:20:37.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.561+0000 7f8c1dffb700 1 -- 172.21.15.40:0/1357530775 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f8c200145e0 con 0x7f8c2815c640 2023-11-08T08:20:37.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.565+0000 7f8c1dffb700 1 --2- 172.21.15.40:0/1357530775 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8c0c064ee0 0x7f8c0c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:37.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.565+0000 7f8c1dffb700 1 -- 172.21.15.40:0/1357530775 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8c20082ab0 con 0x7f8c2815c640 2023-11-08T08:20:37.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.565+0000 7f8c2d6db700 1 --2- 172.21.15.40:0/1357530775 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8c0c064ee0 0x7f8c0c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:37.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.565+0000 7f8c2d6db700 1 --2- 172.21.15.40:0/1357530775 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8c0c064ee0 0x7f8c0c067390 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8c240011e0 tx=0x7f8c24009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:37.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.569+0000 7f8c1dffb700 1 -- 172.21.15.40:0/1357530775 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f8c20050a20 con 0x7f8c2815c640 2023-11-08T08:20:37.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.781+0000 7f8c2e6dd700 1 -- 172.21.15.40:0/1357530775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c2815e4d0 con 0x7f8c2815c640 2023-11-08T08:20:37.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.781+0000 7f8c1dffb700 1 -- 172.21.15.40:0/1357530775 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f8c2815e4d0 con 0x7f8c2815c640 2023-11-08T08:20:37.788 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:37.788 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4958,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2244078655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2244078655},{"type":"v1","addr":"172.21.15.40:6834","nonce":2244078655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/749537066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":749537066},{"type":"v1","addr":"172.21.15.169:6836","nonce":749537066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5328,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2307838686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2307838686},{"type":"v1","addr":"172.21.15.169:6837","nonce":2307838686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:20:34.294628+0000","modified":"2023-11-08T08:20:37.318141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/1791517264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1791517264},{"type":"v1","addr":"172.21.15.40:6835","nonce":1791517264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T08:20:37.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.785+0000 7f8c2e6dd700 1 -- 172.21.15.40:0/1357530775 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8c0c064ee0 msgr2=0x7f8c0c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:37.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.785+0000 7f8c2e6dd700 1 --2- 172.21.15.40:0/1357530775 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8c0c064ee0 0x7f8c0c067390 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8c240011e0 tx=0x7f8c24009040 comp rx=0 tx=0).stop 2023-11-08T08:20:37.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.785+0000 7f8c2e6dd700 1 -- 172.21.15.40:0/1357530775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c2815c640 msgr2=0x7f8c2815e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:37.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.785+0000 7f8c2e6dd700 1 --2- 172.21.15.40:0/1357530775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c2815c640 0x7f8c2815e7b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000efc0 tx=0x7f8c2000c3a0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.785+0000 7f8c2e6dd700 1 -- 172.21.15.40:0/1357530775 shutdown_connections 2023-11-08T08:20:37.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.785+0000 7f8c2e6dd700 1 --2- 172.21.15.40:0/1357530775 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8c0c064ee0 0x7f8c0c067390 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.785+0000 7f8c2e6dd700 1 --2- 172.21.15.40:0/1357530775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c2815c640 0x7f8c2815e7b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.785+0000 7f8c2e6dd700 1 --2- 172.21.15.40:0/1357530775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c28071aa0 0x7f8c280c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.785+0000 7f8c2e6dd700 1 --2- 172.21.15.40:0/1357530775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c28069e00 0x7f8c280c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.785+0000 7f8c2e6dd700 1 -- 172.21.15.40:0/1357530775 >> 172.21.15.40:0/1357530775 conn(0x7f8c280b6520 msgr2=0x7f8c280d1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:37.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.785+0000 7f8c2e6dd700 1 -- 172.21.15.40:0/1357530775 shutdown_connections 2023-11-08T08:20:37.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.785+0000 7f8c2e6dd700 1 -- 172.21.15.40:0/1357530775 wait complete. 2023-11-08T08:20:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 17 2023-11-08T08:20:37.814 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:20:37.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.977+0000 7fc049f90700 1 Processor -- start 2023-11-08T08:20:37.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.977+0000 7fc049f90700 1 -- start start 2023-11-08T08:20:37.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.981+0000 7fc049f90700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc044160710 0x7fc0440cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:37.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.981+0000 7fc049f90700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0440cc860 0x7fc044070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:37.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.981+0000 7fc049f90700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc0440705c0 0x7fc044069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:37.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.981+0000 7fc049f90700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc044070a60 con 0x7fc044160710 2023-11-08T08:20:37.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.981+0000 7fc049f90700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc04406b1c0 con 0x7fc0440705c0 2023-11-08T08:20:37.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.981+0000 7fc049f90700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440ad6c0 con 0x7fc0440cc860 2023-11-08T08:20:37.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.981+0000 7fc048f8e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc044160710 0x7fc0440cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:37.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.981+0000 7fc048f8e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc044160710 0x7fc0440cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48734/0 (socket says 172.21.15.40:48734) 2023-11-08T08:20:37.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.981+0000 7fc048f8e700 1 -- 172.21.15.40:0/2288648994 learned_addr learned my addr 172.21.15.40:0/2288648994 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:37.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.981+0000 7fc03bfff700 1 --2- 172.21.15.40:0/2288648994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0440cc860 0x7fc044070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:37.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.981+0000 7fc04978f700 1 --2- 172.21.15.40:0/2288648994 >> v2:172.21.15.169:3300/0 conn(0x7fc0440705c0 0x7fc044069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:37.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.981+0000 7fc048f8e700 1 -- 172.21.15.40:0/2288648994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0440cc860 msgr2=0x7fc044070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:37.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.981+0000 7fc048f8e700 1 --2- 172.21.15.40:0/2288648994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0440cc860 0x7fc044070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.981+0000 7fc048f8e700 1 -- 172.21.15.40:0/2288648994 >> v2:172.21.15.169:3300/0 conn(0x7fc0440705c0 msgr2=0x7fc044069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:37.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.981+0000 7fc048f8e700 1 --2- 172.21.15.40:0/2288648994 >> v2:172.21.15.169:3300/0 conn(0x7fc0440705c0 0x7fc044069bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.981+0000 7fc048f8e700 1 -- 172.21.15.40:0/2288648994 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0440adc60 con 0x7fc044160710 2023-11-08T08:20:37.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.981+0000 7fc048f8e700 1 --2- 172.21.15.40:0/2288648994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc044160710 0x7fc0440cc320 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc0440b4900 tx=0x7fc040009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54874769a8da89b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:37.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.981+0000 7fc03b7fe700 1 -- 172.21.15.40:0/2288648994 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc040011830 con 0x7fc044160710 2023-11-08T08:20:37.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.981+0000 7fc03b7fe700 1 -- 172.21.15.40:0/2288648994 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc040011990 con 0x7fc044160710 2023-11-08T08:20:37.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.981+0000 7fc03b7fe700 1 -- 172.21.15.40:0/2288648994 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc040011d20 con 0x7fc044160710 2023-11-08T08:20:37.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.985+0000 7fc049f90700 1 -- 172.21.15.40:0/2288648994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc044160710 msgr2=0x7fc0440cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:37.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.985+0000 7fc049f90700 1 --2- 172.21.15.40:0/2288648994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc044160710 0x7fc0440cc320 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc0440b4900 tx=0x7fc040009bf0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.985+0000 7fc049f90700 1 -- 172.21.15.40:0/2288648994 shutdown_connections 2023-11-08T08:20:37.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.985+0000 7fc049f90700 1 --2- 172.21.15.40:0/2288648994 >> v2:172.21.15.169:3300/0 conn(0x7fc0440705c0 0x7fc044069bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.985+0000 7fc049f90700 1 --2- 172.21.15.40:0/2288648994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0440cc860 0x7fc044070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.985+0000 7fc049f90700 1 --2- 172.21.15.40:0/2288648994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc044160710 0x7fc0440cc320 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.985+0000 7fc049f90700 1 -- 172.21.15.40:0/2288648994 >> 172.21.15.40:0/2288648994 conn(0x7fc0440b0f80 msgr2=0x7fc0440ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:37.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.985+0000 7fc049f90700 1 -- 172.21.15.40:0/2288648994 shutdown_connections 2023-11-08T08:20:37.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.985+0000 7fc049f90700 1 -- 172.21.15.40:0/2288648994 wait complete. 2023-11-08T08:20:37.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.989+0000 7fc049f90700 1 Processor -- start 2023-11-08T08:20:37.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.989+0000 7fc049f90700 1 -- start start 2023-11-08T08:20:37.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.989+0000 7fc049f90700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0440705c0 0x7fc0441f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:37.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.989+0000 7fc049f90700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0440cc860 0x7fc0440c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:37.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.989+0000 7fc049f90700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc044160710 0x7fc04415b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:37.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.989+0000 7fc049f90700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440c1830 con 0x7fc0440cc860 2023-11-08T08:20:37.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.989+0000 7fc049f90700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04415d360 con 0x7fc0440705c0 2023-11-08T08:20:37.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.989+0000 7fc049f90700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440c1a80 con 0x7fc044160710 2023-11-08T08:20:37.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.989+0000 7fc03bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0440cc860 0x7fc0440c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:37.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.989+0000 7fc03bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0440cc860 0x7fc0440c43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48748/0 (socket says 172.21.15.40:48748) 2023-11-08T08:20:37.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.989+0000 7fc03bfff700 1 -- 172.21.15.40:0/4228560101 learned_addr learned my addr 172.21.15.40:0/4228560101 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:37.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.989+0000 7fc04978f700 1 --2- 172.21.15.40:0/4228560101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc044160710 0x7fc04415b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:37.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.989+0000 7fc048f8e700 1 --2- 172.21.15.40:0/4228560101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0440705c0 0x7fc0441f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:37.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.989+0000 7fc03bfff700 1 -- 172.21.15.40:0/4228560101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc044160710 msgr2=0x7fc04415b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:37.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.989+0000 7fc03bfff700 1 --2- 172.21.15.40:0/4228560101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc044160710 0x7fc04415b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.989+0000 7fc03bfff700 1 -- 172.21.15.40:0/4228560101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0440705c0 msgr2=0x7fc0441f5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:37.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.989+0000 7fc03bfff700 1 --2- 172.21.15.40:0/4228560101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0440705c0 0x7fc0441f5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:37.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.989+0000 7fc03bfff700 1 -- 172.21.15.40:0/4228560101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc040009570 con 0x7fc0440cc860 2023-11-08T08:20:37.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.989+0000 7fc04978f700 1 --2- 172.21.15.40:0/4228560101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc044160710 0x7fc04415b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:37.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.993+0000 7fc03bfff700 1 --2- 172.21.15.40:0/4228560101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0440cc860 0x7fc0440c43b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc03400ebe0 tx=0x7fc034009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:37.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.993+0000 7fc0397fa700 1 -- 172.21.15.40:0/4228560101 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034009740 con 0x7fc0440cc860 2023-11-08T08:20:38.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.993+0000 7fc0397fa700 1 -- 172.21.15.40:0/4228560101 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03400c8b0 con 0x7fc0440cc860 2023-11-08T08:20:38.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.993+0000 7fc049f90700 1 -- 172.21.15.40:0/4228560101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04406b6b0 con 0x7fc0440cc860 2023-11-08T08:20:38.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.993+0000 7fc0397fa700 1 -- 172.21.15.40:0/4228560101 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03400cc40 con 0x7fc0440cc860 2023-11-08T08:20:38.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.993+0000 7fc049f90700 1 -- 172.21.15.40:0/4228560101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04406b3d0 con 0x7fc0440cc860 2023-11-08T08:20:38.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.993+0000 7fc049f90700 1 -- 172.21.15.40:0/4228560101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0440ad8f0 con 0x7fc0440cc860 2023-11-08T08:20:38.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.997+0000 7fc0397fa700 1 -- 172.21.15.40:0/4228560101 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc034019030 con 0x7fc0440cc860 2023-11-08T08:20:38.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.997+0000 7fc0397fa700 1 --2- 172.21.15.40:0/4228560101 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc030064ee0 0x7fc030067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:38.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.997+0000 7fc0397fa700 1 -- 172.21.15.40:0/4228560101 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc0340820d0 con 0x7fc0440cc860 2023-11-08T08:20:38.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:37.997+0000 7fc048f8e700 1 --2- 172.21.15.40:0/4228560101 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc030064ee0 0x7fc030067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:38.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.001+0000 7fc048f8e700 1 --2- 172.21.15.40:0/4228560101 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc030064ee0 0x7fc030067390 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc040014040 tx=0x7fc040008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:38.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.001+0000 7fc0397fa700 1 -- 172.21.15.40:0/4228560101 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc034050040 con 0x7fc0440cc860 2023-11-08T08:20:38.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.177+0000 7fc049f90700 1 -- 172.21.15.40:0/4228560101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc044073cb0 con 0x7fc0440cc860 2023-11-08T08:20:38.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.181+0000 7fc0397fa700 1 -- 172.21.15.40:0/4228560101 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+18032 (secure 0 0 0) 0x7fc034053660 con 0x7fc0440cc860 2023-11-08T08:20:38.187 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:38.187 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":43,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:20:35.300958+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:20:32.059150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-08T08:20:33.297064+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:20:38.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.185+0000 7fc049f90700 1 -- 172.21.15.40:0/4228560101 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc030064ee0 msgr2=0x7fc030067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:38.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.185+0000 7fc049f90700 1 --2- 172.21.15.40:0/4228560101 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc030064ee0 0x7fc030067390 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc040014040 tx=0x7fc040008030 comp rx=0 tx=0).stop 2023-11-08T08:20:38.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.185+0000 7fc049f90700 1 -- 172.21.15.40:0/4228560101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0440cc860 msgr2=0x7fc0440c43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:38.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.185+0000 7fc049f90700 1 --2- 172.21.15.40:0/4228560101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0440cc860 0x7fc0440c43b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc03400ebe0 tx=0x7fc034009e00 comp rx=0 tx=0).stop 2023-11-08T08:20:38.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.185+0000 7fc049f90700 1 -- 172.21.15.40:0/4228560101 shutdown_connections 2023-11-08T08:20:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.185+0000 7fc049f90700 1 --2- 172.21.15.40:0/4228560101 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc030064ee0 0x7fc030067390 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.185+0000 7fc049f90700 1 --2- 172.21.15.40:0/4228560101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc044160710 0x7fc04415b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.185+0000 7fc049f90700 1 --2- 172.21.15.40:0/4228560101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0440cc860 0x7fc0440c43b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.185+0000 7fc049f90700 1 --2- 172.21.15.40:0/4228560101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0440705c0 0x7fc0441f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.185+0000 7fc049f90700 1 -- 172.21.15.40:0/4228560101 >> 172.21.15.40:0/4228560101 conn(0x7fc0440b0f80 msgr2=0x7fc044160ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:38.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.185+0000 7fc049f90700 1 -- 172.21.15.40:0/4228560101 shutdown_connections 2023-11-08T08:20:38.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.185+0000 7fc049f90700 1 -- 172.21.15.40:0/4228560101 wait complete. 2023-11-08T08:20:38.212 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T08:20:38.213 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T08:20:38.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.389+0000 7f7a0635d700 1 Processor -- start 2023-11-08T08:20:38.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.389+0000 7f7a0635d700 1 -- start start 2023-11-08T08:20:38.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f7a0635d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00070790 0x7f7a00164fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:38.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f7a0635d700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a0015a550 con 0x7f7a000b43a0 2023-11-08T08:20:38.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f7a0635d700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a0006b0a0 con 0x7f7a000b1640 2023-11-08T08:20:38.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f7a0635d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a000caac0 con 0x7f7a00070790 2023-11-08T08:20:38.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f7a0535b700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7a000b43a0 0x7f7a0015e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36754/0 (socket says 172.21.15.40:36754) 2023-11-08T08:20:38.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f7a0535b700 1 -- 172.21.15.40:0/2064884920 learned_addr learned my addr 172.21.15.40:0/2064884920 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:38.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f7a05b5c700 1 --2- 172.21.15.40:0/2064884920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00070790 0x7f7a00164fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:38.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f79f7fff700 1 -- 172.21.15.40:0/2064884920 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3046227038 0 0) 0x7f79fc001110 con 0x7f7a000b43a0 2023-11-08T08:20:38.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f79f7fff700 1 -- 172.21.15.40:0/2064884920 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79ec0035d0 con 0x7f7a000b43a0 2023-11-08T08:20:38.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f79f7fff700 1 -- 172.21.15.40:0/2064884920 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648875832 0 0) 0x7f7a0006b0a0 con 0x7f7a000b1640 2023-11-08T08:20:38.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f79f7fff700 1 -- 172.21.15.40:0/2064884920 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79fc001110 con 0x7f7a000b1640 2023-11-08T08:20:38.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f79f7fff700 1 -- 172.21.15.40:0/2064884920 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4025380954 0 0) 0x7f79ec0035d0 con 0x7f7a000b43a0 2023-11-08T08:20:38.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f79f7fff700 1 -- 172.21.15.40:0/2064884920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00070790 msgr2=0x7f7a00164fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:38.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f79f7fff700 1 --2- 172.21.15.40:0/2064884920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00070790 0x7f7a00164fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:38.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f79f7fff700 1 -- 172.21.15.40:0/2064884920 >> v1:172.21.15.169:6789/0 conn(0x7f7a000b1640 legacy=0x7f7a00070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:38.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f79f7fff700 1 -- 172.21.15.40:0/2064884920 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a000cada0 con 0x7f7a000b43a0 2023-11-08T08:20:38.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f79f7fff700 1 -- 172.21.15.40:0/2064884920 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f79fc001800 con 0x7f7a000b43a0 2023-11-08T08:20:38.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f79f7fff700 1 -- 172.21.15.40:0/2064884920 >> v1:172.21.15.40:6789/0 conn(0x7f7a000b43a0 legacy=0x7f7a0015e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:38.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f79f7fff700 1 --2- 172.21.15.40:0/2064884920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec003be0 0x7f79ec003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:38.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f79f7fff700 1 --2- 172.21.15.40:0/2064884920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ec004870 0x7f79ec004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:38.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.393+0000 7f79f7fff700 1 --2- 172.21.15.40:0/2064884920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00070790 0x7f79ec005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:38.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.397+0000 7f79f7fff700 1 -- 172.21.15.40:0/2064884920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a000caac0 con 0x7f79ec004870 2023-11-08T08:20:38.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.397+0000 7f79f7fff700 1 -- 172.21.15.40:0/2064884920 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec00ff50 con 0x7f79ec003be0 2023-11-08T08:20:38.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.397+0000 7f79f7fff700 1 -- 172.21.15.40:0/2064884920 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec010110 con 0x7f7a00070790 2023-11-08T08:20:38.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.397+0000 7f7a04b5a700 1 --2- 172.21.15.40:0/2064884920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ec004870 0x7f79ec004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:38.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.397+0000 7f7a05b5c700 1 --2- 172.21.15.40:0/2064884920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00070790 0x7f79ec005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:38.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.397+0000 7f7a0535b700 1 --2- 172.21.15.40:0/2064884920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec003be0 0x7f79ec003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:38.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.397+0000 7f7a04b5a700 1 -- 172.21.15.40:0/2064884920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00070790 msgr2=0x7f79ec005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:38.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.397+0000 7f7a04b5a700 1 --2- 172.21.15.40:0/2064884920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00070790 0x7f79ec005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:38.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.397+0000 7f7a04b5a700 1 -- 172.21.15.40:0/2064884920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec003be0 msgr2=0x7f79ec003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:38.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.397+0000 7f7a04b5a700 1 --2- 172.21.15.40:0/2064884920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec003be0 0x7f79ec003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:38.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.397+0000 7f7a04b5a700 1 -- 172.21.15.40:0/2064884920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec0035d0 con 0x7f79ec004870 2023-11-08T08:20:38.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.397+0000 7f7a04b5a700 1 --2- 172.21.15.40:0/2064884920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ec004870 0x7f79ec004ce0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f79f0001960 tx=0x7f79ec0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd149f2ab59dd41 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:38.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.397+0000 7f79f7fff700 1 -- 172.21.15.40:0/2064884920 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0001450 con 0x7f79ec004870 2023-11-08T08:20:38.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.397+0000 7f79f7fff700 1 -- 172.21.15.40:0/2064884920 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f00052f0 con 0x7f79ec004870 2023-11-08T08:20:38.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.397+0000 7f79f7fff700 1 -- 172.21.15.40:0/2064884920 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f00036a0 con 0x7f79ec004870 2023-11-08T08:20:38.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.397+0000 7f7a0635d700 1 -- 172.21.15.40:0/2064884920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ec004870 msgr2=0x7f79ec004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:38.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.397+0000 7f7a0635d700 1 --2- 172.21.15.40:0/2064884920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ec004870 0x7f79ec004ce0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f79f0001960 tx=0x7f79ec0058a0 comp rx=0 tx=0).stop 2023-11-08T08:20:38.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.397+0000 7f7a05b5c700 1 -- 172.21.15.40:0/2064884920 reap_dead start 2023-11-08T08:20:38.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.397+0000 7f7a0635d700 1 -- 172.21.15.40:0/2064884920 shutdown_connections 2023-11-08T08:20:38.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.397+0000 7f7a0635d700 1 -- 172.21.15.40:0/2064884920 >> 172.21.15.40:0/2064884920 conn(0x7f7a000b0f80 msgr2=0x7f7a000ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:38.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.401+0000 7f7a0635d700 1 -- 172.21.15.40:0/2064884920 shutdown_connections 2023-11-08T08:20:38.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.401+0000 7f7a0635d700 1 -- 172.21.15.40:0/2064884920 wait complete. 2023-11-08T08:20:38.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.401+0000 7f7a0635d700 1 Processor -- start 2023-11-08T08:20:38.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.401+0000 7f7a0635d700 1 -- start start 2023-11-08T08:20:38.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a0635d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec004870 0x7f7a001589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:38.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a0635d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a000c4f00 0x7f7a00153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:38.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a0635d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a001f08a0 0x7f7a001f0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:38.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a0635d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a000c5370 con 0x7f7a000c4f00 2023-11-08T08:20:38.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a0635d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a001f1280 con 0x7f79ec004870 2023-11-08T08:20:38.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a0635d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a00153ec0 con 0x7f7a001f08a0 2023-11-08T08:20:38.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a04b5a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a000c4f00 0x7f7a00153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:38.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a04b5a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a000c4f00 0x7f7a00153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48758/0 (socket says 172.21.15.40:48758) 2023-11-08T08:20:38.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a04b5a700 1 -- 172.21.15.40:0/3733686967 learned_addr learned my addr 172.21.15.40:0/3733686967 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:38.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a0535b700 1 --2- 172.21.15.40:0/3733686967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec004870 0x7f7a001589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:38.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a05b5c700 1 --2- 172.21.15.40:0/3733686967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a001f08a0 0x7f7a001f0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:38.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a04b5a700 1 -- 172.21.15.40:0/3733686967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a001f08a0 msgr2=0x7f7a001f0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:38.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a04b5a700 1 --2- 172.21.15.40:0/3733686967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a001f08a0 0x7f7a001f0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:38.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a04b5a700 1 -- 172.21.15.40:0/3733686967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec004870 msgr2=0x7f7a001589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:38.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a04b5a700 1 --2- 172.21.15.40:0/3733686967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec004870 0x7f7a001589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:38.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a04b5a700 1 -- 172.21.15.40:0/3733686967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0006b5d0 con 0x7f7a000c4f00 2023-11-08T08:20:38.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a05b5c700 1 --2- 172.21.15.40:0/3733686967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a001f08a0 0x7f7a001f0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:38.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a0535b700 1 --2- 172.21.15.40:0/3733686967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec004870 0x7f7a001589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:38.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a04b5a700 1 --2- 172.21.15.40:0/3733686967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a000c4f00 0x7f7a00153980 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f79f000c3c0 tx=0x7f79f00012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:38.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f79f5ffb700 1 -- 172.21.15.40:0/3733686967 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0006c90 con 0x7f7a000c4f00 2023-11-08T08:20:38.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f79f5ffb700 1 -- 172.21.15.40:0/3733686967 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f0006df0 con 0x7f7a000c4f00 2023-11-08T08:20:38.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f79f5ffb700 1 -- 172.21.15.40:0/3733686967 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0007240 con 0x7f7a000c4f00 2023-11-08T08:20:38.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a0635d700 1 -- 172.21.15.40:0/3733686967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0006b2f0 con 0x7f7a000c4f00 2023-11-08T08:20:38.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.405+0000 7f7a0635d700 1 -- 172.21.15.40:0/3733686967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a000ca7b0 con 0x7f7a000c4f00 2023-11-08T08:20:38.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.409+0000 7f7a0635d700 1 -- 172.21.15.40:0/3733686967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a00073cb0 con 0x7f7a000c4f00 2023-11-08T08:20:38.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.409+0000 7f79f5ffb700 1 -- 172.21.15.40:0/3733686967 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f79f002d070 con 0x7f7a000c4f00 2023-11-08T08:20:38.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.409+0000 7f79f5ffb700 1 --2- 172.21.15.40:0/3733686967 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f79e4064f10 0x7f79e40673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:38.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.413+0000 7f79f5ffb700 1 -- 172.21.15.40:0/3733686967 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f79f0082e00 con 0x7f7a000c4f00 2023-11-08T08:20:38.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.413+0000 7f7a0535b700 1 --2- 172.21.15.40:0/3733686967 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f79e4064f10 0x7f79e40673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:38.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.413+0000 7f7a0535b700 1 --2- 172.21.15.40:0/3733686967 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f79e4064f10 0x7f79e40673c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f79fc001800 tx=0x7f79fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:38.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.417+0000 7f79f5ffb700 1 -- 172.21.15.40:0/3733686967 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f79f0050d70 con 0x7f7a000c4f00 2023-11-08T08:20:38.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.585+0000 7f7a0635d700 1 -- 172.21.15.40:0/3733686967 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7a00071090 con 0x7f79e4064f10 2023-11-08T08:20:38.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.609+0000 7f79f5ffb700 1 -- 172.21.15.40:0/3733686967 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309006 (secure 0 0 0) 0x7f7a00071090 con 0x7f79e4064f10 2023-11-08T08:20:38.618 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:38.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.617+0000 7f7a0635d700 1 -- 172.21.15.40:0/3733686967 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f79e4064f10 msgr2=0x7f79e40673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:38.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.617+0000 7f7a0635d700 1 --2- 172.21.15.40:0/3733686967 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f79e4064f10 0x7f79e40673c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f79fc001800 tx=0x7f79fc009040 comp rx=0 tx=0).stop 2023-11-08T08:20:38.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.617+0000 7f7a0635d700 1 -- 172.21.15.40:0/3733686967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a000c4f00 msgr2=0x7f7a00153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:38.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.617+0000 7f7a0635d700 1 --2- 172.21.15.40:0/3733686967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a000c4f00 0x7f7a00153980 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f79f000c3c0 tx=0x7f79f00012d0 comp rx=0 tx=0).stop 2023-11-08T08:20:38.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.617+0000 7f7a0635d700 1 -- 172.21.15.40:0/3733686967 shutdown_connections 2023-11-08T08:20:38.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.617+0000 7f7a0635d700 1 --2- 172.21.15.40:0/3733686967 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f79e4064f10 0x7f79e40673c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:38.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.617+0000 7f7a0635d700 1 --2- 172.21.15.40:0/3733686967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a001f08a0 0x7f7a001f0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:38.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.617+0000 7f7a0635d700 1 --2- 172.21.15.40:0/3733686967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a000c4f00 0x7f7a00153980 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:38.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.617+0000 7f7a0635d700 1 --2- 172.21.15.40:0/3733686967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec004870 0x7f7a001589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:38.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.617+0000 7f7a0635d700 1 -- 172.21.15.40:0/3733686967 >> 172.21.15.40:0/3733686967 conn(0x7f7a000b0f80 msgr2=0x7f7a000af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:38.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.617+0000 7f7a0635d700 1 -- 172.21.15.40:0/3733686967 shutdown_connections 2023-11-08T08:20:38.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.617+0000 7f7a0635d700 1 -- 172.21.15.40:0/3733686967 wait complete. 2023-11-08T08:20:38.626 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-11-08T08:20:38.647 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2023-11-08T08:20:37.916687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":71936,"kb_used_data":5056,"kb_used_omap":0,"kb_used_meta":66816,"kb_avail":754902784,"statfs":{"total":773094113280,"available":773020450816,"internally_reserved":0,"allocated":5177344,"data_stored":2931134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68419584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.108352"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.341942+0000","last_change":"2023-11-08T08:20:34.330057+0000","last_active":"2023-11-08T08:20:34.341942+0000","last_peered":"2023-11-08T08:20:34.341942+0000","last_clean":"2023-11-08T08:20:34.341942+0000","last_became_active":"2023-11-08T08:20:34.325719+0000","last_became_peered":"2023-11-08T08:20:34.325719+0000","last_unstale":"2023-11-08T08:20:34.341942+0000","last_undegraded":"2023-11-08T08:20:34.341942+0000","last_fullsized":"2023-11-08T08:20:34.341942+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:56:45.933503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.337783+0000","last_change":"2023-11-08T08:20:34.328948+0000","last_active":"2023-11-08T08:20:34.337783+0000","last_peered":"2023-11-08T08:20:34.337783+0000","last_clean":"2023-11-08T08:20:34.337783+0000","last_became_active":"2023-11-08T08:20:34.328182+0000","last_became_peered":"2023-11-08T08:20:34.328182+0000","last_unstale":"2023-11-08T08:20:34.337783+0000","last_undegraded":"2023-11-08T08:20:34.337783+0000","last_fullsized":"2023-11-08T08:20:34.337783+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:20:27.726248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447461+0000","last_change":"2023-11-08T08:20:34.330263+0000","last_active":"2023-11-08T08:20:35.447461+0000","last_peered":"2023-11-08T08:20:35.447461+0000","last_clean":"2023-11-08T08:20:35.447461+0000","last_became_active":"2023-11-08T08:20:34.328048+0000","last_became_peered":"2023-11-08T08:20:34.328048+0000","last_unstale":"2023-11-08T08:20:35.447461+0000","last_undegraded":"2023-11-08T08:20:35.447461+0000","last_fullsized":"2023-11-08T08:20:35.447461+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:19:44.058497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319582+0000","last_change":"2023-11-08T08:20:34.332656+0000","last_active":"2023-11-08T08:20:35.319582+0000","last_peered":"2023-11-08T08:20:35.319582+0000","last_clean":"2023-11-08T08:20:35.319582+0000","last_became_active":"2023-11-08T08:20:34.332270+0000","last_became_peered":"2023-11-08T08:20:34.332270+0000","last_unstale":"2023-11-08T08:20:35.319582+0000","last_undegraded":"2023-11-08T08:20:35.319582+0000","last_fullsized":"2023-11-08T08:20:35.319582+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:39:57.195619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.331308+0000","last_change":"2023-11-08T08:20:34.323639+0000","last_active":"2023-11-08T08:20:34.331308+0000","last_peered":"2023-11-08T08:20:34.331308+0000","last_clean":"2023-11-08T08:20:34.331308+0000","last_became_active":"2023-11-08T08:20:34.323369+0000","last_became_peered":"2023-11-08T08:20:34.323369+0000","last_unstale":"2023-11-08T08:20:34.331308+0000","last_undegraded":"2023-11-08T08:20:34.331308+0000","last_fullsized":"2023-11-08T08:20:34.331308+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:09:07.316196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439468+0000","last_change":"2023-11-08T08:20:34.349152+0000","last_active":"2023-11-08T08:20:35.439468+0000","last_peered":"2023-11-08T08:20:35.439468+0000","last_clean":"2023-11-08T08:20:35.439468+0000","last_became_active":"2023-11-08T08:20:34.348378+0000","last_became_peered":"2023-11-08T08:20:34.348378+0000","last_unstale":"2023-11-08T08:20:35.439468+0000","last_undegraded":"2023-11-08T08:20:35.439468+0000","last_fullsized":"2023-11-08T08:20:35.439468+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:32:02.079767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.341074+0000","last_change":"2023-11-08T08:20:34.332470+0000","last_active":"2023-11-08T08:20:34.341074+0000","last_peered":"2023-11-08T08:20:34.341074+0000","last_clean":"2023-11-08T08:20:34.341074+0000","last_became_active":"2023-11-08T08:20:34.332186+0000","last_became_peered":"2023-11-08T08:20:34.332186+0000","last_unstale":"2023-11-08T08:20:34.341074+0000","last_undegraded":"2023-11-08T08:20:34.341074+0000","last_fullsized":"2023-11-08T08:20:34.341074+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:43:49.374493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.318943+0000","last_change":"2023-11-08T08:20:34.334000+0000","last_active":"2023-11-08T08:20:35.318943+0000","last_peered":"2023-11-08T08:20:35.318943+0000","last_clean":"2023-11-08T08:20:35.318943+0000","last_became_active":"2023-11-08T08:20:34.333735+0000","last_became_peered":"2023-11-08T08:20:34.333735+0000","last_unstale":"2023-11-08T08:20:35.318943+0000","last_undegraded":"2023-11-08T08:20:35.318943+0000","last_fullsized":"2023-11-08T08:20:35.318943+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:24:12.113358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319969+0000","last_change":"2023-11-08T08:20:34.340403+0000","last_active":"2023-11-08T08:20:35.319969+0000","last_peered":"2023-11-08T08:20:35.319969+0000","last_clean":"2023-11-08T08:20:35.319969+0000","last_became_active":"2023-11-08T08:20:34.339070+0000","last_became_peered":"2023-11-08T08:20:34.339070+0000","last_unstale":"2023-11-08T08:20:35.319969+0000","last_undegraded":"2023-11-08T08:20:35.319969+0000","last_fullsized":"2023-11-08T08:20:35.319969+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:30:43.080172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319976+0000","last_change":"2023-11-08T08:20:34.333087+0000","last_active":"2023-11-08T08:20:35.319976+0000","last_peered":"2023-11-08T08:20:35.319976+0000","last_clean":"2023-11-08T08:20:35.319976+0000","last_became_active":"2023-11-08T08:20:34.332825+0000","last_became_peered":"2023-11-08T08:20:34.332825+0000","last_unstale":"2023-11-08T08:20:35.319976+0000","last_undegraded":"2023-11-08T08:20:35.319976+0000","last_fullsized":"2023-11-08T08:20:35.319976+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:31:59.165324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319686+0000","last_change":"2023-11-08T08:20:34.329030+0000","last_active":"2023-11-08T08:20:35.319686+0000","last_peered":"2023-11-08T08:20:35.319686+0000","last_clean":"2023-11-08T08:20:35.319686+0000","last_became_active":"2023-11-08T08:20:34.328724+0000","last_became_peered":"2023-11-08T08:20:34.328724+0000","last_unstale":"2023-11-08T08:20:35.319686+0000","last_undegraded":"2023-11-08T08:20:35.319686+0000","last_fullsized":"2023-11-08T08:20:35.319686+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:50:55.369713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.343229+0000","last_change":"2023-11-08T08:20:34.328800+0000","last_active":"2023-11-08T08:20:34.343229+0000","last_peered":"2023-11-08T08:20:34.343229+0000","last_clean":"2023-11-08T08:20:34.343229+0000","last_became_active":"2023-11-08T08:20:34.328529+0000","last_became_peered":"2023-11-08T08:20:34.328529+0000","last_unstale":"2023-11-08T08:20:34.343229+0000","last_undegraded":"2023-11-08T08:20:34.343229+0000","last_fullsized":"2023-11-08T08:20:34.343229+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:56:03.805095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.321268+0000","last_change":"2023-11-08T08:20:34.335268+0000","last_active":"2023-11-08T08:20:35.321268+0000","last_peered":"2023-11-08T08:20:35.321268+0000","last_clean":"2023-11-08T08:20:35.321268+0000","last_became_active":"2023-11-08T08:20:34.335024+0000","last_became_peered":"2023-11-08T08:20:34.335024+0000","last_unstale":"2023-11-08T08:20:35.321268+0000","last_undegraded":"2023-11-08T08:20:35.321268+0000","last_fullsized":"2023-11-08T08:20:35.321268+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:28:23.771501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319397+0000","last_change":"2023-11-08T08:20:34.328937+0000","last_active":"2023-11-08T08:20:35.319397+0000","last_peered":"2023-11-08T08:20:35.319397+0000","last_clean":"2023-11-08T08:20:35.319397+0000","last_became_active":"2023-11-08T08:20:34.328625+0000","last_became_peered":"2023-11-08T08:20:34.328625+0000","last_unstale":"2023-11-08T08:20:35.319397+0000","last_undegraded":"2023-11-08T08:20:35.319397+0000","last_fullsized":"2023-11-08T08:20:35.319397+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:32:02.079767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439273+0000","last_change":"2023-11-08T08:20:34.336520+0000","last_active":"2023-11-08T08:20:35.439273+0000","last_peered":"2023-11-08T08:20:35.439273+0000","last_clean":"2023-11-08T08:20:35.439273+0000","last_became_active":"2023-11-08T08:20:34.336251+0000","last_became_peered":"2023-11-08T08:20:34.336251+0000","last_unstale":"2023-11-08T08:20:35.439273+0000","last_undegraded":"2023-11-08T08:20:35.439273+0000","last_fullsized":"2023-11-08T08:20:35.439273+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:15:24.368062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.346936+0000","last_change":"2023-11-08T08:20:34.340453+0000","last_active":"2023-11-08T08:20:34.346936+0000","last_peered":"2023-11-08T08:20:34.346936+0000","last_clean":"2023-11-08T08:20:34.346936+0000","last_became_active":"2023-11-08T08:20:34.340110+0000","last_became_peered":"2023-11-08T08:20:34.340110+0000","last_unstale":"2023-11-08T08:20:34.346936+0000","last_undegraded":"2023-11-08T08:20:34.346936+0000","last_fullsized":"2023-11-08T08:20:34.346936+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:48:42.113335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447809+0000","last_change":"2023-11-08T08:20:34.333209+0000","last_active":"2023-11-08T08:20:35.447809+0000","last_peered":"2023-11-08T08:20:35.447809+0000","last_clean":"2023-11-08T08:20:35.447809+0000","last_became_active":"2023-11-08T08:20:34.332899+0000","last_became_peered":"2023-11-08T08:20:34.332899+0000","last_unstale":"2023-11-08T08:20:35.447809+0000","last_undegraded":"2023-11-08T08:20:35.447809+0000","last_fullsized":"2023-11-08T08:20:35.447809+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:58:23.617005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447721+0000","last_change":"2023-11-08T08:20:34.327381+0000","last_active":"2023-11-08T08:20:35.447721+0000","last_peered":"2023-11-08T08:20:35.447721+0000","last_clean":"2023-11-08T08:20:35.447721+0000","last_became_active":"2023-11-08T08:20:34.325540+0000","last_became_peered":"2023-11-08T08:20:34.325540+0000","last_unstale":"2023-11-08T08:20:35.447721+0000","last_undegraded":"2023-11-08T08:20:35.447721+0000","last_fullsized":"2023-11-08T08:20:35.447721+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:32:02.079767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.348976+0000","last_change":"2023-11-08T08:20:34.342478+0000","last_active":"2023-11-08T08:20:34.348976+0000","last_peered":"2023-11-08T08:20:34.348976+0000","last_clean":"2023-11-08T08:20:34.348976+0000","last_became_active":"2023-11-08T08:20:34.341501+0000","last_became_peered":"2023-11-08T08:20:34.341501+0000","last_unstale":"2023-11-08T08:20:34.348976+0000","last_undegraded":"2023-11-08T08:20:34.348976+0000","last_fullsized":"2023-11-08T08:20:34.348976+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:16:14.114658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.321676+0000","last_change":"2023-11-08T08:20:34.321816+0000","last_active":"2023-11-08T08:20:35.321676+0000","last_peered":"2023-11-08T08:20:35.321676+0000","last_clean":"2023-11-08T08:20:35.321676+0000","last_became_active":"2023-11-08T08:20:34.321573+0000","last_became_peered":"2023-11-08T08:20:34.321573+0000","last_unstale":"2023-11-08T08:20:35.321676+0000","last_undegraded":"2023-11-08T08:20:35.321676+0000","last_fullsized":"2023-11-08T08:20:35.321676+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:27:55.895840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439573+0000","last_change":"2023-11-08T08:20:34.340948+0000","last_active":"2023-11-08T08:20:35.439573+0000","last_peered":"2023-11-08T08:20:35.439573+0000","last_clean":"2023-11-08T08:20:35.439573+0000","last_became_active":"2023-11-08T08:20:34.339080+0000","last_became_peered":"2023-11-08T08:20:34.339080+0000","last_unstale":"2023-11-08T08:20:35.439573+0000","last_undegraded":"2023-11-08T08:20:35.439573+0000","last_fullsized":"2023-11-08T08:20:35.439573+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:47:19.286387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319457+0000","last_change":"2023-11-08T08:20:34.333230+0000","last_active":"2023-11-08T08:20:35.319457+0000","last_peered":"2023-11-08T08:20:35.319457+0000","last_clean":"2023-11-08T08:20:35.319457+0000","last_became_active":"2023-11-08T08:20:34.332947+0000","last_became_peered":"2023-11-08T08:20:34.332947+0000","last_unstale":"2023-11-08T08:20:35.319457+0000","last_undegraded":"2023-11-08T08:20:35.319457+0000","last_fullsized":"2023-11-08T08:20:35.319457+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:47:19.286387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.349339+0000","last_change":"2023-11-08T08:20:34.343766+0000","last_active":"2023-11-08T08:20:34.349339+0000","last_peered":"2023-11-08T08:20:34.349339+0000","last_clean":"2023-11-08T08:20:34.349339+0000","last_became_active":"2023-11-08T08:20:34.343324+0000","last_became_peered":"2023-11-08T08:20:34.343324+0000","last_unstale":"2023-11-08T08:20:34.349339+0000","last_undegraded":"2023-11-08T08:20:34.349339+0000","last_fullsized":"2023-11-08T08:20:34.349339+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:27:55.895840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.346958+0000","last_change":"2023-11-08T08:20:34.341041+0000","last_active":"2023-11-08T08:20:34.346958+0000","last_peered":"2023-11-08T08:20:34.346958+0000","last_clean":"2023-11-08T08:20:34.346958+0000","last_became_active":"2023-11-08T08:20:34.340736+0000","last_became_peered":"2023-11-08T08:20:34.340736+0000","last_unstale":"2023-11-08T08:20:34.346958+0000","last_undegraded":"2023-11-08T08:20:34.346958+0000","last_fullsized":"2023-11-08T08:20:34.346958+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:08:18.698224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.345985+0000","last_change":"2023-11-08T08:20:34.338102+0000","last_active":"2023-11-08T08:20:34.345985+0000","last_peered":"2023-11-08T08:20:34.345985+0000","last_clean":"2023-11-08T08:20:34.345985+0000","last_became_active":"2023-11-08T08:20:34.337750+0000","last_became_peered":"2023-11-08T08:20:34.337750+0000","last_unstale":"2023-11-08T08:20:34.345985+0000","last_undegraded":"2023-11-08T08:20:34.345985+0000","last_fullsized":"2023-11-08T08:20:34.345985+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:31:38.135691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.321738+0000","last_change":"2023-11-08T08:20:34.335934+0000","last_active":"2023-11-08T08:20:35.321738+0000","last_peered":"2023-11-08T08:20:35.321738+0000","last_clean":"2023-11-08T08:20:35.321738+0000","last_became_active":"2023-11-08T08:20:34.335631+0000","last_became_peered":"2023-11-08T08:20:34.335631+0000","last_unstale":"2023-11-08T08:20:35.321738+0000","last_undegraded":"2023-11-08T08:20:35.321738+0000","last_fullsized":"2023-11-08T08:20:35.321738+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:31:38.135691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447591+0000","last_change":"2023-11-08T08:20:34.332562+0000","last_active":"2023-11-08T08:20:35.447591+0000","last_peered":"2023-11-08T08:20:35.447591+0000","last_clean":"2023-11-08T08:20:35.447591+0000","last_became_active":"2023-11-08T08:20:34.332260+0000","last_became_peered":"2023-11-08T08:20:34.332260+0000","last_unstale":"2023-11-08T08:20:35.447591+0000","last_undegraded":"2023-11-08T08:20:35.447591+0000","last_fullsized":"2023-11-08T08:20:35.447591+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:09.194290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.330349+0000","last_change":"2023-11-08T08:20:34.323169+0000","last_active":"2023-11-08T08:20:34.330349+0000","last_peered":"2023-11-08T08:20:34.330349+0000","last_clean":"2023-11-08T08:20:34.330349+0000","last_became_active":"2023-11-08T08:20:34.322681+0000","last_became_peered":"2023-11-08T08:20:34.322681+0000","last_unstale":"2023-11-08T08:20:34.330349+0000","last_undegraded":"2023-11-08T08:20:34.330349+0000","last_fullsized":"2023-11-08T08:20:34.330349+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:31:59.165324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.344473+0000","last_change":"2023-11-08T08:20:34.335087+0000","last_active":"2023-11-08T08:20:34.344473+0000","last_peered":"2023-11-08T08:20:34.344473+0000","last_clean":"2023-11-08T08:20:34.344473+0000","last_became_active":"2023-11-08T08:20:34.334780+0000","last_became_peered":"2023-11-08T08:20:34.334780+0000","last_unstale":"2023-11-08T08:20:34.344473+0000","last_undegraded":"2023-11-08T08:20:34.344473+0000","last_fullsized":"2023-11-08T08:20:34.344473+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:19:14.805477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447340+0000","last_change":"2023-11-08T08:20:34.333075+0000","last_active":"2023-11-08T08:20:35.447340+0000","last_peered":"2023-11-08T08:20:35.447340+0000","last_clean":"2023-11-08T08:20:35.447340+0000","last_became_active":"2023-11-08T08:20:34.332803+0000","last_became_peered":"2023-11-08T08:20:34.332803+0000","last_unstale":"2023-11-08T08:20:35.447340+0000","last_undegraded":"2023-11-08T08:20:35.447340+0000","last_fullsized":"2023-11-08T08:20:35.447340+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:47:19.286387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439279+0000","last_change":"2023-11-08T08:20:34.346083+0000","last_active":"2023-11-08T08:20:35.439279+0000","last_peered":"2023-11-08T08:20:35.439279+0000","last_clean":"2023-11-08T08:20:35.439279+0000","last_became_active":"2023-11-08T08:20:34.344329+0000","last_became_peered":"2023-11-08T08:20:34.344329+0000","last_unstale":"2023-11-08T08:20:35.439279+0000","last_undegraded":"2023-11-08T08:20:35.439279+0000","last_fullsized":"2023-11-08T08:20:35.439279+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:19:44.058497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.321305+0000","last_change":"2023-11-08T08:20:34.331102+0000","last_active":"2023-11-08T08:20:35.321305+0000","last_peered":"2023-11-08T08:20:35.321305+0000","last_clean":"2023-11-08T08:20:35.321305+0000","last_became_active":"2023-11-08T08:20:34.329730+0000","last_became_peered":"2023-11-08T08:20:34.329730+0000","last_unstale":"2023-11-08T08:20:35.321305+0000","last_undegraded":"2023-11-08T08:20:35.321305+0000","last_fullsized":"2023-11-08T08:20:35.321305+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:26:18.631300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.323046+0000","last_change":"2023-11-08T08:20:34.335051+0000","last_active":"2023-11-08T08:20:35.323046+0000","last_peered":"2023-11-08T08:20:35.323046+0000","last_clean":"2023-11-08T08:20:35.323046+0000","last_became_active":"2023-11-08T08:20:34.334797+0000","last_became_peered":"2023-11-08T08:20:34.334797+0000","last_unstale":"2023-11-08T08:20:35.323046+0000","last_undegraded":"2023-11-08T08:20:35.323046+0000","last_fullsized":"2023-11-08T08:20:35.323046+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:19:14.805477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"42'4","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.440339+0000","last_change":"2023-11-08T08:20:33.848558+0000","last_active":"2023-11-08T08:20:35.440339+0000","last_peered":"2023-11-08T08:20:35.440339+0000","last_clean":"2023-11-08T08:20:35.440339+0000","last_became_active":"2023-11-08T08:20:33.848277+0000","last_became_peered":"2023-11-08T08:20:33.848277+0000","last_unstale":"2023-11-08T08:20:35.440339+0000","last_undegraded":"2023-11-08T08:20:35.440339+0000","last_fullsized":"2023-11-08T08:20:35.440339+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:16:43.509880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.340488+0000","last_change":"2023-11-08T08:20:34.332475+0000","last_active":"2023-11-08T08:20:34.340488+0000","last_peered":"2023-11-08T08:20:34.340488+0000","last_clean":"2023-11-08T08:20:34.340488+0000","last_became_active":"2023-11-08T08:20:34.331733+0000","last_became_peered":"2023-11-08T08:20:34.331733+0000","last_unstale":"2023-11-08T08:20:34.340488+0000","last_undegraded":"2023-11-08T08:20:34.340488+0000","last_fullsized":"2023-11-08T08:20:34.340488+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:00:34.711344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447905+0000","last_change":"2023-11-08T08:20:33.850421+0000","last_active":"2023-11-08T08:20:35.447905+0000","last_peered":"2023-11-08T08:20:35.447905+0000","last_clean":"2023-11-08T08:20:35.447905+0000","last_became_active":"2023-11-08T08:20:33.850058+0000","last_became_peered":"2023-11-08T08:20:33.850058+0000","last_unstale":"2023-11-08T08:20:35.447905+0000","last_undegraded":"2023-11-08T08:20:35.447905+0000","last_fullsized":"2023-11-08T08:20:35.447905+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:05:48.739190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.348909+0000","last_change":"2023-11-08T08:20:34.345991+0000","last_active":"2023-11-08T08:20:34.348909+0000","last_peered":"2023-11-08T08:20:34.348909+0000","last_clean":"2023-11-08T08:20:34.348909+0000","last_became_active":"2023-11-08T08:20:34.345699+0000","last_became_peered":"2023-11-08T08:20:34.345699+0000","last_unstale":"2023-11-08T08:20:34.348909+0000","last_undegraded":"2023-11-08T08:20:34.348909+0000","last_fullsized":"2023-11-08T08:20:34.348909+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:39:57.195619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.323557+0000","last_change":"2023-11-08T08:20:33.841990+0000","last_active":"2023-11-08T08:20:35.323557+0000","last_peered":"2023-11-08T08:20:35.323557+0000","last_clean":"2023-11-08T08:20:35.323557+0000","last_became_active":"2023-11-08T08:20:33.841474+0000","last_became_peered":"2023-11-08T08:20:33.841474+0000","last_unstale":"2023-11-08T08:20:35.323557+0000","last_undegraded":"2023-11-08T08:20:35.323557+0000","last_fullsized":"2023-11-08T08:20:35.323557+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:48:41.102650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.323297+0000","last_change":"2023-11-08T08:20:34.330319+0000","last_active":"2023-11-08T08:20:35.323297+0000","last_peered":"2023-11-08T08:20:35.323297+0000","last_clean":"2023-11-08T08:20:35.323297+0000","last_became_active":"2023-11-08T08:20:34.329960+0000","last_became_peered":"2023-11-08T08:20:34.329960+0000","last_unstale":"2023-11-08T08:20:35.323297+0000","last_undegraded":"2023-11-08T08:20:35.323297+0000","last_fullsized":"2023-11-08T08:20:35.323297+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:13:05.557780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.349312+0000","last_change":"2023-11-08T08:20:33.843611+0000","last_active":"2023-11-08T08:20:34.349312+0000","last_peered":"2023-11-08T08:20:34.349312+0000","last_clean":"2023-11-08T08:20:34.349312+0000","last_became_active":"2023-11-08T08:20:33.843232+0000","last_became_peered":"2023-11-08T08:20:33.843232+0000","last_unstale":"2023-11-08T08:20:34.349312+0000","last_undegraded":"2023-11-08T08:20:34.349312+0000","last_fullsized":"2023-11-08T08:20:34.349312+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:26:17.620615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447389+0000","last_change":"2023-11-08T08:20:34.335134+0000","last_active":"2023-11-08T08:20:35.447389+0000","last_peered":"2023-11-08T08:20:35.447389+0000","last_clean":"2023-11-08T08:20:35.447389+0000","last_became_active":"2023-11-08T08:20:34.334843+0000","last_became_peered":"2023-11-08T08:20:34.334843+0000","last_unstale":"2023-11-08T08:20:35.447389+0000","last_undegraded":"2023-11-08T08:20:35.447389+0000","last_fullsized":"2023-11-08T08:20:35.447389+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:30:43.080172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439799+0000","last_change":"2023-11-08T08:20:33.849373+0000","last_active":"2023-11-08T08:20:35.439799+0000","last_peered":"2023-11-08T08:20:35.439799+0000","last_clean":"2023-11-08T08:20:35.439799+0000","last_became_active":"2023-11-08T08:20:33.849121+0000","last_became_peered":"2023-11-08T08:20:33.849121+0000","last_unstale":"2023-11-08T08:20:35.439799+0000","last_undegraded":"2023-11-08T08:20:35.439799+0000","last_fullsized":"2023-11-08T08:20:35.439799+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:09:01.178114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.320223+0000","last_change":"2023-11-08T08:20:34.320058+0000","last_active":"2023-11-08T08:20:35.320223+0000","last_peered":"2023-11-08T08:20:35.320223+0000","last_clean":"2023-11-08T08:20:35.320223+0000","last_became_active":"2023-11-08T08:20:34.319262+0000","last_became_peered":"2023-11-08T08:20:34.319262+0000","last_unstale":"2023-11-08T08:20:35.320223+0000","last_undegraded":"2023-11-08T08:20:35.320223+0000","last_fullsized":"2023-11-08T08:20:35.320223+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:15:24.368062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447674+0000","last_change":"2023-11-08T08:20:33.850496+0000","last_active":"2023-11-08T08:20:35.447674+0000","last_peered":"2023-11-08T08:20:35.447674+0000","last_clean":"2023-11-08T08:20:35.447674+0000","last_became_active":"2023-11-08T08:20:33.850043+0000","last_became_peered":"2023-11-08T08:20:33.850043+0000","last_unstale":"2023-11-08T08:20:35.447674+0000","last_undegraded":"2023-11-08T08:20:35.447674+0000","last_fullsized":"2023-11-08T08:20:35.447674+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:11:10.537633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.318918+0000","last_change":"2023-11-08T08:20:34.331941+0000","last_active":"2023-11-08T08:20:35.318918+0000","last_peered":"2023-11-08T08:20:35.318918+0000","last_clean":"2023-11-08T08:20:35.318918+0000","last_became_active":"2023-11-08T08:20:34.331659+0000","last_became_peered":"2023-11-08T08:20:34.331659+0000","last_unstale":"2023-11-08T08:20:35.318918+0000","last_undegraded":"2023-11-08T08:20:35.318918+0000","last_fullsized":"2023-11-08T08:20:35.318918+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:19:44.058497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.310500+0000","last_change":"2023-11-08T08:20:33.845051+0000","last_active":"2023-11-08T08:20:34.310500+0000","last_peered":"2023-11-08T08:20:34.310500+0000","last_clean":"2023-11-08T08:20:34.310500+0000","last_became_active":"2023-11-08T08:20:33.844642+0000","last_became_peered":"2023-11-08T08:20:33.844642+0000","last_unstale":"2023-11-08T08:20:34.310500+0000","last_undegraded":"2023-11-08T08:20:34.310500+0000","last_fullsized":"2023-11-08T08:20:34.310500+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:09:46.694297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.349316+0000","last_change":"2023-11-08T08:20:34.343702+0000","last_active":"2023-11-08T08:20:34.349316+0000","last_peered":"2023-11-08T08:20:34.349316+0000","last_clean":"2023-11-08T08:20:34.349316+0000","last_became_active":"2023-11-08T08:20:34.343330+0000","last_became_peered":"2023-11-08T08:20:34.343330+0000","last_unstale":"2023-11-08T08:20:34.349316+0000","last_undegraded":"2023-11-08T08:20:34.349316+0000","last_fullsized":"2023-11-08T08:20:34.349316+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:28:23.771501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319001+0000","last_change":"2023-11-08T08:20:33.839789+0000","last_active":"2023-11-08T08:20:35.319001+0000","last_peered":"2023-11-08T08:20:35.319001+0000","last_clean":"2023-11-08T08:20:35.319001+0000","last_became_active":"2023-11-08T08:20:33.839489+0000","last_became_peered":"2023-11-08T08:20:33.839489+0000","last_unstale":"2023-11-08T08:20:35.319001+0000","last_undegraded":"2023-11-08T08:20:35.319001+0000","last_fullsized":"2023-11-08T08:20:35.319001+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:13:04.547095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447814+0000","last_change":"2023-11-08T08:20:34.325628+0000","last_active":"2023-11-08T08:20:35.447814+0000","last_peered":"2023-11-08T08:20:35.447814+0000","last_clean":"2023-11-08T08:20:35.447814+0000","last_became_active":"2023-11-08T08:20:34.325059+0000","last_became_peered":"2023-11-08T08:20:34.325059+0000","last_unstale":"2023-11-08T08:20:35.447814+0000","last_undegraded":"2023-11-08T08:20:35.447814+0000","last_fullsized":"2023-11-08T08:20:35.447814+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:15:24.368062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.448374+0000","last_change":"2023-11-08T08:20:33.846230+0000","last_active":"2023-11-08T08:20:35.448374+0000","last_peered":"2023-11-08T08:20:35.448374+0000","last_clean":"2023-11-08T08:20:35.448374+0000","last_became_active":"2023-11-08T08:20:33.845766+0000","last_became_peered":"2023-11-08T08:20:33.845766+0000","last_unstale":"2023-11-08T08:20:35.448374+0000","last_undegraded":"2023-11-08T08:20:35.448374+0000","last_fullsized":"2023-11-08T08:20:35.448374+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:13:00.589562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.341972+0000","last_change":"2023-11-08T08:20:34.337217+0000","last_active":"2023-11-08T08:20:34.341972+0000","last_peered":"2023-11-08T08:20:34.341972+0000","last_clean":"2023-11-08T08:20:34.341972+0000","last_became_active":"2023-11-08T08:20:34.336944+0000","last_became_peered":"2023-11-08T08:20:34.336944+0000","last_unstale":"2023-11-08T08:20:34.341972+0000","last_undegraded":"2023-11-08T08:20:34.341972+0000","last_fullsized":"2023-11-08T08:20:34.341972+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:09:47.704982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.320332+0000","last_change":"2023-11-08T08:20:33.832698+0000","last_active":"2023-11-08T08:20:35.320332+0000","last_peered":"2023-11-08T08:20:35.320332+0000","last_clean":"2023-11-08T08:20:35.320332+0000","last_became_active":"2023-11-08T08:20:33.832414+0000","last_became_peered":"2023-11-08T08:20:33.832414+0000","last_unstale":"2023-11-08T08:20:35.320332+0000","last_undegraded":"2023-11-08T08:20:35.320332+0000","last_fullsized":"2023-11-08T08:20:35.320332+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:11:41.800061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.440093+0000","last_change":"2023-11-08T08:20:34.347100+0000","last_active":"2023-11-08T08:20:35.440093+0000","last_peered":"2023-11-08T08:20:35.440093+0000","last_clean":"2023-11-08T08:20:35.440093+0000","last_became_active":"2023-11-08T08:20:34.346768+0000","last_became_peered":"2023-11-08T08:20:34.346768+0000","last_unstale":"2023-11-08T08:20:35.440093+0000","last_undegraded":"2023-11-08T08:20:35.440093+0000","last_fullsized":"2023-11-08T08:20:35.440093+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:39:57.195619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.349553+0000","last_change":"2023-11-08T08:20:33.849553+0000","last_active":"2023-11-08T08:20:34.349553+0000","last_peered":"2023-11-08T08:20:34.349553+0000","last_clean":"2023-11-08T08:20:34.349553+0000","last_became_active":"2023-11-08T08:20:33.849304+0000","last_became_peered":"2023-11-08T08:20:33.849304+0000","last_unstale":"2023-11-08T08:20:34.349553+0000","last_undegraded":"2023-11-08T08:20:34.349553+0000","last_fullsized":"2023-11-08T08:20:34.349553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:19:13.794792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.344007+0000","last_change":"2023-11-08T08:20:34.329202+0000","last_active":"2023-11-08T08:20:34.344007+0000","last_peered":"2023-11-08T08:20:34.344007+0000","last_clean":"2023-11-08T08:20:34.344007+0000","last_became_active":"2023-11-08T08:20:34.328557+0000","last_became_peered":"2023-11-08T08:20:34.328557+0000","last_unstale":"2023-11-08T08:20:34.344007+0000","last_undegraded":"2023-11-08T08:20:34.344007+0000","last_fullsized":"2023-11-08T08:20:34.344007+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:13:01.600247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.321824+0000","last_change":"2023-11-08T08:20:33.844322+0000","last_active":"2023-11-08T08:20:35.321824+0000","last_peered":"2023-11-08T08:20:35.321824+0000","last_clean":"2023-11-08T08:20:35.321824+0000","last_became_active":"2023-11-08T08:20:33.844034+0000","last_became_peered":"2023-11-08T08:20:33.844034+0000","last_unstale":"2023-11-08T08:20:35.321824+0000","last_undegraded":"2023-11-08T08:20:35.321824+0000","last_fullsized":"2023-11-08T08:20:35.321824+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:46:19.490236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.349165+0000","last_change":"2023-11-08T08:20:34.344556+0000","last_active":"2023-11-08T08:20:34.349165+0000","last_peered":"2023-11-08T08:20:34.349165+0000","last_clean":"2023-11-08T08:20:34.349165+0000","last_became_active":"2023-11-08T08:20:34.344252+0000","last_became_peered":"2023-11-08T08:20:34.344252+0000","last_unstale":"2023-11-08T08:20:34.349165+0000","last_undegraded":"2023-11-08T08:20:34.349165+0000","last_fullsized":"2023-11-08T08:20:34.349165+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:13:05.557780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439895+0000","last_change":"2023-11-08T08:20:33.845912+0000","last_active":"2023-11-08T08:20:35.439895+0000","last_peered":"2023-11-08T08:20:35.439895+0000","last_clean":"2023-11-08T08:20:35.439895+0000","last_became_active":"2023-11-08T08:20:33.845639+0000","last_became_peered":"2023-11-08T08:20:33.845639+0000","last_unstale":"2023-11-08T08:20:35.439895+0000","last_undegraded":"2023-11-08T08:20:35.439895+0000","last_fullsized":"2023-11-08T08:20:35.439895+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:19:12.865432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.320314+0000","last_change":"2023-11-08T08:20:34.331437+0000","last_active":"2023-11-08T08:20:35.320314+0000","last_peered":"2023-11-08T08:20:35.320314+0000","last_clean":"2023-11-08T08:20:35.320314+0000","last_became_active":"2023-11-08T08:20:34.329416+0000","last_became_peered":"2023-11-08T08:20:34.329416+0000","last_unstale":"2023-11-08T08:20:35.320314+0000","last_undegraded":"2023-11-08T08:20:35.320314+0000","last_fullsized":"2023-11-08T08:20:35.320314+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:17:06.821071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.314630+0000","last_change":"2023-11-08T08:20:33.838897+0000","last_active":"2023-11-08T08:20:34.314630+0000","last_peered":"2023-11-08T08:20:34.314630+0000","last_clean":"2023-11-08T08:20:34.314630+0000","last_became_active":"2023-11-08T08:20:33.838641+0000","last_became_peered":"2023-11-08T08:20:33.838641+0000","last_unstale":"2023-11-08T08:20:34.314630+0000","last_undegraded":"2023-11-08T08:20:34.314630+0000","last_fullsized":"2023-11-08T08:20:34.314630+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:16:13.103973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439282+0000","last_change":"2023-11-08T08:20:34.347906+0000","last_active":"2023-11-08T08:20:35.439282+0000","last_peered":"2023-11-08T08:20:35.439282+0000","last_clean":"2023-11-08T08:20:35.439282+0000","last_became_active":"2023-11-08T08:20:34.347650+0000","last_became_peered":"2023-11-08T08:20:34.347650+0000","last_unstale":"2023-11-08T08:20:35.439282+0000","last_undegraded":"2023-11-08T08:20:35.439282+0000","last_fullsized":"2023-11-08T08:20:35.439282+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:17:06.821071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439391+0000","last_change":"2023-11-08T08:20:33.847825+0000","last_active":"2023-11-08T08:20:35.439391+0000","last_peered":"2023-11-08T08:20:35.439391+0000","last_clean":"2023-11-08T08:20:35.439391+0000","last_became_active":"2023-11-08T08:20:33.847566+0000","last_became_peered":"2023-11-08T08:20:33.847566+0000","last_unstale":"2023-11-08T08:20:35.439391+0000","last_undegraded":"2023-11-08T08:20:35.439391+0000","last_fullsized":"2023-11-08T08:20:35.439391+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:41:47.878758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.342457+0000","last_change":"2023-11-08T08:20:34.336533+0000","last_active":"2023-11-08T08:20:34.342457+0000","last_peered":"2023-11-08T08:20:34.342457+0000","last_clean":"2023-11-08T08:20:34.342457+0000","last_became_active":"2023-11-08T08:20:34.336243+0000","last_became_peered":"2023-11-08T08:20:34.336243+0000","last_unstale":"2023-11-08T08:20:34.342457+0000","last_undegraded":"2023-11-08T08:20:34.342457+0000","last_fullsized":"2023-11-08T08:20:34.342457+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:47:01.311804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319151+0000","last_change":"2023-11-08T08:20:33.832222+0000","last_active":"2023-11-08T08:20:35.319151+0000","last_peered":"2023-11-08T08:20:35.319151+0000","last_clean":"2023-11-08T08:20:35.319151+0000","last_became_active":"2023-11-08T08:20:33.831917+0000","last_became_peered":"2023-11-08T08:20:33.831917+0000","last_unstale":"2023-11-08T08:20:35.319151+0000","last_undegraded":"2023-11-08T08:20:35.319151+0000","last_fullsized":"2023-11-08T08:20:35.319151+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:42:20.705274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.321922+0000","last_change":"2023-11-08T08:20:34.322642+0000","last_active":"2023-11-08T08:20:35.321922+0000","last_peered":"2023-11-08T08:20:35.321922+0000","last_clean":"2023-11-08T08:20:35.321922+0000","last_became_active":"2023-11-08T08:20:34.322367+0000","last_became_peered":"2023-11-08T08:20:34.322367+0000","last_unstale":"2023-11-08T08:20:35.321922+0000","last_undegraded":"2023-11-08T08:20:35.321922+0000","last_fullsized":"2023-11-08T08:20:35.321922+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:16:14.114658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.322094+0000","last_change":"2023-11-08T08:20:33.850157+0000","last_active":"2023-11-08T08:20:35.322094+0000","last_peered":"2023-11-08T08:20:35.322094+0000","last_clean":"2023-11-08T08:20:35.322094+0000","last_became_active":"2023-11-08T08:20:33.848800+0000","last_became_peered":"2023-11-08T08:20:33.848800+0000","last_unstale":"2023-11-08T08:20:35.322094+0000","last_undegraded":"2023-11-08T08:20:35.322094+0000","last_fullsized":"2023-11-08T08:20:35.322094+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:12:15.661299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.320050+0000","last_change":"2023-11-08T08:20:34.344491+0000","last_active":"2023-11-08T08:20:35.320050+0000","last_peered":"2023-11-08T08:20:35.320050+0000","last_clean":"2023-11-08T08:20:35.320050+0000","last_became_active":"2023-11-08T08:20:34.339597+0000","last_became_peered":"2023-11-08T08:20:34.339597+0000","last_unstale":"2023-11-08T08:20:35.320050+0000","last_undegraded":"2023-11-08T08:20:35.320050+0000","last_fullsized":"2023-11-08T08:20:35.320050+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:09.194290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.448058+0000","last_change":"2023-11-08T08:20:33.846503+0000","last_active":"2023-11-08T08:20:35.448058+0000","last_peered":"2023-11-08T08:20:35.448058+0000","last_clean":"2023-11-08T08:20:35.448058+0000","last_became_active":"2023-11-08T08:20:33.846243+0000","last_became_peered":"2023-11-08T08:20:33.846243+0000","last_unstale":"2023-11-08T08:20:35.448058+0000","last_undegraded":"2023-11-08T08:20:35.448058+0000","last_fullsized":"2023-11-08T08:20:35.448058+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:16:43.509880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.345424+0000","last_change":"2023-11-08T08:20:34.337402+0000","last_active":"2023-11-08T08:20:34.345424+0000","last_peered":"2023-11-08T08:20:34.345424+0000","last_clean":"2023-11-08T08:20:34.345424+0000","last_became_active":"2023-11-08T08:20:34.336870+0000","last_became_peered":"2023-11-08T08:20:34.336870+0000","last_unstale":"2023-11-08T08:20:34.345424+0000","last_undegraded":"2023-11-08T08:20:34.345424+0000","last_fullsized":"2023-11-08T08:20:34.345424+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:42:21.715959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.345716+0000","last_change":"2023-11-08T08:20:33.840964+0000","last_active":"2023-11-08T08:20:34.345716+0000","last_peered":"2023-11-08T08:20:34.345716+0000","last_clean":"2023-11-08T08:20:34.345716+0000","last_became_active":"2023-11-08T08:20:33.840724+0000","last_became_peered":"2023-11-08T08:20:33.840724+0000","last_unstale":"2023-11-08T08:20:34.345716+0000","last_undegraded":"2023-11-08T08:20:34.345716+0000","last_fullsized":"2023-11-08T08:20:34.345716+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:56:44.922818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.345161+0000","last_change":"2023-11-08T08:20:34.336911+0000","last_active":"2023-11-08T08:20:34.345161+0000","last_peered":"2023-11-08T08:20:34.345161+0000","last_clean":"2023-11-08T08:20:34.345161+0000","last_became_active":"2023-11-08T08:20:34.336605+0000","last_became_peered":"2023-11-08T08:20:34.336605+0000","last_unstale":"2023-11-08T08:20:34.345161+0000","last_undegraded":"2023-11-08T08:20:34.345161+0000","last_fullsized":"2023-11-08T08:20:34.345161+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:26:18.631300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.440088+0000","last_change":"2023-11-08T08:20:33.846233+0000","last_active":"2023-11-08T08:20:35.440088+0000","last_peered":"2023-11-08T08:20:35.440088+0000","last_clean":"2023-11-08T08:20:35.440088+0000","last_became_active":"2023-11-08T08:20:33.845337+0000","last_became_peered":"2023-11-08T08:20:33.845337+0000","last_unstale":"2023-11-08T08:20:35.440088+0000","last_undegraded":"2023-11-08T08:20:35.440088+0000","last_fullsized":"2023-11-08T08:20:35.440088+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:56:44.922818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439813+0000","last_change":"2023-11-08T08:20:34.348264+0000","last_active":"2023-11-08T08:20:35.439813+0000","last_peered":"2023-11-08T08:20:35.439813+0000","last_clean":"2023-11-08T08:20:35.439813+0000","last_became_active":"2023-11-08T08:20:34.347846+0000","last_became_peered":"2023-11-08T08:20:34.347846+0000","last_unstale":"2023-11-08T08:20:35.439813+0000","last_undegraded":"2023-11-08T08:20:35.439813+0000","last_fullsized":"2023-11-08T08:20:35.439813+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:56:03.805095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439994+0000","last_change":"2023-11-08T08:20:33.844446+0000","last_active":"2023-11-08T08:20:35.439994+0000","last_peered":"2023-11-08T08:20:35.439994+0000","last_clean":"2023-11-08T08:20:35.439994+0000","last_became_active":"2023-11-08T08:20:33.842189+0000","last_became_peered":"2023-11-08T08:20:33.842189+0000","last_unstale":"2023-11-08T08:20:35.439994+0000","last_undegraded":"2023-11-08T08:20:35.439994+0000","last_fullsized":"2023-11-08T08:20:35.439994+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:00:33.700659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.322966+0000","last_change":"2023-11-08T08:20:34.327141+0000","last_active":"2023-11-08T08:20:35.322966+0000","last_peered":"2023-11-08T08:20:35.322966+0000","last_clean":"2023-11-08T08:20:35.322966+0000","last_became_active":"2023-11-08T08:20:34.326847+0000","last_became_peered":"2023-11-08T08:20:34.326847+0000","last_unstale":"2023-11-08T08:20:35.322966+0000","last_undegraded":"2023-11-08T08:20:35.322966+0000","last_fullsized":"2023-11-08T08:20:35.322966+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:31:38.135691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.310726+0000","last_change":"2023-11-08T08:20:33.845802+0000","last_active":"2023-11-08T08:20:34.310726+0000","last_peered":"2023-11-08T08:20:34.310726+0000","last_clean":"2023-11-08T08:20:34.310726+0000","last_became_active":"2023-11-08T08:20:33.845549+0000","last_became_peered":"2023-11-08T08:20:33.845549+0000","last_unstale":"2023-11-08T08:20:34.310726+0000","last_undegraded":"2023-11-08T08:20:34.310726+0000","last_fullsized":"2023-11-08T08:20:34.310726+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:09:01.178114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.321121+0000","last_change":"2023-11-08T08:20:34.327592+0000","last_active":"2023-11-08T08:20:35.321121+0000","last_peered":"2023-11-08T08:20:35.321121+0000","last_clean":"2023-11-08T08:20:35.321121+0000","last_became_active":"2023-11-08T08:20:34.327333+0000","last_became_peered":"2023-11-08T08:20:34.327333+0000","last_unstale":"2023-11-08T08:20:35.321121+0000","last_undegraded":"2023-11-08T08:20:35.321121+0000","last_fullsized":"2023-11-08T08:20:35.321121+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:48:42.113335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319176+0000","last_change":"2023-11-08T08:20:33.833398+0000","last_active":"2023-11-08T08:20:35.319176+0000","last_peered":"2023-11-08T08:20:35.319176+0000","last_clean":"2023-11-08T08:20:35.319176+0000","last_became_active":"2023-11-08T08:20:33.832827+0000","last_became_peered":"2023-11-08T08:20:33.832827+0000","last_unstale":"2023-11-08T08:20:35.319176+0000","last_undegraded":"2023-11-08T08:20:35.319176+0000","last_fullsized":"2023-11-08T08:20:35.319176+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:19:13.794792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319038+0000","last_change":"2023-11-08T08:20:34.328899+0000","last_active":"2023-11-08T08:20:35.319038+0000","last_peered":"2023-11-08T08:20:35.319038+0000","last_clean":"2023-11-08T08:20:35.319038+0000","last_became_active":"2023-11-08T08:20:34.328587+0000","last_became_peered":"2023-11-08T08:20:34.328587+0000","last_unstale":"2023-11-08T08:20:35.319038+0000","last_undegraded":"2023-11-08T08:20:35.319038+0000","last_fullsized":"2023-11-08T08:20:35.319038+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:58:23.617005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.314429+0000","last_change":"2023-11-08T08:20:33.843590+0000","last_active":"2023-11-08T08:20:34.314429+0000","last_peered":"2023-11-08T08:20:34.314429+0000","last_clean":"2023-11-08T08:20:34.314429+0000","last_became_active":"2023-11-08T08:20:33.843281+0000","last_became_peered":"2023-11-08T08:20:33.843281+0000","last_unstale":"2023-11-08T08:20:34.314429+0000","last_undegraded":"2023-11-08T08:20:34.314429+0000","last_fullsized":"2023-11-08T08:20:34.314429+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:27:54.885155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439773+0000","last_change":"2023-11-08T08:20:34.344571+0000","last_active":"2023-11-08T08:20:35.439773+0000","last_peered":"2023-11-08T08:20:35.439773+0000","last_clean":"2023-11-08T08:20:35.439773+0000","last_became_active":"2023-11-08T08:20:34.344023+0000","last_became_peered":"2023-11-08T08:20:34.344023+0000","last_unstale":"2023-11-08T08:20:35.439773+0000","last_undegraded":"2023-11-08T08:20:35.439773+0000","last_fullsized":"2023-11-08T08:20:35.439773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:58:23.617005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.314797+0000","last_change":"2023-11-08T08:20:33.848536+0000","last_active":"2023-11-08T08:20:34.314797+0000","last_peered":"2023-11-08T08:20:34.314797+0000","last_clean":"2023-11-08T08:20:34.314797+0000","last_became_active":"2023-11-08T08:20:33.844523+0000","last_became_peered":"2023-11-08T08:20:33.844523+0000","last_unstale":"2023-11-08T08:20:34.314797+0000","last_undegraded":"2023-11-08T08:20:34.314797+0000","last_fullsized":"2023-11-08T08:20:34.314797+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:28:22.760817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":28,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T08:20:33.839232+0000","last_change":"2023-11-08T08:18:58.771310+0000","last_active":"2023-11-08T08:20:33.839232+0000","last_peered":"2023-11-08T08:20:33.839232+0000","last_clean":"2023-11-08T08:20:33.839232+0000","last_became_active":"2023-11-08T08:18:58.770891+0000","last_became_peered":"2023-11-08T08:18:58.770891+0000","last_unstale":"2023-11-08T08:20:33.839232+0000","last_undegraded":"2023-11-08T08:20:33.839232+0000","last_fullsized":"2023-11-08T08:20:33.839232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439530+0000","last_change":"2023-11-08T08:20:34.346998+0000","last_active":"2023-11-08T08:20:35.439530+0000","last_peered":"2023-11-08T08:20:35.439530+0000","last_clean":"2023-11-08T08:20:35.439530+0000","last_became_active":"2023-11-08T08:20:34.346747+0000","last_became_peered":"2023-11-08T08:20:34.346747+0000","last_unstale":"2023-11-08T08:20:35.439530+0000","last_undegraded":"2023-11-08T08:20:35.439530+0000","last_fullsized":"2023-11-08T08:20:35.439530+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:24:12.113358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447698+0000","last_change":"2023-11-08T08:20:33.851108+0000","last_active":"2023-11-08T08:20:35.447698+0000","last_peered":"2023-11-08T08:20:35.447698+0000","last_clean":"2023-11-08T08:20:35.447698+0000","last_became_active":"2023-11-08T08:20:33.850810+0000","last_became_peered":"2023-11-08T08:20:33.850810+0000","last_unstale":"2023-11-08T08:20:35.447698+0000","last_undegraded":"2023-11-08T08:20:35.447698+0000","last_fullsized":"2023-11-08T08:20:35.447698+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:43:48.363808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.322609+0000","last_change":"2023-11-08T08:18:58.770483+0000","last_active":"2023-11-08T08:20:34.322609+0000","last_peered":"2023-11-08T08:20:34.322609+0000","last_clean":"2023-11-08T08:20:34.322609+0000","last_became_active":"2023-11-08T08:18:58.770040+0000","last_became_peered":"2023-11-08T08:18:58.770040+0000","last_unstale":"2023-11-08T08:20:34.322609+0000","last_undegraded":"2023-11-08T08:20:34.322609+0000","last_fullsized":"2023-11-08T08:20:34.322609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.338049+0000","last_change":"2023-11-08T08:20:34.331904+0000","last_active":"2023-11-08T08:20:34.338049+0000","last_peered":"2023-11-08T08:20:34.338049+0000","last_clean":"2023-11-08T08:20:34.338049+0000","last_became_active":"2023-11-08T08:20:34.331485+0000","last_became_peered":"2023-11-08T08:20:34.331485+0000","last_unstale":"2023-11-08T08:20:34.338049+0000","last_undegraded":"2023-11-08T08:20:34.338049+0000","last_fullsized":"2023-11-08T08:20:34.338049+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:17:06.821071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.322866+0000","last_change":"2023-11-08T08:20:33.848387+0000","last_active":"2023-11-08T08:20:35.322866+0000","last_peered":"2023-11-08T08:20:35.322866+0000","last_clean":"2023-11-08T08:20:35.322866+0000","last_became_active":"2023-11-08T08:20:33.848133+0000","last_became_peered":"2023-11-08T08:20:33.848133+0000","last_unstale":"2023-11-08T08:20:35.322866+0000","last_undegraded":"2023-11-08T08:20:35.322866+0000","last_fullsized":"2023-11-08T08:20:35.322866+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:43:23.168935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T08:20:33.829156+0000","last_change":"2023-11-08T08:18:58.766848+0000","last_active":"2023-11-08T08:20:33.829156+0000","last_peered":"2023-11-08T08:20:33.829156+0000","last_clean":"2023-11-08T08:20:33.829156+0000","last_became_active":"2023-11-08T08:18:58.766378+0000","last_became_peered":"2023-11-08T08:18:58.766378+0000","last_unstale":"2023-11-08T08:20:33.829156+0000","last_undegraded":"2023-11-08T08:20:33.829156+0000","last_fullsized":"2023-11-08T08:20:33.829156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447535+0000","last_change":"2023-11-08T08:20:34.337228+0000","last_active":"2023-11-08T08:20:35.447535+0000","last_peered":"2023-11-08T08:20:35.447535+0000","last_clean":"2023-11-08T08:20:35.447535+0000","last_became_active":"2023-11-08T08:20:34.336988+0000","last_became_peered":"2023-11-08T08:20:34.336988+0000","last_unstale":"2023-11-08T08:20:35.447535+0000","last_undegraded":"2023-11-08T08:20:35.447535+0000","last_fullsized":"2023-11-08T08:20:35.447535+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:50:55.369713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319095+0000","last_change":"2023-11-08T08:20:33.845617+0000","last_active":"2023-11-08T08:20:35.319095+0000","last_peered":"2023-11-08T08:20:35.319095+0000","last_clean":"2023-11-08T08:20:35.319095+0000","last_became_active":"2023-11-08T08:20:33.845315+0000","last_became_peered":"2023-11-08T08:20:33.845315+0000","last_unstale":"2023-11-08T08:20:35.319095+0000","last_undegraded":"2023-11-08T08:20:35.319095+0000","last_fullsized":"2023-11-08T08:20:35.319095+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:26:17.620615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.321408+0000","last_change":"2023-11-08T08:18:58.769764+0000","last_active":"2023-11-08T08:20:34.321408+0000","last_peered":"2023-11-08T08:20:34.321408+0000","last_clean":"2023-11-08T08:20:34.321408+0000","last_became_active":"2023-11-08T08:18:58.769357+0000","last_became_peered":"2023-11-08T08:18:58.769357+0000","last_unstale":"2023-11-08T08:20:34.321408+0000","last_undegraded":"2023-11-08T08:20:34.321408+0000","last_fullsized":"2023-11-08T08:20:34.321408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:13.748094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.347057+0000","last_change":"2023-11-08T08:20:34.341170+0000","last_active":"2023-11-08T08:20:34.347057+0000","last_peered":"2023-11-08T08:20:34.347057+0000","last_clean":"2023-11-08T08:20:34.347057+0000","last_became_active":"2023-11-08T08:20:34.340922+0000","last_became_peered":"2023-11-08T08:20:34.340922+0000","last_unstale":"2023-11-08T08:20:34.347057+0000","last_undegraded":"2023-11-08T08:20:34.347057+0000","last_fullsized":"2023-11-08T08:20:34.347057+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:11:42.810746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"43'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.443933+0000","last_change":"2023-11-08T08:20:33.844867+0000","last_active":"2023-11-08T08:20:35.443933+0000","last_peered":"2023-11-08T08:20:35.443933+0000","last_clean":"2023-11-08T08:20:35.443933+0000","last_became_active":"2023-11-08T08:20:33.844553+0000","last_became_peered":"2023-11-08T08:20:33.844553+0000","last_unstale":"2023-11-08T08:20:35.443933+0000","last_undegraded":"2023-11-08T08:20:35.443933+0000","last_fullsized":"2023-11-08T08:20:35.443933+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:09:46.694297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T08:20:33.827576+0000","last_change":"2023-11-08T08:18:58.771460+0000","last_active":"2023-11-08T08:20:33.827576+0000","last_peered":"2023-11-08T08:20:33.827576+0000","last_clean":"2023-11-08T08:20:33.827576+0000","last_became_active":"2023-11-08T08:18:58.770648+0000","last_became_peered":"2023-11-08T08:18:58.770648+0000","last_unstale":"2023-11-08T08:20:33.827576+0000","last_undegraded":"2023-11-08T08:20:33.827576+0000","last_fullsized":"2023-11-08T08:20:33.827576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319687+0000","last_change":"2023-11-08T08:20:34.326509+0000","last_active":"2023-11-08T08:20:35.319687+0000","last_peered":"2023-11-08T08:20:35.319687+0000","last_clean":"2023-11-08T08:20:35.319687+0000","last_became_active":"2023-11-08T08:20:34.326252+0000","last_became_peered":"2023-11-08T08:20:34.326252+0000","last_unstale":"2023-11-08T08:20:35.319687+0000","last_undegraded":"2023-11-08T08:20:35.319687+0000","last_fullsized":"2023-11-08T08:20:35.319687+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:56:03.805095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.315626+0000","last_change":"2023-11-08T08:20:33.845017+0000","last_active":"2023-11-08T08:20:34.315626+0000","last_peered":"2023-11-08T08:20:34.315626+0000","last_clean":"2023-11-08T08:20:34.315626+0000","last_became_active":"2023-11-08T08:20:33.844760+0000","last_became_peered":"2023-11-08T08:20:33.844760+0000","last_unstale":"2023-11-08T08:20:34.315626+0000","last_undegraded":"2023-11-08T08:20:34.315626+0000","last_fullsized":"2023-11-08T08:20:34.315626+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:31:37.125006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.315620+0000","last_change":"2023-11-08T08:18:58.973732+0000","last_active":"2023-11-08T08:20:34.315620+0000","last_peered":"2023-11-08T08:20:34.315620+0000","last_clean":"2023-11-08T08:20:34.315620+0000","last_became_active":"2023-11-08T08:18:58.973007+0000","last_became_peered":"2023-11-08T08:18:58.973007+0000","last_unstale":"2023-11-08T08:20:34.315620+0000","last_undegraded":"2023-11-08T08:20:34.315620+0000","last_fullsized":"2023-11-08T08:20:34.315620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.344029+0000","last_change":"2023-11-08T08:20:34.329387+0000","last_active":"2023-11-08T08:20:34.344029+0000","last_peered":"2023-11-08T08:20:34.344029+0000","last_clean":"2023-11-08T08:20:34.344029+0000","last_became_active":"2023-11-08T08:20:34.328949+0000","last_became_peered":"2023-11-08T08:20:34.328949+0000","last_unstale":"2023-11-08T08:20:34.344029+0000","last_undegraded":"2023-11-08T08:20:34.344029+0000","last_fullsized":"2023-11-08T08:20:34.344029+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:16:44.520565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.322741+0000","last_change":"2023-11-08T08:20:33.845481+0000","last_active":"2023-11-08T08:20:35.322741+0000","last_peered":"2023-11-08T08:20:35.322741+0000","last_clean":"2023-11-08T08:20:35.322741+0000","last_became_active":"2023-11-08T08:20:33.845213+0000","last_became_peered":"2023-11-08T08:20:33.845213+0000","last_unstale":"2023-11-08T08:20:35.322741+0000","last_undegraded":"2023-11-08T08:20:35.322741+0000","last_fullsized":"2023-11-08T08:20:35.322741+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:08:17.687539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T08:20:33.826052+0000","last_change":"2023-11-08T08:18:58.770136+0000","last_active":"2023-11-08T08:20:33.826052+0000","last_peered":"2023-11-08T08:20:33.826052+0000","last_clean":"2023-11-08T08:20:33.826052+0000","last_became_active":"2023-11-08T08:18:58.769797+0000","last_became_peered":"2023-11-08T08:18:58.769797+0000","last_unstale":"2023-11-08T08:20:33.826052+0000","last_undegraded":"2023-11-08T08:20:33.826052+0000","last_fullsized":"2023-11-08T08:20:33.826052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:29:12.107128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.320158+0000","last_change":"2023-11-08T08:20:34.331889+0000","last_active":"2023-11-08T08:20:35.320158+0000","last_peered":"2023-11-08T08:20:35.320158+0000","last_clean":"2023-11-08T08:20:35.320158+0000","last_became_active":"2023-11-08T08:20:34.331526+0000","last_became_peered":"2023-11-08T08:20:34.331526+0000","last_unstale":"2023-11-08T08:20:35.320158+0000","last_undegraded":"2023-11-08T08:20:35.320158+0000","last_fullsized":"2023-11-08T08:20:35.320158+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:20:27.726248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.345234+0000","last_change":"2023-11-08T08:20:33.848145+0000","last_active":"2023-11-08T08:20:34.345234+0000","last_peered":"2023-11-08T08:20:34.345234+0000","last_clean":"2023-11-08T08:20:34.345234+0000","last_became_active":"2023-11-08T08:20:33.847694+0000","last_became_peered":"2023-11-08T08:20:33.847694+0000","last_unstale":"2023-11-08T08:20:34.345234+0000","last_undegraded":"2023-11-08T08:20:34.345234+0000","last_fullsized":"2023-11-08T08:20:34.345234+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:16:43.509880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T08:20:33.827610+0000","last_change":"2023-11-08T08:18:58.777140+0000","last_active":"2023-11-08T08:20:33.827610+0000","last_peered":"2023-11-08T08:20:33.827610+0000","last_clean":"2023-11-08T08:20:33.827610+0000","last_became_active":"2023-11-08T08:18:58.776834+0000","last_became_peered":"2023-11-08T08:18:58.776834+0000","last_unstale":"2023-11-08T08:20:33.827610+0000","last_undegraded":"2023-11-08T08:20:33.827610+0000","last_fullsized":"2023-11-08T08:20:33.827610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T08:20:33.826410+0000","last_change":"2023-11-08T08:18:53.780089+0000","last_active":"2023-11-08T08:20:33.826410+0000","last_peered":"2023-11-08T08:20:33.826410+0000","last_clean":"2023-11-08T08:20:33.826410+0000","last_became_active":"2023-11-08T08:18:53.779533+0000","last_became_peered":"2023-11-08T08:18:53.779533+0000","last_unstale":"2023-11-08T08:20:33.826410+0000","last_undegraded":"2023-11-08T08:20:33.826410+0000","last_fullsized":"2023-11-08T08:20:33.826410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_clean_scrub_stamp":"2023-11-08T08:18:52.744693+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:58:24.877070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9836,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362004,"statfs":{"total":96636764160,"available":96626692096,"internally_reserved":0,"allocated":1085440,"data_stored":806932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.341}]},{"osd":1,"last update":"Wed Nov 8 08:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.2869999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":3.6949999999999998,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":4.9089999999999998,"5min":4.9089999999999998,"15min":4.9089999999999998},"last":1.1850000000000001}]},{"osd":3,"last update":"Wed Nov 8 08:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.8199999999999998,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":1.431}]},{"osd":4,"last update":"Wed Nov 8 08:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.46},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":3.1360000000000001,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":1.5349999999999999}]},{"osd":5,"last update":"Wed Nov 8 08:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.73,"5min":2.73,"15min":2.73},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.399}]},{"osd":6,"last update":"Wed Nov 8 08:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.462,"15min":1.462},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.264},{"interface":"front","average":{"1min":1.462,"5min":1.462,"15min":1.462},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.4850000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8620,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363220,"statfs":{"total":96636764160,"available":96627937280,"internally_reserved":0,"allocated":495616,"data_stored":217130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.01}]},{"osd":1,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.1859999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.2749999999999999}]},{"osd":3,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.6539999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.3140000000000001}]},{"osd":5,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.476}]},{"osd":7,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":3.133,"5min":3.133,"15min":3.133},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":2.8180000000000001,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.7370000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8680,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363160,"statfs":{"total":96636764160,"available":96627875840,"internally_reserved":0,"allocated":491520,"data_stored":216544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.03}]},{"osd":1,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.212},{"interface":"front","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.89500000000000002}]},{"osd":2,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.173}]},{"osd":3,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":2.9660000000000002,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":3.7999999999999998,"5min":3.7999999999999998,"15min":3.7999999999999998},"last":0.96399999999999997}]},{"osd":4,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":3.073,"5min":3.073,"15min":3.073},"last":1.359}]},{"osd":6,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":3.8650000000000002,"5min":3.8650000000000002,"15min":3.8650000000000002},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.1020000000000001}]},{"osd":7,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.248},{"interface":"front","average":{"1min":1.52,"5min":1.52,"15min":1.52},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":3.9369999999999998,"5min":3.9369999999999998,"15min":3.9369999999999998},"last":0.86899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8816,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94363024,"statfs":{"total":96636764160,"available":96627736576,"internally_reserved":0,"allocated":499712,"data_stored":218022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.492},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":1.2569999999999999}]},{"osd":1,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.095}]},{"osd":2,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.3240000000000001}]},{"osd":3,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.278}]},{"osd":5,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.4740000000000002,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":1.373}]},{"osd":6,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.5960000000000001}]},{"osd":7,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.446,"15min":1.446},"min":{"1min":1.02,"5min":1.02,"15min":1.02},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.472},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":1.6599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8752,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363088,"statfs":{"total":96636764160,"available":96627802112,"internally_reserved":0,"allocated":499712,"data_stored":217096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.77700000000000002}]},{"osd":1,"last update":"Wed Nov 8 08:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.94999999999999996}]},{"osd":2,"last update":"Wed Nov 8 08:19:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.4569999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":0.86499999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:19:51 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.113},{"interface":"front","average":{"1min":1.415,"5min":1.415,"15min":1.415},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.88800000000000001}]},{"osd":5,"last update":"Wed Nov 8 08:19:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":2.2189999999999999,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.018},{"interface":"front","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.3610000000000002,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.1639999999999999}]},{"osd":6,"last update":"Wed Nov 8 08:19:51 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":1.008,"5min":1.008,"15min":1.008},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":1.222},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"min":{"1min":1.091,"5min":1.091,"15min":1.091},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":1.0580000000000001}]},{"osd":7,"last update":"Wed Nov 8 08:19:51 2023","interfaces":[{"interface":"back","average":{"1min":1.623,"5min":1.623,"15min":1.623},"min":{"1min":1.22,"5min":1.22,"15min":1.22},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":1.258},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"min":{"1min":1.123,"5min":1.123,"15min":1.123},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.03}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8640,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363200,"statfs":{"total":96636764160,"available":96627916800,"internally_reserved":0,"allocated":516096,"data_stored":225017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.77400000000000002}]},{"osd":1,"last update":"Wed Nov 8 08:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.115}]},{"osd":3,"last update":"Wed Nov 8 08:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.6099999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.747},{"interface":"front","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":1.2749999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"max":{"1min":2.7010000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.504}]},{"osd":5,"last update":"Wed Nov 8 08:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"max":{"1min":2.3279999999999998,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.24}]},{"osd":6,"last update":"Wed Nov 8 08:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":3.4350000000000001,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.363},{"interface":"front","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"max":{"1min":2.9060000000000001,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":1.3859999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"min":{"1min":1.091,"5min":1.091,"15min":1.091},"max":{"1min":2.867,"5min":2.867,"15min":2.867},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.1679999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8744,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363096,"statfs":{"total":96636764160,"available":96627810304,"internally_reserved":0,"allocated":491520,"data_stored":216544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.1479999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.872},{"interface":"front","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":3.0899999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.97399999999999998}]},{"osd":3,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.3570000000000002,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.0660000000000001}]},{"osd":4,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.228},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":1.034}]},{"osd":5,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.28},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":1.0009999999999999}]},{"osd":6,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.456,"15min":1.456},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.149,"5min":3.149,"15min":3.149},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":1.381}]},{"osd":7,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"max":{"1min":2.2320000000000002,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.3480000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9848,"kb_used_data":1072,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94361992,"statfs":{"total":96636764160,"available":96626679808,"internally_reserved":0,"allocated":1097728,"data_stored":813849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.56000000000000005}]},{"osd":2,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.745}]},{"osd":3,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.964,"5min":1.964,"15min":1.964},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":0.95799999999999996}]},{"osd":4,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"min":{"1min":1.083,"5min":1.083,"15min":1.083},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.2230000000000001}]},{"osd":5,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"min":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"max":{"1min":2.4449999999999998,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.234}]},{"osd":6,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.159},{"interface":"front","average":{"1min":1.607,"5min":1.607,"15min":1.607},"min":{"1min":1.034,"5min":1.034,"15min":1.034},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.99299999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"min":{"1min":1.24,"5min":1.24,"15min":1.24},"max":{"1min":3.0529999999999999,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"min":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"max":{"1min":3.2669999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":1.359}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T08:20:38.654 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T08:20:38.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.873+0000 7f437af18700 1 Processor -- start 2023-11-08T08:20:38.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.873+0000 7f437af18700 1 -- start start 2023-11-08T08:20:38.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.873+0000 7f437af18700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f43740c14e0 0x7f43740c1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:38.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.873+0000 7f437af18700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43740c9050 con 0x7f437415fb60 2023-11-08T08:20:38.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.873+0000 7f437af18700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43740cb970 con 0x7f43740ad5a0 2023-11-08T08:20:38.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.873+0000 7f437af18700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43740c1e90 con 0x7f43740c14e0 2023-11-08T08:20:38.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.873+0000 7f437a717700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f437415fb60 0x7f437415ffc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36764/0 (socket says 172.21.15.40:36764) 2023-11-08T08:20:38.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.873+0000 7f437a717700 1 -- 172.21.15.40:0/2247438389 learned_addr learned my addr 172.21.15.40:0/2247438389 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:38.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.873+0000 7f4379715700 1 --2- 172.21.15.40:0/2247438389 >> v2:172.21.15.169:3300/0 conn(0x7f43740c14e0 0x7f43740c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:38.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.877+0000 7f4378f14700 1 -- 172.21.15.40:0/2247438389 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3245725042 0 0) 0x7f43740c9050 con 0x7f437415fb60 2023-11-08T08:20:38.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.877+0000 7f4378f14700 1 -- 172.21.15.40:0/2247438389 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f435c0035d0 con 0x7f437415fb60 2023-11-08T08:20:38.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.877+0000 7f4379715700 1 -- 172.21.15.40:0/2247438389 >> v1:172.21.15.169:6789/0 conn(0x7f43740ad5a0 legacy=0x7f43740caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:38.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.877+0000 7f4378f14700 1 -- 172.21.15.40:0/2247438389 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925098843 0 0) 0x7f43740cb970 con 0x7f43740ad5a0 2023-11-08T08:20:38.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.877+0000 7f4379715700 1 -- 172.21.15.40:0/2247438389 >> v1:172.21.15.40:6789/0 conn(0x7f437415fb60 legacy=0x7f437415ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:38.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.877+0000 7f4379715700 1 -- 172.21.15.40:0/2247438389 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43740cbc50 con 0x7f43740c14e0 2023-11-08T08:20:38.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.877+0000 7f4379715700 1 --2- 172.21.15.40:0/2247438389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43740c14e0 0x7f43740c1950 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4368009850 tx=0x7f4368009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c78e7d00992ac0d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:38.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.877+0000 7f4378f14700 1 -- 172.21.15.40:0/2247438389 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4368011830 con 0x7f43740c14e0 2023-11-08T08:20:38.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.877+0000 7f4378f14700 1 -- 172.21.15.40:0/2247438389 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4368011990 con 0x7f43740c14e0 2023-11-08T08:20:38.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.877+0000 7f4378f14700 1 -- 172.21.15.40:0/2247438389 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4368011d40 con 0x7f43740c14e0 2023-11-08T08:20:38.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.877+0000 7f437af18700 1 -- 172.21.15.40:0/2247438389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43740c14e0 msgr2=0x7f43740c1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:38.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.877+0000 7f437af18700 1 --2- 172.21.15.40:0/2247438389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43740c14e0 0x7f43740c1950 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4368009850 tx=0x7f4368009b60 comp rx=0 tx=0).stop 2023-11-08T08:20:38.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.877+0000 7f437af18700 1 -- 172.21.15.40:0/2247438389 shutdown_connections 2023-11-08T08:20:38.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.877+0000 7f437af18700 1 --2- 172.21.15.40:0/2247438389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43740c14e0 0x7f43740c1950 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:38.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.877+0000 7f437af18700 1 -- 172.21.15.40:0/2247438389 >> 172.21.15.40:0/2247438389 conn(0x7f43740ce510 msgr2=0x7f43740ce910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:38.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.881+0000 7f437af18700 1 -- 172.21.15.40:0/2247438389 shutdown_connections 2023-11-08T08:20:38.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.881+0000 7f437af18700 1 -- 172.21.15.40:0/2247438389 wait complete. 2023-11-08T08:20:38.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.881+0000 7f437af18700 1 Processor -- start 2023-11-08T08:20:38.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.881+0000 7f437af18700 1 -- start start 2023-11-08T08:20:38.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f437af18700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43740ad5a0 0x7f43741541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:38.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f437af18700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43740c14e0 0x7f43740c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:38.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f437af18700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f437415fb60 0x7f43740c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:38.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f437af18700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43740c3110 con 0x7f43740ad5a0 2023-11-08T08:20:38.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f437af18700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43740c2b50 con 0x7f43740c14e0 2023-11-08T08:20:38.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f437af18700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43740c2e30 con 0x7f437415fb60 2023-11-08T08:20:38.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f4379f16700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43740ad5a0 0x7f43741541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:38.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f4379f16700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43740ad5a0 0x7f43741541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48770/0 (socket says 172.21.15.40:48770) 2023-11-08T08:20:38.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f4379f16700 1 -- 172.21.15.40:0/1026020869 learned_addr learned my addr 172.21.15.40:0/1026020869 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:38.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f4379715700 1 --2- 172.21.15.40:0/1026020869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43740c14e0 0x7f43740c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:38.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f437a717700 1 --2- 172.21.15.40:0/1026020869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f437415fb60 0x7f43740c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:38.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f4379f16700 1 -- 172.21.15.40:0/1026020869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f437415fb60 msgr2=0x7f43740c3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:38.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f4379f16700 1 --2- 172.21.15.40:0/1026020869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f437415fb60 0x7f43740c3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:38.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f4379f16700 1 -- 172.21.15.40:0/1026020869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43740c14e0 msgr2=0x7f43740c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:38.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f4379f16700 1 --2- 172.21.15.40:0/1026020869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43740c14e0 0x7f43740c35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:38.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f4379f16700 1 -- 172.21.15.40:0/1026020869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43680094b0 con 0x7f43740ad5a0 2023-11-08T08:20:38.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f4379f16700 1 --2- 172.21.15.40:0/1026020869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43740ad5a0 0x7f43741541d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f436400c550 tx=0x7f436400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:38.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f43727fc700 1 -- 172.21.15.40:0/1026020869 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43640054e0 con 0x7f43740ad5a0 2023-11-08T08:20:38.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f43727fc700 1 -- 172.21.15.40:0/1026020869 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43640036f0 con 0x7f43740ad5a0 2023-11-08T08:20:38.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f43727fc700 1 -- 172.21.15.40:0/1026020869 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4364003aa0 con 0x7f43740ad5a0 2023-11-08T08:20:38.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f437af18700 1 -- 172.21.15.40:0/1026020869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43740cb630 con 0x7f43740ad5a0 2023-11-08T08:20:38.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.885+0000 7f437af18700 1 -- 172.21.15.40:0/1026020869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43740c8ce0 con 0x7f43740ad5a0 2023-11-08T08:20:38.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.889+0000 7f437af18700 1 -- 172.21.15.40:0/1026020869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43740ad970 con 0x7f43740ad5a0 2023-11-08T08:20:38.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.889+0000 7f43727fc700 1 -- 172.21.15.40:0/1026020869 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4364019030 con 0x7f43740ad5a0 2023-11-08T08:20:38.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.889+0000 7f43727fc700 1 --2- 172.21.15.40:0/1026020869 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4360064ee0 0x7f4360067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:38.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.893+0000 7f43727fc700 1 -- 172.21.15.40:0/1026020869 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f43640812d0 con 0x7f43740ad5a0 2023-11-08T08:20:38.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.893+0000 7f4379715700 1 --2- 172.21.15.40:0/1026020869 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4360064ee0 0x7f4360067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:38.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.893+0000 7f4379715700 1 --2- 172.21.15.40:0/1026020869 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4360064ee0 0x7f4360067390 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f43680090b0 tx=0x7f4368008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:38.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:38.897+0000 7f43727fc700 1 -- 172.21.15.40:0/1026020869 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f436404f330 con 0x7f43740ad5a0 2023-11-08T08:20:39.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.073+0000 7f437af18700 1 -- 172.21.15.40:0/1026020869 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43740c6630 con 0x7f4360064ee0 2023-11-08T08:20:39.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.089+0000 7f43727fc700 1 -- 172.21.15.40:0/1026020869 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309006 (secure 0 0 0) 0x7f43740c6630 con 0x7f4360064ee0 2023-11-08T08:20:39.095 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:39.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.097+0000 7f437af18700 1 -- 172.21.15.40:0/1026020869 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4360064ee0 msgr2=0x7f4360067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:39.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.097+0000 7f437af18700 1 --2- 172.21.15.40:0/1026020869 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4360064ee0 0x7f4360067390 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f43680090b0 tx=0x7f4368008030 comp rx=0 tx=0).stop 2023-11-08T08:20:39.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.097+0000 7f437af18700 1 -- 172.21.15.40:0/1026020869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43740ad5a0 msgr2=0x7f43741541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:39.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.097+0000 7f437af18700 1 --2- 172.21.15.40:0/1026020869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43740ad5a0 0x7f43741541d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f436400c550 tx=0x7f436400c860 comp rx=0 tx=0).stop 2023-11-08T08:20:39.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.097+0000 7f437af18700 1 -- 172.21.15.40:0/1026020869 shutdown_connections 2023-11-08T08:20:39.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.097+0000 7f437af18700 1 --2- 172.21.15.40:0/1026020869 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4360064ee0 0x7f4360067390 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.097+0000 7f437af18700 1 --2- 172.21.15.40:0/1026020869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f437415fb60 0x7f43740c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.097+0000 7f437af18700 1 --2- 172.21.15.40:0/1026020869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43740c14e0 0x7f43740c35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.097+0000 7f437af18700 1 --2- 172.21.15.40:0/1026020869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43740ad5a0 0x7f43741541d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.097+0000 7f437af18700 1 -- 172.21.15.40:0/1026020869 >> 172.21.15.40:0/1026020869 conn(0x7f43740ce510 msgr2=0x7f4374069bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:39.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.097+0000 7f437af18700 1 -- 172.21.15.40:0/1026020869 shutdown_connections 2023-11-08T08:20:39.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.097+0000 7f437af18700 1 -- 172.21.15.40:0/1026020869 wait complete. 2023-11-08T08:20:39.106 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-11-08T08:20:39.125 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2023-11-08T08:20:37.916687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":71936,"kb_used_data":5056,"kb_used_omap":0,"kb_used_meta":66816,"kb_avail":754902784,"statfs":{"total":773094113280,"available":773020450816,"internally_reserved":0,"allocated":5177344,"data_stored":2931134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68419584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.108352"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.341942+0000","last_change":"2023-11-08T08:20:34.330057+0000","last_active":"2023-11-08T08:20:34.341942+0000","last_peered":"2023-11-08T08:20:34.341942+0000","last_clean":"2023-11-08T08:20:34.341942+0000","last_became_active":"2023-11-08T08:20:34.325719+0000","last_became_peered":"2023-11-08T08:20:34.325719+0000","last_unstale":"2023-11-08T08:20:34.341942+0000","last_undegraded":"2023-11-08T08:20:34.341942+0000","last_fullsized":"2023-11-08T08:20:34.341942+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:56:45.933503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.337783+0000","last_change":"2023-11-08T08:20:34.328948+0000","last_active":"2023-11-08T08:20:34.337783+0000","last_peered":"2023-11-08T08:20:34.337783+0000","last_clean":"2023-11-08T08:20:34.337783+0000","last_became_active":"2023-11-08T08:20:34.328182+0000","last_became_peered":"2023-11-08T08:20:34.328182+0000","last_unstale":"2023-11-08T08:20:34.337783+0000","last_undegraded":"2023-11-08T08:20:34.337783+0000","last_fullsized":"2023-11-08T08:20:34.337783+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:20:27.726248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447461+0000","last_change":"2023-11-08T08:20:34.330263+0000","last_active":"2023-11-08T08:20:35.447461+0000","last_peered":"2023-11-08T08:20:35.447461+0000","last_clean":"2023-11-08T08:20:35.447461+0000","last_became_active":"2023-11-08T08:20:34.328048+0000","last_became_peered":"2023-11-08T08:20:34.328048+0000","last_unstale":"2023-11-08T08:20:35.447461+0000","last_undegraded":"2023-11-08T08:20:35.447461+0000","last_fullsized":"2023-11-08T08:20:35.447461+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:19:44.058497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319582+0000","last_change":"2023-11-08T08:20:34.332656+0000","last_active":"2023-11-08T08:20:35.319582+0000","last_peered":"2023-11-08T08:20:35.319582+0000","last_clean":"2023-11-08T08:20:35.319582+0000","last_became_active":"2023-11-08T08:20:34.332270+0000","last_became_peered":"2023-11-08T08:20:34.332270+0000","last_unstale":"2023-11-08T08:20:35.319582+0000","last_undegraded":"2023-11-08T08:20:35.319582+0000","last_fullsized":"2023-11-08T08:20:35.319582+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:39:57.195619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.331308+0000","last_change":"2023-11-08T08:20:34.323639+0000","last_active":"2023-11-08T08:20:34.331308+0000","last_peered":"2023-11-08T08:20:34.331308+0000","last_clean":"2023-11-08T08:20:34.331308+0000","last_became_active":"2023-11-08T08:20:34.323369+0000","last_became_peered":"2023-11-08T08:20:34.323369+0000","last_unstale":"2023-11-08T08:20:34.331308+0000","last_undegraded":"2023-11-08T08:20:34.331308+0000","last_fullsized":"2023-11-08T08:20:34.331308+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:09:07.316196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439468+0000","last_change":"2023-11-08T08:20:34.349152+0000","last_active":"2023-11-08T08:20:35.439468+0000","last_peered":"2023-11-08T08:20:35.439468+0000","last_clean":"2023-11-08T08:20:35.439468+0000","last_became_active":"2023-11-08T08:20:34.348378+0000","last_became_peered":"2023-11-08T08:20:34.348378+0000","last_unstale":"2023-11-08T08:20:35.439468+0000","last_undegraded":"2023-11-08T08:20:35.439468+0000","last_fullsized":"2023-11-08T08:20:35.439468+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:32:02.079767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.341074+0000","last_change":"2023-11-08T08:20:34.332470+0000","last_active":"2023-11-08T08:20:34.341074+0000","last_peered":"2023-11-08T08:20:34.341074+0000","last_clean":"2023-11-08T08:20:34.341074+0000","last_became_active":"2023-11-08T08:20:34.332186+0000","last_became_peered":"2023-11-08T08:20:34.332186+0000","last_unstale":"2023-11-08T08:20:34.341074+0000","last_undegraded":"2023-11-08T08:20:34.341074+0000","last_fullsized":"2023-11-08T08:20:34.341074+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:43:49.374493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.318943+0000","last_change":"2023-11-08T08:20:34.334000+0000","last_active":"2023-11-08T08:20:35.318943+0000","last_peered":"2023-11-08T08:20:35.318943+0000","last_clean":"2023-11-08T08:20:35.318943+0000","last_became_active":"2023-11-08T08:20:34.333735+0000","last_became_peered":"2023-11-08T08:20:34.333735+0000","last_unstale":"2023-11-08T08:20:35.318943+0000","last_undegraded":"2023-11-08T08:20:35.318943+0000","last_fullsized":"2023-11-08T08:20:35.318943+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:24:12.113358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319969+0000","last_change":"2023-11-08T08:20:34.340403+0000","last_active":"2023-11-08T08:20:35.319969+0000","last_peered":"2023-11-08T08:20:35.319969+0000","last_clean":"2023-11-08T08:20:35.319969+0000","last_became_active":"2023-11-08T08:20:34.339070+0000","last_became_peered":"2023-11-08T08:20:34.339070+0000","last_unstale":"2023-11-08T08:20:35.319969+0000","last_undegraded":"2023-11-08T08:20:35.319969+0000","last_fullsized":"2023-11-08T08:20:35.319969+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:30:43.080172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319976+0000","last_change":"2023-11-08T08:20:34.333087+0000","last_active":"2023-11-08T08:20:35.319976+0000","last_peered":"2023-11-08T08:20:35.319976+0000","last_clean":"2023-11-08T08:20:35.319976+0000","last_became_active":"2023-11-08T08:20:34.332825+0000","last_became_peered":"2023-11-08T08:20:34.332825+0000","last_unstale":"2023-11-08T08:20:35.319976+0000","last_undegraded":"2023-11-08T08:20:35.319976+0000","last_fullsized":"2023-11-08T08:20:35.319976+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:31:59.165324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319686+0000","last_change":"2023-11-08T08:20:34.329030+0000","last_active":"2023-11-08T08:20:35.319686+0000","last_peered":"2023-11-08T08:20:35.319686+0000","last_clean":"2023-11-08T08:20:35.319686+0000","last_became_active":"2023-11-08T08:20:34.328724+0000","last_became_peered":"2023-11-08T08:20:34.328724+0000","last_unstale":"2023-11-08T08:20:35.319686+0000","last_undegraded":"2023-11-08T08:20:35.319686+0000","last_fullsized":"2023-11-08T08:20:35.319686+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:50:55.369713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.343229+0000","last_change":"2023-11-08T08:20:34.328800+0000","last_active":"2023-11-08T08:20:34.343229+0000","last_peered":"2023-11-08T08:20:34.343229+0000","last_clean":"2023-11-08T08:20:34.343229+0000","last_became_active":"2023-11-08T08:20:34.328529+0000","last_became_peered":"2023-11-08T08:20:34.328529+0000","last_unstale":"2023-11-08T08:20:34.343229+0000","last_undegraded":"2023-11-08T08:20:34.343229+0000","last_fullsized":"2023-11-08T08:20:34.343229+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:56:03.805095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.321268+0000","last_change":"2023-11-08T08:20:34.335268+0000","last_active":"2023-11-08T08:20:35.321268+0000","last_peered":"2023-11-08T08:20:35.321268+0000","last_clean":"2023-11-08T08:20:35.321268+0000","last_became_active":"2023-11-08T08:20:34.335024+0000","last_became_peered":"2023-11-08T08:20:34.335024+0000","last_unstale":"2023-11-08T08:20:35.321268+0000","last_undegraded":"2023-11-08T08:20:35.321268+0000","last_fullsized":"2023-11-08T08:20:35.321268+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:28:23.771501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319397+0000","last_change":"2023-11-08T08:20:34.328937+0000","last_active":"2023-11-08T08:20:35.319397+0000","last_peered":"2023-11-08T08:20:35.319397+0000","last_clean":"2023-11-08T08:20:35.319397+0000","last_became_active":"2023-11-08T08:20:34.328625+0000","last_became_peered":"2023-11-08T08:20:34.328625+0000","last_unstale":"2023-11-08T08:20:35.319397+0000","last_undegraded":"2023-11-08T08:20:35.319397+0000","last_fullsized":"2023-11-08T08:20:35.319397+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:32:02.079767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439273+0000","last_change":"2023-11-08T08:20:34.336520+0000","last_active":"2023-11-08T08:20:35.439273+0000","last_peered":"2023-11-08T08:20:35.439273+0000","last_clean":"2023-11-08T08:20:35.439273+0000","last_became_active":"2023-11-08T08:20:34.336251+0000","last_became_peered":"2023-11-08T08:20:34.336251+0000","last_unstale":"2023-11-08T08:20:35.439273+0000","last_undegraded":"2023-11-08T08:20:35.439273+0000","last_fullsized":"2023-11-08T08:20:35.439273+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:15:24.368062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.346936+0000","last_change":"2023-11-08T08:20:34.340453+0000","last_active":"2023-11-08T08:20:34.346936+0000","last_peered":"2023-11-08T08:20:34.346936+0000","last_clean":"2023-11-08T08:20:34.346936+0000","last_became_active":"2023-11-08T08:20:34.340110+0000","last_became_peered":"2023-11-08T08:20:34.340110+0000","last_unstale":"2023-11-08T08:20:34.346936+0000","last_undegraded":"2023-11-08T08:20:34.346936+0000","last_fullsized":"2023-11-08T08:20:34.346936+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:48:42.113335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447809+0000","last_change":"2023-11-08T08:20:34.333209+0000","last_active":"2023-11-08T08:20:35.447809+0000","last_peered":"2023-11-08T08:20:35.447809+0000","last_clean":"2023-11-08T08:20:35.447809+0000","last_became_active":"2023-11-08T08:20:34.332899+0000","last_became_peered":"2023-11-08T08:20:34.332899+0000","last_unstale":"2023-11-08T08:20:35.447809+0000","last_undegraded":"2023-11-08T08:20:35.447809+0000","last_fullsized":"2023-11-08T08:20:35.447809+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:58:23.617005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447721+0000","last_change":"2023-11-08T08:20:34.327381+0000","last_active":"2023-11-08T08:20:35.447721+0000","last_peered":"2023-11-08T08:20:35.447721+0000","last_clean":"2023-11-08T08:20:35.447721+0000","last_became_active":"2023-11-08T08:20:34.325540+0000","last_became_peered":"2023-11-08T08:20:34.325540+0000","last_unstale":"2023-11-08T08:20:35.447721+0000","last_undegraded":"2023-11-08T08:20:35.447721+0000","last_fullsized":"2023-11-08T08:20:35.447721+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:32:02.079767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.348976+0000","last_change":"2023-11-08T08:20:34.342478+0000","last_active":"2023-11-08T08:20:34.348976+0000","last_peered":"2023-11-08T08:20:34.348976+0000","last_clean":"2023-11-08T08:20:34.348976+0000","last_became_active":"2023-11-08T08:20:34.341501+0000","last_became_peered":"2023-11-08T08:20:34.341501+0000","last_unstale":"2023-11-08T08:20:34.348976+0000","last_undegraded":"2023-11-08T08:20:34.348976+0000","last_fullsized":"2023-11-08T08:20:34.348976+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:16:14.114658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.321676+0000","last_change":"2023-11-08T08:20:34.321816+0000","last_active":"2023-11-08T08:20:35.321676+0000","last_peered":"2023-11-08T08:20:35.321676+0000","last_clean":"2023-11-08T08:20:35.321676+0000","last_became_active":"2023-11-08T08:20:34.321573+0000","last_became_peered":"2023-11-08T08:20:34.321573+0000","last_unstale":"2023-11-08T08:20:35.321676+0000","last_undegraded":"2023-11-08T08:20:35.321676+0000","last_fullsized":"2023-11-08T08:20:35.321676+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:27:55.895840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439573+0000","last_change":"2023-11-08T08:20:34.340948+0000","last_active":"2023-11-08T08:20:35.439573+0000","last_peered":"2023-11-08T08:20:35.439573+0000","last_clean":"2023-11-08T08:20:35.439573+0000","last_became_active":"2023-11-08T08:20:34.339080+0000","last_became_peered":"2023-11-08T08:20:34.339080+0000","last_unstale":"2023-11-08T08:20:35.439573+0000","last_undegraded":"2023-11-08T08:20:35.439573+0000","last_fullsized":"2023-11-08T08:20:35.439573+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:47:19.286387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319457+0000","last_change":"2023-11-08T08:20:34.333230+0000","last_active":"2023-11-08T08:20:35.319457+0000","last_peered":"2023-11-08T08:20:35.319457+0000","last_clean":"2023-11-08T08:20:35.319457+0000","last_became_active":"2023-11-08T08:20:34.332947+0000","last_became_peered":"2023-11-08T08:20:34.332947+0000","last_unstale":"2023-11-08T08:20:35.319457+0000","last_undegraded":"2023-11-08T08:20:35.319457+0000","last_fullsized":"2023-11-08T08:20:35.319457+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:47:19.286387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.349339+0000","last_change":"2023-11-08T08:20:34.343766+0000","last_active":"2023-11-08T08:20:34.349339+0000","last_peered":"2023-11-08T08:20:34.349339+0000","last_clean":"2023-11-08T08:20:34.349339+0000","last_became_active":"2023-11-08T08:20:34.343324+0000","last_became_peered":"2023-11-08T08:20:34.343324+0000","last_unstale":"2023-11-08T08:20:34.349339+0000","last_undegraded":"2023-11-08T08:20:34.349339+0000","last_fullsized":"2023-11-08T08:20:34.349339+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:27:55.895840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.346958+0000","last_change":"2023-11-08T08:20:34.341041+0000","last_active":"2023-11-08T08:20:34.346958+0000","last_peered":"2023-11-08T08:20:34.346958+0000","last_clean":"2023-11-08T08:20:34.346958+0000","last_became_active":"2023-11-08T08:20:34.340736+0000","last_became_peered":"2023-11-08T08:20:34.340736+0000","last_unstale":"2023-11-08T08:20:34.346958+0000","last_undegraded":"2023-11-08T08:20:34.346958+0000","last_fullsized":"2023-11-08T08:20:34.346958+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:08:18.698224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.345985+0000","last_change":"2023-11-08T08:20:34.338102+0000","last_active":"2023-11-08T08:20:34.345985+0000","last_peered":"2023-11-08T08:20:34.345985+0000","last_clean":"2023-11-08T08:20:34.345985+0000","last_became_active":"2023-11-08T08:20:34.337750+0000","last_became_peered":"2023-11-08T08:20:34.337750+0000","last_unstale":"2023-11-08T08:20:34.345985+0000","last_undegraded":"2023-11-08T08:20:34.345985+0000","last_fullsized":"2023-11-08T08:20:34.345985+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:31:38.135691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.321738+0000","last_change":"2023-11-08T08:20:34.335934+0000","last_active":"2023-11-08T08:20:35.321738+0000","last_peered":"2023-11-08T08:20:35.321738+0000","last_clean":"2023-11-08T08:20:35.321738+0000","last_became_active":"2023-11-08T08:20:34.335631+0000","last_became_peered":"2023-11-08T08:20:34.335631+0000","last_unstale":"2023-11-08T08:20:35.321738+0000","last_undegraded":"2023-11-08T08:20:35.321738+0000","last_fullsized":"2023-11-08T08:20:35.321738+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:31:38.135691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447591+0000","last_change":"2023-11-08T08:20:34.332562+0000","last_active":"2023-11-08T08:20:35.447591+0000","last_peered":"2023-11-08T08:20:35.447591+0000","last_clean":"2023-11-08T08:20:35.447591+0000","last_became_active":"2023-11-08T08:20:34.332260+0000","last_became_peered":"2023-11-08T08:20:34.332260+0000","last_unstale":"2023-11-08T08:20:35.447591+0000","last_undegraded":"2023-11-08T08:20:35.447591+0000","last_fullsized":"2023-11-08T08:20:35.447591+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:09.194290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.330349+0000","last_change":"2023-11-08T08:20:34.323169+0000","last_active":"2023-11-08T08:20:34.330349+0000","last_peered":"2023-11-08T08:20:34.330349+0000","last_clean":"2023-11-08T08:20:34.330349+0000","last_became_active":"2023-11-08T08:20:34.322681+0000","last_became_peered":"2023-11-08T08:20:34.322681+0000","last_unstale":"2023-11-08T08:20:34.330349+0000","last_undegraded":"2023-11-08T08:20:34.330349+0000","last_fullsized":"2023-11-08T08:20:34.330349+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:31:59.165324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.344473+0000","last_change":"2023-11-08T08:20:34.335087+0000","last_active":"2023-11-08T08:20:34.344473+0000","last_peered":"2023-11-08T08:20:34.344473+0000","last_clean":"2023-11-08T08:20:34.344473+0000","last_became_active":"2023-11-08T08:20:34.334780+0000","last_became_peered":"2023-11-08T08:20:34.334780+0000","last_unstale":"2023-11-08T08:20:34.344473+0000","last_undegraded":"2023-11-08T08:20:34.344473+0000","last_fullsized":"2023-11-08T08:20:34.344473+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:19:14.805477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447340+0000","last_change":"2023-11-08T08:20:34.333075+0000","last_active":"2023-11-08T08:20:35.447340+0000","last_peered":"2023-11-08T08:20:35.447340+0000","last_clean":"2023-11-08T08:20:35.447340+0000","last_became_active":"2023-11-08T08:20:34.332803+0000","last_became_peered":"2023-11-08T08:20:34.332803+0000","last_unstale":"2023-11-08T08:20:35.447340+0000","last_undegraded":"2023-11-08T08:20:35.447340+0000","last_fullsized":"2023-11-08T08:20:35.447340+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:47:19.286387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439279+0000","last_change":"2023-11-08T08:20:34.346083+0000","last_active":"2023-11-08T08:20:35.439279+0000","last_peered":"2023-11-08T08:20:35.439279+0000","last_clean":"2023-11-08T08:20:35.439279+0000","last_became_active":"2023-11-08T08:20:34.344329+0000","last_became_peered":"2023-11-08T08:20:34.344329+0000","last_unstale":"2023-11-08T08:20:35.439279+0000","last_undegraded":"2023-11-08T08:20:35.439279+0000","last_fullsized":"2023-11-08T08:20:35.439279+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:19:44.058497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.321305+0000","last_change":"2023-11-08T08:20:34.331102+0000","last_active":"2023-11-08T08:20:35.321305+0000","last_peered":"2023-11-08T08:20:35.321305+0000","last_clean":"2023-11-08T08:20:35.321305+0000","last_became_active":"2023-11-08T08:20:34.329730+0000","last_became_peered":"2023-11-08T08:20:34.329730+0000","last_unstale":"2023-11-08T08:20:35.321305+0000","last_undegraded":"2023-11-08T08:20:35.321305+0000","last_fullsized":"2023-11-08T08:20:35.321305+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:26:18.631300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.323046+0000","last_change":"2023-11-08T08:20:34.335051+0000","last_active":"2023-11-08T08:20:35.323046+0000","last_peered":"2023-11-08T08:20:35.323046+0000","last_clean":"2023-11-08T08:20:35.323046+0000","last_became_active":"2023-11-08T08:20:34.334797+0000","last_became_peered":"2023-11-08T08:20:34.334797+0000","last_unstale":"2023-11-08T08:20:35.323046+0000","last_undegraded":"2023-11-08T08:20:35.323046+0000","last_fullsized":"2023-11-08T08:20:35.323046+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:19:14.805477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"42'4","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.440339+0000","last_change":"2023-11-08T08:20:33.848558+0000","last_active":"2023-11-08T08:20:35.440339+0000","last_peered":"2023-11-08T08:20:35.440339+0000","last_clean":"2023-11-08T08:20:35.440339+0000","last_became_active":"2023-11-08T08:20:33.848277+0000","last_became_peered":"2023-11-08T08:20:33.848277+0000","last_unstale":"2023-11-08T08:20:35.440339+0000","last_undegraded":"2023-11-08T08:20:35.440339+0000","last_fullsized":"2023-11-08T08:20:35.440339+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:16:43.509880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.340488+0000","last_change":"2023-11-08T08:20:34.332475+0000","last_active":"2023-11-08T08:20:34.340488+0000","last_peered":"2023-11-08T08:20:34.340488+0000","last_clean":"2023-11-08T08:20:34.340488+0000","last_became_active":"2023-11-08T08:20:34.331733+0000","last_became_peered":"2023-11-08T08:20:34.331733+0000","last_unstale":"2023-11-08T08:20:34.340488+0000","last_undegraded":"2023-11-08T08:20:34.340488+0000","last_fullsized":"2023-11-08T08:20:34.340488+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:00:34.711344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447905+0000","last_change":"2023-11-08T08:20:33.850421+0000","last_active":"2023-11-08T08:20:35.447905+0000","last_peered":"2023-11-08T08:20:35.447905+0000","last_clean":"2023-11-08T08:20:35.447905+0000","last_became_active":"2023-11-08T08:20:33.850058+0000","last_became_peered":"2023-11-08T08:20:33.850058+0000","last_unstale":"2023-11-08T08:20:35.447905+0000","last_undegraded":"2023-11-08T08:20:35.447905+0000","last_fullsized":"2023-11-08T08:20:35.447905+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:05:48.739190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.348909+0000","last_change":"2023-11-08T08:20:34.345991+0000","last_active":"2023-11-08T08:20:34.348909+0000","last_peered":"2023-11-08T08:20:34.348909+0000","last_clean":"2023-11-08T08:20:34.348909+0000","last_became_active":"2023-11-08T08:20:34.345699+0000","last_became_peered":"2023-11-08T08:20:34.345699+0000","last_unstale":"2023-11-08T08:20:34.348909+0000","last_undegraded":"2023-11-08T08:20:34.348909+0000","last_fullsized":"2023-11-08T08:20:34.348909+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:39:57.195619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.323557+0000","last_change":"2023-11-08T08:20:33.841990+0000","last_active":"2023-11-08T08:20:35.323557+0000","last_peered":"2023-11-08T08:20:35.323557+0000","last_clean":"2023-11-08T08:20:35.323557+0000","last_became_active":"2023-11-08T08:20:33.841474+0000","last_became_peered":"2023-11-08T08:20:33.841474+0000","last_unstale":"2023-11-08T08:20:35.323557+0000","last_undegraded":"2023-11-08T08:20:35.323557+0000","last_fullsized":"2023-11-08T08:20:35.323557+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:48:41.102650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.323297+0000","last_change":"2023-11-08T08:20:34.330319+0000","last_active":"2023-11-08T08:20:35.323297+0000","last_peered":"2023-11-08T08:20:35.323297+0000","last_clean":"2023-11-08T08:20:35.323297+0000","last_became_active":"2023-11-08T08:20:34.329960+0000","last_became_peered":"2023-11-08T08:20:34.329960+0000","last_unstale":"2023-11-08T08:20:35.323297+0000","last_undegraded":"2023-11-08T08:20:35.323297+0000","last_fullsized":"2023-11-08T08:20:35.323297+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:13:05.557780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.349312+0000","last_change":"2023-11-08T08:20:33.843611+0000","last_active":"2023-11-08T08:20:34.349312+0000","last_peered":"2023-11-08T08:20:34.349312+0000","last_clean":"2023-11-08T08:20:34.349312+0000","last_became_active":"2023-11-08T08:20:33.843232+0000","last_became_peered":"2023-11-08T08:20:33.843232+0000","last_unstale":"2023-11-08T08:20:34.349312+0000","last_undegraded":"2023-11-08T08:20:34.349312+0000","last_fullsized":"2023-11-08T08:20:34.349312+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:26:17.620615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447389+0000","last_change":"2023-11-08T08:20:34.335134+0000","last_active":"2023-11-08T08:20:35.447389+0000","last_peered":"2023-11-08T08:20:35.447389+0000","last_clean":"2023-11-08T08:20:35.447389+0000","last_became_active":"2023-11-08T08:20:34.334843+0000","last_became_peered":"2023-11-08T08:20:34.334843+0000","last_unstale":"2023-11-08T08:20:35.447389+0000","last_undegraded":"2023-11-08T08:20:35.447389+0000","last_fullsized":"2023-11-08T08:20:35.447389+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:30:43.080172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439799+0000","last_change":"2023-11-08T08:20:33.849373+0000","last_active":"2023-11-08T08:20:35.439799+0000","last_peered":"2023-11-08T08:20:35.439799+0000","last_clean":"2023-11-08T08:20:35.439799+0000","last_became_active":"2023-11-08T08:20:33.849121+0000","last_became_peered":"2023-11-08T08:20:33.849121+0000","last_unstale":"2023-11-08T08:20:35.439799+0000","last_undegraded":"2023-11-08T08:20:35.439799+0000","last_fullsized":"2023-11-08T08:20:35.439799+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:09:01.178114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.320223+0000","last_change":"2023-11-08T08:20:34.320058+0000","last_active":"2023-11-08T08:20:35.320223+0000","last_peered":"2023-11-08T08:20:35.320223+0000","last_clean":"2023-11-08T08:20:35.320223+0000","last_became_active":"2023-11-08T08:20:34.319262+0000","last_became_peered":"2023-11-08T08:20:34.319262+0000","last_unstale":"2023-11-08T08:20:35.320223+0000","last_undegraded":"2023-11-08T08:20:35.320223+0000","last_fullsized":"2023-11-08T08:20:35.320223+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:15:24.368062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447674+0000","last_change":"2023-11-08T08:20:33.850496+0000","last_active":"2023-11-08T08:20:35.447674+0000","last_peered":"2023-11-08T08:20:35.447674+0000","last_clean":"2023-11-08T08:20:35.447674+0000","last_became_active":"2023-11-08T08:20:33.850043+0000","last_became_peered":"2023-11-08T08:20:33.850043+0000","last_unstale":"2023-11-08T08:20:35.447674+0000","last_undegraded":"2023-11-08T08:20:35.447674+0000","last_fullsized":"2023-11-08T08:20:35.447674+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:11:10.537633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.318918+0000","last_change":"2023-11-08T08:20:34.331941+0000","last_active":"2023-11-08T08:20:35.318918+0000","last_peered":"2023-11-08T08:20:35.318918+0000","last_clean":"2023-11-08T08:20:35.318918+0000","last_became_active":"2023-11-08T08:20:34.331659+0000","last_became_peered":"2023-11-08T08:20:34.331659+0000","last_unstale":"2023-11-08T08:20:35.318918+0000","last_undegraded":"2023-11-08T08:20:35.318918+0000","last_fullsized":"2023-11-08T08:20:35.318918+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:19:44.058497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.310500+0000","last_change":"2023-11-08T08:20:33.845051+0000","last_active":"2023-11-08T08:20:34.310500+0000","last_peered":"2023-11-08T08:20:34.310500+0000","last_clean":"2023-11-08T08:20:34.310500+0000","last_became_active":"2023-11-08T08:20:33.844642+0000","last_became_peered":"2023-11-08T08:20:33.844642+0000","last_unstale":"2023-11-08T08:20:34.310500+0000","last_undegraded":"2023-11-08T08:20:34.310500+0000","last_fullsized":"2023-11-08T08:20:34.310500+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:09:46.694297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.349316+0000","last_change":"2023-11-08T08:20:34.343702+0000","last_active":"2023-11-08T08:20:34.349316+0000","last_peered":"2023-11-08T08:20:34.349316+0000","last_clean":"2023-11-08T08:20:34.349316+0000","last_became_active":"2023-11-08T08:20:34.343330+0000","last_became_peered":"2023-11-08T08:20:34.343330+0000","last_unstale":"2023-11-08T08:20:34.349316+0000","last_undegraded":"2023-11-08T08:20:34.349316+0000","last_fullsized":"2023-11-08T08:20:34.349316+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:28:23.771501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319001+0000","last_change":"2023-11-08T08:20:33.839789+0000","last_active":"2023-11-08T08:20:35.319001+0000","last_peered":"2023-11-08T08:20:35.319001+0000","last_clean":"2023-11-08T08:20:35.319001+0000","last_became_active":"2023-11-08T08:20:33.839489+0000","last_became_peered":"2023-11-08T08:20:33.839489+0000","last_unstale":"2023-11-08T08:20:35.319001+0000","last_undegraded":"2023-11-08T08:20:35.319001+0000","last_fullsized":"2023-11-08T08:20:35.319001+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:13:04.547095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447814+0000","last_change":"2023-11-08T08:20:34.325628+0000","last_active":"2023-11-08T08:20:35.447814+0000","last_peered":"2023-11-08T08:20:35.447814+0000","last_clean":"2023-11-08T08:20:35.447814+0000","last_became_active":"2023-11-08T08:20:34.325059+0000","last_became_peered":"2023-11-08T08:20:34.325059+0000","last_unstale":"2023-11-08T08:20:35.447814+0000","last_undegraded":"2023-11-08T08:20:35.447814+0000","last_fullsized":"2023-11-08T08:20:35.447814+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:15:24.368062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.448374+0000","last_change":"2023-11-08T08:20:33.846230+0000","last_active":"2023-11-08T08:20:35.448374+0000","last_peered":"2023-11-08T08:20:35.448374+0000","last_clean":"2023-11-08T08:20:35.448374+0000","last_became_active":"2023-11-08T08:20:33.845766+0000","last_became_peered":"2023-11-08T08:20:33.845766+0000","last_unstale":"2023-11-08T08:20:35.448374+0000","last_undegraded":"2023-11-08T08:20:35.448374+0000","last_fullsized":"2023-11-08T08:20:35.448374+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:13:00.589562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.341972+0000","last_change":"2023-11-08T08:20:34.337217+0000","last_active":"2023-11-08T08:20:34.341972+0000","last_peered":"2023-11-08T08:20:34.341972+0000","last_clean":"2023-11-08T08:20:34.341972+0000","last_became_active":"2023-11-08T08:20:34.336944+0000","last_became_peered":"2023-11-08T08:20:34.336944+0000","last_unstale":"2023-11-08T08:20:34.341972+0000","last_undegraded":"2023-11-08T08:20:34.341972+0000","last_fullsized":"2023-11-08T08:20:34.341972+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:09:47.704982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.320332+0000","last_change":"2023-11-08T08:20:33.832698+0000","last_active":"2023-11-08T08:20:35.320332+0000","last_peered":"2023-11-08T08:20:35.320332+0000","last_clean":"2023-11-08T08:20:35.320332+0000","last_became_active":"2023-11-08T08:20:33.832414+0000","last_became_peered":"2023-11-08T08:20:33.832414+0000","last_unstale":"2023-11-08T08:20:35.320332+0000","last_undegraded":"2023-11-08T08:20:35.320332+0000","last_fullsized":"2023-11-08T08:20:35.320332+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:11:41.800061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.440093+0000","last_change":"2023-11-08T08:20:34.347100+0000","last_active":"2023-11-08T08:20:35.440093+0000","last_peered":"2023-11-08T08:20:35.440093+0000","last_clean":"2023-11-08T08:20:35.440093+0000","last_became_active":"2023-11-08T08:20:34.346768+0000","last_became_peered":"2023-11-08T08:20:34.346768+0000","last_unstale":"2023-11-08T08:20:35.440093+0000","last_undegraded":"2023-11-08T08:20:35.440093+0000","last_fullsized":"2023-11-08T08:20:35.440093+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:39:57.195619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.349553+0000","last_change":"2023-11-08T08:20:33.849553+0000","last_active":"2023-11-08T08:20:34.349553+0000","last_peered":"2023-11-08T08:20:34.349553+0000","last_clean":"2023-11-08T08:20:34.349553+0000","last_became_active":"2023-11-08T08:20:33.849304+0000","last_became_peered":"2023-11-08T08:20:33.849304+0000","last_unstale":"2023-11-08T08:20:34.349553+0000","last_undegraded":"2023-11-08T08:20:34.349553+0000","last_fullsized":"2023-11-08T08:20:34.349553+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:19:13.794792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.344007+0000","last_change":"2023-11-08T08:20:34.329202+0000","last_active":"2023-11-08T08:20:34.344007+0000","last_peered":"2023-11-08T08:20:34.344007+0000","last_clean":"2023-11-08T08:20:34.344007+0000","last_became_active":"2023-11-08T08:20:34.328557+0000","last_became_peered":"2023-11-08T08:20:34.328557+0000","last_unstale":"2023-11-08T08:20:34.344007+0000","last_undegraded":"2023-11-08T08:20:34.344007+0000","last_fullsized":"2023-11-08T08:20:34.344007+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:13:01.600247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.321824+0000","last_change":"2023-11-08T08:20:33.844322+0000","last_active":"2023-11-08T08:20:35.321824+0000","last_peered":"2023-11-08T08:20:35.321824+0000","last_clean":"2023-11-08T08:20:35.321824+0000","last_became_active":"2023-11-08T08:20:33.844034+0000","last_became_peered":"2023-11-08T08:20:33.844034+0000","last_unstale":"2023-11-08T08:20:35.321824+0000","last_undegraded":"2023-11-08T08:20:35.321824+0000","last_fullsized":"2023-11-08T08:20:35.321824+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:46:19.490236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.349165+0000","last_change":"2023-11-08T08:20:34.344556+0000","last_active":"2023-11-08T08:20:34.349165+0000","last_peered":"2023-11-08T08:20:34.349165+0000","last_clean":"2023-11-08T08:20:34.349165+0000","last_became_active":"2023-11-08T08:20:34.344252+0000","last_became_peered":"2023-11-08T08:20:34.344252+0000","last_unstale":"2023-11-08T08:20:34.349165+0000","last_undegraded":"2023-11-08T08:20:34.349165+0000","last_fullsized":"2023-11-08T08:20:34.349165+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:13:05.557780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439895+0000","last_change":"2023-11-08T08:20:33.845912+0000","last_active":"2023-11-08T08:20:35.439895+0000","last_peered":"2023-11-08T08:20:35.439895+0000","last_clean":"2023-11-08T08:20:35.439895+0000","last_became_active":"2023-11-08T08:20:33.845639+0000","last_became_peered":"2023-11-08T08:20:33.845639+0000","last_unstale":"2023-11-08T08:20:35.439895+0000","last_undegraded":"2023-11-08T08:20:35.439895+0000","last_fullsized":"2023-11-08T08:20:35.439895+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:19:12.865432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.320314+0000","last_change":"2023-11-08T08:20:34.331437+0000","last_active":"2023-11-08T08:20:35.320314+0000","last_peered":"2023-11-08T08:20:35.320314+0000","last_clean":"2023-11-08T08:20:35.320314+0000","last_became_active":"2023-11-08T08:20:34.329416+0000","last_became_peered":"2023-11-08T08:20:34.329416+0000","last_unstale":"2023-11-08T08:20:35.320314+0000","last_undegraded":"2023-11-08T08:20:35.320314+0000","last_fullsized":"2023-11-08T08:20:35.320314+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:17:06.821071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.314630+0000","last_change":"2023-11-08T08:20:33.838897+0000","last_active":"2023-11-08T08:20:34.314630+0000","last_peered":"2023-11-08T08:20:34.314630+0000","last_clean":"2023-11-08T08:20:34.314630+0000","last_became_active":"2023-11-08T08:20:33.838641+0000","last_became_peered":"2023-11-08T08:20:33.838641+0000","last_unstale":"2023-11-08T08:20:34.314630+0000","last_undegraded":"2023-11-08T08:20:34.314630+0000","last_fullsized":"2023-11-08T08:20:34.314630+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:16:13.103973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439282+0000","last_change":"2023-11-08T08:20:34.347906+0000","last_active":"2023-11-08T08:20:35.439282+0000","last_peered":"2023-11-08T08:20:35.439282+0000","last_clean":"2023-11-08T08:20:35.439282+0000","last_became_active":"2023-11-08T08:20:34.347650+0000","last_became_peered":"2023-11-08T08:20:34.347650+0000","last_unstale":"2023-11-08T08:20:35.439282+0000","last_undegraded":"2023-11-08T08:20:35.439282+0000","last_fullsized":"2023-11-08T08:20:35.439282+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:17:06.821071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439391+0000","last_change":"2023-11-08T08:20:33.847825+0000","last_active":"2023-11-08T08:20:35.439391+0000","last_peered":"2023-11-08T08:20:35.439391+0000","last_clean":"2023-11-08T08:20:35.439391+0000","last_became_active":"2023-11-08T08:20:33.847566+0000","last_became_peered":"2023-11-08T08:20:33.847566+0000","last_unstale":"2023-11-08T08:20:35.439391+0000","last_undegraded":"2023-11-08T08:20:35.439391+0000","last_fullsized":"2023-11-08T08:20:35.439391+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:41:47.878758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.342457+0000","last_change":"2023-11-08T08:20:34.336533+0000","last_active":"2023-11-08T08:20:34.342457+0000","last_peered":"2023-11-08T08:20:34.342457+0000","last_clean":"2023-11-08T08:20:34.342457+0000","last_became_active":"2023-11-08T08:20:34.336243+0000","last_became_peered":"2023-11-08T08:20:34.336243+0000","last_unstale":"2023-11-08T08:20:34.342457+0000","last_undegraded":"2023-11-08T08:20:34.342457+0000","last_fullsized":"2023-11-08T08:20:34.342457+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:47:01.311804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319151+0000","last_change":"2023-11-08T08:20:33.832222+0000","last_active":"2023-11-08T08:20:35.319151+0000","last_peered":"2023-11-08T08:20:35.319151+0000","last_clean":"2023-11-08T08:20:35.319151+0000","last_became_active":"2023-11-08T08:20:33.831917+0000","last_became_peered":"2023-11-08T08:20:33.831917+0000","last_unstale":"2023-11-08T08:20:35.319151+0000","last_undegraded":"2023-11-08T08:20:35.319151+0000","last_fullsized":"2023-11-08T08:20:35.319151+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:42:20.705274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.321922+0000","last_change":"2023-11-08T08:20:34.322642+0000","last_active":"2023-11-08T08:20:35.321922+0000","last_peered":"2023-11-08T08:20:35.321922+0000","last_clean":"2023-11-08T08:20:35.321922+0000","last_became_active":"2023-11-08T08:20:34.322367+0000","last_became_peered":"2023-11-08T08:20:34.322367+0000","last_unstale":"2023-11-08T08:20:35.321922+0000","last_undegraded":"2023-11-08T08:20:35.321922+0000","last_fullsized":"2023-11-08T08:20:35.321922+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:16:14.114658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.322094+0000","last_change":"2023-11-08T08:20:33.850157+0000","last_active":"2023-11-08T08:20:35.322094+0000","last_peered":"2023-11-08T08:20:35.322094+0000","last_clean":"2023-11-08T08:20:35.322094+0000","last_became_active":"2023-11-08T08:20:33.848800+0000","last_became_peered":"2023-11-08T08:20:33.848800+0000","last_unstale":"2023-11-08T08:20:35.322094+0000","last_undegraded":"2023-11-08T08:20:35.322094+0000","last_fullsized":"2023-11-08T08:20:35.322094+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:12:15.661299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.320050+0000","last_change":"2023-11-08T08:20:34.344491+0000","last_active":"2023-11-08T08:20:35.320050+0000","last_peered":"2023-11-08T08:20:35.320050+0000","last_clean":"2023-11-08T08:20:35.320050+0000","last_became_active":"2023-11-08T08:20:34.339597+0000","last_became_peered":"2023-11-08T08:20:34.339597+0000","last_unstale":"2023-11-08T08:20:35.320050+0000","last_undegraded":"2023-11-08T08:20:35.320050+0000","last_fullsized":"2023-11-08T08:20:35.320050+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:09.194290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.448058+0000","last_change":"2023-11-08T08:20:33.846503+0000","last_active":"2023-11-08T08:20:35.448058+0000","last_peered":"2023-11-08T08:20:35.448058+0000","last_clean":"2023-11-08T08:20:35.448058+0000","last_became_active":"2023-11-08T08:20:33.846243+0000","last_became_peered":"2023-11-08T08:20:33.846243+0000","last_unstale":"2023-11-08T08:20:35.448058+0000","last_undegraded":"2023-11-08T08:20:35.448058+0000","last_fullsized":"2023-11-08T08:20:35.448058+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:16:43.509880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.345424+0000","last_change":"2023-11-08T08:20:34.337402+0000","last_active":"2023-11-08T08:20:34.345424+0000","last_peered":"2023-11-08T08:20:34.345424+0000","last_clean":"2023-11-08T08:20:34.345424+0000","last_became_active":"2023-11-08T08:20:34.336870+0000","last_became_peered":"2023-11-08T08:20:34.336870+0000","last_unstale":"2023-11-08T08:20:34.345424+0000","last_undegraded":"2023-11-08T08:20:34.345424+0000","last_fullsized":"2023-11-08T08:20:34.345424+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:42:21.715959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.345716+0000","last_change":"2023-11-08T08:20:33.840964+0000","last_active":"2023-11-08T08:20:34.345716+0000","last_peered":"2023-11-08T08:20:34.345716+0000","last_clean":"2023-11-08T08:20:34.345716+0000","last_became_active":"2023-11-08T08:20:33.840724+0000","last_became_peered":"2023-11-08T08:20:33.840724+0000","last_unstale":"2023-11-08T08:20:34.345716+0000","last_undegraded":"2023-11-08T08:20:34.345716+0000","last_fullsized":"2023-11-08T08:20:34.345716+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:56:44.922818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.345161+0000","last_change":"2023-11-08T08:20:34.336911+0000","last_active":"2023-11-08T08:20:34.345161+0000","last_peered":"2023-11-08T08:20:34.345161+0000","last_clean":"2023-11-08T08:20:34.345161+0000","last_became_active":"2023-11-08T08:20:34.336605+0000","last_became_peered":"2023-11-08T08:20:34.336605+0000","last_unstale":"2023-11-08T08:20:34.345161+0000","last_undegraded":"2023-11-08T08:20:34.345161+0000","last_fullsized":"2023-11-08T08:20:34.345161+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:26:18.631300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.440088+0000","last_change":"2023-11-08T08:20:33.846233+0000","last_active":"2023-11-08T08:20:35.440088+0000","last_peered":"2023-11-08T08:20:35.440088+0000","last_clean":"2023-11-08T08:20:35.440088+0000","last_became_active":"2023-11-08T08:20:33.845337+0000","last_became_peered":"2023-11-08T08:20:33.845337+0000","last_unstale":"2023-11-08T08:20:35.440088+0000","last_undegraded":"2023-11-08T08:20:35.440088+0000","last_fullsized":"2023-11-08T08:20:35.440088+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:56:44.922818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439813+0000","last_change":"2023-11-08T08:20:34.348264+0000","last_active":"2023-11-08T08:20:35.439813+0000","last_peered":"2023-11-08T08:20:35.439813+0000","last_clean":"2023-11-08T08:20:35.439813+0000","last_became_active":"2023-11-08T08:20:34.347846+0000","last_became_peered":"2023-11-08T08:20:34.347846+0000","last_unstale":"2023-11-08T08:20:35.439813+0000","last_undegraded":"2023-11-08T08:20:35.439813+0000","last_fullsized":"2023-11-08T08:20:35.439813+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:56:03.805095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439994+0000","last_change":"2023-11-08T08:20:33.844446+0000","last_active":"2023-11-08T08:20:35.439994+0000","last_peered":"2023-11-08T08:20:35.439994+0000","last_clean":"2023-11-08T08:20:35.439994+0000","last_became_active":"2023-11-08T08:20:33.842189+0000","last_became_peered":"2023-11-08T08:20:33.842189+0000","last_unstale":"2023-11-08T08:20:35.439994+0000","last_undegraded":"2023-11-08T08:20:35.439994+0000","last_fullsized":"2023-11-08T08:20:35.439994+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:00:33.700659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.322966+0000","last_change":"2023-11-08T08:20:34.327141+0000","last_active":"2023-11-08T08:20:35.322966+0000","last_peered":"2023-11-08T08:20:35.322966+0000","last_clean":"2023-11-08T08:20:35.322966+0000","last_became_active":"2023-11-08T08:20:34.326847+0000","last_became_peered":"2023-11-08T08:20:34.326847+0000","last_unstale":"2023-11-08T08:20:35.322966+0000","last_undegraded":"2023-11-08T08:20:35.322966+0000","last_fullsized":"2023-11-08T08:20:35.322966+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:31:38.135691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.310726+0000","last_change":"2023-11-08T08:20:33.845802+0000","last_active":"2023-11-08T08:20:34.310726+0000","last_peered":"2023-11-08T08:20:34.310726+0000","last_clean":"2023-11-08T08:20:34.310726+0000","last_became_active":"2023-11-08T08:20:33.845549+0000","last_became_peered":"2023-11-08T08:20:33.845549+0000","last_unstale":"2023-11-08T08:20:34.310726+0000","last_undegraded":"2023-11-08T08:20:34.310726+0000","last_fullsized":"2023-11-08T08:20:34.310726+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:09:01.178114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.321121+0000","last_change":"2023-11-08T08:20:34.327592+0000","last_active":"2023-11-08T08:20:35.321121+0000","last_peered":"2023-11-08T08:20:35.321121+0000","last_clean":"2023-11-08T08:20:35.321121+0000","last_became_active":"2023-11-08T08:20:34.327333+0000","last_became_peered":"2023-11-08T08:20:34.327333+0000","last_unstale":"2023-11-08T08:20:35.321121+0000","last_undegraded":"2023-11-08T08:20:35.321121+0000","last_fullsized":"2023-11-08T08:20:35.321121+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:48:42.113335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319176+0000","last_change":"2023-11-08T08:20:33.833398+0000","last_active":"2023-11-08T08:20:35.319176+0000","last_peered":"2023-11-08T08:20:35.319176+0000","last_clean":"2023-11-08T08:20:35.319176+0000","last_became_active":"2023-11-08T08:20:33.832827+0000","last_became_peered":"2023-11-08T08:20:33.832827+0000","last_unstale":"2023-11-08T08:20:35.319176+0000","last_undegraded":"2023-11-08T08:20:35.319176+0000","last_fullsized":"2023-11-08T08:20:35.319176+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:19:13.794792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319038+0000","last_change":"2023-11-08T08:20:34.328899+0000","last_active":"2023-11-08T08:20:35.319038+0000","last_peered":"2023-11-08T08:20:35.319038+0000","last_clean":"2023-11-08T08:20:35.319038+0000","last_became_active":"2023-11-08T08:20:34.328587+0000","last_became_peered":"2023-11-08T08:20:34.328587+0000","last_unstale":"2023-11-08T08:20:35.319038+0000","last_undegraded":"2023-11-08T08:20:35.319038+0000","last_fullsized":"2023-11-08T08:20:35.319038+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:58:23.617005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.314429+0000","last_change":"2023-11-08T08:20:33.843590+0000","last_active":"2023-11-08T08:20:34.314429+0000","last_peered":"2023-11-08T08:20:34.314429+0000","last_clean":"2023-11-08T08:20:34.314429+0000","last_became_active":"2023-11-08T08:20:33.843281+0000","last_became_peered":"2023-11-08T08:20:33.843281+0000","last_unstale":"2023-11-08T08:20:34.314429+0000","last_undegraded":"2023-11-08T08:20:34.314429+0000","last_fullsized":"2023-11-08T08:20:34.314429+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:27:54.885155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439773+0000","last_change":"2023-11-08T08:20:34.344571+0000","last_active":"2023-11-08T08:20:35.439773+0000","last_peered":"2023-11-08T08:20:35.439773+0000","last_clean":"2023-11-08T08:20:35.439773+0000","last_became_active":"2023-11-08T08:20:34.344023+0000","last_became_peered":"2023-11-08T08:20:34.344023+0000","last_unstale":"2023-11-08T08:20:35.439773+0000","last_undegraded":"2023-11-08T08:20:35.439773+0000","last_fullsized":"2023-11-08T08:20:35.439773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:58:23.617005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.314797+0000","last_change":"2023-11-08T08:20:33.848536+0000","last_active":"2023-11-08T08:20:34.314797+0000","last_peered":"2023-11-08T08:20:34.314797+0000","last_clean":"2023-11-08T08:20:34.314797+0000","last_became_active":"2023-11-08T08:20:33.844523+0000","last_became_peered":"2023-11-08T08:20:33.844523+0000","last_unstale":"2023-11-08T08:20:34.314797+0000","last_undegraded":"2023-11-08T08:20:34.314797+0000","last_fullsized":"2023-11-08T08:20:34.314797+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:28:22.760817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":28,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T08:20:33.839232+0000","last_change":"2023-11-08T08:18:58.771310+0000","last_active":"2023-11-08T08:20:33.839232+0000","last_peered":"2023-11-08T08:20:33.839232+0000","last_clean":"2023-11-08T08:20:33.839232+0000","last_became_active":"2023-11-08T08:18:58.770891+0000","last_became_peered":"2023-11-08T08:18:58.770891+0000","last_unstale":"2023-11-08T08:20:33.839232+0000","last_undegraded":"2023-11-08T08:20:33.839232+0000","last_fullsized":"2023-11-08T08:20:33.839232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.439530+0000","last_change":"2023-11-08T08:20:34.346998+0000","last_active":"2023-11-08T08:20:35.439530+0000","last_peered":"2023-11-08T08:20:35.439530+0000","last_clean":"2023-11-08T08:20:35.439530+0000","last_became_active":"2023-11-08T08:20:34.346747+0000","last_became_peered":"2023-11-08T08:20:34.346747+0000","last_unstale":"2023-11-08T08:20:35.439530+0000","last_undegraded":"2023-11-08T08:20:35.439530+0000","last_fullsized":"2023-11-08T08:20:35.439530+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:24:12.113358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447698+0000","last_change":"2023-11-08T08:20:33.851108+0000","last_active":"2023-11-08T08:20:35.447698+0000","last_peered":"2023-11-08T08:20:35.447698+0000","last_clean":"2023-11-08T08:20:35.447698+0000","last_became_active":"2023-11-08T08:20:33.850810+0000","last_became_peered":"2023-11-08T08:20:33.850810+0000","last_unstale":"2023-11-08T08:20:35.447698+0000","last_undegraded":"2023-11-08T08:20:35.447698+0000","last_fullsized":"2023-11-08T08:20:35.447698+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:43:48.363808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.322609+0000","last_change":"2023-11-08T08:18:58.770483+0000","last_active":"2023-11-08T08:20:34.322609+0000","last_peered":"2023-11-08T08:20:34.322609+0000","last_clean":"2023-11-08T08:20:34.322609+0000","last_became_active":"2023-11-08T08:18:58.770040+0000","last_became_peered":"2023-11-08T08:18:58.770040+0000","last_unstale":"2023-11-08T08:20:34.322609+0000","last_undegraded":"2023-11-08T08:20:34.322609+0000","last_fullsized":"2023-11-08T08:20:34.322609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.338049+0000","last_change":"2023-11-08T08:20:34.331904+0000","last_active":"2023-11-08T08:20:34.338049+0000","last_peered":"2023-11-08T08:20:34.338049+0000","last_clean":"2023-11-08T08:20:34.338049+0000","last_became_active":"2023-11-08T08:20:34.331485+0000","last_became_peered":"2023-11-08T08:20:34.331485+0000","last_unstale":"2023-11-08T08:20:34.338049+0000","last_undegraded":"2023-11-08T08:20:34.338049+0000","last_fullsized":"2023-11-08T08:20:34.338049+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:17:06.821071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.322866+0000","last_change":"2023-11-08T08:20:33.848387+0000","last_active":"2023-11-08T08:20:35.322866+0000","last_peered":"2023-11-08T08:20:35.322866+0000","last_clean":"2023-11-08T08:20:35.322866+0000","last_became_active":"2023-11-08T08:20:33.848133+0000","last_became_peered":"2023-11-08T08:20:33.848133+0000","last_unstale":"2023-11-08T08:20:35.322866+0000","last_undegraded":"2023-11-08T08:20:35.322866+0000","last_fullsized":"2023-11-08T08:20:35.322866+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:43:23.168935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T08:20:33.829156+0000","last_change":"2023-11-08T08:18:58.766848+0000","last_active":"2023-11-08T08:20:33.829156+0000","last_peered":"2023-11-08T08:20:33.829156+0000","last_clean":"2023-11-08T08:20:33.829156+0000","last_became_active":"2023-11-08T08:18:58.766378+0000","last_became_peered":"2023-11-08T08:18:58.766378+0000","last_unstale":"2023-11-08T08:20:33.829156+0000","last_undegraded":"2023-11-08T08:20:33.829156+0000","last_fullsized":"2023-11-08T08:20:33.829156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.447535+0000","last_change":"2023-11-08T08:20:34.337228+0000","last_active":"2023-11-08T08:20:35.447535+0000","last_peered":"2023-11-08T08:20:35.447535+0000","last_clean":"2023-11-08T08:20:35.447535+0000","last_became_active":"2023-11-08T08:20:34.336988+0000","last_became_peered":"2023-11-08T08:20:34.336988+0000","last_unstale":"2023-11-08T08:20:35.447535+0000","last_undegraded":"2023-11-08T08:20:35.447535+0000","last_fullsized":"2023-11-08T08:20:35.447535+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:50:55.369713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319095+0000","last_change":"2023-11-08T08:20:33.845617+0000","last_active":"2023-11-08T08:20:35.319095+0000","last_peered":"2023-11-08T08:20:35.319095+0000","last_clean":"2023-11-08T08:20:35.319095+0000","last_became_active":"2023-11-08T08:20:33.845315+0000","last_became_peered":"2023-11-08T08:20:33.845315+0000","last_unstale":"2023-11-08T08:20:35.319095+0000","last_undegraded":"2023-11-08T08:20:35.319095+0000","last_fullsized":"2023-11-08T08:20:35.319095+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:26:17.620615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.321408+0000","last_change":"2023-11-08T08:18:58.769764+0000","last_active":"2023-11-08T08:20:34.321408+0000","last_peered":"2023-11-08T08:20:34.321408+0000","last_clean":"2023-11-08T08:20:34.321408+0000","last_became_active":"2023-11-08T08:18:58.769357+0000","last_became_peered":"2023-11-08T08:18:58.769357+0000","last_unstale":"2023-11-08T08:20:34.321408+0000","last_undegraded":"2023-11-08T08:20:34.321408+0000","last_fullsized":"2023-11-08T08:20:34.321408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:13.748094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.347057+0000","last_change":"2023-11-08T08:20:34.341170+0000","last_active":"2023-11-08T08:20:34.347057+0000","last_peered":"2023-11-08T08:20:34.347057+0000","last_clean":"2023-11-08T08:20:34.347057+0000","last_became_active":"2023-11-08T08:20:34.340922+0000","last_became_peered":"2023-11-08T08:20:34.340922+0000","last_unstale":"2023-11-08T08:20:34.347057+0000","last_undegraded":"2023-11-08T08:20:34.347057+0000","last_fullsized":"2023-11-08T08:20:34.347057+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:11:42.810746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"43'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.443933+0000","last_change":"2023-11-08T08:20:33.844867+0000","last_active":"2023-11-08T08:20:35.443933+0000","last_peered":"2023-11-08T08:20:35.443933+0000","last_clean":"2023-11-08T08:20:35.443933+0000","last_became_active":"2023-11-08T08:20:33.844553+0000","last_became_peered":"2023-11-08T08:20:33.844553+0000","last_unstale":"2023-11-08T08:20:35.443933+0000","last_undegraded":"2023-11-08T08:20:35.443933+0000","last_fullsized":"2023-11-08T08:20:35.443933+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:09:46.694297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T08:20:33.827576+0000","last_change":"2023-11-08T08:18:58.771460+0000","last_active":"2023-11-08T08:20:33.827576+0000","last_peered":"2023-11-08T08:20:33.827576+0000","last_clean":"2023-11-08T08:20:33.827576+0000","last_became_active":"2023-11-08T08:18:58.770648+0000","last_became_peered":"2023-11-08T08:18:58.770648+0000","last_unstale":"2023-11-08T08:20:33.827576+0000","last_undegraded":"2023-11-08T08:20:33.827576+0000","last_fullsized":"2023-11-08T08:20:33.827576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.319687+0000","last_change":"2023-11-08T08:20:34.326509+0000","last_active":"2023-11-08T08:20:35.319687+0000","last_peered":"2023-11-08T08:20:35.319687+0000","last_clean":"2023-11-08T08:20:35.319687+0000","last_became_active":"2023-11-08T08:20:34.326252+0000","last_became_peered":"2023-11-08T08:20:34.326252+0000","last_unstale":"2023-11-08T08:20:35.319687+0000","last_undegraded":"2023-11-08T08:20:35.319687+0000","last_fullsized":"2023-11-08T08:20:35.319687+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:56:03.805095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.315626+0000","last_change":"2023-11-08T08:20:33.845017+0000","last_active":"2023-11-08T08:20:34.315626+0000","last_peered":"2023-11-08T08:20:34.315626+0000","last_clean":"2023-11-08T08:20:34.315626+0000","last_became_active":"2023-11-08T08:20:33.844760+0000","last_became_peered":"2023-11-08T08:20:33.844760+0000","last_unstale":"2023-11-08T08:20:34.315626+0000","last_undegraded":"2023-11-08T08:20:34.315626+0000","last_fullsized":"2023-11-08T08:20:34.315626+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:31:37.125006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.315620+0000","last_change":"2023-11-08T08:18:58.973732+0000","last_active":"2023-11-08T08:20:34.315620+0000","last_peered":"2023-11-08T08:20:34.315620+0000","last_clean":"2023-11-08T08:20:34.315620+0000","last_became_active":"2023-11-08T08:18:58.973007+0000","last_became_peered":"2023-11-08T08:18:58.973007+0000","last_unstale":"2023-11-08T08:20:34.315620+0000","last_undegraded":"2023-11-08T08:20:34.315620+0000","last_fullsized":"2023-11-08T08:20:34.315620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.344029+0000","last_change":"2023-11-08T08:20:34.329387+0000","last_active":"2023-11-08T08:20:34.344029+0000","last_peered":"2023-11-08T08:20:34.344029+0000","last_clean":"2023-11-08T08:20:34.344029+0000","last_became_active":"2023-11-08T08:20:34.328949+0000","last_became_peered":"2023-11-08T08:20:34.328949+0000","last_unstale":"2023-11-08T08:20:34.344029+0000","last_undegraded":"2023-11-08T08:20:34.344029+0000","last_fullsized":"2023-11-08T08:20:34.344029+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:16:44.520565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.322741+0000","last_change":"2023-11-08T08:20:33.845481+0000","last_active":"2023-11-08T08:20:35.322741+0000","last_peered":"2023-11-08T08:20:35.322741+0000","last_clean":"2023-11-08T08:20:35.322741+0000","last_became_active":"2023-11-08T08:20:33.845213+0000","last_became_peered":"2023-11-08T08:20:33.845213+0000","last_unstale":"2023-11-08T08:20:35.322741+0000","last_undegraded":"2023-11-08T08:20:35.322741+0000","last_fullsized":"2023-11-08T08:20:35.322741+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:08:17.687539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T08:20:33.826052+0000","last_change":"2023-11-08T08:18:58.770136+0000","last_active":"2023-11-08T08:20:33.826052+0000","last_peered":"2023-11-08T08:20:33.826052+0000","last_clean":"2023-11-08T08:20:33.826052+0000","last_became_active":"2023-11-08T08:18:58.769797+0000","last_became_peered":"2023-11-08T08:18:58.769797+0000","last_unstale":"2023-11-08T08:20:33.826052+0000","last_undegraded":"2023-11-08T08:20:33.826052+0000","last_fullsized":"2023-11-08T08:20:33.826052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:29:12.107128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-08T08:20:35.320158+0000","last_change":"2023-11-08T08:20:34.331889+0000","last_active":"2023-11-08T08:20:35.320158+0000","last_peered":"2023-11-08T08:20:35.320158+0000","last_clean":"2023-11-08T08:20:35.320158+0000","last_became_active":"2023-11-08T08:20:34.331526+0000","last_became_peered":"2023-11-08T08:20:34.331526+0000","last_unstale":"2023-11-08T08:20:35.320158+0000","last_undegraded":"2023-11-08T08:20:35.320158+0000","last_fullsized":"2023-11-08T08:20:35.320158+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:33.805072+0000","last_clean_scrub_stamp":"2023-11-08T08:20:33.805072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:20:27.726248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-08T08:20:34.345234+0000","last_change":"2023-11-08T08:20:33.848145+0000","last_active":"2023-11-08T08:20:34.345234+0000","last_peered":"2023-11-08T08:20:34.345234+0000","last_clean":"2023-11-08T08:20:34.345234+0000","last_became_active":"2023-11-08T08:20:33.847694+0000","last_became_peered":"2023-11-08T08:20:33.847694+0000","last_unstale":"2023-11-08T08:20:34.345234+0000","last_undegraded":"2023-11-08T08:20:34.345234+0000","last_fullsized":"2023-11-08T08:20:34.345234+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:20:32.794387+0000","last_clean_scrub_stamp":"2023-11-08T08:20:32.794387+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:16:43.509880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T08:20:33.827610+0000","last_change":"2023-11-08T08:18:58.777140+0000","last_active":"2023-11-08T08:20:33.827610+0000","last_peered":"2023-11-08T08:20:33.827610+0000","last_clean":"2023-11-08T08:20:33.827610+0000","last_became_active":"2023-11-08T08:18:58.776834+0000","last_became_peered":"2023-11-08T08:18:58.776834+0000","last_unstale":"2023-11-08T08:20:33.827610+0000","last_undegraded":"2023-11-08T08:20:33.827610+0000","last_fullsized":"2023-11-08T08:20:33.827610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-08T08:20:33.826410+0000","last_change":"2023-11-08T08:18:53.780089+0000","last_active":"2023-11-08T08:20:33.826410+0000","last_peered":"2023-11-08T08:20:33.826410+0000","last_clean":"2023-11-08T08:20:33.826410+0000","last_became_active":"2023-11-08T08:18:53.779533+0000","last_became_peered":"2023-11-08T08:18:53.779533+0000","last_unstale":"2023-11-08T08:20:33.826410+0000","last_undegraded":"2023-11-08T08:20:33.826410+0000","last_fullsized":"2023-11-08T08:20:33.826410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_clean_scrub_stamp":"2023-11-08T08:18:52.744693+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:58:24.877070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9836,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362004,"statfs":{"total":96636764160,"available":96626692096,"internally_reserved":0,"allocated":1085440,"data_stored":806932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.341}]},{"osd":1,"last update":"Wed Nov 8 08:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.2869999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":3.6949999999999998,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":4.9089999999999998,"5min":4.9089999999999998,"15min":4.9089999999999998},"last":1.1850000000000001}]},{"osd":3,"last update":"Wed Nov 8 08:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.8199999999999998,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":1.431}]},{"osd":4,"last update":"Wed Nov 8 08:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.46},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":3.1360000000000001,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":1.5349999999999999}]},{"osd":5,"last update":"Wed Nov 8 08:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.73,"5min":2.73,"15min":2.73},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.399}]},{"osd":6,"last update":"Wed Nov 8 08:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.462,"15min":1.462},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.264},{"interface":"front","average":{"1min":1.462,"5min":1.462,"15min":1.462},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.4850000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8620,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363220,"statfs":{"total":96636764160,"available":96627937280,"internally_reserved":0,"allocated":495616,"data_stored":217130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.01}]},{"osd":1,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.1859999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.2749999999999999}]},{"osd":3,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.6539999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.3140000000000001}]},{"osd":5,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.476}]},{"osd":7,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":3.133,"5min":3.133,"15min":3.133},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":2.8180000000000001,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.7370000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8680,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363160,"statfs":{"total":96636764160,"available":96627875840,"internally_reserved":0,"allocated":491520,"data_stored":216544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.03}]},{"osd":1,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.212},{"interface":"front","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.89500000000000002}]},{"osd":2,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.173}]},{"osd":3,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":2.9660000000000002,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":3.7999999999999998,"5min":3.7999999999999998,"15min":3.7999999999999998},"last":0.96399999999999997}]},{"osd":4,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":3.073,"5min":3.073,"15min":3.073},"last":1.359}]},{"osd":6,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":3.8650000000000002,"5min":3.8650000000000002,"15min":3.8650000000000002},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.1020000000000001}]},{"osd":7,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.248},{"interface":"front","average":{"1min":1.52,"5min":1.52,"15min":1.52},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":3.9369999999999998,"5min":3.9369999999999998,"15min":3.9369999999999998},"last":0.86899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8816,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94363024,"statfs":{"total":96636764160,"available":96627736576,"internally_reserved":0,"allocated":499712,"data_stored":218022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.492},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":1.2569999999999999}]},{"osd":1,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.095}]},{"osd":2,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.3240000000000001}]},{"osd":3,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.278}]},{"osd":5,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.4740000000000002,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":1.373}]},{"osd":6,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.5960000000000001}]},{"osd":7,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.446,"15min":1.446},"min":{"1min":1.02,"5min":1.02,"15min":1.02},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.472},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":1.6599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8752,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363088,"statfs":{"total":96636764160,"available":96627802112,"internally_reserved":0,"allocated":499712,"data_stored":217096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.77700000000000002}]},{"osd":1,"last update":"Wed Nov 8 08:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.94999999999999996}]},{"osd":2,"last update":"Wed Nov 8 08:19:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.4569999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":0.86499999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:19:51 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.113},{"interface":"front","average":{"1min":1.415,"5min":1.415,"15min":1.415},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.88800000000000001}]},{"osd":5,"last update":"Wed Nov 8 08:19:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":2.2189999999999999,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.018},{"interface":"front","average":{"1min":1.444,"5min":1.444,"15min":1.444},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.3610000000000002,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.1639999999999999}]},{"osd":6,"last update":"Wed Nov 8 08:19:51 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":1.008,"5min":1.008,"15min":1.008},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":1.222},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"min":{"1min":1.091,"5min":1.091,"15min":1.091},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":1.0580000000000001}]},{"osd":7,"last update":"Wed Nov 8 08:19:51 2023","interfaces":[{"interface":"back","average":{"1min":1.623,"5min":1.623,"15min":1.623},"min":{"1min":1.22,"5min":1.22,"15min":1.22},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":1.258},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"min":{"1min":1.123,"5min":1.123,"15min":1.123},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.03}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8640,"kb_used_data":504,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363200,"statfs":{"total":96636764160,"available":96627916800,"internally_reserved":0,"allocated":516096,"data_stored":225017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.77400000000000002}]},{"osd":1,"last update":"Wed Nov 8 08:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.115}]},{"osd":3,"last update":"Wed Nov 8 08:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.6099999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.747},{"interface":"front","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":1.2749999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"max":{"1min":2.7010000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.504}]},{"osd":5,"last update":"Wed Nov 8 08:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"min":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"max":{"1min":2.3279999999999998,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.24}]},{"osd":6,"last update":"Wed Nov 8 08:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":3.4350000000000001,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.363},{"interface":"front","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"max":{"1min":2.9060000000000001,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":1.3859999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"min":{"1min":1.091,"5min":1.091,"15min":1.091},"max":{"1min":2.867,"5min":2.867,"15min":2.867},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.1679999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8744,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363096,"statfs":{"total":96636764160,"available":96627810304,"internally_reserved":0,"allocated":491520,"data_stored":216544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.1479999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.872},{"interface":"front","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":3.0899999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.97399999999999998}]},{"osd":3,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.3570000000000002,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.0660000000000001}]},{"osd":4,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.228},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":1.034}]},{"osd":5,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.28},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":1.0009999999999999}]},{"osd":6,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.456,"15min":1.456},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.149,"5min":3.149,"15min":3.149},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":1.381}]},{"osd":7,"last update":"Wed Nov 8 08:19:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"max":{"1min":2.2320000000000002,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.3480000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9848,"kb_used_data":1072,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94361992,"statfs":{"total":96636764160,"available":96626679808,"internally_reserved":0,"allocated":1097728,"data_stored":813849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.56000000000000005}]},{"osd":2,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.745}]},{"osd":3,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.964,"5min":1.964,"15min":1.964},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":0.95799999999999996}]},{"osd":4,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"min":{"1min":1.083,"5min":1.083,"15min":1.083},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.2230000000000001}]},{"osd":5,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"min":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"max":{"1min":2.4449999999999998,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.234}]},{"osd":6,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.159},{"interface":"front","average":{"1min":1.607,"5min":1.607,"15min":1.607},"min":{"1min":1.034,"5min":1.034,"15min":1.034},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.99299999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"min":{"1min":1.24,"5min":1.24,"15min":1.24},"max":{"1min":3.0529999999999999,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"min":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"max":{"1min":3.2669999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":1.359}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T08:20:39.131 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T08:20:39.132 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-08T08:20:39.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.353+0000 7f681e730700 1 Processor -- start 2023-11-08T08:20:39.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.353+0000 7f681e730700 1 -- start start 2023-11-08T08:20:39.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.357+0000 7f681e730700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f681815c7c0 0x7f681806cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:39.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.357+0000 7f681e730700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f68180cddf0 0x7f68180ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:39.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.357+0000 7f681e730700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68180ce730 con 0x7f6818161030 2023-11-08T08:20:39.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.357+0000 7f681e730700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68180cdba0 con 0x7f68180cddf0 2023-11-08T08:20:39.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.357+0000 7f681e730700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68180cd8d0 con 0x7f681815c7c0 2023-11-08T08:20:39.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.357+0000 7f681d72e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f681815c7c0 0x7f681806cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:39.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.357+0000 7f681d72e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f681815c7c0 0x7f681806cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:55470/0 (socket says 172.21.15.40:55470) 2023-11-08T08:20:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.357+0000 7f681d72e700 1 -- 172.21.15.40:0/469123575 learned_addr learned my addr 172.21.15.40:0/469123575 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.357+0000 7f681cf2d700 1 --1- 172.21.15.40:0/469123575 >> v1:172.21.15.169:6789/0 conn(0x7f6818161030 0x7f68180cd1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:50984/0 (socket says 172.21.15.40:50984) 2023-11-08T08:20:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.357+0000 7f681df2f700 1 --2- 172.21.15.40:0/469123575 >> v2:172.21.15.40:3300/0 conn(0x7f68180cddf0 0x7f68180ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.357+0000 7f681d72e700 1 -- 172.21.15.40:0/469123575 >> v2:172.21.15.40:3300/0 conn(0x7f68180cddf0 msgr2=0x7f68180ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:39.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.357+0000 7f681d72e700 1 --2- 172.21.15.40:0/469123575 >> v2:172.21.15.40:3300/0 conn(0x7f68180cddf0 0x7f68180ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.357+0000 7f681d72e700 1 -- 172.21.15.40:0/469123575 >> v1:172.21.15.169:6789/0 conn(0x7f6818161030 legacy=0x7f68180cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:39.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.357+0000 7f680ffff700 1 -- 172.21.15.40:0/469123575 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852680134 0 0) 0x7f68180ce730 con 0x7f6818161030 2023-11-08T08:20:39.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.357+0000 7f681d72e700 1 -- 172.21.15.40:0/469123575 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681806c010 con 0x7f681815c7c0 2023-11-08T08:20:39.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.357+0000 7f681d72e700 1 --2- 172.21.15.40:0/469123575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f681815c7c0 0x7f681806cdd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6818073500 tx=0x7f6814009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d22f114c57788db3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:39.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.357+0000 7f680ffff700 1 -- 172.21.15.40:0/469123575 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814011830 con 0x7f681815c7c0 2023-11-08T08:20:39.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.357+0000 7f680ffff700 1 -- 172.21.15.40:0/469123575 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6814011990 con 0x7f681815c7c0 2023-11-08T08:20:39.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.357+0000 7f680ffff700 1 -- 172.21.15.40:0/469123575 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814011d20 con 0x7f681815c7c0 2023-11-08T08:20:39.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.361+0000 7f681e730700 1 -- 172.21.15.40:0/469123575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f681815c7c0 msgr2=0x7f681806cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:39.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.361+0000 7f681e730700 1 --2- 172.21.15.40:0/469123575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f681815c7c0 0x7f681806cdd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6818073500 tx=0x7f6814009bf0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.361+0000 7f681e730700 1 -- 172.21.15.40:0/469123575 shutdown_connections 2023-11-08T08:20:39.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.361+0000 7f681e730700 1 --2- 172.21.15.40:0/469123575 >> v2:172.21.15.40:3300/0 conn(0x7f68180cddf0 0x7f68180ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.361+0000 7f681e730700 1 --2- 172.21.15.40:0/469123575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f681815c7c0 0x7f681806cdd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.361+0000 7f681e730700 1 -- 172.21.15.40:0/469123575 >> 172.21.15.40:0/469123575 conn(0x7f68180d3800 msgr2=0x7f68180cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:39.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.361+0000 7f681e730700 1 -- 172.21.15.40:0/469123575 shutdown_connections 2023-11-08T08:20:39.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.361+0000 7f681e730700 1 -- 172.21.15.40:0/469123575 wait complete. 2023-11-08T08:20:39.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.361+0000 7f681e730700 1 Processor -- start 2023-11-08T08:20:39.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.361+0000 7f681e730700 1 -- start start 2023-11-08T08:20:39.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f681e730700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68180cddf0 0x7f68180c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:39.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f681e730700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f681815c7c0 0x7f68181583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:39.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f681e730700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68180b6b60 0x7f68180b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:39.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f681e730700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68180c68d0 con 0x7f68180b6b60 2023-11-08T08:20:39.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f681e730700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68180c95a0 con 0x7f68180cddf0 2023-11-08T08:20:39.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f681e730700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68180cc7e0 con 0x7f681815c7c0 2023-11-08T08:20:39.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f681df2f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68180b6b60 0x7f68180b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:39.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f681df2f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68180b6b60 0x7f68180b6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48778/0 (socket says 172.21.15.40:48778) 2023-11-08T08:20:39.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f681df2f700 1 -- 172.21.15.40:0/1899273359 learned_addr learned my addr 172.21.15.40:0/1899273359 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:39.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f681d72e700 1 --2- 172.21.15.40:0/1899273359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68180cddf0 0x7f68180c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:39.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f681cf2d700 1 --2- 172.21.15.40:0/1899273359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f681815c7c0 0x7f68181583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:39.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f681df2f700 1 -- 172.21.15.40:0/1899273359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f681815c7c0 msgr2=0x7f68181583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:39.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f681df2f700 1 --2- 172.21.15.40:0/1899273359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f681815c7c0 0x7f68181583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f681df2f700 1 -- 172.21.15.40:0/1899273359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68180cddf0 msgr2=0x7f68180c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:39.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f681df2f700 1 --2- 172.21.15.40:0/1899273359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68180cddf0 0x7f68180c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f681df2f700 1 -- 172.21.15.40:0/1899273359 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6814009570 con 0x7f68180b6b60 2023-11-08T08:20:39.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f681df2f700 1 --2- 172.21.15.40:0/1899273359 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68180b6b60 0x7f68180b6fd0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f681000efc0 tx=0x7f681000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:39.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f680dffb700 1 -- 172.21.15.40:0/1899273359 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681000cad0 con 0x7f68180b6b60 2023-11-08T08:20:39.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f680dffb700 1 -- 172.21.15.40:0/1899273359 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681000f040 con 0x7f68180b6b60 2023-11-08T08:20:39.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f681e730700 1 -- 172.21.15.40:0/1899273359 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681806c5d0 con 0x7f68180b6b60 2023-11-08T08:20:39.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f680dffb700 1 -- 172.21.15.40:0/1899273359 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681000ce70 con 0x7f68180b6b60 2023-11-08T08:20:39.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.365+0000 7f681e730700 1 -- 172.21.15.40:0/1899273359 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681806c2f0 con 0x7f68180b6b60 2023-11-08T08:20:39.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.369+0000 7f681e730700 1 -- 172.21.15.40:0/1899273359 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f681806c8b0 con 0x7f68180b6b60 2023-11-08T08:20:39.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.369+0000 7f680dffb700 1 -- 172.21.15.40:0/1899273359 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f6810014480 con 0x7f68180b6b60 2023-11-08T08:20:39.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.373+0000 7f680dffb700 1 --2- 172.21.15.40:0/1899273359 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f67fc064ee0 0x7f67fc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:39.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.373+0000 7f680dffb700 1 -- 172.21.15.40:0/1899273359 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6810082c00 con 0x7f68180b6b60 2023-11-08T08:20:39.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.373+0000 7f681d72e700 1 --2- 172.21.15.40:0/1899273359 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f67fc064ee0 0x7f67fc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:39.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.373+0000 7f681d72e700 1 --2- 172.21.15.40:0/1899273359 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f67fc064ee0 0x7f67fc067390 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6814000c00 tx=0x7f6814008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:39.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.377+0000 7f680dffb700 1 -- 172.21.15.40:0/1899273359 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f6810050b10 con 0x7f68180b6b60 2023-11-08T08:20:39.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.613+0000 7f681e730700 1 -- 172.21.15.40:0/1899273359 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f6818073cb0 con 0x7f68180b6b60 2023-11-08T08:20:39.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.621+0000 7f680dffb700 1 -- 172.21.15.40:0/1899273359 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 184+0+0 (secure 0 0 0) 0x7f68100180e0 con 0x7f68180b6b60 2023-11-08T08:20:39.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.625+0000 7f681e730700 1 -- 172.21.15.40:0/1899273359 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f67fc064ee0 msgr2=0x7f67fc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:39.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.625+0000 7f681e730700 1 --2- 172.21.15.40:0/1899273359 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f67fc064ee0 0x7f67fc067390 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6814000c00 tx=0x7f6814008030 comp rx=0 tx=0).stop 2023-11-08T08:20:39.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.625+0000 7f681e730700 1 -- 172.21.15.40:0/1899273359 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68180b6b60 msgr2=0x7f68180b6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:39.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.625+0000 7f681e730700 1 --2- 172.21.15.40:0/1899273359 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68180b6b60 0x7f68180b6fd0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f681000efc0 tx=0x7f681000c0c0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.625+0000 7f681e730700 1 -- 172.21.15.40:0/1899273359 shutdown_connections 2023-11-08T08:20:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.625+0000 7f681e730700 1 --2- 172.21.15.40:0/1899273359 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f67fc064ee0 0x7f67fc067390 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.625+0000 7f681e730700 1 --2- 172.21.15.40:0/1899273359 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68180b6b60 0x7f68180b6fd0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.625+0000 7f681e730700 1 --2- 172.21.15.40:0/1899273359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f681815c7c0 0x7f68181583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.625+0000 7f681e730700 1 --2- 172.21.15.40:0/1899273359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68180cddf0 0x7f68180c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.625+0000 7f681e730700 1 -- 172.21.15.40:0/1899273359 >> 172.21.15.40:0/1899273359 conn(0x7f68180d3800 msgr2=0x7f68180b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:39.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.625+0000 7f681e730700 1 -- 172.21.15.40:0/1899273359 shutdown_connections 2023-11-08T08:20:39.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.625+0000 7f681e730700 1 -- 172.21.15.40:0/1899273359 wait complete. 2023-11-08T08:20:39.633 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2023-11-08T08:20:39.653 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:20:39.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.829+0000 7fb5fdb5e700 1 Processor -- start 2023-11-08T08:20:39.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.829+0000 7fb5fdb5e700 1 -- start start 2023-11-08T08:20:39.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5fdb5e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5f8163cb0 0x7fb5f8164cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:39.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5fdb5e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb5f80cdfb0 0x7fb5f806cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:39.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5fdb5e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb5f806d2f0 0x7fb5f806bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:39.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5fdb5e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f80cc290 con 0x7fb5f80cdfb0 2023-11-08T08:20:39.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5fdb5e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f806d770 con 0x7fb5f806d2f0 2023-11-08T08:20:39.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5fdb5e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8164080 con 0x7fb5f8163cb0 2023-11-08T08:20:39.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5eeffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb5f80cdfb0 0x7fb5f806cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:39.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5f7fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb5f806d2f0 0x7fb5f806bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:39.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5eeffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb5f80cdfb0 0x7fb5f806cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48780/0 (socket says 172.21.15.40:48780) 2023-11-08T08:20:39.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5eeffd700 1 -- 172.21.15.40:0/2200379665 learned_addr learned my addr 172.21.15.40:0/2200379665 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:39.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5f77fe700 1 --2- 172.21.15.40:0/2200379665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5f8163cb0 0x7fb5f8164cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:39.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5eeffd700 1 -- 172.21.15.40:0/2200379665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5f8163cb0 msgr2=0x7fb5f8164cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:39.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5eeffd700 1 --2- 172.21.15.40:0/2200379665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5f8163cb0 0x7fb5f8164cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5eeffd700 1 -- 172.21.15.40:0/2200379665 >> v2:172.21.15.169:3300/0 conn(0x7fb5f806d2f0 msgr2=0x7fb5f806bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:39.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5eeffd700 1 --2- 172.21.15.40:0/2200379665 >> v2:172.21.15.169:3300/0 conn(0x7fb5f806d2f0 0x7fb5f806bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5eeffd700 1 -- 172.21.15.40:0/2200379665 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f8069bf0 con 0x7fb5f80cdfb0 2023-11-08T08:20:39.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5f7fff700 1 --2- 172.21.15.40:0/2200379665 >> v2:172.21.15.169:3300/0 conn(0x7fb5f806d2f0 0x7fb5f806bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:39.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5eeffd700 1 --2- 172.21.15.40:0/2200379665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5f80cdfb0 0x7fb5f806cdb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb5f806f370 tx=0x7fb5e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa38934b47be0920 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:39.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5f6ffd700 1 -- 172.21.15.40:0/2200379665 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e000e070 con 0x7fb5f80cdfb0 2023-11-08T08:20:39.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5f6ffd700 1 -- 172.21.15.40:0/2200379665 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e000b910 con 0x7fb5f80cdfb0 2023-11-08T08:20:39.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5f6ffd700 1 -- 172.21.15.40:0/2200379665 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e000bcc0 con 0x7fb5f80cdfb0 2023-11-08T08:20:39.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5fdb5e700 1 -- 172.21.15.40:0/2200379665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5f80cdfb0 msgr2=0x7fb5f806cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:39.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.833+0000 7fb5fdb5e700 1 --2- 172.21.15.40:0/2200379665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5f80cdfb0 0x7fb5f806cdb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb5f806f370 tx=0x7fb5e000b330 comp rx=0 tx=0).stop 2023-11-08T08:20:39.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.837+0000 7fb5fdb5e700 1 -- 172.21.15.40:0/2200379665 shutdown_connections 2023-11-08T08:20:39.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.837+0000 7fb5fdb5e700 1 --2- 172.21.15.40:0/2200379665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5f80cdfb0 0x7fb5f806cdb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.837+0000 7fb5fdb5e700 1 --2- 172.21.15.40:0/2200379665 >> v2:172.21.15.169:3300/0 conn(0x7fb5f806d2f0 0x7fb5f806bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.837+0000 7fb5fdb5e700 1 --2- 172.21.15.40:0/2200379665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5f8163cb0 0x7fb5f8164cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.837+0000 7fb5fdb5e700 1 -- 172.21.15.40:0/2200379665 >> 172.21.15.40:0/2200379665 conn(0x7fb5f80cf350 msgr2=0x7fb5f80b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:39.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.837+0000 7fb5fdb5e700 1 -- 172.21.15.40:0/2200379665 shutdown_connections 2023-11-08T08:20:39.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.837+0000 7fb5fdb5e700 1 -- 172.21.15.40:0/2200379665 wait complete. 2023-11-08T08:20:39.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.837+0000 7fb5fdb5e700 1 Processor -- start 2023-11-08T08:20:39.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.837+0000 7fb5fdb5e700 1 -- start start 2023-11-08T08:20:39.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5fdb5e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5f806d2f0 0x7fb5f8153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5fdb5e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5f80cdfb0 0x7fb5f80c2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5fdb5e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5f8163cb0 0x7fb5f80c7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5fdb5e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8154280 con 0x7fb5f8163cb0 2023-11-08T08:20:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5fdb5e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f80c2cd0 con 0x7fb5f806d2f0 2023-11-08T08:20:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5fdb5e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f80c29f0 con 0x7fb5f80cdfb0 2023-11-08T08:20:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5f7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5f8163cb0 0x7fb5f80c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5f7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5f8163cb0 0x7fb5f80c7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48792/0 (socket says 172.21.15.40:48792) 2023-11-08T08:20:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5f7fff700 1 -- 172.21.15.40:0/3032388081 learned_addr learned my addr 172.21.15.40:0/3032388081 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5f77fe700 1 --2- 172.21.15.40:0/3032388081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5f806d2f0 0x7fb5f8153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5eeffd700 1 --2- 172.21.15.40:0/3032388081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5f80cdfb0 0x7fb5f80c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:39.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5f7fff700 1 -- 172.21.15.40:0/3032388081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5f80cdfb0 msgr2=0x7fb5f80c2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:39.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5f7fff700 1 --2- 172.21.15.40:0/3032388081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5f80cdfb0 0x7fb5f80c2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5f7fff700 1 -- 172.21.15.40:0/3032388081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5f806d2f0 msgr2=0x7fb5f8153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:39.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5f7fff700 1 --2- 172.21.15.40:0/3032388081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5f806d2f0 0x7fb5f8153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:39.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5f7fff700 1 -- 172.21.15.40:0/3032388081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f00094b0 con 0x7fb5f8163cb0 2023-11-08T08:20:39.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5f77fe700 1 --2- 172.21.15.40:0/3032388081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5f806d2f0 0x7fb5f8153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:39.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5f7fff700 1 --2- 172.21.15.40:0/3032388081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5f8163cb0 0x7fb5f80c7160 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000e930 tx=0x7fb5f000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:39.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5f4ff9700 1 -- 172.21.15.40:0/3032388081 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0009710 con 0x7fb5f8163cb0 2023-11-08T08:20:39.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5f4ff9700 1 -- 172.21.15.40:0/3032388081 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f000ee80 con 0x7fb5f8163cb0 2023-11-08T08:20:39.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5f4ff9700 1 -- 172.21.15.40:0/3032388081 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0004240 con 0x7fb5f8163cb0 2023-11-08T08:20:39.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5fdb5e700 1 -- 172.21.15.40:0/3032388081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e0009b30 con 0x7fb5f8163cb0 2023-11-08T08:20:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.841+0000 7fb5fdb5e700 1 -- 172.21.15.40:0/3032388081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f80cc4a0 con 0x7fb5f8163cb0 2023-11-08T08:20:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.845+0000 7fb5fdb5e700 1 -- 172.21.15.40:0/3032388081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f81651f0 con 0x7fb5f8163cb0 2023-11-08T08:20:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.845+0000 7fb5f4ff9700 1 -- 172.21.15.40:0/3032388081 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fb5f00043a0 con 0x7fb5f8163cb0 2023-11-08T08:20:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.849+0000 7fb5f4ff9700 1 --2- 172.21.15.40:0/3032388081 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb5d8064ee0 0x7fb5d8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.849+0000 7fb5f4ff9700 1 -- 172.21.15.40:0/3032388081 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb5f0081e20 con 0x7fb5f8163cb0 2023-11-08T08:20:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.849+0000 7fb5f77fe700 1 --2- 172.21.15.40:0/3032388081 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb5d8064ee0 0x7fb5d8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.849+0000 7fb5f77fe700 1 --2- 172.21.15.40:0/3032388081 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb5d8064ee0 0x7fb5d8067390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009990 tx=0x7fb5e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:39.853+0000 7fb5f4ff9700 1 -- 172.21.15.40:0/3032388081 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fb5f004fe20 con 0x7fb5f8163cb0 2023-11-08T08:20:40.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.061+0000 7fb5fdb5e700 1 -- 172.21.15.40:0/3032388081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5f80cc780 con 0x7fb5f8163cb0 2023-11-08T08:20:40.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.061+0000 7fb5f4ff9700 1 -- 172.21.15.40:0/3032388081 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4651 (secure 0 0 0) 0x7fb5f0053440 con 0x7fb5f8163cb0 2023-11-08T08:20:40.068 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:40.068 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4958,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2244078655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2244078655},{"type":"v1","addr":"172.21.15.40:6834","nonce":2244078655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/749537066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":749537066},{"type":"v1","addr":"172.21.15.169:6836","nonce":749537066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5328,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2307838686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2307838686},{"type":"v1","addr":"172.21.15.169:6837","nonce":2307838686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:20:34.294628+0000","modified":"2023-11-08T08:20:37.318141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/1791517264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1791517264},{"type":"v1","addr":"172.21.15.40:6835","nonce":1791517264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T08:20:40.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.065+0000 7fb5fdb5e700 1 -- 172.21.15.40:0/3032388081 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb5d8064ee0 msgr2=0x7fb5d8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:40.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.065+0000 7fb5fdb5e700 1 --2- 172.21.15.40:0/3032388081 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb5d8064ee0 0x7fb5d8067390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009990 tx=0x7fb5e8008040 comp rx=0 tx=0).stop 2023-11-08T08:20:40.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.065+0000 7fb5fdb5e700 1 -- 172.21.15.40:0/3032388081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5f8163cb0 msgr2=0x7fb5f80c7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:40.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.065+0000 7fb5fdb5e700 1 --2- 172.21.15.40:0/3032388081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5f8163cb0 0x7fb5f80c7160 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000e930 tx=0x7fb5f000ec40 comp rx=0 tx=0).stop 2023-11-08T08:20:40.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.065+0000 7fb5fdb5e700 1 -- 172.21.15.40:0/3032388081 shutdown_connections 2023-11-08T08:20:40.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.065+0000 7fb5fdb5e700 1 --2- 172.21.15.40:0/3032388081 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb5d8064ee0 0x7fb5d8067390 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.065+0000 7fb5fdb5e700 1 --2- 172.21.15.40:0/3032388081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5f8163cb0 0x7fb5f80c7160 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.065+0000 7fb5fdb5e700 1 --2- 172.21.15.40:0/3032388081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5f80cdfb0 0x7fb5f80c2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.065+0000 7fb5fdb5e700 1 --2- 172.21.15.40:0/3032388081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5f806d2f0 0x7fb5f8153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.065+0000 7fb5fdb5e700 1 -- 172.21.15.40:0/3032388081 >> 172.21.15.40:0/3032388081 conn(0x7fb5f80cf350 msgr2=0x7fb5f80afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:40.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.065+0000 7fb5fdb5e700 1 -- 172.21.15.40:0/3032388081 shutdown_connections 2023-11-08T08:20:40.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.065+0000 7fb5fdb5e700 1 -- 172.21.15.40:0/3032388081 wait complete. 2023-11-08T08:20:40.073 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 17 2023-11-08T08:20:40.495 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T08:20:34.294628+0000', 'modified': '2023-11-08T08:20:37.318141+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5331}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5331': {'gid': 5331, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.40:6835/1791517264', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 1791517264}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 1791517264}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T08:20:40.496 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T08:20:40.497 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-08T08:20:40.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.689+0000 7f7262d7a700 1 Processor -- start 2023-11-08T08:20:40.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.689+0000 7f7262d7a700 1 -- start start 2023-11-08T08:20:40.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.693+0000 7f7262d7a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f725c15a480 0x7f725c0cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.693+0000 7f7262d7a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725c15a850 con 0x7f725c0c8d10 2023-11-08T08:20:40.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.693+0000 7f7262d7a700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725c0cada0 con 0x7f725c069d60 2023-11-08T08:20:40.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.693+0000 7f7262d7a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725c0ce060 con 0x7f725c15a480 2023-11-08T08:20:40.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.693+0000 7f7260b16700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f725c15a480 0x7f725c0cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.693+0000 7f7253fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f725c0c8d10 0x7f725c0cc7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36770/0 (socket says 172.21.15.40:36770) 2023-11-08T08:20:40.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.693+0000 7f7253fff700 1 -- 172.21.15.40:0/2596441951 learned_addr learned my addr 172.21.15.40:0/2596441951 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:40.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.693+0000 7f7260b16700 1 -- 172.21.15.40:0/2596441951 >> v1:172.21.15.169:6789/0 conn(0x7f725c069d60 legacy=0x7f725c06a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:40.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.693+0000 7f7260b16700 1 -- 172.21.15.40:0/2596441951 >> v1:172.21.15.40:6789/0 conn(0x7f725c0c8d10 legacy=0x7f725c0cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:40.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.693+0000 7f7260b16700 1 -- 172.21.15.40:0/2596441951 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c0ce300 con 0x7f725c15a480 2023-11-08T08:20:40.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.693+0000 7f72537fe700 1 -- 172.21.15.40:0/2596441951 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1520719685 0 0) 0x7f725c15a850 con 0x7f725c0c8d10 2023-11-08T08:20:40.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.697+0000 7f7260b16700 1 --2- 172.21.15.40:0/2596441951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c15a480 0x7f725c0cc290 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7258009850 tx=0x7f7258009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f81ee65d8c5c75 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:40.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.697+0000 7f72537fe700 1 -- 172.21.15.40:0/2596441951 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258011830 con 0x7f725c15a480 2023-11-08T08:20:40.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.697+0000 7f72537fe700 1 -- 172.21.15.40:0/2596441951 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7258011990 con 0x7f725c15a480 2023-11-08T08:20:40.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.697+0000 7f72537fe700 1 -- 172.21.15.40:0/2596441951 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258011d20 con 0x7f725c15a480 2023-11-08T08:20:40.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.697+0000 7f7262d7a700 1 -- 172.21.15.40:0/2596441951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c15a480 msgr2=0x7f725c0cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:40.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.697+0000 7f7262d7a700 1 --2- 172.21.15.40:0/2596441951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c15a480 0x7f725c0cc290 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7258009850 tx=0x7f7258009b60 comp rx=0 tx=0).stop 2023-11-08T08:20:40.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.697+0000 7f7262d7a700 1 -- 172.21.15.40:0/2596441951 shutdown_connections 2023-11-08T08:20:40.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.697+0000 7f7262d7a700 1 --2- 172.21.15.40:0/2596441951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c15a480 0x7f725c0cc290 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.697+0000 7f7262d7a700 1 -- 172.21.15.40:0/2596441951 >> 172.21.15.40:0/2596441951 conn(0x7f725c0cf350 msgr2=0x7f725c0b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:40.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.697+0000 7f7262d7a700 1 -- 172.21.15.40:0/2596441951 shutdown_connections 2023-11-08T08:20:40.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.697+0000 7f7262d7a700 1 -- 172.21.15.40:0/2596441951 wait complete. 2023-11-08T08:20:40.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.701+0000 7f7262d7a700 1 Processor -- start 2023-11-08T08:20:40.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.701+0000 7f7262d7a700 1 -- start start 2023-11-08T08:20:40.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.701+0000 7f7262d7a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c069d60 0x7f725c1ef710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.701+0000 7f7262d7a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725c0c8d10 0x7f725c0af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.701+0000 7f7262d7a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f725c15a480 0x7f725c0af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.701+0000 7f7262d7a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c0b2110 con 0x7f725c069d60 2023-11-08T08:20:40.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.701+0000 7f7262d7a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c1f32a0 con 0x7f725c15a480 2023-11-08T08:20:40.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.701+0000 7f7262d7a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725c1f2ef0 con 0x7f725c0c8d10 2023-11-08T08:20:40.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.701+0000 7f7260b16700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c069d60 0x7f725c1ef710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.701+0000 7f7260b16700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c069d60 0x7f725c1ef710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48806/0 (socket says 172.21.15.40:48806) 2023-11-08T08:20:40.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.701+0000 7f7261317700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f725c15a480 0x7f725c0af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.701+0000 7f7260b16700 1 -- 172.21.15.40:0/1458656177 learned_addr learned my addr 172.21.15.40:0/1458656177 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:40.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.701+0000 7f7253fff700 1 --2- 172.21.15.40:0/1458656177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725c0c8d10 0x7f725c0af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.701+0000 7f7260b16700 1 -- 172.21.15.40:0/1458656177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725c0c8d10 msgr2=0x7f725c0af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:40.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.701+0000 7f7260b16700 1 --2- 172.21.15.40:0/1458656177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725c0c8d10 0x7f725c0af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.701+0000 7f7260b16700 1 -- 172.21.15.40:0/1458656177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f725c15a480 msgr2=0x7f725c0af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:40.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.701+0000 7f7260b16700 1 --2- 172.21.15.40:0/1458656177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f725c15a480 0x7f725c0af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.701+0000 7f7260b16700 1 -- 172.21.15.40:0/1458656177 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72580094b0 con 0x7f725c069d60 2023-11-08T08:20:40.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.705+0000 7f7260b16700 1 --2- 172.21.15.40:0/1458656177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c069d60 0x7f725c1ef710 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f72580090b0 tx=0x7f7258004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:40.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.705+0000 7f72517fa700 1 -- 172.21.15.40:0/1458656177 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725800ba70 con 0x7f725c069d60 2023-11-08T08:20:40.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.705+0000 7f72517fa700 1 -- 172.21.15.40:0/1458656177 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725801f070 con 0x7f725c069d60 2023-11-08T08:20:40.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.705+0000 7f72517fa700 1 -- 172.21.15.40:0/1458656177 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725800bdf0 con 0x7f725c069d60 2023-11-08T08:20:40.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.705+0000 7f7262d7a700 1 -- 172.21.15.40:0/1458656177 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725c0add50 con 0x7f725c069d60 2023-11-08T08:20:40.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.705+0000 7f7262d7a700 1 -- 172.21.15.40:0/1458656177 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725c0caa30 con 0x7f725c069d60 2023-11-08T08:20:40.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.709+0000 7f72517fa700 1 -- 172.21.15.40:0/1458656177 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f725800f050 con 0x7f725c069d60 2023-11-08T08:20:40.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.709+0000 7f72517fa700 1 --2- 172.21.15.40:0/1458656177 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7248064f10 0x7f72480673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.709+0000 7f72517fa700 1 -- 172.21.15.40:0/1458656177 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7258021070 con 0x7f725c069d60 2023-11-08T08:20:40.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7253fff700 1 --2- 172.21.15.40:0/1458656177 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7248064f10 0x7f72480673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7266002740 1 Processor -- start 2023-11-08T08:20:40.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7266002740 1 -- start start 2023-11-08T08:20:40.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7266002740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x21c7ae0 0x2477e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7266002740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7710 0x2478340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7266002740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2212790 0x2478880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7253fff700 1 --2- 172.21.15.40:0/1458656177 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7248064f10 0x7f72480673c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f725c0b1bf0 tx=0x7f724c004a70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:40.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7266002740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x2336ae0 con 0x21c7710 2023-11-08T08:20:40.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7266002740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x23a6bd0 con 0x2212790 2023-11-08T08:20:40.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7266002740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x239fed0 con 0x21c7ae0 2023-11-08T08:20:40.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7253fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7710 0x2478340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7253fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7710 0x2478340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48810/0 (socket says 172.21.15.40:48810) 2023-11-08T08:20:40.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7253fff700 1 -- 172.21.15.40:0/1840064681 learned_addr learned my addr 172.21.15.40:0/1840064681 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:40.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7260b16700 1 --2- 172.21.15.40:0/1840064681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x21c7ae0 0x2477e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7261317700 1 --2- 172.21.15.40:0/1840064681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2212790 0x2478880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7253fff700 1 -- 172.21.15.40:0/1840064681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x21c7ae0 msgr2=0x2477e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:40.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7253fff700 1 --2- 172.21.15.40:0/1840064681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x21c7ae0 0x2477e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7253fff700 1 -- 172.21.15.40:0/1840064681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2212790 msgr2=0x2478880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:40.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7253fff700 1 --2- 172.21.15.40:0/1840064681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2212790 0x2478880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7253fff700 1 -- 172.21.15.40:0/1840064681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x23c5cc0 con 0x21c7710 2023-11-08T08:20:40.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7261317700 1 --2- 172.21.15.40:0/1840064681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2212790 0x2478880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:40.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.713+0000 7f7253fff700 1 --2- 172.21.15.40:0/1840064681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7710 0x2478340 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f724c0137e0 tx=0x7f724c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f4a7457851f22b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:40.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7242ffd700 1 -- 172.21.15.40:0/1840064681 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c0012a0 con 0x21c7710 2023-11-08T08:20:40.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7242ffd700 1 -- 172.21.15.40:0/1840064681 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c001400 con 0x21c7710 2023-11-08T08:20:40.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7242ffd700 1 -- 172.21.15.40:0/1840064681 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c0017b0 con 0x21c7710 2023-11-08T08:20:40.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7266002740 1 -- 172.21.15.40:0/1840064681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7710 msgr2=0x2478340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:40.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7266002740 1 --2- 172.21.15.40:0/1840064681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7710 0x2478340 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f724c0137e0 tx=0x7f724c013000 comp rx=0 tx=0).stop 2023-11-08T08:20:40.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7266002740 1 -- 172.21.15.40:0/1840064681 shutdown_connections 2023-11-08T08:20:40.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7266002740 1 --2- 172.21.15.40:0/1840064681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2212790 0x2478880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7266002740 1 --2- 172.21.15.40:0/1840064681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7710 0x2478340 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7266002740 1 --2- 172.21.15.40:0/1840064681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x21c7ae0 0x2477e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7266002740 1 -- 172.21.15.40:0/1840064681 >> 172.21.15.40:0/1840064681 conn(0x21c7eb0 msgr2=0x2460700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:40.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7266002740 1 -- 172.21.15.40:0/1840064681 shutdown_connections 2023-11-08T08:20:40.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7266002740 1 -- 172.21.15.40:0/1840064681 wait complete. 2023-11-08T08:20:40.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7266002740 1 -- start start 2023-11-08T08:20:40.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7266002740 1 Processor -- start 2023-11-08T08:20:40.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7266002740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x21c7710 0x244e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7266002740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7ae0 0x2450190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7266002740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2212790 0x244ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7266002740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x224cb20 con 0x21c7ae0 2023-11-08T08:20:40.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7266002740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x2461da0 con 0x21c7710 2023-11-08T08:20:40.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7266002740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x2450860 con 0x2212790 2023-11-08T08:20:40.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.717+0000 7f7266002740 10 client.0 _ll_register_callbacks cb 0x23fc0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:20:40.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7253fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7ae0 0x2450190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7253fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7ae0 0x2450190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48814/0 (socket says 172.21.15.40:48814) 2023-11-08T08:20:40.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7253fff700 1 -- 172.21.15.40:0/1889549536 learned_addr learned my addr 172.21.15.40:0/1889549536 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:40.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7260b16700 1 --2- 172.21.15.40:0/1889549536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x21c7710 0x244e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7261317700 1 --2- 172.21.15.40:0/1889549536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2212790 0x244ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7253fff700 1 -- 172.21.15.40:0/1889549536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2212790 msgr2=0x244ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:40.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7253fff700 1 --2- 172.21.15.40:0/1889549536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2212790 0x244ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7253fff700 1 -- 172.21.15.40:0/1889549536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x21c7710 msgr2=0x244e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:40.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7253fff700 1 --2- 172.21.15.40:0/1889549536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x21c7710 0x244e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7253fff700 1 -- 172.21.15.40:0/1889549536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f724c013430 con 0x21c7ae0 2023-11-08T08:20:40.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7260b16700 1 --2- 172.21.15.40:0/1889549536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x21c7710 0x244e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:40.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7253fff700 1 --2- 172.21.15.40:0/1889549536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7ae0 0x2450190 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f724c024040 tx=0x7f724c007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937824891b6f0d69 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:40.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f723b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:20:40.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f723b7fe700 1 -- 172.21.15.40:0/1889549536 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c0012a0 con 0x21c7ae0 2023-11-08T08:20:40.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7266002740 1 -- 172.21.15.40:0/1889549536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2398f50 con 0x21c7ae0 2023-11-08T08:20:40.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f723b7fe700 1 -- 172.21.15.40:0/1889549536 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f724c01b050 con 0x21c7ae0 2023-11-08T08:20:40.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f723b7fe700 1 -- 172.21.15.40:0/1889549536 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f724c004c00 con 0x21c7ae0 2023-11-08T08:20:40.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7266002740 10 client.5511 fetch_fsmap learned FSMap version 17 2023-11-08T08:20:40.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7266002740 1 -- 172.21.15.40:0/1889549536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x22139e0 con 0x21c7ae0 2023-11-08T08:20:40.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f723b7fe700 1 -- 172.21.15.40:0/1889549536 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2398f50 con 0x21c7ae0 2023-11-08T08:20:40.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f723b7fe700 1 -- 172.21.15.40:0/1889549536 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 17) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f724c020630 con 0x21c7ae0 2023-11-08T08:20:40.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7266002740 10 client.5511 fetch_fsmap finished waiting for FSMap version 17 2023-11-08T08:20:40.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7266002740 10 client.5511 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T08:20:40.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7266002740 20 client.5511 populate_metadata read hostname 'smithi040' 2023-11-08T08:20:40.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7266002740 1 --2- 172.21.15.40:0/1889549536 >> [v2:172.21.15.40:6832/1791517264,v1:172.21.15.40:6835/1791517264] conn(0x24756b0 0x245ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7266002740 4 client.5511 mds_command: new command op to 5331 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-08T08:20:40.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7266002740 1 -- 172.21.15.40:0/1889549536 --> [v2:172.21.15.40:6832/1791517264,v1:172.21.15.40:6835/1791517264] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x2398c70 con 0x24756b0 2023-11-08T08:20:40.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.721+0000 7f7260b16700 1 --2- 172.21.15.40:0/1889549536 >> [v2:172.21.15.40:6832/1791517264,v1:172.21.15.40:6835/1791517264] conn(0x24756b0 0x245ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.725+0000 7f7260b16700 1 --2- 172.21.15.40:0/1889549536 >> [v2:172.21.15.40:6832/1791517264,v1:172.21.15.40:6835/1791517264] conn(0x24756b0 0x245ebc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=903a6207910f1cb server_cookie=ca54b938d7491582 in_seq=0 out_seq=0 2023-11-08T08:20:40.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.725+0000 7f723b7fe700 10 client.5511 ms_handle_connect on v2:172.21.15.40:6832/1791517264 2023-11-08T08:20:40.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.725+0000 7f723b7fe700 1 -- 172.21.15.40:0/1889549536 <== mds.0 v2:172.21.15.40:6832/1791517264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x2398c70 con 0x24756b0 2023-11-08T08:20:40.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.725+0000 7f723b7fe700 10 client.5511 handle_command_reply: tid=0 2023-11-08T08:20:40.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.725+0000 7f7266002740 1 client.5511 shutdown 2023-11-08T08:20:40.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.725+0000 7f7266002740 1 -- 172.21.15.40:0/1889549536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7ae0 msgr2=0x2450190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:40.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.725+0000 7f7266002740 1 --2- 172.21.15.40:0/1889549536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7ae0 0x2450190 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f724c024040 tx=0x7f724c007ef0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.725+0000 7f7266002740 1 -- 172.21.15.40:0/1889549536 shutdown_connections 2023-11-08T08:20:40.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.725+0000 7f7266002740 1 --2- 172.21.15.40:0/1889549536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2212790 0x244ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.725+0000 7f7266002740 1 --2- 172.21.15.40:0/1889549536 >> [v2:172.21.15.40:6832/1791517264,v1:172.21.15.40:6835/1791517264] conn(0x24756b0 0x245ebc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.725+0000 7f7266002740 1 --2- 172.21.15.40:0/1889549536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7ae0 0x2450190 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.725+0000 7f7266002740 1 --2- 172.21.15.40:0/1889549536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x21c7710 0x244e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.725+0000 7f723b7fe700 0 client.5511 ms_handle_reset on v2:172.21.15.40:6832/1791517264 2023-11-08T08:20:40.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.725+0000 7f7266002740 1 -- 172.21.15.40:0/1889549536 >> 172.21.15.40:0/1889549536 conn(0x21c7eb0 msgr2=0x24793c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:40.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.725+0000 7f7266002740 1 -- 172.21.15.40:0/1889549536 shutdown_connections 2023-11-08T08:20:40.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.725+0000 7f7266002740 1 -- 172.21.15.40:0/1889549536 wait complete. 2023-11-08T08:20:40.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.725+0000 7f7266002740 20 client.5511 trim_cache size 0 max 16384 2023-11-08T08:20:40.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.745+0000 7f7266002740 1 Processor -- start 2023-11-08T08:20:40.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.745+0000 7f7266002740 1 -- start start 2023-11-08T08:20:40.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f7266002740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x21c7710 0x245c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f7266002740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x21c7ae0 0x245ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f7266002740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2212790 0x245d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f7266002740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x2370ec0 con 0x2212790 2023-11-08T08:20:40.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f7266002740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x23a2a30 con 0x21c7710 2023-11-08T08:20:40.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f7266002740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x239ed60 con 0x21c7ae0 2023-11-08T08:20:40.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f7261317700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2212790 0x245d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f7261317700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2212790 0x245d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48824/0 (socket says 172.21.15.40:48824) 2023-11-08T08:20:40.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f7261317700 1 -- 172.21.15.40:0/1284724563 learned_addr learned my addr 172.21.15.40:0/1284724563 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:40.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f7253fff700 1 --2- 172.21.15.40:0/1284724563 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x21c7ae0 0x245ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f7260b16700 1 --2- 172.21.15.40:0/1284724563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x21c7710 0x245c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f7253fff700 1 -- 172.21.15.40:0/1284724563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x21c7710 msgr2=0x245c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:40.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f7253fff700 1 --2- 172.21.15.40:0/1284724563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x21c7710 0x245c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f7253fff700 1 -- 172.21.15.40:0/1284724563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2212790 msgr2=0x245d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:40.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f7253fff700 1 --2- 172.21.15.40:0/1284724563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2212790 0x245d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f7253fff700 1 -- 172.21.15.40:0/1284724563 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c0cd170 con 0x21c7ae0 2023-11-08T08:20:40.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f7261317700 1 --2- 172.21.15.40:0/1284724563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2212790 0x245d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:40.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f7253fff700 1 --2- 172.21.15.40:0/1284724563 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x21c7ae0 0x245ce90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f724c000c00 tx=0x7f724c02c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93484ce612424f1a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:40.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f72437fe700 1 -- 172.21.15.40:0/1284724563 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c007860 con 0x21c7ae0 2023-11-08T08:20:40.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f72437fe700 1 -- 172.21.15.40:0/1284724563 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c00b030 con 0x21c7ae0 2023-11-08T08:20:40.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.749+0000 7f72437fe700 1 -- 172.21.15.40:0/1284724563 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c020af0 con 0x21c7ae0 2023-11-08T08:20:40.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7266002740 1 -- 172.21.15.40:0/1284724563 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x21c7ae0 msgr2=0x245ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:40.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7266002740 1 --2- 172.21.15.40:0/1284724563 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x21c7ae0 0x245ce90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f724c000c00 tx=0x7f724c02c000 comp rx=0 tx=0).stop 2023-11-08T08:20:40.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7266002740 1 -- 172.21.15.40:0/1284724563 shutdown_connections 2023-11-08T08:20:40.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7266002740 1 --2- 172.21.15.40:0/1284724563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2212790 0x245d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7266002740 1 --2- 172.21.15.40:0/1284724563 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x21c7ae0 0x245ce90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7266002740 1 --2- 172.21.15.40:0/1284724563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x21c7710 0x245c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7266002740 1 -- 172.21.15.40:0/1284724563 >> 172.21.15.40:0/1284724563 conn(0x21c7eb0 msgr2=0x2478b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:40.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7266002740 1 -- 172.21.15.40:0/1284724563 shutdown_connections 2023-11-08T08:20:40.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7266002740 1 -- 172.21.15.40:0/1284724563 wait complete. 2023-11-08T08:20:40.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7266002740 1 -- start start 2023-11-08T08:20:40.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7266002740 1 Processor -- start 2023-11-08T08:20:40.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7266002740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x21c7710 0x2493c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7266002740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7ae0 0x24e2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7266002740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2212790 0x244d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7266002740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x239d2d0 con 0x21c7ae0 2023-11-08T08:20:40.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7266002740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x2375de0 con 0x21c7710 2023-11-08T08:20:40.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7266002740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x24e3170 con 0x2212790 2023-11-08T08:20:40.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7266002740 10 client.0 _ll_register_callbacks cb 0x234c2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:20:40.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7253fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7ae0 0x24e2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7253fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7ae0 0x24e2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48828/0 (socket says 172.21.15.40:48828) 2023-11-08T08:20:40.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.753+0000 7f7253fff700 1 -- 172.21.15.40:0/2197752872 learned_addr learned my addr 172.21.15.40:0/2197752872 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:40.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f7260b16700 1 --2- 172.21.15.40:0/2197752872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x21c7710 0x2493c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f7261317700 1 --2- 172.21.15.40:0/2197752872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2212790 0x244d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f7253fff700 1 -- 172.21.15.40:0/2197752872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2212790 msgr2=0x244d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:40.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f7253fff700 1 --2- 172.21.15.40:0/2197752872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2212790 0x244d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f7253fff700 1 -- 172.21.15.40:0/2197752872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x21c7710 msgr2=0x2493c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:40.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f7253fff700 1 --2- 172.21.15.40:0/2197752872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x21c7710 0x2493c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f7253fff700 1 -- 172.21.15.40:0/2197752872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725c0cada0 con 0x21c7ae0 2023-11-08T08:20:40.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f7253fff700 1 --2- 172.21.15.40:0/2197752872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7ae0 0x24e2c30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f724c013af0 tx=0x7f724c009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4c23af18473b17 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:40.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f723b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:20:40.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f723b7fe700 1 -- 172.21.15.40:0/2197752872 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c020cd0 con 0x21c7ae0 2023-11-08T08:20:40.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f723b7fe700 1 -- 172.21.15.40:0/2197752872 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f724c01b030 con 0x21c7ae0 2023-11-08T08:20:40.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f7266002740 1 -- 172.21.15.40:0/2197752872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f724c001400 con 0x21c7ae0 2023-11-08T08:20:40.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f723b7fe700 1 -- 172.21.15.40:0/2197752872 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f724c013650 con 0x21c7ae0 2023-11-08T08:20:40.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f7266002740 10 client.5517 fetch_fsmap learned FSMap version 17 2023-11-08T08:20:40.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f7266002740 1 -- 172.21.15.40:0/2197752872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x22139e0 con 0x21c7ae0 2023-11-08T08:20:40.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f723b7fe700 1 -- 172.21.15.40:0/2197752872 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c001400 con 0x21c7ae0 2023-11-08T08:20:40.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f723b7fe700 1 -- 172.21.15.40:0/2197752872 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 17) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f724c020630 con 0x21c7ae0 2023-11-08T08:20:40.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f7266002740 10 client.5517 fetch_fsmap finished waiting for FSMap version 17 2023-11-08T08:20:40.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f7266002740 10 client.5517 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T08:20:40.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f7266002740 20 client.5517 populate_metadata read hostname 'smithi040' 2023-11-08T08:20:40.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f7266002740 1 --2- 172.21.15.40:0/2197752872 >> [v2:172.21.15.40:6832/1791517264,v1:172.21.15.40:6835/1791517264] conn(0x244de80 0x2491030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f7266002740 4 client.5517 mds_command: new command op to 5331 tid=0[{"prefix": "status"}] 2023-11-08T08:20:40.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f7266002740 1 -- 172.21.15.40:0/2197752872 --> [v2:172.21.15.40:6832/1791517264,v1:172.21.15.40:6835/1791517264] -- command(tid 0: {"prefix": "status"}) v1 -- 0x2244260 con 0x244de80 2023-11-08T08:20:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.757+0000 7f7260b16700 1 --2- 172.21.15.40:0/2197752872 >> [v2:172.21.15.40:6832/1791517264,v1:172.21.15.40:6835/1791517264] conn(0x244de80 0x2491030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f723b7fe700 10 client.5517 ms_handle_connect on v2:172.21.15.40:6832/1791517264 2023-11-08T08:20:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7260b16700 1 --2- 172.21.15.40:0/2197752872 >> [v2:172.21.15.40:6832/1791517264,v1:172.21.15.40:6835/1791517264] conn(0x244de80 0x2491030 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14e71d81cee864ac server_cookie=327a8ed5ce8582f0 in_seq=0 out_seq=0 2023-11-08T08:20:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f723b7fe700 1 -- 172.21.15.40:0/2197752872 <== mds.0 v2:172.21.15.40:6832/1791517264 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x2244260 con 0x244de80 2023-11-08T08:20:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f723b7fe700 10 client.5517 handle_command_reply: tid=0 2023-11-08T08:20:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7266002740 1 client.5517 shutdown 2023-11-08T08:20:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7266002740 1 -- 172.21.15.40:0/2197752872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7ae0 msgr2=0x24e2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7266002740 1 --2- 172.21.15.40:0/2197752872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7ae0 0x24e2c30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f724c013af0 tx=0x7f724c009030 comp rx=0 tx=0).stop 2023-11-08T08:20:40.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7266002740 1 -- 172.21.15.40:0/2197752872 shutdown_connections 2023-11-08T08:20:40.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7266002740 1 --2- 172.21.15.40:0/2197752872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2212790 0x244d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7266002740 1 --2- 172.21.15.40:0/2197752872 >> [v2:172.21.15.40:6832/1791517264,v1:172.21.15.40:6835/1791517264] conn(0x244de80 0x2491030 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7266002740 1 --2- 172.21.15.40:0/2197752872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x21c7ae0 0x24e2c30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7266002740 1 --2- 172.21.15.40:0/2197752872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x21c7710 0x2493c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f723b7fe700 0 client.5517 ms_handle_reset on v2:172.21.15.40:6832/1791517264 2023-11-08T08:20:40.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7266002740 1 -- 172.21.15.40:0/2197752872 >> 172.21.15.40:0/2197752872 conn(0x21c7eb0 msgr2=0x24793c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:40.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7266002740 1 -- 172.21.15.40:0/2197752872 shutdown_connections 2023-11-08T08:20:40.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7266002740 1 -- 172.21.15.40:0/2197752872 wait complete. 2023-11-08T08:20:40.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7266002740 20 client.5517 trim_cache size 0 max 16384 2023-11-08T08:20:40.778 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:20:40.778 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "54ed45f4-d5b0-41c3-8c3b-b5a18b844f25", 2023-11-08T08:20:40.778 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2023-11-08T08:20:40.779 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 5331, 2023-11-08T08:20:40.779 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-11-08T08:20:40.779 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-11-08T08:20:40.780 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-11-08T08:20:40.780 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 6.4623397899999997, 2023-11-08T08:20:40.780 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 17, 2023-11-08T08:20:40.780 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 43, 2023-11-08T08:20:40.780 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 43, 2023-11-08T08:20:40.781 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 15.334856749 2023-11-08T08:20:40.781 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:20:40.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7262d7a700 1 -- 172.21.15.40:0/1458656177 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7248064f10 msgr2=0x7f72480673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:40.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7262d7a700 1 --2- 172.21.15.40:0/1458656177 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7248064f10 0x7f72480673c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f725c0b1bf0 tx=0x7f724c004a70 comp rx=0 tx=0).stop 2023-11-08T08:20:40.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7262d7a700 1 -- 172.21.15.40:0/1458656177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c069d60 msgr2=0x7f725c1ef710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:40.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7262d7a700 1 --2- 172.21.15.40:0/1458656177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c069d60 0x7f725c1ef710 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f72580090b0 tx=0x7f7258004770 comp rx=0 tx=0).stop 2023-11-08T08:20:40.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7262d7a700 1 -- 172.21.15.40:0/1458656177 shutdown_connections 2023-11-08T08:20:40.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7262d7a700 1 --2- 172.21.15.40:0/1458656177 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7248064f10 0x7f72480673c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7262d7a700 1 --2- 172.21.15.40:0/1458656177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f725c15a480 0x7f725c0af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7262d7a700 1 --2- 172.21.15.40:0/1458656177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f725c0c8d10 0x7f725c0af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7262d7a700 1 --2- 172.21.15.40:0/1458656177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c069d60 0x7f725c1ef710 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7262d7a700 1 -- 172.21.15.40:0/1458656177 >> 172.21.15.40:0/1458656177 conn(0x7f725c0cf350 msgr2=0x7f725c15de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:40.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7262d7a700 1 -- 172.21.15.40:0/1458656177 shutdown_connections 2023-11-08T08:20:40.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.761+0000 7f7262d7a700 1 -- 172.21.15.40:0/1458656177 wait complete. 2023-11-08T08:20:40.791 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T08:20:40.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.973+0000 7f7e5928c700 1 Processor -- start 2023-11-08T08:20:40.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.973+0000 7f7e5928c700 1 -- start start 2023-11-08T08:20:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.977+0000 7f7e5928c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7e5415c6d0 0x7f7e540b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.977+0000 7f7e5928c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e540cfc40 0x7f7e54069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.977+0000 7f7e5928c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5406d5b0 con 0x7f7e5406a130 2023-11-08T08:20:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.977+0000 7f7e5928c700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e540d0400 con 0x7f7e5415c6d0 2023-11-08T08:20:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.977+0000 7f7e5928c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5406a5b0 con 0x7f7e540cfc40 2023-11-08T08:20:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.977+0000 7f7e58a8b700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7e5406a130 0x7f7e540b7950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36772/0 (socket says 172.21.15.40:36772) 2023-11-08T08:20:40.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.977+0000 7f7e58a8b700 1 -- 172.21.15.40:0/1300222187 learned_addr learned my addr 172.21.15.40:0/1300222187 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:40.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.977+0000 7f7e53fff700 1 --2- 172.21.15.40:0/1300222187 >> v2:172.21.15.169:3300/0 conn(0x7f7e5415c6d0 0x7f7e540b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.977+0000 7f7e4bfff700 1 --2- 172.21.15.40:0/1300222187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e540cfc40 0x7f7e54069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.977+0000 7f7e537fe700 1 -- 172.21.15.40:0/1300222187 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1866256084 0 0) 0x7f7e5406d5b0 con 0x7f7e5406a130 2023-11-08T08:20:40.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.977+0000 7f7e537fe700 1 -- 172.21.15.40:0/1300222187 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e340035d0 con 0x7f7e5406a130 2023-11-08T08:20:40.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.977+0000 7f7e4bfff700 1 -- 172.21.15.40:0/1300222187 >> v2:172.21.15.169:3300/0 conn(0x7f7e5415c6d0 msgr2=0x7f7e540b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:40.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.977+0000 7f7e4bfff700 1 --2- 172.21.15.40:0/1300222187 >> v2:172.21.15.169:3300/0 conn(0x7f7e5415c6d0 0x7f7e540b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.977+0000 7f7e4bfff700 1 -- 172.21.15.40:0/1300222187 >> v1:172.21.15.40:6789/0 conn(0x7f7e5406a130 legacy=0x7f7e540b7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:40.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.977+0000 7f7e4bfff700 1 -- 172.21.15.40:0/1300222187 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e54072270 con 0x7f7e540cfc40 2023-11-08T08:20:40.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.977+0000 7f7e4bfff700 1 --2- 172.21.15.40:0/1300222187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e540cfc40 0x7f7e54069bf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009850 tx=0x7f7e3c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbb9bb094e68baff server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:40.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.977+0000 7f7e537fe700 1 -- 172.21.15.40:0/1300222187 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c011830 con 0x7f7e540cfc40 2023-11-08T08:20:40.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.977+0000 7f7e537fe700 1 -- 172.21.15.40:0/1300222187 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3c011990 con 0x7f7e540cfc40 2023-11-08T08:20:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.977+0000 7f7e537fe700 1 -- 172.21.15.40:0/1300222187 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c011d40 con 0x7f7e540cfc40 2023-11-08T08:20:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.981+0000 7f7e5928c700 1 -- 172.21.15.40:0/1300222187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e540cfc40 msgr2=0x7f7e54069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.981+0000 7f7e5928c700 1 --2- 172.21.15.40:0/1300222187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e540cfc40 0x7f7e54069bf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009850 tx=0x7f7e3c009b60 comp rx=0 tx=0).stop 2023-11-08T08:20:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.981+0000 7f7e5928c700 1 -- 172.21.15.40:0/1300222187 shutdown_connections 2023-11-08T08:20:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.981+0000 7f7e5928c700 1 --2- 172.21.15.40:0/1300222187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e540cfc40 0x7f7e54069bf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.981+0000 7f7e5928c700 1 --2- 172.21.15.40:0/1300222187 >> v2:172.21.15.169:3300/0 conn(0x7f7e5415c6d0 0x7f7e540b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.981+0000 7f7e5928c700 1 -- 172.21.15.40:0/1300222187 >> 172.21.15.40:0/1300222187 conn(0x7f7e540b6520 msgr2=0x7f7e54071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.981+0000 7f7e5928c700 1 -- 172.21.15.40:0/1300222187 shutdown_connections 2023-11-08T08:20:40.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.981+0000 7f7e5928c700 1 -- 172.21.15.40:0/1300222187 wait complete. 2023-11-08T08:20:40.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.985+0000 7f7e5928c700 1 Processor -- start 2023-11-08T08:20:40.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.985+0000 7f7e5928c700 1 -- start start 2023-11-08T08:20:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.985+0000 7f7e5928c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e5406a130 0x7f7e540c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.985+0000 7f7e5928c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e540cfc40 0x7f7e5415caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.985+0000 7f7e5928c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e5415c6d0 0x7f7e540cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:40.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.985+0000 7f7e5928c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540c9030 con 0x7f7e5406a130 2023-11-08T08:20:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.985+0000 7f7e5928c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5415d560 con 0x7f7e5415c6d0 2023-11-08T08:20:40.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.985+0000 7f7e5928c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5415d840 con 0x7f7e540cfc40 2023-11-08T08:20:40.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.989+0000 7f7e53fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e5406a130 0x7f7e540c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.989+0000 7f7e53fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e5406a130 0x7f7e540c8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48836/0 (socket says 172.21.15.40:48836) 2023-11-08T08:20:40.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.989+0000 7f7e53fff700 1 -- 172.21.15.40:0/2194846614 learned_addr learned my addr 172.21.15.40:0/2194846614 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:40.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.989+0000 7f7e58a8b700 1 --2- 172.21.15.40:0/2194846614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e5415c6d0 0x7f7e540cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.989+0000 7f7e4bfff700 1 --2- 172.21.15.40:0/2194846614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e540cfc40 0x7f7e5415caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:40.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.989+0000 7f7e53fff700 1 -- 172.21.15.40:0/2194846614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e540cfc40 msgr2=0x7f7e5415caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:40.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.989+0000 7f7e53fff700 1 --2- 172.21.15.40:0/2194846614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e540cfc40 0x7f7e5415caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.989+0000 7f7e53fff700 1 -- 172.21.15.40:0/2194846614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e5415c6d0 msgr2=0x7f7e540cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:40.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.989+0000 7f7e53fff700 1 --2- 172.21.15.40:0/2194846614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e5415c6d0 0x7f7e540cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:40.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.989+0000 7f7e53fff700 1 -- 172.21.15.40:0/2194846614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c0094b0 con 0x7f7e5406a130 2023-11-08T08:20:40.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.989+0000 7f7e53fff700 1 --2- 172.21.15.40:0/2194846614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e5406a130 0x7f7e540c8af0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7e4400e840 tx=0x7f7e4400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:40.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.989+0000 7f7e517fa700 1 -- 172.21.15.40:0/2194846614 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e440095a0 con 0x7f7e5406a130 2023-11-08T08:20:40.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.989+0000 7f7e517fa700 1 -- 172.21.15.40:0/2194846614 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e44013680 con 0x7f7e5406a130 2023-11-08T08:20:40.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.989+0000 7f7e5928c700 1 -- 172.21.15.40:0/2194846614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e54071c20 con 0x7f7e5406a130 2023-11-08T08:20:40.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.989+0000 7f7e517fa700 1 -- 172.21.15.40:0/2194846614 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e440139f0 con 0x7f7e5406a130 2023-11-08T08:20:40.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.989+0000 7f7e5928c700 1 -- 172.21.15.40:0/2194846614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e540d0120 con 0x7f7e5406a130 2023-11-08T08:20:40.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.993+0000 7f7e5928c700 1 -- 172.21.15.40:0/2194846614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e54071f00 con 0x7f7e5406a130 2023-11-08T08:20:40.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.993+0000 7f7e517fa700 1 -- 172.21.15.40:0/2194846614 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7e44023070 con 0x7f7e5406a130 2023-11-08T08:20:41.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.997+0000 7f7e517fa700 1 --2- 172.21.15.40:0/2194846614 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e38064ee0 0x7f7e38067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:41.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.997+0000 7f7e517fa700 1 -- 172.21.15.40:0/2194846614 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7e440820d0 con 0x7f7e5406a130 2023-11-08T08:20:41.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.997+0000 7f7e4bfff700 1 --2- 172.21.15.40:0/2194846614 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e38064ee0 0x7f7e38067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:41.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:40.997+0000 7f7e4bfff700 1 --2- 172.21.15.40:0/2194846614 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e38064ee0 0x7f7e38067390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7e540d0050 tx=0x7f7e3c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:41.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:41.001+0000 7f7e517fa700 1 -- 172.21.15.40:0/2194846614 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7e44050040 con 0x7f7e5406a130 2023-11-08T08:20:41.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:41.225+0000 7f7e5928c700 1 -- 172.21.15.40:0/2194846614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7e54073cb0 con 0x7f7e5406a130 2023-11-08T08:20:41.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:41.225+0000 7f7e517fa700 1 -- 172.21.15.40:0/2194846614 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7e44017070 con 0x7f7e5406a130 2023-11-08T08:20:41.230 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T08:20:41.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:41.229+0000 7f7e5928c700 1 -- 172.21.15.40:0/2194846614 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e38064ee0 msgr2=0x7f7e38067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:41.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:41.229+0000 7f7e5928c700 1 --2- 172.21.15.40:0/2194846614 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e38064ee0 0x7f7e38067390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7e540d0050 tx=0x7f7e3c008030 comp rx=0 tx=0).stop 2023-11-08T08:20:41.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:41.229+0000 7f7e5928c700 1 -- 172.21.15.40:0/2194846614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e5406a130 msgr2=0x7f7e540c8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:41.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:41.229+0000 7f7e5928c700 1 --2- 172.21.15.40:0/2194846614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e5406a130 0x7f7e540c8af0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7e4400e840 tx=0x7f7e4400eb50 comp rx=0 tx=0).stop 2023-11-08T08:20:41.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:41.229+0000 7f7e5928c700 1 -- 172.21.15.40:0/2194846614 shutdown_connections 2023-11-08T08:20:41.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:41.229+0000 7f7e5928c700 1 --2- 172.21.15.40:0/2194846614 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e38064ee0 0x7f7e38067390 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:41.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:41.229+0000 7f7e5928c700 1 --2- 172.21.15.40:0/2194846614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e5415c6d0 0x7f7e540cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:41.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:41.229+0000 7f7e5928c700 1 --2- 172.21.15.40:0/2194846614 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e540cfc40 0x7f7e5415caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:41.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:41.229+0000 7f7e5928c700 1 --2- 172.21.15.40:0/2194846614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e5406a130 0x7f7e540c8af0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:41.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:41.229+0000 7f7e5928c700 1 -- 172.21.15.40:0/2194846614 >> 172.21.15.40:0/2194846614 conn(0x7f7e540b6520 msgr2=0x7f7e541e6720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:41.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:41.229+0000 7f7e5928c700 1 -- 172.21.15.40:0/2194846614 shutdown_connections 2023-11-08T08:20:41.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:41.229+0000 7f7e5928c700 1 -- 172.21.15.40:0/2194846614 wait complete. 2023-11-08T08:20:41.257 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-08T08:20:41.257 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T08:20:41.257 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-11-08T08:20:41.258 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T08:20:41.258 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T08:20:41.258 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-08T08:20:41.259 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-08T08:20:41.259 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T08:20:41.259 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T08:20:41.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:41.260 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-11-08T08:20:41.280 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T08:20:41.281 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T08:20:41.281 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T08:20:41.281 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:20:41.281 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-11-08T08:20:41.282 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:20:41.282 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:20:41.282 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:20:41.282 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2023-11-08T08:20:41.282 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:20:41.283 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:20:41.283 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2023-11-08T08:20:41.283 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T08:20:41.283 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-11-08T08:20:41.283 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2023-11-08T08:20:41.284 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-08T08:20:41.284 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42279sec preferred_lft 42279sec 2023-11-08T08:20:41.284 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link 2023-11-08T08:20:41.284 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:20:41.284 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:20:41.285 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-11-08T08:20:41.285 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2023-11-08T08:20:41.285 INFO:teuthology.orchestra.run.smithi040.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-08T08:20:41.285 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 02:42:94:e4:3d:2e brd ff:ff:ff:ff:ff:ff 2023-11-08T08:20:41.285 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-08T08:20:41.285 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:20:41.286 INFO:teuthology.orchestra.run.smithi040.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T08:20:41.286 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0e:a3:2a:89:6f:47 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:20:41.286 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T08:20:41.286 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:20:41.286 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a4cc:6bff:fe07:287b/64 scope link 2023-11-08T08:20:41.287 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:20:41.287 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T08:20:41.287 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0e:a3:2a:89:6f:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T08:20:41.287 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ca3:2aff:fe89:6f47/64 scope link 2023-11-08T08:20:41.288 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:20:41.288 INFO:teuthology.orchestra.run.smithi040.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T08:20:41.288 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 72:b3:2e:65:5a:02 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T08:20:41.288 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::70b3:2eff:fe65:5a02/64 scope link 2023-11-08T08:20:41.288 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:20:41.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:41.290 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T08:20:41.367 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-11-08T08:20:41.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:41.371 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-11-08T08:20:41.437 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-11-08T08:20:41.437 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T08:20:41.438 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-08T08:20:41.438 INFO:teuthology.orchestra.run.smithi040.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-08T08:20:41.438 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-08T08:20:41.438 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T08:20:41.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:41.439 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-11-08T08:20:41.440 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-11-08T08:20:41.440 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-08T08:20:41.440 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-08T08:20:41.441 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-08T08:20:41.441 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-11-08T08:20:41.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:41.582 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-11-08T08:20:41.599 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T08:20:41.599 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T08:20:41.600 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T08:20:41.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:41.601 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2023-11-08T08:20:41.682 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T08:20:41.683 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T08:20:41.747 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T08:20:41.747 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T08:20:41.748 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T08:20:41.809 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-08T08:20:41.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:41.810 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:20:41.946 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:20:41.947 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T08:20:41.972 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:20:41.973 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T08:20:41.973 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-08T08:20:41.973 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@54ed45f4-d5b0-41c3-8c3b-b5a18b844f25.cephfs=/ 2023-11-08T08:20:41.974 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2023-11-08T08:20:41.978 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T08:20:41.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:41.979 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T08:20:42.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:42.014 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T08:20:42.093 INFO:teuthology.orchestra.run.smithi040.stdout:client5099 2023-11-08T08:20:42.094 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:20:42.095 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:20:42.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:42.160 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2023-11-08T08:20:42.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:42.239 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2023-11-08T08:20:42.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:42.311 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2023-11-08T08:20:42.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:42.377 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2023-11-08T08:20:42.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:42.449 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2023-11-08T08:20:42.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:42.517 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2023-11-08T08:20:42.595 INFO:teuthology.orchestra.run.smithi040.stdout:./ 2023-11-08T08:20:42.595 INFO:teuthology.orchestra.run.smithi040.stdout:./rfile 2023-11-08T08:20:42.595 INFO:teuthology.orchestra.run.smithi040.stdout:./parentdir 2023-11-08T08:20:42.596 INFO:teuthology.orchestra.run.smithi040.stdout:./parentdir/pfile 2023-11-08T08:20:42.596 INFO:teuthology.orchestra.run.smithi040.stdout:./parentdir/childdir 2023-11-08T08:20:42.596 INFO:teuthology.orchestra.run.smithi040.stdout:./parentdir/childdir/cfile 2023-11-08T08:20:42.597 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-08T08:20:42.598 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-11-08T08:20:42.598 DEBUG:teuthology.orchestra.run.smithi040:> import stat 2023-11-08T08:20:42.598 DEBUG:teuthology.orchestra.run.smithi040:> 2023-11-08T08:20:42.599 DEBUG:teuthology.orchestra.run.smithi040:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2023-11-08T08:20:42.599 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-11-08T08:20:42.741 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-11-08T08:20:42.931 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-08T08:20:42.932 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-11-08T08:20:42.932 DEBUG:teuthology.orchestra.run.smithi040:> import stat 2023-11-08T08:20:42.933 DEBUG:teuthology.orchestra.run.smithi040:> 2023-11-08T08:20:42.933 DEBUG:teuthology.orchestra.run.smithi040:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2023-11-08T08:20:42.933 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-11-08T08:20:43.077 INFO:teuthology.orchestra.run.smithi040.stdout:1099511627778 2023-11-08T08:20:43.258 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-08T08:20:43.259 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-11-08T08:20:43.259 DEBUG:teuthology.orchestra.run.smithi040:> import stat 2023-11-08T08:20:43.259 DEBUG:teuthology.orchestra.run.smithi040:> 2023-11-08T08:20:43.260 DEBUG:teuthology.orchestra.run.smithi040:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2023-11-08T08:20:43.260 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-11-08T08:20:43.416 INFO:teuthology.orchestra.run.smithi040.stdout:1099511627776 2023-11-08T08:20:43.595 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-08T08:20:43.596 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-11-08T08:20:43.596 DEBUG:teuthology.orchestra.run.smithi040:> import stat 2023-11-08T08:20:43.596 DEBUG:teuthology.orchestra.run.smithi040:> 2023-11-08T08:20:43.597 DEBUG:teuthology.orchestra.run.smithi040:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2023-11-08T08:20:43.597 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-11-08T08:20:43.733 INFO:teuthology.orchestra.run.smithi040.stdout:1099511628279 2023-11-08T08:20:43.926 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-08T08:20:43.927 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-11-08T08:20:43.927 DEBUG:teuthology.orchestra.run.smithi040:> import stat 2023-11-08T08:20:43.928 DEBUG:teuthology.orchestra.run.smithi040:> 2023-11-08T08:20:43.928 DEBUG:teuthology.orchestra.run.smithi040:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2023-11-08T08:20:43.928 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-11-08T08:20:44.066 INFO:teuthology.orchestra.run.smithi040.stdout:1099511627777 2023-11-08T08:20:44.257 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-08T08:20:44.257 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-11-08T08:20:44.258 DEBUG:teuthology.orchestra.run.smithi040:> import stat 2023-11-08T08:20:44.258 DEBUG:teuthology.orchestra.run.smithi040:> 2023-11-08T08:20:44.259 DEBUG:teuthology.orchestra.run.smithi040:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2023-11-08T08:20:44.259 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-11-08T08:20:44.405 INFO:teuthology.orchestra.run.smithi040.stdout:1099511628280 2023-11-08T08:20:44.595 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:20:44.595 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:20:44.654 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:20:44.654 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:20:44.720 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:20:44.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.941+0000 7f124b524700 1 Processor -- start 2023-11-08T08:20:44.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.941+0000 7f124b524700 1 -- start start 2023-11-08T08:20:44.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.945+0000 7f124b524700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f12441586d0 0x7f124415c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:44.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.945+0000 7f124b524700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12440cf520 0x7f12440c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:44.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.945+0000 7f124b524700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12440c94d0 con 0x7f12440cca50 2023-11-08T08:20:44.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.945+0000 7f124b524700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1244158aa0 con 0x7f12441586d0 2023-11-08T08:20:44.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.945+0000 7f124b524700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12440c9790 con 0x7f12440cf520 2023-11-08T08:20:44.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.945+0000 7f124a522700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f12441586d0 0x7f124415c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:44.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.945+0000 7f124a522700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f12441586d0 0x7f124415c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48838/0 (socket says 172.21.15.40:48838) 2023-11-08T08:20:44.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.945+0000 7f124a522700 1 -- 172.21.15.40:0/4054840874 learned_addr learned my addr 172.21.15.40:0/4054840874 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:44.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.945+0000 7f124ad23700 1 --2- 172.21.15.40:0/4054840874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12440cf520 0x7f12440c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:44.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.945+0000 7f124a522700 1 -- 172.21.15.40:0/4054840874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12440cf520 msgr2=0x7f12440c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:44.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.945+0000 7f124a522700 1 --2- 172.21.15.40:0/4054840874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12440cf520 0x7f12440c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:44.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.945+0000 7f124a522700 1 -- 172.21.15.40:0/4054840874 >> v1:172.21.15.169:6789/0 conn(0x7f12440cca50 legacy=0x7f12440cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:44.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.945+0000 7f124a522700 1 -- 172.21.15.40:0/4054840874 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124415d0f0 con 0x7f12441586d0 2023-11-08T08:20:44.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.945+0000 7f124a522700 1 --2- 172.21.15.40:0/4054840874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12441586d0 0x7f124415c1c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1240009910 tx=0x7f1240009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1de55fe3052db0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:44.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.945+0000 7f1249520700 1 -- 172.21.15.40:0/4054840874 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1240004230 con 0x7f12441586d0 2023-11-08T08:20:44.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.945+0000 7f1249520700 1 -- 172.21.15.40:0/4054840874 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1240004390 con 0x7f12441586d0 2023-11-08T08:20:44.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.945+0000 7f1249520700 1 -- 172.21.15.40:0/4054840874 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1240009e60 con 0x7f12441586d0 2023-11-08T08:20:44.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.949+0000 7f124b524700 1 -- 172.21.15.40:0/4054840874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12441586d0 msgr2=0x7f124415c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:44.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.949+0000 7f124b524700 1 --2- 172.21.15.40:0/4054840874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12441586d0 0x7f124415c1c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1240009910 tx=0x7f1240009c20 comp rx=0 tx=0).stop 2023-11-08T08:20:44.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.949+0000 7f124b524700 1 -- 172.21.15.40:0/4054840874 shutdown_connections 2023-11-08T08:20:44.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.949+0000 7f124b524700 1 --2- 172.21.15.40:0/4054840874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12440cf520 0x7f12440c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:44.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.949+0000 7f124b524700 1 --2- 172.21.15.40:0/4054840874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12441586d0 0x7f124415c1c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:44.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.949+0000 7f124b524700 1 -- 172.21.15.40:0/4054840874 >> 172.21.15.40:0/4054840874 conn(0x7f124406c790 msgr2=0x7f124406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:44.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.949+0000 7f124b524700 1 -- 172.21.15.40:0/4054840874 shutdown_connections 2023-11-08T08:20:44.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.949+0000 7f124b524700 1 -- 172.21.15.40:0/4054840874 wait complete. 2023-11-08T08:20:44.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f124b524700 1 Processor -- start 2023-11-08T08:20:44.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f124b524700 1 -- start start 2023-11-08T08:20:44.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f124b524700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12440cca50 0x7f1244159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:44.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f124b524700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12440cf520 0x7f124415b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:44.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f124b524700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12441586d0 0x7f124414f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:44.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f124b524700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124415a580 con 0x7f12440cf520 2023-11-08T08:20:44.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f124b524700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12440aedf0 con 0x7f12440cca50 2023-11-08T08:20:44.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f124b524700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1244159ce0 con 0x7f12441586d0 2023-11-08T08:20:44.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f1249d21700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12440cf520 0x7f124415b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:44.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f1249d21700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12440cf520 0x7f124415b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48844/0 (socket says 172.21.15.40:48844) 2023-11-08T08:20:44.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f1249d21700 1 -- 172.21.15.40:0/3777687620 learned_addr learned my addr 172.21.15.40:0/3777687620 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:44.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f124a522700 1 --2- 172.21.15.40:0/3777687620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12440cca50 0x7f1244159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:44.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f124ad23700 1 --2- 172.21.15.40:0/3777687620 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12441586d0 0x7f124414f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:44.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f1249d21700 1 -- 172.21.15.40:0/3777687620 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12441586d0 msgr2=0x7f124414f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:44.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f1249d21700 1 --2- 172.21.15.40:0/3777687620 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12441586d0 0x7f124414f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:44.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f1249d21700 1 -- 172.21.15.40:0/3777687620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12440cca50 msgr2=0x7f1244159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:44.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f1249d21700 1 --2- 172.21.15.40:0/3777687620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12440cca50 0x7f1244159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:44.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f1249d21700 1 -- 172.21.15.40:0/3777687620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1240009570 con 0x7f12440cf520 2023-11-08T08:20:44.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f124a522700 1 --2- 172.21.15.40:0/3777687620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12440cca50 0x7f1244159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:20:44.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f124ad23700 1 --2- 172.21.15.40:0/3777687620 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12441586d0 0x7f124414f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:20:44.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f1249d21700 1 --2- 172.21.15.40:0/3777687620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12440cf520 0x7f124415b560 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f123400b4a0 tx=0x7f123400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:44.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f123affd700 1 -- 172.21.15.40:0/3777687620 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1234011830 con 0x7f12440cf520 2023-11-08T08:20:44.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f123affd700 1 -- 172.21.15.40:0/3777687620 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1234011990 con 0x7f12440cf520 2023-11-08T08:20:44.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.953+0000 7f123affd700 1 -- 172.21.15.40:0/3777687620 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1234011d40 con 0x7f12440cf520 2023-11-08T08:20:44.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.957+0000 7f124b524700 1 -- 172.21.15.40:0/3777687620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124415c850 con 0x7f12440cf520 2023-11-08T08:20:44.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.957+0000 7f124b524700 1 -- 172.21.15.40:0/3777687620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124406b520 con 0x7f12440cf520 2023-11-08T08:20:44.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.957+0000 7f124b524700 1 -- 172.21.15.40:0/3777687620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f124415cb30 con 0x7f12440cf520 2023-11-08T08:20:44.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.957+0000 7f123affd700 1 -- 172.21.15.40:0/3777687620 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1234021070 con 0x7f12440cf520 2023-11-08T08:20:44.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.961+0000 7f123affd700 1 --2- 172.21.15.40:0/3777687620 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1228064ee0 0x7f1228067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:44.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.961+0000 7f123affd700 1 -- 172.21.15.40:0/3777687620 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1234082060 con 0x7f12440cf520 2023-11-08T08:20:44.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.961+0000 7f124a522700 1 --2- 172.21.15.40:0/3777687620 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1228064ee0 0x7f1228067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:44.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.961+0000 7f124a522700 1 --2- 172.21.15.40:0/3777687620 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1228064ee0 0x7f1228067390 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1240009910 tx=0x7f1240008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:44.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:44.969+0000 7f123affd700 1 -- 172.21.15.40:0/3777687620 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f123404ffd0 con 0x7f12440cf520 2023-11-08T08:20:45.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.153+0000 7f124b524700 1 -- 172.21.15.40:0/3777687620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12440aef80 con 0x7f12440cf520 2023-11-08T08:20:45.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.153+0000 7f123affd700 1 -- 172.21.15.40:0/3777687620 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+18127 (secure 0 0 0) 0x7f12340535f0 con 0x7f12440cf520 2023-11-08T08:20:45.161 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:45.161 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":44,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:20:44.974097+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:20:32.059150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-08T08:20:33.297064+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":4}]},{"pgid":"7.2e","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:20:45.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.157+0000 7f124b524700 1 -- 172.21.15.40:0/3777687620 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1228064ee0 msgr2=0x7f1228067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:45.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.157+0000 7f124b524700 1 --2- 172.21.15.40:0/3777687620 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1228064ee0 0x7f1228067390 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1240009910 tx=0x7f1240008030 comp rx=0 tx=0).stop 2023-11-08T08:20:45.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.157+0000 7f124b524700 1 -- 172.21.15.40:0/3777687620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12440cf520 msgr2=0x7f124415b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:45.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.157+0000 7f124b524700 1 --2- 172.21.15.40:0/3777687620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12440cf520 0x7f124415b560 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f123400b4a0 tx=0x7f123400b860 comp rx=0 tx=0).stop 2023-11-08T08:20:45.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.157+0000 7f124b524700 1 -- 172.21.15.40:0/3777687620 shutdown_connections 2023-11-08T08:20:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.157+0000 7f124b524700 1 --2- 172.21.15.40:0/3777687620 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1228064ee0 0x7f1228067390 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.157+0000 7f124b524700 1 --2- 172.21.15.40:0/3777687620 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12441586d0 0x7f124414f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.157+0000 7f124b524700 1 --2- 172.21.15.40:0/3777687620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12440cf520 0x7f124415b560 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.157+0000 7f124b524700 1 --2- 172.21.15.40:0/3777687620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12440cca50 0x7f1244159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.157+0000 7f124b524700 1 -- 172.21.15.40:0/3777687620 >> 172.21.15.40:0/3777687620 conn(0x7f124406c790 msgr2=0x7f12440ca080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:45.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.157+0000 7f124b524700 1 -- 172.21.15.40:0/3777687620 shutdown_connections 2023-11-08T08:20:45.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.157+0000 7f124b524700 1 -- 172.21.15.40:0/3777687620 wait complete. 2023-11-08T08:20:45.186 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T08:20:32.059150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T08:20:33.297064+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T08:20:45.187 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:20:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.361+0000 7feaef133700 1 Processor -- start 2023-11-08T08:20:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.361+0000 7feaef133700 1 -- start start 2023-11-08T08:20:45.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feaef133700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7feae80b0db0 0x7feae81602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:45.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feaef133700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae80ae440 con 0x7feae80ae070 2023-11-08T08:20:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feaef133700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae806c780 con 0x7feae80c91e0 2023-11-08T08:20:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feaececf700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7feae80ae070 0x7feae8070f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36778/0 (socket says 172.21.15.40:36778) 2023-11-08T08:20:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feaececf700 1 -- 172.21.15.40:0/3258393803 learned_addr learned my addr 172.21.15.40:0/3258393803 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feaef133700 1 -- 172.21.15.40:0/3258393803 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae806d300 con 0x7feae80b0db0 2023-11-08T08:20:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feae77fe700 1 -- 172.21.15.40:0/3258393803 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3174294521 0 0) 0x7feae80ae440 con 0x7feae80ae070 2023-11-08T08:20:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feae77fe700 1 -- 172.21.15.40:0/3258393803 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fead80035d0 con 0x7feae80ae070 2023-11-08T08:20:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feaed6d0700 1 --2- 172.21.15.40:0/3258393803 >> v2:172.21.15.169:3300/0 conn(0x7feae80b0db0 0x7feae81602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feae77fe700 1 -- 172.21.15.40:0/3258393803 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3208181982 0 0) 0x7fead80035d0 con 0x7feae80ae070 2023-11-08T08:20:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feae77fe700 1 -- 172.21.15.40:0/3258393803 >> v2:172.21.15.169:3300/0 conn(0x7feae80b0db0 msgr2=0x7feae81602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feae77fe700 1 --2- 172.21.15.40:0/3258393803 >> v2:172.21.15.169:3300/0 conn(0x7feae80b0db0 0x7feae81602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feae77fe700 1 -- 172.21.15.40:0/3258393803 >> v1:172.21.15.169:6789/0 conn(0x7feae80c91e0 legacy=0x7feae80b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feae77fe700 1 -- 172.21.15.40:0/3258393803 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae806ca60 con 0x7feae80ae070 2023-11-08T08:20:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feae77fe700 1 -- 172.21.15.40:0/3258393803 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7feadc001670 con 0x7feae80ae070 2023-11-08T08:20:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feae77fe700 1 -- 172.21.15.40:0/3258393803 >> v1:172.21.15.40:6789/0 conn(0x7feae80ae070 legacy=0x7feae8070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feae77fe700 1 --2- 172.21.15.40:0/3258393803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead8003a20 0x7fead8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feae77fe700 1 --2- 172.21.15.40:0/3258393803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fead8004870 0x7fead8004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feae77fe700 1 --2- 172.21.15.40:0/3258393803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fead8005220 0x7fead800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feae77fe700 1 -- 172.21.15.40:0/3258393803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae806d300 con 0x7fead8005220 2023-11-08T08:20:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feae77fe700 1 -- 172.21.15.40:0/3258393803 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead800c1b0 con 0x7fead8004870 2023-11-08T08:20:45.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feae77fe700 1 -- 172.21.15.40:0/3258393803 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead800c370 con 0x7fead8003a20 2023-11-08T08:20:45.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feaed6d0700 1 --2- 172.21.15.40:0/3258393803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fead8005220 0x7fead800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:45.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feaececf700 1 --2- 172.21.15.40:0/3258393803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead8003a20 0x7fead8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:45.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feae7fff700 1 --2- 172.21.15.40:0/3258393803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fead8004870 0x7fead8004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:45.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feaed6d0700 1 -- 172.21.15.40:0/3258393803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead8003a20 msgr2=0x7fead8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:45.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feaed6d0700 1 --2- 172.21.15.40:0/3258393803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead8003a20 0x7fead8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feaed6d0700 1 -- 172.21.15.40:0/3258393803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fead8004870 msgr2=0x7fead8004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:45.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feaed6d0700 1 --2- 172.21.15.40:0/3258393803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fead8004870 0x7fead8004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feaed6d0700 1 -- 172.21.15.40:0/3258393803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead80035d0 con 0x7fead8005220 2023-11-08T08:20:45.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feaed6d0700 1 -- 172.21.15.40:0/3258393803 reap_dead start 2023-11-08T08:20:45.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feaed6d0700 1 --2- 172.21.15.40:0/3258393803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fead8005220 0x7fead800ba30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7feae8066d70 tx=0x7fead80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5738b770bc4539bb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:45.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feae77fe700 1 -- 172.21.15.40:0/3258393803 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae0007df0 con 0x7fead8005220 2023-11-08T08:20:45.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feae77fe700 1 -- 172.21.15.40:0/3258393803 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae0003a40 con 0x7fead8005220 2023-11-08T08:20:45.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.365+0000 7feae77fe700 1 -- 172.21.15.40:0/3258393803 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae0003df0 con 0x7fead8005220 2023-11-08T08:20:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.369+0000 7feaef133700 1 -- 172.21.15.40:0/3258393803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fead8005220 msgr2=0x7fead800ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.369+0000 7feaef133700 1 --2- 172.21.15.40:0/3258393803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fead8005220 0x7fead800ba30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7feae8066d70 tx=0x7fead80058a0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.369+0000 7feaef133700 1 -- 172.21.15.40:0/3258393803 shutdown_connections 2023-11-08T08:20:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.369+0000 7feaef133700 1 --2- 172.21.15.40:0/3258393803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fead8005220 0x7fead800ba30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.369+0000 7feaef133700 1 -- 172.21.15.40:0/3258393803 >> 172.21.15.40:0/3258393803 conn(0x7feae80cec20 msgr2=0x7feae80ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:45.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.369+0000 7feaef133700 1 -- 172.21.15.40:0/3258393803 shutdown_connections 2023-11-08T08:20:45.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.369+0000 7feaef133700 1 -- 172.21.15.40:0/3258393803 wait complete. 2023-11-08T08:20:45.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.369+0000 7feaef133700 1 Processor -- start 2023-11-08T08:20:45.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feaef133700 1 -- start start 2023-11-08T08:20:45.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feaef133700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead8005220 0x7feae8148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:45.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feaef133700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feae815a160 0x7feae815a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:45.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feaef133700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feae80c6670 0x7feae80c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:45.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feaef133700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae815aaf0 con 0x7feae815a160 2023-11-08T08:20:45.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feaef133700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae80c7020 con 0x7feae80c6670 2023-11-08T08:20:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feaef133700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae80c72c0 con 0x7fead8005220 2023-11-08T08:20:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feae7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feae815a160 0x7feae815a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feae7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feae815a160 0x7feae815a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48862/0 (socket says 172.21.15.40:48862) 2023-11-08T08:20:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feae7fff700 1 -- 172.21.15.40:0/2811640313 learned_addr learned my addr 172.21.15.40:0/2811640313 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feaed6d0700 1 --2- 172.21.15.40:0/2811640313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feae80c6670 0x7feae80c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:45.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feaececf700 1 --2- 172.21.15.40:0/2811640313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead8005220 0x7feae8148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:45.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feae7fff700 1 -- 172.21.15.40:0/2811640313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead8005220 msgr2=0x7feae8148ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:45.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feae7fff700 1 --2- 172.21.15.40:0/2811640313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead8005220 0x7feae8148ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feae7fff700 1 -- 172.21.15.40:0/2811640313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feae80c6670 msgr2=0x7feae80c6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:45.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feae7fff700 1 --2- 172.21.15.40:0/2811640313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feae80c6670 0x7feae80c6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feae7fff700 1 -- 172.21.15.40:0/2811640313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae806d020 con 0x7feae815a160 2023-11-08T08:20:45.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feae7fff700 1 --2- 172.21.15.40:0/2811640313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feae815a160 0x7feae815a5b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fead400e580 tx=0x7fead400e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:45.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feae57fa700 1 -- 172.21.15.40:0/2811640313 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead40052e0 con 0x7feae815a160 2023-11-08T08:20:45.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feae57fa700 1 -- 172.21.15.40:0/2811640313 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead4005440 con 0x7feae815a160 2023-11-08T08:20:45.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feaef133700 1 -- 172.21.15.40:0/2811640313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae806cd40 con 0x7feae815a160 2023-11-08T08:20:45.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.373+0000 7feae57fa700 1 -- 172.21.15.40:0/2811640313 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead40038e0 con 0x7feae815a160 2023-11-08T08:20:45.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.377+0000 7feaef133700 1 -- 172.21.15.40:0/2811640313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae80ce960 con 0x7feae815a160 2023-11-08T08:20:45.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.377+0000 7feaef133700 1 -- 172.21.15.40:0/2811640313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae80726c0 con 0x7feae815a160 2023-11-08T08:20:45.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.377+0000 7feae57fa700 1 -- 172.21.15.40:0/2811640313 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fead401b030 con 0x7feae815a160 2023-11-08T08:20:45.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.381+0000 7feae57fa700 1 --2- 172.21.15.40:0/2811640313 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feacc064ee0 0x7feacc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:45.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.381+0000 7feae57fa700 1 -- 172.21.15.40:0/2811640313 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fead4081660 con 0x7feae815a160 2023-11-08T08:20:45.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.381+0000 7feaececf700 1 --2- 172.21.15.40:0/2811640313 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feacc064ee0 0x7feacc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:45.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.381+0000 7feaececf700 1 --2- 172.21.15.40:0/2811640313 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feacc064ee0 0x7feacc067390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feadc001050 tx=0x7feadc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:45.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.385+0000 7feae57fa700 1 -- 172.21.15.40:0/2811640313 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fead404c400 con 0x7feae815a160 2023-11-08T08:20:45.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.561+0000 7feaef133700 1 -- 172.21.15.40:0/2811640313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feae815d100 con 0x7feae815a160 2023-11-08T08:20:45.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.561+0000 7feae57fa700 1 -- 172.21.15.40:0/2811640313 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7fead40190a0 con 0x7feae815a160 2023-11-08T08:20:45.566 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:20:45.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.565+0000 7feaef133700 1 -- 172.21.15.40:0/2811640313 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feacc064ee0 msgr2=0x7feacc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:45.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.565+0000 7feaef133700 1 --2- 172.21.15.40:0/2811640313 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feacc064ee0 0x7feacc067390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feadc001050 tx=0x7feadc009040 comp rx=0 tx=0).stop 2023-11-08T08:20:45.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.565+0000 7feaef133700 1 -- 172.21.15.40:0/2811640313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feae815a160 msgr2=0x7feae815a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:45.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.565+0000 7feaef133700 1 --2- 172.21.15.40:0/2811640313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feae815a160 0x7feae815a5b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fead400e580 tx=0x7fead400e890 comp rx=0 tx=0).stop 2023-11-08T08:20:45.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.565+0000 7feaef133700 1 -- 172.21.15.40:0/2811640313 shutdown_connections 2023-11-08T08:20:45.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.565+0000 7feaef133700 1 --2- 172.21.15.40:0/2811640313 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7feacc064ee0 0x7feacc067390 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.565+0000 7feaef133700 1 --2- 172.21.15.40:0/2811640313 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feae80c6670 0x7feae80c6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.565+0000 7feaef133700 1 --2- 172.21.15.40:0/2811640313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feae815a160 0x7feae815a5b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.565+0000 7feaef133700 1 --2- 172.21.15.40:0/2811640313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead8005220 0x7feae8148ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.565+0000 7feaef133700 1 -- 172.21.15.40:0/2811640313 >> 172.21.15.40:0/2811640313 conn(0x7feae80cec20 msgr2=0x7feae80cc760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:45.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.565+0000 7feaef133700 1 -- 172.21.15.40:0/2811640313 shutdown_connections 2023-11-08T08:20:45.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.565+0000 7feaef133700 1 -- 172.21.15.40:0/2811640313 wait complete. 2023-11-08T08:20:45.594 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:20:45.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.773+0000 7fcbcf59e700 1 Processor -- start 2023-11-08T08:20:45.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.773+0000 7fcbcf59e700 1 -- start start 2023-11-08T08:20:45.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.777+0000 7fcbcf59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbd0070620 0x7fcbd00b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:45.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.777+0000 7fcbcf59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd015c520 con 0x7fcbd015c150 2023-11-08T08:20:45.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.777+0000 7fcbcf59e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd006cc50 con 0x7fcbd0069bf0 2023-11-08T08:20:45.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.777+0000 7fcbcf59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd00b4240 con 0x7fcbd0070620 2023-11-08T08:20:45.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.777+0000 7fcbce59c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcbd015c150 0x7fcbd006ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36792/0 (socket says 172.21.15.40:36792) 2023-11-08T08:20:45.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.777+0000 7fcbce59c700 1 -- 172.21.15.40:0/3866237795 learned_addr learned my addr 172.21.15.40:0/3866237795 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.777+0000 7fcbcdd9b700 1 --2- 172.21.15.40:0/3866237795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbd0070620 0x7fcbd00b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcd59a700 1 -- 172.21.15.40:0/3866237795 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2876823662 0 0) 0x7fcbd015c520 con 0x7fcbd015c150 2023-11-08T08:20:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcd59a700 1 -- 172.21.15.40:0/3866237795 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbb80035d0 con 0x7fcbd015c150 2023-11-08T08:20:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcd59a700 1 -- 172.21.15.40:0/3866237795 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 855762600 0 0) 0x7fcbd006cc50 con 0x7fcbd0069bf0 2023-11-08T08:20:45.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcd59a700 1 -- 172.21.15.40:0/3866237795 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbd015c520 con 0x7fcbd0069bf0 2023-11-08T08:20:45.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcd59a700 1 -- 172.21.15.40:0/3866237795 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2418310450 0 0) 0x7fcbb80035d0 con 0x7fcbd015c150 2023-11-08T08:20:45.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcd59a700 1 -- 172.21.15.40:0/3866237795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbd0070620 msgr2=0x7fcbd00b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:45.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcd59a700 1 --2- 172.21.15.40:0/3866237795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbd0070620 0x7fcbd00b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcd59a700 1 -- 172.21.15.40:0/3866237795 >> v1:172.21.15.169:6789/0 conn(0x7fcbd0069bf0 legacy=0x7fcbd006a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcd59a700 1 -- 172.21.15.40:0/3866237795 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd00d2450 con 0x7fcbd015c150 2023-11-08T08:20:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcd59a700 1 -- 172.21.15.40:0/3866237795 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7fcbc0001670 con 0x7fcbd015c150 2023-11-08T08:20:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcd59a700 1 -- 172.21.15.40:0/3866237795 >> v1:172.21.15.40:6789/0 conn(0x7fcbd015c150 legacy=0x7fcbd006ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcd59a700 1 --2- 172.21.15.40:0/3866237795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbd0070620 0x7fcbb8003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcd59a700 1 --2- 172.21.15.40:0/3866237795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbb8004140 0x7fcbb8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcd59a700 1 --2- 172.21.15.40:0/3866237795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbb8004db0 0x7fcbb8005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:45.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcd59a700 1 -- 172.21.15.40:0/3866237795 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd00b4240 con 0x7fcbb8004140 2023-11-08T08:20:45.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcd59a700 1 -- 172.21.15.40:0/3866237795 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb800bae0 con 0x7fcbb8004db0 2023-11-08T08:20:45.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcd59a700 1 -- 172.21.15.40:0/3866237795 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb800bca0 con 0x7fcbd0070620 2023-11-08T08:20:45.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcdd9b700 1 --2- 172.21.15.40:0/3866237795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbb8004140 0x7fcbb8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:45.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbced9d700 1 --2- 172.21.15.40:0/3866237795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbb8004db0 0x7fcbb8005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcdd9b700 1 -- 172.21.15.40:0/3866237795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbd0070620 msgr2=0x7fcbb8003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcdd9b700 1 --2- 172.21.15.40:0/3866237795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbd0070620 0x7fcbb8003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcdd9b700 1 -- 172.21.15.40:0/3866237795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbb8004db0 msgr2=0x7fcbb8005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcdd9b700 1 --2- 172.21.15.40:0/3866237795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbb8004db0 0x7fcbb8005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcdd9b700 1 -- 172.21.15.40:0/3866237795 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb80035d0 con 0x7fcbb8004140 2023-11-08T08:20:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcdd9b700 1 --2- 172.21.15.40:0/3866237795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbb8004140 0x7fcbb8004870 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0070a10 tx=0x7fcbb80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f1d1ffc45e1e0a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcd59a700 1 -- 172.21.15.40:0/3866237795 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4005a40 con 0x7fcbb8004140 2023-11-08T08:20:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcd59a700 1 -- 172.21.15.40:0/3866237795 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc4005ba0 con 0x7fcbb8004140 2023-11-08T08:20:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcd59a700 1 -- 172.21.15.40:0/3866237795 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc400e6c0 con 0x7fcbb8004140 2023-11-08T08:20:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcf59e700 1 -- 172.21.15.40:0/3866237795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbb8004140 msgr2=0x7fcbb8004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.781+0000 7fcbcf59e700 1 --2- 172.21.15.40:0/3866237795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbb8004140 0x7fcbb8004870 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0070a10 tx=0x7fcbb80058a0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.785+0000 7fcbced9d700 1 -- 172.21.15.40:0/3866237795 reap_dead start 2023-11-08T08:20:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.785+0000 7fcbcf59e700 1 -- 172.21.15.40:0/3866237795 shutdown_connections 2023-11-08T08:20:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.785+0000 7fcbcf59e700 1 -- 172.21.15.40:0/3866237795 >> 172.21.15.40:0/3866237795 conn(0x7fcbd00d3680 msgr2=0x7fcbd00cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.785+0000 7fcbcf59e700 1 -- 172.21.15.40:0/3866237795 shutdown_connections 2023-11-08T08:20:45.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.785+0000 7fcbcf59e700 1 -- 172.21.15.40:0/3866237795 wait complete. 2023-11-08T08:20:45.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.789+0000 7fcbcf59e700 1 Processor -- start 2023-11-08T08:20:45.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.789+0000 7fcbcf59e700 1 -- start start 2023-11-08T08:20:45.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.789+0000 7fcbcf59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbb8004140 0x7fcbd00cf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:45.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.789+0000 7fcbcf59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbd00c9a40 0x7fcbd0157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:45.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.789+0000 7fcbcf59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbd015c6b0 0x7fcbd015cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:45.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.789+0000 7fcbcf59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd00ced20 con 0x7fcbd015c6b0 2023-11-08T08:20:45.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.789+0000 7fcbcf59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd00c9f20 con 0x7fcbb8004140 2023-11-08T08:20:45.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.789+0000 7fcbcf59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd00ca200 con 0x7fcbd00c9a40 2023-11-08T08:20:45.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.789+0000 7fcbced9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbd015c6b0 0x7fcbd015cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:45.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.789+0000 7fcbced9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbd015c6b0 0x7fcbd015cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48872/0 (socket says 172.21.15.40:48872) 2023-11-08T08:20:45.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.789+0000 7fcbced9d700 1 -- 172.21.15.40:0/3616157754 learned_addr learned my addr 172.21.15.40:0/3616157754 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:45.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.789+0000 7fcbcdd9b700 1 --2- 172.21.15.40:0/3616157754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbd00c9a40 0x7fcbd0157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:45.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.789+0000 7fcbce59c700 1 --2- 172.21.15.40:0/3616157754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbb8004140 0x7fcbd00cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:45.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.789+0000 7fcbced9d700 1 -- 172.21.15.40:0/3616157754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbd00c9a40 msgr2=0x7fcbd0157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:45.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.789+0000 7fcbced9d700 1 --2- 172.21.15.40:0/3616157754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbd00c9a40 0x7fcbd0157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.789+0000 7fcbced9d700 1 -- 172.21.15.40:0/3616157754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbb8004140 msgr2=0x7fcbd00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:45.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.789+0000 7fcbced9d700 1 --2- 172.21.15.40:0/3616157754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbb8004140 0x7fcbd00cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.789+0000 7fcbced9d700 1 -- 172.21.15.40:0/3616157754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd00d05e0 con 0x7fcbd015c6b0 2023-11-08T08:20:45.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.793+0000 7fcbced9d700 1 --2- 172.21.15.40:0/3616157754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbd015c6b0 0x7fcbd015cb50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00731d0 tx=0x7fcbc800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:45.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.793+0000 7fcbbeffd700 1 -- 172.21.15.40:0/3616157754 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8003c30 con 0x7fcbd015c6b0 2023-11-08T08:20:45.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.793+0000 7fcbbeffd700 1 -- 172.21.15.40:0/3616157754 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc8003db0 con 0x7fcbd015c6b0 2023-11-08T08:20:45.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.793+0000 7fcbbeffd700 1 -- 172.21.15.40:0/3616157754 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8005310 con 0x7fcbd015c6b0 2023-11-08T08:20:45.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.793+0000 7fcbcf59e700 1 -- 172.21.15.40:0/3616157754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd00d2730 con 0x7fcbd015c6b0 2023-11-08T08:20:45.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.793+0000 7fcbcf59e700 1 -- 172.21.15.40:0/3616157754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd00d2140 con 0x7fcbd015c6b0 2023-11-08T08:20:45.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.793+0000 7fcbcf59e700 1 -- 172.21.15.40:0/3616157754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd0073d20 con 0x7fcbd015c6b0 2023-11-08T08:20:45.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.793+0000 7fcbbeffd700 1 -- 172.21.15.40:0/3616157754 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fcbc8005470 con 0x7fcbd015c6b0 2023-11-08T08:20:45.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.797+0000 7fcbbeffd700 1 --2- 172.21.15.40:0/3616157754 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcbb0064f10 0x7fcbb00673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:45.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.797+0000 7fcbbeffd700 1 -- 172.21.15.40:0/3616157754 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fcbc80822f0 con 0x7fcbd015c6b0 2023-11-08T08:20:45.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.797+0000 7fcbce59c700 1 --2- 172.21.15.40:0/3616157754 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcbb0064f10 0x7fcbb00673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:45.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.797+0000 7fcbce59c700 1 --2- 172.21.15.40:0/3616157754 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcbb0064f10 0x7fcbb00673c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00017a0 tx=0x7fcbc0001570 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:45.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.801+0000 7fcbbeffd700 1 -- 172.21.15.40:0/3616157754 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fcbc8050220 con 0x7fcbd015c6b0 2023-11-08T08:20:45.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.977+0000 7fcbcf59e700 1 -- 172.21.15.40:0/3616157754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcbd00d0150 con 0x7fcbd015c6b0 2023-11-08T08:20:45.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.977+0000 7fcbbeffd700 1 -- 172.21.15.40:0/3616157754 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v44) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcbc8053840 con 0x7fcbd015c6b0 2023-11-08T08:20:45.981 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:20:45.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.981+0000 7fcbcf59e700 1 -- 172.21.15.40:0/3616157754 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcbb0064f10 msgr2=0x7fcbb00673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:45.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.981+0000 7fcbcf59e700 1 --2- 172.21.15.40:0/3616157754 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcbb0064f10 0x7fcbb00673c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00017a0 tx=0x7fcbc0001570 comp rx=0 tx=0).stop 2023-11-08T08:20:45.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.981+0000 7fcbcf59e700 1 -- 172.21.15.40:0/3616157754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbd015c6b0 msgr2=0x7fcbd015cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:45.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.981+0000 7fcbcf59e700 1 --2- 172.21.15.40:0/3616157754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbd015c6b0 0x7fcbd015cb50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00731d0 tx=0x7fcbc800ee70 comp rx=0 tx=0).stop 2023-11-08T08:20:45.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.981+0000 7fcbcf59e700 1 -- 172.21.15.40:0/3616157754 shutdown_connections 2023-11-08T08:20:45.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.981+0000 7fcbcf59e700 1 --2- 172.21.15.40:0/3616157754 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcbb0064f10 0x7fcbb00673c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.981+0000 7fcbcf59e700 1 --2- 172.21.15.40:0/3616157754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbd015c6b0 0x7fcbd015cb50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.981+0000 7fcbcf59e700 1 --2- 172.21.15.40:0/3616157754 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbd00c9a40 0x7fcbd0157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.981+0000 7fcbcf59e700 1 --2- 172.21.15.40:0/3616157754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbb8004140 0x7fcbd00cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:45.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.981+0000 7fcbcf59e700 1 -- 172.21.15.40:0/3616157754 >> 172.21.15.40:0/3616157754 conn(0x7fcbd00d3680 msgr2=0x7fcbd006ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:45.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.981+0000 7fcbcf59e700 1 -- 172.21.15.40:0/3616157754 shutdown_connections 2023-11-08T08:20:45.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:45.981+0000 7fcbcf59e700 1 -- 172.21.15.40:0/3616157754 wait complete. 2023-11-08T08:20:46.010 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T08:20:46.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.181+0000 7fe59ca0f700 1 Processor -- start 2023-11-08T08:20:46.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.181+0000 7fe59ca0f700 1 -- start start 2023-11-08T08:20:46.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe59ca0f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe59815a110 0x7fe59806b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:46.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe59ca0f700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe5980b2470 0x7fe5980caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:46.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe59ca0f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe598071600 con 0x7fe5980c9560 2023-11-08T08:20:46.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe59ca0f700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5980cc2c0 con 0x7fe5980b2470 2023-11-08T08:20:46.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe59ca0f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59815a4e0 con 0x7fe59815a110 2023-11-08T08:20:46.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe596ffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe5980c9560 0x7fe5980b1d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36804/0 (socket says 172.21.15.40:36804) 2023-11-08T08:20:46.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe596ffd700 1 -- 172.21.15.40:0/2515121838 learned_addr learned my addr 172.21.15.40:0/2515121838 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:46.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe597fff700 1 --2- 172.21.15.40:0/2515121838 >> v2:172.21.15.169:3300/0 conn(0x7fe5980b2470 0x7fe5980caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:46.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe5977fe700 1 --2- 172.21.15.40:0/2515121838 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe59815a110 0x7fe59806b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe5967fc700 1 -- 172.21.15.40:0/2515121838 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2721820676 0 0) 0x7fe598071600 con 0x7fe5980c9560 2023-11-08T08:20:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe5967fc700 1 -- 172.21.15.40:0/2515121838 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5800035d0 con 0x7fe5980c9560 2023-11-08T08:20:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe5967fc700 1 -- 172.21.15.40:0/2515121838 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2083757079 0 0) 0x7fe5800035d0 con 0x7fe5980c9560 2023-11-08T08:20:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe5967fc700 1 -- 172.21.15.40:0/2515121838 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe59815a110 msgr2=0x7fe59806b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe5967fc700 1 --2- 172.21.15.40:0/2515121838 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe59815a110 0x7fe59806b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe5967fc700 1 -- 172.21.15.40:0/2515121838 >> v2:172.21.15.169:3300/0 conn(0x7fe5980b2470 msgr2=0x7fe5980caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:46.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe5967fc700 1 --2- 172.21.15.40:0/2515121838 >> v2:172.21.15.169:3300/0 conn(0x7fe5980b2470 0x7fe5980caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe5967fc700 1 -- 172.21.15.40:0/2515121838 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe598071850 con 0x7fe5980c9560 2023-11-08T08:20:46.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe5967fc700 1 -- 172.21.15.40:0/2515121838 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7fe58c001670 con 0x7fe5980c9560 2023-11-08T08:20:46.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe5967fc700 1 -- 172.21.15.40:0/2515121838 >> v1:172.21.15.40:6789/0 conn(0x7fe5980c9560 legacy=0x7fe5980b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:46.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe5967fc700 1 --2- 172.21.15.40:0/2515121838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe580003a20 0x7fe580003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:46.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe5967fc700 1 --2- 172.21.15.40:0/2515121838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe580004450 0x7fe580004910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:46.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe5967fc700 1 --2- 172.21.15.40:0/2515121838 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe59815a110 0x7fe580004fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:46.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe5967fc700 1 -- 172.21.15.40:0/2515121838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5980cc2c0 con 0x7fe580003a20 2023-11-08T08:20:46.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe5967fc700 1 -- 172.21.15.40:0/2515121838 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59815a4e0 con 0x7fe580004450 2023-11-08T08:20:46.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe5967fc700 1 -- 172.21.15.40:0/2515121838 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58000ba30 con 0x7fe59815a110 2023-11-08T08:20:46.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe5977fe700 1 --2- 172.21.15.40:0/2515121838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe580003a20 0x7fe580003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:46.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe596ffd700 1 --2- 172.21.15.40:0/2515121838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe580004450 0x7fe580004910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:46.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.185+0000 7fe597fff700 1 --2- 172.21.15.40:0/2515121838 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe59815a110 0x7fe580004fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:46.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.189+0000 7fe5977fe700 1 -- 172.21.15.40:0/2515121838 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe59815a110 msgr2=0x7fe580004fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:46.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.189+0000 7fe5977fe700 1 --2- 172.21.15.40:0/2515121838 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe59815a110 0x7fe580004fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.189+0000 7fe5977fe700 1 -- 172.21.15.40:0/2515121838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe580004450 msgr2=0x7fe580004910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:46.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.189+0000 7fe5977fe700 1 --2- 172.21.15.40:0/2515121838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe580004450 0x7fe580004910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.189+0000 7fe5977fe700 1 -- 172.21.15.40:0/2515121838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5800035d0 con 0x7fe580003a20 2023-11-08T08:20:46.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.189+0000 7fe5977fe700 1 --2- 172.21.15.40:0/2515121838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe580003a20 0x7fe580003e30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe588009b70 tx=0x7fe5800058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470add3e985c3f09 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:46.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.189+0000 7fe5967fc700 1 -- 172.21.15.40:0/2515121838 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58800e6c0 con 0x7fe580003a20 2023-11-08T08:20:46.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.189+0000 7fe5967fc700 1 -- 172.21.15.40:0/2515121838 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58800e820 con 0x7fe580003a20 2023-11-08T08:20:46.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.189+0000 7fe5967fc700 1 -- 172.21.15.40:0/2515121838 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58800eb90 con 0x7fe580003a20 2023-11-08T08:20:46.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.189+0000 7fe59ca0f700 1 -- 172.21.15.40:0/2515121838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe580003a20 msgr2=0x7fe580003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:46.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.189+0000 7fe59ca0f700 1 --2- 172.21.15.40:0/2515121838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe580003a20 0x7fe580003e30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe588009b70 tx=0x7fe5800058a0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.189+0000 7fe597fff700 1 -- 172.21.15.40:0/2515121838 reap_dead start 2023-11-08T08:20:46.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.189+0000 7fe59ca0f700 1 -- 172.21.15.40:0/2515121838 shutdown_connections 2023-11-08T08:20:46.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.189+0000 7fe59ca0f700 1 -- 172.21.15.40:0/2515121838 >> 172.21.15.40:0/2515121838 conn(0x7fe5980b0a20 msgr2=0x7fe5980b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:46.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.189+0000 7fe59ca0f700 1 -- 172.21.15.40:0/2515121838 shutdown_connections 2023-11-08T08:20:46.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.189+0000 7fe59ca0f700 1 -- 172.21.15.40:0/2515121838 wait complete. 2023-11-08T08:20:46.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.193+0000 7fe59ca0f700 1 Processor -- start 2023-11-08T08:20:46.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.193+0000 7fe59ca0f700 1 -- start start 2023-11-08T08:20:46.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.193+0000 7fe59ca0f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe598149120 0x7fe5981e1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:46.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.193+0000 7fe59ca0f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5981494f0 0x7fe5981e1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:46.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.193+0000 7fe59ca0f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5980c2a00 0x7fe5980c2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:46.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.193+0000 7fe59ca0f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5980c33b0 con 0x7fe5980c2a00 2023-11-08T08:20:46.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.193+0000 7fe59ca0f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe598156510 con 0x7fe5981494f0 2023-11-08T08:20:46.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.193+0000 7fe59ca0f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5980c35e0 con 0x7fe598149120 2023-11-08T08:20:46.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.197+0000 7fe597fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5980c2a00 0x7fe5980c2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:46.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.197+0000 7fe597fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5980c2a00 0x7fe5980c2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48894/0 (socket says 172.21.15.40:48894) 2023-11-08T08:20:46.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.197+0000 7fe597fff700 1 -- 172.21.15.40:0/1129228084 learned_addr learned my addr 172.21.15.40:0/1129228084 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:46.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.197+0000 7fe5977fe700 1 --2- 172.21.15.40:0/1129228084 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe598149120 0x7fe5981e1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:46.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.197+0000 7fe596ffd700 1 --2- 172.21.15.40:0/1129228084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5981494f0 0x7fe5981e1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:46.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.197+0000 7fe597fff700 1 -- 172.21.15.40:0/1129228084 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe598149120 msgr2=0x7fe5981e1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:46.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.197+0000 7fe597fff700 1 --2- 172.21.15.40:0/1129228084 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe598149120 0x7fe5981e1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.197+0000 7fe597fff700 1 -- 172.21.15.40:0/1129228084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5981494f0 msgr2=0x7fe5981e1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:46.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.197+0000 7fe597fff700 1 --2- 172.21.15.40:0/1129228084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5981494f0 0x7fe5981e1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.197+0000 7fe597fff700 1 -- 172.21.15.40:0/1129228084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59806a980 con 0x7fe5980c2a00 2023-11-08T08:20:46.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.197+0000 7fe597fff700 1 --2- 172.21.15.40:0/1129228084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5980c2a00 0x7fe5980c2e70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe598070700 tx=0x7fe5900114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:46.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.197+0000 7fe57ffff700 1 -- 172.21.15.40:0/1129228084 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59000e070 con 0x7fe5980c2a00 2023-11-08T08:20:46.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.197+0000 7fe57ffff700 1 -- 172.21.15.40:0/1129228084 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe590003d10 con 0x7fe5980c2a00 2023-11-08T08:20:46.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.197+0000 7fe57ffff700 1 -- 172.21.15.40:0/1129228084 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe590011960 con 0x7fe5980c2a00 2023-11-08T08:20:46.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.197+0000 7fe59ca0f700 1 -- 172.21.15.40:0/1129228084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59806a6a0 con 0x7fe5980c2a00 2023-11-08T08:20:46.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.197+0000 7fe59ca0f700 1 -- 172.21.15.40:0/1129228084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe59806ac60 con 0x7fe5980c2a00 2023-11-08T08:20:46.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.201+0000 7fe59ca0f700 1 -- 172.21.15.40:0/1129228084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe598073d20 con 0x7fe5980c2a00 2023-11-08T08:20:46.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.201+0000 7fe57ffff700 1 -- 172.21.15.40:0/1129228084 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fe590011ac0 con 0x7fe5980c2a00 2023-11-08T08:20:46.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.201+0000 7fe57ffff700 1 --2- 172.21.15.40:0/1129228084 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe584064ee0 0x7fe584067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:46.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.201+0000 7fe57ffff700 1 -- 172.21.15.40:0/1129228084 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fe590082670 con 0x7fe5980c2a00 2023-11-08T08:20:46.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.205+0000 7fe5977fe700 1 --2- 172.21.15.40:0/1129228084 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe584064ee0 0x7fe584067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:46.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.205+0000 7fe5977fe700 1 --2- 172.21.15.40:0/1129228084 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe584064ee0 0x7fe584067390 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe5980c99e0 tx=0x7fe58801f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:46.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.209+0000 7fe57ffff700 1 -- 172.21.15.40:0/1129228084 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fe5900505a0 con 0x7fe5980c2a00 2023-11-08T08:20:46.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.377+0000 7fe59ca0f700 1 -- 172.21.15.40:0/1129228084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe5980c9240 con 0x7fe5980c2a00 2023-11-08T08:20:46.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.381+0000 7fe57ffff700 1 -- 172.21.15.40:0/1129228084 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe590053bc0 con 0x7fe5980c2a00 2023-11-08T08:20:46.385 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:20:46.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.381+0000 7fe59ca0f700 1 -- 172.21.15.40:0/1129228084 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe584064ee0 msgr2=0x7fe584067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:46.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.381+0000 7fe59ca0f700 1 --2- 172.21.15.40:0/1129228084 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe584064ee0 0x7fe584067390 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe5980c99e0 tx=0x7fe58801f040 comp rx=0 tx=0).stop 2023-11-08T08:20:46.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.381+0000 7fe59ca0f700 1 -- 172.21.15.40:0/1129228084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5980c2a00 msgr2=0x7fe5980c2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:46.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.381+0000 7fe59ca0f700 1 --2- 172.21.15.40:0/1129228084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5980c2a00 0x7fe5980c2e70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe598070700 tx=0x7fe5900114b0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.381+0000 7fe59ca0f700 1 -- 172.21.15.40:0/1129228084 shutdown_connections 2023-11-08T08:20:46.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.381+0000 7fe59ca0f700 1 --2- 172.21.15.40:0/1129228084 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe584064ee0 0x7fe584067390 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.381+0000 7fe59ca0f700 1 --2- 172.21.15.40:0/1129228084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5980c2a00 0x7fe5980c2e70 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.381+0000 7fe59ca0f700 1 --2- 172.21.15.40:0/1129228084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5981494f0 0x7fe5981e1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.381+0000 7fe59ca0f700 1 --2- 172.21.15.40:0/1129228084 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe598149120 0x7fe5981e1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.381+0000 7fe59ca0f700 1 -- 172.21.15.40:0/1129228084 >> 172.21.15.40:0/1129228084 conn(0x7fe5980b0a20 msgr2=0x7fe5980b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:46.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.385+0000 7fe59ca0f700 1 -- 172.21.15.40:0/1129228084 shutdown_connections 2023-11-08T08:20:46.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.385+0000 7fe59ca0f700 1 -- 172.21.15.40:0/1129228084 wait complete. 2023-11-08T08:20:46.411 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T08:20:46.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.585+0000 7f86d6929700 1 Processor -- start 2023-11-08T08:20:46.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.585+0000 7f86d6929700 1 -- start start 2023-11-08T08:20:46.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.589+0000 7f86d6929700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f86d015c150 0x7f86d00b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:46.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.589+0000 7f86d6929700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86d0070360 0x7f86d0160340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:46.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.589+0000 7f86d6929700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d015c520 con 0x7f86d006ff10 2023-11-08T08:20:46.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.589+0000 7f86d6929700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86d00b4240 con 0x7f86d015c150 2023-11-08T08:20:46.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.589+0000 7f86d6929700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d006ac00 con 0x7f86d0070360 2023-11-08T08:20:46.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.589+0000 7f86d5927700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f86d015c150 0x7f86d00b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:46.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.589+0000 7f86d5126700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f86d006ff10 0x7f86d015fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36814/0 (socket says 172.21.15.40:36814) 2023-11-08T08:20:46.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.589+0000 7f86d5126700 1 -- 172.21.15.40:0/1895376966 learned_addr learned my addr 172.21.15.40:0/1895376966 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:46.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.589+0000 7f86d6128700 1 --2- 172.21.15.40:0/1895376966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86d0070360 0x7f86d0160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:46.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.593+0000 7f86d6128700 1 -- 172.21.15.40:0/1895376966 >> v2:172.21.15.169:3300/0 conn(0x7f86d015c150 msgr2=0x7f86d00b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:46.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.593+0000 7f86d4925700 1 -- 172.21.15.40:0/1895376966 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110044597 0 0) 0x7f86d015c520 con 0x7f86d006ff10 2023-11-08T08:20:46.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.593+0000 7f86d6128700 1 --2- 172.21.15.40:0/1895376966 >> v2:172.21.15.169:3300/0 conn(0x7f86d015c150 0x7f86d00b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.593+0000 7f86d6128700 1 -- 172.21.15.40:0/1895376966 >> v1:172.21.15.40:6789/0 conn(0x7f86d006ff10 legacy=0x7f86d015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:46.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.593+0000 7f86d6128700 1 -- 172.21.15.40:0/1895376966 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d006cc50 con 0x7f86d0070360 2023-11-08T08:20:46.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.593+0000 7f86d5927700 1 --2- 172.21.15.40:0/1895376966 >> v2:172.21.15.169:3300/0 conn(0x7f86d015c150 0x7f86d00b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:46.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.593+0000 7f86d6128700 1 --2- 172.21.15.40:0/1895376966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86d0070360 0x7f86d0160340 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f86c8009d90 tx=0x7f86c800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efe127fa8e8e861a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:46.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.593+0000 7f86d4925700 1 -- 172.21.15.40:0/1895376966 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c800e070 con 0x7f86d0070360 2023-11-08T08:20:46.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.593+0000 7f86d4925700 1 -- 172.21.15.40:0/1895376966 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c800bb20 con 0x7f86d0070360 2023-11-08T08:20:46.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.593+0000 7f86d4925700 1 -- 172.21.15.40:0/1895376966 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c8012430 con 0x7f86d0070360 2023-11-08T08:20:46.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.593+0000 7f86d6929700 1 -- 172.21.15.40:0/1895376966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86d0070360 msgr2=0x7f86d0160340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:46.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.593+0000 7f86d6929700 1 --2- 172.21.15.40:0/1895376966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86d0070360 0x7f86d0160340 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f86c8009d90 tx=0x7f86c800b330 comp rx=0 tx=0).stop 2023-11-08T08:20:46.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.593+0000 7f86d6929700 1 -- 172.21.15.40:0/1895376966 shutdown_connections 2023-11-08T08:20:46.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.593+0000 7f86d6929700 1 --2- 172.21.15.40:0/1895376966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86d0070360 0x7f86d0160340 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.593+0000 7f86d6929700 1 --2- 172.21.15.40:0/1895376966 >> v2:172.21.15.169:3300/0 conn(0x7f86d015c150 0x7f86d00b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.593+0000 7f86d6929700 1 -- 172.21.15.40:0/1895376966 >> 172.21.15.40:0/1895376966 conn(0x7f86d00d3680 msgr2=0x7f86d00cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:46.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.593+0000 7f86d6929700 1 -- 172.21.15.40:0/1895376966 shutdown_connections 2023-11-08T08:20:46.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.593+0000 7f86d6929700 1 -- 172.21.15.40:0/1895376966 wait complete. 2023-11-08T08:20:46.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.597+0000 7f86d6929700 1 Processor -- start 2023-11-08T08:20:46.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.601+0000 7f86d6929700 1 -- start start 2023-11-08T08:20:46.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.601+0000 7f86d6929700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86d006ff10 0x7f86d00b8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:46.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.601+0000 7f86d6929700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d0070360 0x7f86d01f13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:46.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.601+0000 7f86d6929700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86d015c150 0x7f86d0150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:46.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.601+0000 7f86d6929700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d01f18e0 con 0x7f86d0070360 2023-11-08T08:20:46.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.601+0000 7f86d6929700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d00b7260 con 0x7f86d006ff10 2023-11-08T08:20:46.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.601+0000 7f86d6929700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d00b7540 con 0x7f86d015c150 2023-11-08T08:20:46.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.601+0000 7f86d5126700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d0070360 0x7f86d01f13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:46.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.601+0000 7f86d5126700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d0070360 0x7f86d01f13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48896/0 (socket says 172.21.15.40:48896) 2023-11-08T08:20:46.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.601+0000 7f86d5126700 1 -- 172.21.15.40:0/2653374645 learned_addr learned my addr 172.21.15.40:0/2653374645 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:46.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.601+0000 7f86d5927700 1 --2- 172.21.15.40:0/2653374645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86d006ff10 0x7f86d00b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:46.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.601+0000 7f86d6128700 1 --2- 172.21.15.40:0/2653374645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86d015c150 0x7f86d0150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:46.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.601+0000 7f86d5126700 1 -- 172.21.15.40:0/2653374645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86d015c150 msgr2=0x7f86d0150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:46.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.601+0000 7f86d5126700 1 --2- 172.21.15.40:0/2653374645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86d015c150 0x7f86d0150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.601+0000 7f86d5126700 1 -- 172.21.15.40:0/2653374645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86d006ff10 msgr2=0x7f86d00b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:46.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.601+0000 7f86d5126700 1 --2- 172.21.15.40:0/2653374645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86d006ff10 0x7f86d00b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.601+0000 7f86d5126700 1 -- 172.21.15.40:0/2653374645 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86cc0094b0 con 0x7f86d0070360 2023-11-08T08:20:46.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.601+0000 7f86d5927700 1 --2- 172.21.15.40:0/2653374645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86d006ff10 0x7f86d00b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:46.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.601+0000 7f86d6128700 1 --2- 172.21.15.40:0/2653374645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86d015c150 0x7f86d0150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:46.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.601+0000 7f86d5126700 1 --2- 172.21.15.40:0/2653374645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d0070360 0x7f86d01f13a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f86c000c700 tx=0x7f86c000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:46.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.605+0000 7f86c67fc700 1 -- 172.21.15.40:0/2653374645 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c00074e0 con 0x7f86d0070360 2023-11-08T08:20:46.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.605+0000 7f86c67fc700 1 -- 172.21.15.40:0/2653374645 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c0007640 con 0x7f86d0070360 2023-11-08T08:20:46.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.605+0000 7f86c67fc700 1 -- 172.21.15.40:0/2653374645 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0005a50 con 0x7f86d0070360 2023-11-08T08:20:46.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.605+0000 7f86d6929700 1 -- 172.21.15.40:0/2653374645 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c8009a70 con 0x7f86d0070360 2023-11-08T08:20:46.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.605+0000 7f86d6929700 1 -- 172.21.15.40:0/2653374645 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d00d2730 con 0x7f86d0070360 2023-11-08T08:20:46.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.605+0000 7f86d6929700 1 -- 172.21.15.40:0/2653374645 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d00d2450 con 0x7f86d0070360 2023-11-08T08:20:46.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.605+0000 7f86c67fc700 1 -- 172.21.15.40:0/2653374645 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f86c0019030 con 0x7f86d0070360 2023-11-08T08:20:46.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.609+0000 7f86c67fc700 1 --2- 172.21.15.40:0/2653374645 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f86b4064ee0 0x7f86b4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:46.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.609+0000 7f86c67fc700 1 -- 172.21.15.40:0/2653374645 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f86c0081470 con 0x7f86d0070360 2023-11-08T08:20:46.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.609+0000 7f86d5927700 1 --2- 172.21.15.40:0/2653374645 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f86b4064ee0 0x7f86b4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:46.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.609+0000 7f86d5927700 1 --2- 172.21.15.40:0/2653374645 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f86b4064ee0 0x7f86b4067390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009d30 tx=0x7f86cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:46.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.613+0000 7f86c67fc700 1 -- 172.21.15.40:0/2653374645 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f86c004a400 con 0x7f86d0070360 2023-11-08T08:20:46.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.785+0000 7f86d6929700 1 -- 172.21.15.40:0/2653374645 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f86d00d05e0 con 0x7f86d0070360 2023-11-08T08:20:46.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.789+0000 7f86c67fc700 1 -- 172.21.15.40:0/2653374645 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+11 (secure 0 0 0) 0x7f86c0017020 con 0x7f86d0070360 2023-11-08T08:20:46.793 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-11-08T08:20:46.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.789+0000 7f86d6929700 1 -- 172.21.15.40:0/2653374645 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f86b4064ee0 msgr2=0x7f86b4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:46.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.789+0000 7f86d6929700 1 --2- 172.21.15.40:0/2653374645 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f86b4064ee0 0x7f86b4067390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009d30 tx=0x7f86cc008040 comp rx=0 tx=0).stop 2023-11-08T08:20:46.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.789+0000 7f86d6929700 1 -- 172.21.15.40:0/2653374645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d0070360 msgr2=0x7f86d01f13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:46.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.789+0000 7f86d6929700 1 --2- 172.21.15.40:0/2653374645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d0070360 0x7f86d01f13a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f86c000c700 tx=0x7f86c000cac0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.789+0000 7f86d6929700 1 -- 172.21.15.40:0/2653374645 shutdown_connections 2023-11-08T08:20:46.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.789+0000 7f86d6929700 1 --2- 172.21.15.40:0/2653374645 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f86b4064ee0 0x7f86b4067390 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.789+0000 7f86d6929700 1 --2- 172.21.15.40:0/2653374645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86d015c150 0x7f86d0150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.789+0000 7f86d6929700 1 --2- 172.21.15.40:0/2653374645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d0070360 0x7f86d01f13a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.789+0000 7f86d6929700 1 --2- 172.21.15.40:0/2653374645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86d006ff10 0x7f86d00b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:46.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.789+0000 7f86d6929700 1 -- 172.21.15.40:0/2653374645 >> 172.21.15.40:0/2653374645 conn(0x7f86d00d3680 msgr2=0x7f86d00c8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:46.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.789+0000 7f86d6929700 1 -- 172.21.15.40:0/2653374645 shutdown_connections 2023-11-08T08:20:46.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.789+0000 7f86d6929700 1 -- 172.21.15.40:0/2653374645 wait complete. 2023-11-08T08:20:46.820 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T08:20:46.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.989+0000 7fc308ab8700 1 Processor -- start 2023-11-08T08:20:46.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.989+0000 7fc308ab8700 1 -- start start 2023-11-08T08:20:46.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc308ab8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc30415c640 0x7fc3040d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:46.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc308ab8700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc304069e00 0x7fc30406a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:46.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc308ab8700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc30406c600 con 0x7fc304071aa0 2023-11-08T08:20:46.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc308ab8700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc30406cef0 con 0x7fc304069e00 2023-11-08T08:20:46.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc308ab8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30406a7c0 con 0x7fc30415c640 2023-11-08T08:20:47.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc2fbfff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc304071aa0 0x7fc30406d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36830/0 (socket says 172.21.15.40:36830) 2023-11-08T08:20:47.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc2fbfff700 1 -- 172.21.15.40:0/2637080481 learned_addr learned my addr 172.21.15.40:0/2637080481 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:47.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc3037fe700 1 --2- 172.21.15.40:0/2637080481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc30415c640 0x7fc3040d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:47.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc303fff700 1 --2- 172.21.15.40:0/2637080481 >> v2:172.21.15.169:3300/0 conn(0x7fc304069e00 0x7fc30406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:47.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc302ffd700 1 -- 172.21.15.40:0/2637080481 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2669674822 0 0) 0x7fc30406c600 con 0x7fc304071aa0 2023-11-08T08:20:47.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc302ffd700 1 -- 172.21.15.40:0/2637080481 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2e40035d0 con 0x7fc304071aa0 2023-11-08T08:20:47.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc302ffd700 1 -- 172.21.15.40:0/2637080481 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3051760741 0 0) 0x7fc2e40035d0 con 0x7fc304071aa0 2023-11-08T08:20:47.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc302ffd700 1 -- 172.21.15.40:0/2637080481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc30415c640 msgr2=0x7fc3040d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:47.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc302ffd700 1 --2- 172.21.15.40:0/2637080481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc30415c640 0x7fc3040d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:47.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc302ffd700 1 -- 172.21.15.40:0/2637080481 >> v2:172.21.15.169:3300/0 conn(0x7fc304069e00 msgr2=0x7fc30406a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:47.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc302ffd700 1 --2- 172.21.15.40:0/2637080481 >> v2:172.21.15.169:3300/0 conn(0x7fc304069e00 0x7fc30406a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:47.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc302ffd700 1 -- 172.21.15.40:0/2637080481 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3040722f0 con 0x7fc304071aa0 2023-11-08T08:20:47.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc302ffd700 1 -- 172.21.15.40:0/2637080481 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7fc2ec001670 con 0x7fc304071aa0 2023-11-08T08:20:47.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc302ffd700 1 -- 172.21.15.40:0/2637080481 >> v1:172.21.15.40:6789/0 conn(0x7fc304071aa0 legacy=0x7fc30406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:47.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc302ffd700 1 --2- 172.21.15.40:0/2637080481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc30415c640 0x7fc2e4003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:47.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc302ffd700 1 --2- 172.21.15.40:0/2637080481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2e4003fa0 0x7fc2e4004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:47.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc302ffd700 1 --2- 172.21.15.40:0/2637080481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2e4004db0 0x7fc2e4005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:47.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc302ffd700 1 -- 172.21.15.40:0/2637080481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30406cef0 con 0x7fc2e4003fa0 2023-11-08T08:20:47.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc302ffd700 1 -- 172.21.15.40:0/2637080481 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30406a7c0 con 0x7fc2e4004db0 2023-11-08T08:20:47.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc302ffd700 1 -- 172.21.15.40:0/2637080481 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e400ba30 con 0x7fc30415c640 2023-11-08T08:20:47.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc303fff700 1 --2- 172.21.15.40:0/2637080481 >> v2:172.21.15.169:3300/0 conn(0x7fc304069e00 0x7fc30406a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:47.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc2fbfff700 1 --2- 172.21.15.40:0/2637080481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2e4003fa0 0x7fc2e4004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:47.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc3037fe700 1 --2- 172.21.15.40:0/2637080481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc30415c640 0x7fc2e4003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:47.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.993+0000 7fc303fff700 1 --2- 172.21.15.40:0/2637080481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2e4004db0 0x7fc2e4005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:47.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.997+0000 7fc2fbfff700 1 -- 172.21.15.40:0/2637080481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc30415c640 msgr2=0x7fc2e4003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:47.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.997+0000 7fc2fbfff700 1 --2- 172.21.15.40:0/2637080481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc30415c640 0x7fc2e4003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:47.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.997+0000 7fc2fbfff700 1 -- 172.21.15.40:0/2637080481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2e4004db0 msgr2=0x7fc2e4005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:47.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.997+0000 7fc2fbfff700 1 --2- 172.21.15.40:0/2637080481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2e4004db0 0x7fc2e4005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:47.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.997+0000 7fc2fbfff700 1 -- 172.21.15.40:0/2637080481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e40035d0 con 0x7fc2e4003fa0 2023-11-08T08:20:47.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.997+0000 7fc2fbfff700 1 --2- 172.21.15.40:0/2637080481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2e4003fa0 0x7fc2e4004870 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00c320 tx=0x7fc2e40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e29897e795fe7d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:47.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.997+0000 7fc302ffd700 1 -- 172.21.15.40:0/2637080481 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec00ce60 con 0x7fc2e4003fa0 2023-11-08T08:20:47.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.997+0000 7fc302ffd700 1 -- 172.21.15.40:0/2637080481 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec005980 con 0x7fc2e4003fa0 2023-11-08T08:20:47.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.997+0000 7fc302ffd700 1 -- 172.21.15.40:0/2637080481 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec005d10 con 0x7fc2e4003fa0 2023-11-08T08:20:47.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.997+0000 7fc308ab8700 1 -- 172.21.15.40:0/2637080481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2e4003fa0 msgr2=0x7fc2e4004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:47.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.997+0000 7fc308ab8700 1 --2- 172.21.15.40:0/2637080481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2e4003fa0 0x7fc2e4004870 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00c320 tx=0x7fc2e40058a0 comp rx=0 tx=0).stop 2023-11-08T08:20:47.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.997+0000 7fc303fff700 1 -- 172.21.15.40:0/2637080481 reap_dead start 2023-11-08T08:20:47.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.997+0000 7fc308ab8700 1 -- 172.21.15.40:0/2637080481 shutdown_connections 2023-11-08T08:20:47.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.997+0000 7fc308ab8700 1 -- 172.21.15.40:0/2637080481 >> 172.21.15.40:0/2637080481 conn(0x7fc3040b6520 msgr2=0x7fc304071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:47.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.997+0000 7fc308ab8700 1 -- 172.21.15.40:0/2637080481 shutdown_connections 2023-11-08T08:20:47.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:46.997+0000 7fc308ab8700 1 -- 172.21.15.40:0/2637080481 wait complete. 2023-11-08T08:20:47.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.001+0000 7fc308ab8700 1 Processor -- start 2023-11-08T08:20:47.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.001+0000 7fc308ab8700 1 -- start start 2023-11-08T08:20:47.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.001+0000 7fc308ab8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc30409bfd0 0x7fc3040b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:47.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.001+0000 7fc308ab8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3040cc420 0x7fc3040b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:47.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.001+0000 7fc308ab8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3040caad0 0x7fc3040ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:47.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.001+0000 7fc308ab8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040cbca0 con 0x7fc30409bfd0 2023-11-08T08:20:47.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.001+0000 7fc308ab8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040cb090 con 0x7fc3040cc420 2023-11-08T08:20:47.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.001+0000 7fc308ab8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040cb370 con 0x7fc3040caad0 2023-11-08T08:20:47.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.001+0000 7fc3037fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc30409bfd0 0x7fc3040b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:47.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.001+0000 7fc3037fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc30409bfd0 0x7fc3040b3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48902/0 (socket says 172.21.15.40:48902) 2023-11-08T08:20:47.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.001+0000 7fc3037fe700 1 -- 172.21.15.40:0/2600604736 learned_addr learned my addr 172.21.15.40:0/2600604736 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:47.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.001+0000 7fc303fff700 1 --2- 172.21.15.40:0/2600604736 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3040caad0 0x7fc3040ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:47.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.001+0000 7fc2fbfff700 1 --2- 172.21.15.40:0/2600604736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3040cc420 0x7fc3040b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:47.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.005+0000 7fc3037fe700 1 -- 172.21.15.40:0/2600604736 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3040caad0 msgr2=0x7fc3040ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:47.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.005+0000 7fc3037fe700 1 --2- 172.21.15.40:0/2600604736 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3040caad0 0x7fc3040ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:47.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.005+0000 7fc3037fe700 1 -- 172.21.15.40:0/2600604736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3040cc420 msgr2=0x7fc3040b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:47.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.005+0000 7fc3037fe700 1 --2- 172.21.15.40:0/2600604736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3040cc420 0x7fc3040b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:47.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.005+0000 7fc3037fe700 1 -- 172.21.15.40:0/2600604736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30415ca10 con 0x7fc30409bfd0 2023-11-08T08:20:47.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.005+0000 7fc3037fe700 1 --2- 172.21.15.40:0/2600604736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc30409bfd0 0x7fc3040b3530 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4011a20 tx=0x7fc2f4011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:47.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.005+0000 7fc300ff9700 1 -- 172.21.15.40:0/2600604736 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4011ec0 con 0x7fc30409bfd0 2023-11-08T08:20:47.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.005+0000 7fc300ff9700 1 -- 172.21.15.40:0/2600604736 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f400f940 con 0x7fc30409bfd0 2023-11-08T08:20:47.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.005+0000 7fc300ff9700 1 -- 172.21.15.40:0/2600604736 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f400fcd0 con 0x7fc30409bfd0 2023-11-08T08:20:47.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.005+0000 7fc308ab8700 1 -- 172.21.15.40:0/2600604736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30406da20 con 0x7fc30409bfd0 2023-11-08T08:20:47.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.005+0000 7fc308ab8700 1 -- 172.21.15.40:0/2600604736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc304072010 con 0x7fc30409bfd0 2023-11-08T08:20:47.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.005+0000 7fc308ab8700 1 -- 172.21.15.40:0/2600604736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc304073cb0 con 0x7fc30409bfd0 2023-11-08T08:20:47.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.009+0000 7fc300ff9700 1 -- 172.21.15.40:0/2600604736 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc2f4014440 con 0x7fc30409bfd0 2023-11-08T08:20:47.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.013+0000 7fc300ff9700 1 --2- 172.21.15.40:0/2600604736 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc2e8064ee0 0x7fc2e8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:47.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.013+0000 7fc300ff9700 1 -- 172.21.15.40:0/2600604736 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fc2f4082c90 con 0x7fc30409bfd0 2023-11-08T08:20:47.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.013+0000 7fc2fbfff700 1 --2- 172.21.15.40:0/2600604736 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc2e8064ee0 0x7fc2e8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:47.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.013+0000 7fc2fbfff700 1 --2- 172.21.15.40:0/2600604736 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc2e8064ee0 0x7fc2e8067390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc304063810 tx=0x7fc2ec00a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:47.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.017+0000 7fc300ff9700 1 -- 172.21.15.40:0/2600604736 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc2f4050ca0 con 0x7fc30409bfd0 2023-11-08T08:20:47.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.201+0000 7fc308ab8700 1 -- 172.21.15.40:0/2600604736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc3040cbfb0 con 0x7fc30409bfd0 2023-11-08T08:20:47.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.205+0000 7fc300ff9700 1 -- 172.21.15.40:0/2600604736 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc2f40542c0 con 0x7fc30409bfd0 2023-11-08T08:20:47.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.205+0000 7fc308ab8700 1 -- 172.21.15.40:0/2600604736 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc2e8064ee0 msgr2=0x7fc2e8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:47.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.205+0000 7fc308ab8700 1 --2- 172.21.15.40:0/2600604736 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc2e8064ee0 0x7fc2e8067390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc304063810 tx=0x7fc2ec00a000 comp rx=0 tx=0).stop 2023-11-08T08:20:47.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.205+0000 7fc308ab8700 1 -- 172.21.15.40:0/2600604736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc30409bfd0 msgr2=0x7fc3040b3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:47.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.205+0000 7fc308ab8700 1 --2- 172.21.15.40:0/2600604736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc30409bfd0 0x7fc3040b3530 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4011a20 tx=0x7fc2f4011d30 comp rx=0 tx=0).stop 2023-11-08T08:20:47.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.205+0000 7fc308ab8700 1 -- 172.21.15.40:0/2600604736 shutdown_connections 2023-11-08T08:20:47.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.205+0000 7fc308ab8700 1 --2- 172.21.15.40:0/2600604736 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc2e8064ee0 0x7fc2e8067390 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:47.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.205+0000 7fc308ab8700 1 --2- 172.21.15.40:0/2600604736 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3040caad0 0x7fc3040ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:47.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.205+0000 7fc308ab8700 1 --2- 172.21.15.40:0/2600604736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3040cc420 0x7fc3040b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:47.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.205+0000 7fc308ab8700 1 --2- 172.21.15.40:0/2600604736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc30409bfd0 0x7fc3040b3530 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:47.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.205+0000 7fc308ab8700 1 -- 172.21.15.40:0/2600604736 >> 172.21.15.40:0/2600604736 conn(0x7fc3040b6520 msgr2=0x7fc3040d11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:47.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.205+0000 7fc308ab8700 1 -- 172.21.15.40:0/2600604736 shutdown_connections 2023-11-08T08:20:47.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.205+0000 7fc308ab8700 1 -- 172.21.15.40:0/2600604736 wait complete. 2023-11-08T08:20:47.214 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-11-08T08:20:47.236 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-08T08:20:47.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:47.237 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-08T08:20:47.341 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-11-08T08:20:47.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:20:47.342 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T08:20:47.356 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:20:47.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.577+0000 7fa43cc6d700 1 Processor -- start 2023-11-08T08:20:47.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.581+0000 7fa43cc6d700 1 -- start start 2023-11-08T08:20:47.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.581+0000 7fa43cc6d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43815c1a0 0x7fa43806c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.581+0000 7fa43cc6d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa4380cdeb0 0x7fa438070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.581+0000 7fa43cc6d700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4380adf90 con 0x7fa4380705c0 2023-11-08T08:20:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.581+0000 7fa43cc6d700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa438069bf0 con 0x7fa4380cdeb0 2023-11-08T08:20:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.581+0000 7fa43cc6d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa438070a40 con 0x7fa43815c1a0 2023-11-08T08:20:47.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.581+0000 7fa437fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa4380705c0 0x7fa43815fd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32910/0 (socket says 172.21.15.40:32910) 2023-11-08T08:20:47.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.581+0000 7fa437fff700 1 -- 172.21.15.40:0/3204989902 learned_addr learned my addr 172.21.15.40:0/3204989902 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:47.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.581+0000 7fa4377fe700 1 --2- 172.21.15.40:0/3204989902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43815c1a0 0x7fa43806c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:47.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.581+0000 7fa436ffd700 1 --2- 172.21.15.40:0/3204989902 >> v2:172.21.15.169:3300/0 conn(0x7fa4380cdeb0 0x7fa438070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:47.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.581+0000 7fa4367fc700 1 -- 172.21.15.40:0/3204989902 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198455480 0 0) 0x7fa4380adf90 con 0x7fa4380705c0 2023-11-08T08:20:47.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.581+0000 7fa4367fc700 1 -- 172.21.15.40:0/3204989902 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4200035d0 con 0x7fa4380705c0 2023-11-08T08:20:47.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.581+0000 7fa4377fe700 1 -- 172.21.15.40:0/3204989902 >> v2:172.21.15.169:3300/0 conn(0x7fa4380cdeb0 msgr2=0x7fa438070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:47.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.581+0000 7fa4377fe700 1 --2- 172.21.15.40:0/3204989902 >> v2:172.21.15.169:3300/0 conn(0x7fa4380cdeb0 0x7fa438070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:47.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.581+0000 7fa4377fe700 1 -- 172.21.15.40:0/3204989902 >> v1:172.21.15.40:6789/0 conn(0x7fa4380705c0 legacy=0x7fa43815fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:47.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.581+0000 7fa4377fe700 1 -- 172.21.15.40:0/3204989902 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4380ce370 con 0x7fa43815c1a0 2023-11-08T08:20:47.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.585+0000 7fa4377fe700 1 --2- 172.21.15.40:0/3204989902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43815c1a0 0x7fa43806c8b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa428009850 tx=0x7fa428009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cd83bd5bd208f92 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:47.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.585+0000 7fa4367fc700 1 -- 172.21.15.40:0/3204989902 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa428011830 con 0x7fa43815c1a0 2023-11-08T08:20:47.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.585+0000 7fa4367fc700 1 -- 172.21.15.40:0/3204989902 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa428011990 con 0x7fa43815c1a0 2023-11-08T08:20:47.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.585+0000 7fa4367fc700 1 -- 172.21.15.40:0/3204989902 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa428011d40 con 0x7fa43815c1a0 2023-11-08T08:20:47.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.585+0000 7fa43cc6d700 1 -- 172.21.15.40:0/3204989902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43815c1a0 msgr2=0x7fa43806c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:47.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.585+0000 7fa43cc6d700 1 --2- 172.21.15.40:0/3204989902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43815c1a0 0x7fa43806c8b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa428009850 tx=0x7fa428009b60 comp rx=0 tx=0).stop 2023-11-08T08:20:47.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.585+0000 7fa43cc6d700 1 -- 172.21.15.40:0/3204989902 shutdown_connections 2023-11-08T08:20:47.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.585+0000 7fa43cc6d700 1 --2- 172.21.15.40:0/3204989902 >> v2:172.21.15.169:3300/0 conn(0x7fa4380cdeb0 0x7fa438070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:47.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.585+0000 7fa43cc6d700 1 --2- 172.21.15.40:0/3204989902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa43815c1a0 0x7fa43806c8b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:47.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.585+0000 7fa43cc6d700 1 -- 172.21.15.40:0/3204989902 >> 172.21.15.40:0/3204989902 conn(0x7fa4380b0f80 msgr2=0x7fa4380ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:47.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.585+0000 7fa43cc6d700 1 -- 172.21.15.40:0/3204989902 shutdown_connections 2023-11-08T08:20:47.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.585+0000 7fa43cc6d700 1 -- 172.21.15.40:0/3204989902 wait complete. 2023-11-08T08:20:47.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.589+0000 7fa43cc6d700 1 Processor -- start 2023-11-08T08:20:47.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.589+0000 7fa43cc6d700 1 -- start start 2023-11-08T08:20:47.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa43cc6d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4380705c0 0x7fa4380ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:47.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa43cc6d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4380cdeb0 0x7fa43814c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:47.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa43cc6d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa43815c1a0 0x7fa4380c1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:47.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa43cc6d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4380b27f0 con 0x7fa43815c1a0 2023-11-08T08:20:47.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa43cc6d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4380b2ad0 con 0x7fa4380cdeb0 2023-11-08T08:20:47.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa43cc6d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4380b2db0 con 0x7fa4380705c0 2023-11-08T08:20:47.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa4377fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4380705c0 0x7fa4380ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:47.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa437fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa43815c1a0 0x7fa4380c1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:47.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa4377fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4380705c0 0x7fa4380ad3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:56256/0 (socket says 172.21.15.40:56256) 2023-11-08T08:20:47.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa436ffd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4380cdeb0 0x7fa43814c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:47.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa4377fe700 1 -- 172.21.15.40:0/2621818599 learned_addr learned my addr 172.21.15.40:0/2621818599 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:47.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa4377fe700 1 -- 172.21.15.40:0/2621818599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4380cdeb0 msgr2=0x7fa43814c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:47.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa4377fe700 1 --2- 172.21.15.40:0/2621818599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4380cdeb0 0x7fa43814c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:47.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa4377fe700 1 -- 172.21.15.40:0/2621818599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa43815c1a0 msgr2=0x7fa4380c1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:47.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa4377fe700 1 --2- 172.21.15.40:0/2621818599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa43815c1a0 0x7fa4380c1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:47.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa4377fe700 1 -- 172.21.15.40:0/2621818599 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4280094b0 con 0x7fa4380705c0 2023-11-08T08:20:47.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa4377fe700 1 --2- 172.21.15.40:0/2621818599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4380705c0 0x7fa4380ad3f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa4280090b0 tx=0x7fa428004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:47.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa41ffff700 1 -- 172.21.15.40:0/2621818599 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa428004380 con 0x7fa4380705c0 2023-11-08T08:20:47.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa41ffff700 1 -- 172.21.15.40:0/2621818599 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42801f070 con 0x7fa4380705c0 2023-11-08T08:20:47.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa41ffff700 1 -- 172.21.15.40:0/2621818599 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4280036c0 con 0x7fa4380705c0 2023-11-08T08:20:47.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa43cc6d700 1 -- 172.21.15.40:0/2621818599 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4380cc780 con 0x7fa4380705c0 2023-11-08T08:20:47.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.593+0000 7fa43cc6d700 1 -- 172.21.15.40:0/2621818599 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4380cc4a0 con 0x7fa4380705c0 2023-11-08T08:20:47.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.597+0000 7fa43cc6d700 1 -- 172.21.15.40:0/2621818599 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43806cdf0 con 0x7fa4380705c0 2023-11-08T08:20:47.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.597+0000 7fa41ffff700 1 -- 172.21.15.40:0/2621818599 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa42802a020 con 0x7fa4380705c0 2023-11-08T08:20:47.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.597+0000 7fa41ffff700 1 --2- 172.21.15.40:0/2621818599 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa424064f30 0x7fa4240673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:47.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.597+0000 7fa41ffff700 1 -- 172.21.15.40:0/2621818599 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fa428083400 con 0x7fa4380705c0 2023-11-08T08:20:47.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.601+0000 7fa436ffd700 1 --2- 172.21.15.40:0/2621818599 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa424064f30 0x7fa4240673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:47.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.601+0000 7fa436ffd700 1 --2- 172.21.15.40:0/2621818599 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa424064f30 0x7fa4240673e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa4381f4d80 tx=0x7fa42c009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:47.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.605+0000 7fa41ffff700 1 -- 172.21.15.40:0/2621818599 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa4280513b0 con 0x7fa4380705c0 2023-11-08T08:20:47.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.817+0000 7fa43cc6d700 1 -- 172.21.15.40:0/2621818599 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa438073cb0 con 0x7fa4380705c0 2023-11-08T08:20:47.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.817+0000 7fa41ffff700 1 -- 172.21.15.40:0/2621818599 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4651 (secure 0 0 0) 0x7fa428006ce0 con 0x7fa4380705c0 2023-11-08T08:20:47.824 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:47.825 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4958,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2244078655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2244078655},{"type":"v1","addr":"172.21.15.40:6834","nonce":2244078655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/749537066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":749537066},{"type":"v1","addr":"172.21.15.169:6836","nonce":749537066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5328,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2307838686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2307838686},{"type":"v1","addr":"172.21.15.169:6837","nonce":2307838686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:20:34.294628+0000","modified":"2023-11-08T08:20:37.318141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/1791517264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1791517264},{"type":"v1","addr":"172.21.15.40:6835","nonce":1791517264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T08:20:47.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.825+0000 7fa43cc6d700 1 -- 172.21.15.40:0/2621818599 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa424064f30 msgr2=0x7fa4240673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:47.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.825+0000 7fa43cc6d700 1 --2- 172.21.15.40:0/2621818599 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa424064f30 0x7fa4240673e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa4381f4d80 tx=0x7fa42c009240 comp rx=0 tx=0).stop 2023-11-08T08:20:47.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.825+0000 7fa43cc6d700 1 -- 172.21.15.40:0/2621818599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4380705c0 msgr2=0x7fa4380ad3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:47.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.825+0000 7fa43cc6d700 1 --2- 172.21.15.40:0/2621818599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4380705c0 0x7fa4380ad3f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa4280090b0 tx=0x7fa428004770 comp rx=0 tx=0).stop 2023-11-08T08:20:47.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.825+0000 7fa43cc6d700 1 -- 172.21.15.40:0/2621818599 shutdown_connections 2023-11-08T08:20:47.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.825+0000 7fa43cc6d700 1 --2- 172.21.15.40:0/2621818599 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa424064f30 0x7fa4240673e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:47.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.825+0000 7fa43cc6d700 1 --2- 172.21.15.40:0/2621818599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa43815c1a0 0x7fa4380c1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:47.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.825+0000 7fa43cc6d700 1 --2- 172.21.15.40:0/2621818599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4380cdeb0 0x7fa43814c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:47.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.825+0000 7fa43cc6d700 1 --2- 172.21.15.40:0/2621818599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4380705c0 0x7fa4380ad3f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:47.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.825+0000 7fa43cc6d700 1 -- 172.21.15.40:0/2621818599 >> 172.21.15.40:0/2621818599 conn(0x7fa4380b0f80 msgr2=0x7fa4380af370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:47.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.825+0000 7fa43cc6d700 1 -- 172.21.15.40:0/2621818599 shutdown_connections 2023-11-08T08:20:47.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:47.825+0000 7fa43cc6d700 1 -- 172.21.15.40:0/2621818599 wait complete. 2023-11-08T08:20:47.832 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 17 2023-11-08T08:20:47.852 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-11-08T08:20:48.065 INFO:teuthology.orchestra.run.smithi040.stdout:{"message":"","return_code":0} 2023-11-08T08:20:48.066 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-08T08:20:48.067 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:20:48.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.237+0000 7fbcd0924700 1 Processor -- start 2023-11-08T08:20:48.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.237+0000 7fbcd0924700 1 -- start start 2023-11-08T08:20:48.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.237+0000 7fbcd0924700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbccc0b43a0 0x7fbccc15e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:48.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.237+0000 7fbcd0924700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbccc0b1640 0x7fbccc070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:48.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.237+0000 7fbcd0924700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbccc0c9040 0x7fbccc0705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:48.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.237+0000 7fbcd0924700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbccc15a550 con 0x7fbccc0b1640 2023-11-08T08:20:48.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.237+0000 7fbcd0924700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbccc06b0a0 con 0x7fbccc0c9040 2023-11-08T08:20:48.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.237+0000 7fbcd0924700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0caac0 con 0x7fbccc0b43a0 2023-11-08T08:20:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.237+0000 7fbccaffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbccc0b1640 0x7fbccc070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.237+0000 7fbccaffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbccc0b1640 0x7fbccc070080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37544/0 (socket says 172.21.15.40:37544) 2023-11-08T08:20:48.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.237+0000 7fbccaffd700 1 -- 172.21.15.40:0/4116349017 learned_addr learned my addr 172.21.15.40:0/4116349017 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.237+0000 7fbccb7fe700 1 --2- 172.21.15.40:0/4116349017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbccc0b43a0 0x7fbccc15e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.237+0000 7fbccbfff700 1 --2- 172.21.15.40:0/4116349017 >> v2:172.21.15.169:3300/0 conn(0x7fbccc0c9040 0x7fbccc0705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.241+0000 7fbccaffd700 1 -- 172.21.15.40:0/4116349017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbccc0b43a0 msgr2=0x7fbccc15e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.241+0000 7fbccaffd700 1 --2- 172.21.15.40:0/4116349017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbccc0b43a0 0x7fbccc15e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:48.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.241+0000 7fbccaffd700 1 -- 172.21.15.40:0/4116349017 >> v2:172.21.15.169:3300/0 conn(0x7fbccc0c9040 msgr2=0x7fbccc0705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.241+0000 7fbccaffd700 1 --2- 172.21.15.40:0/4116349017 >> v2:172.21.15.169:3300/0 conn(0x7fbccc0c9040 0x7fbccc0705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.241+0000 7fbccaffd700 1 -- 172.21.15.40:0/4116349017 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc0cada0 con 0x7fbccc0b1640 2023-11-08T08:20:48.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.241+0000 7fbccaffd700 1 --2- 172.21.15.40:0/4116349017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbccc0b1640 0x7fbccc070080 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0c9f60 tx=0x7fbcc0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d13509809d3d5e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.241+0000 7fbcca7fc700 1 -- 172.21.15.40:0/4116349017 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0011830 con 0x7fbccc0b1640 2023-11-08T08:20:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.241+0000 7fbcca7fc700 1 -- 172.21.15.40:0/4116349017 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc0011990 con 0x7fbccc0b1640 2023-11-08T08:20:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.241+0000 7fbcca7fc700 1 -- 172.21.15.40:0/4116349017 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0011d20 con 0x7fbccc0b1640 2023-11-08T08:20:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.241+0000 7fbcd0924700 1 -- 172.21.15.40:0/4116349017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbccc0b1640 msgr2=0x7fbccc070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.241+0000 7fbcd0924700 1 --2- 172.21.15.40:0/4116349017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbccc0b1640 0x7fbccc070080 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0c9f60 tx=0x7fbcc0009bf0 comp rx=0 tx=0).stop 2023-11-08T08:20:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.241+0000 7fbcd0924700 1 -- 172.21.15.40:0/4116349017 shutdown_connections 2023-11-08T08:20:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.241+0000 7fbcd0924700 1 --2- 172.21.15.40:0/4116349017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbccc0b1640 0x7fbccc070080 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.241+0000 7fbcd0924700 1 --2- 172.21.15.40:0/4116349017 >> v2:172.21.15.169:3300/0 conn(0x7fbccc0c9040 0x7fbccc0705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:48.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.241+0000 7fbcd0924700 1 --2- 172.21.15.40:0/4116349017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbccc0b43a0 0x7fbccc15e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.241+0000 7fbcd0924700 1 -- 172.21.15.40:0/4116349017 >> 172.21.15.40:0/4116349017 conn(0x7fbccc0b0f80 msgr2=0x7fbccc0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.241+0000 7fbcd0924700 1 -- 172.21.15.40:0/4116349017 shutdown_connections 2023-11-08T08:20:48.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.241+0000 7fbcd0924700 1 -- 172.21.15.40:0/4116349017 wait complete. 2023-11-08T08:20:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.245+0000 7fbcd0924700 1 Processor -- start 2023-11-08T08:20:48.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.245+0000 7fbcd0924700 1 -- start start 2023-11-08T08:20:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.245+0000 7fbcd0924700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc0b1640 0x7fbccc1f3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.245+0000 7fbcd0924700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbccc0b43a0 0x7fbccc0c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.245+0000 7fbcd0924700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbccc0c9040 0x7fbccc0c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.245+0000 7fbcd0924700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc1f3980 con 0x7fbccc0b43a0 2023-11-08T08:20:48.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.245+0000 7fbcd0924700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc1f64d0 con 0x7fbccc0b1640 2023-11-08T08:20:48.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.245+0000 7fbcd0924700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc1f67b0 con 0x7fbccc0c9040 2023-11-08T08:20:48.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.245+0000 7fbccaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbccc0b43a0 0x7fbccc0c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:48.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.245+0000 7fbccaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbccc0b43a0 0x7fbccc0c6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37546/0 (socket says 172.21.15.40:37546) 2023-11-08T08:20:48.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.245+0000 7fbccaffd700 1 -- 172.21.15.40:0/4226797696 learned_addr learned my addr 172.21.15.40:0/4226797696 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:48.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.245+0000 7fbccb7fe700 1 --2- 172.21.15.40:0/4226797696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc0b1640 0x7fbccc1f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:48.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.245+0000 7fbccbfff700 1 --2- 172.21.15.40:0/4226797696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbccc0c9040 0x7fbccc0c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:48.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.249+0000 7fbccaffd700 1 -- 172.21.15.40:0/4226797696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbccc0c9040 msgr2=0x7fbccc0c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:48.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.249+0000 7fbccaffd700 1 --2- 172.21.15.40:0/4226797696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbccc0c9040 0x7fbccc0c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:48.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.249+0000 7fbccaffd700 1 -- 172.21.15.40:0/4226797696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc0b1640 msgr2=0x7fbccc1f3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:48.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.249+0000 7fbccaffd700 1 --2- 172.21.15.40:0/4226797696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc0b1640 0x7fbccc1f3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:48.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.249+0000 7fbccaffd700 1 -- 172.21.15.40:0/4226797696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc0009570 con 0x7fbccc0b43a0 2023-11-08T08:20:48.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.249+0000 7fbccb7fe700 1 --2- 172.21.15.40:0/4226797696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc0b1640 0x7fbccc1f3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:48.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.249+0000 7fbccaffd700 1 --2- 172.21.15.40:0/4226797696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbccc0b43a0 0x7fbccc0c6460 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009860 tx=0x7fbcc0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:48.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.249+0000 7fbcb3fff700 1 -- 172.21.15.40:0/4226797696 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc00044d0 con 0x7fbccc0b43a0 2023-11-08T08:20:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.249+0000 7fbcb3fff700 1 -- 172.21.15.40:0/4226797696 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc001f070 con 0x7fbccc0b43a0 2023-11-08T08:20:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.249+0000 7fbcd0924700 1 -- 172.21.15.40:0/4226797696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbccc06b5d0 con 0x7fbccc0b43a0 2023-11-08T08:20:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.249+0000 7fbcb3fff700 1 -- 172.21.15.40:0/4226797696 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0003900 con 0x7fbccc0b43a0 2023-11-08T08:20:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.249+0000 7fbcd0924700 1 -- 172.21.15.40:0/4226797696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbccc06b2f0 con 0x7fbccc0b43a0 2023-11-08T08:20:48.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.249+0000 7fbcd0924700 1 -- 172.21.15.40:0/4226797696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbccc15eb10 con 0x7fbccc0b43a0 2023-11-08T08:20:48.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.253+0000 7fbcb3fff700 1 -- 172.21.15.40:0/4226797696 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fbcc000f050 con 0x7fbccc0b43a0 2023-11-08T08:20:48.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.253+0000 7fbcb3fff700 1 --2- 172.21.15.40:0/4226797696 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbcb8064f10 0x7fbcb80673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:48.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.253+0000 7fbcb3fff700 1 -- 172.21.15.40:0/4226797696 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fbcc0021070 con 0x7fbccc0b43a0 2023-11-08T08:20:48.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.253+0000 7fbccb7fe700 1 --2- 172.21.15.40:0/4226797696 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbcb8064f10 0x7fbcb80673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:48.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.253+0000 7fbccb7fe700 1 --2- 172.21.15.40:0/4226797696 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbcb8064f10 0x7fbcb80673c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009bc0 tx=0x7fbcbc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:48.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.261+0000 7fbcb3fff700 1 -- 172.21.15.40:0/4226797696 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fbcc00517b0 con 0x7fbccc0b43a0 2023-11-08T08:20:48.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.469+0000 7fbcd0924700 1 -- 172.21.15.40:0/4226797696 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbccc073cb0 con 0x7fbccc0b43a0 2023-11-08T08:20:48.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.473+0000 7fbcb3fff700 1 -- 172.21.15.40:0/4226797696 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4651 (secure 0 0 0) 0x7fbcc0054dd0 con 0x7fbccc0b43a0 2023-11-08T08:20:48.477 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:48.478 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4958,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2244078655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2244078655},{"type":"v1","addr":"172.21.15.40:6834","nonce":2244078655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/749537066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":749537066},{"type":"v1","addr":"172.21.15.169:6836","nonce":749537066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5328,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2307838686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2307838686},{"type":"v1","addr":"172.21.15.169:6837","nonce":2307838686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:20:34.294628+0000","modified":"2023-11-08T08:20:37.318141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/1791517264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1791517264},{"type":"v1","addr":"172.21.15.40:6835","nonce":1791517264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T08:20:48.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.473+0000 7fbcd0924700 1 -- 172.21.15.40:0/4226797696 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbcb8064f10 msgr2=0x7fbcb80673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:48.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.473+0000 7fbcd0924700 1 --2- 172.21.15.40:0/4226797696 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbcb8064f10 0x7fbcb80673c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009bc0 tx=0x7fbcbc008040 comp rx=0 tx=0).stop 2023-11-08T08:20:48.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.473+0000 7fbcd0924700 1 -- 172.21.15.40:0/4226797696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbccc0b43a0 msgr2=0x7fbccc0c6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:48.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.473+0000 7fbcd0924700 1 --2- 172.21.15.40:0/4226797696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbccc0b43a0 0x7fbccc0c6460 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009860 tx=0x7fbcc0004770 comp rx=0 tx=0).stop 2023-11-08T08:20:48.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.473+0000 7fbcd0924700 1 -- 172.21.15.40:0/4226797696 shutdown_connections 2023-11-08T08:20:48.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.473+0000 7fbcd0924700 1 --2- 172.21.15.40:0/4226797696 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbcb8064f10 0x7fbcb80673c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:48.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.473+0000 7fbcd0924700 1 --2- 172.21.15.40:0/4226797696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbccc0c9040 0x7fbccc0c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:48.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.473+0000 7fbcd0924700 1 --2- 172.21.15.40:0/4226797696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbccc0b43a0 0x7fbccc0c6460 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:48.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.473+0000 7fbcd0924700 1 --2- 172.21.15.40:0/4226797696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbccc0b1640 0x7fbccc1f3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:48.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.473+0000 7fbcd0924700 1 -- 172.21.15.40:0/4226797696 >> 172.21.15.40:0/4226797696 conn(0x7fbccc0b0f80 msgr2=0x7fbccc06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:48.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.473+0000 7fbcd0924700 1 -- 172.21.15.40:0/4226797696 shutdown_connections 2023-11-08T08:20:48.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:48.477+0000 7fbcd0924700 1 -- 172.21.15.40:0/4226797696 wait complete. 2023-11-08T08:20:48.483 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 17 2023-11-08T08:20:48.502 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parentdir mytag 2023-11-08T08:20:48.710 INFO:teuthology.orchestra.run.smithi040.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2023-11-08T08:20:48.711 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2023-11-08T08:20:56.539 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:20:56.547 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:20:58.712 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2023-11-08T08:20:58.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.781+0000 7fd15ca37140 1 Processor -- start 2023-11-08T08:20:58.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.781+0000 7fd15ca37140 1 -- start start 2023-11-08T08:20:58.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.781+0000 7fd15ca37140 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561416623700 0x56141664bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:58.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.781+0000 7fd15ca37140 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561416656980 0x56141664c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:58.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.781+0000 7fd15ca37140 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614166e1f40 0x56141664d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:58.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.781+0000 7fd15ca37140 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141654b240 con 0x561416656980 2023-11-08T08:20:58.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.781+0000 7fd15ca37140 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141664d710 con 0x5614166e1f40 2023-11-08T08:20:58.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.781+0000 7fd15ca37140 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x56141664f430 con 0x561416623700 2023-11-08T08:20:58.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.781+0000 7fd159d6b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561416656980 0x56141664c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:58.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.781+0000 7fd159d6b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561416656980 0x56141664c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42460/0 (socket says 172.21.15.40:42460) 2023-11-08T08:20:58.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.781+0000 7fd159d6b700 1 -- 172.21.15.40:0/1081177657 learned_addr learned my addr 172.21.15.40:0/1081177657 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:58.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.781+0000 7fd15a56c700 1 --2- 172.21.15.40:0/1081177657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561416623700 0x56141664bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:58.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.781+0000 7fd15ad6d700 1 --2- 172.21.15.40:0/1081177657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614166e1f40 0x56141664d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.781+0000 7fd159d6b700 1 -- 172.21.15.40:0/1081177657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561416623700 msgr2=0x56141664bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.781+0000 7fd159d6b700 1 --2- 172.21.15.40:0/1081177657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561416623700 0x56141664bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.781+0000 7fd159d6b700 1 -- 172.21.15.40:0/1081177657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614166e1f40 msgr2=0x56141664d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.781+0000 7fd159d6b700 1 --2- 172.21.15.40:0/1081177657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614166e1f40 0x56141664d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.781+0000 7fd159d6b700 1 -- 172.21.15.40:0/1081177657 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd150005d20 con 0x561416656980 2023-11-08T08:20:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.785+0000 7fd159d6b700 1 --2- 172.21.15.40:0/1081177657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561416656980 0x56141664c0b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x56141662c1c0 tx=0x7fd15000d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d318ee43c72514ae server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.785+0000 7fd158d69700 1 -- 172.21.15.40:0/1081177657 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15001a040 con 0x561416656980 2023-11-08T08:20:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.785+0000 7fd158d69700 1 -- 172.21.15.40:0/1081177657 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd150008900 con 0x561416656980 2023-11-08T08:20:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.785+0000 7fd158d69700 1 -- 172.21.15.40:0/1081177657 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd150008cb0 con 0x561416656980 2023-11-08T08:20:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.785+0000 7fd15ca37140 1 -- 172.21.15.40:0/1081177657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561416656980 msgr2=0x56141664c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.785+0000 7fd15ca37140 1 --2- 172.21.15.40:0/1081177657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561416656980 0x56141664c0b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x56141662c1c0 tx=0x7fd15000d000 comp rx=0 tx=0).stop 2023-11-08T08:20:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.785+0000 7fd15ca37140 1 -- 172.21.15.40:0/1081177657 shutdown_connections 2023-11-08T08:20:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.785+0000 7fd15ca37140 1 --2- 172.21.15.40:0/1081177657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614166e1f40 0x56141664d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.785+0000 7fd15ca37140 1 --2- 172.21.15.40:0/1081177657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561416656980 0x56141664c0b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.785+0000 7fd15ca37140 1 --2- 172.21.15.40:0/1081177657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561416623700 0x56141664bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.785+0000 7fd15ca37140 1 -- 172.21.15.40:0/1081177657 >> 172.21.15.40:0/1081177657 conn(0x561416629c30 msgr2=0x5614166715c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.785+0000 7fd15ca37140 1 -- 172.21.15.40:0/1081177657 shutdown_connections 2023-11-08T08:20:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.785+0000 7fd15ca37140 1 -- 172.21.15.40:0/1081177657 wait complete. 2023-11-08T08:20:58.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.785+0000 7fd15ca37140 1 Processor -- start 2023-11-08T08:20:58.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.785+0000 7fd15ca37140 1 -- start start 2023-11-08T08:20:58.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd15ca37140 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561416623700 0x5614166523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd15ca37140 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561416656980 0x5614166df9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd15ca37140 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5614166e1f40 0x5614166dff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd15ca37140 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141664bec0 con 0x5614166e1f40 2023-11-08T08:20:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd15ca37140 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56141664ea90 con 0x561416623700 2023-11-08T08:20:58.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd15ca37140 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x56141664ecc0 con 0x561416656980 2023-11-08T08:20:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd15ad6d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5614166e1f40 0x5614166dff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd15ad6d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5614166e1f40 0x5614166dff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42476/0 (socket says 172.21.15.40:42476) 2023-11-08T08:20:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd15ad6d700 1 -- 172.21.15.40:0/1241930319 learned_addr learned my addr 172.21.15.40:0/1241930319 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd159d6b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561416656980 0x5614166df9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:58.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd15a56c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561416623700 0x5614166523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd15ad6d700 1 -- 172.21.15.40:0/1241930319 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561416656980 msgr2=0x5614166df9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd15ad6d700 1 --2- 172.21.15.40:0/1241930319 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561416656980 0x5614166df9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd15ad6d700 1 -- 172.21.15.40:0/1241930319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561416623700 msgr2=0x5614166523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd15ad6d700 1 --2- 172.21.15.40:0/1241930319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561416623700 0x5614166523d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd15ad6d700 1 -- 172.21.15.40:0/1241930319 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56141657d760 con 0x5614166e1f40 2023-11-08T08:20:58.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd15ad6d700 1 --2- 172.21.15.40:0/1241930319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5614166e1f40 0x5614166dff30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd15400f8a0 tx=0x7fd15400fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd14b7fe700 1 -- 172.21.15.40:0/1241930319 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd154013640 con 0x5614166e1f40 2023-11-08T08:20:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd14b7fe700 1 -- 172.21.15.40:0/1241930319 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1540137c0 con 0x5614166e1f40 2023-11-08T08:20:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd14b7fe700 1 -- 172.21.15.40:0/1241930319 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd154013b70 con 0x5614166e1f40 2023-11-08T08:20:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd15ca37140 1 -- 172.21.15.40:0/1241930319 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561416563390 con 0x5614166e1f40 2023-11-08T08:20:58.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.789+0000 7fd15ca37140 1 -- 172.21.15.40:0/1241930319 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614165568b0 con 0x5614166e1f40 2023-11-08T08:20:58.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.793+0000 7fd14b7fe700 1 -- 172.21.15.40:0/1241930319 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd154019030 con 0x5614166e1f40 2023-11-08T08:20:58.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.793+0000 7fd14b7fe700 1 --2- 172.21.15.40:0/1241930319 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd13c063890 0x7fd13c065d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:58.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.797+0000 7fd14b7fe700 1 -- 172.21.15.40:0/1241930319 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fd154080e20 con 0x5614166e1f40 2023-11-08T08:20:58.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.797+0000 7fd15ca37140 1 --2- 172.21.15.40:0/1241930319 >> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] conn(0x561416652910 0x5614166e5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:58.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.797+0000 7fd15ca37140 1 -- 172.21.15.40:0/1241930319 --> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] -- osd_op(unknown.0.0:1 6.10 6:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45) v8 -- 0x56141666fa60 con 0x561416652910 2023-11-08T08:20:58.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.797+0000 7fd15a56c700 1 --2- 172.21.15.40:0/1241930319 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd13c063890 0x7fd13c065d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:58.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.797+0000 7fd159d6b700 1 --2- 172.21.15.40:0/1241930319 >> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] conn(0x561416652910 0x5614166e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:58.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.797+0000 7fd159d6b700 1 --2- 172.21.15.40:0/1241930319 >> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] conn(0x561416652910 0x5614166e5660 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:58.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.797+0000 7fd15a56c700 1 --2- 172.21.15.40:0/1241930319 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd13c063890 0x7fd13c065d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0011e0 tx=0x7fd14c001cf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:58.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.797+0000 7fd159d6b700 1 -- 172.21.15.40:0/1241930319 <== osd.0 v2:172.21.15.40:6808/35482 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fd150026040 con 0x561416652910 2023-11-08T08:20:58.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.797+0000 7fd15ca37140 1 -- 172.21.15.40:0/1241930319 >> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] conn(0x561416652910 msgr2=0x5614166e5660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:58.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.797+0000 7fd15ca37140 1 --2- 172.21.15.40:0/1241930319 >> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] conn(0x561416652910 0x5614166e5660 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.797+0000 7fd15ca37140 1 -- 172.21.15.40:0/1241930319 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd13c063890 msgr2=0x7fd13c065d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:58.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.797+0000 7fd15ca37140 1 --2- 172.21.15.40:0/1241930319 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd13c063890 0x7fd13c065d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0011e0 tx=0x7fd14c001cf0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.797+0000 7fd15ca37140 1 -- 172.21.15.40:0/1241930319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5614166e1f40 msgr2=0x5614166dff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:58.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.797+0000 7fd15ca37140 1 --2- 172.21.15.40:0/1241930319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5614166e1f40 0x5614166dff30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd15400f8a0 tx=0x7fd15400fbb0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.797+0000 7fd15ad6d700 1 -- 172.21.15.40:0/1241930319 reap_dead start 2023-11-08T08:20:58.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.797+0000 7fd15ca37140 1 -- 172.21.15.40:0/1241930319 shutdown_connections 2023-11-08T08:20:58.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.797+0000 7fd15ca37140 1 -- 172.21.15.40:0/1241930319 >> 172.21.15.40:0/1241930319 conn(0x561416629c30 msgr2=0x5614166e0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:58.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.797+0000 7fd15ca37140 1 -- 172.21.15.40:0/1241930319 shutdown_connections 2023-11-08T08:20:58.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.797+0000 7fd15ca37140 1 -- 172.21.15.40:0/1241930319 wait complete. 2023-11-08T08:20:58.810 INFO:teuthology.orchestra.run.smithi040.stdout:mytag 2023-11-08T08:20:58.811 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2023-11-08T08:20:58.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.877+0000 7f61279e2140 1 Processor -- start 2023-11-08T08:20:58.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.877+0000 7f61279e2140 1 -- start start 2023-11-08T08:20:58.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f61279e2140 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590b4915c20 0x5590b49f2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:58.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f61279e2140 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590b4963970 0x5590b4944fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:58.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f61279e2140 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590b49f1960 0x5590b49ee5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:58.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f61279e2140 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590b48aa180 con 0x5590b4915c20 2023-11-08T08:20:58.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f61279e2140 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590b483d240 con 0x5590b4963970 2023-11-08T08:20:58.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f61279e2140 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590b49ee370 con 0x5590b49f1960 2023-11-08T08:20:58.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f6125517700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590b4915c20 0x5590b49f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:58.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f6125517700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590b4915c20 0x5590b49f2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42484/0 (socket says 172.21.15.40:42484) 2023-11-08T08:20:58.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f6125517700 1 -- 172.21.15.40:0/923980967 learned_addr learned my addr 172.21.15.40:0/923980967 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:58.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f6125d18700 1 --2- 172.21.15.40:0/923980967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590b49f1960 0x5590b49ee5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:58.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f6124d16700 1 --2- 172.21.15.40:0/923980967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590b4963970 0x5590b4944fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:58.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f6125517700 1 -- 172.21.15.40:0/923980967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590b49f1960 msgr2=0x5590b49ee5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:58.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f6125517700 1 --2- 172.21.15.40:0/923980967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590b49f1960 0x5590b49ee5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f6125517700 1 -- 172.21.15.40:0/923980967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590b4963970 msgr2=0x5590b4944fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:58.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f6125517700 1 --2- 172.21.15.40:0/923980967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590b4963970 0x5590b4944fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f6125517700 1 -- 172.21.15.40:0/923980967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611800a100 con 0x5590b4915c20 2023-11-08T08:20:58.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f6125d18700 1 --2- 172.21.15.40:0/923980967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590b49f1960 0x5590b49ee5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:20:58.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f6124d16700 1 --2- 172.21.15.40:0/923980967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590b4963970 0x5590b4944fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:20:58.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f6125517700 1 --2- 172.21.15.40:0/923980967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590b4915c20 0x5590b49f2100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6118005f10 tx=0x7f6118007030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa247022f3f3053 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:58.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f611f7fe700 1 -- 172.21.15.40:0/923980967 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611800f590 con 0x5590b4915c20 2023-11-08T08:20:58.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f611f7fe700 1 -- 172.21.15.40:0/923980967 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611800f6f0 con 0x5590b4915c20 2023-11-08T08:20:58.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f611f7fe700 1 -- 172.21.15.40:0/923980967 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611800fca0 con 0x5590b4915c20 2023-11-08T08:20:58.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f61279e2140 1 -- 172.21.15.40:0/923980967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590b4915c20 msgr2=0x5590b49f2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:58.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f61279e2140 1 --2- 172.21.15.40:0/923980967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590b4915c20 0x5590b49f2100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6118005f10 tx=0x7f6118007030 comp rx=0 tx=0).stop 2023-11-08T08:20:58.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f61279e2140 1 -- 172.21.15.40:0/923980967 shutdown_connections 2023-11-08T08:20:58.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f61279e2140 1 --2- 172.21.15.40:0/923980967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590b49f1960 0x5590b49ee5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f61279e2140 1 --2- 172.21.15.40:0/923980967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590b4963970 0x5590b4944fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f61279e2140 1 --2- 172.21.15.40:0/923980967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590b4915c20 0x5590b49f2100 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f61279e2140 1 -- 172.21.15.40:0/923980967 >> 172.21.15.40:0/923980967 conn(0x5590b491bc30 msgr2=0x5590b49f4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f61279e2140 1 -- 172.21.15.40:0/923980967 shutdown_connections 2023-11-08T08:20:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.881+0000 7f61279e2140 1 -- 172.21.15.40:0/923980967 wait complete. 2023-11-08T08:20:58.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.885+0000 7f61279e2140 1 Processor -- start 2023-11-08T08:20:58.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.885+0000 7f61279e2140 1 -- start start 2023-11-08T08:20:58.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.885+0000 7f61279e2140 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590b4915c20 0x5590b4a811f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:58.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.885+0000 7f61279e2140 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590b4963970 0x5590b4a81730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:58.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.885+0000 7f61279e2140 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590b49f1960 0x5590b4a7fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:58.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.885+0000 7f61279e2140 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590b491a4c0 con 0x5590b4915c20 2023-11-08T08:20:58.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.885+0000 7f61279e2140 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590b491a760 con 0x5590b49f1960 2023-11-08T08:20:58.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.885+0000 7f61279e2140 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590b491aa00 con 0x5590b4963970 2023-11-08T08:20:58.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.885+0000 7f6125517700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590b4915c20 0x5590b4a811f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:58.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.885+0000 7f6125517700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590b4915c20 0x5590b4a811f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42488/0 (socket says 172.21.15.40:42488) 2023-11-08T08:20:58.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.885+0000 7f6125517700 1 -- 172.21.15.40:0/1061043228 learned_addr learned my addr 172.21.15.40:0/1061043228 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:58.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.885+0000 7f6124d16700 1 --2- 172.21.15.40:0/1061043228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590b4963970 0x5590b4a81730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:58.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.885+0000 7f6125d18700 1 --2- 172.21.15.40:0/1061043228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590b49f1960 0x5590b4a7fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:58.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.889+0000 7f6125517700 1 -- 172.21.15.40:0/1061043228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590b4963970 msgr2=0x5590b4a81730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:58.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.889+0000 7f6125517700 1 --2- 172.21.15.40:0/1061043228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590b4963970 0x5590b4a81730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.889+0000 7f6125517700 1 -- 172.21.15.40:0/1061043228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590b49f1960 msgr2=0x5590b4a7fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:58.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.889+0000 7f6125517700 1 --2- 172.21.15.40:0/1061043228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590b49f1960 0x5590b4a7fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.889+0000 7f6125517700 1 -- 172.21.15.40:0/1061043228 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590b4855390 con 0x5590b4915c20 2023-11-08T08:20:58.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.889+0000 7f6124d16700 1 --2- 172.21.15.40:0/1061043228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590b4963970 0x5590b4a81730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:20:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.889+0000 7f6125d18700 1 --2- 172.21.15.40:0/1061043228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590b49f1960 0x5590b4a7fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.889+0000 7f6125517700 1 --2- 172.21.15.40:0/1061043228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590b4915c20 0x5590b4a811f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6118005f10 tx=0x7f6118003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.889+0000 7f611e7fc700 1 -- 172.21.15.40:0/1061043228 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6118010630 con 0x5590b4915c20 2023-11-08T08:20:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.889+0000 7f611e7fc700 1 -- 172.21.15.40:0/1061043228 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6118009030 con 0x5590b4915c20 2023-11-08T08:20:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.889+0000 7f611e7fc700 1 -- 172.21.15.40:0/1061043228 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6118010990 con 0x5590b4915c20 2023-11-08T08:20:58.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.889+0000 7f61279e2140 1 -- 172.21.15.40:0/1061043228 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5590b48488b0 con 0x5590b4915c20 2023-11-08T08:20:58.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.889+0000 7f61279e2140 1 -- 172.21.15.40:0/1061043228 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590b48555d0 con 0x5590b4915c20 2023-11-08T08:20:58.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.893+0000 7f611e7fc700 1 -- 172.21.15.40:0/1061043228 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f6118034070 con 0x5590b4915c20 2023-11-08T08:20:58.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.893+0000 7f611e7fc700 1 --2- 172.21.15.40:0/1061043228 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6108063840 0x7f6108065cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:58.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.893+0000 7f611e7fc700 1 -- 172.21.15.40:0/1061043228 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f6118082ff0 con 0x5590b4915c20 2023-11-08T08:20:58.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.893+0000 7f61279e2140 1 --2- 172.21.15.40:0/1061043228 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x5590b4a81c70 0x5590b4a7a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:58.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.893+0000 7f61279e2140 1 -- 172.21.15.40:0/1061043228 --> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] -- osd_op(unknown.0.0:1 6.7 6:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45) v8 -- 0x5590b4919500 con 0x5590b4a81c70 2023-11-08T08:20:58.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.893+0000 7f6124d16700 1 --2- 172.21.15.40:0/1061043228 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6108063840 0x7f6108065cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:58.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.893+0000 7f6125d18700 1 --2- 172.21.15.40:0/1061043228 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x5590b4a81c70 0x5590b4a7a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:58.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.897+0000 7f6124d16700 1 --2- 172.21.15.40:0/1061043228 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6108063840 0x7f6108065cf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6110002fd0 tx=0x7f6110001860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:58.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.897+0000 7f6125d18700 1 --2- 172.21.15.40:0/1061043228 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x5590b4a81c70 0x5590b4a7a380 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:58.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.897+0000 7f6125d18700 1 -- 172.21.15.40:0/1061043228 <== osd.5 v2:172.21.15.169:6809/35793 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f612000e540 con 0x5590b4a81c70 2023-11-08T08:20:58.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.897+0000 7f61279e2140 1 -- 172.21.15.40:0/1061043228 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x5590b4a81c70 msgr2=0x5590b4a7a380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:58.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.897+0000 7f61279e2140 1 --2- 172.21.15.40:0/1061043228 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x5590b4a81c70 0x5590b4a7a380 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.897+0000 7f61279e2140 1 -- 172.21.15.40:0/1061043228 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6108063840 msgr2=0x7f6108065cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:58.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.897+0000 7f61279e2140 1 --2- 172.21.15.40:0/1061043228 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6108063840 0x7f6108065cf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6110002fd0 tx=0x7f6110001860 comp rx=0 tx=0).stop 2023-11-08T08:20:58.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.897+0000 7f61279e2140 1 -- 172.21.15.40:0/1061043228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590b4915c20 msgr2=0x5590b4a811f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:58.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.897+0000 7f61279e2140 1 --2- 172.21.15.40:0/1061043228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590b4915c20 0x5590b4a811f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6118005f10 tx=0x7f6118003330 comp rx=0 tx=0).stop 2023-11-08T08:20:58.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.897+0000 7f6125d18700 1 -- 172.21.15.40:0/1061043228 reap_dead start 2023-11-08T08:20:58.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.897+0000 7f61279e2140 1 -- 172.21.15.40:0/1061043228 shutdown_connections 2023-11-08T08:20:58.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.897+0000 7f61279e2140 1 -- 172.21.15.40:0/1061043228 >> 172.21.15.40:0/1061043228 conn(0x5590b491bc30 msgr2=0x5590b49f4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:58.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.897+0000 7f61279e2140 1 -- 172.21.15.40:0/1061043228 shutdown_connections 2023-11-08T08:20:58.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.897+0000 7f61279e2140 1 -- 172.21.15.40:0/1061043228 wait complete. 2023-11-08T08:20:58.911 INFO:teuthology.orchestra.run.smithi040.stdout:mytag 2023-11-08T08:20:58.911 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2023-11-08T08:20:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.985+0000 7efe8329d140 1 Processor -- start 2023-11-08T08:20:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.985+0000 7efe8329d140 1 -- start start 2023-11-08T08:20:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.985+0000 7efe8329d140 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653d12b7700 0x5653d141a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.985+0000 7efe8329d140 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5653d12ea980 0x5653d12ddbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.985+0000 7efe8329d140 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653d1375830 0x5653d1392580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.985+0000 7efe8329d140 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d11df240 con 0x5653d1375830 2023-11-08T08:20:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.985+0000 7efe8329d140 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d14194f0 con 0x5653d12b7700 2023-11-08T08:20:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.985+0000 7efe8329d140 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653d1419790 con 0x5653d12ea980 2023-11-08T08:20:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.985+0000 7efe815d3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653d1375830 0x5653d1392580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.985+0000 7efe815d3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653d1375830 0x5653d1392580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42512/0 (socket says 172.21.15.40:42512) 2023-11-08T08:20:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.985+0000 7efe815d3700 1 -- 172.21.15.40:0/1794746019 learned_addr learned my addr 172.21.15.40:0/1794746019 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:58.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.985+0000 7efe785d1700 1 --2- 172.21.15.40:0/1794746019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5653d12ea980 0x5653d12ddbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.985+0000 7efe80dd2700 1 --2- 172.21.15.40:0/1794746019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653d12b7700 0x5653d141a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.989+0000 7efe815d3700 1 -- 172.21.15.40:0/1794746019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5653d12ea980 msgr2=0x5653d12ddbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.989+0000 7efe815d3700 1 --2- 172.21.15.40:0/1794746019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5653d12ea980 0x5653d12ddbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.989+0000 7efe815d3700 1 -- 172.21.15.40:0/1794746019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653d12b7700 msgr2=0x5653d141a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.989+0000 7efe815d3700 1 --2- 172.21.15.40:0/1794746019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653d12b7700 0x5653d141a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.989+0000 7efe815d3700 1 -- 172.21.15.40:0/1794746019 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7000a100 con 0x5653d1375830 2023-11-08T08:20:58.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.989+0000 7efe80dd2700 1 --2- 172.21.15.40:0/1794746019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653d12b7700 0x5653d141a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:58.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.989+0000 7efe815d3700 1 --2- 172.21.15.40:0/1794746019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653d1375830 0x5653d1392580 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7efe7c001c60 tx=0x7efe7c00b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c167a26df020535 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:58.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.989+0000 7efe7b7fe700 1 -- 172.21.15.40:0/1794746019 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c014070 con 0x5653d1375830 2023-11-08T08:20:58.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.989+0000 7efe7b7fe700 1 -- 172.21.15.40:0/1794746019 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7c008030 con 0x5653d1375830 2023-11-08T08:20:58.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.989+0000 7efe7b7fe700 1 -- 172.21.15.40:0/1794746019 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c00bce0 con 0x5653d1375830 2023-11-08T08:20:58.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.989+0000 7efe8329d140 1 -- 172.21.15.40:0/1794746019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653d1375830 msgr2=0x5653d1392580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:58.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.989+0000 7efe8329d140 1 --2- 172.21.15.40:0/1794746019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653d1375830 0x5653d1392580 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7efe7c001c60 tx=0x7efe7c00b4c0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.989+0000 7efe8329d140 1 -- 172.21.15.40:0/1794746019 shutdown_connections 2023-11-08T08:20:58.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.989+0000 7efe8329d140 1 --2- 172.21.15.40:0/1794746019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653d1375830 0x5653d1392580 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.989+0000 7efe8329d140 1 --2- 172.21.15.40:0/1794746019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5653d12ea980 0x5653d12ddbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.989+0000 7efe8329d140 1 --2- 172.21.15.40:0/1794746019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653d12b7700 0x5653d141a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:58.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.989+0000 7efe8329d140 1 -- 172.21.15.40:0/1794746019 >> 172.21.15.40:0/1794746019 conn(0x5653d12bdc30 msgr2=0x5653d1375c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:58.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.989+0000 7efe8329d140 1 -- 172.21.15.40:0/1794746019 shutdown_connections 2023-11-08T08:20:58.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.989+0000 7efe8329d140 1 -- 172.21.15.40:0/1794746019 wait complete. 2023-11-08T08:20:59.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.989+0000 7efe8329d140 1 Processor -- start 2023-11-08T08:20:59.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.993+0000 7efe8329d140 1 -- start start 2023-11-08T08:20:59.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.993+0000 7efe8329d140 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653d12b7700 0x5653d12e7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.993+0000 7efe8329d140 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5653d12ea980 0x5653d1374a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.993+0000 7efe8329d140 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653d1375830 0x5653d13055c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.993+0000 7efe8329d140 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d12e7cd0 con 0x5653d1375830 2023-11-08T08:20:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.993+0000 7efe8329d140 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d1420970 con 0x5653d12b7700 2023-11-08T08:20:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.993+0000 7efe8329d140 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653d141fa50 con 0x5653d12ea980 2023-11-08T08:20:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.993+0000 7efe815d3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653d1375830 0x5653d13055c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.993+0000 7efe815d3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653d1375830 0x5653d13055c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42524/0 (socket says 172.21.15.40:42524) 2023-11-08T08:20:59.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.993+0000 7efe815d3700 1 -- 172.21.15.40:0/931014869 learned_addr learned my addr 172.21.15.40:0/931014869 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:59.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.993+0000 7efe80dd2700 1 --2- 172.21.15.40:0/931014869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653d12b7700 0x5653d12e7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.993+0000 7efe785d1700 1 --2- 172.21.15.40:0/931014869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5653d12ea980 0x5653d1374a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.993+0000 7efe815d3700 1 -- 172.21.15.40:0/931014869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5653d12ea980 msgr2=0x5653d1374a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.993+0000 7efe815d3700 1 --2- 172.21.15.40:0/931014869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5653d12ea980 0x5653d1374a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.993+0000 7efe815d3700 1 -- 172.21.15.40:0/931014869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653d12b7700 msgr2=0x5653d12e7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.993+0000 7efe815d3700 1 --2- 172.21.15.40:0/931014869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5653d12b7700 0x5653d12e7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.993+0000 7efe815d3700 1 -- 172.21.15.40:0/931014869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c00a040 con 0x5653d1375830 2023-11-08T08:20:59.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.993+0000 7efe815d3700 1 --2- 172.21.15.40:0/931014869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653d1375830 0x5653d13055c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00b810 tx=0x7efe7c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:59.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.997+0000 7efe7a7fc700 1 -- 172.21.15.40:0/931014869 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c014040 con 0x5653d1375830 2023-11-08T08:20:59.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.997+0000 7efe7a7fc700 1 -- 172.21.15.40:0/931014869 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7c0018f0 con 0x5653d1375830 2023-11-08T08:20:59.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.997+0000 7efe7a7fc700 1 -- 172.21.15.40:0/931014869 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c01b670 con 0x5653d1375830 2023-11-08T08:20:59.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.997+0000 7efe8329d140 1 -- 172.21.15.40:0/931014869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5653d11ea8b0 con 0x5653d1375830 2023-11-08T08:20:59.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.997+0000 7efe8329d140 1 -- 172.21.15.40:0/931014869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe68005de0 con 0x5653d1375830 2023-11-08T08:20:59.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:58.997+0000 7efe7a7fc700 1 -- 172.21.15.40:0/931014869 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7efe7c02b070 con 0x5653d1375830 2023-11-08T08:20:59.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.001+0000 7efe7a7fc700 1 --2- 172.21.15.40:0/931014869 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efe60063840 0x7efe60065cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.001+0000 7efe7a7fc700 1 -- 172.21.15.40:0/931014869 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7efe7c081ed0 con 0x5653d1375830 2023-11-08T08:20:59.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.001+0000 7efe80dd2700 1 --2- 172.21.15.40:0/931014869 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efe60063840 0x7efe60065cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.001+0000 7efe8329d140 1 --2- 172.21.15.40:0/931014869 >> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] conn(0x5653d1425620 0x5653d1425a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.001+0000 7efe8329d140 1 -- 172.21.15.40:0/931014869 --> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] -- osd_op(unknown.0.0:1 7.36 7:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45) v8 -- 0x5653d1374fb0 con 0x5653d1425620 2023-11-08T08:20:59.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.001+0000 7efe785d1700 1 --2- 172.21.15.40:0/931014869 >> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] conn(0x5653d1425620 0x5653d1425a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.001+0000 7efe80dd2700 1 --2- 172.21.15.40:0/931014869 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efe60063840 0x7efe60065cf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efe70005fd0 tx=0x7efe70007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:59.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.001+0000 7efe785d1700 1 --2- 172.21.15.40:0/931014869 >> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] conn(0x5653d1425620 0x5653d1425a20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:59.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.001+0000 7efe785d1700 1 -- 172.21.15.40:0/931014869 <== osd.0 v2:172.21.15.40:6808/35482 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7efe68019040 con 0x5653d1425620 2023-11-08T08:20:59.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.005+0000 7efe8329d140 1 -- 172.21.15.40:0/931014869 >> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] conn(0x5653d1425620 msgr2=0x5653d1425a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.005+0000 7efe8329d140 1 --2- 172.21.15.40:0/931014869 >> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] conn(0x5653d1425620 0x5653d1425a20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.005+0000 7efe8329d140 1 -- 172.21.15.40:0/931014869 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efe60063840 msgr2=0x7efe60065cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.005+0000 7efe8329d140 1 --2- 172.21.15.40:0/931014869 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efe60063840 0x7efe60065cf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efe70005fd0 tx=0x7efe70007040 comp rx=0 tx=0).stop 2023-11-08T08:20:59.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.005+0000 7efe8329d140 1 -- 172.21.15.40:0/931014869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653d1375830 msgr2=0x5653d13055c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.005+0000 7efe8329d140 1 --2- 172.21.15.40:0/931014869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5653d1375830 0x5653d13055c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00b810 tx=0x7efe7c000bc0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.005+0000 7efe815d3700 1 -- 172.21.15.40:0/931014869 reap_dead start 2023-11-08T08:20:59.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.005+0000 7efe8329d140 1 -- 172.21.15.40:0/931014869 shutdown_connections 2023-11-08T08:20:59.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.005+0000 7efe8329d140 1 -- 172.21.15.40:0/931014869 >> 172.21.15.40:0/931014869 conn(0x5653d12bdc30 msgr2=0x5653d1375c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:59.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.005+0000 7efe8329d140 1 -- 172.21.15.40:0/931014869 shutdown_connections 2023-11-08T08:20:59.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.005+0000 7efe8329d140 1 -- 172.21.15.40:0/931014869 wait complete. 2023-11-08T08:20:59.015 INFO:teuthology.orchestra.run.smithi040.stdout:mytag 2023-11-08T08:20:59.016 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2023-11-08T08:20:59.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.089+0000 7f4324064140 1 Processor -- start 2023-11-08T08:20:59.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.089+0000 7f4324064140 1 -- start start 2023-11-08T08:20:59.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.089+0000 7f4324064140 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5607bf57d700 0x5607bf5a7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.089+0000 7f4324064140 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5607bf5b0980 0x5607bf6581c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.089+0000 7f4324064140 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5607bf63b830 0x5607bf658700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.089+0000 7f4324064140 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607bf5a9c20 con 0x5607bf57d700 2023-11-08T08:20:59.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.089+0000 7f4324064140 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607bf5a9ec0 con 0x5607bf5b0980 2023-11-08T08:20:59.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.089+0000 7f4324064140 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607bf5aa160 con 0x5607bf63b830 2023-11-08T08:20:59.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.089+0000 7f4321b99700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5607bf57d700 0x5607bf5a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4321b99700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5607bf57d700 0x5607bf5a7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42548/0 (socket says 172.21.15.40:42548) 2023-11-08T08:20:59.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4321b99700 1 -- 172.21.15.40:0/1737518333 learned_addr learned my addr 172.21.15.40:0/1737518333 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:59.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f432239a700 1 --2- 172.21.15.40:0/1737518333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5607bf63b830 0x5607bf658700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4319398700 1 --2- 172.21.15.40:0/1737518333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5607bf5b0980 0x5607bf6581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4321b99700 1 -- 172.21.15.40:0/1737518333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5607bf63b830 msgr2=0x5607bf658700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:59.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4321b99700 1 --2- 172.21.15.40:0/1737518333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5607bf63b830 0x5607bf658700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4321b99700 1 -- 172.21.15.40:0/1737518333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5607bf5b0980 msgr2=0x5607bf6581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:59.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4321b99700 1 --2- 172.21.15.40:0/1737518333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5607bf5b0980 0x5607bf6581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4321b99700 1 -- 172.21.15.40:0/1737518333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430c005de0 con 0x5607bf57d700 2023-11-08T08:20:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4321b99700 1 --2- 172.21.15.40:0/1737518333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5607bf57d700 0x5607bf5a7780 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f43140096b0 tx=0x7f43140099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1485e53b73020c67 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:59.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4320b97700 1 -- 172.21.15.40:0/1737518333 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314010800 con 0x5607bf57d700 2023-11-08T08:20:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4320b97700 1 -- 172.21.15.40:0/1737518333 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4314010960 con 0x5607bf57d700 2023-11-08T08:20:59.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4320b97700 1 -- 172.21.15.40:0/1737518333 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314010cd0 con 0x5607bf57d700 2023-11-08T08:20:59.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4324064140 1 -- 172.21.15.40:0/1737518333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5607bf57d700 msgr2=0x5607bf5a7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4324064140 1 --2- 172.21.15.40:0/1737518333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5607bf57d700 0x5607bf5a7780 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f43140096b0 tx=0x7f43140099c0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4324064140 1 -- 172.21.15.40:0/1737518333 shutdown_connections 2023-11-08T08:20:59.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4324064140 1 --2- 172.21.15.40:0/1737518333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5607bf63b830 0x5607bf658700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4324064140 1 --2- 172.21.15.40:0/1737518333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5607bf5b0980 0x5607bf6581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4324064140 1 --2- 172.21.15.40:0/1737518333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5607bf57d700 0x5607bf5a7780 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4324064140 1 -- 172.21.15.40:0/1737518333 >> 172.21.15.40:0/1737518333 conn(0x5607bf583c30 msgr2=0x5607bf5a6520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4324064140 1 -- 172.21.15.40:0/1737518333 shutdown_connections 2023-11-08T08:20:59.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.093+0000 7f4324064140 1 -- 172.21.15.40:0/1737518333 wait complete. 2023-11-08T08:20:59.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f4324064140 1 Processor -- start 2023-11-08T08:20:59.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f4324064140 1 -- start start 2023-11-08T08:20:59.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f4324064140 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5607bf5b0980 0x5607bf6df4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f4324064140 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5607bf63b830 0x5607bf5a6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f4324064140 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5607bf5a6c50 0x5607bf6e4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f4324064140 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607bf6de830 con 0x5607bf5a6c50 2023-11-08T08:20:59.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f4324064140 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607bf6dead0 con 0x5607bf63b830 2023-11-08T08:20:59.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f4324064140 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607bf6ded70 con 0x5607bf5b0980 2023-11-08T08:20:59.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f432239a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5607bf5a6c50 0x5607bf6e4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f432239a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5607bf5a6c50 0x5607bf6e4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42550/0 (socket says 172.21.15.40:42550) 2023-11-08T08:20:59.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f432239a700 1 -- 172.21.15.40:0/1056910293 learned_addr learned my addr 172.21.15.40:0/1056910293 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:59.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f4319398700 1 --2- 172.21.15.40:0/1056910293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5607bf63b830 0x5607bf5a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f4321b99700 1 --2- 172.21.15.40:0/1056910293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5607bf5b0980 0x5607bf6df4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f432239a700 1 -- 172.21.15.40:0/1056910293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5607bf5b0980 msgr2=0x5607bf6df4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f432239a700 1 --2- 172.21.15.40:0/1056910293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5607bf5b0980 0x5607bf6df4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f432239a700 1 -- 172.21.15.40:0/1056910293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5607bf63b830 msgr2=0x5607bf5a6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f432239a700 1 --2- 172.21.15.40:0/1056910293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5607bf63b830 0x5607bf5a6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f432239a700 1 -- 172.21.15.40:0/1056910293 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43140092e0 con 0x5607bf5a6c50 2023-11-08T08:20:59.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f432239a700 1 --2- 172.21.15.40:0/1056910293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5607bf5a6c50 0x5607bf6e4650 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f431c00a440 tx=0x7f431c00a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:59.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f431b7fe700 1 -- 172.21.15.40:0/1056910293 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c0087f0 con 0x5607bf5a6c50 2023-11-08T08:20:59.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f431b7fe700 1 -- 172.21.15.40:0/1056910293 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c008950 con 0x5607bf5a6c50 2023-11-08T08:20:59.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f4324064140 1 -- 172.21.15.40:0/1056910293 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5607bf4d7760 con 0x5607bf5a6c50 2023-11-08T08:20:59.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f431b7fe700 1 -- 172.21.15.40:0/1056910293 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c008d00 con 0x5607bf5a6c50 2023-11-08T08:20:59.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.097+0000 7f4324064140 1 -- 172.21.15.40:0/1056910293 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607bf4bd390 con 0x5607bf5a6c50 2023-11-08T08:20:59.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.101+0000 7f431b7fe700 1 -- 172.21.15.40:0/1056910293 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f431c01b020 con 0x5607bf5a6c50 2023-11-08T08:20:59.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.105+0000 7f431b7fe700 1 --2- 172.21.15.40:0/1056910293 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4304063810 0x7f4304065cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.105+0000 7f431b7fe700 1 -- 172.21.15.40:0/1056910293 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f431c081090 con 0x5607bf5a6c50 2023-11-08T08:20:59.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.105+0000 7f4324064140 1 --2- 172.21.15.40:0/1056910293 >> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] conn(0x5607bf6e1420 0x5607bf6e17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.105+0000 7f4324064140 1 -- 172.21.15.40:0/1056910293 --> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] -- osd_op(unknown.0.0:1 7.19 7:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45) v8 -- 0x5607bf6e1d30 con 0x5607bf6e1420 2023-11-08T08:20:59.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.105+0000 7f4321b99700 1 --2- 172.21.15.40:0/1056910293 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4304063810 0x7f4304065cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.105+0000 7f4319398700 1 --2- 172.21.15.40:0/1056910293 >> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] conn(0x5607bf6e1420 0x5607bf6e17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.105+0000 7f4321b99700 1 --2- 172.21.15.40:0/1056910293 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4304063810 0x7f4304065cc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x5607bf584c10 tx=0x7f4314011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:59.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.105+0000 7f4319398700 1 --2- 172.21.15.40:0/1056910293 >> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] conn(0x5607bf6e1420 0x5607bf6e17f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:59.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.109+0000 7f4319398700 1 -- 172.21.15.40:0/1056910293 <== osd.6 v2:172.21.15.169:6800/35791 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f430c008b80 con 0x5607bf6e1420 2023-11-08T08:20:59.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.109+0000 7f4324064140 1 -- 172.21.15.40:0/1056910293 >> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] conn(0x5607bf6e1420 msgr2=0x5607bf6e17f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.109+0000 7f4324064140 1 --2- 172.21.15.40:0/1056910293 >> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] conn(0x5607bf6e1420 0x5607bf6e17f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.109+0000 7f4324064140 1 -- 172.21.15.40:0/1056910293 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4304063810 msgr2=0x7f4304065cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.109+0000 7f4324064140 1 --2- 172.21.15.40:0/1056910293 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4304063810 0x7f4304065cc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x5607bf584c10 tx=0x7f4314011000 comp rx=0 tx=0).stop 2023-11-08T08:20:59.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.109+0000 7f4324064140 1 -- 172.21.15.40:0/1056910293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5607bf5a6c50 msgr2=0x5607bf6e4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.109+0000 7f4324064140 1 --2- 172.21.15.40:0/1056910293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5607bf5a6c50 0x5607bf6e4650 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f431c00a440 tx=0x7f431c00a800 comp rx=0 tx=0).stop 2023-11-08T08:20:59.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.109+0000 7f432239a700 1 -- 172.21.15.40:0/1056910293 reap_dead start 2023-11-08T08:20:59.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.109+0000 7f4324064140 1 -- 172.21.15.40:0/1056910293 shutdown_connections 2023-11-08T08:20:59.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.109+0000 7f4324064140 1 -- 172.21.15.40:0/1056910293 >> 172.21.15.40:0/1056910293 conn(0x5607bf583c30 msgr2=0x5607bf657ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:59.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.109+0000 7f4324064140 1 -- 172.21.15.40:0/1056910293 shutdown_connections 2023-11-08T08:20:59.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.109+0000 7f4324064140 1 -- 172.21.15.40:0/1056910293 wait complete. 2023-11-08T08:20:59.126 INFO:teuthology.orchestra.run.smithi040.stdout:mytag 2023-11-08T08:20:59.127 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2023-11-08T08:20:59.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.193+0000 7f0b7c07c140 1 Processor -- start 2023-11-08T08:20:59.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.193+0000 7f0b7c07c140 1 -- start start 2023-11-08T08:20:59.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.193+0000 7f0b7c07c140 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2b2fbd700 0x55f2b2fe7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.193+0000 7f0b7c07c140 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2b2fbdad0 0x55f2b300b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.193+0000 7f0b7c07c140 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2b307bf40 0x55f2b3081e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.193+0000 7f0b7c07c140 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b2f52180 con 0x55f2b307bf40 2023-11-08T08:20:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.193+0000 7f0b7c07c140 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b2ee5240 con 0x55f2b2fbdad0 2023-11-08T08:20:59.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.193+0000 7f0b7c07c140 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2b2fe76e0 con 0x55f2b2fbd700 2023-11-08T08:20:59.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.193+0000 7f0b7a3b2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2b307bf40 0x55f2b3081e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.193+0000 7f0b7a3b2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2b307bf40 0x55f2b3081e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42560/0 (socket says 172.21.15.40:42560) 2023-11-08T08:20:59.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.193+0000 7f0b7a3b2700 1 -- 172.21.15.40:0/518756838 learned_addr learned my addr 172.21.15.40:0/518756838 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:59.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.193+0000 7f0b793b0700 1 --2- 172.21.15.40:0/518756838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2b2fbdad0 0x55f2b300b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.193+0000 7f0b79bb1700 1 --2- 172.21.15.40:0/518756838 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2b2fbd700 0x55f2b2fe7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.197+0000 7f0b7a3b2700 1 -- 172.21.15.40:0/518756838 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2b2fbd700 msgr2=0x55f2b2fe7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:59.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.197+0000 7f0b7a3b2700 1 --2- 172.21.15.40:0/518756838 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2b2fbd700 0x55f2b2fe7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.197+0000 7f0b7a3b2700 1 -- 172.21.15.40:0/518756838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2b2fbdad0 msgr2=0x55f2b300b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:20:59.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.197+0000 7f0b7a3b2700 1 --2- 172.21.15.40:0/518756838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2b2fbdad0 0x55f2b300b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.197+0000 7f0b7a3b2700 1 -- 172.21.15.40:0/518756838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b74005d20 con 0x55f2b307bf40 2023-11-08T08:20:59.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.197+0000 7f0b79bb1700 1 --2- 172.21.15.40:0/518756838 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2b2fbd700 0x55f2b2fe7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.197+0000 7f0b7a3b2700 1 --2- 172.21.15.40:0/518756838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2b307bf40 0x55f2b3081e60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400ab50 tx=0x7f0b7400d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31bb5b26bfce0496 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.197+0000 7f0b6bfff700 1 -- 172.21.15.40:0/518756838 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74008ac0 con 0x55f2b307bf40 2023-11-08T08:20:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.197+0000 7f0b6bfff700 1 -- 172.21.15.40:0/518756838 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b74008c40 con 0x55f2b307bf40 2023-11-08T08:20:59.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.197+0000 7f0b6bfff700 1 -- 172.21.15.40:0/518756838 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74012410 con 0x55f2b307bf40 2023-11-08T08:20:59.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.197+0000 7f0b7c07c140 1 -- 172.21.15.40:0/518756838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2b307bf40 msgr2=0x55f2b3081e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.197+0000 7f0b7c07c140 1 --2- 172.21.15.40:0/518756838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2b307bf40 0x55f2b3081e60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400ab50 tx=0x7f0b7400d000 comp rx=0 tx=0).stop 2023-11-08T08:20:59.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.197+0000 7f0b7c07c140 1 -- 172.21.15.40:0/518756838 shutdown_connections 2023-11-08T08:20:59.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.197+0000 7f0b7c07c140 1 --2- 172.21.15.40:0/518756838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2b307bf40 0x55f2b3081e60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.197+0000 7f0b7c07c140 1 --2- 172.21.15.40:0/518756838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2b2fbdad0 0x55f2b300b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.197+0000 7f0b7c07c140 1 --2- 172.21.15.40:0/518756838 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2b2fbd700 0x55f2b2fe7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.197+0000 7f0b7c07c140 1 -- 172.21.15.40:0/518756838 >> 172.21.15.40:0/518756838 conn(0x55f2b2fc3c30 msgr2=0x55f2b2fe5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:59.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.197+0000 7f0b7c07c140 1 -- 172.21.15.40:0/518756838 shutdown_connections 2023-11-08T08:20:59.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.197+0000 7f0b7c07c140 1 -- 172.21.15.40:0/518756838 wait complete. 2023-11-08T08:20:59.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.201+0000 7f0b7c07c140 1 Processor -- start 2023-11-08T08:20:59.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.201+0000 7f0b7c07c140 1 -- start start 2023-11-08T08:20:59.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.201+0000 7f0b7c07c140 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2b2fbd700 0x55f2b2fe7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.201+0000 7f0b7c07c140 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2b2fbdad0 0x55f2b2fec3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.201+0000 7f0b7c07c140 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2b307bf40 0x55f2b312d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.201+0000 7f0b7c07c140 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b307ea00 con 0x55f2b307bf40 2023-11-08T08:20:59.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.201+0000 7f0b7c07c140 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b307eca0 con 0x55f2b2fbdad0 2023-11-08T08:20:59.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.201+0000 7f0b7c07c140 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2b307ef40 con 0x55f2b2fbd700 2023-11-08T08:20:59.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.201+0000 7f0b7a3b2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2b307bf40 0x55f2b312d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.201+0000 7f0b7a3b2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2b307bf40 0x55f2b312d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42572/0 (socket says 172.21.15.40:42572) 2023-11-08T08:20:59.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.201+0000 7f0b7a3b2700 1 -- 172.21.15.40:0/2494975354 learned_addr learned my addr 172.21.15.40:0/2494975354 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:59.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.201+0000 7f0b793b0700 1 --2- 172.21.15.40:0/2494975354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2b2fbdad0 0x55f2b2fec3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.201+0000 7f0b79bb1700 1 --2- 172.21.15.40:0/2494975354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2b2fbd700 0x55f2b2fe7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.201+0000 7f0b7a3b2700 1 -- 172.21.15.40:0/2494975354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2b2fbd700 msgr2=0x55f2b2fe7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.201+0000 7f0b7a3b2700 1 --2- 172.21.15.40:0/2494975354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2b2fbd700 0x55f2b2fe7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.201+0000 7f0b7a3b2700 1 -- 172.21.15.40:0/2494975354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2b2fbdad0 msgr2=0x55f2b2fec3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.201+0000 7f0b7a3b2700 1 --2- 172.21.15.40:0/2494975354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2b2fbdad0 0x55f2b2fec3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.201+0000 7f0b7a3b2700 1 -- 172.21.15.40:0/2494975354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2b2efd390 con 0x55f2b307bf40 2023-11-08T08:20:59.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.205+0000 7f0b7a3b2700 1 --2- 172.21.15.40:0/2494975354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2b307bf40 0x55f2b312d460 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400c010 tx=0x7f0b74002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:59.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.205+0000 7f0b6affd700 1 -- 172.21.15.40:0/2494975354 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b740126e0 con 0x55f2b307bf40 2023-11-08T08:20:59.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.205+0000 7f0b6affd700 1 -- 172.21.15.40:0/2494975354 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b74012840 con 0x55f2b307bf40 2023-11-08T08:20:59.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.205+0000 7f0b7c07c140 1 -- 172.21.15.40:0/2494975354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f2b2ef08b0 con 0x55f2b307bf40 2023-11-08T08:20:59.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.205+0000 7f0b6affd700 1 -- 172.21.15.40:0/2494975354 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74012bf0 con 0x55f2b307bf40 2023-11-08T08:20:59.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.205+0000 7f0b7c07c140 1 -- 172.21.15.40:0/2494975354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2b2efd5d0 con 0x55f2b307bf40 2023-11-08T08:20:59.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.209+0000 7f0b6affd700 1 -- 172.21.15.40:0/2494975354 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f0b74022070 con 0x55f2b307bf40 2023-11-08T08:20:59.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.209+0000 7f0b6affd700 1 --2- 172.21.15.40:0/2494975354 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0b5c063840 0x7f0b5c065cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.209+0000 7f0b6affd700 1 -- 172.21.15.40:0/2494975354 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f0b7400ae40 con 0x55f2b307bf40 2023-11-08T08:20:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.209+0000 7f0b79bb1700 1 --2- 172.21.15.40:0/2494975354 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0b5c063840 0x7f0b5c065cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.209+0000 7f0b7c07c140 1 --2- 172.21.15.40:0/2494975354 >> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] conn(0x55f2b2fec910 0x55f2b312b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.209+0000 7f0b7c07c140 1 -- 172.21.15.40:0/2494975354 --> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] -- osd_op(unknown.0.0:1 7.12 7:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45) v8 -- 0x55f2b312b580 con 0x55f2b2fec910 2023-11-08T08:20:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.209+0000 7f0b793b0700 1 --2- 172.21.15.40:0/2494975354 >> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] conn(0x55f2b2fec910 0x55f2b312b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.209+0000 7f0b79bb1700 1 --2- 172.21.15.40:0/2494975354 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0b5c063840 0x7f0b5c065cf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c003730 tx=0x7f0b6c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.209+0000 7f0b793b0700 1 --2- 172.21.15.40:0/2494975354 >> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] conn(0x55f2b2fec910 0x55f2b312b040 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.213+0000 7f0b793b0700 1 -- 172.21.15.40:0/2494975354 <== osd.6 v2:172.21.15.169:6800/35791 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f0b7000f760 con 0x55f2b2fec910 2023-11-08T08:20:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2023-11-08T08:20:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.213+0000 7f0b7c07c140 1 -- 172.21.15.40:0/2494975354 >> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] conn(0x55f2b2fec910 msgr2=0x55f2b312b040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.213+0000 7f0b7c07c140 1 --2- 172.21.15.40:0/2494975354 >> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] conn(0x55f2b2fec910 0x55f2b312b040 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.213+0000 7f0b7c07c140 1 -- 172.21.15.40:0/2494975354 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0b5c063840 msgr2=0x7f0b5c065cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.213+0000 7f0b7c07c140 1 --2- 172.21.15.40:0/2494975354 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0b5c063840 0x7f0b5c065cf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c003730 tx=0x7f0b6c008040 comp rx=0 tx=0).stop 2023-11-08T08:20:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.213+0000 7f0b7c07c140 1 -- 172.21.15.40:0/2494975354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2b307bf40 msgr2=0x55f2b312d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.213+0000 7f0b7c07c140 1 --2- 172.21.15.40:0/2494975354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f2b307bf40 0x55f2b312d460 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400c010 tx=0x7f0b74002fb0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.213+0000 7f0b7a3b2700 1 -- 172.21.15.40:0/2494975354 reap_dead start 2023-11-08T08:20:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.213+0000 7f0b7c07c140 1 -- 172.21.15.40:0/2494975354 shutdown_connections 2023-11-08T08:20:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.213+0000 7f0b7c07c140 1 -- 172.21.15.40:0/2494975354 >> 172.21.15.40:0/2494975354 conn(0x55f2b2fc3c30 msgr2=0x55f2b2fe5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.213+0000 7f0b7c07c140 1 -- 172.21.15.40:0/2494975354 shutdown_connections 2023-11-08T08:20:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.213+0000 7f0b7c07c140 1 -- 172.21.15.40:0/2494975354 wait complete. 2023-11-08T08:20:59.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T08:20:59.225 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T08:20:59.226 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T08:20:59.270 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T08:20:59.314 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T08:20:59.315 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:20:59.315 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:20:59.336 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16355412k,nr_inodes=4088853,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6645 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T08:20:59.336 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:20:59.337 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:20:59.374 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352316k,nr_inodes=4088079,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1804 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T08:20:59.375 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:20:59.376 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:20:59.397 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16355412k,nr_inodes=4088853,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6645 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T08:20:59.398 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:20:59.398 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:20:59.441 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352316k,nr_inodes=4088079,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1804 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T08:20:59.442 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:20:59.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.613+0000 7fc8d6e0b700 1 Processor -- start 2023-11-08T08:20:59.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.613+0000 7fc8d6e0b700 1 -- start start 2023-11-08T08:20:59.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.617+0000 7fc8d6e0b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8d0070080 0x7fc8d00704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.617+0000 7fc8d6e0b700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d00ad6c0 con 0x7fc8d0164250 2023-11-08T08:20:59.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.617+0000 7fc8d6e0b700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d00adc60 con 0x7fc8d0160710 2023-11-08T08:20:59.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.617+0000 7fc8d6e0b700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d006b1c0 con 0x7fc8d0070080 2023-11-08T08:20:59.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.617+0000 7fc8d660a700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc8d0164250 0x7fc8d01646d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60156/0 (socket says 172.21.15.40:60156) 2023-11-08T08:20:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.617+0000 7fc8d660a700 1 -- 172.21.15.40:0/3275947707 learned_addr learned my addr 172.21.15.40:0/3275947707 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:20:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.617+0000 7fc8d5608700 1 --2- 172.21.15.40:0/3275947707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8d0070080 0x7fc8d00704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.617+0000 7fc8d4e07700 1 -- 172.21.15.40:0/3275947707 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512322318 0 0) 0x7fc8d00ad6c0 con 0x7fc8d0164250 2023-11-08T08:20:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.617+0000 7fc8d4e07700 1 -- 172.21.15.40:0/3275947707 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8bc0035d0 con 0x7fc8d0164250 2023-11-08T08:20:59.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.617+0000 7fc8d4e07700 1 -- 172.21.15.40:0/3275947707 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585401069 0 0) 0x7fc8d00adc60 con 0x7fc8d0160710 2023-11-08T08:20:59.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.617+0000 7fc8d4e07700 1 -- 172.21.15.40:0/3275947707 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8d00ad6c0 con 0x7fc8d0160710 2023-11-08T08:20:59.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.617+0000 7fc8d5608700 1 -- 172.21.15.40:0/3275947707 >> v1:172.21.15.169:6789/0 conn(0x7fc8d0160710 legacy=0x7fc8d00cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.617+0000 7fc8d5608700 1 -- 172.21.15.40:0/3275947707 >> v1:172.21.15.40:6789/0 conn(0x7fc8d0164250 legacy=0x7fc8d01646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.617+0000 7fc8d5608700 1 -- 172.21.15.40:0/3275947707 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d00709f0 con 0x7fc8d0070080 2023-11-08T08:20:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.617+0000 7fc8d5608700 1 --2- 172.21.15.40:0/3275947707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8d0070080 0x7fc8d00704b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009880 tx=0x7fc8c0009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ab3862b358e7f1f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.617+0000 7fc8d4e07700 1 -- 172.21.15.40:0/3275947707 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0011830 con 0x7fc8d0070080 2023-11-08T08:20:59.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.617+0000 7fc8d4e07700 1 -- 172.21.15.40:0/3275947707 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c0011990 con 0x7fc8d0070080 2023-11-08T08:20:59.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.617+0000 7fc8d4e07700 1 -- 172.21.15.40:0/3275947707 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0011d20 con 0x7fc8d0070080 2023-11-08T08:20:59.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.621+0000 7fc8d6e0b700 1 -- 172.21.15.40:0/3275947707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8d0070080 msgr2=0x7fc8d00704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.621+0000 7fc8d6e0b700 1 --2- 172.21.15.40:0/3275947707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8d0070080 0x7fc8d00704b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009880 tx=0x7fc8c0009b90 comp rx=0 tx=0).stop 2023-11-08T08:20:59.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.621+0000 7fc8d6e0b700 1 -- 172.21.15.40:0/3275947707 shutdown_connections 2023-11-08T08:20:59.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.621+0000 7fc8d6e0b700 1 --2- 172.21.15.40:0/3275947707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8d0070080 0x7fc8d00704b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.621+0000 7fc8d6e0b700 1 -- 172.21.15.40:0/3275947707 >> 172.21.15.40:0/3275947707 conn(0x7fc8d00b0f80 msgr2=0x7fc8d00ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:59.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.621+0000 7fc8d6e0b700 1 -- 172.21.15.40:0/3275947707 shutdown_connections 2023-11-08T08:20:59.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.621+0000 7fc8d6e0b700 1 -- 172.21.15.40:0/3275947707 wait complete. 2023-11-08T08:20:59.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.621+0000 7fc8d6e0b700 1 Processor -- start 2023-11-08T08:20:59.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.621+0000 7fc8d6e0b700 1 -- start start 2023-11-08T08:20:59.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8d6e0b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8d0070080 0x7fc8d00c0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8d6e0b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d0160710 0x7fc8d015cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8d6e0b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8d0164250 0x7fc8d015b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8d6e0b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d015de90 con 0x7fc8d0160710 2023-11-08T08:20:59.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8d6e0b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d015d020 con 0x7fc8d0164250 2023-11-08T08:20:59.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8d6e0b700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d015d280 con 0x7fc8d0070080 2023-11-08T08:20:59.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8d5608700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d0160710 0x7fc8d015cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8d5608700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d0160710 0x7fc8d015cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42586/0 (socket says 172.21.15.40:42586) 2023-11-08T08:20:59.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8d5608700 1 -- 172.21.15.40:0/939412436 learned_addr learned my addr 172.21.15.40:0/939412436 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:20:59.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8d660a700 1 --2- 172.21.15.40:0/939412436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8d0164250 0x7fc8d015b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8d5e09700 1 --2- 172.21.15.40:0/939412436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8d0070080 0x7fc8d00c0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8d5608700 1 -- 172.21.15.40:0/939412436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8d0070080 msgr2=0x7fc8d00c0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8d5608700 1 --2- 172.21.15.40:0/939412436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8d0070080 0x7fc8d00c0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8d5608700 1 -- 172.21.15.40:0/939412436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8d0164250 msgr2=0x7fc8d015b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8d5608700 1 --2- 172.21.15.40:0/939412436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8d0164250 0x7fc8d015b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8d5608700 1 -- 172.21.15.40:0/939412436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c00094b0 con 0x7fc8d0160710 2023-11-08T08:20:59.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8d660a700 1 --2- 172.21.15.40:0/939412436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8d0164250 0x7fc8d015b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:20:59.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8d5608700 1 --2- 172.21.15.40:0/939412436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d0160710 0x7fc8d015cae0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0013040 tx=0x7fc8c0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:59.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8c67fc700 1 -- 172.21.15.40:0/939412436 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c001a8c0 con 0x7fc8d0160710 2023-11-08T08:20:59.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8c67fc700 1 -- 172.21.15.40:0/939412436 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c001e070 con 0x7fc8d0160710 2023-11-08T08:20:59.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8d6e0b700 1 -- 172.21.15.40:0/939412436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d006b3d0 con 0x7fc8d0160710 2023-11-08T08:20:59.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.625+0000 7fc8c67fc700 1 -- 172.21.15.40:0/939412436 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c001ac60 con 0x7fc8d0160710 2023-11-08T08:20:59.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.629+0000 7fc8d6e0b700 1 -- 172.21.15.40:0/939412436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d00cca30 con 0x7fc8d0160710 2023-11-08T08:20:59.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.629+0000 7fc8d6e0b700 1 -- 172.21.15.40:0/939412436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d006b6b0 con 0x7fc8d0160710 2023-11-08T08:20:59.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.629+0000 7fc8c67fc700 1 -- 172.21.15.40:0/939412436 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc8c000f050 con 0x7fc8d0160710 2023-11-08T08:20:59.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.629+0000 7fc8c67fc700 1 --2- 172.21.15.40:0/939412436 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc8b4064ee0 0x7fc8b4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:20:59.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.633+0000 7fc8c67fc700 1 -- 172.21.15.40:0/939412436 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fc8c0082e30 con 0x7fc8d0160710 2023-11-08T08:20:59.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.633+0000 7fc8d5e09700 1 --2- 172.21.15.40:0/939412436 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc8b4064ee0 0x7fc8b4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:20:59.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.633+0000 7fc8d5e09700 1 --2- 172.21.15.40:0/939412436 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc8b4064ee0 0x7fc8b4067390 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0011e0 tx=0x7fc8cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:20:59.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.637+0000 7fc8c67fc700 1 -- 172.21.15.40:0/939412436 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc8c0050e50 con 0x7fc8d0160710 2023-11-08T08:20:59.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.841+0000 7fc8d6e0b700 1 -- 172.21.15.40:0/939412436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8d0073ce0 con 0x7fc8d0160710 2023-11-08T08:20:59.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.841+0000 7fc8c67fc700 1 -- 172.21.15.40:0/939412436 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4651 (secure 0 0 0) 0x7fc8c0020020 con 0x7fc8d0160710 2023-11-08T08:20:59.848 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:20:59.849 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4958,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2244078655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2244078655},{"type":"v1","addr":"172.21.15.40:6834","nonce":2244078655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/749537066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":749537066},{"type":"v1","addr":"172.21.15.169:6836","nonce":749537066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5328,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2307838686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2307838686},{"type":"v1","addr":"172.21.15.169:6837","nonce":2307838686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:20:34.294628+0000","modified":"2023-11-08T08:20:37.318141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/1791517264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1791517264},{"type":"v1","addr":"172.21.15.40:6835","nonce":1791517264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T08:20:59.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.845+0000 7fc8d6e0b700 1 -- 172.21.15.40:0/939412436 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc8b4064ee0 msgr2=0x7fc8b4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.845+0000 7fc8d6e0b700 1 --2- 172.21.15.40:0/939412436 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc8b4064ee0 0x7fc8b4067390 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0011e0 tx=0x7fc8cc009040 comp rx=0 tx=0).stop 2023-11-08T08:20:59.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.845+0000 7fc8d6e0b700 1 -- 172.21.15.40:0/939412436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d0160710 msgr2=0x7fc8d015cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:20:59.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.845+0000 7fc8d6e0b700 1 --2- 172.21.15.40:0/939412436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d0160710 0x7fc8d015cae0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0013040 tx=0x7fc8c0004770 comp rx=0 tx=0).stop 2023-11-08T08:20:59.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.845+0000 7fc8d6e0b700 1 -- 172.21.15.40:0/939412436 shutdown_connections 2023-11-08T08:20:59.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.845+0000 7fc8d6e0b700 1 --2- 172.21.15.40:0/939412436 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc8b4064ee0 0x7fc8b4067390 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.845+0000 7fc8d6e0b700 1 --2- 172.21.15.40:0/939412436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8d0164250 0x7fc8d015b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.845+0000 7fc8d6e0b700 1 --2- 172.21.15.40:0/939412436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d0160710 0x7fc8d015cae0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.845+0000 7fc8d6e0b700 1 --2- 172.21.15.40:0/939412436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8d0070080 0x7fc8d00c0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:20:59.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.845+0000 7fc8d6e0b700 1 -- 172.21.15.40:0/939412436 >> 172.21.15.40:0/939412436 conn(0x7fc8d00b0f80 msgr2=0x7fc8d0160fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:20:59.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.845+0000 7fc8d6e0b700 1 -- 172.21.15.40:0/939412436 shutdown_connections 2023-11-08T08:20:59.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:20:59.845+0000 7fc8d6e0b700 1 -- 172.21.15.40:0/939412436 wait complete. 2023-11-08T08:20:59.854 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 17 2023-11-08T08:20:59.876 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:21:00.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.057+0000 7f4ecf59e700 1 Processor -- start 2023-11-08T08:21:00.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.057+0000 7f4ecf59e700 1 -- start start 2023-11-08T08:21:00.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.061+0000 7f4ecf59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ed0160710 0x7f4ed00cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:00.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.061+0000 7f4ecf59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed00cc860 0x7f4ed0070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:00.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.061+0000 7f4ecf59e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed00ad6c0 con 0x7f4ed00705c0 2023-11-08T08:21:00.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.061+0000 7f4ecf59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ed0070a60 con 0x7f4ed0160710 2023-11-08T08:21:00.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.061+0000 7f4ecf59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed006b1c0 con 0x7f4ed00cc860 2023-11-08T08:21:00.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.061+0000 7f4ece59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ed0160710 0x7f4ed00cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.061+0000 7f4ece59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ed0160710 0x7f4ed00cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42592/0 (socket says 172.21.15.40:42592) 2023-11-08T08:21:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.061+0000 7f4ece59c700 1 -- 172.21.15.40:0/1910467235 learned_addr learned my addr 172.21.15.40:0/1910467235 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.061+0000 7f4ec7fff700 1 --2- 172.21.15.40:0/1910467235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed00cc860 0x7f4ed0070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.061+0000 7f4ece59c700 1 -- 172.21.15.40:0/1910467235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed00cc860 msgr2=0x7f4ed0070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:00.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.061+0000 7f4ece59c700 1 --2- 172.21.15.40:0/1910467235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed00cc860 0x7f4ed0070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.061+0000 7f4ece59c700 1 -- 172.21.15.40:0/1910467235 >> v1:172.21.15.169:6789/0 conn(0x7f4ed00705c0 legacy=0x7f4ed0069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:00.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.061+0000 7f4ece59c700 1 -- 172.21.15.40:0/1910467235 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed00adc60 con 0x7f4ed0160710 2023-11-08T08:21:00.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.061+0000 7f4ece59c700 1 --2- 172.21.15.40:0/1910467235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ed0160710 0x7f4ed00cc320 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009910 tx=0x7f4ec0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f86583213f75dd1d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.061+0000 7f4ecdd9b700 1 -- 172.21.15.40:0/1910467235 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0011830 con 0x7f4ed0160710 2023-11-08T08:21:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.061+0000 7f4ecdd9b700 1 -- 172.21.15.40:0/1910467235 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec0011990 con 0x7f4ed0160710 2023-11-08T08:21:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.061+0000 7f4ecdd9b700 1 -- 172.21.15.40:0/1910467235 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0011d40 con 0x7f4ed0160710 2023-11-08T08:21:00.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.065+0000 7f4ecf59e700 1 -- 172.21.15.40:0/1910467235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ed0160710 msgr2=0x7f4ed00cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:00.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.065+0000 7f4ecf59e700 1 --2- 172.21.15.40:0/1910467235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ed0160710 0x7f4ed00cc320 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009910 tx=0x7f4ec0009c20 comp rx=0 tx=0).stop 2023-11-08T08:21:00.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.065+0000 7f4ecf59e700 1 -- 172.21.15.40:0/1910467235 shutdown_connections 2023-11-08T08:21:00.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.065+0000 7f4ecf59e700 1 --2- 172.21.15.40:0/1910467235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed00cc860 0x7f4ed0070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.065+0000 7f4ecf59e700 1 --2- 172.21.15.40:0/1910467235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ed0160710 0x7f4ed00cc320 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.065+0000 7f4ecf59e700 1 -- 172.21.15.40:0/1910467235 >> 172.21.15.40:0/1910467235 conn(0x7f4ed00b0f80 msgr2=0x7f4ed00ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:00.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.065+0000 7f4ecf59e700 1 -- 172.21.15.40:0/1910467235 shutdown_connections 2023-11-08T08:21:00.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.065+0000 7f4ecf59e700 1 -- 172.21.15.40:0/1910467235 wait complete. 2023-11-08T08:21:00.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.069+0000 7f4ecf59e700 1 Processor -- start 2023-11-08T08:21:00.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.069+0000 7f4ecf59e700 1 -- start start 2023-11-08T08:21:00.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.069+0000 7f4ecf59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ed00705c0 0x7f4ed0150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:00.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.069+0000 7f4ecf59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ed00cc860 0x7f4ed00b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:00.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.069+0000 7f4ecf59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed0160710 0x7f4ed015da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:00.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.069+0000 7f4ecf59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed01506b0 con 0x7f4ed00705c0 2023-11-08T08:21:00.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.069+0000 7f4ecf59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed00b1c90 con 0x7f4ed00cc860 2023-11-08T08:21:00.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.069+0000 7f4ecf59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed00b19b0 con 0x7f4ed0160710 2023-11-08T08:21:00.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.069+0000 7f4ece59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ed00705c0 0x7f4ed0150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:00.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.069+0000 7f4ece59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ed00705c0 0x7f4ed0150170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42594/0 (socket says 172.21.15.40:42594) 2023-11-08T08:21:00.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.069+0000 7f4ece59c700 1 -- 172.21.15.40:0/239653440 learned_addr learned my addr 172.21.15.40:0/239653440 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:00.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.069+0000 7f4ec7fff700 1 --2- 172.21.15.40:0/239653440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ed00cc860 0x7f4ed00b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:00.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.069+0000 7f4eced9d700 1 --2- 172.21.15.40:0/239653440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed0160710 0x7f4ed015da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:00.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.069+0000 7f4ece59c700 1 -- 172.21.15.40:0/239653440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed0160710 msgr2=0x7f4ed015da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:00.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.069+0000 7f4ece59c700 1 --2- 172.21.15.40:0/239653440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed0160710 0x7f4ed015da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.069+0000 7f4ece59c700 1 -- 172.21.15.40:0/239653440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ed00cc860 msgr2=0x7f4ed00b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:00.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.069+0000 7f4ece59c700 1 --2- 172.21.15.40:0/239653440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ed00cc860 0x7f4ed00b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.069+0000 7f4ece59c700 1 -- 172.21.15.40:0/239653440 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec0009570 con 0x7f4ed00705c0 2023-11-08T08:21:00.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.069+0000 7f4ece59c700 1 --2- 172.21.15.40:0/239653440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ed00705c0 0x7f4ed0150170 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0000c00 tx=0x7f4ec0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:00.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.073+0000 7f4ec6ffd700 1 -- 172.21.15.40:0/239653440 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0004610 con 0x7f4ed00705c0 2023-11-08T08:21:00.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.073+0000 7f4ec6ffd700 1 -- 172.21.15.40:0/239653440 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec001e070 con 0x7f4ed00705c0 2023-11-08T08:21:00.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.073+0000 7f4ec6ffd700 1 -- 172.21.15.40:0/239653440 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec00038e0 con 0x7f4ed00705c0 2023-11-08T08:21:00.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.073+0000 7f4ecf59e700 1 -- 172.21.15.40:0/239653440 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed006b6b0 con 0x7f4ed00705c0 2023-11-08T08:21:00.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.073+0000 7f4ecf59e700 1 -- 172.21.15.40:0/239653440 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed006b3d0 con 0x7f4ed00705c0 2023-11-08T08:21:00.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.073+0000 7f4ecf59e700 1 -- 172.21.15.40:0/239653440 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed00ad8f0 con 0x7f4ed00705c0 2023-11-08T08:21:00.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.073+0000 7f4ec6ffd700 1 -- 172.21.15.40:0/239653440 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4ec002a020 con 0x7f4ed00705c0 2023-11-08T08:21:00.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.077+0000 7f4ec6ffd700 1 --2- 172.21.15.40:0/239653440 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4eb0064ee0 0x7f4eb0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:00.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.077+0000 7f4ec6ffd700 1 -- 172.21.15.40:0/239653440 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f4ec00831f0 con 0x7f4ed00705c0 2023-11-08T08:21:00.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.077+0000 7f4ec7fff700 1 --2- 172.21.15.40:0/239653440 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4eb0064ee0 0x7f4eb0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:00.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.077+0000 7f4ec7fff700 1 --2- 172.21.15.40:0/239653440 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4eb0064ee0 0x7f4eb0067390 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4ed015bd40 tx=0x7f4eb8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:00.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.081+0000 7f4ec6ffd700 1 -- 172.21.15.40:0/239653440 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4ec0051180 con 0x7f4ed00705c0 2023-11-08T08:21:00.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.261+0000 7f4ecf59e700 1 -- 172.21.15.40:0/239653440 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ed00c1a40 con 0x7f4ed00705c0 2023-11-08T08:21:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.261+0000 7f4ec6ffd700 1 -- 172.21.15.40:0/239653440 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+18127 (secure 0 0 0) 0x7f4ed00c1a40 con 0x7f4ed00705c0 2023-11-08T08:21:00.268 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:00.268 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":45,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:20:45.994346+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:20:32.059150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-08T08:20:33.297064+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":4}]},{"pgid":"7.2e","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:21:00.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.265+0000 7f4ecf59e700 1 -- 172.21.15.40:0/239653440 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4eb0064ee0 msgr2=0x7f4eb0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:00.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.265+0000 7f4ecf59e700 1 --2- 172.21.15.40:0/239653440 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4eb0064ee0 0x7f4eb0067390 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4ed015bd40 tx=0x7f4eb8008040 comp rx=0 tx=0).stop 2023-11-08T08:21:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.265+0000 7f4ecf59e700 1 -- 172.21.15.40:0/239653440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ed00705c0 msgr2=0x7f4ed0150170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.265+0000 7f4ecf59e700 1 --2- 172.21.15.40:0/239653440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ed00705c0 0x7f4ed0150170 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0000c00 tx=0x7f4ec0004770 comp rx=0 tx=0).stop 2023-11-08T08:21:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.265+0000 7f4ecf59e700 1 -- 172.21.15.40:0/239653440 shutdown_connections 2023-11-08T08:21:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.265+0000 7f4ecf59e700 1 --2- 172.21.15.40:0/239653440 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4eb0064ee0 0x7f4eb0067390 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.265+0000 7f4ecf59e700 1 --2- 172.21.15.40:0/239653440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ed0160710 0x7f4ed015da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.265+0000 7f4ecf59e700 1 --2- 172.21.15.40:0/239653440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ed00cc860 0x7f4ed00b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.265+0000 7f4ecf59e700 1 --2- 172.21.15.40:0/239653440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ed00705c0 0x7f4ed0150170 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.265+0000 7f4ecf59e700 1 -- 172.21.15.40:0/239653440 >> 172.21.15.40:0/239653440 conn(0x7f4ed00b0f80 msgr2=0x7f4ed006bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:00.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.265+0000 7f4ecf59e700 1 -- 172.21.15.40:0/239653440 shutdown_connections 2023-11-08T08:21:00.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.265+0000 7f4ecf59e700 1 -- 172.21.15.40:0/239653440 wait complete. 2023-11-08T08:21:00.299 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T08:20:32.059150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T08:20:33.297064+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T08:21:00.300 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:21:00.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.481+0000 7f344f59e700 1 Processor -- start 2023-11-08T08:21:00.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.481+0000 7f344f59e700 1 -- start start 2023-11-08T08:21:00.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.485+0000 7f344f59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3450070620 0x7f34500b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:00.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.485+0000 7f344f59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3450069bf0 0x7f345006a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:00.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.485+0000 7f344f59e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f345015c520 con 0x7f345015c150 2023-11-08T08:21:00.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.485+0000 7f344f59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34500b4240 con 0x7f3450070620 2023-11-08T08:21:00.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.485+0000 7f344f59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345006cc50 con 0x7f3450069bf0 2023-11-08T08:21:00.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.485+0000 7f344dd9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3450070620 0x7f34500b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:00.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.485+0000 7f344dd9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3450070620 0x7f34500b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42602/0 (socket says 172.21.15.40:42602) 2023-11-08T08:21:00.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.485+0000 7f344dd9b700 1 -- 172.21.15.40:0/1158565689 learned_addr learned my addr 172.21.15.40:0/1158565689 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:00.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.485+0000 7f344e59c700 1 --1- 172.21.15.40:0/1158565689 >> v1:172.21.15.169:6789/0 conn(0x7f345015c150 0x7f345006ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:51158/0 (socket says 172.21.15.40:51158) 2023-11-08T08:21:00.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.485+0000 7f344ed9d700 1 --2- 172.21.15.40:0/1158565689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3450069bf0 0x7f345006a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:00.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.485+0000 7f344dd9b700 1 -- 172.21.15.40:0/1158565689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3450069bf0 msgr2=0x7f345006a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:00.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.485+0000 7f344dd9b700 1 --2- 172.21.15.40:0/1158565689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3450069bf0 0x7f345006a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.485+0000 7f344dd9b700 1 -- 172.21.15.40:0/1158565689 >> v1:172.21.15.169:6789/0 conn(0x7f345015c150 legacy=0x7f345006ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:00.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.485+0000 7f344dd9b700 1 -- 172.21.15.40:0/1158565689 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34500d2450 con 0x7f3450070620 2023-11-08T08:21:00.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.485+0000 7f344dd9b700 1 --2- 172.21.15.40:0/1158565689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3450070620 0x7f34500b3d00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3444009910 tx=0x7f3444009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226617ca2fafe4f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:00.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.489+0000 7f344d59a700 1 -- 172.21.15.40:0/1158565689 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3444005e50 con 0x7f3450070620 2023-11-08T08:21:00.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.489+0000 7f344d59a700 1 -- 172.21.15.40:0/1158565689 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3444006030 con 0x7f3450070620 2023-11-08T08:21:00.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.489+0000 7f344d59a700 1 -- 172.21.15.40:0/1158565689 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34440063e0 con 0x7f3450070620 2023-11-08T08:21:00.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.489+0000 7f344f59e700 1 -- 172.21.15.40:0/1158565689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3450070620 msgr2=0x7f34500b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:00.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.489+0000 7f344f59e700 1 --2- 172.21.15.40:0/1158565689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3450070620 0x7f34500b3d00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3444009910 tx=0x7f3444009c20 comp rx=0 tx=0).stop 2023-11-08T08:21:00.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.489+0000 7f344f59e700 1 -- 172.21.15.40:0/1158565689 shutdown_connections 2023-11-08T08:21:00.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.489+0000 7f344f59e700 1 --2- 172.21.15.40:0/1158565689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3450069bf0 0x7f345006a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.489+0000 7f344f59e700 1 --2- 172.21.15.40:0/1158565689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3450070620 0x7f34500b3d00 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.489+0000 7f344f59e700 1 -- 172.21.15.40:0/1158565689 >> 172.21.15.40:0/1158565689 conn(0x7f34500d3680 msgr2=0x7f34500cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.489+0000 7f344f59e700 1 -- 172.21.15.40:0/1158565689 shutdown_connections 2023-11-08T08:21:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.489+0000 7f344f59e700 1 -- 172.21.15.40:0/1158565689 wait complete. 2023-11-08T08:21:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.493+0000 7f344f59e700 1 Processor -- start 2023-11-08T08:21:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.493+0000 7f344f59e700 1 -- start start 2023-11-08T08:21:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.493+0000 7f344f59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3450069bf0 0x7f3450156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.493+0000 7f344f59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3450070620 0x7f34500cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.493+0000 7f344f59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f345015c150 0x7f34500b7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.493+0000 7f344f59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34500b7380 con 0x7f345015c150 2023-11-08T08:21:00.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.493+0000 7f344f59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3450156790 con 0x7f3450070620 2023-11-08T08:21:00.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.493+0000 7f344f59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34500cfde0 con 0x7f3450069bf0 2023-11-08T08:21:00.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.493+0000 7f344ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f345015c150 0x7f34500b7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:00.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.493+0000 7f344ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f345015c150 0x7f34500b7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42616/0 (socket says 172.21.15.40:42616) 2023-11-08T08:21:00.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.493+0000 7f344ed9d700 1 -- 172.21.15.40:0/402274649 learned_addr learned my addr 172.21.15.40:0/402274649 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:00.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.493+0000 7f344e59c700 1 --2- 172.21.15.40:0/402274649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3450069bf0 0x7f3450156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:00.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.493+0000 7f344dd9b700 1 --2- 172.21.15.40:0/402274649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3450070620 0x7f34500cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:00.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.493+0000 7f344ed9d700 1 -- 172.21.15.40:0/402274649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3450069bf0 msgr2=0x7f3450156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:00.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.493+0000 7f344ed9d700 1 --2- 172.21.15.40:0/402274649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3450069bf0 0x7f3450156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.493+0000 7f344ed9d700 1 -- 172.21.15.40:0/402274649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3450070620 msgr2=0x7f34500cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.493+0000 7f344ed9d700 1 --2- 172.21.15.40:0/402274649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3450070620 0x7f34500cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.493+0000 7f344ed9d700 1 -- 172.21.15.40:0/402274649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3444009520 con 0x7f345015c150 2023-11-08T08:21:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.493+0000 7f344ed9d700 1 --2- 172.21.15.40:0/402274649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f345015c150 0x7f34500b7ab0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3448009dd0 tx=0x7f344800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.497+0000 7f343effd700 1 -- 172.21.15.40:0/402274649 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344800cd40 con 0x7f345015c150 2023-11-08T08:21:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.497+0000 7f343effd700 1 -- 172.21.15.40:0/402274649 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344800f040 con 0x7f345015c150 2023-11-08T08:21:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.497+0000 7f344f59e700 1 -- 172.21.15.40:0/402274649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34500d05e0 con 0x7f345015c150 2023-11-08T08:21:00.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.497+0000 7f343effd700 1 -- 172.21.15.40:0/402274649 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34480146c0 con 0x7f345015c150 2023-11-08T08:21:00.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.497+0000 7f344f59e700 1 -- 172.21.15.40:0/402274649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34500d2730 con 0x7f345015c150 2023-11-08T08:21:00.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.497+0000 7f344f59e700 1 -- 172.21.15.40:0/402274649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34500d2140 con 0x7f345015c150 2023-11-08T08:21:00.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.501+0000 7f343effd700 1 -- 172.21.15.40:0/402274649 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f3448014820 con 0x7f345015c150 2023-11-08T08:21:00.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.501+0000 7f343effd700 1 --2- 172.21.15.40:0/402274649 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3430064ee0 0x7f3430067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:00.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.501+0000 7f343effd700 1 -- 172.21.15.40:0/402274649 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f3448082a20 con 0x7f345015c150 2023-11-08T08:21:00.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.501+0000 7f344e59c700 1 --2- 172.21.15.40:0/402274649 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3430064ee0 0x7f3430067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:00.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.501+0000 7f344e59c700 1 --2- 172.21.15.40:0/402274649 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3430064ee0 0x7f3430067390 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3440000f60 tx=0x7f3440007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:00.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.505+0000 7f343effd700 1 -- 172.21.15.40:0/402274649 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f3448050a40 con 0x7f345015c150 2023-11-08T08:21:00.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.685+0000 7f344f59e700 1 -- 172.21.15.40:0/402274649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3450073d20 con 0x7f345015c150 2023-11-08T08:21:00.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.685+0000 7f343effd700 1 -- 172.21.15.40:0/402274649 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f344800cea0 con 0x7f345015c150 2023-11-08T08:21:00.691 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:21:00.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.689+0000 7f344f59e700 1 -- 172.21.15.40:0/402274649 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3430064ee0 msgr2=0x7f3430067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:00.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.689+0000 7f344f59e700 1 --2- 172.21.15.40:0/402274649 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3430064ee0 0x7f3430067390 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3440000f60 tx=0x7f3440007d30 comp rx=0 tx=0).stop 2023-11-08T08:21:00.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.689+0000 7f344f59e700 1 -- 172.21.15.40:0/402274649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f345015c150 msgr2=0x7f34500b7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:00.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.689+0000 7f344f59e700 1 --2- 172.21.15.40:0/402274649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f345015c150 0x7f34500b7ab0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3448009dd0 tx=0x7f344800c3a0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.693+0000 7f344f59e700 1 -- 172.21.15.40:0/402274649 shutdown_connections 2023-11-08T08:21:00.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.693+0000 7f344f59e700 1 --2- 172.21.15.40:0/402274649 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3430064ee0 0x7f3430067390 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.693+0000 7f344f59e700 1 --2- 172.21.15.40:0/402274649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f345015c150 0x7f34500b7ab0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.693+0000 7f344f59e700 1 --2- 172.21.15.40:0/402274649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3450070620 0x7f34500cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.693+0000 7f344f59e700 1 --2- 172.21.15.40:0/402274649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3450069bf0 0x7f3450156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.693+0000 7f344f59e700 1 -- 172.21.15.40:0/402274649 >> 172.21.15.40:0/402274649 conn(0x7f34500d3680 msgr2=0x7f3450155990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:00.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.693+0000 7f344f59e700 1 -- 172.21.15.40:0/402274649 shutdown_connections 2023-11-08T08:21:00.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.693+0000 7f344f59e700 1 -- 172.21.15.40:0/402274649 wait complete. 2023-11-08T08:21:00.723 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:21:00.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.905+0000 7f037074f700 1 Processor -- start 2023-11-08T08:21:00.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.905+0000 7f037074f700 1 -- start start 2023-11-08T08:21:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.909+0000 7f037074f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f03680aed60 0x7f0368165080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.909+0000 7f037074f700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f036806c680 0x7f03680adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.909+0000 7f037074f700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f036806afb0 con 0x7f036806c2b0 2023-11-08T08:21:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.909+0000 7f037074f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03680ca750 con 0x7f03680aed60 2023-11-08T08:21:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.909+0000 7f037074f700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036806b290 con 0x7f036806c680 2023-11-08T08:21:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.909+0000 7f036dcea700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f03680aed60 0x7f0368165080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.909+0000 7f036ecec700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f036806c680 0x7f03680adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:00.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.909+0000 7f036dcea700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f03680aed60 0x7f0368165080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42626/0 (socket says 172.21.15.40:42626) 2023-11-08T08:21:00.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.909+0000 7f036dcea700 1 -- 172.21.15.40:0/2906125604 learned_addr learned my addr 172.21.15.40:0/2906125604 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:00.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.909+0000 7f036dcea700 1 -- 172.21.15.40:0/2906125604 >> v2:172.21.15.169:3300/0 conn(0x7f036806c680 msgr2=0x7f03680adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:00.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.909+0000 7f036dcea700 1 --2- 172.21.15.40:0/2906125604 >> v2:172.21.15.169:3300/0 conn(0x7f036806c680 0x7f03680adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.909+0000 7f036d4e9700 1 -- 172.21.15.40:0/2906125604 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 747373841 0 0) 0x7f036806afb0 con 0x7f036806c2b0 2023-11-08T08:21:00.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.909+0000 7f036dcea700 1 -- 172.21.15.40:0/2906125604 >> v1:172.21.15.169:6789/0 conn(0x7f036806c2b0 legacy=0x7f03681639d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:00.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.909+0000 7f036dcea700 1 -- 172.21.15.40:0/2906125604 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03680c8f40 con 0x7f03680aed60 2023-11-08T08:21:00.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.913+0000 7f036dcea700 1 --2- 172.21.15.40:0/2906125604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03680aed60 0x7f0368165080 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0368073190 tx=0x7f0358009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7effa0158508755 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:00.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.913+0000 7f036d4e9700 1 -- 172.21.15.40:0/2906125604 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0358011830 con 0x7f03680aed60 2023-11-08T08:21:00.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.913+0000 7f036d4e9700 1 -- 172.21.15.40:0/2906125604 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0358011990 con 0x7f03680aed60 2023-11-08T08:21:00.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.913+0000 7f036d4e9700 1 -- 172.21.15.40:0/2906125604 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0358011d20 con 0x7f03680aed60 2023-11-08T08:21:00.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.913+0000 7f037074f700 1 -- 172.21.15.40:0/2906125604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03680aed60 msgr2=0x7f0368165080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:00.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.913+0000 7f037074f700 1 --2- 172.21.15.40:0/2906125604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03680aed60 0x7f0368165080 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0368073190 tx=0x7f0358009bf0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.913+0000 7f037074f700 1 -- 172.21.15.40:0/2906125604 shutdown_connections 2023-11-08T08:21:00.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.913+0000 7f037074f700 1 --2- 172.21.15.40:0/2906125604 >> v2:172.21.15.169:3300/0 conn(0x7f036806c680 0x7f03680adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.913+0000 7f037074f700 1 --2- 172.21.15.40:0/2906125604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03680aed60 0x7f0368165080 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:00.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.913+0000 7f037074f700 1 -- 172.21.15.40:0/2906125604 >> 172.21.15.40:0/2906125604 conn(0x7f03680cf580 msgr2=0x7f03680cb370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.913+0000 7f037074f700 1 -- 172.21.15.40:0/2906125604 shutdown_connections 2023-11-08T08:21:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.913+0000 7f037074f700 1 -- 172.21.15.40:0/2906125604 wait complete. 2023-11-08T08:21:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.917+0000 7f037074f700 1 Processor -- start 2023-11-08T08:21:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.917+0000 7f037074f700 1 -- start start 2023-11-08T08:21:01.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.917+0000 7f037074f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f036806c2b0 0x7f03680b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:01.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.917+0000 7f037074f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f036806c680 0x7f03680b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:01.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.917+0000 7f037074f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03680aed60 0x7f03680b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:01.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.917+0000 7f037074f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03681ea030 con 0x7f036806c2b0 2023-11-08T08:21:01.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.917+0000 7f037074f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036814d640 con 0x7f03680aed60 2023-11-08T08:21:01.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.917+0000 7f037074f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03680b47c0 con 0x7f036806c680 2023-11-08T08:21:01.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.917+0000 7f036e4eb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f036806c2b0 0x7f03680b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:01.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.917+0000 7f036e4eb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f036806c2b0 0x7f03680b4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42630/0 (socket says 172.21.15.40:42630) 2023-11-08T08:21:01.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.917+0000 7f036e4eb700 1 -- 172.21.15.40:0/1340766637 learned_addr learned my addr 172.21.15.40:0/1340766637 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:01.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.917+0000 7f036ecec700 1 --2- 172.21.15.40:0/1340766637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03680aed60 0x7f03680b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:01.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.917+0000 7f036dcea700 1 --2- 172.21.15.40:0/1340766637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f036806c680 0x7f03680b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:01.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.917+0000 7f036e4eb700 1 -- 172.21.15.40:0/1340766637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f036806c680 msgr2=0x7f03680b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:01.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.917+0000 7f036e4eb700 1 --2- 172.21.15.40:0/1340766637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f036806c680 0x7f03680b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.917+0000 7f036e4eb700 1 -- 172.21.15.40:0/1340766637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03680aed60 msgr2=0x7f03680b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:01.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.917+0000 7f036e4eb700 1 --2- 172.21.15.40:0/1340766637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03680aed60 0x7f03680b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.917+0000 7f036e4eb700 1 -- 172.21.15.40:0/1340766637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0358009570 con 0x7f036806c2b0 2023-11-08T08:21:01.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.917+0000 7f036e4eb700 1 --2- 172.21.15.40:0/1340766637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f036806c2b0 0x7f03680b4280 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f03640011e0 tx=0x7f036400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:01.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.921+0000 7f035effd700 1 -- 172.21.15.40:0/1340766637 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03640054e0 con 0x7f036806c2b0 2023-11-08T08:21:01.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.921+0000 7f035effd700 1 -- 172.21.15.40:0/1340766637 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03640036a0 con 0x7f036806c2b0 2023-11-08T08:21:01.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.921+0000 7f035effd700 1 -- 172.21.15.40:0/1340766637 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364003a50 con 0x7f036806c2b0 2023-11-08T08:21:01.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.921+0000 7f037074f700 1 -- 172.21.15.40:0/1340766637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03680ad800 con 0x7f036806c2b0 2023-11-08T08:21:01.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.921+0000 7f037074f700 1 -- 172.21.15.40:0/1340766637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03680c9220 con 0x7f036806c2b0 2023-11-08T08:21:01.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.921+0000 7f037074f700 1 -- 172.21.15.40:0/1340766637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03681640e0 con 0x7f036806c2b0 2023-11-08T08:21:01.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.921+0000 7f035effd700 1 -- 172.21.15.40:0/1340766637 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f0364019030 con 0x7f036806c2b0 2023-11-08T08:21:01.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.925+0000 7f035effd700 1 --2- 172.21.15.40:0/1340766637 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f034c064ee0 0x7f034c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:01.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.925+0000 7f035effd700 1 -- 172.21.15.40:0/1340766637 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f03640812a0 con 0x7f036806c2b0 2023-11-08T08:21:01.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.925+0000 7f036dcea700 1 --2- 172.21.15.40:0/1340766637 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f034c064ee0 0x7f034c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:01.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.925+0000 7f036dcea700 1 --2- 172.21.15.40:0/1340766637 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f034c064ee0 0x7f034c067390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0368073190 tx=0x7f0358008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:01.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:00.933+0000 7f035effd700 1 -- 172.21.15.40:0/1340766637 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f036404f2c0 con 0x7f036806c2b0 2023-11-08T08:21:01.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.109+0000 7f037074f700 1 -- 172.21.15.40:0/1340766637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f03681505e0 con 0x7f036806c2b0 2023-11-08T08:21:01.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.109+0000 7f035effd700 1 -- 172.21.15.40:0/1340766637 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0364017020 con 0x7f036806c2b0 2023-11-08T08:21:01.116 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:21:01.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.117+0000 7f037074f700 1 -- 172.21.15.40:0/1340766637 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f034c064ee0 msgr2=0x7f034c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:01.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.117+0000 7f037074f700 1 --2- 172.21.15.40:0/1340766637 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f034c064ee0 0x7f034c067390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0368073190 tx=0x7f0358008030 comp rx=0 tx=0).stop 2023-11-08T08:21:01.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.117+0000 7f037074f700 1 -- 172.21.15.40:0/1340766637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f036806c2b0 msgr2=0x7f03680b4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:01.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.117+0000 7f037074f700 1 --2- 172.21.15.40:0/1340766637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f036806c2b0 0x7f03680b4280 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f03640011e0 tx=0x7f036400ca30 comp rx=0 tx=0).stop 2023-11-08T08:21:01.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.117+0000 7f037074f700 1 -- 172.21.15.40:0/1340766637 shutdown_connections 2023-11-08T08:21:01.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.117+0000 7f037074f700 1 --2- 172.21.15.40:0/1340766637 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f034c064ee0 0x7f034c067390 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.117+0000 7f037074f700 1 --2- 172.21.15.40:0/1340766637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03680aed60 0x7f03680b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.117+0000 7f037074f700 1 --2- 172.21.15.40:0/1340766637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f036806c680 0x7f03680b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.117+0000 7f037074f700 1 --2- 172.21.15.40:0/1340766637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f036806c2b0 0x7f03680b4280 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.117+0000 7f037074f700 1 -- 172.21.15.40:0/1340766637 >> 172.21.15.40:0/1340766637 conn(0x7f03680cf580 msgr2=0x7f03681621c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:01.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.117+0000 7f037074f700 1 -- 172.21.15.40:0/1340766637 shutdown_connections 2023-11-08T08:21:01.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.117+0000 7f037074f700 1 -- 172.21.15.40:0/1340766637 wait complete. 2023-11-08T08:21:01.144 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T08:21:01.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.329+0000 7f94fc619700 1 Processor -- start 2023-11-08T08:21:01.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.329+0000 7f94fc619700 1 -- start start 2023-11-08T08:21:01.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.333+0000 7f94fc619700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f94f40ada10 0x7f94f4160cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:01.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.333+0000 7f94fc619700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94f406b220 0x7f94f40c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:01.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.333+0000 7f94fc619700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94f406c040 0x7f94f406c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:01.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.333+0000 7f94fc619700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f41608b0 con 0x7f94f406b220 2023-11-08T08:21:01.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.333+0000 7f94fc619700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f406ca00 con 0x7f94f40ada10 2023-11-08T08:21:01.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.333+0000 7f94fc619700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40ca3b0 con 0x7f94f406c040 2023-11-08T08:21:01.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.333+0000 7f94f9bb4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94f406b220 0x7f94f40c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:01.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.333+0000 7f94f9bb4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94f406b220 0x7f94f40c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42632/0 (socket says 172.21.15.40:42632) 2023-11-08T08:21:01.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.333+0000 7f94f9bb4700 1 -- 172.21.15.40:0/3639247206 learned_addr learned my addr 172.21.15.40:0/3639247206 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:01.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.333+0000 7f94fa3b5700 1 --2- 172.21.15.40:0/3639247206 >> v2:172.21.15.169:3300/0 conn(0x7f94f40ada10 0x7f94f4160cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:01.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.333+0000 7f94fabb6700 1 --2- 172.21.15.40:0/3639247206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94f406c040 0x7f94f406c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:01.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.333+0000 7f94f9bb4700 1 -- 172.21.15.40:0/3639247206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94f406c040 msgr2=0x7f94f406c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:01.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.333+0000 7f94f9bb4700 1 --2- 172.21.15.40:0/3639247206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94f406c040 0x7f94f406c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.333+0000 7f94f9bb4700 1 -- 172.21.15.40:0/3639247206 >> v2:172.21.15.169:3300/0 conn(0x7f94f40ada10 msgr2=0x7f94f4160cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:01.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.333+0000 7f94f9bb4700 1 --2- 172.21.15.40:0/3639247206 >> v2:172.21.15.169:3300/0 conn(0x7f94f40ada10 0x7f94f4160cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.333+0000 7f94f9bb4700 1 -- 172.21.15.40:0/3639247206 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f41601f0 con 0x7f94f406b220 2023-11-08T08:21:01.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.333+0000 7f94fabb6700 1 --2- 172.21.15.40:0/3639247206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94f406c040 0x7f94f406c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:01.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.337+0000 7f94f9bb4700 1 --2- 172.21.15.40:0/3639247206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f406b220 0x7f94f40c9b80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f94f40700b0 tx=0x7f94e4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2220a68092a7de7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:01.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.337+0000 7f94f93b3700 1 -- 172.21.15.40:0/3639247206 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e4011830 con 0x7f94f406b220 2023-11-08T08:21:01.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.337+0000 7f94f93b3700 1 -- 172.21.15.40:0/3639247206 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e4011990 con 0x7f94f406b220 2023-11-08T08:21:01.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.337+0000 7f94f93b3700 1 -- 172.21.15.40:0/3639247206 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e4011d20 con 0x7f94f406b220 2023-11-08T08:21:01.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.337+0000 7f94fc619700 1 -- 172.21.15.40:0/3639247206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f406b220 msgr2=0x7f94f40c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:01.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.337+0000 7f94fc619700 1 --2- 172.21.15.40:0/3639247206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f406b220 0x7f94f40c9b80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f94f40700b0 tx=0x7f94e4009bf0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.337+0000 7f94fc619700 1 -- 172.21.15.40:0/3639247206 shutdown_connections 2023-11-08T08:21:01.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.337+0000 7f94fc619700 1 --2- 172.21.15.40:0/3639247206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f406b220 0x7f94f40c9b80 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.337+0000 7f94fc619700 1 --2- 172.21.15.40:0/3639247206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94f406c040 0x7f94f406c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.337+0000 7f94fc619700 1 --2- 172.21.15.40:0/3639247206 >> v2:172.21.15.169:3300/0 conn(0x7f94f40ada10 0x7f94f4160cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.337+0000 7f94fc619700 1 -- 172.21.15.40:0/3639247206 >> 172.21.15.40:0/3639247206 conn(0x7f94f40cf580 msgr2=0x7f94f40cb370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:01.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.337+0000 7f94fc619700 1 -- 172.21.15.40:0/3639247206 shutdown_connections 2023-11-08T08:21:01.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.337+0000 7f94fc619700 1 -- 172.21.15.40:0/3639247206 wait complete. 2023-11-08T08:21:01.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.341+0000 7f94fc619700 1 Processor -- start 2023-11-08T08:21:01.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.341+0000 7f94fc619700 1 -- start start 2023-11-08T08:21:01.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.341+0000 7f94fc619700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f406b220 0x7f94f4151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:01.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.341+0000 7f94fc619700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94f406c040 0x7f94f40afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:01.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.341+0000 7f94fc619700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94f40ada10 0x7f94f41597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:01.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.341+0000 7f94fc619700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40ade80 con 0x7f94f406b220 2023-11-08T08:21:01.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.341+0000 7f94fc619700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40ae160 con 0x7f94f40ada10 2023-11-08T08:21:01.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.341+0000 7f94fc619700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40ae440 con 0x7f94f406c040 2023-11-08T08:21:01.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.341+0000 7f94fa3b5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f406b220 0x7f94f4151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:01.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.341+0000 7f94fa3b5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f406b220 0x7f94f4151520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42648/0 (socket says 172.21.15.40:42648) 2023-11-08T08:21:01.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.341+0000 7f94fa3b5700 1 -- 172.21.15.40:0/1588180137 learned_addr learned my addr 172.21.15.40:0/1588180137 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:01.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.341+0000 7f94f9bb4700 1 --2- 172.21.15.40:0/1588180137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94f406c040 0x7f94f40afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:01.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.341+0000 7f94fabb6700 1 --2- 172.21.15.40:0/1588180137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94f40ada10 0x7f94f41597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:01.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.341+0000 7f94fa3b5700 1 -- 172.21.15.40:0/1588180137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94f406c040 msgr2=0x7f94f40afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:01.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.341+0000 7f94fa3b5700 1 --2- 172.21.15.40:0/1588180137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94f406c040 0x7f94f40afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.341+0000 7f94fa3b5700 1 -- 172.21.15.40:0/1588180137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94f40ada10 msgr2=0x7f94f41597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:01.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.341+0000 7f94fa3b5700 1 --2- 172.21.15.40:0/1588180137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94f40ada10 0x7f94f41597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.341+0000 7f94fa3b5700 1 -- 172.21.15.40:0/1588180137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e4009570 con 0x7f94f406b220 2023-11-08T08:21:01.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.345+0000 7f94fa3b5700 1 --2- 172.21.15.40:0/1588180137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f406b220 0x7f94f4151520 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f94f000e840 tx=0x7f94f000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:01.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.345+0000 7f94eaffd700 1 -- 172.21.15.40:0/1588180137 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f00095a0 con 0x7f94f406b220 2023-11-08T08:21:01.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.345+0000 7f94eaffd700 1 -- 172.21.15.40:0/1588180137 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f000c690 con 0x7f94f406b220 2023-11-08T08:21:01.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.345+0000 7f94fc619700 1 -- 172.21.15.40:0/1588180137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f40ca0c0 con 0x7f94f406b220 2023-11-08T08:21:01.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.345+0000 7f94eaffd700 1 -- 172.21.15.40:0/1588180137 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f000ca00 con 0x7f94f406b220 2023-11-08T08:21:01.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.345+0000 7f94fc619700 1 -- 172.21.15.40:0/1588180137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f41604d0 con 0x7f94f406b220 2023-11-08T08:21:01.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.345+0000 7f94fc619700 1 -- 172.21.15.40:0/1588180137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f40ca690 con 0x7f94f406b220 2023-11-08T08:21:01.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.349+0000 7f94eaffd700 1 -- 172.21.15.40:0/1588180137 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f94f0019030 con 0x7f94f406b220 2023-11-08T08:21:01.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.349+0000 7f94eaffd700 1 --2- 172.21.15.40:0/1588180137 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f94d8064f30 0x7f94d80673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:01.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.349+0000 7f94eaffd700 1 -- 172.21.15.40:0/1588180137 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f94f0081f30 con 0x7f94f406b220 2023-11-08T08:21:01.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.349+0000 7f94f9bb4700 1 --2- 172.21.15.40:0/1588180137 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f94d8064f30 0x7f94d80673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:01.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.353+0000 7f94f9bb4700 1 --2- 172.21.15.40:0/1588180137 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f94d8064f30 0x7f94d80673e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f94e4000c00 tx=0x7f94e4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:01.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.357+0000 7f94eaffd700 1 -- 172.21.15.40:0/1588180137 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f94f004fee0 con 0x7f94f406b220 2023-11-08T08:21:01.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.537+0000 7f94fc619700 1 -- 172.21.15.40:0/1588180137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f94f40726c0 con 0x7f94f406b220 2023-11-08T08:21:01.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.537+0000 7f94eaffd700 1 -- 172.21.15.40:0/1588180137 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+11 (secure 0 0 0) 0x7f94f0053500 con 0x7f94f406b220 2023-11-08T08:21:01.543 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:21:01.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.541+0000 7f94fc619700 1 -- 172.21.15.40:0/1588180137 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f94d8064f30 msgr2=0x7f94d80673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:01.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.541+0000 7f94fc619700 1 --2- 172.21.15.40:0/1588180137 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f94d8064f30 0x7f94d80673e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f94e4000c00 tx=0x7f94e4008030 comp rx=0 tx=0).stop 2023-11-08T08:21:01.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.541+0000 7f94fc619700 1 -- 172.21.15.40:0/1588180137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f406b220 msgr2=0x7f94f4151520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:01.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.541+0000 7f94fc619700 1 --2- 172.21.15.40:0/1588180137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f406b220 0x7f94f4151520 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f94f000e840 tx=0x7f94f000ec00 comp rx=0 tx=0).stop 2023-11-08T08:21:01.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.541+0000 7f94fc619700 1 -- 172.21.15.40:0/1588180137 shutdown_connections 2023-11-08T08:21:01.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.541+0000 7f94fc619700 1 --2- 172.21.15.40:0/1588180137 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f94d8064f30 0x7f94d80673e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.541+0000 7f94fc619700 1 --2- 172.21.15.40:0/1588180137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94f40ada10 0x7f94f41597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.541+0000 7f94fc619700 1 --2- 172.21.15.40:0/1588180137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94f406c040 0x7f94f40afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.541+0000 7f94fc619700 1 --2- 172.21.15.40:0/1588180137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94f406b220 0x7f94f4151520 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.541+0000 7f94fc619700 1 -- 172.21.15.40:0/1588180137 >> 172.21.15.40:0/1588180137 conn(0x7f94f40cf580 msgr2=0x7f94f40cda20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:01.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.541+0000 7f94fc619700 1 -- 172.21.15.40:0/1588180137 shutdown_connections 2023-11-08T08:21:01.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.541+0000 7f94fc619700 1 -- 172.21.15.40:0/1588180137 wait complete. 2023-11-08T08:21:01.568 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T08:21:01.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.753+0000 7f70a1199700 1 Processor -- start 2023-11-08T08:21:01.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.753+0000 7f70a1199700 1 -- start start 2023-11-08T08:21:01.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.753+0000 7f70a1199700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709c0d0160 0x7f709c0b2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:01.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.753+0000 7f70a1199700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f709c0ce560 con 0x7f709c1567d0 2023-11-08T08:21:01.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.753+0000 7f70a1199700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f709c0b2d90 con 0x7f709c0b2950 2023-11-08T08:21:01.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.753+0000 7f70a1199700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709c0ced70 con 0x7f709c0d0160 2023-11-08T08:21:01.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.753+0000 7f709ad9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f709c1567d0 0x7f709c15d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60160/0 (socket says 172.21.15.40:60160) 2023-11-08T08:21:01.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.753+0000 7f709ad9d700 1 -- 172.21.15.40:0/1896649004 learned_addr learned my addr 172.21.15.40:0/1896649004 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:01.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.753+0000 7f709a59c700 1 --2- 172.21.15.40:0/1896649004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709c0d0160 0x7f709c0b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:01.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.753+0000 7f7099d9b700 1 -- 172.21.15.40:0/1896649004 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2250518663 0 0) 0x7f709c0ce560 con 0x7f709c1567d0 2023-11-08T08:21:01.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.753+0000 7f7099d9b700 1 -- 172.21.15.40:0/1896649004 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70840035d0 con 0x7f709c1567d0 2023-11-08T08:21:01.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.753+0000 7f709a59c700 1 -- 172.21.15.40:0/1896649004 >> v1:172.21.15.169:6789/0 conn(0x7f709c0b2950 legacy=0x7f709c06ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:01.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.753+0000 7f709a59c700 1 -- 172.21.15.40:0/1896649004 >> v1:172.21.15.40:6789/0 conn(0x7f709c1567d0 legacy=0x7f709c15d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:01.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.753+0000 7f709a59c700 1 -- 172.21.15.40:0/1896649004 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c15c870 con 0x7f709c0d0160 2023-11-08T08:21:01.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.757+0000 7f709a59c700 1 --2- 172.21.15.40:0/1896649004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709c0d0160 0x7f709c0b2410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7090009880 tx=0x7f7090009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f10a9b099a6a76a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:01.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.757+0000 7f7099d9b700 1 -- 172.21.15.40:0/1896649004 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090011830 con 0x7f709c0d0160 2023-11-08T08:21:01.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.757+0000 7f7099d9b700 1 -- 172.21.15.40:0/1896649004 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7090011990 con 0x7f709c0d0160 2023-11-08T08:21:01.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.757+0000 7f7099d9b700 1 -- 172.21.15.40:0/1896649004 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090011d20 con 0x7f709c0d0160 2023-11-08T08:21:01.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.757+0000 7f70a1199700 1 -- 172.21.15.40:0/1896649004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709c0d0160 msgr2=0x7f709c0b2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:01.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.757+0000 7f70a1199700 1 --2- 172.21.15.40:0/1896649004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709c0d0160 0x7f709c0b2410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7090009880 tx=0x7f7090009b90 comp rx=0 tx=0).stop 2023-11-08T08:21:01.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.757+0000 7f70a1199700 1 -- 172.21.15.40:0/1896649004 shutdown_connections 2023-11-08T08:21:01.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.757+0000 7f70a1199700 1 --2- 172.21.15.40:0/1896649004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709c0d0160 0x7f709c0b2410 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.757+0000 7f70a1199700 1 -- 172.21.15.40:0/1896649004 >> 172.21.15.40:0/1896649004 conn(0x7f709c0cd410 msgr2=0x7f709c0cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:01.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.757+0000 7f70a1199700 1 -- 172.21.15.40:0/1896649004 shutdown_connections 2023-11-08T08:21:01.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.757+0000 7f70a1199700 1 -- 172.21.15.40:0/1896649004 wait complete. 2023-11-08T08:21:01.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.761+0000 7f70a1199700 1 Processor -- start 2023-11-08T08:21:01.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.761+0000 7f70a1199700 1 -- start start 2023-11-08T08:21:01.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.761+0000 7f70a1199700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f709c0b2950 0x7f709c151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:01.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.761+0000 7f70a1199700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f709c0d0160 0x7f709c151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:01.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.761+0000 7f70a1199700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709c1567d0 0x7f709c15e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:01.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.761+0000 7f70a1199700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c0c5ef0 con 0x7f709c0d0160 2023-11-08T08:21:01.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.761+0000 7f70a1199700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c0c61d0 con 0x7f709c0b2950 2023-11-08T08:21:01.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.761+0000 7f70a1199700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709c0c64b0 con 0x7f709c1567d0 2023-11-08T08:21:01.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.765+0000 7f709a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f709c0d0160 0x7f709c151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:01.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.765+0000 7f709a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f709c0d0160 0x7f709c151510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42656/0 (socket says 172.21.15.40:42656) 2023-11-08T08:21:01.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.765+0000 7f709a59c700 1 -- 172.21.15.40:0/1055472970 learned_addr learned my addr 172.21.15.40:0/1055472970 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:01.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.765+0000 7f709ad9d700 1 --2- 172.21.15.40:0/1055472970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f709c0b2950 0x7f709c151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:01.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.765+0000 7f709b59e700 1 --2- 172.21.15.40:0/1055472970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709c1567d0 0x7f709c15e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:01.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.765+0000 7f709a59c700 1 -- 172.21.15.40:0/1055472970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709c1567d0 msgr2=0x7f709c15e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:01.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.765+0000 7f709a59c700 1 --2- 172.21.15.40:0/1055472970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709c1567d0 0x7f709c15e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.765+0000 7f709a59c700 1 -- 172.21.15.40:0/1055472970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f709c0b2950 msgr2=0x7f709c151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:01.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.765+0000 7f709a59c700 1 --2- 172.21.15.40:0/1055472970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f709c0b2950 0x7f709c151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.765+0000 7f709a59c700 1 -- 172.21.15.40:0/1055472970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70900094b0 con 0x7f709c0d0160 2023-11-08T08:21:01.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.765+0000 7f709ad9d700 1 --2- 172.21.15.40:0/1055472970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f709c0b2950 0x7f709c151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:01.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.765+0000 7f709a59c700 1 --2- 172.21.15.40:0/1055472970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f709c0d0160 0x7f709c151510 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7090013040 tx=0x7f7090004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:01.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.765+0000 7f70837fe700 1 -- 172.21.15.40:0/1055472970 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70900038c0 con 0x7f709c0d0160 2023-11-08T08:21:01.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.765+0000 7f70837fe700 1 -- 172.21.15.40:0/1055472970 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7090003a40 con 0x7f709c0d0160 2023-11-08T08:21:01.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.765+0000 7f70837fe700 1 -- 172.21.15.40:0/1055472970 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090003df0 con 0x7f709c0d0160 2023-11-08T08:21:01.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.765+0000 7f70a1199700 1 -- 172.21.15.40:0/1055472970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709c0cea90 con 0x7f709c0d0160 2023-11-08T08:21:01.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.765+0000 7f70a1199700 1 -- 172.21.15.40:0/1055472970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709c0ce7b0 con 0x7f709c0d0160 2023-11-08T08:21:01.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.769+0000 7f70a1199700 1 -- 172.21.15.40:0/1055472970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709c15caf0 con 0x7f709c0d0160 2023-11-08T08:21:01.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.769+0000 7f70837fe700 1 -- 172.21.15.40:0/1055472970 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f709000f050 con 0x7f709c0d0160 2023-11-08T08:21:01.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.769+0000 7f70837fe700 1 --2- 172.21.15.40:0/1055472970 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7088064f10 0x7f70880673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:01.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.769+0000 7f70837fe700 1 -- 172.21.15.40:0/1055472970 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f7090083660 con 0x7f709c0d0160 2023-11-08T08:21:01.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.773+0000 7f709ad9d700 1 --2- 172.21.15.40:0/1055472970 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7088064f10 0x7f70880673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:01.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.773+0000 7f709ad9d700 1 --2- 172.21.15.40:0/1055472970 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7088064f10 0x7f70880673c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f708c001560 tx=0x7f708c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:01.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.777+0000 7f70837fe700 1 -- 172.21.15.40:0/1055472970 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7090051590 con 0x7f709c0d0160 2023-11-08T08:21:01.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.953+0000 7f70a1199700 1 -- 172.21.15.40:0/1055472970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f709c0726c0 con 0x7f709c0d0160 2023-11-08T08:21:01.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.953+0000 7f70837fe700 1 -- 172.21.15.40:0/1055472970 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+11 (secure 0 0 0) 0x7f709002a0a0 con 0x7f709c0d0160 2023-11-08T08:21:01.958 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-11-08T08:21:01.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.957+0000 7f70a1199700 1 -- 172.21.15.40:0/1055472970 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7088064f10 msgr2=0x7f70880673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:01.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.957+0000 7f70a1199700 1 --2- 172.21.15.40:0/1055472970 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7088064f10 0x7f70880673c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f708c001560 tx=0x7f708c009040 comp rx=0 tx=0).stop 2023-11-08T08:21:01.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.957+0000 7f70a1199700 1 -- 172.21.15.40:0/1055472970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f709c0d0160 msgr2=0x7f709c151510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:01.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.957+0000 7f70a1199700 1 --2- 172.21.15.40:0/1055472970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f709c0d0160 0x7f709c151510 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7090013040 tx=0x7f7090004770 comp rx=0 tx=0).stop 2023-11-08T08:21:01.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.957+0000 7f70a1199700 1 -- 172.21.15.40:0/1055472970 shutdown_connections 2023-11-08T08:21:01.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.957+0000 7f70a1199700 1 --2- 172.21.15.40:0/1055472970 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7088064f10 0x7f70880673c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.957+0000 7f70a1199700 1 --2- 172.21.15.40:0/1055472970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f709c1567d0 0x7f709c15e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.957+0000 7f70a1199700 1 --2- 172.21.15.40:0/1055472970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f709c0d0160 0x7f709c151510 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.957+0000 7f70a1199700 1 --2- 172.21.15.40:0/1055472970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f709c0b2950 0x7f709c151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:01.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.957+0000 7f70a1199700 1 -- 172.21.15.40:0/1055472970 >> 172.21.15.40:0/1055472970 conn(0x7f709c0cd410 msgr2=0x7f709c0b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:01.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.957+0000 7f70a1199700 1 -- 172.21.15.40:0/1055472970 shutdown_connections 2023-11-08T08:21:01.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:01.957+0000 7f70a1199700 1 -- 172.21.15.40:0/1055472970 wait complete. 2023-11-08T08:21:01.985 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:21:02.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.173+0000 7f7fae584700 1 Processor -- start 2023-11-08T08:21:02.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.177+0000 7f7fae584700 1 -- start start 2023-11-08T08:21:02.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.177+0000 7f7fae584700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa815c640 0x7f7fa80d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:02.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.177+0000 7f7fae584700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7fa8071aa0 0x7f7fa806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:02.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.177+0000 7f7fae584700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa806c600 con 0x7f7fa8069e00 2023-11-08T08:21:02.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.177+0000 7f7fae584700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa806cef0 con 0x7f7fa8071aa0 2023-11-08T08:21:02.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.177+0000 7f7fae584700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa80722f0 con 0x7f7fa815c640 2023-11-08T08:21:02.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.177+0000 7f7fadd83700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7fa8069e00 0x7f7fa80d0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60168/0 (socket says 172.21.15.40:60168) 2023-11-08T08:21:02.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.177+0000 7f7fadd83700 1 -- 172.21.15.40:0/1619210630 learned_addr learned my addr 172.21.15.40:0/1619210630 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:02.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.177+0000 7f7facd81700 1 --2- 172.21.15.40:0/1619210630 >> v2:172.21.15.40:3300/0 conn(0x7f7fa8071aa0 0x7f7fa806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:02.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.181+0000 7f7fad582700 1 --2- 172.21.15.40:0/1619210630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa815c640 0x7f7fa80d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:02.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.181+0000 7f7fad582700 1 -- 172.21.15.40:0/1619210630 >> v2:172.21.15.40:3300/0 conn(0x7f7fa8071aa0 msgr2=0x7f7fa806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:02.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.181+0000 7f7fad582700 1 --2- 172.21.15.40:0/1619210630 >> v2:172.21.15.40:3300/0 conn(0x7f7fa8071aa0 0x7f7fa806d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:02.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.181+0000 7f7fad582700 1 -- 172.21.15.40:0/1619210630 >> v1:172.21.15.40:6789/0 conn(0x7f7fa8069e00 legacy=0x7f7fa80d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:02.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.181+0000 7f7fad582700 1 -- 172.21.15.40:0/1619210630 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa815ca10 con 0x7f7fa815c640 2023-11-08T08:21:02.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.181+0000 7f7f9ffff700 1 -- 172.21.15.40:0/1619210630 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3186833880 0 0) 0x7f7fa806c600 con 0x7f7fa8069e00 2023-11-08T08:21:02.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.181+0000 7f7fad582700 1 --2- 172.21.15.40:0/1619210630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa815c640 0x7f7fa80d31e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009850 tx=0x7f7fa4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb5ab5b0b3353799 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:02.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.181+0000 7f7f9ffff700 1 -- 172.21.15.40:0/1619210630 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4011830 con 0x7f7fa815c640 2023-11-08T08:21:02.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.181+0000 7f7f9ffff700 1 -- 172.21.15.40:0/1619210630 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa4011990 con 0x7f7fa815c640 2023-11-08T08:21:02.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.181+0000 7f7f9ffff700 1 -- 172.21.15.40:0/1619210630 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4011d40 con 0x7f7fa815c640 2023-11-08T08:21:02.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.181+0000 7f7fae584700 1 -- 172.21.15.40:0/1619210630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa815c640 msgr2=0x7f7fa80d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:02.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.181+0000 7f7fae584700 1 --2- 172.21.15.40:0/1619210630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa815c640 0x7f7fa80d31e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009850 tx=0x7f7fa4009b60 comp rx=0 tx=0).stop 2023-11-08T08:21:02.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.181+0000 7f7fae584700 1 -- 172.21.15.40:0/1619210630 shutdown_connections 2023-11-08T08:21:02.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.181+0000 7f7fae584700 1 --2- 172.21.15.40:0/1619210630 >> v2:172.21.15.40:3300/0 conn(0x7f7fa8071aa0 0x7f7fa806d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:02.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.181+0000 7f7fae584700 1 --2- 172.21.15.40:0/1619210630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa815c640 0x7f7fa80d31e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:02.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.181+0000 7f7fae584700 1 -- 172.21.15.40:0/1619210630 >> 172.21.15.40:0/1619210630 conn(0x7f7fa80b6520 msgr2=0x7f7fa8071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:02.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.185+0000 7f7fae584700 1 -- 172.21.15.40:0/1619210630 shutdown_connections 2023-11-08T08:21:02.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.185+0000 7f7fae584700 1 -- 172.21.15.40:0/1619210630 wait complete. 2023-11-08T08:21:02.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.185+0000 7f7fae584700 1 Processor -- start 2023-11-08T08:21:02.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7fae584700 1 -- start start 2023-11-08T08:21:02.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7fae584700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa8069e00 0x7f7fa80ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:02.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7fae584700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fa8071aa0 0x7f7fa80c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:02.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7fae584700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fa815c640 0x7f7fa80c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:02.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7fae584700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa815bd30 con 0x7f7fa8071aa0 2023-11-08T08:21:02.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7fae584700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa815a320 con 0x7f7fa815c640 2023-11-08T08:21:02.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7fae584700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa815a5e0 con 0x7f7fa8069e00 2023-11-08T08:21:02.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7facd81700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fa8071aa0 0x7f7fa80c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:02.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7facd81700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fa8071aa0 0x7f7fa80c7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42674/0 (socket says 172.21.15.40:42674) 2023-11-08T08:21:02.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7facd81700 1 -- 172.21.15.40:0/2634360099 learned_addr learned my addr 172.21.15.40:0/2634360099 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:02.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7fad582700 1 --2- 172.21.15.40:0/2634360099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa8069e00 0x7f7fa80ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:02.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7fadd83700 1 --2- 172.21.15.40:0/2634360099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fa815c640 0x7f7fa80c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:02.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7facd81700 1 -- 172.21.15.40:0/2634360099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa8069e00 msgr2=0x7f7fa80ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:02.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7facd81700 1 --2- 172.21.15.40:0/2634360099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa8069e00 0x7f7fa80ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:02.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7facd81700 1 -- 172.21.15.40:0/2634360099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fa815c640 msgr2=0x7f7fa80c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:02.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7facd81700 1 --2- 172.21.15.40:0/2634360099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fa815c640 0x7f7fa80c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:02.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7facd81700 1 -- 172.21.15.40:0/2634360099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa40094b0 con 0x7f7fa8071aa0 2023-11-08T08:21:02.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7facd81700 1 --2- 172.21.15.40:0/2634360099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fa8071aa0 0x7f7fa80c7fc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800e840 tx=0x7f7f9800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:02.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7f9dffb700 1 -- 172.21.15.40:0/2634360099 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f980095a0 con 0x7f7fa8071aa0 2023-11-08T08:21:02.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7f9dffb700 1 -- 172.21.15.40:0/2634360099 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f98013680 con 0x7f7fa8071aa0 2023-11-08T08:21:02.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7fae584700 1 -- 172.21.15.40:0/2634360099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa806c830 con 0x7f7fa8071aa0 2023-11-08T08:21:02.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.189+0000 7f7f9dffb700 1 -- 172.21.15.40:0/2634360099 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f980139f0 con 0x7f7fa8071aa0 2023-11-08T08:21:02.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.193+0000 7f7fae584700 1 -- 172.21.15.40:0/2634360099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa8072010 con 0x7f7fa8071aa0 2023-11-08T08:21:02.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.193+0000 7f7fae584700 1 -- 172.21.15.40:0/2634360099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa806cb10 con 0x7f7fa8071aa0 2023-11-08T08:21:02.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.197+0000 7f7f9dffb700 1 -- 172.21.15.40:0/2634360099 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7f98019030 con 0x7f7fa8071aa0 2023-11-08T08:21:02.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.197+0000 7f7f9dffb700 1 --2- 172.21.15.40:0/2634360099 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7f8c064f10 0x7f7f8c0673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:02.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.197+0000 7f7f9dffb700 1 -- 172.21.15.40:0/2634360099 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f7f98081e90 con 0x7f7fa8071aa0 2023-11-08T08:21:02.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.197+0000 7f7fad582700 1 --2- 172.21.15.40:0/2634360099 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7f8c064f10 0x7f7f8c0673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:02.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.201+0000 7f7fad582700 1 --2- 172.21.15.40:0/2634360099 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7f8c064f10 0x7f7f8c0673c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40097a0 tx=0x7f7fa4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:02.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.205+0000 7f7f9dffb700 1 -- 172.21.15.40:0/2634360099 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7f9804fdc0 con 0x7f7fa8071aa0 2023-11-08T08:21:02.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.416+0000 7f7fae584700 1 -- 172.21.15.40:0/2634360099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fa8073cb0 con 0x7f7fa8071aa0 2023-11-08T08:21:02.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.416+0000 7f7f9dffb700 1 -- 172.21.15.40:0/2634360099 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f7f980533e0 con 0x7f7fa8071aa0 2023-11-08T08:21:02.424 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:02.425 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4958,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2244078655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2244078655},{"type":"v1","addr":"172.21.15.40:6834","nonce":2244078655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/749537066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":749537066},{"type":"v1","addr":"172.21.15.169:6836","nonce":749537066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5328,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2307838686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2307838686},{"type":"v1","addr":"172.21.15.169:6837","nonce":2307838686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:20:34.294628+0000","modified":"2023-11-08T08:20:37.318141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/1791517264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1791517264},{"type":"v1","addr":"172.21.15.40:6835","nonce":1791517264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T08:21:02.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.420+0000 7f7fae584700 1 -- 172.21.15.40:0/2634360099 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7f8c064f10 msgr2=0x7f7f8c0673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:02.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.420+0000 7f7fae584700 1 --2- 172.21.15.40:0/2634360099 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7f8c064f10 0x7f7f8c0673c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40097a0 tx=0x7f7fa4008030 comp rx=0 tx=0).stop 2023-11-08T08:21:02.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.420+0000 7f7fae584700 1 -- 172.21.15.40:0/2634360099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fa8071aa0 msgr2=0x7f7fa80c7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:02.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.420+0000 7f7fae584700 1 --2- 172.21.15.40:0/2634360099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fa8071aa0 0x7f7fa80c7fc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800e840 tx=0x7f7f9800eb50 comp rx=0 tx=0).stop 2023-11-08T08:21:02.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.420+0000 7f7fae584700 1 -- 172.21.15.40:0/2634360099 shutdown_connections 2023-11-08T08:21:02.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.420+0000 7f7fae584700 1 --2- 172.21.15.40:0/2634360099 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7f8c064f10 0x7f7f8c0673c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:02.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.420+0000 7f7fae584700 1 --2- 172.21.15.40:0/2634360099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fa815c640 0x7f7fa80c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:02.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.420+0000 7f7fae584700 1 --2- 172.21.15.40:0/2634360099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fa8071aa0 0x7f7fa80c7fc0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:02.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.420+0000 7f7fae584700 1 --2- 172.21.15.40:0/2634360099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fa8069e00 0x7f7fa80ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:02.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.420+0000 7f7fae584700 1 -- 172.21.15.40:0/2634360099 >> 172.21.15.40:0/2634360099 conn(0x7f7fa80b6520 msgr2=0x7f7fa81e4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:02.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.420+0000 7f7fae584700 1 -- 172.21.15.40:0/2634360099 shutdown_connections 2023-11-08T08:21:02.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.420+0000 7f7fae584700 1 -- 172.21.15.40:0/2634360099 wait complete. 2023-11-08T08:21:02.430 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 17 2023-11-08T08:21:02.451 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:21:02.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.620+0000 7f79b2390700 1 Processor -- start 2023-11-08T08:21:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.624+0000 7f79b2390700 1 -- start start 2023-11-08T08:21:02.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.624+0000 7f79b2390700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79ac070080 0x7f79ac0704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:02.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.624+0000 7f79b2390700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79ac0ad6c0 con 0x7f79ac164250 2023-11-08T08:21:02.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.624+0000 7f79b2390700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79ac0adc60 con 0x7f79ac160710 2023-11-08T08:21:02.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.624+0000 7f79b2390700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ac06b1c0 con 0x7f79ac070080 2023-11-08T08:21:02.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.624+0000 7f79b0b8d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79ac070080 0x7f79ac0704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:02.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.624+0000 7f79b0b8d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79ac070080 0x7f79ac0704b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42686/0 (socket says 172.21.15.40:42686) 2023-11-08T08:21:02.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.624+0000 7f79b0b8d700 1 -- 172.21.15.40:0/129762809 learned_addr learned my addr 172.21.15.40:0/129762809 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:02.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.624+0000 7f79b0b8d700 1 -- 172.21.15.40:0/129762809 >> v1:172.21.15.169:6789/0 conn(0x7f79ac160710 legacy=0x7f79ac0cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:02.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.624+0000 7f79b0b8d700 1 -- 172.21.15.40:0/129762809 >> v1:172.21.15.40:6789/0 conn(0x7f79ac164250 legacy=0x7f79ac1646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:02.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.624+0000 7f79b0b8d700 1 -- 172.21.15.40:0/129762809 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ac0709f0 con 0x7f79ac070080 2023-11-08T08:21:02.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.628+0000 7f79b0b8d700 1 --2- 172.21.15.40:0/129762809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ac070080 0x7f79ac0704b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f799c009910 tx=0x7f799c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228d1107ac7e6968 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:02.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.628+0000 7f79a3fff700 1 -- 172.21.15.40:0/129762809 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c00ba60 con 0x7f79ac070080 2023-11-08T08:21:02.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.628+0000 7f79a3fff700 1 -- 172.21.15.40:0/129762809 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799c00bbc0 con 0x7f79ac070080 2023-11-08T08:21:02.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.628+0000 7f79a3fff700 1 -- 172.21.15.40:0/129762809 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c009e60 con 0x7f79ac070080 2023-11-08T08:21:02.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.628+0000 7f79b2390700 1 -- 172.21.15.40:0/129762809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ac070080 msgr2=0x7f79ac0704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:02.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.628+0000 7f79b2390700 1 --2- 172.21.15.40:0/129762809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ac070080 0x7f79ac0704b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f799c009910 tx=0x7f799c009c20 comp rx=0 tx=0).stop 2023-11-08T08:21:02.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.628+0000 7f79b2390700 1 -- 172.21.15.40:0/129762809 shutdown_connections 2023-11-08T08:21:02.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.628+0000 7f79b2390700 1 --2- 172.21.15.40:0/129762809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ac070080 0x7f79ac0704b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:02.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.628+0000 7f79b2390700 1 -- 172.21.15.40:0/129762809 >> 172.21.15.40:0/129762809 conn(0x7f79ac0b0f80 msgr2=0x7f79ac0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:02.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.628+0000 7f79b2390700 1 -- 172.21.15.40:0/129762809 shutdown_connections 2023-11-08T08:21:02.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.628+0000 7f79b2390700 1 -- 172.21.15.40:0/129762809 wait complete. 2023-11-08T08:21:02.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.632+0000 7f79b2390700 1 Processor -- start 2023-11-08T08:21:02.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.632+0000 7f79b2390700 1 -- start start 2023-11-08T08:21:02.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.632+0000 7f79b2390700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ac070080 0x7f79ac1614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:02.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.632+0000 7f79b2390700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ac160710 0x7f79ac15ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:02.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.632+0000 7f79b2390700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ac164250 0x7f79ac0c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:02.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.632+0000 7f79b2390700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac15e240 con 0x7f79ac160710 2023-11-08T08:21:02.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.632+0000 7f79b2390700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1f61b0 con 0x7f79ac070080 2023-11-08T08:21:02.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.632+0000 7f79b2390700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1620b0 con 0x7f79ac164250 2023-11-08T08:21:02.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.632+0000 7f79b0b8d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ac160710 0x7f79ac15ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:02.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.632+0000 7f79b0b8d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ac160710 0x7f79ac15ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42702/0 (socket says 172.21.15.40:42702) 2023-11-08T08:21:02.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.632+0000 7f79b0b8d700 1 -- 172.21.15.40:0/3322853711 learned_addr learned my addr 172.21.15.40:0/3322853711 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:02.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.632+0000 7f79b138e700 1 --2- 172.21.15.40:0/3322853711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ac070080 0x7f79ac1614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:02.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.632+0000 7f79b1b8f700 1 --2- 172.21.15.40:0/3322853711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ac164250 0x7f79ac0c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:02.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.632+0000 7f79b0b8d700 1 -- 172.21.15.40:0/3322853711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ac164250 msgr2=0x7f79ac0c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:02.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.632+0000 7f79b0b8d700 1 --2- 172.21.15.40:0/3322853711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ac164250 0x7f79ac0c39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:02.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.632+0000 7f79b0b8d700 1 -- 172.21.15.40:0/3322853711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ac070080 msgr2=0x7f79ac1614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:02.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.632+0000 7f79b0b8d700 1 --2- 172.21.15.40:0/3322853711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ac070080 0x7f79ac1614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:02.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.632+0000 7f79b0b8d700 1 -- 172.21.15.40:0/3322853711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c009570 con 0x7f79ac160710 2023-11-08T08:21:02.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.632+0000 7f79b138e700 1 --2- 172.21.15.40:0/3322853711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ac070080 0x7f79ac1614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:02.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.632+0000 7f79b0b8d700 1 --2- 172.21.15.40:0/3322853711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ac160710 0x7f79ac15ae40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f799c000c00 tx=0x7f799c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:02.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.636+0000 7f79a1ffb700 1 -- 172.21.15.40:0/3322853711 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c0045d0 con 0x7f79ac160710 2023-11-08T08:21:02.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.636+0000 7f79a1ffb700 1 -- 172.21.15.40:0/3322853711 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799c01f070 con 0x7f79ac160710 2023-11-08T08:21:02.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.636+0000 7f79b2390700 1 -- 172.21.15.40:0/3322853711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ac06b3d0 con 0x7f79ac160710 2023-11-08T08:21:02.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.636+0000 7f79a1ffb700 1 -- 172.21.15.40:0/3322853711 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c0038e0 con 0x7f79ac160710 2023-11-08T08:21:02.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.636+0000 7f79b2390700 1 -- 172.21.15.40:0/3322853711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ac0cca30 con 0x7f79ac160710 2023-11-08T08:21:02.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.636+0000 7f79b2390700 1 -- 172.21.15.40:0/3322853711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ac06b6b0 con 0x7f79ac160710 2023-11-08T08:21:02.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.640+0000 7f79a1ffb700 1 -- 172.21.15.40:0/3322853711 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f799c00f050 con 0x7f79ac160710 2023-11-08T08:21:02.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.640+0000 7f79a1ffb700 1 --2- 172.21.15.40:0/3322853711 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7990064f10 0x7f79900673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:02.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.640+0000 7f79a1ffb700 1 -- 172.21.15.40:0/3322853711 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f799c021070 con 0x7f79ac160710 2023-11-08T08:21:02.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.640+0000 7f79b138e700 1 --2- 172.21.15.40:0/3322853711 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7990064f10 0x7f79900673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:02.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.640+0000 7f79b138e700 1 --2- 172.21.15.40:0/3322853711 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7990064f10 0x7f79900673c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f79ac1600f0 tx=0x7f79a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:02.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.648+0000 7f79a1ffb700 1 -- 172.21.15.40:0/3322853711 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f799c051a20 con 0x7f79ac160710 2023-11-08T08:21:02.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.824+0000 7f79b2390700 1 -- 172.21.15.40:0/3322853711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79ac162220 con 0x7f79ac160710 2023-11-08T08:21:02.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.828+0000 7f79a1ffb700 1 -- 172.21.15.40:0/3322853711 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+18127 (secure 0 0 0) 0x7f799c015090 con 0x7f79ac160710 2023-11-08T08:21:02.833 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:02.833 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":45,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:20:45.994346+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:20:32.059150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-08T08:20:33.297064+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":4}]},{"pgid":"7.2e","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:21:02.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.828+0000 7f79b2390700 1 -- 172.21.15.40:0/3322853711 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7990064f10 msgr2=0x7f79900673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:02.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.828+0000 7f79b2390700 1 --2- 172.21.15.40:0/3322853711 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7990064f10 0x7f79900673c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f79ac1600f0 tx=0x7f79a8009040 comp rx=0 tx=0).stop 2023-11-08T08:21:02.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.828+0000 7f79b2390700 1 -- 172.21.15.40:0/3322853711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ac160710 msgr2=0x7f79ac15ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:02.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.828+0000 7f79b2390700 1 --2- 172.21.15.40:0/3322853711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ac160710 0x7f79ac15ae40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f799c000c00 tx=0x7f799c004770 comp rx=0 tx=0).stop 2023-11-08T08:21:02.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.828+0000 7f79b2390700 1 -- 172.21.15.40:0/3322853711 shutdown_connections 2023-11-08T08:21:02.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.828+0000 7f79b2390700 1 --2- 172.21.15.40:0/3322853711 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7990064f10 0x7f79900673c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:02.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.828+0000 7f79b2390700 1 --2- 172.21.15.40:0/3322853711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ac164250 0x7f79ac0c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:02.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.828+0000 7f79b2390700 1 --2- 172.21.15.40:0/3322853711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ac160710 0x7f79ac15ae40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:02.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.828+0000 7f79b2390700 1 --2- 172.21.15.40:0/3322853711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ac070080 0x7f79ac1614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:02.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.828+0000 7f79b2390700 1 -- 172.21.15.40:0/3322853711 >> 172.21.15.40:0/3322853711 conn(0x7f79ac0b0f80 msgr2=0x7f79ac0c4930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:02.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.828+0000 7f79b2390700 1 -- 172.21.15.40:0/3322853711 shutdown_connections 2023-11-08T08:21:02.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:02.828+0000 7f79b2390700 1 -- 172.21.15.40:0/3322853711 wait complete. 2023-11-08T08:21:02.859 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T08:21:02.859 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:21:03.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.040+0000 7efd3e848700 1 Processor -- start 2023-11-08T08:21:03.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.040+0000 7efd3e848700 1 -- start start 2023-11-08T08:21:03.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.044+0000 7efd3e848700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd38156820 0x7efd380b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:03.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.044+0000 7efd3e848700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd380d0670 con 0x7efd3815fe30 2023-11-08T08:21:03.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.044+0000 7efd3e848700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd380d0c30 con 0x7efd38071640 2023-11-08T08:21:03.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.044+0000 7efd3e848700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd38070b50 con 0x7efd38156820 2023-11-08T08:21:03.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.044+0000 7efd3e047700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efd3815fe30 0x7efd38069bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60176/0 (socket says 172.21.15.40:60176) 2023-11-08T08:21:03.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.044+0000 7efd3e047700 1 -- 172.21.15.40:0/1515505917 learned_addr learned my addr 172.21.15.40:0/1515505917 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:03.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.044+0000 7efd3d846700 1 --2- 172.21.15.40:0/1515505917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd38156820 0x7efd380b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:03.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.044+0000 7efd3c844700 1 -- 172.21.15.40:0/1515505917 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 244697398 0 0) 0x7efd380d0670 con 0x7efd3815fe30 2023-11-08T08:21:03.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.044+0000 7efd3c844700 1 -- 172.21.15.40:0/1515505917 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd240035d0 con 0x7efd3815fe30 2023-11-08T08:21:03.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.044+0000 7efd3d846700 1 -- 172.21.15.40:0/1515505917 >> v1:172.21.15.169:6789/0 conn(0x7efd38071640 legacy=0x7efd380d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:03.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.044+0000 7efd3c844700 1 -- 172.21.15.40:0/1515505917 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1080003432 0 0) 0x7efd380d0c30 con 0x7efd38071640 2023-11-08T08:21:03.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.044+0000 7efd3d846700 1 -- 172.21.15.40:0/1515505917 >> v1:172.21.15.40:6789/0 conn(0x7efd3815fe30 legacy=0x7efd38069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:03.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.044+0000 7efd3d846700 1 -- 172.21.15.40:0/1515505917 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd38070e30 con 0x7efd38156820 2023-11-08T08:21:03.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.048+0000 7efd3d846700 1 --2- 172.21.15.40:0/1515505917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd38156820 0x7efd380b1ee0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efd34009880 tx=0x7efd34009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d603a8e33c7ea847 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:03.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.048+0000 7efd3c844700 1 -- 172.21.15.40:0/1515505917 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34011830 con 0x7efd38156820 2023-11-08T08:21:03.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.048+0000 7efd3c844700 1 -- 172.21.15.40:0/1515505917 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd34011990 con 0x7efd38156820 2023-11-08T08:21:03.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.048+0000 7efd3c844700 1 -- 172.21.15.40:0/1515505917 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34011d20 con 0x7efd38156820 2023-11-08T08:21:03.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.048+0000 7efd3e848700 1 -- 172.21.15.40:0/1515505917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd38156820 msgr2=0x7efd380b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:03.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.048+0000 7efd3e848700 1 --2- 172.21.15.40:0/1515505917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd38156820 0x7efd380b1ee0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efd34009880 tx=0x7efd34009b90 comp rx=0 tx=0).stop 2023-11-08T08:21:03.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.048+0000 7efd3e848700 1 -- 172.21.15.40:0/1515505917 shutdown_connections 2023-11-08T08:21:03.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.048+0000 7efd3e848700 1 --2- 172.21.15.40:0/1515505917 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd38156820 0x7efd380b1ee0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:03.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.048+0000 7efd3e848700 1 -- 172.21.15.40:0/1515505917 >> 172.21.15.40:0/1515505917 conn(0x7efd3806cb80 msgr2=0x7efd380b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:03.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.048+0000 7efd3e848700 1 -- 172.21.15.40:0/1515505917 shutdown_connections 2023-11-08T08:21:03.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.048+0000 7efd3e848700 1 -- 172.21.15.40:0/1515505917 wait complete. 2023-11-08T08:21:03.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.052+0000 7efd3e848700 1 Processor -- start 2023-11-08T08:21:03.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.052+0000 7efd3e848700 1 -- start start 2023-11-08T08:21:03.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd3e848700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd38071640 0x7efd380d3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:03.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd3e848700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd38156820 0x7efd3815e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:03.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd3e848700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd3815fe30 0x7efd380cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:03.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd3e848700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3815e860 con 0x7efd3815fe30 2023-11-08T08:21:03.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd3e848700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd380cff60 con 0x7efd38071640 2023-11-08T08:21:03.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd3e848700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd380b7960 con 0x7efd38156820 2023-11-08T08:21:03.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd3e047700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd3815fe30 0x7efd380cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:03.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd3e047700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd3815fe30 0x7efd380cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42704/0 (socket says 172.21.15.40:42704) 2023-11-08T08:21:03.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd3e047700 1 -- 172.21.15.40:0/937265630 learned_addr learned my addr 172.21.15.40:0/937265630 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:03.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd3d045700 1 --2- 172.21.15.40:0/937265630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd38156820 0x7efd3815e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:03.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd3d846700 1 --2- 172.21.15.40:0/937265630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd38071640 0x7efd380d3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:03.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd3e047700 1 -- 172.21.15.40:0/937265630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd38156820 msgr2=0x7efd3815e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:03.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd3e047700 1 --2- 172.21.15.40:0/937265630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd38156820 0x7efd3815e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:03.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd3e047700 1 -- 172.21.15.40:0/937265630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd38071640 msgr2=0x7efd380d3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:03.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd3e047700 1 --2- 172.21.15.40:0/937265630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd38071640 0x7efd380d3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:03.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd3e047700 1 -- 172.21.15.40:0/937265630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd340094b0 con 0x7efd3815fe30 2023-11-08T08:21:03.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd3e047700 1 --2- 172.21.15.40:0/937265630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd3815fe30 0x7efd380cfa20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efd3000cb40 tx=0x7efd3000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:03.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd2e7fc700 1 -- 172.21.15.40:0/937265630 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30007500 con 0x7efd3815fe30 2023-11-08T08:21:03.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd3e848700 1 -- 172.21.15.40:0/937265630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd380b2420 con 0x7efd3815fe30 2023-11-08T08:21:03.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd2e7fc700 1 -- 172.21.15.40:0/937265630 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd30007660 con 0x7efd3815fe30 2023-11-08T08:21:03.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd2e7fc700 1 -- 172.21.15.40:0/937265630 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd300079f0 con 0x7efd3815fe30 2023-11-08T08:21:03.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.056+0000 7efd3e848700 1 -- 172.21.15.40:0/937265630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd380d08c0 con 0x7efd3815fe30 2023-11-08T08:21:03.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.060+0000 7efd3e848700 1 -- 172.21.15.40:0/937265630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd38156bf0 con 0x7efd3815fe30 2023-11-08T08:21:03.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.060+0000 7efd2e7fc700 1 -- 172.21.15.40:0/937265630 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7efd30019030 con 0x7efd3815fe30 2023-11-08T08:21:03.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.060+0000 7efd2e7fc700 1 --2- 172.21.15.40:0/937265630 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efd1c064ee0 0x7efd1c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:03.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.060+0000 7efd2e7fc700 1 -- 172.21.15.40:0/937265630 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7efd30081a10 con 0x7efd3815fe30 2023-11-08T08:21:03.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.064+0000 7efd3d846700 1 --2- 172.21.15.40:0/937265630 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efd1c064ee0 0x7efd1c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:03.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.064+0000 7efd3d846700 1 --2- 172.21.15.40:0/937265630 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efd1c064ee0 0x7efd1c067390 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efd340090b0 tx=0x7efd3401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:03.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.068+0000 7efd2e7fc700 1 -- 172.21.15.40:0/937265630 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7efd3004f8c0 con 0x7efd3815fe30 2023-11-08T08:21:03.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.276+0000 7efd3e848700 1 -- 172.21.15.40:0/937265630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd38073cb0 con 0x7efd3815fe30 2023-11-08T08:21:03.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.280+0000 7efd2e7fc700 1 -- 172.21.15.40:0/937265630 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4651 (secure 0 0 0) 0x7efd30052ee0 con 0x7efd3815fe30 2023-11-08T08:21:03.285 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:03.285 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4958,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2244078655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2244078655},{"type":"v1","addr":"172.21.15.40:6834","nonce":2244078655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/749537066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":749537066},{"type":"v1","addr":"172.21.15.169:6836","nonce":749537066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5328,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2307838686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2307838686},{"type":"v1","addr":"172.21.15.169:6837","nonce":2307838686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:20:34.294628+0000","modified":"2023-11-08T08:20:37.318141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/1791517264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1791517264},{"type":"v1","addr":"172.21.15.40:6835","nonce":1791517264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T08:21:03.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.280+0000 7efd3e848700 1 -- 172.21.15.40:0/937265630 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efd1c064ee0 msgr2=0x7efd1c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.280+0000 7efd3e848700 1 --2- 172.21.15.40:0/937265630 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efd1c064ee0 0x7efd1c067390 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efd340090b0 tx=0x7efd3401c040 comp rx=0 tx=0).stop 2023-11-08T08:21:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.280+0000 7efd3e848700 1 -- 172.21.15.40:0/937265630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd3815fe30 msgr2=0x7efd380cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.280+0000 7efd3e848700 1 --2- 172.21.15.40:0/937265630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd3815fe30 0x7efd380cfa20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efd3000cb40 tx=0x7efd3000cb70 comp rx=0 tx=0).stop 2023-11-08T08:21:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.280+0000 7efd3e848700 1 -- 172.21.15.40:0/937265630 shutdown_connections 2023-11-08T08:21:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.280+0000 7efd3e848700 1 --2- 172.21.15.40:0/937265630 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efd1c064ee0 0x7efd1c067390 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.280+0000 7efd3e848700 1 --2- 172.21.15.40:0/937265630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd3815fe30 0x7efd380cfa20 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.280+0000 7efd3e848700 1 --2- 172.21.15.40:0/937265630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd38156820 0x7efd3815e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.280+0000 7efd3e848700 1 --2- 172.21.15.40:0/937265630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd38071640 0x7efd380d3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.280+0000 7efd3e848700 1 -- 172.21.15.40:0/937265630 >> 172.21.15.40:0/937265630 conn(0x7efd3806cb80 msgr2=0x7efd38151530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.280+0000 7efd3e848700 1 -- 172.21.15.40:0/937265630 shutdown_connections 2023-11-08T08:21:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.280+0000 7efd3e848700 1 -- 172.21.15.40:0/937265630 wait complete. 2023-11-08T08:21:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 17 2023-11-08T08:21:03.309 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:21:03.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.484+0000 7fa3ea970700 1 Processor -- start 2023-11-08T08:21:03.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.484+0000 7fa3ea970700 1 -- start start 2023-11-08T08:21:03.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.484+0000 7fa3ea970700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3e415c640 0x7fa3e40d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:03.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.484+0000 7fa3ea970700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa3e4071aa0 0x7fa3e406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:03.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.484+0000 7fa3ea970700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e406c600 con 0x7fa3e4069e00 2023-11-08T08:21:03.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.484+0000 7fa3ea970700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e406cef0 con 0x7fa3e4071aa0 2023-11-08T08:21:03.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.484+0000 7fa3ea970700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e40722f0 con 0x7fa3e415c640 2023-11-08T08:21:03.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.488+0000 7fa3e916d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa3e4071aa0 0x7fa3e406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:03.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.488+0000 7fa3e916d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa3e4071aa0 0x7fa3e406d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:58924/0 (socket says 172.21.15.40:58924) 2023-11-08T08:21:03.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.488+0000 7fa3e916d700 1 -- 172.21.15.40:0/1791228278 learned_addr learned my addr 172.21.15.40:0/1791228278 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:03.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.488+0000 7fa3e996e700 1 --2- 172.21.15.40:0/1791228278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3e415c640 0x7fa3e40d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:03.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.488+0000 7fa3e916d700 1 -- 172.21.15.40:0/1791228278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3e415c640 msgr2=0x7fa3e40d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:03.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.488+0000 7fa3e916d700 1 --2- 172.21.15.40:0/1791228278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3e415c640 0x7fa3e40d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:03.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.488+0000 7fa3e916d700 1 -- 172.21.15.40:0/1791228278 >> v1:172.21.15.169:6789/0 conn(0x7fa3e4069e00 legacy=0x7fa3e40d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:03.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.488+0000 7fa3e916d700 1 -- 172.21.15.40:0/1791228278 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e415ca10 con 0x7fa3e4071aa0 2023-11-08T08:21:03.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.488+0000 7fa3e996e700 1 --2- 172.21.15.40:0/1791228278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3e415c640 0x7fa3e40d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:03.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.488+0000 7fa3e916d700 1 --2- 172.21.15.40:0/1791228278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e4071aa0 0x7fa3e406d310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009910 tx=0x7fa3d4009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b17cbfbe24306a2b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:03.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.488+0000 7fa3e896c700 1 -- 172.21.15.40:0/1791228278 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d400ba60 con 0x7fa3e4071aa0 2023-11-08T08:21:03.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.488+0000 7fa3e896c700 1 -- 172.21.15.40:0/1791228278 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d400bbc0 con 0x7fa3e4071aa0 2023-11-08T08:21:03.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.488+0000 7fa3e896c700 1 -- 172.21.15.40:0/1791228278 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4009e60 con 0x7fa3e4071aa0 2023-11-08T08:21:03.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.488+0000 7fa3ea970700 1 -- 172.21.15.40:0/1791228278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e4071aa0 msgr2=0x7fa3e406d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:03.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.488+0000 7fa3ea970700 1 --2- 172.21.15.40:0/1791228278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e4071aa0 0x7fa3e406d310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009910 tx=0x7fa3d4009c20 comp rx=0 tx=0).stop 2023-11-08T08:21:03.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.488+0000 7fa3ea970700 1 -- 172.21.15.40:0/1791228278 shutdown_connections 2023-11-08T08:21:03.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.488+0000 7fa3ea970700 1 --2- 172.21.15.40:0/1791228278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e4071aa0 0x7fa3e406d310 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:03.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.488+0000 7fa3ea970700 1 --2- 172.21.15.40:0/1791228278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3e415c640 0x7fa3e40d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:03.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.488+0000 7fa3ea970700 1 -- 172.21.15.40:0/1791228278 >> 172.21.15.40:0/1791228278 conn(0x7fa3e40b6520 msgr2=0x7fa3e4071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:03.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.492+0000 7fa3ea970700 1 -- 172.21.15.40:0/1791228278 shutdown_connections 2023-11-08T08:21:03.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.492+0000 7fa3ea970700 1 -- 172.21.15.40:0/1791228278 wait complete. 2023-11-08T08:21:03.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.492+0000 7fa3ea970700 1 Processor -- start 2023-11-08T08:21:03.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.492+0000 7fa3ea970700 1 -- start start 2023-11-08T08:21:03.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.496+0000 7fa3ea970700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3e4069e00 0x7fa3e40cf780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:03.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.496+0000 7fa3ea970700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e4071aa0 0x7fa3e40cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:03.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.496+0000 7fa3ea970700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e415c640 0x7fa3e41f1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:03.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.496+0000 7fa3ea970700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e406a620 con 0x7fa3e415c640 2023-11-08T08:21:03.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.496+0000 7fa3ea970700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e40cbc10 con 0x7fa3e4071aa0 2023-11-08T08:21:03.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.496+0000 7fa3ea970700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e40cb240 con 0x7fa3e4069e00 2023-11-08T08:21:03.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.496+0000 7fa3ea16f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e415c640 0x7fa3e41f1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:03.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.496+0000 7fa3ea16f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e415c640 0x7fa3e41f1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42716/0 (socket says 172.21.15.40:42716) 2023-11-08T08:21:03.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.496+0000 7fa3ea16f700 1 -- 172.21.15.40:0/3986220527 learned_addr learned my addr 172.21.15.40:0/3986220527 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:03.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.496+0000 7fa3e996e700 1 --2- 172.21.15.40:0/3986220527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3e4069e00 0x7fa3e40cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:03.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.496+0000 7fa3e916d700 1 --2- 172.21.15.40:0/3986220527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e4071aa0 0x7fa3e40cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:03.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.496+0000 7fa3ea16f700 1 -- 172.21.15.40:0/3986220527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3e4069e00 msgr2=0x7fa3e40cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:03.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.496+0000 7fa3ea16f700 1 --2- 172.21.15.40:0/3986220527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3e4069e00 0x7fa3e40cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:03.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.496+0000 7fa3ea16f700 1 -- 172.21.15.40:0/3986220527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e4071aa0 msgr2=0x7fa3e40cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:03.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.496+0000 7fa3ea16f700 1 --2- 172.21.15.40:0/3986220527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e4071aa0 0x7fa3e40cb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:03.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.496+0000 7fa3ea16f700 1 -- 172.21.15.40:0/3986220527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d4009570 con 0x7fa3e415c640 2023-11-08T08:21:03.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.500+0000 7fa3ea16f700 1 --2- 172.21.15.40:0/3986220527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e415c640 0x7fa3e41f1310 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00cb80 tx=0x7fa3dc00ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:03.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.500+0000 7fa3da7fc700 1 -- 172.21.15.40:0/3986220527 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc003c20 con 0x7fa3e415c640 2023-11-08T08:21:03.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.500+0000 7fa3da7fc700 1 -- 172.21.15.40:0/3986220527 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3dc003d80 con 0x7fa3e415c640 2023-11-08T08:21:03.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.500+0000 7fa3ea970700 1 -- 172.21.15.40:0/3986220527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e406c830 con 0x7fa3e415c640 2023-11-08T08:21:03.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.500+0000 7fa3da7fc700 1 -- 172.21.15.40:0/3986220527 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc00aaa0 con 0x7fa3e415c640 2023-11-08T08:21:03.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.500+0000 7fa3ea970700 1 -- 172.21.15.40:0/3986220527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e4072010 con 0x7fa3e415c640 2023-11-08T08:21:03.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.500+0000 7fa3ea970700 1 -- 172.21.15.40:0/3986220527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e406cb10 con 0x7fa3e415c640 2023-11-08T08:21:03.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.504+0000 7fa3da7fc700 1 -- 172.21.15.40:0/3986220527 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa3dc010050 con 0x7fa3e415c640 2023-11-08T08:21:03.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.504+0000 7fa3da7fc700 1 --2- 172.21.15.40:0/3986220527 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa3c8064f10 0x7fa3c80673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:03.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.504+0000 7fa3da7fc700 1 -- 172.21.15.40:0/3986220527 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fa3dc081de0 con 0x7fa3e415c640 2023-11-08T08:21:03.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.504+0000 7fa3e996e700 1 --2- 172.21.15.40:0/3986220527 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa3c8064f10 0x7fa3c80673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:03.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.504+0000 7fa3e996e700 1 --2- 172.21.15.40:0/3986220527 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa3c8064f10 0x7fa3c80673c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00099f0 tx=0x7fa3e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:03.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.508+0000 7fa3da7fc700 1 -- 172.21.15.40:0/3986220527 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa3dc04fe00 con 0x7fa3e415c640 2023-11-08T08:21:03.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.716+0000 7fa3ea970700 1 -- 172.21.15.40:0/3986220527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3e40cb3f0 con 0x7fa3e415c640 2023-11-08T08:21:03.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.716+0000 7fa3da7fc700 1 -- 172.21.15.40:0/3986220527 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4651 (secure 0 0 0) 0x7fa3e40cb3f0 con 0x7fa3e415c640 2023-11-08T08:21:03.723 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:03.723 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4958,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2244078655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2244078655},{"type":"v1","addr":"172.21.15.40:6834","nonce":2244078655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/749537066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":749537066},{"type":"v1","addr":"172.21.15.169:6836","nonce":749537066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5328,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2307838686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2307838686},{"type":"v1","addr":"172.21.15.169:6837","nonce":2307838686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:20:34.294628+0000","modified":"2023-11-08T08:20:37.318141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/1791517264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1791517264},{"type":"v1","addr":"172.21.15.40:6835","nonce":1791517264}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T08:21:03.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.720+0000 7fa3ea970700 1 -- 172.21.15.40:0/3986220527 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa3c8064f10 msgr2=0x7fa3c80673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:03.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.720+0000 7fa3ea970700 1 --2- 172.21.15.40:0/3986220527 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa3c8064f10 0x7fa3c80673c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00099f0 tx=0x7fa3e0008040 comp rx=0 tx=0).stop 2023-11-08T08:21:03.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.720+0000 7fa3ea970700 1 -- 172.21.15.40:0/3986220527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e415c640 msgr2=0x7fa3e41f1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:03.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.720+0000 7fa3ea970700 1 --2- 172.21.15.40:0/3986220527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e415c640 0x7fa3e41f1310 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00cb80 tx=0x7fa3dc00ce90 comp rx=0 tx=0).stop 2023-11-08T08:21:03.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.720+0000 7fa3ea970700 1 -- 172.21.15.40:0/3986220527 shutdown_connections 2023-11-08T08:21:03.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.720+0000 7fa3ea970700 1 --2- 172.21.15.40:0/3986220527 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa3c8064f10 0x7fa3c80673c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:03.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.720+0000 7fa3ea970700 1 --2- 172.21.15.40:0/3986220527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3e415c640 0x7fa3e41f1310 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:03.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.720+0000 7fa3ea970700 1 --2- 172.21.15.40:0/3986220527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e4071aa0 0x7fa3e40cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.720+0000 7fa3ea970700 1 --2- 172.21.15.40:0/3986220527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3e4069e00 0x7fa3e40cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.720+0000 7fa3ea970700 1 -- 172.21.15.40:0/3986220527 >> 172.21.15.40:0/3986220527 conn(0x7fa3e40b6520 msgr2=0x7fa3e40afd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.720+0000 7fa3ea970700 1 -- 172.21.15.40:0/3986220527 shutdown_connections 2023-11-08T08:21:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.720+0000 7fa3ea970700 1 -- 172.21.15.40:0/3986220527 wait complete. 2023-11-08T08:21:03.729 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 17 2023-11-08T08:21:03.747 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:21:03.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.920+0000 7f30653ef700 1 Processor -- start 2023-11-08T08:21:03.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.920+0000 7f30653ef700 1 -- start start 2023-11-08T08:21:03.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.920+0000 7f30653ef700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f30600b43a0 0x7f306015e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:03.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.920+0000 7f30653ef700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306015a550 con 0x7f30600b1640 2023-11-08T08:21:03.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.920+0000 7f30653ef700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306006b0a0 con 0x7f3060070790 2023-11-08T08:21:03.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.920+0000 7f30653ef700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30600caac0 con 0x7f30600b43a0 2023-11-08T08:21:03.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.924+0000 7f30577fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f30600b1640 0x7f3060070080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60180/0 (socket says 172.21.15.40:60180) 2023-11-08T08:21:03.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.924+0000 7f30577fe700 1 -- 172.21.15.40:0/492762124 learned_addr learned my addr 172.21.15.40:0/492762124 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:03.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.924+0000 7f305ffff700 1 --2- 172.21.15.40:0/492762124 >> v2:172.21.15.169:3300/0 conn(0x7f30600b43a0 0x7f306015e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:03.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.924+0000 7f305ffff700 1 -- 172.21.15.40:0/492762124 >> v1:172.21.15.169:6789/0 conn(0x7f3060070790 legacy=0x7f306006bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:03.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.924+0000 7f305ffff700 1 -- 172.21.15.40:0/492762124 >> v1:172.21.15.40:6789/0 conn(0x7f30600b1640 legacy=0x7f3060070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:03.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.924+0000 7f305ffff700 1 -- 172.21.15.40:0/492762124 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30600cada0 con 0x7f30600b43a0 2023-11-08T08:21:03.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.924+0000 7f305ffff700 1 --2- 172.21.15.40:0/492762124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30600b43a0 0x7f306015e5d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3050009850 tx=0x7f3050009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb260656ee401d7d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:03.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.924+0000 7f305f7fe700 1 -- 172.21.15.40:0/492762124 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050011830 con 0x7f30600b43a0 2023-11-08T08:21:03.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.924+0000 7f305f7fe700 1 -- 172.21.15.40:0/492762124 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3050011990 con 0x7f30600b43a0 2023-11-08T08:21:03.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.924+0000 7f305f7fe700 1 -- 172.21.15.40:0/492762124 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050011d20 con 0x7f30600b43a0 2023-11-08T08:21:03.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.924+0000 7f30653ef700 1 -- 172.21.15.40:0/492762124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30600b43a0 msgr2=0x7f306015e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:03.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.924+0000 7f30653ef700 1 --2- 172.21.15.40:0/492762124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30600b43a0 0x7f306015e5d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3050009850 tx=0x7f3050009b60 comp rx=0 tx=0).stop 2023-11-08T08:21:03.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.924+0000 7f30653ef700 1 -- 172.21.15.40:0/492762124 shutdown_connections 2023-11-08T08:21:03.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.924+0000 7f30653ef700 1 --2- 172.21.15.40:0/492762124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30600b43a0 0x7f306015e5d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:03.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.924+0000 7f30653ef700 1 -- 172.21.15.40:0/492762124 >> 172.21.15.40:0/492762124 conn(0x7f30600b0f80 msgr2=0x7f30600ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:03.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.924+0000 7f30653ef700 1 -- 172.21.15.40:0/492762124 shutdown_connections 2023-11-08T08:21:03.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.924+0000 7f30653ef700 1 -- 172.21.15.40:0/492762124 wait complete. 2023-11-08T08:21:03.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.928+0000 7f30653ef700 1 Processor -- start 2023-11-08T08:21:03.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.928+0000 7f30653ef700 1 -- start start 2023-11-08T08:21:03.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f30653ef700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3060070790 0x7f30601f2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:03.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f30653ef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30600b1640 0x7f30601f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:03.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f30653ef700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30600b43a0 0x7f3060158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:03.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f30653ef700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30601f2890 con 0x7f30600b1640 2023-11-08T08:21:03.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f30653ef700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306015bbf0 con 0x7f30600b43a0 2023-11-08T08:21:03.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f30653ef700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306015bed0 con 0x7f3060070790 2023-11-08T08:21:03.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f30577fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30600b1640 0x7f30601f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:03.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f30577fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30600b1640 0x7f30601f2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42724/0 (socket says 172.21.15.40:42724) 2023-11-08T08:21:03.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f30577fe700 1 -- 172.21.15.40:0/4207603962 learned_addr learned my addr 172.21.15.40:0/4207603962 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:03.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f305ffff700 1 --2- 172.21.15.40:0/4207603962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3060070790 0x7f30601f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:03.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f3064bee700 1 --2- 172.21.15.40:0/4207603962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30600b43a0 0x7f3060158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:03.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f30577fe700 1 -- 172.21.15.40:0/4207603962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3060070790 msgr2=0x7f30601f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:03.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f30577fe700 1 --2- 172.21.15.40:0/4207603962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3060070790 0x7f30601f2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:03.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f30577fe700 1 -- 172.21.15.40:0/4207603962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30600b43a0 msgr2=0x7f3060158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:03.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f30577fe700 1 --2- 172.21.15.40:0/4207603962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30600b43a0 0x7f3060158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:03.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f30577fe700 1 -- 172.21.15.40:0/4207603962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30500094b0 con 0x7f30600b1640 2023-11-08T08:21:03.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f3064bee700 1 --2- 172.21.15.40:0/4207603962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30600b43a0 0x7f3060158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:03.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f30577fe700 1 --2- 172.21.15.40:0/4207603962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30600b1640 0x7f30601f2a70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f304800cba0 tx=0x7f304800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:03.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f305d7fa700 1 -- 172.21.15.40:0/4207603962 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304800a7f0 con 0x7f30600b1640 2023-11-08T08:21:03.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f305d7fa700 1 -- 172.21.15.40:0/4207603962 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f304800a950 con 0x7f30600b1640 2023-11-08T08:21:03.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f30653ef700 1 -- 172.21.15.40:0/4207603962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306006b5d0 con 0x7f30600b1640 2023-11-08T08:21:03.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f305d7fa700 1 -- 172.21.15.40:0/4207603962 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304800acc0 con 0x7f30600b1640 2023-11-08T08:21:03.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.932+0000 7f30653ef700 1 -- 172.21.15.40:0/4207603962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306006b2f0 con 0x7f30600b1640 2023-11-08T08:21:03.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.936+0000 7f30653ef700 1 -- 172.21.15.40:0/4207603962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306015eb10 con 0x7f30600b1640 2023-11-08T08:21:03.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.936+0000 7f305d7fa700 1 -- 172.21.15.40:0/4207603962 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f3048019030 con 0x7f30600b1640 2023-11-08T08:21:03.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.940+0000 7f305d7fa700 1 --2- 172.21.15.40:0/4207603962 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3040064ee0 0x7f3040067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:03.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.940+0000 7f305d7fa700 1 -- 172.21.15.40:0/4207603962 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f30480816e0 con 0x7f30600b1640 2023-11-08T08:21:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.940+0000 7f305ffff700 1 --2- 172.21.15.40:0/4207603962 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3040064ee0 0x7f3040067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:03.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.940+0000 7f305ffff700 1 --2- 172.21.15.40:0/4207603962 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3040064ee0 0x7f3040067390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3050014040 tx=0x7f305001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:03.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:03.948+0000 7f305d7fa700 1 -- 172.21.15.40:0/4207603962 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f304804a400 con 0x7f30600b1640 2023-11-08T08:21:04.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.128+0000 7f30653ef700 1 -- 172.21.15.40:0/4207603962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f306015a550 con 0x7f30600b1640 2023-11-08T08:21:04.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.132+0000 7f305d7fa700 1 -- 172.21.15.40:0/4207603962 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+18127 (secure 0 0 0) 0x7f306015a550 con 0x7f30600b1640 2023-11-08T08:21:04.139 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:04.139 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":45,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:20:45.994346+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:20:32.059150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-08T08:20:33.297064+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":4}]},{"pgid":"7.2e","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:21:04.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.136+0000 7f30653ef700 1 -- 172.21.15.40:0/4207603962 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3040064ee0 msgr2=0x7f3040067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:04.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.136+0000 7f30653ef700 1 --2- 172.21.15.40:0/4207603962 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3040064ee0 0x7f3040067390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3050014040 tx=0x7f305001c040 comp rx=0 tx=0).stop 2023-11-08T08:21:04.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.136+0000 7f30653ef700 1 -- 172.21.15.40:0/4207603962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30600b1640 msgr2=0x7f30601f2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:04.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.136+0000 7f30653ef700 1 --2- 172.21.15.40:0/4207603962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30600b1640 0x7f30601f2a70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f304800cba0 tx=0x7f304800cbd0 comp rx=0 tx=0).stop 2023-11-08T08:21:04.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.136+0000 7f30653ef700 1 -- 172.21.15.40:0/4207603962 shutdown_connections 2023-11-08T08:21:04.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.136+0000 7f30653ef700 1 --2- 172.21.15.40:0/4207603962 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3040064ee0 0x7f3040067390 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:04.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.136+0000 7f30653ef700 1 --2- 172.21.15.40:0/4207603962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30600b43a0 0x7f3060158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:04.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.136+0000 7f30653ef700 1 --2- 172.21.15.40:0/4207603962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30600b1640 0x7f30601f2a70 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:04.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.136+0000 7f30653ef700 1 --2- 172.21.15.40:0/4207603962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3060070790 0x7f30601f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:04.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.136+0000 7f30653ef700 1 -- 172.21.15.40:0/4207603962 >> 172.21.15.40:0/4207603962 conn(0x7f30600b0f80 msgr2=0x7f3060069bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:04.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.140+0000 7f30653ef700 1 -- 172.21.15.40:0/4207603962 shutdown_connections 2023-11-08T08:21:04.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.140+0000 7f30653ef700 1 -- 172.21.15.40:0/4207603962 wait complete. 2023-11-08T08:21:04.169 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T08:21:04.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.340+0000 7fd3e8726700 1 Processor -- start 2023-11-08T08:21:04.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.340+0000 7fd3e8726700 1 -- start start 2023-11-08T08:21:04.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.344+0000 7fd3e8726700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd3e01587b0 0x7fd3e006cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:04.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.344+0000 7fd3e8726700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3e006ac00 0x7fd3e006b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:04.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.344+0000 7fd3e8726700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3e006b540 con 0x7fd3e006d010 2023-11-08T08:21:04.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.344+0000 7fd3e8726700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3e00adfe0 con 0x7fd3e01587b0 2023-11-08T08:21:04.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.344+0000 7fd3e8726700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e0158b80 con 0x7fd3e006ac00 2023-11-08T08:21:04.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.344+0000 7fd3e64c2700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd3e01587b0 0x7fd3e006cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:04.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.344+0000 7fd3e64c2700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd3e01587b0 0x7fd3e006cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42730/0 (socket says 172.21.15.40:42730) 2023-11-08T08:21:04.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.344+0000 7fd3e64c2700 1 -- 172.21.15.40:0/3665102416 learned_addr learned my addr 172.21.15.40:0/3665102416 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:04.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.348+0000 7fd3e6cc3700 1 --2- 172.21.15.40:0/3665102416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3e006ac00 0x7fd3e006b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:04.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.348+0000 7fd3e64c2700 1 -- 172.21.15.40:0/3665102416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3e006ac00 msgr2=0x7fd3e006b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:04.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.348+0000 7fd3e64c2700 1 --2- 172.21.15.40:0/3665102416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3e006ac00 0x7fd3e006b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:04.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.348+0000 7fd3e64c2700 1 -- 172.21.15.40:0/3665102416 >> v1:172.21.15.169:6789/0 conn(0x7fd3e006d010 legacy=0x7fd3e00c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:04.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.348+0000 7fd3e64c2700 1 -- 172.21.15.40:0/3665102416 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e006d520 con 0x7fd3e01587b0 2023-11-08T08:21:04.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.348+0000 7fd3e64c2700 1 --2- 172.21.15.40:0/3665102416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e01587b0 0x7fd3e006cad0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009910 tx=0x7fd3dc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27206f40e8522b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:04.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.348+0000 7fd3e54c0700 1 -- 172.21.15.40:0/3665102416 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc00ba60 con 0x7fd3e01587b0 2023-11-08T08:21:04.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.348+0000 7fd3e54c0700 1 -- 172.21.15.40:0/3665102416 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc00bbc0 con 0x7fd3e01587b0 2023-11-08T08:21:04.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.348+0000 7fd3e54c0700 1 -- 172.21.15.40:0/3665102416 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc009e60 con 0x7fd3e01587b0 2023-11-08T08:21:04.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.348+0000 7fd3e8726700 1 -- 172.21.15.40:0/3665102416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e01587b0 msgr2=0x7fd3e006cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:04.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.348+0000 7fd3e8726700 1 --2- 172.21.15.40:0/3665102416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e01587b0 0x7fd3e006cad0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009910 tx=0x7fd3dc009c20 comp rx=0 tx=0).stop 2023-11-08T08:21:04.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.348+0000 7fd3e8726700 1 -- 172.21.15.40:0/3665102416 shutdown_connections 2023-11-08T08:21:04.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.348+0000 7fd3e8726700 1 --2- 172.21.15.40:0/3665102416 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3e006ac00 0x7fd3e006b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:04.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.348+0000 7fd3e8726700 1 --2- 172.21.15.40:0/3665102416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e01587b0 0x7fd3e006cad0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:04.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.348+0000 7fd3e8726700 1 -- 172.21.15.40:0/3665102416 >> 172.21.15.40:0/3665102416 conn(0x7fd3e00cf350 msgr2=0x7fd3e00b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:04.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.348+0000 7fd3e8726700 1 -- 172.21.15.40:0/3665102416 shutdown_connections 2023-11-08T08:21:04.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.348+0000 7fd3e8726700 1 -- 172.21.15.40:0/3665102416 wait complete. 2023-11-08T08:21:04.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.352+0000 7fd3e8726700 1 Processor -- start 2023-11-08T08:21:04.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.352+0000 7fd3e8726700 1 -- start start 2023-11-08T08:21:04.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3e8726700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3e006ac00 0x7fd3e00c3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:04.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3e8726700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3e006d010 0x7fd3e0158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:04.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3e8726700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e01587b0 0x7fd3e00b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:04.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3e8726700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e00c3580 con 0x7fd3e01587b0 2023-11-08T08:21:04.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3e8726700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e00c4700 con 0x7fd3e006ac00 2023-11-08T08:21:04.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3e8726700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e00c49e0 con 0x7fd3e006d010 2023-11-08T08:21:04.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3e6cc3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e01587b0 0x7fd3e00b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:04.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3e6cc3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e01587b0 0x7fd3e00b14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42740/0 (socket says 172.21.15.40:42740) 2023-11-08T08:21:04.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3e6cc3700 1 -- 172.21.15.40:0/1401573976 learned_addr learned my addr 172.21.15.40:0/1401573976 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:04.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3e64c2700 1 --2- 172.21.15.40:0/1401573976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3e006ac00 0x7fd3e00c3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:04.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3e5cc1700 1 --2- 172.21.15.40:0/1401573976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3e006d010 0x7fd3e0158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:04.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3e6cc3700 1 -- 172.21.15.40:0/1401573976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3e006d010 msgr2=0x7fd3e0158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:04.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3e6cc3700 1 --2- 172.21.15.40:0/1401573976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3e006d010 0x7fd3e0158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:04.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3e6cc3700 1 -- 172.21.15.40:0/1401573976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3e006ac00 msgr2=0x7fd3e00c3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:04.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3e6cc3700 1 --2- 172.21.15.40:0/1401573976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3e006ac00 0x7fd3e00c3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:04.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3e6cc3700 1 -- 172.21.15.40:0/1401573976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3dc009570 con 0x7fd3e01587b0 2023-11-08T08:21:04.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3e6cc3700 1 --2- 172.21.15.40:0/1401573976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e01587b0 0x7fd3e00b14f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8007e40 tx=0x7fd3d800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:04.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3d6ffd700 1 -- 172.21.15.40:0/1401573976 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d800ede0 con 0x7fd3e01587b0 2023-11-08T08:21:04.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3d6ffd700 1 -- 172.21.15.40:0/1401573976 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d8014420 con 0x7fd3e01587b0 2023-11-08T08:21:04.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3e8726700 1 -- 172.21.15.40:0/1401573976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e006dae0 con 0x7fd3e01587b0 2023-11-08T08:21:04.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3d6ffd700 1 -- 172.21.15.40:0/1401573976 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d80147d0 con 0x7fd3e01587b0 2023-11-08T08:21:04.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.356+0000 7fd3e8726700 1 -- 172.21.15.40:0/1401573976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e006d800 con 0x7fd3e01587b0 2023-11-08T08:21:04.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.360+0000 7fd3e8726700 1 -- 172.21.15.40:0/1401573976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e00c91a0 con 0x7fd3e01587b0 2023-11-08T08:21:04.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.360+0000 7fd3d6ffd700 1 -- 172.21.15.40:0/1401573976 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd3d8012050 con 0x7fd3e01587b0 2023-11-08T08:21:04.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.360+0000 7fd3d6ffd700 1 --2- 172.21.15.40:0/1401573976 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd3c4064ee0 0x7fd3c4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:04.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.364+0000 7fd3d6ffd700 1 -- 172.21.15.40:0/1401573976 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fd3d8082740 con 0x7fd3e01587b0 2023-11-08T08:21:04.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.364+0000 7fd3e64c2700 1 --2- 172.21.15.40:0/1401573976 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd3c4064ee0 0x7fd3c4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:04.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.364+0000 7fd3e64c2700 1 --2- 172.21.15.40:0/1401573976 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd3c4064ee0 0x7fd3c4067390 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc000c00 tx=0x7fd3dc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:04.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.368+0000 7fd3d6ffd700 1 -- 172.21.15.40:0/1401573976 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd3d8050670 con 0x7fd3e01587b0 2023-11-08T08:21:04.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:04.576+0000 7fd3e8726700 1 -- 172.21.15.40:0/1401573976 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd3e00c37f0 con 0x7fd3e01587b0 2023-11-08T08:21:05.030 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-11-08T08:21:05.024+0000 7f513abf9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T08:21:05.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.036+0000 7fd3d6ffd700 1 -- 172.21.15.40:0/1401573976 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd3d8053c90 con 0x7fd3e01587b0 2023-11-08T08:21:05.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.044+0000 7fd3e8726700 1 -- 172.21.15.40:0/1401573976 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd3c4064ee0 msgr2=0x7fd3c4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:05.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.044+0000 7fd3e8726700 1 --2- 172.21.15.40:0/1401573976 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd3c4064ee0 0x7fd3c4067390 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc000c00 tx=0x7fd3dc008030 comp rx=0 tx=0).stop 2023-11-08T08:21:05.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.044+0000 7fd3e8726700 1 -- 172.21.15.40:0/1401573976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e01587b0 msgr2=0x7fd3e00b14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:05.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.044+0000 7fd3e8726700 1 --2- 172.21.15.40:0/1401573976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e01587b0 0x7fd3e00b14f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8007e40 tx=0x7fd3d800e3c0 comp rx=0 tx=0).stop 2023-11-08T08:21:05.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.044+0000 7fd3e8726700 1 -- 172.21.15.40:0/1401573976 shutdown_connections 2023-11-08T08:21:05.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.044+0000 7fd3e8726700 1 --2- 172.21.15.40:0/1401573976 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd3c4064ee0 0x7fd3c4067390 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:05.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.044+0000 7fd3e8726700 1 --2- 172.21.15.40:0/1401573976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3e01587b0 0x7fd3e00b14f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:05.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.044+0000 7fd3e8726700 1 --2- 172.21.15.40:0/1401573976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3e006d010 0x7fd3e0158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:05.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.044+0000 7fd3e8726700 1 --2- 172.21.15.40:0/1401573976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3e006ac00 0x7fd3e00c3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:05.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.044+0000 7fd3e8726700 1 -- 172.21.15.40:0/1401573976 >> 172.21.15.40:0/1401573976 conn(0x7fd3e00cf350 msgr2=0x7fd3e0069bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:05.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.044+0000 7fd3e8726700 1 -- 172.21.15.40:0/1401573976 shutdown_connections 2023-11-08T08:21:05.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.044+0000 7fd3e8726700 1 -- 172.21.15.40:0/1401573976 wait complete. 2023-11-08T08:21:05.054 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T08:21:05.070 INFO:tasks.ceph.mds.c.smithi040.stderr: -6293> 2023-11-08T08:20:36.229+0000 7fba7688d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T08:21:05.083 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T08:21:05.139 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-11-08T08:21:05.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.264+0000 7f52a5ed1700 1 Processor -- start 2023-11-08T08:21:05.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.264+0000 7f52a5ed1700 1 -- start start 2023-11-08T08:21:05.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.264+0000 7f52a5ed1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f52a006c2b0 0x7f52a0165080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:05.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.264+0000 7f52a5ed1700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f52a006c680 0x7f52a00adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:05.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.264+0000 7f52a5ed1700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52a006afb0 con 0x7f52a00aed60 2023-11-08T08:21:05.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.264+0000 7f52a5ed1700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a00ca750 con 0x7f52a006c2b0 2023-11-08T08:21:05.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.264+0000 7f52a5ed1700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a006b290 con 0x7f52a006c680 2023-11-08T08:21:05.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f529effd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f52a00aed60 0x7f52a01639d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60182/0 (socket says 172.21.15.40:60182) 2023-11-08T08:21:05.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f529effd700 1 -- 172.21.15.40:0/652433833 learned_addr learned my addr 172.21.15.40:0/652433833 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:05.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f529f7fe700 1 --2- 172.21.15.40:0/652433833 >> v2:172.21.15.40:3300/0 conn(0x7f52a006c2b0 0x7f52a0165080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:05.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f529ffff700 1 --2- 172.21.15.40:0/652433833 >> v2:172.21.15.169:3300/0 conn(0x7f52a006c680 0x7f52a00adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:05.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f529e7fc700 1 -- 172.21.15.40:0/652433833 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621046375 0 0) 0x7f52a006afb0 con 0x7f52a00aed60 2023-11-08T08:21:05.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f529e7fc700 1 -- 172.21.15.40:0/652433833 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52880035d0 con 0x7f52a00aed60 2023-11-08T08:21:05.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f529f7fe700 1 -- 172.21.15.40:0/652433833 >> v2:172.21.15.169:3300/0 conn(0x7f52a006c680 msgr2=0x7f52a00adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:05.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f529f7fe700 1 --2- 172.21.15.40:0/652433833 >> v2:172.21.15.169:3300/0 conn(0x7f52a006c680 0x7f52a00adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:05.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f529f7fe700 1 -- 172.21.15.40:0/652433833 >> v1:172.21.15.40:6789/0 conn(0x7f52a00aed60 legacy=0x7f52a01639d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:05.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f529f7fe700 1 -- 172.21.15.40:0/652433833 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a00c8f40 con 0x7f52a006c2b0 2023-11-08T08:21:05.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f529f7fe700 1 --2- 172.21.15.40:0/652433833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a006c2b0 0x7f52a0165080 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f52a0073190 tx=0x7f5290009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9c4d97c0e49b88 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:05.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f529e7fc700 1 -- 172.21.15.40:0/652433833 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290011830 con 0x7f52a006c2b0 2023-11-08T08:21:05.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f529e7fc700 1 -- 172.21.15.40:0/652433833 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5290011990 con 0x7f52a006c2b0 2023-11-08T08:21:05.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f529e7fc700 1 -- 172.21.15.40:0/652433833 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290011d20 con 0x7f52a006c2b0 2023-11-08T08:21:05.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f52a5ed1700 1 -- 172.21.15.40:0/652433833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a006c2b0 msgr2=0x7f52a0165080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:05.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f52a5ed1700 1 --2- 172.21.15.40:0/652433833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a006c2b0 0x7f52a0165080 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f52a0073190 tx=0x7f5290009b30 comp rx=0 tx=0).stop 2023-11-08T08:21:05.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f52a5ed1700 1 -- 172.21.15.40:0/652433833 shutdown_connections 2023-11-08T08:21:05.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f52a5ed1700 1 --2- 172.21.15.40:0/652433833 >> v2:172.21.15.169:3300/0 conn(0x7f52a006c680 0x7f52a00adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:05.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f52a5ed1700 1 --2- 172.21.15.40:0/652433833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a006c2b0 0x7f52a0165080 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:05.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f52a5ed1700 1 -- 172.21.15.40:0/652433833 >> 172.21.15.40:0/652433833 conn(0x7f52a00cf580 msgr2=0x7f52a00cb370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:05.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f52a5ed1700 1 -- 172.21.15.40:0/652433833 shutdown_connections 2023-11-08T08:21:05.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.268+0000 7f52a5ed1700 1 -- 172.21.15.40:0/652433833 wait complete. 2023-11-08T08:21:05.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.272+0000 7f52a5ed1700 1 Processor -- start 2023-11-08T08:21:05.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.276+0000 7f52a5ed1700 1 -- start start 2023-11-08T08:21:05.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.276+0000 7f52a5ed1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a006c2b0 0x7f52a00b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:05.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.276+0000 7f52a5ed1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52a006c680 0x7f52a0158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:05.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.276+0000 7f52a5ed1700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52a00aed60 0x7f52a015bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:05.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.276+0000 7f52a5ed1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a015c790 con 0x7f52a006c2b0 2023-11-08T08:21:05.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.276+0000 7f52a5ed1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a00b2d70 con 0x7f52a006c680 2023-11-08T08:21:05.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.276+0000 7f52a5ed1700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a015c4b0 con 0x7f52a00aed60 2023-11-08T08:21:05.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.276+0000 7f529f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a006c2b0 0x7f52a00b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:05.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.276+0000 7f529f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a006c2b0 0x7f52a00b2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42758/0 (socket says 172.21.15.40:42758) 2023-11-08T08:21:05.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.276+0000 7f529f7fe700 1 -- 172.21.15.40:0/3728263738 learned_addr learned my addr 172.21.15.40:0/3728263738 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:05.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.276+0000 7f529effd700 1 --2- 172.21.15.40:0/3728263738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52a006c680 0x7f52a0158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:05.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.276+0000 7f529ffff700 1 --2- 172.21.15.40:0/3728263738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52a00aed60 0x7f52a015bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:05.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.276+0000 7f529f7fe700 1 -- 172.21.15.40:0/3728263738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52a00aed60 msgr2=0x7f52a015bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:05.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.276+0000 7f529f7fe700 1 --2- 172.21.15.40:0/3728263738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52a00aed60 0x7f52a015bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:05.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.276+0000 7f529f7fe700 1 -- 172.21.15.40:0/3728263738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52a006c680 msgr2=0x7f52a0158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:05.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.276+0000 7f529f7fe700 1 --2- 172.21.15.40:0/3728263738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52a006c680 0x7f52a0158a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:05.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.276+0000 7f529f7fe700 1 -- 172.21.15.40:0/3728263738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52900094b0 con 0x7f52a006c2b0 2023-11-08T08:21:05.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.276+0000 7f529ffff700 1 --2- 172.21.15.40:0/3728263738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52a00aed60 0x7f52a015bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:05.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.276+0000 7f529f7fe700 1 --2- 172.21.15.40:0/3728263738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a006c2b0 0x7f52a00b2830 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5290009b70 tx=0x7f5290004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:05.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.280+0000 7f5287fff700 1 -- 172.21.15.40:0/3728263738 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290004480 con 0x7f52a006c2b0 2023-11-08T08:21:05.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.280+0000 7f5287fff700 1 -- 172.21.15.40:0/3728263738 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529001f070 con 0x7f52a006c2b0 2023-11-08T08:21:05.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.280+0000 7f52a5ed1700 1 -- 172.21.15.40:0/3728263738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a00ad800 con 0x7f52a006c2b0 2023-11-08T08:21:05.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.280+0000 7f5287fff700 1 -- 172.21.15.40:0/3728263738 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52900038c0 con 0x7f52a006c2b0 2023-11-08T08:21:05.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.280+0000 7f52a5ed1700 1 -- 172.21.15.40:0/3728263738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a00c9220 con 0x7f52a006c2b0 2023-11-08T08:21:05.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.280+0000 7f52a5ed1700 1 -- 172.21.15.40:0/3728263738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a01640e0 con 0x7f52a006c2b0 2023-11-08T08:21:05.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.284+0000 7f5287fff700 1 -- 172.21.15.40:0/3728263738 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f529002a020 con 0x7f52a006c2b0 2023-11-08T08:21:05.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.284+0000 7f5287fff700 1 --2- 172.21.15.40:0/3728263738 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f528c064ee0 0x7f528c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:05.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.284+0000 7f5287fff700 1 -- 172.21.15.40:0/3728263738 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6526+0+0 (secure 0 0 0) 0x7f5290083660 con 0x7f52a006c2b0 2023-11-08T08:21:05.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.284+0000 7f529effd700 1 --2- 172.21.15.40:0/3728263738 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f528c064ee0 0x7f528c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:05.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.284+0000 7f529effd700 1 --2- 172.21.15.40:0/3728263738 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f528c064ee0 0x7f528c067390 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f52a015cb70 tx=0x7f5294009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:05.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.288+0000 7f5287fff700 1 -- 172.21.15.40:0/3728263738 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5290051540 con 0x7f52a006c2b0 2023-11-08T08:21:05.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:05.496+0000 7f52a5ed1700 1 -- 172.21.15.40:0/3728263738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f52a00726c0 con 0x7f52a006c2b0 2023-11-08T08:21:06.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.040+0000 7f5287fff700 1 -- 172.21.15.40:0/3728263738 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5290054b60 con 0x7f52a006c2b0 2023-11-08T08:21:06.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.044+0000 7f52a5ed1700 1 -- 172.21.15.40:0/3728263738 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f528c064ee0 msgr2=0x7f528c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:06.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.044+0000 7f52a5ed1700 1 --2- 172.21.15.40:0/3728263738 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f528c064ee0 0x7f528c067390 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f52a015cb70 tx=0x7f5294009040 comp rx=0 tx=0).stop 2023-11-08T08:21:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.044+0000 7f52a5ed1700 1 -- 172.21.15.40:0/3728263738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a006c2b0 msgr2=0x7f52a00b2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.044+0000 7f52a5ed1700 1 --2- 172.21.15.40:0/3728263738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a006c2b0 0x7f52a00b2830 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5290009b70 tx=0x7f5290004770 comp rx=0 tx=0).stop 2023-11-08T08:21:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.044+0000 7f52a5ed1700 1 -- 172.21.15.40:0/3728263738 shutdown_connections 2023-11-08T08:21:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.044+0000 7f52a5ed1700 1 --2- 172.21.15.40:0/3728263738 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f528c064ee0 0x7f528c067390 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:06.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.044+0000 7f52a5ed1700 1 --2- 172.21.15.40:0/3728263738 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52a00aed60 0x7f52a015bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.044+0000 7f52a5ed1700 1 --2- 172.21.15.40:0/3728263738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52a006c680 0x7f52a0158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.044+0000 7f52a5ed1700 1 --2- 172.21.15.40:0/3728263738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52a006c2b0 0x7f52a00b2830 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:06.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.044+0000 7f52a5ed1700 1 -- 172.21.15.40:0/3728263738 >> 172.21.15.40:0/3728263738 conn(0x7f52a00cf580 msgr2=0x7f52a00af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:06.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.044+0000 7f52a5ed1700 1 -- 172.21.15.40:0/3728263738 shutdown_connections 2023-11-08T08:21:06.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.044+0000 7f52a5ed1700 1 -- 172.21.15.40:0/3728263738 wait complete. 2023-11-08T08:21:06.076 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T08:21:06.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.272+0000 7f8b3f59e700 1 Processor -- start 2023-11-08T08:21:06.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.272+0000 7f8b3f59e700 1 -- start start 2023-11-08T08:21:06.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.272+0000 7f8b3f59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b4015c6e0 0x7f8b40071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:06.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.272+0000 7f8b3f59e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8b4006a510 0x7f8b4006c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:06.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.272+0000 7f8b3f59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b4015cab0 con 0x7f8b40071fe0 2023-11-08T08:21:06.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.272+0000 7f8b3f59e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b400d2800 con 0x7f8b4006a510 2023-11-08T08:21:06.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.272+0000 7f8b3f59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400702d0 con 0x7f8b4015c6e0 2023-11-08T08:21:06.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.272+0000 7f8b3dd9b700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8b40071fe0 0x7f8b40069e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60192/0 (socket says 172.21.15.40:60192) 2023-11-08T08:21:06.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.272+0000 7f8b3dd9b700 1 -- 172.21.15.40:0/3136109531 learned_addr learned my addr 172.21.15.40:0/3136109531 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:06.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.272+0000 7f8b3ed9d700 1 --2- 172.21.15.40:0/3136109531 >> v2:172.21.15.169:3300/0 conn(0x7f8b4006a510 0x7f8b4006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:06.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.272+0000 7f8b3d59a700 1 -- 172.21.15.40:0/3136109531 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2199310241 0 0) 0x7f8b4015cab0 con 0x7f8b40071fe0 2023-11-08T08:21:06.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3d59a700 1 -- 172.21.15.40:0/3136109531 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b280035d0 con 0x7f8b40071fe0 2023-11-08T08:21:06.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3e59c700 1 --2- 172.21.15.40:0/3136109531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b4015c6e0 0x7f8b40071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:06.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3d59a700 1 -- 172.21.15.40:0/3136109531 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3390456759 0 0) 0x7f8b280035d0 con 0x7f8b40071fe0 2023-11-08T08:21:06.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3d59a700 1 -- 172.21.15.40:0/3136109531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b4015c6e0 msgr2=0x7f8b40071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3d59a700 1 --2- 172.21.15.40:0/3136109531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b4015c6e0 0x7f8b40071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3d59a700 1 -- 172.21.15.40:0/3136109531 >> v2:172.21.15.169:3300/0 conn(0x7f8b4006a510 msgr2=0x7f8b4006c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3d59a700 1 --2- 172.21.15.40:0/3136109531 >> v2:172.21.15.169:3300/0 conn(0x7f8b4006a510 0x7f8b4006c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3d59a700 1 -- 172.21.15.40:0/3136109531 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b400d2a10 con 0x7f8b40071fe0 2023-11-08T08:21:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3d59a700 1 -- 172.21.15.40:0/3136109531 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f8b34001670 con 0x7f8b40071fe0 2023-11-08T08:21:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3ed9d700 1 --2- 172.21.15.40:0/3136109531 >> v2:172.21.15.169:3300/0 conn(0x7f8b4006a510 0x7f8b4006c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:06.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3d59a700 1 -- 172.21.15.40:0/3136109531 >> v1:172.21.15.40:6789/0 conn(0x7f8b40071fe0 legacy=0x7f8b40069e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:06.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3d59a700 1 --2- 172.21.15.40:0/3136109531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b28003a20 0x7f8b28003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:06.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3d59a700 1 --2- 172.21.15.40:0/3136109531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b4015c6e0 0x7f8b28004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:06.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3d59a700 1 --2- 172.21.15.40:0/3136109531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b28004db0 0x7f8b28005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:06.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3d59a700 1 -- 172.21.15.40:0/3136109531 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400d2800 con 0x7f8b28003a20 2023-11-08T08:21:06.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3d59a700 1 -- 172.21.15.40:0/3136109531 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400702d0 con 0x7f8b28004db0 2023-11-08T08:21:06.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3d59a700 1 -- 172.21.15.40:0/3136109531 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2800bae0 con 0x7f8b4015c6e0 2023-11-08T08:21:06.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3e59c700 1 --2- 172.21.15.40:0/3136109531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b28003a20 0x7f8b28003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:06.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3dd9b700 1 --2- 172.21.15.40:0/3136109531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b4015c6e0 0x7f8b28004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:06.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3ed9d700 1 --2- 172.21.15.40:0/3136109531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b28004db0 0x7f8b28005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3e59c700 1 -- 172.21.15.40:0/3136109531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b4015c6e0 msgr2=0x7f8b28004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3e59c700 1 --2- 172.21.15.40:0/3136109531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b4015c6e0 0x7f8b28004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3e59c700 1 -- 172.21.15.40:0/3136109531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b28004db0 msgr2=0x7f8b28005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:06.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3e59c700 1 --2- 172.21.15.40:0/3136109531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b28004db0 0x7f8b28005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:06.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3e59c700 1 -- 172.21.15.40:0/3136109531 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b280035d0 con 0x7f8b28003a20 2023-11-08T08:21:06.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3e59c700 1 --2- 172.21.15.40:0/3136109531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b28003a20 0x7f8b28003e30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8b300098b0 tx=0x7f8b280058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af99a4d361c4a0f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:06.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3d59a700 1 -- 172.21.15.40:0/3136109531 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3000c070 con 0x7f8b28003a20 2023-11-08T08:21:06.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3d59a700 1 -- 172.21.15.40:0/3136109531 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b30013610 con 0x7f8b28003a20 2023-11-08T08:21:06.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.276+0000 7f8b3d59a700 1 -- 172.21.15.40:0/3136109531 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b300139c0 con 0x7f8b28003a20 2023-11-08T08:21:06.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.280+0000 7f8b3f59e700 1 -- 172.21.15.40:0/3136109531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b28003a20 msgr2=0x7f8b28003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:06.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.280+0000 7f8b3f59e700 1 --2- 172.21.15.40:0/3136109531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b28003a20 0x7f8b28003e30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8b300098b0 tx=0x7f8b280058a0 comp rx=0 tx=0).stop 2023-11-08T08:21:06.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.280+0000 7f8b3ed9d700 1 -- 172.21.15.40:0/3136109531 reap_dead start 2023-11-08T08:21:06.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.280+0000 7f8b3f59e700 1 -- 172.21.15.40:0/3136109531 shutdown_connections 2023-11-08T08:21:06.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.280+0000 7f8b3f59e700 1 -- 172.21.15.40:0/3136109531 >> 172.21.15.40:0/3136109531 conn(0x7f8b400b65b0 msgr2=0x7f8b40071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:06.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.280+0000 7f8b3f59e700 1 -- 172.21.15.40:0/3136109531 shutdown_connections 2023-11-08T08:21:06.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.280+0000 7f8b3f59e700 1 -- 172.21.15.40:0/3136109531 wait complete. 2023-11-08T08:21:06.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.284+0000 7f8b3f59e700 1 Processor -- start 2023-11-08T08:21:06.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.284+0000 7f8b3f59e700 1 -- start start 2023-11-08T08:21:06.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.284+0000 7f8b3f59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b28003a20 0x7f8b40158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:06.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.284+0000 7f8b3f59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4015aee0 0x7f8b400b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:06.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.284+0000 7f8b3f59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b4015b330 0x7f8b400b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:06.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.284+0000 7f8b3f59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b40158640 con 0x7f8b4015b330 2023-11-08T08:21:06.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.284+0000 7f8b3f59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400b31f0 con 0x7f8b4015aee0 2023-11-08T08:21:06.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.284+0000 7f8b3f59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400b33f0 con 0x7f8b28003a20 2023-11-08T08:21:06.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.284+0000 7f8b3ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b4015b330 0x7f8b400b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:06.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.284+0000 7f8b3ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b4015b330 0x7f8b400b2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42770/0 (socket says 172.21.15.40:42770) 2023-11-08T08:21:06.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.284+0000 7f8b3ed9d700 1 -- 172.21.15.40:0/689196852 learned_addr learned my addr 172.21.15.40:0/689196852 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:06.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.284+0000 7f8b3e59c700 1 --2- 172.21.15.40:0/689196852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b28003a20 0x7f8b40158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:06.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.284+0000 7f8b3dd9b700 1 --2- 172.21.15.40:0/689196852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4015aee0 0x7f8b400b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:06.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.284+0000 7f8b3ed9d700 1 -- 172.21.15.40:0/689196852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b28003a20 msgr2=0x7f8b40158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:06.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.284+0000 7f8b3ed9d700 1 --2- 172.21.15.40:0/689196852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b28003a20 0x7f8b40158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:06.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.284+0000 7f8b3ed9d700 1 -- 172.21.15.40:0/689196852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4015aee0 msgr2=0x7f8b400b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:06.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.284+0000 7f8b3ed9d700 1 --2- 172.21.15.40:0/689196852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4015aee0 0x7f8b400b2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.284+0000 7f8b3ed9d700 1 -- 172.21.15.40:0/689196852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b400d2fd0 con 0x7f8b4015b330 2023-11-08T08:21:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.284+0000 7f8b3ed9d700 1 --2- 172.21.15.40:0/689196852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b4015b330 0x7f8b400b2cb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8b4006ef00 tx=0x7f8b38010440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.288+0000 7f8b2effd700 1 -- 172.21.15.40:0/689196852 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3801b070 con 0x7f8b4015b330 2023-11-08T08:21:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.288+0000 7f8b2effd700 1 -- 172.21.15.40:0/689196852 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b38003a40 con 0x7f8b4015b330 2023-11-08T08:21:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.288+0000 7f8b2effd700 1 -- 172.21.15.40:0/689196852 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38003bd0 con 0x7f8b4015b330 2023-11-08T08:21:06.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.288+0000 7f8b3f59e700 1 -- 172.21.15.40:0/689196852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b400d2cf0 con 0x7f8b4015b330 2023-11-08T08:21:06.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.288+0000 7f8b3f59e700 1 -- 172.21.15.40:0/689196852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b400d32b0 con 0x7f8b4015b330 2023-11-08T08:21:06.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.288+0000 7f8b3f59e700 1 -- 172.21.15.40:0/689196852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b40158780 con 0x7f8b4015b330 2023-11-08T08:21:06.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.288+0000 7f8b2effd700 1 -- 172.21.15.40:0/689196852 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f8b38010a20 con 0x7f8b4015b330 2023-11-08T08:21:06.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.292+0000 7f8b2effd700 1 --2- 172.21.15.40:0/689196852 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8b20064ee0 0x7f8b20067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:06.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.292+0000 7f8b2effd700 1 -- 172.21.15.40:0/689196852 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6526+0+0 (secure 0 0 0) 0x7f8b38082650 con 0x7f8b4015b330 2023-11-08T08:21:06.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.292+0000 7f8b3e59c700 1 --2- 172.21.15.40:0/689196852 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8b20064ee0 0x7f8b20067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:06.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.292+0000 7f8b3e59c700 1 --2- 172.21.15.40:0/689196852 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8b20064ee0 0x7f8b20067390 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8b30008010 tx=0x7f8b3001e040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:06.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.300+0000 7f8b2effd700 1 -- 172.21.15.40:0/689196852 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f8b40158780 con 0x7f8b4015b330 2023-11-08T08:21:06.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:06.488+0000 7f8b3f59e700 1 -- 172.21.15.40:0/689196852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b40158780 con 0x7f8b4015b330 2023-11-08T08:21:07.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.164+0000 7f8b2effd700 1 -- 172.21.15.40:0/689196852 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v47) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8b40158780 con 0x7f8b4015b330 2023-11-08T08:21:07.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.168+0000 7f8b3f59e700 1 -- 172.21.15.40:0/689196852 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8b20064ee0 msgr2=0x7f8b20067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:07.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.168+0000 7f8b3f59e700 1 --2- 172.21.15.40:0/689196852 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8b20064ee0 0x7f8b20067390 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8b30008010 tx=0x7f8b3001e040 comp rx=0 tx=0).stop 2023-11-08T08:21:07.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.168+0000 7f8b3f59e700 1 -- 172.21.15.40:0/689196852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b4015b330 msgr2=0x7f8b400b2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:07.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.168+0000 7f8b3f59e700 1 --2- 172.21.15.40:0/689196852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b4015b330 0x7f8b400b2cb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8b4006ef00 tx=0x7f8b38010440 comp rx=0 tx=0).stop 2023-11-08T08:21:07.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.168+0000 7f8b3f59e700 1 -- 172.21.15.40:0/689196852 shutdown_connections 2023-11-08T08:21:07.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.168+0000 7f8b3f59e700 1 --2- 172.21.15.40:0/689196852 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8b20064ee0 0x7f8b20067390 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:07.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.168+0000 7f8b3f59e700 1 --2- 172.21.15.40:0/689196852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b4015b330 0x7f8b400b2cb0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:07.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.168+0000 7f8b3f59e700 1 --2- 172.21.15.40:0/689196852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4015aee0 0x7f8b400b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:07.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.168+0000 7f8b3f59e700 1 --2- 172.21.15.40:0/689196852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b28003a20 0x7f8b40158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:07.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.168+0000 7f8b3f59e700 1 -- 172.21.15.40:0/689196852 >> 172.21.15.40:0/689196852 conn(0x7f8b400b65b0 msgr2=0x7f8b40071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:07.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.168+0000 7f8b3f59e700 1 -- 172.21.15.40:0/689196852 shutdown_connections 2023-11-08T08:21:07.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.168+0000 7f8b3f59e700 1 -- 172.21.15.40:0/689196852 wait complete. 2023-11-08T08:21:07.178 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2023-11-08T08:21:07.199 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T08:21:07.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.380+0000 7fa1f6abc700 1 Processor -- start 2023-11-08T08:21:07.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.380+0000 7fa1f6abc700 1 -- start start 2023-11-08T08:21:07.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f6abc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1f00c1c90 0x7fa1f016b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:07.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f6abc700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f0165140 con 0x7fa1f00ca940 2023-11-08T08:21:07.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f6abc700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f00cced0 con 0x7fa1f00ccb00 2023-11-08T08:21:07.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f6abc700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f0160da0 con 0x7fa1f00c1c90 2023-11-08T08:21:07.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f52b9700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa1f00ca940 0x7fa1f00c1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45076/0 (socket says 172.21.15.40:45076) 2023-11-08T08:21:07.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f52b9700 1 -- 172.21.15.40:0/486100456 learned_addr learned my addr 172.21.15.40:0/486100456 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:07.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f62bb700 1 --2- 172.21.15.40:0/486100456 >> v2:172.21.15.40:3300/0 conn(0x7fa1f00c1c90 0x7fa1f016b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:07.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f4ab8700 1 -- 172.21.15.40:0/486100456 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 85036462 0 0) 0x7fa1f0165140 con 0x7fa1f00ca940 2023-11-08T08:21:07.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f4ab8700 1 -- 172.21.15.40:0/486100456 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1dc0035d0 con 0x7fa1f00ca940 2023-11-08T08:21:07.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f4ab8700 1 -- 172.21.15.40:0/486100456 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 273613122 0 0) 0x7fa1f00cced0 con 0x7fa1f00ccb00 2023-11-08T08:21:07.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f4ab8700 1 -- 172.21.15.40:0/486100456 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1f0165140 con 0x7fa1f00ccb00 2023-11-08T08:21:07.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f4ab8700 1 -- 172.21.15.40:0/486100456 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2286514815 0 0) 0x7fa1dc0035d0 con 0x7fa1f00ca940 2023-11-08T08:21:07.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f4ab8700 1 -- 172.21.15.40:0/486100456 >> v2:172.21.15.40:3300/0 conn(0x7fa1f00c1c90 msgr2=0x7fa1f016b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:07.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f4ab8700 1 --2- 172.21.15.40:0/486100456 >> v2:172.21.15.40:3300/0 conn(0x7fa1f00c1c90 0x7fa1f016b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:07.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f4ab8700 1 -- 172.21.15.40:0/486100456 >> v1:172.21.15.169:6789/0 conn(0x7fa1f00ccb00 legacy=0x7fa1f00ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:07.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f4ab8700 1 -- 172.21.15.40:0/486100456 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f0164530 con 0x7fa1f00ca940 2023-11-08T08:21:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f4ab8700 1 -- 172.21.15.40:0/486100456 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7fa1e0001670 con 0x7fa1f00ca940 2023-11-08T08:21:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f4ab8700 1 -- 172.21.15.40:0/486100456 >> v1:172.21.15.40:6789/0 conn(0x7fa1f00ca940 legacy=0x7fa1f00c1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f4ab8700 1 --2- 172.21.15.40:0/486100456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1dc003bb0 0x7fa1dc003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f4ab8700 1 --2- 172.21.15.40:0/486100456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1dc004870 0x7fa1dc004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f4ab8700 1 --2- 172.21.15.40:0/486100456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1dc005290 0x7fa1dc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:07.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f4ab8700 1 -- 172.21.15.40:0/486100456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f0160da0 con 0x7fa1dc004870 2023-11-08T08:21:07.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f4ab8700 1 -- 172.21.15.40:0/486100456 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc00c1d0 con 0x7fa1dc005290 2023-11-08T08:21:07.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f4ab8700 1 -- 172.21.15.40:0/486100456 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc00c390 con 0x7fa1dc003bb0 2023-11-08T08:21:07.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f62bb700 1 --2- 172.21.15.40:0/486100456 >> v2:172.21.15.40:3300/0 conn(0x7fa1f00c1c90 0x7fa1f016b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:21:07.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f5aba700 1 --2- 172.21.15.40:0/486100456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1dc003bb0 0x7fa1dc003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:07.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.384+0000 7fa1f52b9700 1 --2- 172.21.15.40:0/486100456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1dc004870 0x7fa1dc004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:07.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.388+0000 7fa1f62bb700 1 --2- 172.21.15.40:0/486100456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1dc005290 0x7fa1dc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:07.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.388+0000 7fa1f52b9700 1 -- 172.21.15.40:0/486100456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1dc003bb0 msgr2=0x7fa1dc003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:07.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.388+0000 7fa1f52b9700 1 --2- 172.21.15.40:0/486100456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1dc003bb0 0x7fa1dc003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:07.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.388+0000 7fa1f52b9700 1 -- 172.21.15.40:0/486100456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1dc005290 msgr2=0x7fa1dc00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:07.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.388+0000 7fa1f52b9700 1 --2- 172.21.15.40:0/486100456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1dc005290 0x7fa1dc00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:07.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.388+0000 7fa1f52b9700 1 -- 172.21.15.40:0/486100456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc0035d0 con 0x7fa1dc004870 2023-11-08T08:21:07.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.388+0000 7fa1f62bb700 1 -- 172.21.15.40:0/486100456 reap_dead start 2023-11-08T08:21:07.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.388+0000 7fa1f5aba700 1 --2- 172.21.15.40:0/486100456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1dc003bb0 0x7fa1dc003fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:21:07.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.388+0000 7fa1f52b9700 1 --2- 172.21.15.40:0/486100456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1dc004870 0x7fa1dc004ce0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000c240 tx=0x7fa1dc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295839a118feb93f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:07.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.388+0000 7fa1f4ab8700 1 -- 172.21.15.40:0/486100456 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e000cd40 con 0x7fa1dc004870 2023-11-08T08:21:07.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.388+0000 7fa1f4ab8700 1 -- 172.21.15.40:0/486100456 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e0007b30 con 0x7fa1dc004870 2023-11-08T08:21:07.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.388+0000 7fa1f4ab8700 1 -- 172.21.15.40:0/486100456 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e000cea0 con 0x7fa1dc004870 2023-11-08T08:21:07.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.388+0000 7fa1f6abc700 1 -- 172.21.15.40:0/486100456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1dc004870 msgr2=0x7fa1dc004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:07.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.388+0000 7fa1f6abc700 1 --2- 172.21.15.40:0/486100456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1dc004870 0x7fa1dc004ce0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000c240 tx=0x7fa1dc0058a0 comp rx=0 tx=0).stop 2023-11-08T08:21:07.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.388+0000 7fa1f6abc700 1 -- 172.21.15.40:0/486100456 shutdown_connections 2023-11-08T08:21:07.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.388+0000 7fa1f6abc700 1 --2- 172.21.15.40:0/486100456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1dc004870 0x7fa1dc004ce0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:07.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.388+0000 7fa1f6abc700 1 -- 172.21.15.40:0/486100456 >> 172.21.15.40:0/486100456 conn(0x7fa1f00ce5b0 msgr2=0x7fa1f00ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:07.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.388+0000 7fa1f6abc700 1 -- 172.21.15.40:0/486100456 shutdown_connections 2023-11-08T08:21:07.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.388+0000 7fa1f6abc700 1 -- 172.21.15.40:0/486100456 wait complete. 2023-11-08T08:21:07.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.392+0000 7fa1f6abc700 1 Processor -- start 2023-11-08T08:21:07.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.392+0000 7fa1f6abc700 1 -- start start 2023-11-08T08:21:07.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1f6abc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1dc004870 0x7fa1f00c5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:07.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1f6abc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1f0163930 0x7fa1f01f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:07.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1f6abc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1f015f200 0x7fa1f015f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:07.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1f6abc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f00c84a0 con 0x7fa1f015f200 2023-11-08T08:21:07.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1f6abc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f015fbb0 con 0x7fa1dc004870 2023-11-08T08:21:07.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1f6abc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f0163da0 con 0x7fa1f0163930 2023-11-08T08:21:07.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1f62bb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1f015f200 0x7fa1f015f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:07.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1f62bb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1f015f200 0x7fa1f015f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43200/0 (socket says 172.21.15.40:43200) 2023-11-08T08:21:07.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1f62bb700 1 -- 172.21.15.40:0/1974454988 learned_addr learned my addr 172.21.15.40:0/1974454988 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:07.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1f5aba700 1 --2- 172.21.15.40:0/1974454988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1dc004870 0x7fa1f00c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:07.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1f52b9700 1 --2- 172.21.15.40:0/1974454988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1f0163930 0x7fa1f01f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:07.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1f62bb700 1 -- 172.21.15.40:0/1974454988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1f0163930 msgr2=0x7fa1f01f5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:07.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1f62bb700 1 --2- 172.21.15.40:0/1974454988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1f0163930 0x7fa1f01f5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:07.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1f62bb700 1 -- 172.21.15.40:0/1974454988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1dc004870 msgr2=0x7fa1f00c5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:07.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1f62bb700 1 --2- 172.21.15.40:0/1974454988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1dc004870 0x7fa1f00c5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:07.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1f62bb700 1 -- 172.21.15.40:0/1974454988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f0164af0 con 0x7fa1f015f200 2023-11-08T08:21:07.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1f62bb700 1 --2- 172.21.15.40:0/1974454988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1f015f200 0x7fa1f015f670 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00c2080 tx=0x7fa1e800f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:07.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1e67fc700 1 -- 172.21.15.40:0/1974454988 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e800e040 con 0x7fa1f015f200 2023-11-08T08:21:07.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1e67fc700 1 -- 172.21.15.40:0/1974454988 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e800f040 con 0x7fa1f015f200 2023-11-08T08:21:07.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1f6abc700 1 -- 172.21.15.40:0/1974454988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f0164810 con 0x7fa1f015f200 2023-11-08T08:21:07.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1e67fc700 1 -- 172.21.15.40:0/1974454988 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8015430 con 0x7fa1f015f200 2023-11-08T08:21:07.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.396+0000 7fa1f6abc700 1 -- 172.21.15.40:0/1974454988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f0164dd0 con 0x7fa1f015f200 2023-11-08T08:21:07.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.400+0000 7fa1f6abc700 1 -- 172.21.15.40:0/1974454988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f01619e0 con 0x7fa1f015f200 2023-11-08T08:21:07.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.400+0000 7fa1e67fc700 1 -- 172.21.15.40:0/1974454988 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa1e8015590 con 0x7fa1f015f200 2023-11-08T08:21:07.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.404+0000 7fa1e67fc700 1 --2- 172.21.15.40:0/1974454988 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa1d4064ee0 0x7fa1d4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:07.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.404+0000 7fa1e67fc700 1 -- 172.21.15.40:0/1974454988 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6088+0+0 (secure 0 0 0) 0x7fa1e8082b00 con 0x7fa1f015f200 2023-11-08T08:21:07.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.404+0000 7fa1f5aba700 1 --2- 172.21.15.40:0/1974454988 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa1d4064ee0 0x7fa1d4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:07.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.404+0000 7fa1f5aba700 1 --2- 172.21.15.40:0/1974454988 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa1d4064ee0 0x7fa1d4067390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec000c00 tx=0x7fa1ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:07.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.408+0000 7fa1e67fc700 1 -- 172.21.15.40:0/1974454988 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa1f01619e0 con 0x7fa1f015f200 2023-11-08T08:21:07.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:07.592+0000 7fa1f6abc700 1 -- 172.21.15.40:0/1974454988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa1f01619e0 con 0x7fa1f015f200 2023-11-08T08:21:08.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.172+0000 7fa1e67fc700 1 -- 172.21.15.40:0/1974454988 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v48) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa1f01619e0 con 0x7fa1f015f200 2023-11-08T08:21:08.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.184+0000 7fa1f6abc700 1 -- 172.21.15.40:0/1974454988 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa1d4064ee0 msgr2=0x7fa1d4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:08.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.184+0000 7fa1f6abc700 1 --2- 172.21.15.40:0/1974454988 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa1d4064ee0 0x7fa1d4067390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec000c00 tx=0x7fa1ec009040 comp rx=0 tx=0).stop 2023-11-08T08:21:08.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.184+0000 7fa1f6abc700 1 -- 172.21.15.40:0/1974454988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1f015f200 msgr2=0x7fa1f015f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:08.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.184+0000 7fa1f6abc700 1 --2- 172.21.15.40:0/1974454988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1f015f200 0x7fa1f015f670 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00c2080 tx=0x7fa1e800f490 comp rx=0 tx=0).stop 2023-11-08T08:21:08.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.184+0000 7fa1f6abc700 1 -- 172.21.15.40:0/1974454988 shutdown_connections 2023-11-08T08:21:08.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.184+0000 7fa1f6abc700 1 --2- 172.21.15.40:0/1974454988 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa1d4064ee0 0x7fa1d4067390 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:08.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.184+0000 7fa1f6abc700 1 --2- 172.21.15.40:0/1974454988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1f015f200 0x7fa1f015f670 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:08.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.184+0000 7fa1f6abc700 1 --2- 172.21.15.40:0/1974454988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1f0163930 0x7fa1f01f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:08.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.184+0000 7fa1f6abc700 1 --2- 172.21.15.40:0/1974454988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1dc004870 0x7fa1f00c5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:08.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.184+0000 7fa1f6abc700 1 -- 172.21.15.40:0/1974454988 >> 172.21.15.40:0/1974454988 conn(0x7fa1f00ce5b0 msgr2=0x7fa1f006b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:08.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.184+0000 7fa1f6abc700 1 -- 172.21.15.40:0/1974454988 shutdown_connections 2023-11-08T08:21:08.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.188+0000 7fa1f6abc700 1 -- 172.21.15.40:0/1974454988 wait complete. 2023-11-08T08:21:08.202 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2023-11-08T08:21:08.222 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-08T08:21:08.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.396+0000 7f221f59e700 1 Processor -- start 2023-11-08T08:21:08.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.396+0000 7f221f59e700 1 -- start start 2023-11-08T08:21:08.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.396+0000 7f221f59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22200cdeb0 0x7f2220070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:08.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.396+0000 7f221f59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22200adf90 con 0x7f222015c1a0 2023-11-08T08:21:08.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.396+0000 7f221f59e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22200ce370 con 0x7f22200705c0 2023-11-08T08:21:08.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.396+0000 7f221f59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2220069bf0 con 0x7f22200cdeb0 2023-11-08T08:21:08.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.396+0000 7f221e59c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f222015c1a0 0x7f222006c8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45088/0 (socket says 172.21.15.40:45088) 2023-11-08T08:21:08.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221e59c700 1 -- 172.21.15.40:0/84541169 learned_addr learned my addr 172.21.15.40:0/84541169 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:08.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221dd9b700 1 --2- 172.21.15.40:0/84541169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22200cdeb0 0x7f2220070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:08.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221d59a700 1 -- 172.21.15.40:0/84541169 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926461750 0 0) 0x7f22200adf90 con 0x7f222015c1a0 2023-11-08T08:21:08.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221d59a700 1 -- 172.21.15.40:0/84541169 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22080035d0 con 0x7f222015c1a0 2023-11-08T08:21:08.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221d59a700 1 -- 172.21.15.40:0/84541169 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391734697 0 0) 0x7f22200ce370 con 0x7f22200705c0 2023-11-08T08:21:08.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221d59a700 1 -- 172.21.15.40:0/84541169 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22200adf90 con 0x7f22200705c0 2023-11-08T08:21:08.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221d59a700 1 -- 172.21.15.40:0/84541169 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1584456878 0 0) 0x7f22080035d0 con 0x7f222015c1a0 2023-11-08T08:21:08.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221d59a700 1 -- 172.21.15.40:0/84541169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22200cdeb0 msgr2=0x7f2220070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:08.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221d59a700 1 --2- 172.21.15.40:0/84541169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22200cdeb0 0x7f2220070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:08.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221d59a700 1 -- 172.21.15.40:0/84541169 >> v1:172.21.15.169:6789/0 conn(0x7f22200705c0 legacy=0x7f222006a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:08.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221d59a700 1 -- 172.21.15.40:0/84541169 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22200cc4a0 con 0x7f222015c1a0 2023-11-08T08:21:08.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221d59a700 1 -- 172.21.15.40:0/84541169 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f2210001670 con 0x7f222015c1a0 2023-11-08T08:21:08.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221d59a700 1 -- 172.21.15.40:0/84541169 >> v1:172.21.15.40:6789/0 conn(0x7f222015c1a0 legacy=0x7f222006c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:08.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221d59a700 1 --2- 172.21.15.40:0/84541169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2208003be0 0x7f2208003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:08.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221d59a700 1 --2- 172.21.15.40:0/84541169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2208004870 0x7f2208004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:08.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221d59a700 1 --2- 172.21.15.40:0/84541169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2208005220 0x7f220800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:08.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221d59a700 1 -- 172.21.15.40:0/84541169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2220069bf0 con 0x7f2208004870 2023-11-08T08:21:08.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221d59a700 1 -- 172.21.15.40:0/84541169 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220800c080 con 0x7f2208005220 2023-11-08T08:21:08.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221d59a700 1 -- 172.21.15.40:0/84541169 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220800c240 con 0x7f2208003be0 2023-11-08T08:21:08.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221dd9b700 1 --2- 172.21.15.40:0/84541169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22200cdeb0 0x7f2220070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:21:08.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221ed9d700 1 --2- 172.21.15.40:0/84541169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2208005220 0x7f220800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:08.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221e59c700 1 --2- 172.21.15.40:0/84541169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2208003be0 0x7f2208003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:08.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221dd9b700 1 --2- 172.21.15.40:0/84541169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2208004870 0x7f2208004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:08.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221ed9d700 1 -- 172.21.15.40:0/84541169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2208003be0 msgr2=0x7f2208003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:08.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221ed9d700 1 --2- 172.21.15.40:0/84541169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2208003be0 0x7f2208003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:08.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221ed9d700 1 -- 172.21.15.40:0/84541169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2208004870 msgr2=0x7f2208004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:08.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221ed9d700 1 --2- 172.21.15.40:0/84541169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2208004870 0x7f2208004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:08.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.400+0000 7f221ed9d700 1 -- 172.21.15.40:0/84541169 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22080035d0 con 0x7f2208005220 2023-11-08T08:21:08.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.404+0000 7f221ed9d700 1 --2- 172.21.15.40:0/84541169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2208005220 0x7f220800ba30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f221800c320 tx=0x7f22080058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a31291ea240ba01a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:08.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.404+0000 7f221d59a700 1 -- 172.21.15.40:0/84541169 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218001f20 con 0x7f2208005220 2023-11-08T08:21:08.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.404+0000 7f221d59a700 1 -- 172.21.15.40:0/84541169 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2218003a20 con 0x7f2208005220 2023-11-08T08:21:08.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.404+0000 7f221d59a700 1 -- 172.21.15.40:0/84541169 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221800cea0 con 0x7f2208005220 2023-11-08T08:21:08.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.404+0000 7f221f59e700 1 -- 172.21.15.40:0/84541169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2208005220 msgr2=0x7f220800ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:08.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.404+0000 7f221f59e700 1 --2- 172.21.15.40:0/84541169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2208005220 0x7f220800ba30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f221800c320 tx=0x7f22080058a0 comp rx=0 tx=0).stop 2023-11-08T08:21:08.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.404+0000 7f221ed9d700 1 -- 172.21.15.40:0/84541169 reap_dead start 2023-11-08T08:21:08.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.404+0000 7f221f59e700 1 -- 172.21.15.40:0/84541169 shutdown_connections 2023-11-08T08:21:08.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.404+0000 7f221f59e700 1 -- 172.21.15.40:0/84541169 >> 172.21.15.40:0/84541169 conn(0x7f22200b0f80 msgr2=0x7f22200ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:08.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.404+0000 7f221f59e700 1 -- 172.21.15.40:0/84541169 shutdown_connections 2023-11-08T08:21:08.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.404+0000 7f221f59e700 1 -- 172.21.15.40:0/84541169 wait complete. 2023-11-08T08:21:08.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.408+0000 7f221f59e700 1 Processor -- start 2023-11-08T08:21:08.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.408+0000 7f221f59e700 1 -- start start 2023-11-08T08:21:08.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.408+0000 7f221f59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2208005220 0x7f2220154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:08.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.408+0000 7f221f59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22200b1640 0x7f22200c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:08.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.408+0000 7f221f59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22200b2b50 0x7f222015a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:08.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.408+0000 7f221f59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22200b2730 con 0x7f22200b2b50 2023-11-08T08:21:08.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.408+0000 7f221f59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2220155320 con 0x7f2208005220 2023-11-08T08:21:08.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.408+0000 7f221f59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22200b1ab0 con 0x7f22200b1640 2023-11-08T08:21:08.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.408+0000 7f221ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22200b2b50 0x7f222015a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:08.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.408+0000 7f221ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22200b2b50 0x7f222015a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43218/0 (socket says 172.21.15.40:43218) 2023-11-08T08:21:08.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.408+0000 7f221ed9d700 1 -- 172.21.15.40:0/1470890006 learned_addr learned my addr 172.21.15.40:0/1470890006 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:08.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.408+0000 7f221e59c700 1 --2- 172.21.15.40:0/1470890006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2208005220 0x7f2220154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:08.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.408+0000 7f221dd9b700 1 --2- 172.21.15.40:0/1470890006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22200b1640 0x7f22200c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:08.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.408+0000 7f221ed9d700 1 -- 172.21.15.40:0/1470890006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22200b1640 msgr2=0x7f22200c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:08.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.408+0000 7f221ed9d700 1 --2- 172.21.15.40:0/1470890006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22200b1640 0x7f22200c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:08.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.408+0000 7f221ed9d700 1 -- 172.21.15.40:0/1470890006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2208005220 msgr2=0x7f2220154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:08.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.408+0000 7f221ed9d700 1 --2- 172.21.15.40:0/1470890006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2208005220 0x7f2220154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:08.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.408+0000 7f221ed9d700 1 -- 172.21.15.40:0/1470890006 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222015c570 con 0x7f22200b2b50 2023-11-08T08:21:08.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.408+0000 7f221dd9b700 1 --2- 172.21.15.40:0/1470890006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22200b1640 0x7f22200c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:08.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.408+0000 7f221ed9d700 1 --2- 172.21.15.40:0/1470890006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22200b2b50 0x7f222015a7e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f220800c210 tx=0x7f2218001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:08.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.412+0000 7f220effd700 1 -- 172.21.15.40:0/1470890006 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22180050d0 con 0x7f22200b2b50 2023-11-08T08:21:08.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.412+0000 7f220effd700 1 -- 172.21.15.40:0/1470890006 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2218005230 con 0x7f22200b2b50 2023-11-08T08:21:08.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.412+0000 7f220effd700 1 -- 172.21.15.40:0/1470890006 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22180055c0 con 0x7f22200b2b50 2023-11-08T08:21:08.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.412+0000 7f221f59e700 1 -- 172.21.15.40:0/1470890006 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22200cc780 con 0x7f22200b2b50 2023-11-08T08:21:08.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.412+0000 7f221f59e700 1 -- 172.21.15.40:0/1470890006 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2220070080 con 0x7f22200b2b50 2023-11-08T08:21:08.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.412+0000 7f221f59e700 1 -- 172.21.15.40:0/1470890006 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22200b2870 con 0x7f22200b2b50 2023-11-08T08:21:08.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.412+0000 7f220effd700 1 -- 172.21.15.40:0/1470890006 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f2218010050 con 0x7f22200b2b50 2023-11-08T08:21:08.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.416+0000 7f220effd700 1 --2- 172.21.15.40:0/1470890006 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2200064ee0 0x7f2200067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:08.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.416+0000 7f220effd700 1 -- 172.21.15.40:0/1470890006 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f2218083110 con 0x7f22200b2b50 2023-11-08T08:21:08.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.416+0000 7f221e59c700 1 --2- 172.21.15.40:0/1470890006 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2200064ee0 0x7f2200067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:08.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.416+0000 7f221e59c700 1 --2- 172.21.15.40:0/1470890006 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2200064ee0 0x7f2200067390 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2210001200 tx=0x7f2210001df0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:08.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.420+0000 7f220effd700 1 -- 172.21.15.40:0/1470890006 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f22200b2870 con 0x7f22200b2b50 2023-11-08T08:21:08.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.596+0000 7f221f59e700 1 -- 172.21.15.40:0/1470890006 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f22200b2870 con 0x7f22200b2b50 2023-11-08T08:21:08.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.604+0000 7f220effd700 1 -- 172.21.15.40:0/1470890006 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f22200b2870 con 0x7f22200b2b50 2023-11-08T08:21:08.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.608+0000 7f221f59e700 1 -- 172.21.15.40:0/1470890006 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2200064ee0 msgr2=0x7f2200067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:08.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.608+0000 7f221f59e700 1 --2- 172.21.15.40:0/1470890006 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2200064ee0 0x7f2200067390 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2210001200 tx=0x7f2210001df0 comp rx=0 tx=0).stop 2023-11-08T08:21:08.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.608+0000 7f221f59e700 1 -- 172.21.15.40:0/1470890006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22200b2b50 msgr2=0x7f222015a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:08.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.608+0000 7f221f59e700 1 --2- 172.21.15.40:0/1470890006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22200b2b50 0x7f222015a7e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f220800c210 tx=0x7f2218001650 comp rx=0 tx=0).stop 2023-11-08T08:21:08.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.608+0000 7f221f59e700 1 -- 172.21.15.40:0/1470890006 shutdown_connections 2023-11-08T08:21:08.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.608+0000 7f221f59e700 1 --2- 172.21.15.40:0/1470890006 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2200064ee0 0x7f2200067390 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:08.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.608+0000 7f221f59e700 1 --2- 172.21.15.40:0/1470890006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22200b2b50 0x7f222015a7e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:08.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.608+0000 7f221f59e700 1 --2- 172.21.15.40:0/1470890006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22200b1640 0x7f22200c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:08.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.608+0000 7f221f59e700 1 --2- 172.21.15.40:0/1470890006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2208005220 0x7f2220154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:08.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.608+0000 7f221f59e700 1 -- 172.21.15.40:0/1470890006 >> 172.21.15.40:0/1470890006 conn(0x7f22200b0f80 msgr2=0x7f22200ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:08.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.608+0000 7f221f59e700 1 -- 172.21.15.40:0/1470890006 shutdown_connections 2023-11-08T08:21:08.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.608+0000 7f221f59e700 1 -- 172.21.15.40:0/1470890006 wait complete. 2023-11-08T08:21:08.641 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-08T08:21:08.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.812+0000 7f53799a0700 1 Processor -- start 2023-11-08T08:21:08.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.812+0000 7f53799a0700 1 -- start start 2023-11-08T08:21:08.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.812+0000 7f53799a0700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f537415a480 0x7f53740cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:08.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.812+0000 7f53799a0700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5374069d60 0x7f537406a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:08.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.812+0000 7f53799a0700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537415a850 con 0x7f53740c8d10 2023-11-08T08:21:08.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.812+0000 7f53799a0700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53740ce060 con 0x7f5374069d60 2023-11-08T08:21:08.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.812+0000 7f53799a0700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53740ce300 con 0x7f537415a480 2023-11-08T08:21:08.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.812+0000 7f536a7fc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f53740c8d10 0x7f53740cc7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45102/0 (socket says 172.21.15.40:45102) 2023-11-08T08:21:08.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.812+0000 7f536a7fc700 1 -- 172.21.15.40:0/1547837551 learned_addr learned my addr 172.21.15.40:0/1547837551 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:08.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.812+0000 7f53737fe700 1 --2- 172.21.15.40:0/1547837551 >> v2:172.21.15.40:3300/0 conn(0x7f5374069d60 0x7f537406a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:08.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.812+0000 7f5372ffd700 1 --2- 172.21.15.40:0/1547837551 >> v2:172.21.15.169:3300/0 conn(0x7f537415a480 0x7f53740cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:08.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.812+0000 7f53737fe700 1 -- 172.21.15.40:0/1547837551 >> v2:172.21.15.169:3300/0 conn(0x7f537415a480 msgr2=0x7f53740cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:08.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.812+0000 7f53737fe700 1 --2- 172.21.15.40:0/1547837551 >> v2:172.21.15.169:3300/0 conn(0x7f537415a480 0x7f53740cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:08.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.812+0000 7f53737fe700 1 -- 172.21.15.40:0/1547837551 >> v1:172.21.15.40:6789/0 conn(0x7f53740c8d10 legacy=0x7f53740cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:08.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.812+0000 7f53737fe700 1 -- 172.21.15.40:0/1547837551 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53740cada0 con 0x7f5374069d60 2023-11-08T08:21:08.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.816+0000 7f53737fe700 1 --2- 172.21.15.40:0/1547837551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5374069d60 0x7f537406a1e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f53740c9390 tx=0x7f536c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892550e43d306620 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:08.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.816+0000 7f53727fc700 1 -- 172.21.15.40:0/1547837551 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c011830 con 0x7f5374069d60 2023-11-08T08:21:08.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.816+0000 7f53727fc700 1 -- 172.21.15.40:0/1547837551 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536c011990 con 0x7f5374069d60 2023-11-08T08:21:08.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.816+0000 7f53727fc700 1 -- 172.21.15.40:0/1547837551 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c011d40 con 0x7f5374069d60 2023-11-08T08:21:08.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.816+0000 7f53799a0700 1 -- 172.21.15.40:0/1547837551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5374069d60 msgr2=0x7f537406a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:08.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.816+0000 7f53799a0700 1 --2- 172.21.15.40:0/1547837551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5374069d60 0x7f537406a1e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f53740c9390 tx=0x7f536c009b30 comp rx=0 tx=0).stop 2023-11-08T08:21:08.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.816+0000 7f53799a0700 1 -- 172.21.15.40:0/1547837551 shutdown_connections 2023-11-08T08:21:08.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.816+0000 7f53799a0700 1 --2- 172.21.15.40:0/1547837551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5374069d60 0x7f537406a1e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:08.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.816+0000 7f53799a0700 1 --2- 172.21.15.40:0/1547837551 >> v2:172.21.15.169:3300/0 conn(0x7f537415a480 0x7f53740cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:08.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.816+0000 7f53799a0700 1 -- 172.21.15.40:0/1547837551 >> 172.21.15.40:0/1547837551 conn(0x7f53740cf350 msgr2=0x7f53740b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:08.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.816+0000 7f53799a0700 1 -- 172.21.15.40:0/1547837551 shutdown_connections 2023-11-08T08:21:08.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.816+0000 7f53799a0700 1 -- 172.21.15.40:0/1547837551 wait complete. 2023-11-08T08:21:08.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.820+0000 7f53799a0700 1 Processor -- start 2023-11-08T08:21:08.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.820+0000 7f53799a0700 1 -- start start 2023-11-08T08:21:08.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.820+0000 7f53799a0700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5374069d60 0x7f53740b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:08.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.820+0000 7f53799a0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53740c8d10 0x7f53740c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:08.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.820+0000 7f53799a0700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f537415a480 0x7f53740b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:08.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.820+0000 7f53799a0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5374156e90 con 0x7f53740c8d10 2023-11-08T08:21:08.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.820+0000 7f53799a0700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53740b2730 con 0x7f5374069d60 2023-11-08T08:21:08.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.820+0000 7f53799a0700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53740b2a10 con 0x7f537415a480 2023-11-08T08:21:08.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.820+0000 7f536a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53740c8d10 0x7f53740c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:08.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.820+0000 7f536a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53740c8d10 0x7f53740c4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43240/0 (socket says 172.21.15.40:43240) 2023-11-08T08:21:08.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.820+0000 7f536a7fc700 1 -- 172.21.15.40:0/3497130479 learned_addr learned my addr 172.21.15.40:0/3497130479 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:08.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.820+0000 7f5372ffd700 1 --2- 172.21.15.40:0/3497130479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5374069d60 0x7f53740b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:08.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.820+0000 7f53737fe700 1 --2- 172.21.15.40:0/3497130479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f537415a480 0x7f53740b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:08.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.820+0000 7f536a7fc700 1 -- 172.21.15.40:0/3497130479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f537415a480 msgr2=0x7f53740b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:08.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.820+0000 7f536a7fc700 1 --2- 172.21.15.40:0/3497130479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f537415a480 0x7f53740b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:08.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.820+0000 7f536a7fc700 1 -- 172.21.15.40:0/3497130479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5374069d60 msgr2=0x7f53740b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:08.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.820+0000 7f536a7fc700 1 --2- 172.21.15.40:0/3497130479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5374069d60 0x7f53740b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:08.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.824+0000 7f536a7fc700 1 -- 172.21.15.40:0/3497130479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c0094b0 con 0x7f53740c8d10 2023-11-08T08:21:08.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.824+0000 7f53737fe700 1 --2- 172.21.15.40:0/3497130479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f537415a480 0x7f53740b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:08.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.824+0000 7f536a7fc700 1 --2- 172.21.15.40:0/3497130479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53740c8d10 0x7f53740c4a40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f535c001eb0 tx=0x7f535c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:08.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.824+0000 7f5370ff9700 1 -- 172.21.15.40:0/3497130479 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c005d90 con 0x7f53740c8d10 2023-11-08T08:21:08.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.824+0000 7f5370ff9700 1 -- 172.21.15.40:0/3497130479 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535c0070d0 con 0x7f53740c8d10 2023-11-08T08:21:08.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.824+0000 7f53799a0700 1 -- 172.21.15.40:0/3497130479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53740add50 con 0x7f53740c8d10 2023-11-08T08:21:08.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.824+0000 7f5370ff9700 1 -- 172.21.15.40:0/3497130479 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c007440 con 0x7f53740c8d10 2023-11-08T08:21:08.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.824+0000 7f53799a0700 1 -- 172.21.15.40:0/3497130479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53740caa30 con 0x7f53740c8d10 2023-11-08T08:21:08.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.824+0000 7f53799a0700 1 -- 172.21.15.40:0/3497130479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53740ca6e0 con 0x7f53740c8d10 2023-11-08T08:21:08.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.828+0000 7f5370ff9700 1 -- 172.21.15.40:0/3497130479 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f535c019030 con 0x7f53740c8d10 2023-11-08T08:21:08.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.828+0000 7f5370ff9700 1 --2- 172.21.15.40:0/3497130479 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5354064ee0 0x7f5354067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:08.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.828+0000 7f5370ff9700 1 -- 172.21.15.40:0/3497130479 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f535c080f00 con 0x7f53740c8d10 2023-11-08T08:21:08.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.828+0000 7f5372ffd700 1 --2- 172.21.15.40:0/3497130479 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5354064ee0 0x7f5354067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:08.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.832+0000 7f5372ffd700 1 --2- 172.21.15.40:0/3497130479 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5354064ee0 0x7f5354067390 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5364003b50 tx=0x7f5364009830 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:08.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:08.836+0000 7f5370ff9700 1 -- 172.21.15.40:0/3497130479 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f535c04a400 con 0x7f53740c8d10 2023-11-08T08:21:09.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.080+0000 7f53799a0700 1 -- 172.21.15.40:0/3497130479 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f5374159630 con 0x7f53740c8d10 2023-11-08T08:21:09.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.184+0000 7f5370ff9700 1 -- 172.21.15.40:0/3497130479 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v138) v1 ==== 142+0+0 (secure 0 0 0) 0x7f535c04f240 con 0x7f53740c8d10 2023-11-08T08:21:09.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.192+0000 7f53799a0700 1 -- 172.21.15.40:0/3497130479 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5354064ee0 msgr2=0x7f5354067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:09.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.192+0000 7f53799a0700 1 --2- 172.21.15.40:0/3497130479 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5354064ee0 0x7f5354067390 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5364003b50 tx=0x7f5364009830 comp rx=0 tx=0).stop 2023-11-08T08:21:09.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.192+0000 7f53799a0700 1 -- 172.21.15.40:0/3497130479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53740c8d10 msgr2=0x7f53740c4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:09.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.192+0000 7f53799a0700 1 --2- 172.21.15.40:0/3497130479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53740c8d10 0x7f53740c4a40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f535c001eb0 tx=0x7f535c00c7b0 comp rx=0 tx=0).stop 2023-11-08T08:21:09.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.192+0000 7f53799a0700 1 -- 172.21.15.40:0/3497130479 shutdown_connections 2023-11-08T08:21:09.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.192+0000 7f53799a0700 1 --2- 172.21.15.40:0/3497130479 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5354064ee0 0x7f5354067390 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:09.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.192+0000 7f53799a0700 1 --2- 172.21.15.40:0/3497130479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f537415a480 0x7f53740b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:09.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.192+0000 7f53799a0700 1 --2- 172.21.15.40:0/3497130479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53740c8d10 0x7f53740c4a40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:09.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.192+0000 7f53799a0700 1 --2- 172.21.15.40:0/3497130479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5374069d60 0x7f53740b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:09.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.192+0000 7f53799a0700 1 -- 172.21.15.40:0/3497130479 >> 172.21.15.40:0/3497130479 conn(0x7f53740cf350 msgr2=0x7f53740cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:09.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.192+0000 7f53799a0700 1 -- 172.21.15.40:0/3497130479 shutdown_connections 2023-11-08T08:21:09.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.192+0000 7f53799a0700 1 -- 172.21.15.40:0/3497130479 wait complete. 2023-11-08T08:21:09.221 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2023-11-08T08:21:09.222 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2023-11-08T08:21:09.222 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-08T08:21:09.223 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-08T08:21:09.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.408+0000 7f6fd0a5d700 1 Processor -- start 2023-11-08T08:21:09.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.412+0000 7f6fd0a5d700 1 -- start start 2023-11-08T08:21:09.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.412+0000 7f6fd0a5d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6fcc0d0f40 0x7f6fcc0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:09.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.412+0000 7f6fd0a5d700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fcc0c89b0 con 0x7f6fcc0ccc20 2023-11-08T08:21:09.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.412+0000 7f6fd0a5d700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fcc070ca0 con 0x7f6fcc0ca660 2023-11-08T08:21:09.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.412+0000 7f6fd0a5d700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fcc0caa30 con 0x7f6fcc0d0f40 2023-11-08T08:21:09.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.412+0000 7f6fcad9d700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f6fcc0ca660 0x7f6fcc0d0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:55478/0 (socket says 172.21.15.40:55478) 2023-11-08T08:21:09.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.412+0000 7f6fca59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6fcc0d0f40 0x7f6fcc0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:09.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.412+0000 7f6fcad9d700 1 -- 172.21.15.40:0/1039326030 learned_addr learned my addr 172.21.15.40:0/1039326030 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:09.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.416+0000 7f6fc9d9b700 1 -- 172.21.15.40:0/1039326030 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 858764730 0 0) 0x7f6fcc070ca0 con 0x7f6fcc0ca660 2023-11-08T08:21:09.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.416+0000 7f6fc9d9b700 1 -- 172.21.15.40:0/1039326030 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fb40035d0 con 0x7f6fcc0ca660 2023-11-08T08:21:09.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.416+0000 7f6fca59c700 1 -- 172.21.15.40:0/1039326030 >> v1:172.21.15.169:6789/0 conn(0x7f6fcc0ca660 legacy=0x7f6fcc0d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:09.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.416+0000 7f6fc9d9b700 1 -- 172.21.15.40:0/1039326030 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390843073 0 0) 0x7f6fcc0c89b0 con 0x7f6fcc0ccc20 2023-11-08T08:21:09.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.416+0000 7f6fca59c700 1 -- 172.21.15.40:0/1039326030 >> v1:172.21.15.40:6789/0 conn(0x7f6fcc0ccc20 legacy=0x7f6fcc0cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:09.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.416+0000 7f6fca59c700 1 -- 172.21.15.40:0/1039326030 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fcc0ce540 con 0x7f6fcc0d0f40 2023-11-08T08:21:09.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.416+0000 7f6fca59c700 1 --2- 172.21.15.40:0/1039326030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fcc0d0f40 0x7f6fcc0705a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009850 tx=0x7f6fc0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107eeb8c37390d0a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:09.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.416+0000 7f6fc9d9b700 1 -- 172.21.15.40:0/1039326030 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0011830 con 0x7f6fcc0d0f40 2023-11-08T08:21:09.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.416+0000 7f6fc9d9b700 1 -- 172.21.15.40:0/1039326030 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc0011990 con 0x7f6fcc0d0f40 2023-11-08T08:21:09.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.416+0000 7f6fc9d9b700 1 -- 172.21.15.40:0/1039326030 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0011d40 con 0x7f6fcc0d0f40 2023-11-08T08:21:09.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.416+0000 7f6fd0a5d700 1 -- 172.21.15.40:0/1039326030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fcc0d0f40 msgr2=0x7f6fcc0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:09.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.416+0000 7f6fd0a5d700 1 --2- 172.21.15.40:0/1039326030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fcc0d0f40 0x7f6fcc0705a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009850 tx=0x7f6fc0009b60 comp rx=0 tx=0).stop 2023-11-08T08:21:09.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.416+0000 7f6fd0a5d700 1 -- 172.21.15.40:0/1039326030 shutdown_connections 2023-11-08T08:21:09.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.416+0000 7f6fd0a5d700 1 --2- 172.21.15.40:0/1039326030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fcc0d0f40 0x7f6fcc0705a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:09.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.416+0000 7f6fd0a5d700 1 -- 172.21.15.40:0/1039326030 >> 172.21.15.40:0/1039326030 conn(0x7f6fcc06f220 msgr2=0x7f6fcc06fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:09.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.416+0000 7f6fd0a5d700 1 -- 172.21.15.40:0/1039326030 shutdown_connections 2023-11-08T08:21:09.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.420+0000 7f6fd0a5d700 1 -- 172.21.15.40:0/1039326030 wait complete. 2023-11-08T08:21:09.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.420+0000 7f6fd0a5d700 1 Processor -- start 2023-11-08T08:21:09.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.420+0000 7f6fd0a5d700 1 -- start start 2023-11-08T08:21:09.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.420+0000 7f6fd0a5d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fcc0ca660 0x7f6fcc1590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:09.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.420+0000 7f6fd0a5d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fcc0ccc20 0x7f6fcc0ac420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:09.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.420+0000 7f6fd0a5d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6fcc0d0f40 0x7f6fcc06c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:09.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.420+0000 7f6fd0a5d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc0af3f0 con 0x7f6fcc0ccc20 2023-11-08T08:21:09.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.420+0000 7f6fd0a5d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc0c5a70 con 0x7f6fcc0d0f40 2023-11-08T08:21:09.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.420+0000 7f6fd0a5d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc0acee0 con 0x7f6fcc0ca660 2023-11-08T08:21:09.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.424+0000 7f6fca59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fcc0ccc20 0x7f6fcc0ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:09.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.424+0000 7f6fca59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fcc0ccc20 0x7f6fcc0ac420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43248/0 (socket says 172.21.15.40:43248) 2023-11-08T08:21:09.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.424+0000 7f6fca59c700 1 -- 172.21.15.40:0/494356829 learned_addr learned my addr 172.21.15.40:0/494356829 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:09.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.424+0000 7f6fcad9d700 1 --2- 172.21.15.40:0/494356829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fcc0ca660 0x7f6fcc1590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:09.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.424+0000 7f6fcb59e700 1 --2- 172.21.15.40:0/494356829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6fcc0d0f40 0x7f6fcc06c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:09.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.424+0000 7f6fca59c700 1 -- 172.21.15.40:0/494356829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fcc0ca660 msgr2=0x7f6fcc1590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:09.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.424+0000 7f6fca59c700 1 --2- 172.21.15.40:0/494356829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fcc0ca660 0x7f6fcc1590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:09.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.424+0000 7f6fca59c700 1 -- 172.21.15.40:0/494356829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6fcc0d0f40 msgr2=0x7f6fcc06c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:09.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.424+0000 7f6fca59c700 1 --2- 172.21.15.40:0/494356829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6fcc0d0f40 0x7f6fcc06c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:09.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.424+0000 7f6fca59c700 1 -- 172.21.15.40:0/494356829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc00094b0 con 0x7f6fcc0ccc20 2023-11-08T08:21:09.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.424+0000 7f6fcb59e700 1 --2- 172.21.15.40:0/494356829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6fcc0d0f40 0x7f6fcc06c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:09.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.424+0000 7f6fca59c700 1 --2- 172.21.15.40:0/494356829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fcc0ccc20 0x7f6fcc0ac420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0013000 tx=0x7f6fc0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:09.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.424+0000 7f6fb37fe700 1 -- 172.21.15.40:0/494356829 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc001ab70 con 0x7f6fcc0ccc20 2023-11-08T08:21:09.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.424+0000 7f6fb37fe700 1 -- 172.21.15.40:0/494356829 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc001f070 con 0x7f6fcc0ccc20 2023-11-08T08:21:09.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.424+0000 7f6fb37fe700 1 -- 172.21.15.40:0/494356829 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc001b430 con 0x7f6fcc0ccc20 2023-11-08T08:21:09.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.424+0000 7f6fd0a5d700 1 -- 172.21.15.40:0/494356829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fcc0ceb00 con 0x7f6fcc0ccc20 2023-11-08T08:21:09.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.424+0000 7f6fd0a5d700 1 -- 172.21.15.40:0/494356829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fcc0ce820 con 0x7f6fcc0ccc20 2023-11-08T08:21:09.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.424+0000 7f6fd0a5d700 1 -- 172.21.15.40:0/494356829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fcc0cede0 con 0x7f6fcc0ccc20 2023-11-08T08:21:09.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.428+0000 7f6fb37fe700 1 -- 172.21.15.40:0/494356829 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f6fc000f050 con 0x7f6fcc0ccc20 2023-11-08T08:21:09.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.428+0000 7f6fb37fe700 1 --2- 172.21.15.40:0/494356829 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6fb8064ee0 0x7f6fb8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.428+0000 7f6fb37fe700 1 -- 172.21.15.40:0/494356829 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f6fc0083390 con 0x7f6fcc0ccc20 2023-11-08T08:21:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.432+0000 7f6fcad9d700 1 --2- 172.21.15.40:0/494356829 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6fb8064ee0 0x7f6fb8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:09.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.432+0000 7f6fcad9d700 1 --2- 172.21.15.40:0/494356829 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6fb8064ee0 0x7f6fb8067390 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc005980 tx=0x7f6fbc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:09.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.436+0000 7f6fb37fe700 1 -- 172.21.15.40:0/494356829 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f6fc0051660 con 0x7f6fcc0ccc20 2023-11-08T08:21:09.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:09.672+0000 7f6fd0a5d700 1 -- 172.21.15.40:0/494356829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f6fcc073cb0 con 0x7f6fcc0ccc20 2023-11-08T08:21:10.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.196+0000 7f6fb37fe700 1 -- 172.21.15.40:0/494356829 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v139) v1 ==== 152+0+0 (secure 0 0 0) 0x7f6fc0054c40 con 0x7f6fcc0ccc20 2023-11-08T08:21:10.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.200+0000 7f6fd0a5d700 1 -- 172.21.15.40:0/494356829 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6fb8064ee0 msgr2=0x7f6fb8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:10.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.200+0000 7f6fd0a5d700 1 --2- 172.21.15.40:0/494356829 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6fb8064ee0 0x7f6fb8067390 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc005980 tx=0x7f6fbc009040 comp rx=0 tx=0).stop 2023-11-08T08:21:10.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.200+0000 7f6fd0a5d700 1 -- 172.21.15.40:0/494356829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fcc0ccc20 msgr2=0x7f6fcc0ac420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:10.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.200+0000 7f6fd0a5d700 1 --2- 172.21.15.40:0/494356829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fcc0ccc20 0x7f6fcc0ac420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0013000 tx=0x7f6fc0004770 comp rx=0 tx=0).stop 2023-11-08T08:21:10.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.200+0000 7f6fd0a5d700 1 -- 172.21.15.40:0/494356829 shutdown_connections 2023-11-08T08:21:10.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.200+0000 7f6fd0a5d700 1 --2- 172.21.15.40:0/494356829 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6fb8064ee0 0x7f6fb8067390 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:10.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.200+0000 7f6fd0a5d700 1 --2- 172.21.15.40:0/494356829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6fcc0d0f40 0x7f6fcc06c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:10.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.200+0000 7f6fd0a5d700 1 --2- 172.21.15.40:0/494356829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fcc0ccc20 0x7f6fcc0ac420 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:10.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.200+0000 7f6fd0a5d700 1 --2- 172.21.15.40:0/494356829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6fcc0ca660 0x7f6fcc1590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:10.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.200+0000 7f6fd0a5d700 1 -- 172.21.15.40:0/494356829 >> 172.21.15.40:0/494356829 conn(0x7f6fcc06f220 msgr2=0x7f6fcc0b6020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:10.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.200+0000 7f6fd0a5d700 1 -- 172.21.15.40:0/494356829 shutdown_connections 2023-11-08T08:21:10.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.200+0000 7f6fd0a5d700 1 -- 172.21.15.40:0/494356829 wait complete. 2023-11-08T08:21:10.234 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-08T08:21:10.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.424+0000 7f26d15ef700 1 Processor -- start 2023-11-08T08:21:10.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.424+0000 7f26d15ef700 1 -- start start 2023-11-08T08:21:10.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.428+0000 7f26d15ef700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f26cc070620 0x7f26cc0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:10.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.428+0000 7f26d15ef700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f26cc069bf0 0x7f26cc069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:10.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.428+0000 7f26d15ef700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26cc15c520 con 0x7f26cc15c150 2023-11-08T08:21:10.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.428+0000 7f26d15ef700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26cc0b4240 con 0x7f26cc069bf0 2023-11-08T08:21:10.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.428+0000 7f26d15ef700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26cc06cc50 con 0x7f26cc070620 2023-11-08T08:21:10.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.428+0000 7f26d0dee700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f26cc069bf0 0x7f26cc069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:10.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.428+0000 7f26d0dee700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f26cc069bf0 0x7f26cc069ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43262/0 (socket says 172.21.15.40:43262) 2023-11-08T08:21:10.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.428+0000 7f26d0dee700 1 -- 172.21.15.40:0/636318714 learned_addr learned my addr 172.21.15.40:0/636318714 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:10.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.428+0000 7f26cb7fe700 1 --2- 172.21.15.40:0/636318714 >> v2:172.21.15.169:3300/0 conn(0x7f26cc070620 0x7f26cc0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:10.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.428+0000 7f26d0dee700 1 -- 172.21.15.40:0/636318714 >> v2:172.21.15.169:3300/0 conn(0x7f26cc070620 msgr2=0x7f26cc0b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:10.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.428+0000 7f26d0dee700 1 --2- 172.21.15.40:0/636318714 >> v2:172.21.15.169:3300/0 conn(0x7f26cc070620 0x7f26cc0b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:10.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.428+0000 7f26d0dee700 1 -- 172.21.15.40:0/636318714 >> v1:172.21.15.169:6789/0 conn(0x7f26cc15c150 legacy=0x7f26cc06ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:10.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.428+0000 7f26d0dee700 1 -- 172.21.15.40:0/636318714 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26cc0d2450 con 0x7f26cc069bf0 2023-11-08T08:21:10.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.428+0000 7f26d0dee700 1 --2- 172.21.15.40:0/636318714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26cc069bf0 0x7f26cc069ff0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f26cc066d30 tx=0x7f26c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc78219f33fac2f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.428+0000 7f26caffd700 1 -- 172.21.15.40:0/636318714 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c400e070 con 0x7f26cc069bf0 2023-11-08T08:21:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.428+0000 7f26caffd700 1 -- 172.21.15.40:0/636318714 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c400b910 con 0x7f26cc069bf0 2023-11-08T08:21:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.428+0000 7f26caffd700 1 -- 172.21.15.40:0/636318714 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c400bcc0 con 0x7f26cc069bf0 2023-11-08T08:21:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.432+0000 7f26d15ef700 1 -- 172.21.15.40:0/636318714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26cc069bf0 msgr2=0x7f26cc069ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.432+0000 7f26d15ef700 1 --2- 172.21.15.40:0/636318714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26cc069bf0 0x7f26cc069ff0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f26cc066d30 tx=0x7f26c400b330 comp rx=0 tx=0).stop 2023-11-08T08:21:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.432+0000 7f26d15ef700 1 -- 172.21.15.40:0/636318714 shutdown_connections 2023-11-08T08:21:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.432+0000 7f26d15ef700 1 --2- 172.21.15.40:0/636318714 >> v2:172.21.15.169:3300/0 conn(0x7f26cc070620 0x7f26cc0b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.432+0000 7f26d15ef700 1 --2- 172.21.15.40:0/636318714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26cc069bf0 0x7f26cc069ff0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.432+0000 7f26d15ef700 1 -- 172.21.15.40:0/636318714 >> 172.21.15.40:0/636318714 conn(0x7f26cc0d3680 msgr2=0x7f26cc0cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.432+0000 7f26d15ef700 1 -- 172.21.15.40:0/636318714 shutdown_connections 2023-11-08T08:21:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.432+0000 7f26d15ef700 1 -- 172.21.15.40:0/636318714 wait complete. 2023-11-08T08:21:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.436+0000 7f26d15ef700 1 Processor -- start 2023-11-08T08:21:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.436+0000 7f26d15ef700 1 -- start start 2023-11-08T08:21:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.436+0000 7f26d15ef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26cc069bf0 0x7f26cc1ef030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.436+0000 7f26d15ef700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26cc070620 0x7f26cc1f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.436+0000 7f26d15ef700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26cc15c150 0x7f26cc1501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.436+0000 7f26d15ef700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc0b6b00 con 0x7f26cc069bf0 2023-11-08T08:21:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.436+0000 7f26d15ef700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc1f2a00 con 0x7f26cc070620 2023-11-08T08:21:10.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.436+0000 7f26d15ef700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc0b5ef0 con 0x7f26cc15c150 2023-11-08T08:21:10.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.436+0000 7f26cbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26cc069bf0 0x7f26cc1ef030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:10.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.436+0000 7f26cbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26cc069bf0 0x7f26cc1ef030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43276/0 (socket says 172.21.15.40:43276) 2023-11-08T08:21:10.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.436+0000 7f26cbfff700 1 -- 172.21.15.40:0/761596450 learned_addr learned my addr 172.21.15.40:0/761596450 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.436+0000 7f26cb7fe700 1 --2- 172.21.15.40:0/761596450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26cc070620 0x7f26cc1f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.436+0000 7f26d0dee700 1 --2- 172.21.15.40:0/761596450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26cc15c150 0x7f26cc1501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.436+0000 7f26cbfff700 1 -- 172.21.15.40:0/761596450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26cc15c150 msgr2=0x7f26cc1501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.436+0000 7f26cbfff700 1 --2- 172.21.15.40:0/761596450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26cc15c150 0x7f26cc1501a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.436+0000 7f26cbfff700 1 -- 172.21.15.40:0/761596450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26cc070620 msgr2=0x7f26cc1f24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.436+0000 7f26cbfff700 1 --2- 172.21.15.40:0/761596450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26cc070620 0x7f26cc1f24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.436+0000 7f26cbfff700 1 -- 172.21.15.40:0/761596450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c4009b30 con 0x7f26cc069bf0 2023-11-08T08:21:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.440+0000 7f26cbfff700 1 --2- 172.21.15.40:0/761596450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26cc069bf0 0x7f26cc1ef030 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00c440 tx=0x7f26bc00c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.440+0000 7f26c8ff9700 1 -- 172.21.15.40:0/761596450 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc0052d0 con 0x7f26cc069bf0 2023-11-08T08:21:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.440+0000 7f26c8ff9700 1 -- 172.21.15.40:0/761596450 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26bc005430 con 0x7f26cc069bf0 2023-11-08T08:21:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.440+0000 7f26d15ef700 1 -- 172.21.15.40:0/761596450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26cc0d05e0 con 0x7f26cc069bf0 2023-11-08T08:21:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.440+0000 7f26c8ff9700 1 -- 172.21.15.40:0/761596450 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc0055a0 con 0x7f26cc069bf0 2023-11-08T08:21:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.440+0000 7f26d15ef700 1 -- 172.21.15.40:0/761596450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26cc0d2730 con 0x7f26cc069bf0 2023-11-08T08:21:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.440+0000 7f26d15ef700 1 -- 172.21.15.40:0/761596450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26cc0d2140 con 0x7f26cc069bf0 2023-11-08T08:21:10.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.444+0000 7f26c8ff9700 1 -- 172.21.15.40:0/761596450 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f26bc010050 con 0x7f26cc069bf0 2023-11-08T08:21:10.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.444+0000 7f26c8ff9700 1 --2- 172.21.15.40:0/761596450 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f26b8064ee0 0x7f26b8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:10.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.444+0000 7f26c8ff9700 1 -- 172.21.15.40:0/761596450 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f26bc080d90 con 0x7f26cc069bf0 2023-11-08T08:21:10.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.444+0000 7f26cb7fe700 1 --2- 172.21.15.40:0/761596450 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f26b8064ee0 0x7f26b8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.448+0000 7f26cb7fe700 1 --2- 172.21.15.40:0/761596450 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f26b8064ee0 0x7f26b8067390 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f26cc150cf0 tx=0x7f26c00091c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:10.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.448+0000 7f26c8ff9700 1 -- 172.21.15.40:0/761596450 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f26bc049400 con 0x7f26cc069bf0 2023-11-08T08:21:10.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.624+0000 7f26d15ef700 1 -- 172.21.15.40:0/761596450 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f26cc073d20 con 0x7f26cc069bf0 2023-11-08T08:21:10.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.632+0000 7f26c8ff9700 1 -- 172.21.15.40:0/761596450 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f26bc04f020 con 0x7f26cc069bf0 2023-11-08T08:21:10.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.636+0000 7f26d15ef700 1 -- 172.21.15.40:0/761596450 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f26b8064ee0 msgr2=0x7f26b8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:10.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.636+0000 7f26d15ef700 1 --2- 172.21.15.40:0/761596450 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f26b8064ee0 0x7f26b8067390 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f26cc150cf0 tx=0x7f26c00091c0 comp rx=0 tx=0).stop 2023-11-08T08:21:10.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.636+0000 7f26d15ef700 1 -- 172.21.15.40:0/761596450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26cc069bf0 msgr2=0x7f26cc1ef030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:10.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.636+0000 7f26d15ef700 1 --2- 172.21.15.40:0/761596450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26cc069bf0 0x7f26cc1ef030 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00c440 tx=0x7f26bc00c750 comp rx=0 tx=0).stop 2023-11-08T08:21:10.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.636+0000 7f26d15ef700 1 -- 172.21.15.40:0/761596450 shutdown_connections 2023-11-08T08:21:10.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.636+0000 7f26d15ef700 1 --2- 172.21.15.40:0/761596450 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f26b8064ee0 0x7f26b8067390 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:10.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.636+0000 7f26d15ef700 1 --2- 172.21.15.40:0/761596450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26cc15c150 0x7f26cc1501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:10.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.636+0000 7f26d15ef700 1 --2- 172.21.15.40:0/761596450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26cc070620 0x7f26cc1f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:10.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.636+0000 7f26d15ef700 1 --2- 172.21.15.40:0/761596450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26cc069bf0 0x7f26cc1ef030 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:10.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.636+0000 7f26d15ef700 1 -- 172.21.15.40:0/761596450 >> 172.21.15.40:0/761596450 conn(0x7f26cc0d3680 msgr2=0x7f26cc0b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:10.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.636+0000 7f26d15ef700 1 -- 172.21.15.40:0/761596450 shutdown_connections 2023-11-08T08:21:10.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:10.640+0000 7f26d15ef700 1 -- 172.21.15.40:0/761596450 wait complete. 2023-11-08T08:21:10.666 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T08:21:10.666 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T08:21:10.705 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T08:21:10.752 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T08:21:10.753 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:21:10.753 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:21:10.774 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16355412k,nr_inodes=4088853,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6645 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T08:21:10.774 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:21:10.775 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:21:10.815 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352316k,nr_inodes=4088079,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1804 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T08:21:10.816 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:21:10.816 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:21:10.839 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16355412k,nr_inodes=4088853,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6645 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T08:21:10.839 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:21:10.840 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:21:10.883 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352316k,nr_inodes=4088079,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1804 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T08:21:10.885 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-08T08:21:10.885 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:21:11.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.064+0000 7f77e0b56700 1 Processor -- start 2023-11-08T08:21:11.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.068+0000 7f77e0b56700 1 -- start start 2023-11-08T08:21:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.068+0000 7f77e0b56700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77dc15c1a0 0x7f77dc06c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.068+0000 7f77e0b56700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f77dc070790 0x7f77dc06a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.068+0000 7f77e0b56700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77dc0adf90 con 0x7f77dc0cdeb0 2023-11-08T08:21:11.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.068+0000 7f77e0b56700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77dc069bf0 con 0x7f77dc070790 2023-11-08T08:21:11.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.068+0000 7f77e0b56700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc0ce370 con 0x7f77dc15c1a0 2023-11-08T08:21:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77dbfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f77dc070790 0x7f77dc06a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77db7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77dc15c1a0 0x7f77dc06c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77d3fff700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f77dc0cdeb0 0x7f77dc070080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:55488/0 (socket says 172.21.15.40:55488) 2023-11-08T08:21:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77d3fff700 1 -- 172.21.15.40:0/4073424895 learned_addr learned my addr 172.21.15.40:0/4073424895 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:11.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77dbfff700 1 -- 172.21.15.40:0/4073424895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77dc15c1a0 msgr2=0x7f77dc06c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:11.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77dbfff700 1 --2- 172.21.15.40:0/4073424895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77dc15c1a0 0x7f77dc06c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:11.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77dbfff700 1 -- 172.21.15.40:0/4073424895 >> v1:172.21.15.169:6789/0 conn(0x7f77dc0cdeb0 legacy=0x7f77dc070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:11.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77dbfff700 1 -- 172.21.15.40:0/4073424895 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc0cc4a0 con 0x7f77dc070790 2023-11-08T08:21:11.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77daffd700 1 -- 172.21.15.40:0/4073424895 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2468960523 0 0) 0x7f77dc0adf90 con 0x7f77dc0cdeb0 2023-11-08T08:21:11.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77dbfff700 1 --2- 172.21.15.40:0/4073424895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77dc070790 0x7f77dc06a010 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009d90 tx=0x7f77d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51456d9e052ae3b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:11.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77daffd700 1 -- 172.21.15.40:0/4073424895 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d400e070 con 0x7f77dc070790 2023-11-08T08:21:11.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77daffd700 1 -- 172.21.15.40:0/4073424895 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d400bb20 con 0x7f77dc070790 2023-11-08T08:21:11.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77daffd700 1 -- 172.21.15.40:0/4073424895 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d4012430 con 0x7f77dc070790 2023-11-08T08:21:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77e0b56700 1 -- 172.21.15.40:0/4073424895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77dc070790 msgr2=0x7f77dc06a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77e0b56700 1 --2- 172.21.15.40:0/4073424895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77dc070790 0x7f77dc06a010 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009d90 tx=0x7f77d400b330 comp rx=0 tx=0).stop 2023-11-08T08:21:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77e0b56700 1 -- 172.21.15.40:0/4073424895 shutdown_connections 2023-11-08T08:21:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77e0b56700 1 --2- 172.21.15.40:0/4073424895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77dc070790 0x7f77dc06a010 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77e0b56700 1 --2- 172.21.15.40:0/4073424895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77dc15c1a0 0x7f77dc06c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:11.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77e0b56700 1 -- 172.21.15.40:0/4073424895 >> 172.21.15.40:0/4073424895 conn(0x7f77dc0b0f80 msgr2=0x7f77dc0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:11.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77e0b56700 1 -- 172.21.15.40:0/4073424895 shutdown_connections 2023-11-08T08:21:11.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.072+0000 7f77e0b56700 1 -- 172.21.15.40:0/4073424895 wait complete. 2023-11-08T08:21:11.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.076+0000 7f77e0b56700 1 Processor -- start 2023-11-08T08:21:11.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.076+0000 7f77e0b56700 1 -- start start 2023-11-08T08:21:11.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.076+0000 7f77e0b56700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77dc070790 0x7f77dc0ab860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:11.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.076+0000 7f77e0b56700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77dc0cdeb0 0x7f77dc152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:11.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.076+0000 7f77e0b56700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77dc15c1a0 0x7f77dc15ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:11.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.076+0000 7f77e0b56700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc0abef0 con 0x7f77dc070790 2023-11-08T08:21:11.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.076+0000 7f77e0b56700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc0ac1d0 con 0x7f77dc15c1a0 2023-11-08T08:21:11.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.076+0000 7f77e0b56700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc0ac4b0 con 0x7f77dc0cdeb0 2023-11-08T08:21:11.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.076+0000 7f77db7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77dc070790 0x7f77dc0ab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:11.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.080+0000 7f77db7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77dc070790 0x7f77dc0ab860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43296/0 (socket says 172.21.15.40:43296) 2023-11-08T08:21:11.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.080+0000 7f77db7fe700 1 -- 172.21.15.40:0/1910540095 learned_addr learned my addr 172.21.15.40:0/1910540095 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:11.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.080+0000 7f77d3fff700 1 --2- 172.21.15.40:0/1910540095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77dc0cdeb0 0x7f77dc152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:11.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.080+0000 7f77dbfff700 1 --2- 172.21.15.40:0/1910540095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77dc15c1a0 0x7f77dc15ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:11.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.080+0000 7f77db7fe700 1 -- 172.21.15.40:0/1910540095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77dc0cdeb0 msgr2=0x7f77dc152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:11.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.080+0000 7f77db7fe700 1 --2- 172.21.15.40:0/1910540095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77dc0cdeb0 0x7f77dc152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:11.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.080+0000 7f77db7fe700 1 -- 172.21.15.40:0/1910540095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77dc15c1a0 msgr2=0x7f77dc15ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:11.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.080+0000 7f77db7fe700 1 --2- 172.21.15.40:0/1910540095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77dc15c1a0 0x7f77dc15ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:11.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.080+0000 7f77db7fe700 1 -- 172.21.15.40:0/1910540095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d4009a70 con 0x7f77dc070790 2023-11-08T08:21:11.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.080+0000 7f77dbfff700 1 --2- 172.21.15.40:0/1910540095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77dc15c1a0 0x7f77dc15ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:11.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.080+0000 7f77db7fe700 1 --2- 172.21.15.40:0/1910540095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77dc070790 0x7f77dc0ab860 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00e840 tx=0x7f77cc00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:11.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.080+0000 7f77d8ff9700 1 -- 172.21.15.40:0/1910540095 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc0095a0 con 0x7f77dc070790 2023-11-08T08:21:11.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.080+0000 7f77d8ff9700 1 -- 172.21.15.40:0/1910540095 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77cc00c690 con 0x7f77dc070790 2023-11-08T08:21:11.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.080+0000 7f77e0b56700 1 -- 172.21.15.40:0/1910540095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77dc06cdf0 con 0x7f77dc070790 2023-11-08T08:21:11.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.080+0000 7f77d8ff9700 1 -- 172.21.15.40:0/1910540095 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc00ca00 con 0x7f77dc070790 2023-11-08T08:21:11.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.080+0000 7f77e0b56700 1 -- 172.21.15.40:0/1910540095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77dc0cc780 con 0x7f77dc070790 2023-11-08T08:21:11.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.080+0000 7f77e0b56700 1 -- 172.21.15.40:0/1910540095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77dc15c570 con 0x7f77dc070790 2023-11-08T08:21:11.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.084+0000 7f77d8ff9700 1 -- 172.21.15.40:0/1910540095 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f77cc019030 con 0x7f77dc070790 2023-11-08T08:21:11.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.084+0000 7f77d8ff9700 1 --2- 172.21.15.40:0/1910540095 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f77bc064ee0 0x7f77bc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:11.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.084+0000 7f77d8ff9700 1 -- 172.21.15.40:0/1910540095 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f77cc081c40 con 0x7f77dc070790 2023-11-08T08:21:11.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.084+0000 7f77d3fff700 1 --2- 172.21.15.40:0/1910540095 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f77bc064ee0 0x7f77bc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:11.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.088+0000 7f77d3fff700 1 --2- 172.21.15.40:0/1910540095 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f77bc064ee0 0x7f77bc067390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0c1e30 tx=0x7f77c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:11.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.088+0000 7f77d8ff9700 1 -- 172.21.15.40:0/1910540095 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f77cc04fed0 con 0x7f77dc070790 2023-11-08T08:21:11.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.300+0000 7f77e0b56700 1 -- 172.21.15.40:0/1910540095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77dc073cb0 con 0x7f77dc070790 2023-11-08T08:21:11.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.300+0000 7f77d8ff9700 1 -- 172.21.15.40:0/1910540095 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f77cc017de0 con 0x7f77dc070790 2023-11-08T08:21:11.307 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:11.307 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4958,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/2244078655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2244078655},{"type":"v1","addr":"172.21.15.40:6834","nonce":2244078655}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/749537066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":749537066},{"type":"v1","addr":"172.21.15.169:6836","nonce":749537066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5328,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2307838686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2307838686},{"type":"v1","addr":"172.21.15.169:6837","nonce":2307838686}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5700,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3819586020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3819586020},{"type":"v1","addr":"172.21.15.40:6835","nonce":3819586020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2023-11-08T08:21:11.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.304+0000 7f77e0b56700 1 -- 172.21.15.40:0/1910540095 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f77bc064ee0 msgr2=0x7f77bc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:11.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.304+0000 7f77e0b56700 1 --2- 172.21.15.40:0/1910540095 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f77bc064ee0 0x7f77bc067390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0c1e30 tx=0x7f77c4009040 comp rx=0 tx=0).stop 2023-11-08T08:21:11.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.304+0000 7f77e0b56700 1 -- 172.21.15.40:0/1910540095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77dc070790 msgr2=0x7f77dc0ab860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:11.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.304+0000 7f77e0b56700 1 --2- 172.21.15.40:0/1910540095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77dc070790 0x7f77dc0ab860 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00e840 tx=0x7f77cc00ec00 comp rx=0 tx=0).stop 2023-11-08T08:21:11.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.304+0000 7f77e0b56700 1 -- 172.21.15.40:0/1910540095 shutdown_connections 2023-11-08T08:21:11.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.304+0000 7f77e0b56700 1 --2- 172.21.15.40:0/1910540095 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f77bc064ee0 0x7f77bc067390 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:11.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.304+0000 7f77e0b56700 1 --2- 172.21.15.40:0/1910540095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77dc15c1a0 0x7f77dc15ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:11.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.304+0000 7f77e0b56700 1 --2- 172.21.15.40:0/1910540095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77dc0cdeb0 0x7f77dc152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:11.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.304+0000 7f77e0b56700 1 --2- 172.21.15.40:0/1910540095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77dc070790 0x7f77dc0ab860 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:11.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.304+0000 7f77e0b56700 1 -- 172.21.15.40:0/1910540095 >> 172.21.15.40:0/1910540095 conn(0x7f77dc0b0f80 msgr2=0x7f77dc155ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:11.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.304+0000 7f77e0b56700 1 -- 172.21.15.40:0/1910540095 shutdown_connections 2023-11-08T08:21:11.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.304+0000 7f77e0b56700 1 -- 172.21.15.40:0/1910540095 wait complete. 2023-11-08T08:21:11.313 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 19 2023-11-08T08:21:11.334 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T08:21:11.334 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T08:21:11.335 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T08:21:11.335 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:21:11.335 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T08:21:11.335 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T08:21:11.336 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T08:21:11.336 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:21:11.336 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T08:21:11.337 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T08:21:11.337 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T08:21:11.337 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:21:11.337 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T08:21:11.338 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T08:21:11.338 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T08:21:11.338 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:21:11.354 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T08:21:11.354 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T08:21:11.363 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T08:21:11.363 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T08:21:11.364 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T08:21:11.365 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T08:21:11.365 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T08:21:11.366 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T08:21:11.414 INFO:tasks.ceph.mds.b:Started 2023-11-08T08:21:11.414 DEBUG:teuthology.parallel:result is None 2023-11-08T08:21:11.415 INFO:tasks.ceph.mds.a:Started 2023-11-08T08:21:11.415 DEBUG:teuthology.parallel:result is None 2023-11-08T08:21:11.419 INFO:tasks.ceph.mds.d:Started 2023-11-08T08:21:11.419 DEBUG:teuthology.parallel:result is None 2023-11-08T08:21:11.420 INFO:tasks.ceph.mds.c:Started 2023-11-08T08:21:11.420 DEBUG:teuthology.parallel:result is None 2023-11-08T08:21:11.420 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-08T08:21:11.534 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-11-08T08:21:11.534 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2023-11-08T08:21:11.539 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2023-11-08T08:21:11.547 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2023-11-08T08:21:11.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.644+0000 7f2719e98700 1 Processor -- start 2023-11-08T08:21:11.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.644+0000 7f2719e98700 1 -- start start 2023-11-08T08:21:11.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.644+0000 7f2719e98700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f271415c640 0x7f27140d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:11.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.644+0000 7f2719e98700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2714069e00 0x7f271406a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:11.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.644+0000 7f2719e98700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f271406c600 con 0x7f2714071aa0 2023-11-08T08:21:11.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.644+0000 7f2719e98700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f271406cef0 con 0x7f2714069e00 2023-11-08T08:21:11.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.644+0000 7f2719e98700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271406a7c0 con 0x7f271415c640 2023-11-08T08:21:11.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f2719697700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2714069e00 0x7f271406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:11.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f2713fff700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f2714071aa0 0x7f271406d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:55518/0 (socket says 172.21.15.40:55518) 2023-11-08T08:21:11.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f2713fff700 1 -- 172.21.15.40:0/1127903283 learned_addr learned my addr 172.21.15.40:0/1127903283 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:11.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f2719697700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2714069e00 0x7f271406a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43312/0 (socket says 172.21.15.40:43312) 2023-11-08T08:21:11.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f2718e96700 1 --2- 172.21.15.40:0/1127903283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f271415c640 0x7f27140d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:11.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f2719697700 1 -- 172.21.15.40:0/1127903283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f271415c640 msgr2=0x7f27140d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:11.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f2719697700 1 --2- 172.21.15.40:0/1127903283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f271415c640 0x7f27140d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:11.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f2719697700 1 -- 172.21.15.40:0/1127903283 >> v1:172.21.15.169:6789/0 conn(0x7f2714071aa0 legacy=0x7f271406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:11.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f2719697700 1 -- 172.21.15.40:0/1127903283 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27140722f0 con 0x7f2714069e00 2023-11-08T08:21:11.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f2719697700 1 --2- 172.21.15.40:0/1127903283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2714069e00 0x7f271406a280 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f270c009d90 tx=0x7f270c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb610c63f3682203 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:11.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f27137fe700 1 -- 172.21.15.40:0/1127903283 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c00e070 con 0x7f2714069e00 2023-11-08T08:21:11.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f27137fe700 1 -- 172.21.15.40:0/1127903283 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270c00bb20 con 0x7f2714069e00 2023-11-08T08:21:11.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f27137fe700 1 -- 172.21.15.40:0/1127903283 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c012430 con 0x7f2714069e00 2023-11-08T08:21:11.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f2719e98700 1 -- 172.21.15.40:0/1127903283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2714069e00 msgr2=0x7f271406a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:11.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f2719e98700 1 --2- 172.21.15.40:0/1127903283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2714069e00 0x7f271406a280 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f270c009d90 tx=0x7f270c00b330 comp rx=0 tx=0).stop 2023-11-08T08:21:11.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f2719e98700 1 -- 172.21.15.40:0/1127903283 shutdown_connections 2023-11-08T08:21:11.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f2719e98700 1 --2- 172.21.15.40:0/1127903283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2714069e00 0x7f271406a280 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:11.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f2719e98700 1 --2- 172.21.15.40:0/1127903283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f271415c640 0x7f27140d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:11.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f2719e98700 1 -- 172.21.15.40:0/1127903283 >> 172.21.15.40:0/1127903283 conn(0x7f27140b6520 msgr2=0x7f2714071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:11.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f2719e98700 1 -- 172.21.15.40:0/1127903283 shutdown_connections 2023-11-08T08:21:11.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.648+0000 7f2719e98700 1 -- 172.21.15.40:0/1127903283 wait complete. 2023-11-08T08:21:11.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.652+0000 7f2719e98700 1 Processor -- start 2023-11-08T08:21:11.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.652+0000 7f2719e98700 1 -- start start 2023-11-08T08:21:11.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.652+0000 7f2719e98700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2714069e00 0x7f271409bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:11.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.652+0000 7f2719e98700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2714071aa0 0x7f27140b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:11.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.652+0000 7f2719e98700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f271415c640 0x7f2714159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:11.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.652+0000 7f2719e98700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27140b44d0 con 0x7f2714071aa0 2023-11-08T08:21:11.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.652+0000 7f2719e98700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27140b47b0 con 0x7f2714069e00 2023-11-08T08:21:11.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.652+0000 7f2719e98700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27140b4a90 con 0x7f271415c640 2023-11-08T08:21:11.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.652+0000 7f2713fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2714071aa0 0x7f27140b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:11.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.652+0000 7f2718e96700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2714069e00 0x7f271409bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:11.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.652+0000 7f2713fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2714071aa0 0x7f27140b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43324/0 (socket says 172.21.15.40:43324) 2023-11-08T08:21:11.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.652+0000 7f2713fff700 1 -- 172.21.15.40:0/2195627685 learned_addr learned my addr 172.21.15.40:0/2195627685 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:11.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.652+0000 7f2718e96700 1 --2- 172.21.15.40:0/2195627685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2714069e00 0x7f271409bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:36870/0 (socket says 172.21.15.40:36870) 2023-11-08T08:21:11.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.652+0000 7f2719697700 1 --2- 172.21.15.40:0/2195627685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f271415c640 0x7f2714159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:11.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.656+0000 7f2713fff700 1 -- 172.21.15.40:0/2195627685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f271415c640 msgr2=0x7f2714159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:11.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.656+0000 7f2713fff700 1 --2- 172.21.15.40:0/2195627685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f271415c640 0x7f2714159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:11.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.656+0000 7f2713fff700 1 -- 172.21.15.40:0/2195627685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2714069e00 msgr2=0x7f271409bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:11.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.656+0000 7f2713fff700 1 --2- 172.21.15.40:0/2195627685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2714069e00 0x7f271409bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:11.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.656+0000 7f2713fff700 1 -- 172.21.15.40:0/2195627685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270c009a70 con 0x7f2714071aa0 2023-11-08T08:21:11.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.656+0000 7f2713fff700 1 --2- 172.21.15.40:0/2195627685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2714071aa0 0x7f27140b3a80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2700001eb0 tx=0x7f270000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:11.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.656+0000 7f27117fa700 1 -- 172.21.15.40:0/2195627685 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700005d90 con 0x7f2714071aa0 2023-11-08T08:21:11.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.656+0000 7f27117fa700 1 -- 172.21.15.40:0/2195627685 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27000070d0 con 0x7f2714071aa0 2023-11-08T08:21:11.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.656+0000 7f27117fa700 1 -- 172.21.15.40:0/2195627685 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700007440 con 0x7f2714071aa0 2023-11-08T08:21:11.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.656+0000 7f2719e98700 1 -- 172.21.15.40:0/2195627685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f271415ca10 con 0x7f2714071aa0 2023-11-08T08:21:11.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.656+0000 7f2719e98700 1 -- 172.21.15.40:0/2195627685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f271406da20 con 0x7f2714071aa0 2023-11-08T08:21:11.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.656+0000 7f2719e98700 1 -- 172.21.15.40:0/2195627685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2714072010 con 0x7f2714071aa0 2023-11-08T08:21:11.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.660+0000 7f27117fa700 1 -- 172.21.15.40:0/2195627685 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f2700019030 con 0x7f2714071aa0 2023-11-08T08:21:11.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.660+0000 7f27117fa700 1 --2- 172.21.15.40:0/2195627685 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f26f8064ee0 0x7f26f8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:11.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.660+0000 7f27117fa700 1 -- 172.21.15.40:0/2195627685 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f2700080e70 con 0x7f2714071aa0 2023-11-08T08:21:11.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.660+0000 7f2718e96700 1 --2- 172.21.15.40:0/2195627685 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f26f8064ee0 0x7f26f8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:11.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.660+0000 7f2718e96700 1 --2- 172.21.15.40:0/2195627685 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f26f8064ee0 0x7f26f8067390 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f27140715b0 tx=0x7f2708009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:11.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.664+0000 7f27117fa700 1 -- 172.21.15.40:0/2195627685 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f270004a400 con 0x7f2714071aa0 2023-11-08T08:21:11.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:11.876+0000 7f2719e98700 1 -- 172.21.15.40:0/2195627685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2714069bf0 con 0x7f2714071aa0 2023-11-08T08:21:12.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:12.820+0000 7f27117fa700 1 -- 172.21.15.40:0/2195627685 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v53) v1 ==== 97+0+0 (secure 0 0 0) 0x7f270004f1b0 con 0x7f2714071aa0 2023-11-08T08:21:12.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:12.828+0000 7f2719e98700 1 -- 172.21.15.40:0/2195627685 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f26f8064ee0 msgr2=0x7f26f8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:12.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:12.828+0000 7f2719e98700 1 --2- 172.21.15.40:0/2195627685 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f26f8064ee0 0x7f26f8067390 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f27140715b0 tx=0x7f2708009240 comp rx=0 tx=0).stop 2023-11-08T08:21:12.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:12.828+0000 7f2719e98700 1 -- 172.21.15.40:0/2195627685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2714071aa0 msgr2=0x7f27140b3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:12.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:12.828+0000 7f2719e98700 1 --2- 172.21.15.40:0/2195627685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2714071aa0 0x7f27140b3a80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2700001eb0 tx=0x7f270000c7b0 comp rx=0 tx=0).stop 2023-11-08T08:21:12.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:12.828+0000 7f2719e98700 1 -- 172.21.15.40:0/2195627685 shutdown_connections 2023-11-08T08:21:12.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:12.828+0000 7f2719e98700 1 --2- 172.21.15.40:0/2195627685 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f26f8064ee0 0x7f26f8067390 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:12.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:12.828+0000 7f2719e98700 1 --2- 172.21.15.40:0/2195627685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f271415c640 0x7f2714159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:12.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:12.828+0000 7f2719e98700 1 --2- 172.21.15.40:0/2195627685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2714071aa0 0x7f27140b3a80 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:12.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:12.828+0000 7f2719e98700 1 --2- 172.21.15.40:0/2195627685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2714069e00 0x7f271409bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:12.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:12.828+0000 7f2719e98700 1 -- 172.21.15.40:0/2195627685 >> 172.21.15.40:0/2195627685 conn(0x7f27140b6520 msgr2=0x7f271406d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:12.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:12.828+0000 7f2719e98700 1 -- 172.21.15.40:0/2195627685 shutdown_connections 2023-11-08T08:21:12.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:12.828+0000 7f2719e98700 1 -- 172.21.15.40:0/2195627685 wait complete. 2023-11-08T08:21:12.836 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2023-11-08T08:21:12.856 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-08T08:21:13.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.036+0000 7f0b3ab69700 1 Processor -- start 2023-11-08T08:21:13.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.036+0000 7f0b3ab69700 1 -- start start 2023-11-08T08:21:13.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.040+0000 7f0b3ab69700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b34160710 0x7f0b340cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.040+0000 7f0b3ab69700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0b34070790 0x7f0b34164250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.040+0000 7f0b3ab69700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b340ad6c0 con 0x7f0b340cc860 2023-11-08T08:21:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.040+0000 7f0b3ab69700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3406b1c0 con 0x7f0b34070790 2023-11-08T08:21:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.040+0000 7f0b3ab69700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b340adc60 con 0x7f0b34160710 2023-11-08T08:21:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.040+0000 7f0b3a368700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0b34070790 0x7f0b34164250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.040+0000 7f0b39b67700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b34160710 0x7f0b340cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.040+0000 7f0b39b67700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b34160710 0x7f0b340cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:37244/0 (socket says 172.21.15.40:37244) 2023-11-08T08:21:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.040+0000 7f0b39b67700 1 -- 172.21.15.40:0/2132806616 learned_addr learned my addr 172.21.15.40:0/2132806616 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.040+0000 7f0b39366700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f0b340cc860 0x7f0b34070080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:55528/0 (socket says 172.21.15.40:55528) 2023-11-08T08:21:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.040+0000 7f0b39b67700 1 -- 172.21.15.40:0/2132806616 >> v2:172.21.15.40:3300/0 conn(0x7f0b34070790 msgr2=0x7f0b34164250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:13.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.040+0000 7f0b39b67700 1 --2- 172.21.15.40:0/2132806616 >> v2:172.21.15.40:3300/0 conn(0x7f0b34070790 0x7f0b34164250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:13.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.040+0000 7f0b39b67700 1 -- 172.21.15.40:0/2132806616 >> v1:172.21.15.169:6789/0 conn(0x7f0b340cc860 legacy=0x7f0b34070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:13.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.040+0000 7f0b39b67700 1 -- 172.21.15.40:0/2132806616 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3406b3d0 con 0x7f0b34160710 2023-11-08T08:21:13.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.040+0000 7f0b39b67700 1 --2- 172.21.15.40:0/2132806616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b34160710 0x7f0b340cc320 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009910 tx=0x7f0b30009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b81fd08d00ed39d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:13.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.044+0000 7f0b38b65700 1 -- 172.21.15.40:0/2132806616 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b30011830 con 0x7f0b34160710 2023-11-08T08:21:13.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.044+0000 7f0b38b65700 1 -- 172.21.15.40:0/2132806616 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b30011990 con 0x7f0b34160710 2023-11-08T08:21:13.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.044+0000 7f0b38b65700 1 -- 172.21.15.40:0/2132806616 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b30011d40 con 0x7f0b34160710 2023-11-08T08:21:13.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.044+0000 7f0b3ab69700 1 -- 172.21.15.40:0/2132806616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b34160710 msgr2=0x7f0b340cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:13.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.044+0000 7f0b3ab69700 1 --2- 172.21.15.40:0/2132806616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b34160710 0x7f0b340cc320 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009910 tx=0x7f0b30009c20 comp rx=0 tx=0).stop 2023-11-08T08:21:13.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.044+0000 7f0b3ab69700 1 -- 172.21.15.40:0/2132806616 shutdown_connections 2023-11-08T08:21:13.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.044+0000 7f0b3ab69700 1 --2- 172.21.15.40:0/2132806616 >> v2:172.21.15.40:3300/0 conn(0x7f0b34070790 0x7f0b34164250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:13.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.044+0000 7f0b3ab69700 1 --2- 172.21.15.40:0/2132806616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b34160710 0x7f0b340cc320 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:13.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.044+0000 7f0b3ab69700 1 -- 172.21.15.40:0/2132806616 >> 172.21.15.40:0/2132806616 conn(0x7f0b340b0f80 msgr2=0x7f0b340ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:13.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.044+0000 7f0b3ab69700 1 -- 172.21.15.40:0/2132806616 shutdown_connections 2023-11-08T08:21:13.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.044+0000 7f0b3ab69700 1 -- 172.21.15.40:0/2132806616 wait complete. 2023-11-08T08:21:13.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.048+0000 7f0b3ab69700 1 Processor -- start 2023-11-08T08:21:13.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.048+0000 7f0b3ab69700 1 -- start start 2023-11-08T08:21:13.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.048+0000 7f0b3ab69700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b34070790 0x7f0b3414a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:13.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.048+0000 7f0b3ab69700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b340cc860 0x7f0b3414a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:13.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.048+0000 7f0b3ab69700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b34160710 0x7f0b340ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:13.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.048+0000 7f0b3ab69700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3414da00 con 0x7f0b34160710 2023-11-08T08:21:13.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.048+0000 7f0b3ab69700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3414ce80 con 0x7f0b340cc860 2023-11-08T08:21:13.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.048+0000 7f0b3ab69700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3414d160 con 0x7f0b34070790 2023-11-08T08:21:13.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.048+0000 7f0b3a368700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b34160710 0x7f0b340ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:13.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.048+0000 7f0b39b67700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b34070790 0x7f0b3414a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:13.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.048+0000 7f0b3a368700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b34160710 0x7f0b340ae2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43336/0 (socket says 172.21.15.40:43336) 2023-11-08T08:21:13.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.048+0000 7f0b3a368700 1 -- 172.21.15.40:0/3664305667 learned_addr learned my addr 172.21.15.40:0/3664305667 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:13.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.048+0000 7f0b39366700 1 --2- 172.21.15.40:0/3664305667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b340cc860 0x7f0b3414a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:13.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.052+0000 7f0b39b67700 1 -- 172.21.15.40:0/3664305667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b340cc860 msgr2=0x7f0b3414a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:13.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.052+0000 7f0b39b67700 1 --2- 172.21.15.40:0/3664305667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b340cc860 0x7f0b3414a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:13.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.052+0000 7f0b39b67700 1 -- 172.21.15.40:0/3664305667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b34160710 msgr2=0x7f0b340ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:13.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.052+0000 7f0b39b67700 1 --2- 172.21.15.40:0/3664305667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b34160710 0x7f0b340ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:13.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.052+0000 7f0b39b67700 1 -- 172.21.15.40:0/3664305667 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b30009570 con 0x7f0b34070790 2023-11-08T08:21:13.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.052+0000 7f0b39b67700 1 --2- 172.21.15.40:0/3664305667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b34070790 0x7f0b3414a9a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0b30016000 tx=0x7f0b30004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:13.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.052+0000 7f0b2a7fc700 1 -- 172.21.15.40:0/3664305667 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b30004510 con 0x7f0b34070790 2023-11-08T08:21:13.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.052+0000 7f0b2a7fc700 1 -- 172.21.15.40:0/3664305667 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3001e070 con 0x7f0b34070790 2023-11-08T08:21:13.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.052+0000 7f0b3ab69700 1 -- 172.21.15.40:0/3664305667 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b340ad8f0 con 0x7f0b34070790 2023-11-08T08:21:13.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.052+0000 7f0b2a7fc700 1 -- 172.21.15.40:0/3664305667 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b30003900 con 0x7f0b34070790 2023-11-08T08:21:13.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.052+0000 7f0b3ab69700 1 -- 172.21.15.40:0/3664305667 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3406b6b0 con 0x7f0b34070790 2023-11-08T08:21:13.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.052+0000 7f0b3ab69700 1 -- 172.21.15.40:0/3664305667 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b34160ae0 con 0x7f0b34070790 2023-11-08T08:21:13.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.056+0000 7f0b2a7fc700 1 -- 172.21.15.40:0/3664305667 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f0b3000f050 con 0x7f0b34070790 2023-11-08T08:21:13.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.056+0000 7f0b2a7fc700 1 --2- 172.21.15.40:0/3664305667 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0b18064ee0 0x7f0b18067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:13.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.056+0000 7f0b2a7fc700 1 -- 172.21.15.40:0/3664305667 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f0b30082980 con 0x7f0b34070790 2023-11-08T08:21:13.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.056+0000 7f0b39366700 1 --2- 172.21.15.40:0/3664305667 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0b18064ee0 0x7f0b18067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:13.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.056+0000 7f0b39366700 1 --2- 172.21.15.40:0/3664305667 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0b18064ee0 0x7f0b18067390 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0b341f93f0 tx=0x7f0b24007ef0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:13.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.064+0000 7f0b2a7fc700 1 -- 172.21.15.40:0/3664305667 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f0b30050c40 con 0x7f0b34070790 2023-11-08T08:21:13.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.300+0000 7f0b3ab69700 1 -- 172.21.15.40:0/3664305667 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0b34073cb0 con 0x7f0b34070790 2023-11-08T08:21:13.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.304+0000 7f0b2a7fc700 1 -- 172.21.15.40:0/3664305667 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f0b30054260 con 0x7f0b34070790 2023-11-08T08:21:13.309 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:13.309 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:21:13.310 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2023-11-08T08:21:13.310 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:13.310 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2023-11-08T08:21:13.310 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA3REtlj6YZGRAABMVq4pKyrp6U3tbqShtmuw==", 2023-11-08T08:21:13.310 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:21:13.311 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:21:13.311 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-11-08T08:21:13.311 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-11-08T08:21:13.311 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:21:13.311 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:13.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:13.312 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:13.312 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2023-11-08T08:21:13.312 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA3REtlAfqKLRAAmNg22Rw+FWmzeRDBKQtjfw==", 2023-11-08T08:21:13.312 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:21:13.313 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:21:13.313 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-11-08T08:21:13.313 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-11-08T08:21:13.313 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:21:13.313 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:13.314 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:13.314 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:13.314 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2023-11-08T08:21:13.314 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA3REtlRseiJBAAZFNQO+PK+kioVBn+S/54Hw==", 2023-11-08T08:21:13.314 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:21:13.315 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:21:13.315 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-11-08T08:21:13.315 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-11-08T08:21:13.315 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:21:13.315 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:13.316 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:13.316 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:13.316 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2023-11-08T08:21:13.316 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA3REtlZHCiOBAAW8bJst1bqTZ2jPOr+wXcpg==", 2023-11-08T08:21:13.316 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:21:13.316 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:21:13.317 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-11-08T08:21:13.317 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-11-08T08:21:13.317 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:21:13.317 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:13.317 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:13.318 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:13.318 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2023-11-08T08:21:13.318 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA7REtlvWsaLBAAws4XZXHoYl9KnhBp6LEh0w==", 2023-11-08T08:21:13.318 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:21:13.319 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:21:13.319 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:21:13.319 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:21:13.319 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:13.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:13.319 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:13.320 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2023-11-08T08:21:13.320 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA+REtl8FYFOhAAOY5q9LJAl6PS+essHcF8cw==", 2023-11-08T08:21:13.320 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:21:13.320 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:21:13.320 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:21:13.321 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:21:13.321 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:13.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:13.321 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:13.321 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2023-11-08T08:21:13.322 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBCREtljU1ODBAA1gVE4sX2QNdGHIjdRue/rA==", 2023-11-08T08:21:13.322 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:21:13.322 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:21:13.322 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:21:13.322 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:21:13.323 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:13.323 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:13.323 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:13.323 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2023-11-08T08:21:13.323 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBFREtlRZsSGxAAh7v5f7TVZNQozESV6k641g==", 2023-11-08T08:21:13.324 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:21:13.324 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:21:13.324 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:21:13.324 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:21:13.324 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:13.325 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:13.325 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:13.325 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2023-11-08T08:21:13.325 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBLREtla+qaNRAA0jaCFarD/RgWGp1ltYXQdA==", 2023-11-08T08:21:13.325 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:21:13.326 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:21:13.326 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:21:13.326 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:21:13.326 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:13.326 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:13.327 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:13.327 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2023-11-08T08:21:13.327 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPREtltHKNCRAA/TKgyxOGRkN5y/RHWr8vSQ==", 2023-11-08T08:21:13.327 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:21:13.327 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:21:13.328 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:21:13.328 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:21:13.328 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:13.328 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:13.328 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:13.329 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2023-11-08T08:21:13.329 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBSREtlpXC4GRAAZl6YWPVnbX3o1tyfEkR7CA==", 2023-11-08T08:21:13.329 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:21:13.329 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:21:13.329 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:21:13.330 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:21:13.330 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:13.330 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:13.330 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:13.330 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2023-11-08T08:21:13.331 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBVREtlvL/4JxAAZtGdthYoFzdMRs3DMcRXNw==", 2023-11-08T08:21:13.331 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:21:13.331 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:21:13.331 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:21:13.331 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:21:13.332 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:13.332 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:13.332 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:13.332 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2023-11-08T08:21:13.332 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA4REtl0YCpBBAAtap/irtK+GkSmR8g37HRVQ==", 2023-11-08T08:21:13.333 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:21:13.333 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:21:13.333 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2023-11-08T08:21:13.333 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-08T08:21:13.333 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:13.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:13.334 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:13.334 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2023-11-08T08:21:13.334 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA4REtlgapFERAAxg8GS7iiH06t2mCF5ImFXA==", 2023-11-08T08:21:13.334 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:21:13.334 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:21:13.335 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-11-08T08:21:13.335 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-11-08T08:21:13.335 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-11-08T08:21:13.335 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:13.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:13.336 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:13.336 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2023-11-08T08:21:13.336 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA4REtldlrMChAAC9+syjsFWWFwJGXPw82zwQ==", 2023-11-08T08:21:13.336 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:21:13.336 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:21:13.337 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-11-08T08:21:13.337 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-11-08T08:21:13.337 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-11-08T08:21:13.337 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:13.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:13.338 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:13.338 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2023-11-08T08:21:13.338 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA4REtllHEDFxAACJSjTZQaOTnyA0K3X/igXQ==", 2023-11-08T08:21:13.338 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:21:13.338 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:21:13.339 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-11-08T08:21:13.339 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-11-08T08:21:13.339 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-11-08T08:21:13.339 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:13.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:13.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:13.340 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2023-11-08T08:21:13.340 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA2REtlqqakKRAAzyr9EbqU+Pj0m2weeru80g==", 2023-11-08T08:21:13.340 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:21:13.340 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-11-08T08:21:13.341 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-11-08T08:21:13.341 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-11-08T08:21:13.341 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:21:13.341 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:13.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:13.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:13.342 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2023-11-08T08:21:13.342 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA3REtl6YeRERAAKW6fA2vM7HOpZGCulFFLzQ==", 2023-11-08T08:21:13.342 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:21:13.342 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-11-08T08:21:13.343 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-11-08T08:21:13.343 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:21:13.343 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:13.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:13.343 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:13.343 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2023-11-08T08:21:13.344 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA3REtlmuaiChAAsI2681k5qbe8I0hdIXHJGg==", 2023-11-08T08:21:13.344 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:21:13.344 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-11-08T08:21:13.344 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-11-08T08:21:13.344 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:21:13.345 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:13.345 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:13.345 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-11-08T08:21:13.345 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:21:13.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.304+0000 7f0b3ab69700 1 -- 172.21.15.40:0/3664305667 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0b18064ee0 msgr2=0x7f0b18067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:13.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.304+0000 7f0b3ab69700 1 --2- 172.21.15.40:0/3664305667 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0b18064ee0 0x7f0b18067390 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0b341f93f0 tx=0x7f0b24007ef0 comp rx=0 tx=0).stop 2023-11-08T08:21:13.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.304+0000 7f0b3ab69700 1 -- 172.21.15.40:0/3664305667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b34070790 msgr2=0x7f0b3414a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.304+0000 7f0b3ab69700 1 --2- 172.21.15.40:0/3664305667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b34070790 0x7f0b3414a9a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0b30016000 tx=0x7f0b30004770 comp rx=0 tx=0).stop 2023-11-08T08:21:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.304+0000 7f0b3ab69700 1 -- 172.21.15.40:0/3664305667 shutdown_connections 2023-11-08T08:21:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.304+0000 7f0b3ab69700 1 --2- 172.21.15.40:0/3664305667 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0b18064ee0 0x7f0b18067390 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.304+0000 7f0b3ab69700 1 --2- 172.21.15.40:0/3664305667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b34160710 0x7f0b340ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.304+0000 7f0b3ab69700 1 --2- 172.21.15.40:0/3664305667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b340cc860 0x7f0b3414a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.304+0000 7f0b3ab69700 1 --2- 172.21.15.40:0/3664305667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b34070790 0x7f0b3414a9a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.308+0000 7f0b3ab69700 1 -- 172.21.15.40:0/3664305667 >> 172.21.15.40:0/3664305667 conn(0x7f0b340b0f80 msgr2=0x7f0b340cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.308+0000 7f0b3ab69700 1 -- 172.21.15.40:0/3664305667 shutdown_connections 2023-11-08T08:21:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.308+0000 7f0b3ab69700 1 -- 172.21.15.40:0/3664305667 wait complete. 2023-11-08T08:21:13.350 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:21:13.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.520+0000 7f9c1759e700 1 Processor -- start 2023-11-08T08:21:13.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.520+0000 7f9c1759e700 1 -- start start 2023-11-08T08:21:13.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.524+0000 7f9c1759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c180b1640 0x7f9c18070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:13.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.524+0000 7f9c1759e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c1815a550 con 0x7f9c180c9040 2023-11-08T08:21:13.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.524+0000 7f9c1759e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c1806b0a0 con 0x7f9c180b43a0 2023-11-08T08:21:13.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.524+0000 7f9c1759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c180caac0 con 0x7f9c180b1640 2023-11-08T08:21:13.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.524+0000 7f9c16d9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9c180c9040 0x7f9c1806bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45110/0 (socket says 172.21.15.40:45110) 2023-11-08T08:21:13.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.524+0000 7f9c16d9d700 1 -- 172.21.15.40:0/345167886 learned_addr learned my addr 172.21.15.40:0/345167886 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:13.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.524+0000 7f9c0dd9b700 1 --2- 172.21.15.40:0/345167886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c180b1640 0x7f9c18070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:13.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.524+0000 7f9c15d9b700 1 -- 172.21.15.40:0/345167886 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3819779236 0 0) 0x7f9c10001490 con 0x7f9c180c9040 2023-11-08T08:21:13.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.524+0000 7f9c15d9b700 1 -- 172.21.15.40:0/345167886 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c040035d0 con 0x7f9c180c9040 2023-11-08T08:21:13.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.524+0000 7f9c15d9b700 1 -- 172.21.15.40:0/345167886 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259051238 0 0) 0x7f9c1806b0a0 con 0x7f9c180b43a0 2023-11-08T08:21:13.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.524+0000 7f9c15d9b700 1 -- 172.21.15.40:0/345167886 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c10001490 con 0x7f9c180b43a0 2023-11-08T08:21:13.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.524+0000 7f9c0dd9b700 1 -- 172.21.15.40:0/345167886 >> v1:172.21.15.169:6789/0 conn(0x7f9c180b43a0 legacy=0x7f9c1815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:13.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.524+0000 7f9c0dd9b700 1 -- 172.21.15.40:0/345167886 >> v1:172.21.15.40:6789/0 conn(0x7f9c180c9040 legacy=0x7f9c1806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:13.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.524+0000 7f9c0dd9b700 1 -- 172.21.15.40:0/345167886 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c180cada0 con 0x7f9c180b1640 2023-11-08T08:21:13.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.528+0000 7f9c0dd9b700 1 --2- 172.21.15.40:0/345167886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c180b1640 0x7f9c18070080 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009880 tx=0x7f9c00009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3357237cee59f118 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:13.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.528+0000 7f9c15d9b700 1 -- 172.21.15.40:0/345167886 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00011830 con 0x7f9c180b1640 2023-11-08T08:21:13.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.528+0000 7f9c15d9b700 1 -- 172.21.15.40:0/345167886 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c00011990 con 0x7f9c180b1640 2023-11-08T08:21:13.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.528+0000 7f9c15d9b700 1 -- 172.21.15.40:0/345167886 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00011d20 con 0x7f9c180b1640 2023-11-08T08:21:13.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.528+0000 7f9c1759e700 1 -- 172.21.15.40:0/345167886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c180b1640 msgr2=0x7f9c18070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:13.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.528+0000 7f9c1759e700 1 --2- 172.21.15.40:0/345167886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c180b1640 0x7f9c18070080 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009880 tx=0x7f9c00009b90 comp rx=0 tx=0).stop 2023-11-08T08:21:13.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.528+0000 7f9c1759e700 1 -- 172.21.15.40:0/345167886 shutdown_connections 2023-11-08T08:21:13.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.528+0000 7f9c1759e700 1 --2- 172.21.15.40:0/345167886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c180b1640 0x7f9c18070080 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:13.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.528+0000 7f9c1759e700 1 -- 172.21.15.40:0/345167886 >> 172.21.15.40:0/345167886 conn(0x7f9c180b0f80 msgr2=0x7f9c180ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:13.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.528+0000 7f9c1759e700 1 -- 172.21.15.40:0/345167886 shutdown_connections 2023-11-08T08:21:13.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.528+0000 7f9c1759e700 1 -- 172.21.15.40:0/345167886 wait complete. 2023-11-08T08:21:13.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.532+0000 7f9c1759e700 1 Processor -- start 2023-11-08T08:21:13.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.532+0000 7f9c1759e700 1 -- start start 2023-11-08T08:21:13.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.532+0000 7f9c1759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c180b1640 0x7f9c18155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:13.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.532+0000 7f9c1759e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c180b43a0 0x7f9c18155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:13.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.532+0000 7f9c1759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c180c9040 0x7f9c1815bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:13.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.532+0000 7f9c1759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1815bb50 con 0x7f9c180b1640 2023-11-08T08:21:13.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.532+0000 7f9c1759e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c180c8210 con 0x7f9c180b43a0 2023-11-08T08:21:13.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.532+0000 7f9c1759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c180c7c50 con 0x7f9c180c9040 2023-11-08T08:21:13.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.536+0000 7f9c1659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c180b1640 0x7f9c18155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:13.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.536+0000 7f9c1659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c180b1640 0x7f9c18155660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43350/0 (socket says 172.21.15.40:43350) 2023-11-08T08:21:13.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.536+0000 7f9c1659c700 1 -- 172.21.15.40:0/432069115 learned_addr learned my addr 172.21.15.40:0/432069115 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:13.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.536+0000 7f9c16d9d700 1 --2- 172.21.15.40:0/432069115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c180c9040 0x7f9c1815bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:13.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.536+0000 7f9c0dd9b700 1 --2- 172.21.15.40:0/432069115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c180b43a0 0x7f9c18155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:13.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.536+0000 7f9c1659c700 1 -- 172.21.15.40:0/432069115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c180c9040 msgr2=0x7f9c1815bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:13.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.536+0000 7f9c1659c700 1 --2- 172.21.15.40:0/432069115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c180c9040 0x7f9c1815bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:13.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.536+0000 7f9c1659c700 1 -- 172.21.15.40:0/432069115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c180b43a0 msgr2=0x7f9c18155e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:13.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.536+0000 7f9c1659c700 1 --2- 172.21.15.40:0/432069115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c180b43a0 0x7f9c18155e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:13.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.536+0000 7f9c1659c700 1 -- 172.21.15.40:0/432069115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c000094b0 con 0x7f9c180b1640 2023-11-08T08:21:13.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.536+0000 7f9c1659c700 1 --2- 172.21.15.40:0/432069115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c180b1640 0x7f9c18155660 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c730 tx=0x7f9c0800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:13.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.536+0000 7f9c0f7fe700 1 -- 172.21.15.40:0/432069115 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c080052d0 con 0x7f9c180b1640 2023-11-08T08:21:13.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.536+0000 7f9c1759e700 1 -- 172.21.15.40:0/432069115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c1806b5d0 con 0x7f9c180b1640 2023-11-08T08:21:13.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.536+0000 7f9c0f7fe700 1 -- 172.21.15.40:0/432069115 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c08005450 con 0x7f9c180b1640 2023-11-08T08:21:13.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.536+0000 7f9c0f7fe700 1 -- 172.21.15.40:0/432069115 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c080038e0 con 0x7f9c180b1640 2023-11-08T08:21:13.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.536+0000 7f9c1759e700 1 -- 172.21.15.40:0/432069115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1806b2f0 con 0x7f9c180b1640 2023-11-08T08:21:13.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.540+0000 7f9c1759e700 1 -- 172.21.15.40:0/432069115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c180ca7b0 con 0x7f9c180b1640 2023-11-08T08:21:13.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.540+0000 7f9c0f7fe700 1 -- 172.21.15.40:0/432069115 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f9c08019030 con 0x7f9c180b1640 2023-11-08T08:21:13.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.540+0000 7f9c0f7fe700 1 --2- 172.21.15.40:0/432069115 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9bf8064f10 0x7f9bf80673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:13.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.540+0000 7f9c0f7fe700 1 -- 172.21.15.40:0/432069115 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f9c08081160 con 0x7f9c180b1640 2023-11-08T08:21:13.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.544+0000 7f9c0dd9b700 1 --2- 172.21.15.40:0/432069115 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9bf8064f10 0x7f9bf80673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:13.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.544+0000 7f9c0dd9b700 1 --2- 172.21.15.40:0/432069115 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9bf8064f10 0x7f9bf80673c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9c000090b0 tx=0x7f9c0001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:13.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.548+0000 7f9c0f7fe700 1 -- 172.21.15.40:0/432069115 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f9c0804f310 con 0x7f9c180b1640 2023-11-08T08:21:13.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.732+0000 7f9c1759e700 1 -- 172.21.15.40:0/432069115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c18073cb0 con 0x7f9c180b1640 2023-11-08T08:21:13.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.732+0000 7f9c0f7fe700 1 -- 172.21.15.40:0/432069115 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+14376 (secure 0 0 0) 0x7f9c08017090 con 0x7f9c180b1640 2023-11-08T08:21:13.739 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:13.740 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":53,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:21:12.812870+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:21:13.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.736+0000 7f9c1759e700 1 -- 172.21.15.40:0/432069115 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9bf8064f10 msgr2=0x7f9bf80673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:13.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.736+0000 7f9c1759e700 1 --2- 172.21.15.40:0/432069115 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9bf8064f10 0x7f9bf80673c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9c000090b0 tx=0x7f9c0001c040 comp rx=0 tx=0).stop 2023-11-08T08:21:13.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.736+0000 7f9c1759e700 1 -- 172.21.15.40:0/432069115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c180b1640 msgr2=0x7f9c18155660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:13.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.736+0000 7f9c1759e700 1 --2- 172.21.15.40:0/432069115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c180b1640 0x7f9c18155660 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c730 tx=0x7f9c0800ca40 comp rx=0 tx=0).stop 2023-11-08T08:21:13.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.736+0000 7f9c1759e700 1 -- 172.21.15.40:0/432069115 shutdown_connections 2023-11-08T08:21:13.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.736+0000 7f9c1759e700 1 --2- 172.21.15.40:0/432069115 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9bf8064f10 0x7f9bf80673c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:13.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.736+0000 7f9c1759e700 1 --2- 172.21.15.40:0/432069115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c180c9040 0x7f9c1815bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:13.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.736+0000 7f9c1759e700 1 --2- 172.21.15.40:0/432069115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c180b43a0 0x7f9c18155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:13.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.736+0000 7f9c1759e700 1 --2- 172.21.15.40:0/432069115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c180b1640 0x7f9c18155660 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:13.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.736+0000 7f9c1759e700 1 -- 172.21.15.40:0/432069115 >> 172.21.15.40:0/432069115 conn(0x7f9c180b0f80 msgr2=0x7f9c18159d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:13.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.736+0000 7f9c1759e700 1 -- 172.21.15.40:0/432069115 shutdown_connections 2023-11-08T08:21:13.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.736+0000 7f9c1759e700 1 -- 172.21.15.40:0/432069115 wait complete. 2023-11-08T08:21:13.766 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T08:21:13.767 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:21:13.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.960+0000 7f68616c4700 1 Processor -- start 2023-11-08T08:21:13.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.960+0000 7f68616c4700 1 -- start start 2023-11-08T08:21:13.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.964+0000 7f68616c4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f685c15c150 0x7f685c0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:13.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.964+0000 7f68616c4700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685c15c520 con 0x7f685c06ff10 2023-11-08T08:21:13.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.964+0000 7f68616c4700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685c06ac00 con 0x7f685c070360 2023-11-08T08:21:13.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.964+0000 7f68616c4700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685c0b4240 con 0x7f685c15c150 2023-11-08T08:21:13.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.964+0000 7f685bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f685c15c150 0x7f685c0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:13.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.964+0000 7f685b7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f685c06ff10 0x7f685c15fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45120/0 (socket says 172.21.15.40:45120) 2023-11-08T08:21:13.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.964+0000 7f685b7fe700 1 -- 172.21.15.40:0/2031855453 learned_addr learned my addr 172.21.15.40:0/2031855453 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:13.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.964+0000 7f685bfff700 1 -- 172.21.15.40:0/2031855453 >> v1:172.21.15.169:6789/0 conn(0x7f685c070360 legacy=0x7f685c160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:13.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.964+0000 7f685bfff700 1 -- 172.21.15.40:0/2031855453 >> v1:172.21.15.40:6789/0 conn(0x7f685c06ff10 legacy=0x7f685c15fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:13.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.964+0000 7f685bfff700 1 -- 172.21.15.40:0/2031855453 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c160a50 con 0x7f685c15c150 2023-11-08T08:21:13.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.964+0000 7f685affd700 1 -- 172.21.15.40:0/2031855453 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3178065027 0 0) 0x7f685c15c520 con 0x7f685c06ff10 2023-11-08T08:21:13.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.964+0000 7f685bfff700 1 --2- 172.21.15.40:0/2031855453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f685c15c150 0x7f685c0b3d00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f684c009850 tx=0x7f684c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7caed2eb00790111 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:13.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.964+0000 7f685affd700 1 -- 172.21.15.40:0/2031855453 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c011830 con 0x7f685c15c150 2023-11-08T08:21:13.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.964+0000 7f685affd700 1 -- 172.21.15.40:0/2031855453 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684c011990 con 0x7f685c15c150 2023-11-08T08:21:13.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.964+0000 7f685affd700 1 -- 172.21.15.40:0/2031855453 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c011d20 con 0x7f685c15c150 2023-11-08T08:21:13.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.968+0000 7f68616c4700 1 -- 172.21.15.40:0/2031855453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f685c15c150 msgr2=0x7f685c0b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:13.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.968+0000 7f68616c4700 1 --2- 172.21.15.40:0/2031855453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f685c15c150 0x7f685c0b3d00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f684c009850 tx=0x7f684c009b60 comp rx=0 tx=0).stop 2023-11-08T08:21:13.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.968+0000 7f68616c4700 1 -- 172.21.15.40:0/2031855453 shutdown_connections 2023-11-08T08:21:13.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.968+0000 7f68616c4700 1 --2- 172.21.15.40:0/2031855453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f685c15c150 0x7f685c0b3d00 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:13.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.968+0000 7f68616c4700 1 -- 172.21.15.40:0/2031855453 >> 172.21.15.40:0/2031855453 conn(0x7f685c0d3680 msgr2=0x7f685c0cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:13.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.968+0000 7f68616c4700 1 -- 172.21.15.40:0/2031855453 shutdown_connections 2023-11-08T08:21:13.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.968+0000 7f68616c4700 1 -- 172.21.15.40:0/2031855453 wait complete. 2023-11-08T08:21:13.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.968+0000 7f68616c4700 1 Processor -- start 2023-11-08T08:21:13.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.968+0000 7f68616c4700 1 -- start start 2023-11-08T08:21:13.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f68616c4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f685c06ff10 0x7f685c14d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:13.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f68616c4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f685c070360 0x7f685c0cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:13.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f68616c4700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f685c15c150 0x7f685c0cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:13.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f68616c4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c14dfd0 con 0x7f685c06ff10 2023-11-08T08:21:13.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f68616c4700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c14fc10 con 0x7f685c070360 2023-11-08T08:21:13.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f68616c4700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0ca450 con 0x7f685c15c150 2023-11-08T08:21:13.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f685bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f685c06ff10 0x7f685c14d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:13.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f685bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f685c06ff10 0x7f685c14d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43368/0 (socket says 172.21.15.40:43368) 2023-11-08T08:21:13.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f685bfff700 1 -- 172.21.15.40:0/3520425100 learned_addr learned my addr 172.21.15.40:0/3520425100 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:13.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f6860ec3700 1 --2- 172.21.15.40:0/3520425100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f685c15c150 0x7f685c0cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:13.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f685b7fe700 1 --2- 172.21.15.40:0/3520425100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f685c070360 0x7f685c0cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:13.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f685bfff700 1 -- 172.21.15.40:0/3520425100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f685c15c150 msgr2=0x7f685c0cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:13.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f685bfff700 1 --2- 172.21.15.40:0/3520425100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f685c15c150 0x7f685c0cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:13.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f685bfff700 1 -- 172.21.15.40:0/3520425100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f685c070360 msgr2=0x7f685c0cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:13.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f685bfff700 1 --2- 172.21.15.40:0/3520425100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f685c070360 0x7f685c0cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:13.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f685bfff700 1 -- 172.21.15.40:0/3520425100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c0094b0 con 0x7f685c06ff10 2023-11-08T08:21:13.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f685bfff700 1 --2- 172.21.15.40:0/3520425100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f685c06ff10 0x7f685c14d080 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f684c014040 tx=0x7f684c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:13.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f6858ff9700 1 -- 172.21.15.40:0/3520425100 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c00ba70 con 0x7f685c06ff10 2023-11-08T08:21:13.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f6858ff9700 1 -- 172.21.15.40:0/3520425100 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684c01e070 con 0x7f685c06ff10 2023-11-08T08:21:13.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f6858ff9700 1 -- 172.21.15.40:0/3520425100 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c00bdf0 con 0x7f685c06ff10 2023-11-08T08:21:13.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f68616c4700 1 -- 172.21.15.40:0/3520425100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685c0d2450 con 0x7f685c06ff10 2023-11-08T08:21:13.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.972+0000 7f68616c4700 1 -- 172.21.15.40:0/3520425100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685c06cc50 con 0x7f685c06ff10 2023-11-08T08:21:13.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.976+0000 7f68616c4700 1 -- 172.21.15.40:0/3520425100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685c0d2730 con 0x7f685c06ff10 2023-11-08T08:21:13.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.976+0000 7f6858ff9700 1 -- 172.21.15.40:0/3520425100 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f684c02a020 con 0x7f685c06ff10 2023-11-08T08:21:13.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.976+0000 7f6858ff9700 1 --2- 172.21.15.40:0/3520425100 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6848064ee0 0x7f6848067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:13.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.976+0000 7f6858ff9700 1 -- 172.21.15.40:0/3520425100 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f684c082fb0 con 0x7f685c06ff10 2023-11-08T08:21:13.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.980+0000 7f685b7fe700 1 --2- 172.21.15.40:0/3520425100 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6848064ee0 0x7f6848067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:13.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.980+0000 7f685b7fe700 1 --2- 172.21.15.40:0/3520425100 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6848064ee0 0x7f6848067390 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f685c0c9200 tx=0x7f6850009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:13.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:13.984+0000 7f6858ff9700 1 -- 172.21.15.40:0/3520425100 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f684c051270 con 0x7f685c06ff10 2023-11-08T08:21:14.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.156+0000 7f68616c4700 1 -- 172.21.15.40:0/3520425100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f685c073d20 con 0x7f685c06ff10 2023-11-08T08:21:14.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.160+0000 7f6858ff9700 1 -- 172.21.15.40:0/3520425100 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7f684c054890 con 0x7f685c06ff10 2023-11-08T08:21:14.165 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:21:14.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.160+0000 7f68616c4700 1 -- 172.21.15.40:0/3520425100 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6848064ee0 msgr2=0x7f6848067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:14.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.160+0000 7f68616c4700 1 --2- 172.21.15.40:0/3520425100 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6848064ee0 0x7f6848067390 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f685c0c9200 tx=0x7f6850009040 comp rx=0 tx=0).stop 2023-11-08T08:21:14.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.160+0000 7f68616c4700 1 -- 172.21.15.40:0/3520425100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f685c06ff10 msgr2=0x7f685c14d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:14.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.160+0000 7f68616c4700 1 --2- 172.21.15.40:0/3520425100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f685c06ff10 0x7f685c14d080 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f684c014040 tx=0x7f684c004770 comp rx=0 tx=0).stop 2023-11-08T08:21:14.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.160+0000 7f68616c4700 1 -- 172.21.15.40:0/3520425100 shutdown_connections 2023-11-08T08:21:14.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.160+0000 7f68616c4700 1 --2- 172.21.15.40:0/3520425100 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6848064ee0 0x7f6848067390 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:14.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.160+0000 7f68616c4700 1 --2- 172.21.15.40:0/3520425100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f685c15c150 0x7f685c0cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:14.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.160+0000 7f68616c4700 1 --2- 172.21.15.40:0/3520425100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f685c070360 0x7f685c0cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:14.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.160+0000 7f68616c4700 1 --2- 172.21.15.40:0/3520425100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f685c06ff10 0x7f685c14d080 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:14.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.160+0000 7f68616c4700 1 -- 172.21.15.40:0/3520425100 >> 172.21.15.40:0/3520425100 conn(0x7f685c0d3680 msgr2=0x7f685c156810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:14.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.164+0000 7f68616c4700 1 -- 172.21.15.40:0/3520425100 shutdown_connections 2023-11-08T08:21:14.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.164+0000 7f68616c4700 1 -- 172.21.15.40:0/3520425100 wait complete. 2023-11-08T08:21:14.190 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:21:14.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.388+0000 7f53999a8700 1 Processor -- start 2023-11-08T08:21:14.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.388+0000 7f53999a8700 1 -- start start 2023-11-08T08:21:14.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.392+0000 7f53999a8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f539415c150 0x7f53940b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:14.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.392+0000 7f53999a8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f539406ff10 0x7f5394070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:14.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.392+0000 7f53999a8700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f539415fc30 0x7f53941600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:14.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.392+0000 7f53999a8700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53940b4240 con 0x7f539415c150 2023-11-08T08:21:14.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.392+0000 7f53999a8700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f539415c520 con 0x7f539415fc30 2023-11-08T08:21:14.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.392+0000 7f53999a8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53940708a0 con 0x7f539406ff10 2023-11-08T08:21:14.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.392+0000 7f53989a6700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f539415c150 0x7f53940b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:14.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.392+0000 7f53989a6700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f539415c150 0x7f53940b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43376/0 (socket says 172.21.15.40:43376) 2023-11-08T08:21:14.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.392+0000 7f53989a6700 1 -- 172.21.15.40:0/3857789202 learned_addr learned my addr 172.21.15.40:0/3857789202 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:14.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.392+0000 7f5393fff700 1 --2- 172.21.15.40:0/3857789202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f539406ff10 0x7f5394070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:14.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.392+0000 7f53991a7700 1 --2- 172.21.15.40:0/3857789202 >> v2:172.21.15.169:3300/0 conn(0x7f539415fc30 0x7f53941600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:14.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.392+0000 7f53989a6700 1 -- 172.21.15.40:0/3857789202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f539406ff10 msgr2=0x7f5394070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:14.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.392+0000 7f53989a6700 1 --2- 172.21.15.40:0/3857789202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f539406ff10 0x7f5394070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:14.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.392+0000 7f53989a6700 1 -- 172.21.15.40:0/3857789202 >> v2:172.21.15.169:3300/0 conn(0x7f539415fc30 msgr2=0x7f53941600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:14.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.392+0000 7f53989a6700 1 --2- 172.21.15.40:0/3857789202 >> v2:172.21.15.169:3300/0 conn(0x7f539415fc30 0x7f53941600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:14.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.392+0000 7f53989a6700 1 -- 172.21.15.40:0/3857789202 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539406cc50 con 0x7f539415c150 2023-11-08T08:21:14.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.392+0000 7f53989a6700 1 --2- 172.21.15.40:0/3857789202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539415c150 0x7f53940b3d00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5394071460 tx=0x7f5388009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c271d3607426852 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.392+0000 7f53937fe700 1 -- 172.21.15.40:0/3857789202 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5388011830 con 0x7f539415c150 2023-11-08T08:21:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.396+0000 7f53937fe700 1 -- 172.21.15.40:0/3857789202 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5388011990 con 0x7f539415c150 2023-11-08T08:21:14.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.396+0000 7f53937fe700 1 -- 172.21.15.40:0/3857789202 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5388011d40 con 0x7f539415c150 2023-11-08T08:21:14.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.396+0000 7f53999a8700 1 -- 172.21.15.40:0/3857789202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539415c150 msgr2=0x7f53940b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:14.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.396+0000 7f53999a8700 1 --2- 172.21.15.40:0/3857789202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539415c150 0x7f53940b3d00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5394071460 tx=0x7f5388009c20 comp rx=0 tx=0).stop 2023-11-08T08:21:14.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.396+0000 7f53999a8700 1 -- 172.21.15.40:0/3857789202 shutdown_connections 2023-11-08T08:21:14.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.396+0000 7f53999a8700 1 --2- 172.21.15.40:0/3857789202 >> v2:172.21.15.169:3300/0 conn(0x7f539415fc30 0x7f53941600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:14.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.396+0000 7f53999a8700 1 --2- 172.21.15.40:0/3857789202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f539406ff10 0x7f5394070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:14.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.396+0000 7f53999a8700 1 --2- 172.21.15.40:0/3857789202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539415c150 0x7f53940b3d00 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:14.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.396+0000 7f53999a8700 1 -- 172.21.15.40:0/3857789202 >> 172.21.15.40:0/3857789202 conn(0x7f53940d3680 msgr2=0x7f53940cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.396+0000 7f53999a8700 1 -- 172.21.15.40:0/3857789202 shutdown_connections 2023-11-08T08:21:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.396+0000 7f53999a8700 1 -- 172.21.15.40:0/3857789202 wait complete. 2023-11-08T08:21:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.396+0000 7f53999a8700 1 Processor -- start 2023-11-08T08:21:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.396+0000 7f53999a8700 1 -- start start 2023-11-08T08:21:14.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f53999a8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f539406ff10 0x7f53940cf340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:14.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f53999a8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539415c150 0x7f5394157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f53999a8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f539415fc30 0x7f53941581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f53999a8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5394159480 con 0x7f539415c150 2023-11-08T08:21:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f53999a8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5394159760 con 0x7f539415fc30 2023-11-08T08:21:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f53999a8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5394159a40 con 0x7f539406ff10 2023-11-08T08:21:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f5393fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539415c150 0x7f5394157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f5393fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539415c150 0x7f5394157c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43384/0 (socket says 172.21.15.40:43384) 2023-11-08T08:21:14.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f5393fff700 1 -- 172.21.15.40:0/1917461410 learned_addr learned my addr 172.21.15.40:0/1917461410 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:14.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f53991a7700 1 --2- 172.21.15.40:0/1917461410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f539415fc30 0x7f53941581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f53989a6700 1 --2- 172.21.15.40:0/1917461410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f539406ff10 0x7f53940cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f5393fff700 1 -- 172.21.15.40:0/1917461410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f539406ff10 msgr2=0x7f53940cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f5393fff700 1 --2- 172.21.15.40:0/1917461410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f539406ff10 0x7f53940cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f5393fff700 1 -- 172.21.15.40:0/1917461410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f539415fc30 msgr2=0x7f53941581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f5393fff700 1 --2- 172.21.15.40:0/1917461410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f539415fc30 0x7f53941581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f5393fff700 1 -- 172.21.15.40:0/1917461410 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5388009570 con 0x7f539415c150 2023-11-08T08:21:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f5393fff700 1 --2- 172.21.15.40:0/1917461410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539415c150 0x7f5394157c70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f538000e930 tx=0x7f538000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f53917fa700 1 -- 172.21.15.40:0/1917461410 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380005a40 con 0x7f539415c150 2023-11-08T08:21:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f53917fa700 1 -- 172.21.15.40:0/1917461410 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5380005ba0 con 0x7f539415c150 2023-11-08T08:21:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f53917fa700 1 -- 172.21.15.40:0/1917461410 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380013680 con 0x7f539415c150 2023-11-08T08:21:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f53999a8700 1 -- 172.21.15.40:0/1917461410 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53940d2730 con 0x7f539415c150 2023-11-08T08:21:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.400+0000 7f53999a8700 1 -- 172.21.15.40:0/1917461410 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53940d2450 con 0x7f539415c150 2023-11-08T08:21:14.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.404+0000 7f53999a8700 1 -- 172.21.15.40:0/1917461410 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53940d05e0 con 0x7f539415c150 2023-11-08T08:21:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.404+0000 7f53917fa700 1 -- 172.21.15.40:0/1917461410 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5380022070 con 0x7f539415c150 2023-11-08T08:21:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.408+0000 7f53917fa700 1 --2- 172.21.15.40:0/1917461410 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5378064ee0 0x7f5378067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.408+0000 7f53917fa700 1 -- 172.21.15.40:0/1917461410 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f5380081d50 con 0x7f539415c150 2023-11-08T08:21:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.408+0000 7f53989a6700 1 --2- 172.21.15.40:0/1917461410 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5378064ee0 0x7f5378067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.408+0000 7f53989a6700 1 --2- 172.21.15.40:0/1917461410 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5378064ee0 0x7f5378067390 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5388009860 tx=0x7f5388008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:14.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.412+0000 7f53917fa700 1 -- 172.21.15.40:0/1917461410 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5380050010 con 0x7f539415c150 2023-11-08T08:21:14.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.588+0000 7f53999a8700 1 -- 172.21.15.40:0/1917461410 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5394073d20 con 0x7f539415c150 2023-11-08T08:21:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.588+0000 7f53917fa700 1 -- 172.21.15.40:0/1917461410 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v53) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5380053630 con 0x7f539415c150 2023-11-08T08:21:14.595 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:21:14.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.592+0000 7f53999a8700 1 -- 172.21.15.40:0/1917461410 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5378064ee0 msgr2=0x7f5378067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:14.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.592+0000 7f53999a8700 1 --2- 172.21.15.40:0/1917461410 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5378064ee0 0x7f5378067390 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5388009860 tx=0x7f5388008030 comp rx=0 tx=0).stop 2023-11-08T08:21:14.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.592+0000 7f53999a8700 1 -- 172.21.15.40:0/1917461410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539415c150 msgr2=0x7f5394157c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:14.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.592+0000 7f53999a8700 1 --2- 172.21.15.40:0/1917461410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539415c150 0x7f5394157c70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f538000e930 tx=0x7f538000ec40 comp rx=0 tx=0).stop 2023-11-08T08:21:14.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.592+0000 7f53999a8700 1 -- 172.21.15.40:0/1917461410 shutdown_connections 2023-11-08T08:21:14.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.592+0000 7f53999a8700 1 --2- 172.21.15.40:0/1917461410 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5378064ee0 0x7f5378067390 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:14.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.592+0000 7f53999a8700 1 --2- 172.21.15.40:0/1917461410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f539415fc30 0x7f53941581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:14.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.592+0000 7f53999a8700 1 --2- 172.21.15.40:0/1917461410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f539415c150 0x7f5394157c70 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:14.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.592+0000 7f53999a8700 1 --2- 172.21.15.40:0/1917461410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f539406ff10 0x7f53940cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:14.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.592+0000 7f53999a8700 1 -- 172.21.15.40:0/1917461410 >> 172.21.15.40:0/1917461410 conn(0x7f53940d3680 msgr2=0x7f53940c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:14.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.592+0000 7f53999a8700 1 -- 172.21.15.40:0/1917461410 shutdown_connections 2023-11-08T08:21:14.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.592+0000 7f53999a8700 1 -- 172.21.15.40:0/1917461410 wait complete. 2023-11-08T08:21:14.620 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T08:21:14.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.784+0000 7fdc7248c700 1 Processor -- start 2023-11-08T08:21:14.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.788+0000 7fdc7248c700 1 -- start start 2023-11-08T08:21:14.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.788+0000 7fdc7248c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdc6c071aa0 0x7fdc6c06d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:14.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.788+0000 7fdc7248c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc6c069e00 0x7fdc6c06a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:14.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.788+0000 7fdc7248c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc6c06c600 con 0x7fdc6c15c640 2023-11-08T08:21:14.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.788+0000 7fdc7248c700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc6c06cef0 con 0x7fdc6c071aa0 2023-11-08T08:21:14.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.788+0000 7fdc7248c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0d38f0 con 0x7fdc6c069e00 2023-11-08T08:21:14.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.788+0000 7fdc7148a700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdc6c15c640 0x7fdc6c0d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45122/0 (socket says 172.21.15.40:45122) 2023-11-08T08:21:14.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.788+0000 7fdc7148a700 1 -- 172.21.15.40:0/438208656 learned_addr learned my addr 172.21.15.40:0/438208656 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:14.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.788+0000 7fdc70c89700 1 --2- 172.21.15.40:0/438208656 >> v2:172.21.15.40:3300/0 conn(0x7fdc6c071aa0 0x7fdc6c06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:14.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.788+0000 7fdc71c8b700 1 --2- 172.21.15.40:0/438208656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc6c069e00 0x7fdc6c06a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:14.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.788+0000 7fdc70c89700 1 -- 172.21.15.40:0/438208656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc6c069e00 msgr2=0x7fdc6c06a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:14.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.788+0000 7fdc70c89700 1 --2- 172.21.15.40:0/438208656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc6c069e00 0x7fdc6c06a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:14.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.788+0000 7fdc70c89700 1 -- 172.21.15.40:0/438208656 >> v1:172.21.15.40:6789/0 conn(0x7fdc6c15c640 legacy=0x7fdc6c0d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:14.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.788+0000 7fdc70c89700 1 -- 172.21.15.40:0/438208656 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c0722f0 con 0x7fdc6c071aa0 2023-11-08T08:21:14.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.788+0000 7fdc63fff700 1 -- 172.21.15.40:0/438208656 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661086323 0 0) 0x7fdc6c06c600 con 0x7fdc6c15c640 2023-11-08T08:21:14.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.792+0000 7fdc70c89700 1 --2- 172.21.15.40:0/438208656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc6c071aa0 0x7fdc6c06d310 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009850 tx=0x7fdc5c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9f78f42afe7437 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:14.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.792+0000 7fdc63fff700 1 -- 172.21.15.40:0/438208656 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c011830 con 0x7fdc6c071aa0 2023-11-08T08:21:14.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.792+0000 7fdc63fff700 1 -- 172.21.15.40:0/438208656 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5c011990 con 0x7fdc6c071aa0 2023-11-08T08:21:14.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.792+0000 7fdc63fff700 1 -- 172.21.15.40:0/438208656 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c011d40 con 0x7fdc6c071aa0 2023-11-08T08:21:14.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.792+0000 7fdc7248c700 1 -- 172.21.15.40:0/438208656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc6c071aa0 msgr2=0x7fdc6c06d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:14.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.792+0000 7fdc7248c700 1 --2- 172.21.15.40:0/438208656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc6c071aa0 0x7fdc6c06d310 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009850 tx=0x7fdc5c009b60 comp rx=0 tx=0).stop 2023-11-08T08:21:14.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.792+0000 7fdc7248c700 1 -- 172.21.15.40:0/438208656 shutdown_connections 2023-11-08T08:21:14.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.792+0000 7fdc7248c700 1 --2- 172.21.15.40:0/438208656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc6c069e00 0x7fdc6c06a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:14.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.792+0000 7fdc7248c700 1 --2- 172.21.15.40:0/438208656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc6c071aa0 0x7fdc6c06d310 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:14.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.792+0000 7fdc7248c700 1 -- 172.21.15.40:0/438208656 >> 172.21.15.40:0/438208656 conn(0x7fdc6c0b6520 msgr2=0x7fdc6c071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:14.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.792+0000 7fdc7248c700 1 -- 172.21.15.40:0/438208656 shutdown_connections 2023-11-08T08:21:14.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.792+0000 7fdc7248c700 1 -- 172.21.15.40:0/438208656 wait complete. 2023-11-08T08:21:14.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.796+0000 7fdc7248c700 1 Processor -- start 2023-11-08T08:21:14.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.796+0000 7fdc7248c700 1 -- start start 2023-11-08T08:21:14.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.796+0000 7fdc7248c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc6c069e00 0x7fdc6c0b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:14.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.796+0000 7fdc7248c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc6c071aa0 0x7fdc6c158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:14.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.796+0000 7fdc7248c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc6c15c640 0x7fdc6c0c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:14.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.796+0000 7fdc7248c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0c7f50 con 0x7fdc6c069e00 2023-11-08T08:21:14.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.796+0000 7fdc7248c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0cb940 con 0x7fdc6c071aa0 2023-11-08T08:21:14.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.796+0000 7fdc7248c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0cbc20 con 0x7fdc6c15c640 2023-11-08T08:21:14.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.800+0000 7fdc7148a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc6c069e00 0x7fdc6c0b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:14.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.800+0000 7fdc7148a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc6c069e00 0x7fdc6c0b24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43406/0 (socket says 172.21.15.40:43406) 2023-11-08T08:21:14.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.800+0000 7fdc7148a700 1 -- 172.21.15.40:0/2516637272 learned_addr learned my addr 172.21.15.40:0/2516637272 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:14.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.800+0000 7fdc70c89700 1 --2- 172.21.15.40:0/2516637272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc6c071aa0 0x7fdc6c158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:14.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.800+0000 7fdc71c8b700 1 --2- 172.21.15.40:0/2516637272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc6c15c640 0x7fdc6c0c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:14.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.800+0000 7fdc7148a700 1 -- 172.21.15.40:0/2516637272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc6c15c640 msgr2=0x7fdc6c0c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:14.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.800+0000 7fdc7148a700 1 --2- 172.21.15.40:0/2516637272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc6c15c640 0x7fdc6c0c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:14.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.800+0000 7fdc7148a700 1 -- 172.21.15.40:0/2516637272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc6c071aa0 msgr2=0x7fdc6c158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:14.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.800+0000 7fdc7148a700 1 --2- 172.21.15.40:0/2516637272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc6c071aa0 0x7fdc6c158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:14.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.800+0000 7fdc7148a700 1 -- 172.21.15.40:0/2516637272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc5c0094b0 con 0x7fdc6c069e00 2023-11-08T08:21:14.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.800+0000 7fdc7148a700 1 --2- 172.21.15.40:0/2516637272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc6c069e00 0x7fdc6c0b24e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdc6800c580 tx=0x7fdc6800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:14.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.800+0000 7fdc61ffb700 1 -- 172.21.15.40:0/2516637272 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc68004ca0 con 0x7fdc6c069e00 2023-11-08T08:21:14.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.800+0000 7fdc61ffb700 1 -- 172.21.15.40:0/2516637272 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc68004e20 con 0x7fdc6c069e00 2023-11-08T08:21:14.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.800+0000 7fdc7248c700 1 -- 172.21.15.40:0/2516637272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6c072010 con 0x7fdc6c069e00 2023-11-08T08:21:14.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.800+0000 7fdc61ffb700 1 -- 172.21.15.40:0/2516637272 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc68007310 con 0x7fdc6c069e00 2023-11-08T08:21:14.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.800+0000 7fdc7248c700 1 -- 172.21.15.40:0/2516637272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6c15ca10 con 0x7fdc6c069e00 2023-11-08T08:21:14.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.804+0000 7fdc7248c700 1 -- 172.21.15.40:0/2516637272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc6c06c830 con 0x7fdc6c069e00 2023-11-08T08:21:14.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.804+0000 7fdc61ffb700 1 -- 172.21.15.40:0/2516637272 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fdc6801a020 con 0x7fdc6c069e00 2023-11-08T08:21:14.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.808+0000 7fdc61ffb700 1 --2- 172.21.15.40:0/2516637272 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdc50064ee0 0x7fdc50067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:14.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.808+0000 7fdc61ffb700 1 -- 172.21.15.40:0/2516637272 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fdc68081860 con 0x7fdc6c069e00 2023-11-08T08:21:14.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.808+0000 7fdc70c89700 1 --2- 172.21.15.40:0/2516637272 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdc50064ee0 0x7fdc50067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:14.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.808+0000 7fdc70c89700 1 --2- 172.21.15.40:0/2516637272 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdc50064ee0 0x7fdc50067390 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0090b0 tx=0x7fdc5c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:14.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.812+0000 7fdc61ffb700 1 -- 172.21.15.40:0/2516637272 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fdc6804fc10 con 0x7fdc6c069e00 2023-11-08T08:21:14.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.988+0000 7fdc7248c700 1 -- 172.21.15.40:0/2516637272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdc6c0c8090 con 0x7fdc6c069e00 2023-11-08T08:21:14.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.992+0000 7fdc61ffb700 1 -- 172.21.15.40:0/2516637272 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v53) v1 ==== 84+0+126 (secure 0 0 0) 0x7fdc6c0c8090 con 0x7fdc6c069e00 2023-11-08T08:21:14.997 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:14.998 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2023-11-08T08:21:14.998 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:14.998 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2023-11-08T08:21:14.998 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2023-11-08T08:21:14.998 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:14.999 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:14.999 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 2, 2023-11-08T08:21:14.999 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": "rbd" 2023-11-08T08:21:14.999 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:14.999 INFO:teuthology.orchestra.run.smithi040.stdout:] 2023-11-08T08:21:15.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.992+0000 7fdc7248c700 1 -- 172.21.15.40:0/2516637272 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdc50064ee0 msgr2=0x7fdc50067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:15.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.992+0000 7fdc7248c700 1 --2- 172.21.15.40:0/2516637272 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdc50064ee0 0x7fdc50067390 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0090b0 tx=0x7fdc5c008030 comp rx=0 tx=0).stop 2023-11-08T08:21:15.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.992+0000 7fdc7248c700 1 -- 172.21.15.40:0/2516637272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc6c069e00 msgr2=0x7fdc6c0b24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:15.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.992+0000 7fdc7248c700 1 --2- 172.21.15.40:0/2516637272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc6c069e00 0x7fdc6c0b24e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdc6800c580 tx=0x7fdc6800c890 comp rx=0 tx=0).stop 2023-11-08T08:21:15.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.992+0000 7fdc7248c700 1 -- 172.21.15.40:0/2516637272 shutdown_connections 2023-11-08T08:21:15.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.992+0000 7fdc7248c700 1 --2- 172.21.15.40:0/2516637272 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdc50064ee0 0x7fdc50067390 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:15.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.992+0000 7fdc7248c700 1 --2- 172.21.15.40:0/2516637272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc6c15c640 0x7fdc6c0c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:15.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.992+0000 7fdc7248c700 1 --2- 172.21.15.40:0/2516637272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc6c071aa0 0x7fdc6c158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:15.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.992+0000 7fdc7248c700 1 --2- 172.21.15.40:0/2516637272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc6c069e00 0x7fdc6c0b24e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:15.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.992+0000 7fdc7248c700 1 -- 172.21.15.40:0/2516637272 >> 172.21.15.40:0/2516637272 conn(0x7fdc6c0b6520 msgr2=0x7fdc6c0b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:15.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.996+0000 7fdc7248c700 1 -- 172.21.15.40:0/2516637272 shutdown_connections 2023-11-08T08:21:15.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:14.996+0000 7fdc7248c700 1 -- 172.21.15.40:0/2516637272 wait complete. 2023-11-08T08:21:15.019 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T08:21:15.020 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T08:21:15.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.196+0000 7fea95ff9700 1 Processor -- start 2023-11-08T08:21:15.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.196+0000 7fea95ff9700 1 -- start start 2023-11-08T08:21:15.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea95ff9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fea9015c6e0 0x7fea90071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:15.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea95ff9700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fea90071fe0 0x7fea90069e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:15.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea95ff9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea9006a340 0x7fea900b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:15.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea95ff9700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea900d2800 con 0x7fea9015c6e0 2023-11-08T08:21:15.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea95ff9700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea9006a7c0 con 0x7fea90071fe0 2023-11-08T08:21:15.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea95ff9700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9015cab0 con 0x7fea9006a340 2023-11-08T08:21:15.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea94ff7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fea9015c6e0 0x7fea90071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:15.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea94ff7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fea9015c6e0 0x7fea90071aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43416/0 (socket says 172.21.15.40:43416) 2023-11-08T08:21:15.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea94ff7700 1 -- 172.21.15.40:0/949918094 learned_addr learned my addr 172.21.15.40:0/949918094 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:15.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea87fff700 1 --2- 172.21.15.40:0/949918094 >> v2:172.21.15.169:3300/0 conn(0x7fea90071fe0 0x7fea90069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:15.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea94ff7700 1 -- 172.21.15.40:0/949918094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea9006a340 msgr2=0x7fea900b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:15.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea957f8700 1 --2- 172.21.15.40:0/949918094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea9006a340 0x7fea900b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:15.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea94ff7700 1 --2- 172.21.15.40:0/949918094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea9006a340 0x7fea900b79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:15.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea94ff7700 1 -- 172.21.15.40:0/949918094 >> v2:172.21.15.169:3300/0 conn(0x7fea90071fe0 msgr2=0x7fea90069e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:15.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea94ff7700 1 --2- 172.21.15.40:0/949918094 >> v2:172.21.15.169:3300/0 conn(0x7fea90071fe0 0x7fea90069e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:15.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea94ff7700 1 -- 172.21.15.40:0/949918094 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea900702d0 con 0x7fea9015c6e0 2023-11-08T08:21:15.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea957f8700 1 --2- 172.21.15.40:0/949918094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea9006a340 0x7fea900b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:21:15.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea87fff700 1 --2- 172.21.15.40:0/949918094 >> v2:172.21.15.169:3300/0 conn(0x7fea90071fe0 0x7fea90069e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:15.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea94ff7700 1 --2- 172.21.15.40:0/949918094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9015c6e0 0x7fea90071aa0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fea9006f9b0 tx=0x7fea8c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac6800ab0e8bc6d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:15.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea877fe700 1 -- 172.21.15.40:0/949918094 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c00b8a0 con 0x7fea9015c6e0 2023-11-08T08:21:15.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea877fe700 1 -- 172.21.15.40:0/949918094 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8c00ba00 con 0x7fea9015c6e0 2023-11-08T08:21:15.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.200+0000 7fea877fe700 1 -- 172.21.15.40:0/949918094 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c00bd90 con 0x7fea9015c6e0 2023-11-08T08:21:15.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.204+0000 7fea95ff9700 1 -- 172.21.15.40:0/949918094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9015c6e0 msgr2=0x7fea90071aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:15.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.204+0000 7fea95ff9700 1 --2- 172.21.15.40:0/949918094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9015c6e0 0x7fea90071aa0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fea9006f9b0 tx=0x7fea8c009bf0 comp rx=0 tx=0).stop 2023-11-08T08:21:15.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.204+0000 7fea95ff9700 1 -- 172.21.15.40:0/949918094 shutdown_connections 2023-11-08T08:21:15.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.204+0000 7fea95ff9700 1 --2- 172.21.15.40:0/949918094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea9006a340 0x7fea900b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:15.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.204+0000 7fea95ff9700 1 --2- 172.21.15.40:0/949918094 >> v2:172.21.15.169:3300/0 conn(0x7fea90071fe0 0x7fea90069e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:15.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.204+0000 7fea95ff9700 1 --2- 172.21.15.40:0/949918094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9015c6e0 0x7fea90071aa0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:15.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.204+0000 7fea95ff9700 1 -- 172.21.15.40:0/949918094 >> 172.21.15.40:0/949918094 conn(0x7fea900b65b0 msgr2=0x7fea90071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:15.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.204+0000 7fea95ff9700 1 -- 172.21.15.40:0/949918094 shutdown_connections 2023-11-08T08:21:15.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.204+0000 7fea95ff9700 1 -- 172.21.15.40:0/949918094 wait complete. 2023-11-08T08:21:15.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.204+0000 7fea95ff9700 1 Processor -- start 2023-11-08T08:21:15.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.208+0000 7fea95ff9700 1 -- start start 2023-11-08T08:21:15.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.208+0000 7fea95ff9700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea9006a340 0x7fea900c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:15.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.208+0000 7fea95ff9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea90071fe0 0x7fea900cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:15.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.208+0000 7fea95ff9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9015c6e0 0x7fea9006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:15.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.208+0000 7fea95ff9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea900ce0d0 con 0x7fea9015c6e0 2023-11-08T08:21:15.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.208+0000 7fea95ff9700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea900cdc40 con 0x7fea90071fe0 2023-11-08T08:21:15.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.208+0000 7fea95ff9700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea900c8ff0 con 0x7fea9006a340 2023-11-08T08:21:15.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.208+0000 7fea957f8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9015c6e0 0x7fea9006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:15.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.208+0000 7fea957f8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9015c6e0 0x7fea9006d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43418/0 (socket says 172.21.15.40:43418) 2023-11-08T08:21:15.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.208+0000 7fea957f8700 1 -- 172.21.15.40:0/4074393558 learned_addr learned my addr 172.21.15.40:0/4074393558 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:15.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.208+0000 7fea94ff7700 1 --2- 172.21.15.40:0/4074393558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea9006a340 0x7fea900c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:15.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.208+0000 7fea87fff700 1 --2- 172.21.15.40:0/4074393558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea90071fe0 0x7fea900cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:15.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.208+0000 7fea957f8700 1 -- 172.21.15.40:0/4074393558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea9006a340 msgr2=0x7fea900c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:15.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.208+0000 7fea957f8700 1 --2- 172.21.15.40:0/4074393558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea9006a340 0x7fea900c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:15.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.208+0000 7fea957f8700 1 -- 172.21.15.40:0/4074393558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea90071fe0 msgr2=0x7fea900cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:15.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.208+0000 7fea957f8700 1 --2- 172.21.15.40:0/4074393558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea90071fe0 0x7fea900cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:15.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.208+0000 7fea957f8700 1 -- 172.21.15.40:0/4074393558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8c009570 con 0x7fea9015c6e0 2023-11-08T08:21:15.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.208+0000 7fea957f8700 1 --2- 172.21.15.40:0/4074393558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9015c6e0 0x7fea9006d1c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fea8800cef0 tx=0x7fea8800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:15.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.208+0000 7fea857fa700 1 -- 172.21.15.40:0/4074393558 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8800add0 con 0x7fea9015c6e0 2023-11-08T08:21:15.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.212+0000 7fea857fa700 1 -- 172.21.15.40:0/4074393558 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea88015440 con 0x7fea9015c6e0 2023-11-08T08:21:15.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.212+0000 7fea857fa700 1 -- 172.21.15.40:0/4074393558 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea880157f0 con 0x7fea9015c6e0 2023-11-08T08:21:15.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.212+0000 7fea95ff9700 1 -- 172.21.15.40:0/4074393558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea900d2cf0 con 0x7fea9015c6e0 2023-11-08T08:21:15.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.212+0000 7fea95ff9700 1 -- 172.21.15.40:0/4074393558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea900d2a10 con 0x7fea9015c6e0 2023-11-08T08:21:15.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.212+0000 7fea95ff9700 1 -- 172.21.15.40:0/4074393558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea900d2fd0 con 0x7fea9015c6e0 2023-11-08T08:21:15.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.212+0000 7fea857fa700 1 -- 172.21.15.40:0/4074393558 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fea8801c030 con 0x7fea9015c6e0 2023-11-08T08:21:15.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.216+0000 7fea857fa700 1 --2- 172.21.15.40:0/4074393558 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fea7c064f10 0x7fea7c0673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:15.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.216+0000 7fea857fa700 1 -- 172.21.15.40:0/4074393558 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fea88082150 con 0x7fea9015c6e0 2023-11-08T08:21:15.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.216+0000 7fea94ff7700 1 --2- 172.21.15.40:0/4074393558 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fea7c064f10 0x7fea7c0673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:15.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.216+0000 7fea94ff7700 1 --2- 172.21.15.40:0/4074393558 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fea7c064f10 0x7fea7c0673c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009860 tx=0x7fea8c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:15.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.220+0000 7fea857fa700 1 -- 172.21.15.40:0/4074393558 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fea88050410 con 0x7fea9015c6e0 2023-11-08T08:21:15.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:15.392+0000 7fea95ff9700 1 -- 172.21.15.40:0/4074393558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fea90073d30 con 0x7fea9015c6e0 2023-11-08T08:21:16.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.252+0000 7fea857fa700 1 -- 172.21.15.40:0/4074393558 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v54) v1 ==== 138+0+0 (secure 0 0 0) 0x7fea8801a0b0 con 0x7fea9015c6e0 2023-11-08T08:21:16.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.260+0000 7fea95ff9700 1 -- 172.21.15.40:0/4074393558 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fea7c064f10 msgr2=0x7fea7c0673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:16.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.260+0000 7fea95ff9700 1 --2- 172.21.15.40:0/4074393558 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fea7c064f10 0x7fea7c0673c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009860 tx=0x7fea8c008030 comp rx=0 tx=0).stop 2023-11-08T08:21:16.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.260+0000 7fea95ff9700 1 -- 172.21.15.40:0/4074393558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9015c6e0 msgr2=0x7fea9006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:16.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.260+0000 7fea95ff9700 1 --2- 172.21.15.40:0/4074393558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9015c6e0 0x7fea9006d1c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fea8800cef0 tx=0x7fea8800a340 comp rx=0 tx=0).stop 2023-11-08T08:21:16.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.260+0000 7fea95ff9700 1 -- 172.21.15.40:0/4074393558 shutdown_connections 2023-11-08T08:21:16.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.260+0000 7fea95ff9700 1 --2- 172.21.15.40:0/4074393558 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fea7c064f10 0x7fea7c0673c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:16.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.260+0000 7fea95ff9700 1 --2- 172.21.15.40:0/4074393558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9015c6e0 0x7fea9006d1c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:16.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.260+0000 7fea95ff9700 1 --2- 172.21.15.40:0/4074393558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea90071fe0 0x7fea900cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:16.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.260+0000 7fea95ff9700 1 --2- 172.21.15.40:0/4074393558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea9006a340 0x7fea900c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:16.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.260+0000 7fea95ff9700 1 -- 172.21.15.40:0/4074393558 >> 172.21.15.40:0/4074393558 conn(0x7fea900b65b0 msgr2=0x7fea900b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:16.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.260+0000 7fea95ff9700 1 -- 172.21.15.40:0/4074393558 shutdown_connections 2023-11-08T08:21:16.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.260+0000 7fea95ff9700 1 -- 172.21.15.40:0/4074393558 wait complete. 2023-11-08T08:21:16.271 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2023-11-08T08:21:16.292 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T08:21:16.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.480+0000 7f54f5bd2700 1 Processor -- start 2023-11-08T08:21:16.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.480+0000 7f54f5bd2700 1 -- start start 2023-11-08T08:21:16.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.484+0000 7f54f5bd2700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f54f015c7c0 0x7f54f006cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:16.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.484+0000 7f54f5bd2700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54f0161030 0x7f54f00cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:16.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.484+0000 7f54f5bd2700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54f00cdba0 con 0x7f54f00cd700 2023-11-08T08:21:16.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.484+0000 7f54f5bd2700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f006c010 con 0x7f54f015c7c0 2023-11-08T08:21:16.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.484+0000 7f54f5bd2700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f006c2f0 con 0x7f54f0161030 2023-11-08T08:21:16.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.484+0000 7f54f53d1700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f54f00cd700 0x7f54f00cddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45132/0 (socket says 172.21.15.40:45132) 2023-11-08T08:21:16.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.484+0000 7f54f4bd0700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f54f015c7c0 0x7f54f006cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:16.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.484+0000 7f54f53d1700 1 -- 172.21.15.40:0/2265658153 learned_addr learned my addr 172.21.15.40:0/2265658153 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:16.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.484+0000 7f54effff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54f0161030 0x7f54f00cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:16.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.488+0000 7f54f4bd0700 1 -- 172.21.15.40:0/2265658153 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54f0161030 msgr2=0x7f54f00cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:16.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.488+0000 7f54f4bd0700 1 --2- 172.21.15.40:0/2265658153 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54f0161030 0x7f54f00cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:16.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.488+0000 7f54f4bd0700 1 -- 172.21.15.40:0/2265658153 >> v1:172.21.15.40:6789/0 conn(0x7f54f00cd700 legacy=0x7f54f00cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:16.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.488+0000 7f54f4bd0700 1 -- 172.21.15.40:0/2265658153 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f006c5d0 con 0x7f54f015c7c0 2023-11-08T08:21:16.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.488+0000 7f54ef7fe700 1 -- 172.21.15.40:0/2265658153 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 885702047 0 0) 0x7f54f00cdba0 con 0x7f54f00cd700 2023-11-08T08:21:16.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.488+0000 7f54f4bd0700 1 --2- 172.21.15.40:0/2265658153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f015c7c0 0x7f54f006cdd0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009850 tx=0x7f54e4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104a85cad801e00b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:16.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.488+0000 7f54ef7fe700 1 -- 172.21.15.40:0/2265658153 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e4011830 con 0x7f54f015c7c0 2023-11-08T08:21:16.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.488+0000 7f54ef7fe700 1 -- 172.21.15.40:0/2265658153 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e4011990 con 0x7f54f015c7c0 2023-11-08T08:21:16.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.488+0000 7f54ef7fe700 1 -- 172.21.15.40:0/2265658153 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e4011d40 con 0x7f54f015c7c0 2023-11-08T08:21:16.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.488+0000 7f54f5bd2700 1 -- 172.21.15.40:0/2265658153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f015c7c0 msgr2=0x7f54f006cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:16.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.488+0000 7f54f5bd2700 1 --2- 172.21.15.40:0/2265658153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f015c7c0 0x7f54f006cdd0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009850 tx=0x7f54e4009b60 comp rx=0 tx=0).stop 2023-11-08T08:21:16.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.488+0000 7f54f5bd2700 1 -- 172.21.15.40:0/2265658153 shutdown_connections 2023-11-08T08:21:16.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.488+0000 7f54f5bd2700 1 --2- 172.21.15.40:0/2265658153 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54f0161030 0x7f54f00cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:16.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.488+0000 7f54f5bd2700 1 --2- 172.21.15.40:0/2265658153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f015c7c0 0x7f54f006cdd0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:16.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.488+0000 7f54f5bd2700 1 -- 172.21.15.40:0/2265658153 >> 172.21.15.40:0/2265658153 conn(0x7f54f00d3800 msgr2=0x7f54f00cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:16.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.488+0000 7f54f5bd2700 1 -- 172.21.15.40:0/2265658153 shutdown_connections 2023-11-08T08:21:16.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.488+0000 7f54f5bd2700 1 -- 172.21.15.40:0/2265658153 wait complete. 2023-11-08T08:21:16.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.492+0000 7f54f5bd2700 1 Processor -- start 2023-11-08T08:21:16.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.492+0000 7f54f5bd2700 1 -- start start 2023-11-08T08:21:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54f5bd2700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54f00cd700 0x7f54f00c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54f5bd2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54f015c7c0 0x7f54f00d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:16.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54f5bd2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f00d23c0 0x7f54f0157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:16.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54f5bd2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f00c9640 con 0x7f54f00d23c0 2023-11-08T08:21:16.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54f5bd2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f00d2840 con 0x7f54f015c7c0 2023-11-08T08:21:16.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54f5bd2700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f00c6740 con 0x7f54f00cd700 2023-11-08T08:21:16.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54f53d1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f00d23c0 0x7f54f0157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:16.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54f53d1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f00d23c0 0x7f54f0157da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43442/0 (socket says 172.21.15.40:43442) 2023-11-08T08:21:16.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54f53d1700 1 -- 172.21.15.40:0/890245621 learned_addr learned my addr 172.21.15.40:0/890245621 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:16.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54effff700 1 --2- 172.21.15.40:0/890245621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54f015c7c0 0x7f54f00d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:16.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54f4bd0700 1 --2- 172.21.15.40:0/890245621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54f00cd700 0x7f54f00c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:16.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54f53d1700 1 -- 172.21.15.40:0/890245621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54f00cd700 msgr2=0x7f54f00c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:16.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54f53d1700 1 --2- 172.21.15.40:0/890245621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54f00cd700 0x7f54f00c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:16.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54f53d1700 1 -- 172.21.15.40:0/890245621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54f015c7c0 msgr2=0x7f54f00d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:16.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54f53d1700 1 --2- 172.21.15.40:0/890245621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54f015c7c0 0x7f54f00d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:16.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54f53d1700 1 -- 172.21.15.40:0/890245621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e40094b0 con 0x7f54f00d23c0 2023-11-08T08:21:16.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54f4bd0700 1 --2- 172.21.15.40:0/890245621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54f00cd700 0x7f54f00c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:21:16.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54effff700 1 --2- 172.21.15.40:0/890245621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54f015c7c0 0x7f54f00d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:21:16.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54f53d1700 1 --2- 172.21.15.40:0/890245621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f00d23c0 0x7f54f0157da0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f54e800cc10 tx=0x7f54e80021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:16.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54ed7fa700 1 -- 172.21.15.40:0/890245621 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e800ab80 con 0x7f54f00d23c0 2023-11-08T08:21:16.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54ed7fa700 1 -- 172.21.15.40:0/890245621 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e800ad00 con 0x7f54f00d23c0 2023-11-08T08:21:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54ed7fa700 1 -- 172.21.15.40:0/890245621 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e8007310 con 0x7f54f00d23c0 2023-11-08T08:21:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54f5bd2700 1 -- 172.21.15.40:0/890245621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f015cb90 con 0x7f54f00d23c0 2023-11-08T08:21:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.496+0000 7f54f5bd2700 1 -- 172.21.15.40:0/890245621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f006c8b0 con 0x7f54f00d23c0 2023-11-08T08:21:16.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.500+0000 7f54f5bd2700 1 -- 172.21.15.40:0/890245621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f00ce500 con 0x7f54f00d23c0 2023-11-08T08:21:16.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.500+0000 7f54ed7fa700 1 -- 172.21.15.40:0/890245621 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f54e8010070 con 0x7f54f00d23c0 2023-11-08T08:21:16.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.504+0000 7f54ed7fa700 1 --2- 172.21.15.40:0/890245621 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f54d4064ee0 0x7f54d4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:16.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.504+0000 7f54ed7fa700 1 -- 172.21.15.40:0/890245621 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f54e8081850 con 0x7f54f00d23c0 2023-11-08T08:21:16.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.504+0000 7f54f4bd0700 1 --2- 172.21.15.40:0/890245621 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f54d4064ee0 0x7f54d4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:16.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.504+0000 7f54f4bd0700 1 --2- 172.21.15.40:0/890245621 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f54d4064ee0 0x7f54d4067390 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f54f00c97e0 tx=0x7f54e4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:16.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.512+0000 7f54ed7fa700 1 -- 172.21.15.40:0/890245621 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f54e804f9a0 con 0x7f54f00d23c0 2023-11-08T08:21:16.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:16.688+0000 7f54f5bd2700 1 -- 172.21.15.40:0/890245621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f54f00c6940 con 0x7f54f00d23c0 2023-11-08T08:21:17.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.260+0000 7f54ed7fa700 1 -- 172.21.15.40:0/890245621 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v55) v1 ==== 170+0+0 (secure 0 0 0) 0x7f54f00c6940 con 0x7f54f00d23c0 2023-11-08T08:21:17.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.272+0000 7f54f5bd2700 1 -- 172.21.15.40:0/890245621 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f54d4064ee0 msgr2=0x7f54d4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:17.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.272+0000 7f54f5bd2700 1 --2- 172.21.15.40:0/890245621 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f54d4064ee0 0x7f54d4067390 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f54f00c97e0 tx=0x7f54e4008030 comp rx=0 tx=0).stop 2023-11-08T08:21:17.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.272+0000 7f54f5bd2700 1 -- 172.21.15.40:0/890245621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f00d23c0 msgr2=0x7f54f0157da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:17.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.272+0000 7f54f5bd2700 1 --2- 172.21.15.40:0/890245621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f00d23c0 0x7f54f0157da0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f54e800cc10 tx=0x7f54e80021c0 comp rx=0 tx=0).stop 2023-11-08T08:21:17.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.276+0000 7f54f5bd2700 1 -- 172.21.15.40:0/890245621 shutdown_connections 2023-11-08T08:21:17.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.276+0000 7f54f5bd2700 1 --2- 172.21.15.40:0/890245621 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f54d4064ee0 0x7f54d4067390 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:17.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.276+0000 7f54f5bd2700 1 --2- 172.21.15.40:0/890245621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54f00d23c0 0x7f54f0157da0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:17.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.276+0000 7f54f5bd2700 1 --2- 172.21.15.40:0/890245621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54f015c7c0 0x7f54f00d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:17.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.276+0000 7f54f5bd2700 1 --2- 172.21.15.40:0/890245621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54f00cd700 0x7f54f00c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.276+0000 7f54f5bd2700 1 -- 172.21.15.40:0/890245621 >> 172.21.15.40:0/890245621 conn(0x7f54f00d3800 msgr2=0x7f54f00cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.276+0000 7f54f5bd2700 1 -- 172.21.15.40:0/890245621 shutdown_connections 2023-11-08T08:21:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.276+0000 7f54f5bd2700 1 -- 172.21.15.40:0/890245621 wait complete. 2023-11-08T08:21:17.294 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2023-11-08T08:21:17.318 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T08:21:17.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.488+0000 7fc30359e700 1 Processor -- start 2023-11-08T08:21:17.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.488+0000 7fc30359e700 1 -- start start 2023-11-08T08:21:17.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.488+0000 7fc30359e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc30415c150 0x7fc3040b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:17.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.488+0000 7fc30359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc30406ff10 0x7fc304070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:17.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.488+0000 7fc30359e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc30415c520 con 0x7fc30415fc30 2023-11-08T08:21:17.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.488+0000 7fc30359e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3040b4240 con 0x7fc30415c150 2023-11-08T08:21:17.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.488+0000 7fc30359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040708a0 con 0x7fc30406ff10 2023-11-08T08:21:17.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.492+0000 7fc30259c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc30415c150 0x7fc3040b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:17.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.492+0000 7fc302d9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc30415fc30 0x7fc3041600b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33918/0 (socket says 172.21.15.40:33918) 2023-11-08T08:21:17.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.492+0000 7fc302d9d700 1 -- 172.21.15.40:0/159317030 learned_addr learned my addr 172.21.15.40:0/159317030 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:17.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.492+0000 7fc301d9b700 1 --2- 172.21.15.40:0/159317030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc30406ff10 0x7fc304070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:17.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.492+0000 7fc30159a700 1 -- 172.21.15.40:0/159317030 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2831126231 0 0) 0x7fc30415c520 con 0x7fc30415fc30 2023-11-08T08:21:17.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.492+0000 7fc30159a700 1 -- 172.21.15.40:0/159317030 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2ec0035d0 con 0x7fc30415fc30 2023-11-08T08:21:17.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.492+0000 7fc30259c700 1 -- 172.21.15.40:0/159317030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc30406ff10 msgr2=0x7fc304070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:17.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.492+0000 7fc30259c700 1 --2- 172.21.15.40:0/159317030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc30406ff10 0x7fc304070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:17.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.492+0000 7fc30259c700 1 -- 172.21.15.40:0/159317030 >> v1:172.21.15.40:6789/0 conn(0x7fc30415fc30 legacy=0x7fc3041600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:17.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.492+0000 7fc30259c700 1 -- 172.21.15.40:0/159317030 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30406ac00 con 0x7fc30415c150 2023-11-08T08:21:17.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.492+0000 7fc301d9b700 1 --2- 172.21.15.40:0/159317030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc30406ff10 0x7fc304070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:21:17.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.492+0000 7fc30259c700 1 --2- 172.21.15.40:0/159317030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc30415c150 0x7fc3040b3d00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009850 tx=0x7fc2f4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d5009a1ce1afe7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:17.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.492+0000 7fc30159a700 1 -- 172.21.15.40:0/159317030 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4011830 con 0x7fc30415c150 2023-11-08T08:21:17.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.492+0000 7fc30159a700 1 -- 172.21.15.40:0/159317030 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f4011990 con 0x7fc30415c150 2023-11-08T08:21:17.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.492+0000 7fc30159a700 1 -- 172.21.15.40:0/159317030 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4011d40 con 0x7fc30415c150 2023-11-08T08:21:17.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.496+0000 7fc30359e700 1 -- 172.21.15.40:0/159317030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc30415c150 msgr2=0x7fc3040b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:17.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.496+0000 7fc30359e700 1 --2- 172.21.15.40:0/159317030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc30415c150 0x7fc3040b3d00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009850 tx=0x7fc2f4009b60 comp rx=0 tx=0).stop 2023-11-08T08:21:17.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.496+0000 7fc30359e700 1 -- 172.21.15.40:0/159317030 shutdown_connections 2023-11-08T08:21:17.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.496+0000 7fc30359e700 1 --2- 172.21.15.40:0/159317030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc30406ff10 0x7fc304070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:17.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.496+0000 7fc30359e700 1 --2- 172.21.15.40:0/159317030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc30415c150 0x7fc3040b3d00 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:17.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.496+0000 7fc30359e700 1 -- 172.21.15.40:0/159317030 >> 172.21.15.40:0/159317030 conn(0x7fc3040d3680 msgr2=0x7fc3040cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:17.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.496+0000 7fc30359e700 1 -- 172.21.15.40:0/159317030 shutdown_connections 2023-11-08T08:21:17.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.496+0000 7fc30359e700 1 -- 172.21.15.40:0/159317030 wait complete. 2023-11-08T08:21:17.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.496+0000 7fc30359e700 1 Processor -- start 2023-11-08T08:21:17.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.496+0000 7fc30359e700 1 -- start start 2023-11-08T08:21:17.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc30359e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc30406ff10 0x7fc3041f1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:17.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc30359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc30415c150 0x7fc3040c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:17.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc30359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc30415fc30 0x7fc3040c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:17.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc30359e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30415e230 con 0x7fc30415fc30 2023-11-08T08:21:17.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc30359e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30415e510 con 0x7fc30406ff10 2023-11-08T08:21:17.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc30359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30415e7f0 con 0x7fc30415c150 2023-11-08T08:21:17.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc302d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc30415fc30 0x7fc3040c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:17.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc302d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc30415fc30 0x7fc3040c9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51368/0 (socket says 172.21.15.40:51368) 2023-11-08T08:21:17.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc302d9d700 1 -- 172.21.15.40:0/2637652282 learned_addr learned my addr 172.21.15.40:0/2637652282 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:17.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc301d9b700 1 --2- 172.21.15.40:0/2637652282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc30415c150 0x7fc3040c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:17.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc30259c700 1 --2- 172.21.15.40:0/2637652282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc30406ff10 0x7fc3041f1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:17.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc302d9d700 1 -- 172.21.15.40:0/2637652282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc30415c150 msgr2=0x7fc3040c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:17.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc302d9d700 1 --2- 172.21.15.40:0/2637652282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc30415c150 0x7fc3040c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:17.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc302d9d700 1 -- 172.21.15.40:0/2637652282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc30406ff10 msgr2=0x7fc3041f1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:17.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc302d9d700 1 --2- 172.21.15.40:0/2637652282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc30406ff10 0x7fc3041f1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:17.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc302d9d700 1 -- 172.21.15.40:0/2637652282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f8009450 con 0x7fc30415fc30 2023-11-08T08:21:17.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc302d9d700 1 --2- 172.21.15.40:0/2637652282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc30415fc30 0x7fc3040c9f80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00caa0 tx=0x7fc2fc00cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:17.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc2f2ffd700 1 -- 172.21.15.40:0/2637652282 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc007480 con 0x7fc30415fc30 2023-11-08T08:21:17.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc2f2ffd700 1 -- 172.21.15.40:0/2637652282 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2fc0075e0 con 0x7fc30415fc30 2023-11-08T08:21:17.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc30359e700 1 -- 172.21.15.40:0/2637652282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f40094b0 con 0x7fc30415fc30 2023-11-08T08:21:17.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc2f2ffd700 1 -- 172.21.15.40:0/2637652282 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc007970 con 0x7fc30415fc30 2023-11-08T08:21:17.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.500+0000 7fc30359e700 1 -- 172.21.15.40:0/2637652282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3040d2450 con 0x7fc30415fc30 2023-11-08T08:21:17.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.504+0000 7fc30359e700 1 -- 172.21.15.40:0/2637652282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc30406cc50 con 0x7fc30415fc30 2023-11-08T08:21:17.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.504+0000 7fc2f2ffd700 1 -- 172.21.15.40:0/2637652282 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc2fc019030 con 0x7fc30415fc30 2023-11-08T08:21:17.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.508+0000 7fc2f2ffd700 1 --2- 172.21.15.40:0/2637652282 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc2e4064ee0 0x7fc2e4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:17.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.508+0000 7fc2f2ffd700 1 -- 172.21.15.40:0/2637652282 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fc2fc081750 con 0x7fc30415fc30 2023-11-08T08:21:17.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.508+0000 7fc30259c700 1 --2- 172.21.15.40:0/2637652282 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc2e4064ee0 0x7fc2e4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.508+0000 7fc30259c700 1 --2- 172.21.15.40:0/2637652282 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc2e4064ee0 0x7fc2e4067390 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc2f40090b0 tx=0x7fc2f4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.516+0000 7fc2f2ffd700 1 -- 172.21.15.40:0/2637652282 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc2fc04f810 con 0x7fc30415fc30 2023-11-08T08:21:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.724+0000 7fc30359e700 1 -- 172.21.15.40:0/2637652282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc3040d2730 con 0x7fc30415fc30 2023-11-08T08:21:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.736+0000 7fc2f2ffd700 1 -- 172.21.15.40:0/2637652282 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc2fc07fe60 con 0x7fc30415fc30 2023-11-08T08:21:17.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.752+0000 7fc30359e700 1 -- 172.21.15.40:0/2637652282 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc2e4064ee0 msgr2=0x7fc2e4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:17.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.752+0000 7fc30359e700 1 --2- 172.21.15.40:0/2637652282 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc2e4064ee0 0x7fc2e4067390 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc2f40090b0 tx=0x7fc2f4008030 comp rx=0 tx=0).stop 2023-11-08T08:21:17.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.752+0000 7fc30359e700 1 -- 172.21.15.40:0/2637652282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc30415fc30 msgr2=0x7fc3040c9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:17.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.752+0000 7fc30359e700 1 --2- 172.21.15.40:0/2637652282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc30415fc30 0x7fc3040c9f80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00caa0 tx=0x7fc2fc00cad0 comp rx=0 tx=0).stop 2023-11-08T08:21:17.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.752+0000 7fc30359e700 1 -- 172.21.15.40:0/2637652282 shutdown_connections 2023-11-08T08:21:17.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.752+0000 7fc30359e700 1 --2- 172.21.15.40:0/2637652282 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc2e4064ee0 0x7fc2e4067390 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:17.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.752+0000 7fc30359e700 1 --2- 172.21.15.40:0/2637652282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc30415fc30 0x7fc3040c9f80 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:17.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.752+0000 7fc30359e700 1 --2- 172.21.15.40:0/2637652282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc30415c150 0x7fc3040c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:17.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.752+0000 7fc30359e700 1 --2- 172.21.15.40:0/2637652282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc30406ff10 0x7fc3041f1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:17.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.752+0000 7fc30359e700 1 -- 172.21.15.40:0/2637652282 >> 172.21.15.40:0/2637652282 conn(0x7fc3040d3680 msgr2=0x7fc3040b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:17.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.756+0000 7fc30359e700 1 -- 172.21.15.40:0/2637652282 shutdown_connections 2023-11-08T08:21:17.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.756+0000 7fc30359e700 1 -- 172.21.15.40:0/2637652282 wait complete. 2023-11-08T08:21:17.768 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 8 and data pool 9 2023-11-08T08:21:17.796 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:21:17.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.976+0000 7ff5d56e7700 1 Processor -- start 2023-11-08T08:21:17.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.980+0000 7ff5d56e7700 1 -- start start 2023-11-08T08:21:17.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.980+0000 7ff5d56e7700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff5d015c6d0 0x7ff5d00b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:17.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.980+0000 7ff5d56e7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5d006a300 0x7ff5d006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:17.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.980+0000 7ff5d56e7700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d0072270 con 0x7ff5d00cfc40 2023-11-08T08:21:17.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.980+0000 7ff5d56e7700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d00d0400 con 0x7ff5d015c6d0 2023-11-08T08:21:17.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.980+0000 7ff5d56e7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d00d0120 con 0x7ff5d006a300 2023-11-08T08:21:17.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.980+0000 7ff5ceffd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff5d015c6d0 0x7ff5d00b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:17.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.980+0000 7ff5cf7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5d006a300 0x7ff5d006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:17.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.980+0000 7ff5ceffd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff5d015c6d0 0x7ff5d00b17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:35984/0 (socket says 172.21.15.40:35984) 2023-11-08T08:21:17.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.980+0000 7ff5ceffd700 1 -- 172.21.15.40:0/3645079459 learned_addr learned my addr 172.21.15.40:0/3645079459 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:17.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.984+0000 7ff5ceffd700 1 -- 172.21.15.40:0/3645079459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5d006a300 msgr2=0x7ff5d006c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:17.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.984+0000 7ff5ceffd700 1 --2- 172.21.15.40:0/3645079459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5d006a300 0x7ff5d006c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:17.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.984+0000 7ff5ceffd700 1 -- 172.21.15.40:0/3645079459 >> v1:172.21.15.169:6789/0 conn(0x7ff5d00cfc40 legacy=0x7ff5d0069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:17.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.984+0000 7ff5ceffd700 1 -- 172.21.15.40:0/3645079459 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d0071c20 con 0x7ff5d015c6d0 2023-11-08T08:21:17.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.984+0000 7ff5ceffd700 1 --2- 172.21.15.40:0/3645079459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5d015c6d0 0x7ff5d00b17a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009910 tx=0x7ff5c0009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bfd9100187fec7b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:17.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.984+0000 7ff5ce7fc700 1 -- 172.21.15.40:0/3645079459 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0011830 con 0x7ff5d015c6d0 2023-11-08T08:21:17.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.984+0000 7ff5ce7fc700 1 -- 172.21.15.40:0/3645079459 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c0011990 con 0x7ff5d015c6d0 2023-11-08T08:21:17.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.984+0000 7ff5ce7fc700 1 -- 172.21.15.40:0/3645079459 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0011d40 con 0x7ff5d015c6d0 2023-11-08T08:21:17.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.984+0000 7ff5d56e7700 1 -- 172.21.15.40:0/3645079459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5d015c6d0 msgr2=0x7ff5d00b17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:17.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.984+0000 7ff5d56e7700 1 --2- 172.21.15.40:0/3645079459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5d015c6d0 0x7ff5d00b17a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009910 tx=0x7ff5c0009c20 comp rx=0 tx=0).stop 2023-11-08T08:21:17.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.984+0000 7ff5d56e7700 1 -- 172.21.15.40:0/3645079459 shutdown_connections 2023-11-08T08:21:17.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.984+0000 7ff5d56e7700 1 --2- 172.21.15.40:0/3645079459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5d006a300 0x7ff5d006c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:17.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.984+0000 7ff5d56e7700 1 --2- 172.21.15.40:0/3645079459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5d015c6d0 0x7ff5d00b17a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:17.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.984+0000 7ff5d56e7700 1 -- 172.21.15.40:0/3645079459 >> 172.21.15.40:0/3645079459 conn(0x7ff5d00b6520 msgr2=0x7ff5d0071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:17.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.984+0000 7ff5d56e7700 1 -- 172.21.15.40:0/3645079459 shutdown_connections 2023-11-08T08:21:17.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.984+0000 7ff5d56e7700 1 -- 172.21.15.40:0/3645079459 wait complete. 2023-11-08T08:21:17.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.988+0000 7ff5d56e7700 1 Processor -- start 2023-11-08T08:21:17.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.988+0000 7ff5d56e7700 1 -- start start 2023-11-08T08:21:17.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.988+0000 7ff5d56e7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5d006a300 0x7ff5d0158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:17.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.988+0000 7ff5d56e7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5d00cfc40 0x7ff5d00b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:17.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.988+0000 7ff5d56e7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5d015c6d0 0x7ff5d00b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:17.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.988+0000 7ff5d56e7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d01f6080 con 0x7ff5d006a300 2023-11-08T08:21:17.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.988+0000 7ff5d56e7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d01f5500 con 0x7ff5d00cfc40 2023-11-08T08:21:17.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.988+0000 7ff5d56e7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d01f57e0 con 0x7ff5d015c6d0 2023-11-08T08:21:17.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.988+0000 7ff5ceffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5d006a300 0x7ff5d0158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:17.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.988+0000 7ff5ceffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5d006a300 0x7ff5d0158930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51376/0 (socket says 172.21.15.40:51376) 2023-11-08T08:21:17.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.988+0000 7ff5ceffd700 1 -- 172.21.15.40:0/2146087425 learned_addr learned my addr 172.21.15.40:0/2146087425 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:17.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.992+0000 7ff5c7fff700 1 --2- 172.21.15.40:0/2146087425 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5d00cfc40 0x7ff5d00b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:17.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.992+0000 7ff5cf7fe700 1 --2- 172.21.15.40:0/2146087425 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5d015c6d0 0x7ff5d00b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:17.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.992+0000 7ff5ceffd700 1 -- 172.21.15.40:0/2146087425 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5d015c6d0 msgr2=0x7ff5d00b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:17.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.992+0000 7ff5ceffd700 1 --2- 172.21.15.40:0/2146087425 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5d015c6d0 0x7ff5d00b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:17.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.992+0000 7ff5ceffd700 1 -- 172.21.15.40:0/2146087425 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5d00cfc40 msgr2=0x7ff5d00b0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:17.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.992+0000 7ff5ceffd700 1 --2- 172.21.15.40:0/2146087425 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5d00cfc40 0x7ff5d00b0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:17.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.992+0000 7ff5ceffd700 1 -- 172.21.15.40:0/2146087425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c0009570 con 0x7ff5d006a300 2023-11-08T08:21:17.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.992+0000 7ff5ceffd700 1 --2- 172.21.15.40:0/2146087425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5d006a300 0x7ff5d0158930 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff5d006ecc0 tx=0x7ff5c0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:18.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.992+0000 7ff5c77fe700 1 -- 172.21.15.40:0/2146087425 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0004590 con 0x7ff5d006a300 2023-11-08T08:21:18.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.992+0000 7ff5c77fe700 1 -- 172.21.15.40:0/2146087425 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c001e070 con 0x7ff5d006a300 2023-11-08T08:21:18.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.992+0000 7ff5c77fe700 1 -- 172.21.15.40:0/2146087425 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0003900 con 0x7ff5d006a300 2023-11-08T08:21:18.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.992+0000 7ff5d56e7700 1 -- 172.21.15.40:0/2146087425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d015caa0 con 0x7ff5d006a300 2023-11-08T08:21:18.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.992+0000 7ff5d56e7700 1 -- 172.21.15.40:0/2146087425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d0071f00 con 0x7ff5d006a300 2023-11-08T08:21:18.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.992+0000 7ff5d56e7700 1 -- 172.21.15.40:0/2146087425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d0073cb0 con 0x7ff5d006a300 2023-11-08T08:21:18.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.996+0000 7ff5c77fe700 1 -- 172.21.15.40:0/2146087425 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff5c000f050 con 0x7ff5d006a300 2023-11-08T08:21:18.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.996+0000 7ff5c77fe700 1 --2- 172.21.15.40:0/2146087425 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff5b0064ee0 0x7ff5b0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:18.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.996+0000 7ff5c77fe700 1 -- 172.21.15.40:0/2146087425 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff5c0082dd0 con 0x7ff5d006a300 2023-11-08T08:21:18.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:17.996+0000 7ff5c7fff700 1 --2- 172.21.15.40:0/2146087425 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff5b0064ee0 0x7ff5b0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:18.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.000+0000 7ff5c7fff700 1 --2- 172.21.15.40:0/2146087425 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff5b0064ee0 0x7ff5b0067390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00cdac0 tx=0x7ff5b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:18.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.004+0000 7ff5c77fe700 1 -- 172.21.15.40:0/2146087425 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ff5c0050d40 con 0x7ff5d006a300 2023-11-08T08:21:18.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.180+0000 7ff5d56e7700 1 -- 172.21.15.40:0/2146087425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5d00cd810 con 0x7ff5d006a300 2023-11-08T08:21:18.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.180+0000 7ff5c77fe700 1 -- 172.21.15.40:0/2146087425 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+18097 (secure 0 0 0) 0x7ff5c0020090 con 0x7ff5d006a300 2023-11-08T08:21:18.189 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:18.189 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":56,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:21:17.729220+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:21:15.464908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-08T08:21:16.755373+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:21:18.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.184+0000 7ff5d56e7700 1 -- 172.21.15.40:0/2146087425 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff5b0064ee0 msgr2=0x7ff5b0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:18.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.184+0000 7ff5d56e7700 1 --2- 172.21.15.40:0/2146087425 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff5b0064ee0 0x7ff5b0067390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00cdac0 tx=0x7ff5b8009040 comp rx=0 tx=0).stop 2023-11-08T08:21:18.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.184+0000 7ff5d56e7700 1 -- 172.21.15.40:0/2146087425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5d006a300 msgr2=0x7ff5d0158930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:18.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.184+0000 7ff5d56e7700 1 --2- 172.21.15.40:0/2146087425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5d006a300 0x7ff5d0158930 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff5d006ecc0 tx=0x7ff5c0004770 comp rx=0 tx=0).stop 2023-11-08T08:21:18.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.184+0000 7ff5d56e7700 1 -- 172.21.15.40:0/2146087425 shutdown_connections 2023-11-08T08:21:18.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.184+0000 7ff5d56e7700 1 --2- 172.21.15.40:0/2146087425 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff5b0064ee0 0x7ff5b0067390 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:18.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.184+0000 7ff5d56e7700 1 --2- 172.21.15.40:0/2146087425 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5d015c6d0 0x7ff5d00b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:18.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.184+0000 7ff5d56e7700 1 --2- 172.21.15.40:0/2146087425 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5d00cfc40 0x7ff5d00b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:18.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.184+0000 7ff5d56e7700 1 --2- 172.21.15.40:0/2146087425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5d006a300 0x7ff5d0158930 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:18.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.184+0000 7ff5d56e7700 1 -- 172.21.15.40:0/2146087425 >> 172.21.15.40:0/2146087425 conn(0x7ff5d00b6520 msgr2=0x7ff5d015b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:18.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.184+0000 7ff5d56e7700 1 -- 172.21.15.40:0/2146087425 shutdown_connections 2023-11-08T08:21:18.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.184+0000 7ff5d56e7700 1 -- 172.21.15.40:0/2146087425 wait complete. 2023-11-08T08:21:18.215 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:21:18.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.392+0000 7f5afec20700 1 Processor -- start 2023-11-08T08:21:18.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.392+0000 7f5afec20700 1 -- start start 2023-11-08T08:21:18.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.396+0000 7f5afec20700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5af80704b0 0x7f5af8164960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:18.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.396+0000 7f5afec20700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af80ad6c0 con 0x7f5af8070080 2023-11-08T08:21:18.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.396+0000 7f5afec20700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af80adc60 con 0x7f5af8160710 2023-11-08T08:21:18.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.396+0000 7f5afec20700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af806b1c0 con 0x7f5af80704b0 2023-11-08T08:21:18.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.396+0000 7f5afd41d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5af8070080 0x7f5af8164250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33934/0 (socket says 172.21.15.40:33934) 2023-11-08T08:21:18.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.396+0000 7f5afd41d700 1 -- 172.21.15.40:0/672731689 learned_addr learned my addr 172.21.15.40:0/672731689 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:18.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.396+0000 7f5afe41f700 1 --2- 172.21.15.40:0/672731689 >> v2:172.21.15.40:3300/0 conn(0x7f5af80704b0 0x7f5af8164960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:18.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.396+0000 7f5afcc1c700 1 -- 172.21.15.40:0/672731689 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142169742 0 0) 0x7f5af80ad6c0 con 0x7f5af8070080 2023-11-08T08:21:18.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.396+0000 7f5afcc1c700 1 -- 172.21.15.40:0/672731689 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ae40035d0 con 0x7f5af8070080 2023-11-08T08:21:18.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.396+0000 7f5afcc1c700 1 -- 172.21.15.40:0/672731689 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 600149912 0 0) 0x7f5af80adc60 con 0x7f5af8160710 2023-11-08T08:21:18.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.396+0000 7f5afcc1c700 1 -- 172.21.15.40:0/672731689 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af80ad6c0 con 0x7f5af8160710 2023-11-08T08:21:18.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.396+0000 7f5afe41f700 1 -- 172.21.15.40:0/672731689 >> v1:172.21.15.169:6789/0 conn(0x7f5af8160710 legacy=0x7f5af80cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:18.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.396+0000 7f5afe41f700 1 -- 172.21.15.40:0/672731689 >> v1:172.21.15.40:6789/0 conn(0x7f5af8070080 legacy=0x7f5af8164250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:18.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.396+0000 7f5afcc1c700 1 -- 172.21.15.40:0/672731689 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3966067718 0 0) 0x7f5ae40035d0 con 0x7f5af8070080 2023-11-08T08:21:18.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.396+0000 7f5afe41f700 1 -- 172.21.15.40:0/672731689 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af80cca30 con 0x7f5af80704b0 2023-11-08T08:21:18.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.396+0000 7f5afe41f700 1 --2- 172.21.15.40:0/672731689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5af80704b0 0x7f5af8164960 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009d90 tx=0x7f5af000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de04bdc71a7f6665 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:18.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.400+0000 7f5afcc1c700 1 -- 172.21.15.40:0/672731689 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af000e070 con 0x7f5af80704b0 2023-11-08T08:21:18.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.400+0000 7f5afcc1c700 1 -- 172.21.15.40:0/672731689 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af0003a40 con 0x7f5af80704b0 2023-11-08T08:21:18.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.400+0000 7f5afcc1c700 1 -- 172.21.15.40:0/672731689 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8001670 con 0x7f5af80704b0 2023-11-08T08:21:18.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.400+0000 7f5afec20700 1 -- 172.21.15.40:0/672731689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5af80704b0 msgr2=0x7f5af8164960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:18.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.400+0000 7f5afec20700 1 --2- 172.21.15.40:0/672731689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5af80704b0 0x7f5af8164960 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009d90 tx=0x7f5af000b330 comp rx=0 tx=0).stop 2023-11-08T08:21:18.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.400+0000 7f5afec20700 1 -- 172.21.15.40:0/672731689 shutdown_connections 2023-11-08T08:21:18.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.400+0000 7f5afec20700 1 --2- 172.21.15.40:0/672731689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5af80704b0 0x7f5af8164960 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:18.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.400+0000 7f5afec20700 1 -- 172.21.15.40:0/672731689 >> 172.21.15.40:0/672731689 conn(0x7f5af80b0f80 msgr2=0x7f5af80ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:18.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.400+0000 7f5afec20700 1 -- 172.21.15.40:0/672731689 shutdown_connections 2023-11-08T08:21:18.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.400+0000 7f5afec20700 1 -- 172.21.15.40:0/672731689 wait complete. 2023-11-08T08:21:18.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.404+0000 7f5afec20700 1 Processor -- start 2023-11-08T08:21:18.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.404+0000 7f5afec20700 1 -- start start 2023-11-08T08:21:18.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.404+0000 7f5afec20700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5af8070080 0x7f5af80b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:18.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.404+0000 7f5afec20700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af80704b0 0x7f5af815ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:18.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.404+0000 7f5afec20700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5af8160710 0x7f5af80c0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:18.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.404+0000 7f5afec20700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af815f430 con 0x7f5af8160710 2023-11-08T08:21:18.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.404+0000 7f5afec20700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af815f710 con 0x7f5af80704b0 2023-11-08T08:21:18.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.404+0000 7f5afec20700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af815f9f0 con 0x7f5af8070080 2023-11-08T08:21:18.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.408+0000 7f5afe41f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5af8160710 0x7f5af80c0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:18.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.408+0000 7f5afe41f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5af8160710 0x7f5af80c0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51396/0 (socket says 172.21.15.40:51396) 2023-11-08T08:21:18.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.408+0000 7f5afe41f700 1 -- 172.21.15.40:0/2756454309 learned_addr learned my addr 172.21.15.40:0/2756454309 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:18.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.408+0000 7f5afdc1e700 1 --2- 172.21.15.40:0/2756454309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5af8070080 0x7f5af80b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:18.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.408+0000 7f5afd41d700 1 --2- 172.21.15.40:0/2756454309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af80704b0 0x7f5af815ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:18.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.408+0000 7f5afe41f700 1 -- 172.21.15.40:0/2756454309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5af8070080 msgr2=0x7f5af80b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:18.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.408+0000 7f5afe41f700 1 --2- 172.21.15.40:0/2756454309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5af8070080 0x7f5af80b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:18.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.408+0000 7f5afe41f700 1 -- 172.21.15.40:0/2756454309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af80704b0 msgr2=0x7f5af815ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:18.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.408+0000 7f5afe41f700 1 --2- 172.21.15.40:0/2756454309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af80704b0 0x7f5af815ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:18.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.408+0000 7f5afe41f700 1 -- 172.21.15.40:0/2756454309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af0009a70 con 0x7f5af8160710 2023-11-08T08:21:18.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.408+0000 7f5afd41d700 1 --2- 172.21.15.40:0/2756454309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af80704b0 0x7f5af815ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:18.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.408+0000 7f5afe41f700 1 --2- 172.21.15.40:0/2756454309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5af8160710 0x7f5af80c0ac0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5af806bc30 tx=0x7f5af0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:18.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.408+0000 7f5aee7fc700 1 -- 172.21.15.40:0/2756454309 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af00044c0 con 0x7f5af8160710 2023-11-08T08:21:18.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.408+0000 7f5aee7fc700 1 -- 172.21.15.40:0/2756454309 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af0004640 con 0x7f5af8160710 2023-11-08T08:21:18.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.408+0000 7f5afec20700 1 -- 172.21.15.40:0/2756454309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af806b6b0 con 0x7f5af8160710 2023-11-08T08:21:18.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.408+0000 7f5aee7fc700 1 -- 172.21.15.40:0/2756454309 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af001d470 con 0x7f5af8160710 2023-11-08T08:21:18.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.408+0000 7f5afec20700 1 -- 172.21.15.40:0/2756454309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af806b3d0 con 0x7f5af8160710 2023-11-08T08:21:18.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.408+0000 7f5afec20700 1 -- 172.21.15.40:0/2756454309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af80ad8f0 con 0x7f5af8160710 2023-11-08T08:21:18.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.412+0000 7f5aee7fc700 1 -- 172.21.15.40:0/2756454309 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5af0018070 con 0x7f5af8160710 2023-11-08T08:21:18.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.412+0000 7f5aee7fc700 1 --2- 172.21.15.40:0/2756454309 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5adc064ee0 0x7f5adc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:18.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.412+0000 7f5aee7fc700 1 -- 172.21.15.40:0/2756454309 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5af00833f0 con 0x7f5af8160710 2023-11-08T08:21:18.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.412+0000 7f5afdc1e700 1 --2- 172.21.15.40:0/2756454309 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5adc064ee0 0x7f5adc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:18.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.416+0000 7f5afdc1e700 1 --2- 172.21.15.40:0/2756454309 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5adc064ee0 0x7f5adc067390 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5af40011e0 tx=0x7f5af4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:18.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.420+0000 7f5aee7fc700 1 -- 172.21.15.40:0/2756454309 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5af0051360 con 0x7f5af8160710 2023-11-08T08:21:18.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.596+0000 7f5afec20700 1 -- 172.21.15.40:0/2756454309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5af8073cb0 con 0x7f5af8160710 2023-11-08T08:21:18.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.600+0000 7f5aee7fc700 1 -- 172.21.15.40:0/2756454309 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+18097 (secure 0 0 0) 0x7f5af0054980 con 0x7f5af8160710 2023-11-08T08:21:18.605 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:18.605 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":56,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:21:17.729220+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:21:15.464908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-08T08:21:16.755373+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:21:18.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.600+0000 7f5afec20700 1 -- 172.21.15.40:0/2756454309 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5adc064ee0 msgr2=0x7f5adc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:18.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.600+0000 7f5afec20700 1 --2- 172.21.15.40:0/2756454309 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5adc064ee0 0x7f5adc067390 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5af40011e0 tx=0x7f5af4009040 comp rx=0 tx=0).stop 2023-11-08T08:21:18.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.600+0000 7f5afec20700 1 -- 172.21.15.40:0/2756454309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5af8160710 msgr2=0x7f5af80c0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:18.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.600+0000 7f5afec20700 1 --2- 172.21.15.40:0/2756454309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5af8160710 0x7f5af80c0ac0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5af806bc30 tx=0x7f5af0004af0 comp rx=0 tx=0).stop 2023-11-08T08:21:18.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.600+0000 7f5afec20700 1 -- 172.21.15.40:0/2756454309 shutdown_connections 2023-11-08T08:21:18.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.600+0000 7f5afec20700 1 --2- 172.21.15.40:0/2756454309 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5adc064ee0 0x7f5adc067390 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:18.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.600+0000 7f5afec20700 1 --2- 172.21.15.40:0/2756454309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5af8160710 0x7f5af80c0ac0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:18.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.600+0000 7f5afec20700 1 --2- 172.21.15.40:0/2756454309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af80704b0 0x7f5af815ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:18.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.600+0000 7f5afec20700 1 --2- 172.21.15.40:0/2756454309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5af8070080 0x7f5af80b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:18.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.600+0000 7f5afec20700 1 -- 172.21.15.40:0/2756454309 >> 172.21.15.40:0/2756454309 conn(0x7f5af80b0f80 msgr2=0x7f5af80ae9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:18.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.604+0000 7f5afec20700 1 -- 172.21.15.40:0/2756454309 shutdown_connections 2023-11-08T08:21:18.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.604+0000 7f5afec20700 1 -- 172.21.15.40:0/2756454309 wait complete. 2023-11-08T08:21:18.632 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T08:21:18.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.808+0000 7f3d7359e700 1 Processor -- start 2023-11-08T08:21:18.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.808+0000 7f3d7359e700 1 -- start start 2023-11-08T08:21:18.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.808+0000 7f3d7359e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3d7415cbd0 0x7f3d74069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:18.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.808+0000 7f3d7359e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3d740ce660 0x7f3d740cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:18.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d7359e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d740cd0b0 con 0x7f3d741586b0 2023-11-08T08:21:18.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d7359e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d740c4d70 con 0x7f3d740ce660 2023-11-08T08:21:18.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d7359e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d74158a80 con 0x7f3d7415cbd0 2023-11-08T08:21:18.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d7259c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3d741586b0 0x7f3d7406c6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33948/0 (socket says 172.21.15.40:33948) 2023-11-08T08:21:18.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d7259c700 1 -- 172.21.15.40:0/525439427 learned_addr learned my addr 172.21.15.40:0/525439427 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:18.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d71d9b700 1 --2- 172.21.15.40:0/525439427 >> v2:172.21.15.169:3300/0 conn(0x7f3d7415cbd0 0x7f3d74069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:18.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d72d9d700 1 --2- 172.21.15.40:0/525439427 >> v2:172.21.15.40:3300/0 conn(0x7f3d740ce660 0x7f3d740cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:18.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d7159a700 1 -- 172.21.15.40:0/525439427 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223330922 0 0) 0x7f3d740cd0b0 con 0x7f3d741586b0 2023-11-08T08:21:18.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d7159a700 1 -- 172.21.15.40:0/525439427 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d5c0035d0 con 0x7f3d741586b0 2023-11-08T08:21:18.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d72d9d700 1 -- 172.21.15.40:0/525439427 >> v2:172.21.15.169:3300/0 conn(0x7f3d7415cbd0 msgr2=0x7f3d74069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:18.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d72d9d700 1 --2- 172.21.15.40:0/525439427 >> v2:172.21.15.169:3300/0 conn(0x7f3d7415cbd0 0x7f3d74069bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:18.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d72d9d700 1 -- 172.21.15.40:0/525439427 >> v1:172.21.15.40:6789/0 conn(0x7f3d741586b0 legacy=0x7f3d7406c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:18.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d72d9d700 1 -- 172.21.15.40:0/525439427 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d740c4820 con 0x7f3d740ce660 2023-11-08T08:21:18.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d71d9b700 1 --2- 172.21.15.40:0/525439427 >> v2:172.21.15.169:3300/0 conn(0x7f3d7415cbd0 0x7f3d74069bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:21:18.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d72d9d700 1 --2- 172.21.15.40:0/525439427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d740ce660 0x7f3d740cd540 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3d74071e30 tx=0x7f3d6c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9db84400e23c655 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:18.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d7159a700 1 -- 172.21.15.40:0/525439427 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c00f070 con 0x7f3d740ce660 2023-11-08T08:21:18.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d7159a700 1 -- 172.21.15.40:0/525439427 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6c00a2e0 con 0x7f3d740ce660 2023-11-08T08:21:18.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d7159a700 1 -- 172.21.15.40:0/525439427 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c00cb50 con 0x7f3d740ce660 2023-11-08T08:21:18.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d7359e700 1 -- 172.21.15.40:0/525439427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d740ce660 msgr2=0x7f3d740cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:18.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d7359e700 1 --2- 172.21.15.40:0/525439427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d740ce660 0x7f3d740cd540 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3d74071e30 tx=0x7f3d6c00c330 comp rx=0 tx=0).stop 2023-11-08T08:21:18.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d7359e700 1 -- 172.21.15.40:0/525439427 shutdown_connections 2023-11-08T08:21:18.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d7359e700 1 --2- 172.21.15.40:0/525439427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d740ce660 0x7f3d740cd540 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:18.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d7359e700 1 --2- 172.21.15.40:0/525439427 >> v2:172.21.15.169:3300/0 conn(0x7f3d7415cbd0 0x7f3d74069bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:18.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d7359e700 1 -- 172.21.15.40:0/525439427 >> 172.21.15.40:0/525439427 conn(0x7f3d740b1790 msgr2=0x7f3d740ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:18.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d7359e700 1 -- 172.21.15.40:0/525439427 shutdown_connections 2023-11-08T08:21:18.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.812+0000 7f3d7359e700 1 -- 172.21.15.40:0/525439427 wait complete. 2023-11-08T08:21:18.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.816+0000 7f3d7359e700 1 Processor -- start 2023-11-08T08:21:18.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.816+0000 7f3d7359e700 1 -- start start 2023-11-08T08:21:18.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.816+0000 7f3d7359e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d740ce660 0x7f3d741459d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:18.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.816+0000 7f3d7359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d741586b0 0x7f3d74158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:18.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.816+0000 7f3d7359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d7415cbd0 0x7f3d740ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:18.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.816+0000 7f3d7359e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d74159340 con 0x7f3d7415cbd0 2023-11-08T08:21:18.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.816+0000 7f3d7359e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d740b4b30 con 0x7f3d740ce660 2023-11-08T08:21:18.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.816+0000 7f3d7359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d74146320 con 0x7f3d741586b0 2023-11-08T08:21:18.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.816+0000 7f3d72d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d7415cbd0 0x7f3d740ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:18.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.816+0000 7f3d72d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d7415cbd0 0x7f3d740ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51414/0 (socket says 172.21.15.40:51414) 2023-11-08T08:21:18.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.816+0000 7f3d72d9d700 1 -- 172.21.15.40:0/333542916 learned_addr learned my addr 172.21.15.40:0/333542916 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:18.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.816+0000 7f3d7259c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d740ce660 0x7f3d741459d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:18.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.816+0000 7f3d71d9b700 1 --2- 172.21.15.40:0/333542916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d741586b0 0x7f3d74158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:18.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.816+0000 7f3d72d9d700 1 -- 172.21.15.40:0/333542916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d741586b0 msgr2=0x7f3d74158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:18.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.816+0000 7f3d72d9d700 1 --2- 172.21.15.40:0/333542916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d741586b0 0x7f3d74158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:18.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.816+0000 7f3d72d9d700 1 -- 172.21.15.40:0/333542916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d740ce660 msgr2=0x7f3d741459d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:18.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.816+0000 7f3d72d9d700 1 --2- 172.21.15.40:0/333542916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d740ce660 0x7f3d741459d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:18.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.816+0000 7f3d72d9d700 1 -- 172.21.15.40:0/333542916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d680094b0 con 0x7f3d7415cbd0 2023-11-08T08:21:18.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.816+0000 7f3d72d9d700 1 --2- 172.21.15.40:0/333542916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d7415cbd0 0x7f3d740ca3b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c00a470 tx=0x7f3d6c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:18.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.820+0000 7f3d62ffd700 1 -- 172.21.15.40:0/333542916 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c004450 con 0x7f3d7415cbd0 2023-11-08T08:21:18.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.820+0000 7f3d62ffd700 1 -- 172.21.15.40:0/333542916 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6c0045b0 con 0x7f3d7415cbd0 2023-11-08T08:21:18.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.820+0000 7f3d7359e700 1 -- 172.21.15.40:0/333542916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6c00ab80 con 0x7f3d7415cbd0 2023-11-08T08:21:18.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.820+0000 7f3d62ffd700 1 -- 172.21.15.40:0/333542916 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c004960 con 0x7f3d7415cbd0 2023-11-08T08:21:18.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.820+0000 7f3d7359e700 1 -- 172.21.15.40:0/333542916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d740c4a90 con 0x7f3d7415cbd0 2023-11-08T08:21:18.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.820+0000 7f3d7359e700 1 -- 172.21.15.40:0/333542916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d740ccd80 con 0x7f3d7415cbd0 2023-11-08T08:21:18.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.820+0000 7f3d62ffd700 1 -- 172.21.15.40:0/333542916 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f3d6c022030 con 0x7f3d7415cbd0 2023-11-08T08:21:18.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.824+0000 7f3d62ffd700 1 --2- 172.21.15.40:0/333542916 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3d54064ee0 0x7f3d54067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:18.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.824+0000 7f3d62ffd700 1 -- 172.21.15.40:0/333542916 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3d6c083590 con 0x7f3d7415cbd0 2023-11-08T08:21:18.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.824+0000 7f3d7259c700 1 --2- 172.21.15.40:0/333542916 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3d54064ee0 0x7f3d54067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:18.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.828+0000 7f3d7259c700 1 --2- 172.21.15.40:0/333542916 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3d54064ee0 0x7f3d54067390 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3d64007fd0 tx=0x7f3d64007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:18.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:18.832+0000 7f3d62ffd700 1 -- 172.21.15.40:0/333542916 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f3d6c051500 con 0x7f3d7415cbd0 2023-11-08T08:21:19.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.040+0000 7f3d7359e700 1 -- 172.21.15.40:0/333542916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3d7415d040 con 0x7f3d7415cbd0 2023-11-08T08:21:19.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.752+0000 7f3d62ffd700 1 -- 172.21.15.40:0/333542916 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3d6c054b20 con 0x7f3d7415cbd0 2023-11-08T08:21:19.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.756+0000 7f3d7359e700 1 -- 172.21.15.40:0/333542916 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3d54064ee0 msgr2=0x7f3d54067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.756+0000 7f3d7359e700 1 --2- 172.21.15.40:0/333542916 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3d54064ee0 0x7f3d54067390 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3d64007fd0 tx=0x7f3d64007de0 comp rx=0 tx=0).stop 2023-11-08T08:21:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.756+0000 7f3d7359e700 1 -- 172.21.15.40:0/333542916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d7415cbd0 msgr2=0x7f3d740ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.756+0000 7f3d7359e700 1 --2- 172.21.15.40:0/333542916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d7415cbd0 0x7f3d740ca3b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c00a470 tx=0x7f3d6c004af0 comp rx=0 tx=0).stop 2023-11-08T08:21:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.756+0000 7f3d7359e700 1 -- 172.21.15.40:0/333542916 shutdown_connections 2023-11-08T08:21:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.756+0000 7f3d7359e700 1 --2- 172.21.15.40:0/333542916 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3d54064ee0 0x7f3d54067390 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:19.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.756+0000 7f3d7359e700 1 --2- 172.21.15.40:0/333542916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d7415cbd0 0x7f3d740ca3b0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:19.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.756+0000 7f3d7359e700 1 --2- 172.21.15.40:0/333542916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d741586b0 0x7f3d74158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:19.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.756+0000 7f3d7359e700 1 --2- 172.21.15.40:0/333542916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d740ce660 0x7f3d741459d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:19.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.756+0000 7f3d7359e700 1 -- 172.21.15.40:0/333542916 >> 172.21.15.40:0/333542916 conn(0x7f3d740b1790 msgr2=0x7f3d7414fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:19.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.756+0000 7f3d7359e700 1 -- 172.21.15.40:0/333542916 shutdown_connections 2023-11-08T08:21:19.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.756+0000 7f3d7359e700 1 -- 172.21.15.40:0/333542916 wait complete. 2023-11-08T08:21:19.788 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-08T08:21:19.788 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T08:21:19.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.972+0000 7f5ad3377700 1 Processor -- start 2023-11-08T08:21:19.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.972+0000 7f5ad3377700 1 -- start start 2023-11-08T08:21:19.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.976+0000 7f5ad3377700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5acc070620 0x7f5acc0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:19.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.976+0000 7f5ad3377700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5acc15c520 con 0x7f5acc15c150 2023-11-08T08:21:19.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.976+0000 7f5ad3377700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5acc06cc50 con 0x7f5acc069bf0 2023-11-08T08:21:19.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.976+0000 7f5ad3377700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5acc0b4240 con 0x7f5acc070620 2023-11-08T08:21:19.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.976+0000 7f5ad2375700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5acc15c150 0x7f5acc06ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33960/0 (socket says 172.21.15.40:33960) 2023-11-08T08:21:19.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.976+0000 7f5ad2375700 1 -- 172.21.15.40:0/945051770 learned_addr learned my addr 172.21.15.40:0/945051770 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:19.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.976+0000 7f5ad1b74700 1 --2- 172.21.15.40:0/945051770 >> v2:172.21.15.40:3300/0 conn(0x7f5acc070620 0x7f5acc0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:19.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.976+0000 7f5ad1373700 1 -- 172.21.15.40:0/945051770 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512437313 0 0) 0x7f5acc15c520 con 0x7f5acc15c150 2023-11-08T08:21:19.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.976+0000 7f5ad1373700 1 -- 172.21.15.40:0/945051770 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ab80035d0 con 0x7f5acc15c150 2023-11-08T08:21:19.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.976+0000 7f5ad1b74700 1 -- 172.21.15.40:0/945051770 >> v1:172.21.15.169:6789/0 conn(0x7f5acc069bf0 legacy=0x7f5acc069ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:19.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.976+0000 7f5ad1b74700 1 -- 172.21.15.40:0/945051770 >> v1:172.21.15.40:6789/0 conn(0x7f5acc15c150 legacy=0x7f5acc06ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:19.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.976+0000 7f5ad1b74700 1 -- 172.21.15.40:0/945051770 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5acc0d2450 con 0x7f5acc070620 2023-11-08T08:21:19.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.976+0000 7f5ad1b74700 1 --2- 172.21.15.40:0/945051770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5acc070620 0x7f5acc0b3d00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009840 tx=0x7f5abc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e90362ea17fe3c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:19.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.980+0000 7f5ad1373700 1 -- 172.21.15.40:0/945051770 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5abc005e50 con 0x7f5acc070620 2023-11-08T08:21:19.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.980+0000 7f5ad1373700 1 -- 172.21.15.40:0/945051770 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5abc006030 con 0x7f5acc070620 2023-11-08T08:21:19.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.980+0000 7f5ad1373700 1 -- 172.21.15.40:0/945051770 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5abc0063c0 con 0x7f5acc070620 2023-11-08T08:21:19.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.980+0000 7f5ad3377700 1 -- 172.21.15.40:0/945051770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5acc070620 msgr2=0x7f5acc0b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:19.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.980+0000 7f5ad3377700 1 --2- 172.21.15.40:0/945051770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5acc070620 0x7f5acc0b3d00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009840 tx=0x7f5abc009b50 comp rx=0 tx=0).stop 2023-11-08T08:21:19.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.980+0000 7f5ad3377700 1 -- 172.21.15.40:0/945051770 shutdown_connections 2023-11-08T08:21:19.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.980+0000 7f5ad3377700 1 --2- 172.21.15.40:0/945051770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5acc070620 0x7f5acc0b3d00 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:19.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.980+0000 7f5ad3377700 1 -- 172.21.15.40:0/945051770 >> 172.21.15.40:0/945051770 conn(0x7f5acc0d3680 msgr2=0x7f5acc0cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:19.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.980+0000 7f5ad3377700 1 -- 172.21.15.40:0/945051770 shutdown_connections 2023-11-08T08:21:19.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.980+0000 7f5ad3377700 1 -- 172.21.15.40:0/945051770 wait complete. 2023-11-08T08:21:19.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.984+0000 7f5ad3377700 1 Processor -- start 2023-11-08T08:21:19.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.984+0000 7f5ad3377700 1 -- start start 2023-11-08T08:21:19.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.984+0000 7f5ad3377700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5acc069bf0 0x7f5acc0cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:19.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.984+0000 7f5ad3377700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5acc070620 0x7f5acc14f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:19.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.984+0000 7f5ad3377700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5acc15c150 0x7f5acc14f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:19.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.984+0000 7f5ad3377700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc0cf310 con 0x7f5acc15c150 2023-11-08T08:21:19.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.984+0000 7f5ad3377700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc0b69a0 con 0x7f5acc069bf0 2023-11-08T08:21:19.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.984+0000 7f5ad3377700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc0b6f60 con 0x7f5acc070620 2023-11-08T08:21:19.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.984+0000 7f5ad2b76700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5acc15c150 0x7f5acc14f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:19.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.984+0000 7f5ad2b76700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5acc15c150 0x7f5acc14f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51424/0 (socket says 172.21.15.40:51424) 2023-11-08T08:21:19.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.984+0000 7f5ad2b76700 1 -- 172.21.15.40:0/1825743132 learned_addr learned my addr 172.21.15.40:0/1825743132 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:19.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.984+0000 7f5ad1b74700 1 --2- 172.21.15.40:0/1825743132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5acc070620 0x7f5acc14f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:19.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.984+0000 7f5ad2375700 1 --2- 172.21.15.40:0/1825743132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5acc069bf0 0x7f5acc0cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:19.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.984+0000 7f5ad2b76700 1 -- 172.21.15.40:0/1825743132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5acc070620 msgr2=0x7f5acc14f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:19.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.984+0000 7f5ad2b76700 1 --2- 172.21.15.40:0/1825743132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5acc070620 0x7f5acc14f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:19.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.984+0000 7f5ad2b76700 1 -- 172.21.15.40:0/1825743132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5acc069bf0 msgr2=0x7f5acc0cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:19.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.984+0000 7f5ad2b76700 1 --2- 172.21.15.40:0/1825743132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5acc069bf0 0x7f5acc0cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:19.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.984+0000 7f5ad2b76700 1 -- 172.21.15.40:0/1825743132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc009450 con 0x7f5acc15c150 2023-11-08T08:21:19.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.984+0000 7f5ad2b76700 1 --2- 172.21.15.40:0/1825743132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5acc15c150 0x7f5acc14f690 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400cc80 tx=0x7f5ac400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:19.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.988+0000 7f5ac2ffd700 1 -- 172.21.15.40:0/1825743132 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4003c20 con 0x7f5acc15c150 2023-11-08T08:21:19.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.988+0000 7f5ac2ffd700 1 -- 172.21.15.40:0/1825743132 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac4003d80 con 0x7f5acc15c150 2023-11-08T08:21:19.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.988+0000 7f5ac2ffd700 1 -- 172.21.15.40:0/1825743132 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac400aaa0 con 0x7f5acc15c150 2023-11-08T08:21:19.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.988+0000 7f5ad3377700 1 -- 172.21.15.40:0/1825743132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5acc0d05e0 con 0x7f5acc15c150 2023-11-08T08:21:19.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.988+0000 7f5ad3377700 1 -- 172.21.15.40:0/1825743132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5acc0d2730 con 0x7f5acc15c150 2023-11-08T08:21:19.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.988+0000 7f5ad3377700 1 -- 172.21.15.40:0/1825743132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5acc0d2140 con 0x7f5acc15c150 2023-11-08T08:21:19.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.988+0000 7f5ac2ffd700 1 -- 172.21.15.40:0/1825743132 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5ac4010050 con 0x7f5acc15c150 2023-11-08T08:21:19.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.992+0000 7f5ac2ffd700 1 --2- 172.21.15.40:0/1825743132 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5ab0064ee0 0x7f5ab0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:19.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.992+0000 7f5ac2ffd700 1 -- 172.21.15.40:0/1825743132 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5ac4081e30 con 0x7f5acc15c150 2023-11-08T08:21:19.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.992+0000 7f5ad2375700 1 --2- 172.21.15.40:0/1825743132 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5ab0064ee0 0x7f5ab0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:20.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.992+0000 7f5ad2375700 1 --2- 172.21.15.40:0/1825743132 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5ab0064ee0 0x7f5ab0067390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8007fd0 tx=0x7f5ac8007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:20.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:19.996+0000 7f5ac2ffd700 1 -- 172.21.15.40:0/1825743132 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5ac404fe90 con 0x7f5acc15c150 2023-11-08T08:21:20.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.204+0000 7f5ad3377700 1 -- 172.21.15.40:0/1825743132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5acc073d20 con 0x7f5acc15c150 2023-11-08T08:21:20.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.764+0000 7f5ac2ffd700 1 -- 172.21.15.40:0/1825743132 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5ac40534b0 con 0x7f5acc15c150 2023-11-08T08:21:20.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.768+0000 7f5ad3377700 1 -- 172.21.15.40:0/1825743132 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5ab0064ee0 msgr2=0x7f5ab0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:20.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.768+0000 7f5ad3377700 1 --2- 172.21.15.40:0/1825743132 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5ab0064ee0 0x7f5ab0067390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8007fd0 tx=0x7f5ac8007de0 comp rx=0 tx=0).stop 2023-11-08T08:21:20.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.768+0000 7f5ad3377700 1 -- 172.21.15.40:0/1825743132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5acc15c150 msgr2=0x7f5acc14f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:20.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.768+0000 7f5ad3377700 1 --2- 172.21.15.40:0/1825743132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5acc15c150 0x7f5acc14f690 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400cc80 tx=0x7f5ac400cf90 comp rx=0 tx=0).stop 2023-11-08T08:21:20.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.768+0000 7f5ad3377700 1 -- 172.21.15.40:0/1825743132 shutdown_connections 2023-11-08T08:21:20.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.768+0000 7f5ad3377700 1 --2- 172.21.15.40:0/1825743132 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5ab0064ee0 0x7f5ab0067390 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:20.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.768+0000 7f5ad3377700 1 --2- 172.21.15.40:0/1825743132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5acc15c150 0x7f5acc14f690 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:20.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.768+0000 7f5ad3377700 1 --2- 172.21.15.40:0/1825743132 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5acc070620 0x7f5acc14f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:20.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.768+0000 7f5ad3377700 1 --2- 172.21.15.40:0/1825743132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5acc069bf0 0x7f5acc0cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:20.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.768+0000 7f5ad3377700 1 -- 172.21.15.40:0/1825743132 >> 172.21.15.40:0/1825743132 conn(0x7f5acc0d3680 msgr2=0x7f5acc0c7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:20.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.768+0000 7f5ad3377700 1 -- 172.21.15.40:0/1825743132 shutdown_connections 2023-11-08T08:21:20.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.768+0000 7f5ad3377700 1 -- 172.21.15.40:0/1825743132 wait complete. 2023-11-08T08:21:20.799 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T08:21:20.800 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:21:20.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.988+0000 7f8ace62f700 1 Processor -- start 2023-11-08T08:21:20.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.988+0000 7f8ace62f700 1 -- start start 2023-11-08T08:21:21.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.992+0000 7f8ace62f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ac8160710 0x7f8ac80cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:21.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.992+0000 7f8ace62f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8ac8070790 0x7f8ac8164250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:21.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.992+0000 7f8ace62f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac80ad6c0 con 0x7f8ac80cc860 2023-11-08T08:21:21.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.992+0000 7f8ace62f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac806b1c0 con 0x7f8ac8070790 2023-11-08T08:21:21.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.992+0000 7f8ace62f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac80adc60 con 0x7f8ac8160710 2023-11-08T08:21:21.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.992+0000 7f8acde2e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8ac8070790 0x7f8ac8164250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:21.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.992+0000 7f8acce2c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8ac80cc860 0x7f8ac8070080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33964/0 (socket says 172.21.15.40:33964) 2023-11-08T08:21:21.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.992+0000 7f8acce2c700 1 -- 172.21.15.40:0/2610727602 learned_addr learned my addr 172.21.15.40:0/2610727602 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:21.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.992+0000 7f8acd62d700 1 --2- 172.21.15.40:0/2610727602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ac8160710 0x7f8ac80cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:21.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.992+0000 7f8acde2e700 1 -- 172.21.15.40:0/2610727602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ac8160710 msgr2=0x7f8ac80cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:21.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.992+0000 7f8acde2e700 1 --2- 172.21.15.40:0/2610727602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ac8160710 0x7f8ac80cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.992+0000 7f8acde2e700 1 -- 172.21.15.40:0/2610727602 >> v1:172.21.15.40:6789/0 conn(0x7f8ac80cc860 legacy=0x7f8ac8070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:21.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.992+0000 7f8acde2e700 1 -- 172.21.15.40:0/2610727602 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac806b3d0 con 0x7f8ac8070790 2023-11-08T08:21:21.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.992+0000 7f8acde2e700 1 --2- 172.21.15.40:0/2610727602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8070790 0x7f8ac8164250 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009d90 tx=0x7f8ac000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c585cf0093fcee server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:21.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.992+0000 7f8abffff700 1 -- 172.21.15.40:0/2610727602 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac000e070 con 0x7f8ac8070790 2023-11-08T08:21:21.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.992+0000 7f8abffff700 1 -- 172.21.15.40:0/2610727602 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac000bbd0 con 0x7f8ac8070790 2023-11-08T08:21:21.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.992+0000 7f8abffff700 1 -- 172.21.15.40:0/2610727602 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0012430 con 0x7f8ac8070790 2023-11-08T08:21:21.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.996+0000 7f8ace62f700 1 -- 172.21.15.40:0/2610727602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8070790 msgr2=0x7f8ac8164250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:21.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.996+0000 7f8ace62f700 1 --2- 172.21.15.40:0/2610727602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8070790 0x7f8ac8164250 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009d90 tx=0x7f8ac000b330 comp rx=0 tx=0).stop 2023-11-08T08:21:21.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.996+0000 7f8ace62f700 1 -- 172.21.15.40:0/2610727602 shutdown_connections 2023-11-08T08:21:21.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.996+0000 7f8ace62f700 1 --2- 172.21.15.40:0/2610727602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8070790 0x7f8ac8164250 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.996+0000 7f8ace62f700 1 --2- 172.21.15.40:0/2610727602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ac8160710 0x7f8ac80cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.996+0000 7f8ace62f700 1 -- 172.21.15.40:0/2610727602 >> 172.21.15.40:0/2610727602 conn(0x7f8ac80b0f80 msgr2=0x7f8ac80ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:21.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.996+0000 7f8ace62f700 1 -- 172.21.15.40:0/2610727602 shutdown_connections 2023-11-08T08:21:21.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:20.996+0000 7f8ace62f700 1 -- 172.21.15.40:0/2610727602 wait complete. 2023-11-08T08:21:21.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8ace62f700 1 Processor -- start 2023-11-08T08:21:21.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8ace62f700 1 -- start start 2023-11-08T08:21:21.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8ace62f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8070790 0x7f8ac81f9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:21.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8ace62f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ac80cc860 0x7f8ac81f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:21.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8ace62f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ac8160710 0x7f8ac80c1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8ace62f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac81f6c40 con 0x7f8ac8070790 2023-11-08T08:21:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8ace62f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac81fa1b0 con 0x7f8ac80cc860 2023-11-08T08:21:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8ace62f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac81f9910 con 0x7f8ac8160710 2023-11-08T08:21:21.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8acd62d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8070790 0x7f8ac81f9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8acd62d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8070790 0x7f8ac81f9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51450/0 (socket says 172.21.15.40:51450) 2023-11-08T08:21:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8acd62d700 1 -- 172.21.15.40:0/1777759768 learned_addr learned my addr 172.21.15.40:0/1777759768 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:21.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8acce2c700 1 --2- 172.21.15.40:0/1777759768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ac80cc860 0x7f8ac81f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:21.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8acde2e700 1 --2- 172.21.15.40:0/1777759768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ac8160710 0x7f8ac80c1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:21.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8acd62d700 1 -- 172.21.15.40:0/1777759768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ac8160710 msgr2=0x7f8ac80c1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:21.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8acd62d700 1 --2- 172.21.15.40:0/1777759768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ac8160710 0x7f8ac80c1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8acd62d700 1 -- 172.21.15.40:0/1777759768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ac80cc860 msgr2=0x7f8ac81f6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:21.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8acd62d700 1 --2- 172.21.15.40:0/1777759768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ac80cc860 0x7f8ac81f6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8acd62d700 1 -- 172.21.15.40:0/1777759768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac0009a70 con 0x7f8ac8070790 2023-11-08T08:21:21.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8acce2c700 1 --2- 172.21.15.40:0/1777759768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ac80cc860 0x7f8ac81f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:21.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8acd62d700 1 --2- 172.21.15.40:0/1777759768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8070790 0x7f8ac81f9250 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400d800 tx=0x7f8ac400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:21.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.000+0000 7f8abdffb700 1 -- 172.21.15.40:0/1777759768 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac40125f0 con 0x7f8ac8070790 2023-11-08T08:21:21.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.004+0000 7f8abdffb700 1 -- 172.21.15.40:0/1777759768 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac4012750 con 0x7f8ac8070790 2023-11-08T08:21:21.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.004+0000 7f8abdffb700 1 -- 172.21.15.40:0/1777759768 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4012ac0 con 0x7f8ac8070790 2023-11-08T08:21:21.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.004+0000 7f8ace62f700 1 -- 172.21.15.40:0/1777759768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac80ad8f0 con 0x7f8ac8070790 2023-11-08T08:21:21.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.004+0000 7f8ace62f700 1 -- 172.21.15.40:0/1777759768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac806b6b0 con 0x7f8ac8070790 2023-11-08T08:21:21.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.004+0000 7f8ace62f700 1 -- 172.21.15.40:0/1777759768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac8160ae0 con 0x7f8ac8070790 2023-11-08T08:21:21.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.004+0000 7f8abdffb700 1 -- 172.21.15.40:0/1777759768 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f8ac4012c20 con 0x7f8ac8070790 2023-11-08T08:21:21.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.008+0000 7f8abdffb700 1 --2- 172.21.15.40:0/1777759768 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8aac064ee0 0x7f8aac067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:21.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.008+0000 7f8abdffb700 1 -- 172.21.15.40:0/1777759768 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8ac4082600 con 0x7f8ac8070790 2023-11-08T08:21:21.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.008+0000 7f8acce2c700 1 --2- 172.21.15.40:0/1777759768 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8aac064ee0 0x7f8aac067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:21.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.008+0000 7f8acce2c700 1 --2- 172.21.15.40:0/1777759768 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8aac064ee0 0x7f8aac067390 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8007fd0 tx=0x7f8ab8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:21.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.012+0000 7f8abdffb700 1 -- 172.21.15.40:0/1777759768 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f8ac4050570 con 0x7f8ac8070790 2023-11-08T08:21:21.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.224+0000 7f8ace62f700 1 -- 172.21.15.40:0/1777759768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ac8073cb0 con 0x7f8ac8070790 2023-11-08T08:21:21.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.224+0000 7f8abdffb700 1 -- 172.21.15.40:0/1777759768 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f8ac4053b90 con 0x7f8ac8070790 2023-11-08T08:21:21.231 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:21.231 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1549516710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1549516710},{"type":"v1","addr":"172.21.15.169:6837","nonce":1549516710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5728,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/912870125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":912870125},{"type":"v1","addr":"172.21.15.169:6835","nonce":912870125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5772,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1045606427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1045606427},{"type":"v1","addr":"172.21.15.40:6834","nonce":1045606427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:21:17.730002+0000","modified":"2023-11-08T08:21:20.752976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2471311394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2471311394},{"type":"v1","addr":"172.21.15.40:6835","nonce":2471311394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T08:21:21.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.228+0000 7f8ace62f700 1 -- 172.21.15.40:0/1777759768 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8aac064ee0 msgr2=0x7f8aac067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:21.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.228+0000 7f8ace62f700 1 --2- 172.21.15.40:0/1777759768 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8aac064ee0 0x7f8aac067390 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8007fd0 tx=0x7f8ab8009040 comp rx=0 tx=0).stop 2023-11-08T08:21:21.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.228+0000 7f8ace62f700 1 -- 172.21.15.40:0/1777759768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8070790 msgr2=0x7f8ac81f9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:21.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.228+0000 7f8ace62f700 1 --2- 172.21.15.40:0/1777759768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8070790 0x7f8ac81f9250 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400d800 tx=0x7f8ac400dbc0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.228+0000 7f8ace62f700 1 -- 172.21.15.40:0/1777759768 shutdown_connections 2023-11-08T08:21:21.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.228+0000 7f8ace62f700 1 --2- 172.21.15.40:0/1777759768 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8aac064ee0 0x7f8aac067390 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.228+0000 7f8ace62f700 1 --2- 172.21.15.40:0/1777759768 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ac8160710 0x7f8ac80c1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.228+0000 7f8ace62f700 1 --2- 172.21.15.40:0/1777759768 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ac80cc860 0x7f8ac81f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.228+0000 7f8ace62f700 1 --2- 172.21.15.40:0/1777759768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8070790 0x7f8ac81f9250 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.228+0000 7f8ace62f700 1 -- 172.21.15.40:0/1777759768 >> 172.21.15.40:0/1777759768 conn(0x7f8ac80b0f80 msgr2=0x7f8ac80af950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.228+0000 7f8ace62f700 1 -- 172.21.15.40:0/1777759768 shutdown_connections 2023-11-08T08:21:21.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.228+0000 7f8ace62f700 1 -- 172.21.15.40:0/1777759768 wait complete. 2023-11-08T08:21:21.237 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 25 2023-11-08T08:21:21.257 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:21:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.432+0000 7fa7c7ba7700 1 Processor -- start 2023-11-08T08:21:21.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.432+0000 7fa7c7ba7700 1 -- start start 2023-11-08T08:21:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.432+0000 7fa7c7ba7700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa7c01587b0 0x7fa7c006cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c7ba7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa7c006ac00 0x7fa7c006b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c7ba7700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7c006b540 con 0x7fa7c006d010 2023-11-08T08:21:21.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c7ba7700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c00adfe0 con 0x7fa7c006ac00 2023-11-08T08:21:21.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c7ba7700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c0158b80 con 0x7fa7c01587b0 2023-11-08T08:21:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c6144700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa7c006ac00 0x7fa7c006b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:21.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c6144700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa7c006ac00 0x7fa7c006b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51466/0 (socket says 172.21.15.40:51466) 2023-11-08T08:21:21.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c6144700 1 -- 172.21.15.40:0/1801212695 learned_addr learned my addr 172.21.15.40:0/1801212695 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:21.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c5943700 1 --2- 172.21.15.40:0/1801212695 >> v2:172.21.15.169:3300/0 conn(0x7fa7c01587b0 0x7fa7c006cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:21.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c6144700 1 -- 172.21.15.40:0/1801212695 >> v2:172.21.15.169:3300/0 conn(0x7fa7c01587b0 msgr2=0x7fa7c006cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:21.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c6144700 1 --2- 172.21.15.40:0/1801212695 >> v2:172.21.15.169:3300/0 conn(0x7fa7c01587b0 0x7fa7c006cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c6144700 1 -- 172.21.15.40:0/1801212695 >> v1:172.21.15.169:6789/0 conn(0x7fa7c006d010 legacy=0x7fa7c00c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:21.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c6144700 1 -- 172.21.15.40:0/1801212695 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c006d520 con 0x7fa7c006ac00 2023-11-08T08:21:21.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c6144700 1 --2- 172.21.15.40:0/1801212695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c006ac00 0x7fa7c006b000 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00b1290 tx=0x7fa7b800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca170e42b6f98681 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:21.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c4941700 1 -- 172.21.15.40:0/1801212695 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8010070 con 0x7fa7c006ac00 2023-11-08T08:21:21.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c4941700 1 -- 172.21.15.40:0/1801212695 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b800db80 con 0x7fa7c006ac00 2023-11-08T08:21:21.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c4941700 1 -- 172.21.15.40:0/1801212695 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8016400 con 0x7fa7c006ac00 2023-11-08T08:21:21.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c7ba7700 1 -- 172.21.15.40:0/1801212695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c006ac00 msgr2=0x7fa7c006b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c7ba7700 1 --2- 172.21.15.40:0/1801212695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c006ac00 0x7fa7c006b000 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00b1290 tx=0x7fa7b800d430 comp rx=0 tx=0).stop 2023-11-08T08:21:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c7ba7700 1 -- 172.21.15.40:0/1801212695 shutdown_connections 2023-11-08T08:21:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c7ba7700 1 --2- 172.21.15.40:0/1801212695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c006ac00 0x7fa7c006b000 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c7ba7700 1 --2- 172.21.15.40:0/1801212695 >> v2:172.21.15.169:3300/0 conn(0x7fa7c01587b0 0x7fa7c006cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.436+0000 7fa7c7ba7700 1 -- 172.21.15.40:0/1801212695 >> 172.21.15.40:0/1801212695 conn(0x7fa7c00cf350 msgr2=0x7fa7c00b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.440+0000 7fa7c7ba7700 1 -- 172.21.15.40:0/1801212695 shutdown_connections 2023-11-08T08:21:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.440+0000 7fa7c7ba7700 1 -- 172.21.15.40:0/1801212695 wait complete. 2023-11-08T08:21:21.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.440+0000 7fa7c7ba7700 1 Processor -- start 2023-11-08T08:21:21.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.440+0000 7fa7c7ba7700 1 -- start start 2023-11-08T08:21:21.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7c7ba7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7c006ac00 0x7fa7c014fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:21.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7c7ba7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7c006d010 0x7fa7c00ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:21.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7c7ba7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c01587b0 0x7fa7c00ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:21.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7c7ba7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c00ac4c0 con 0x7fa7c01587b0 2023-11-08T08:21:21.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7c7ba7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c00ab860 con 0x7fa7c006d010 2023-11-08T08:21:21.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7c7ba7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c00abb20 con 0x7fa7c006ac00 2023-11-08T08:21:21.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7c6144700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c01587b0 0x7fa7c00ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:21.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7c6144700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c01587b0 0x7fa7c00ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51468/0 (socket says 172.21.15.40:51468) 2023-11-08T08:21:21.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7c6144700 1 -- 172.21.15.40:0/935445107 learned_addr learned my addr 172.21.15.40:0/935445107 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:21.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7c5142700 1 --2- 172.21.15.40:0/935445107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7c006d010 0x7fa7c00ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:21.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7c5943700 1 --2- 172.21.15.40:0/935445107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7c006ac00 0x7fa7c014fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:21.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7c6144700 1 -- 172.21.15.40:0/935445107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7c006ac00 msgr2=0x7fa7c014fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:21.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7c6144700 1 --2- 172.21.15.40:0/935445107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7c006ac00 0x7fa7c014fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7c6144700 1 -- 172.21.15.40:0/935445107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7c006d010 msgr2=0x7fa7c00ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:21.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7c6144700 1 --2- 172.21.15.40:0/935445107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7c006d010 0x7fa7c00ae200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7c6144700 1 -- 172.21.15.40:0/935445107 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b800bd40 con 0x7fa7c01587b0 2023-11-08T08:21:21.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7c6144700 1 --2- 172.21.15.40:0/935445107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c01587b0 0x7fa7c00ae740 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa7c006f500 tx=0x7fa7b8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:21.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7b67fc700 1 -- 172.21.15.40:0/935445107 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8016400 con 0x7fa7c01587b0 2023-11-08T08:21:21.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7b67fc700 1 -- 172.21.15.40:0/935445107 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b80179b0 con 0x7fa7c01587b0 2023-11-08T08:21:21.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7b67fc700 1 -- 172.21.15.40:0/935445107 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8017d60 con 0x7fa7c01587b0 2023-11-08T08:21:21.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7c7ba7700 1 -- 172.21.15.40:0/935445107 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c006dae0 con 0x7fa7c01587b0 2023-11-08T08:21:21.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.444+0000 7fa7c7ba7700 1 -- 172.21.15.40:0/935445107 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c006d800 con 0x7fa7c01587b0 2023-11-08T08:21:21.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.448+0000 7fa7c7ba7700 1 -- 172.21.15.40:0/935445107 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c00c91a0 con 0x7fa7c01587b0 2023-11-08T08:21:21.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.448+0000 7fa7b67fc700 1 -- 172.21.15.40:0/935445107 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa7b8014070 con 0x7fa7c01587b0 2023-11-08T08:21:21.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.452+0000 7fa7b67fc700 1 --2- 172.21.15.40:0/935445107 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa7a4064ee0 0x7fa7a4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:21.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.452+0000 7fa7b67fc700 1 -- 172.21.15.40:0/935445107 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa7b8083920 con 0x7fa7c01587b0 2023-11-08T08:21:21.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.452+0000 7fa7c5943700 1 --2- 172.21.15.40:0/935445107 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa7a4064ee0 0x7fa7a4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:21.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.452+0000 7fa7c5943700 1 --2- 172.21.15.40:0/935445107 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa7a4064ee0 0x7fa7a4067390 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc007a60 tx=0x7fa7bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:21.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.456+0000 7fa7b67fc700 1 -- 172.21.15.40:0/935445107 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa7b8051890 con 0x7fa7c01587b0 2023-11-08T08:21:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.632+0000 7fa7c7ba7700 1 -- 172.21.15.40:0/935445107 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7c00ac600 con 0x7fa7c01587b0 2023-11-08T08:21:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.632+0000 7fa7b67fc700 1 -- 172.21.15.40:0/935445107 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+18084 (secure 0 0 0) 0x7fa7c00ac600 con 0x7fa7c01587b0 2023-11-08T08:21:21.639 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:21.640 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:21:18.736039+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:21:15.464908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-08T08:21:16.755373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:21:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.636+0000 7fa7c7ba7700 1 -- 172.21.15.40:0/935445107 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa7a4064ee0 msgr2=0x7fa7a4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:21.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.636+0000 7fa7c7ba7700 1 --2- 172.21.15.40:0/935445107 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa7a4064ee0 0x7fa7a4067390 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc007a60 tx=0x7fa7bc008040 comp rx=0 tx=0).stop 2023-11-08T08:21:21.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.636+0000 7fa7c7ba7700 1 -- 172.21.15.40:0/935445107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c01587b0 msgr2=0x7fa7c00ae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:21.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.636+0000 7fa7c7ba7700 1 --2- 172.21.15.40:0/935445107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c01587b0 0x7fa7c00ae740 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa7c006f500 tx=0x7fa7b8004af0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.636+0000 7fa7c7ba7700 1 -- 172.21.15.40:0/935445107 shutdown_connections 2023-11-08T08:21:21.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.636+0000 7fa7c7ba7700 1 --2- 172.21.15.40:0/935445107 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa7a4064ee0 0x7fa7a4067390 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.636+0000 7fa7c7ba7700 1 --2- 172.21.15.40:0/935445107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7c01587b0 0x7fa7c00ae740 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.636+0000 7fa7c7ba7700 1 --2- 172.21.15.40:0/935445107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7c006d010 0x7fa7c00ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.636+0000 7fa7c7ba7700 1 --2- 172.21.15.40:0/935445107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7c006ac00 0x7fa7c014fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.636+0000 7fa7c7ba7700 1 -- 172.21.15.40:0/935445107 >> 172.21.15.40:0/935445107 conn(0x7fa7c00cf350 msgr2=0x7fa7c015ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:21.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.636+0000 7fa7c7ba7700 1 -- 172.21.15.40:0/935445107 shutdown_connections 2023-11-08T08:21:21.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.636+0000 7fa7c7ba7700 1 -- 172.21.15.40:0/935445107 wait complete. 2023-11-08T08:21:21.662 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T08:21:21.662 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T08:21:21.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.836+0000 7fe054828700 1 Processor -- start 2023-11-08T08:21:21.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.836+0000 7fe054828700 1 -- start start 2023-11-08T08:21:21.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.840+0000 7fe054828700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe050160710 0x7fe0500cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:21.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.840+0000 7fe054828700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0500ad6c0 con 0x7fe0500cc860 2023-11-08T08:21:21.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.840+0000 7fe054828700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0500adc60 con 0x7fe050070790 2023-11-08T08:21:21.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.840+0000 7fe054828700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe05006b1c0 con 0x7fe050160710 2023-11-08T08:21:21.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.840+0000 7fe04effd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe0500cc860 0x7fe050070080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33974/0 (socket says 172.21.15.40:33974) 2023-11-08T08:21:21.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.840+0000 7fe04effd700 1 -- 172.21.15.40:0/1630494420 learned_addr learned my addr 172.21.15.40:0/1630494420 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:21.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.840+0000 7fe04f7fe700 1 --2- 172.21.15.40:0/1630494420 >> v2:172.21.15.40:3300/0 conn(0x7fe050160710 0x7fe0500cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:21.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.840+0000 7fe04e7fc700 1 -- 172.21.15.40:0/1630494420 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 625339610 0 0) 0x7fe0500ad6c0 con 0x7fe0500cc860 2023-11-08T08:21:21.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.840+0000 7fe04e7fc700 1 -- 172.21.15.40:0/1630494420 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0380035d0 con 0x7fe0500cc860 2023-11-08T08:21:21.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.840+0000 7fe04e7fc700 1 -- 172.21.15.40:0/1630494420 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2697995212 0 0) 0x7fe0500adc60 con 0x7fe050070790 2023-11-08T08:21:21.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.844+0000 7fe04f7fe700 1 -- 172.21.15.40:0/1630494420 >> v1:172.21.15.169:6789/0 conn(0x7fe050070790 legacy=0x7fe050164250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:21.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.844+0000 7fe04f7fe700 1 -- 172.21.15.40:0/1630494420 >> v1:172.21.15.40:6789/0 conn(0x7fe0500cc860 legacy=0x7fe050070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:21.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.844+0000 7fe04f7fe700 1 -- 172.21.15.40:0/1630494420 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05006b3d0 con 0x7fe050160710 2023-11-08T08:21:21.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.844+0000 7fe04f7fe700 1 --2- 172.21.15.40:0/1630494420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe050160710 0x7fe0500cc320 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe040009850 tx=0x7fe040009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a50c3296f3809bc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:21.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.844+0000 7fe04e7fc700 1 -- 172.21.15.40:0/1630494420 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040011830 con 0x7fe050160710 2023-11-08T08:21:21.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.844+0000 7fe04e7fc700 1 -- 172.21.15.40:0/1630494420 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe040011990 con 0x7fe050160710 2023-11-08T08:21:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.844+0000 7fe04e7fc700 1 -- 172.21.15.40:0/1630494420 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040011d40 con 0x7fe050160710 2023-11-08T08:21:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.844+0000 7fe054828700 1 -- 172.21.15.40:0/1630494420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe050160710 msgr2=0x7fe0500cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.844+0000 7fe054828700 1 --2- 172.21.15.40:0/1630494420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe050160710 0x7fe0500cc320 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe040009850 tx=0x7fe040009b60 comp rx=0 tx=0).stop 2023-11-08T08:21:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.844+0000 7fe054828700 1 -- 172.21.15.40:0/1630494420 shutdown_connections 2023-11-08T08:21:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.844+0000 7fe054828700 1 --2- 172.21.15.40:0/1630494420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe050160710 0x7fe0500cc320 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.844+0000 7fe054828700 1 -- 172.21.15.40:0/1630494420 >> 172.21.15.40:0/1630494420 conn(0x7fe0500b0f80 msgr2=0x7fe0500ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.844+0000 7fe054828700 1 -- 172.21.15.40:0/1630494420 shutdown_connections 2023-11-08T08:21:21.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.844+0000 7fe054828700 1 -- 172.21.15.40:0/1630494420 wait complete. 2023-11-08T08:21:21.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.848+0000 7fe054828700 1 Processor -- start 2023-11-08T08:21:21.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.848+0000 7fe054828700 1 -- start start 2023-11-08T08:21:21.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.848+0000 7fe054828700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe050070790 0x7fe05015b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.848+0000 7fe054828700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0500cc860 0x7fe0500b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe054828700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe050160710 0x7fe0500c1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe054828700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0500c2080 con 0x7fe050070790 2023-11-08T08:21:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe054828700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0500c2360 con 0x7fe0500cc860 2023-11-08T08:21:21.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe054828700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0500c2640 con 0x7fe050160710 2023-11-08T08:21:21.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe04f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe050070790 0x7fe05015b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:21.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe04f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe050070790 0x7fe05015b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51478/0 (socket says 172.21.15.40:51478) 2023-11-08T08:21:21.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe04f7fe700 1 -- 172.21.15.40:0/2683592820 learned_addr learned my addr 172.21.15.40:0/2683592820 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:21.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe04f7fe700 1 -- 172.21.15.40:0/2683592820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe050160710 msgr2=0x7fe0500c1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:21.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe04effd700 1 --2- 172.21.15.40:0/2683592820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0500cc860 0x7fe0500b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:21.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe04ffff700 1 --2- 172.21.15.40:0/2683592820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe050160710 0x7fe0500c1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:21.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe04f7fe700 1 --2- 172.21.15.40:0/2683592820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe050160710 0x7fe0500c1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe04f7fe700 1 -- 172.21.15.40:0/2683592820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0500cc860 msgr2=0x7fe0500b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:21.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe04f7fe700 1 --2- 172.21.15.40:0/2683592820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0500cc860 0x7fe0500b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:21.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe04f7fe700 1 -- 172.21.15.40:0/2683592820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0400094b0 con 0x7fe050070790 2023-11-08T08:21:21.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe04effd700 1 --2- 172.21.15.40:0/2683592820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0500cc860 0x7fe0500b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:21:21.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe04ffff700 1 --2- 172.21.15.40:0/2683592820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe050160710 0x7fe0500c1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:21:21.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe04f7fe700 1 --2- 172.21.15.40:0/2683592820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe050070790 0x7fe05015b1e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe0400090b0 tx=0x7fe040004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:21.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe037fff700 1 -- 172.21.15.40:0/2683592820 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040019a00 con 0x7fe050070790 2023-11-08T08:21:21.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe037fff700 1 -- 172.21.15.40:0/2683592820 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04001e070 con 0x7fe050070790 2023-11-08T08:21:21.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe037fff700 1 -- 172.21.15.40:0/2683592820 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040019dc0 con 0x7fe050070790 2023-11-08T08:21:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe054828700 1 -- 172.21.15.40:0/2683592820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0500ad8f0 con 0x7fe050070790 2023-11-08T08:21:21.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.852+0000 7fe054828700 1 -- 172.21.15.40:0/2683592820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05006b6b0 con 0x7fe050070790 2023-11-08T08:21:21.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.856+0000 7fe054828700 1 -- 172.21.15.40:0/2683592820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe050160ae0 con 0x7fe050070790 2023-11-08T08:21:21.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.856+0000 7fe037fff700 1 -- 172.21.15.40:0/2683592820 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fe04000f050 con 0x7fe050070790 2023-11-08T08:21:21.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.856+0000 7fe037fff700 1 --2- 172.21.15.40:0/2683592820 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe03c064f30 0x7fe03c0673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:21.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.856+0000 7fe037fff700 1 -- 172.21.15.40:0/2683592820 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe040082ee0 con 0x7fe050070790 2023-11-08T08:21:21.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.860+0000 7fe04effd700 1 --2- 172.21.15.40:0/2683592820 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe03c064f30 0x7fe03c0673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:21.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.860+0000 7fe04effd700 1 --2- 172.21.15.40:0/2683592820 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe03c064f30 0x7fe03c0673e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe0501f91c0 tx=0x7fe044009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:21.864+0000 7fe037fff700 1 -- 172.21.15.40:0/2683592820 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fe040050fe0 con 0x7fe050070790 2023-11-08T08:21:22.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.044+0000 7fe054828700 1 -- 172.21.15.40:0/2683592820 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe0500c24d0 con 0x7fe03c064f30 2023-11-08T08:21:22.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.064+0000 7fe037fff700 1 -- 172.21.15.40:0/2683592820 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308752 (secure 0 0 0) 0x7fe0500c24d0 con 0x7fe03c064f30 2023-11-08T08:21:22.076 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:22.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.072+0000 7fe054828700 1 -- 172.21.15.40:0/2683592820 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe03c064f30 msgr2=0x7fe03c0673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:22.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.072+0000 7fe054828700 1 --2- 172.21.15.40:0/2683592820 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe03c064f30 0x7fe03c0673e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe0501f91c0 tx=0x7fe044009040 comp rx=0 tx=0).stop 2023-11-08T08:21:22.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.072+0000 7fe054828700 1 -- 172.21.15.40:0/2683592820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe050070790 msgr2=0x7fe05015b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:22.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.072+0000 7fe054828700 1 --2- 172.21.15.40:0/2683592820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe050070790 0x7fe05015b1e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe0400090b0 tx=0x7fe040004770 comp rx=0 tx=0).stop 2023-11-08T08:21:22.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.072+0000 7fe054828700 1 -- 172.21.15.40:0/2683592820 shutdown_connections 2023-11-08T08:21:22.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.072+0000 7fe054828700 1 --2- 172.21.15.40:0/2683592820 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe03c064f30 0x7fe03c0673e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:22.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.072+0000 7fe054828700 1 --2- 172.21.15.40:0/2683592820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe050160710 0x7fe0500c1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:22.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.072+0000 7fe054828700 1 --2- 172.21.15.40:0/2683592820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0500cc860 0x7fe0500b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:22.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.072+0000 7fe054828700 1 --2- 172.21.15.40:0/2683592820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe050070790 0x7fe05015b1e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:22.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.072+0000 7fe054828700 1 -- 172.21.15.40:0/2683592820 >> 172.21.15.40:0/2683592820 conn(0x7fe0500b0f80 msgr2=0x7fe0501635b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:22.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.072+0000 7fe054828700 1 -- 172.21.15.40:0/2683592820 shutdown_connections 2023-11-08T08:21:22.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.072+0000 7fe054828700 1 -- 172.21.15.40:0/2683592820 wait complete. 2023-11-08T08:21:22.083 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-11-08T08:21:22.100 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2023-11-08T08:21:21.932804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":76680,"kb_used_data":6536,"kb_used_omap":0,"kb_used_meta":70080,"kb_avail":754898040,"statfs":{"total":773094113280,"available":773015592960,"internally_reserved":0,"allocated":6692864,"data_stored":3695060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":71761920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.674823"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.754199+0000","last_change":"2023-11-08T08:21:17.774658+0000","last_active":"2023-11-08T08:21:18.754199+0000","last_peered":"2023-11-08T08:21:18.754199+0000","last_clean":"2023-11-08T08:21:18.754199+0000","last_became_active":"2023-11-08T08:21:17.774405+0000","last_became_peered":"2023-11-08T08:21:17.774405+0000","last_unstale":"2023-11-08T08:21:18.754199+0000","last_undegraded":"2023-11-08T08:21:18.754199+0000","last_fullsized":"2023-11-08T08:21:18.754199+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:50:53.616882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809742+0000","last_change":"2023-11-08T08:21:17.778052+0000","last_active":"2023-11-08T08:21:18.809742+0000","last_peered":"2023-11-08T08:21:18.809742+0000","last_clean":"2023-11-08T08:21:18.809742+0000","last_became_active":"2023-11-08T08:21:17.777801+0000","last_became_peered":"2023-11-08T08:21:17.777801+0000","last_unstale":"2023-11-08T08:21:18.809742+0000","last_undegraded":"2023-11-08T08:21:18.809742+0000","last_fullsized":"2023-11-08T08:21:18.809742+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:02:46.560811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813846+0000","last_change":"2023-11-08T08:21:17.775681+0000","last_active":"2023-11-08T08:21:18.813846+0000","last_peered":"2023-11-08T08:21:18.813846+0000","last_clean":"2023-11-08T08:21:18.813846+0000","last_became_active":"2023-11-08T08:21:17.771243+0000","last_became_peered":"2023-11-08T08:21:17.771243+0000","last_unstale":"2023-11-08T08:21:18.813846+0000","last_undegraded":"2023-11-08T08:21:18.813846+0000","last_fullsized":"2023-11-08T08:21:18.813846+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:50:29.721556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812505+0000","last_change":"2023-11-08T08:21:17.771572+0000","last_active":"2023-11-08T08:21:18.812505+0000","last_peered":"2023-11-08T08:21:18.812505+0000","last_clean":"2023-11-08T08:21:18.812505+0000","last_became_active":"2023-11-08T08:21:17.771307+0000","last_became_peered":"2023-11-08T08:21:17.771307+0000","last_unstale":"2023-11-08T08:21:18.812505+0000","last_undegraded":"2023-11-08T08:21:18.812505+0000","last_fullsized":"2023-11-08T08:21:18.812505+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:03:09.691918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753856+0000","last_change":"2023-11-08T08:21:17.772356+0000","last_active":"2023-11-08T08:21:18.753856+0000","last_peered":"2023-11-08T08:21:18.753856+0000","last_clean":"2023-11-08T08:21:18.753856+0000","last_became_active":"2023-11-08T08:21:17.771165+0000","last_became_peered":"2023-11-08T08:21:17.771165+0000","last_unstale":"2023-11-08T08:21:18.753856+0000","last_undegraded":"2023-11-08T08:21:18.753856+0000","last_fullsized":"2023-11-08T08:21:18.753856+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:59:38.074076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.752732+0000","last_change":"2023-11-08T08:21:17.775859+0000","last_active":"2023-11-08T08:21:18.752732+0000","last_peered":"2023-11-08T08:21:18.752732+0000","last_clean":"2023-11-08T08:21:18.752732+0000","last_became_active":"2023-11-08T08:21:17.775611+0000","last_became_peered":"2023-11-08T08:21:17.775611+0000","last_unstale":"2023-11-08T08:21:18.752732+0000","last_undegraded":"2023-11-08T08:21:18.752732+0000","last_fullsized":"2023-11-08T08:21:18.752732+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:53:38.208291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.754987+0000","last_change":"2023-11-08T08:21:17.776518+0000","last_active":"2023-11-08T08:21:18.754987+0000","last_peered":"2023-11-08T08:21:18.754987+0000","last_clean":"2023-11-08T08:21:18.754987+0000","last_became_active":"2023-11-08T08:21:17.776229+0000","last_became_peered":"2023-11-08T08:21:17.776229+0000","last_unstale":"2023-11-08T08:21:18.754987+0000","last_undegraded":"2023-11-08T08:21:18.754987+0000","last_fullsized":"2023-11-08T08:21:18.754987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:19:00.004764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809201+0000","last_change":"2023-11-08T08:21:17.765200+0000","last_active":"2023-11-08T08:21:18.809201+0000","last_peered":"2023-11-08T08:21:18.809201+0000","last_clean":"2023-11-08T08:21:18.809201+0000","last_became_active":"2023-11-08T08:21:17.764941+0000","last_became_peered":"2023-11-08T08:21:17.764941+0000","last_unstale":"2023-11-08T08:21:18.809201+0000","last_undegraded":"2023-11-08T08:21:18.809201+0000","last_fullsized":"2023-11-08T08:21:18.809201+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:39:20.550673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.755114+0000","last_change":"2023-11-08T08:21:17.765151+0000","last_active":"2023-11-08T08:21:18.755114+0000","last_peered":"2023-11-08T08:21:18.755114+0000","last_clean":"2023-11-08T08:21:18.755114+0000","last_became_active":"2023-11-08T08:21:17.762399+0000","last_became_peered":"2023-11-08T08:21:17.762399+0000","last_unstale":"2023-11-08T08:21:18.755114+0000","last_undegraded":"2023-11-08T08:21:18.755114+0000","last_fullsized":"2023-11-08T08:21:18.755114+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:54:22.302215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.808407+0000","last_change":"2023-11-08T08:21:17.773195+0000","last_active":"2023-11-08T08:21:18.808407+0000","last_peered":"2023-11-08T08:21:18.808407+0000","last_clean":"2023-11-08T08:21:18.808407+0000","last_became_active":"2023-11-08T08:21:17.772925+0000","last_became_peered":"2023-11-08T08:21:17.772925+0000","last_unstale":"2023-11-08T08:21:18.808407+0000","last_undegraded":"2023-11-08T08:21:18.808407+0000","last_fullsized":"2023-11-08T08:21:18.808407+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:32:15.406882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.808110+0000","last_change":"2023-11-08T08:21:17.773953+0000","last_active":"2023-11-08T08:21:18.808110+0000","last_peered":"2023-11-08T08:21:18.808110+0000","last_clean":"2023-11-08T08:21:18.808110+0000","last_became_active":"2023-11-08T08:21:17.773694+0000","last_became_peered":"2023-11-08T08:21:17.773694+0000","last_unstale":"2023-11-08T08:21:18.808110+0000","last_undegraded":"2023-11-08T08:21:18.808110+0000","last_fullsized":"2023-11-08T08:21:18.808110+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:27:03.140038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812189+0000","last_change":"2023-11-08T08:21:17.770468+0000","last_active":"2023-11-08T08:21:18.812189+0000","last_peered":"2023-11-08T08:21:18.812189+0000","last_clean":"2023-11-08T08:21:18.812189+0000","last_became_active":"2023-11-08T08:21:17.767802+0000","last_became_peered":"2023-11-08T08:21:17.767802+0000","last_unstale":"2023-11-08T08:21:18.812189+0000","last_undegraded":"2023-11-08T08:21:18.812189+0000","last_fullsized":"2023-11-08T08:21:18.812189+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:16:37.763141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813638+0000","last_change":"2023-11-08T08:21:17.756611+0000","last_active":"2023-11-08T08:21:18.813638+0000","last_peered":"2023-11-08T08:21:18.813638+0000","last_clean":"2023-11-08T08:21:18.813638+0000","last_became_active":"2023-11-08T08:21:17.756239+0000","last_became_peered":"2023-11-08T08:21:17.756239+0000","last_unstale":"2023-11-08T08:21:18.813638+0000","last_undegraded":"2023-11-08T08:21:18.813638+0000","last_fullsized":"2023-11-08T08:21:18.813638+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:14:54.711453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.755105+0000","last_change":"2023-11-08T08:21:17.766210+0000","last_active":"2023-11-08T08:21:18.755105+0000","last_peered":"2023-11-08T08:21:18.755105+0000","last_clean":"2023-11-08T08:21:18.755105+0000","last_became_active":"2023-11-08T08:21:17.765915+0000","last_became_peered":"2023-11-08T08:21:17.765915+0000","last_unstale":"2023-11-08T08:21:18.755105+0000","last_undegraded":"2023-11-08T08:21:18.755105+0000","last_fullsized":"2023-11-08T08:21:18.755105+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:53:39.019269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.755144+0000","last_change":"2023-11-08T08:21:17.773028+0000","last_active":"2023-11-08T08:21:18.755144+0000","last_peered":"2023-11-08T08:21:18.755144+0000","last_clean":"2023-11-08T08:21:18.755144+0000","last_became_active":"2023-11-08T08:21:17.772766+0000","last_became_peered":"2023-11-08T08:21:17.772766+0000","last_unstale":"2023-11-08T08:21:18.755144+0000","last_undegraded":"2023-11-08T08:21:18.755144+0000","last_fullsized":"2023-11-08T08:21:18.755144+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:02:19.326469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.752244+0000","last_change":"2023-11-08T08:21:17.761867+0000","last_active":"2023-11-08T08:21:18.752244+0000","last_peered":"2023-11-08T08:21:18.752244+0000","last_clean":"2023-11-08T08:21:18.752244+0000","last_became_active":"2023-11-08T08:21:17.761389+0000","last_became_peered":"2023-11-08T08:21:17.761389+0000","last_unstale":"2023-11-08T08:21:18.752244+0000","last_undegraded":"2023-11-08T08:21:18.752244+0000","last_fullsized":"2023-11-08T08:21:18.752244+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:02:46.560811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.810297+0000","last_change":"2023-11-08T08:21:17.778277+0000","last_active":"2023-11-08T08:21:18.810297+0000","last_peered":"2023-11-08T08:21:18.810297+0000","last_clean":"2023-11-08T08:21:18.810297+0000","last_became_active":"2023-11-08T08:21:17.777595+0000","last_became_peered":"2023-11-08T08:21:17.777595+0000","last_unstale":"2023-11-08T08:21:18.810297+0000","last_undegraded":"2023-11-08T08:21:18.810297+0000","last_fullsized":"2023-11-08T08:21:18.810297+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:18:18.548903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.814453+0000","last_change":"2023-11-08T08:21:17.776225+0000","last_active":"2023-11-08T08:21:18.814453+0000","last_peered":"2023-11-08T08:21:18.814453+0000","last_clean":"2023-11-08T08:21:18.814453+0000","last_became_active":"2023-11-08T08:21:17.775934+0000","last_became_peered":"2023-11-08T08:21:17.775934+0000","last_unstale":"2023-11-08T08:21:18.814453+0000","last_undegraded":"2023-11-08T08:21:18.814453+0000","last_fullsized":"2023-11-08T08:21:18.814453+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:33:04.463667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753962+0000","last_change":"2023-11-08T08:21:17.774429+0000","last_active":"2023-11-08T08:21:18.753962+0000","last_peered":"2023-11-08T08:21:18.753962+0000","last_clean":"2023-11-08T08:21:18.753962+0000","last_became_active":"2023-11-08T08:21:17.774122+0000","last_became_peered":"2023-11-08T08:21:17.774122+0000","last_unstale":"2023-11-08T08:21:18.753962+0000","last_undegraded":"2023-11-08T08:21:18.753962+0000","last_fullsized":"2023-11-08T08:21:18.753962+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:40:31.145969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809814+0000","last_change":"2023-11-08T08:21:17.783562+0000","last_active":"2023-11-08T08:21:18.809814+0000","last_peered":"2023-11-08T08:21:18.809814+0000","last_clean":"2023-11-08T08:21:18.809814+0000","last_became_active":"2023-11-08T08:21:17.783251+0000","last_became_peered":"2023-11-08T08:21:17.783251+0000","last_unstale":"2023-11-08T08:21:18.809814+0000","last_undegraded":"2023-11-08T08:21:18.809814+0000","last_fullsized":"2023-11-08T08:21:18.809814+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:03:09.691918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813300+0000","last_change":"2023-11-08T08:21:17.784886+0000","last_active":"2023-11-08T08:21:18.813300+0000","last_peered":"2023-11-08T08:21:18.813300+0000","last_clean":"2023-11-08T08:21:18.813300+0000","last_became_active":"2023-11-08T08:21:17.783682+0000","last_became_peered":"2023-11-08T08:21:17.783682+0000","last_unstale":"2023-11-08T08:21:18.813300+0000","last_undegraded":"2023-11-08T08:21:18.813300+0000","last_fullsized":"2023-11-08T08:21:18.813300+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:05:45.264423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809545+0000","last_change":"2023-11-08T08:21:17.776227+0000","last_active":"2023-11-08T08:21:18.809545+0000","last_peered":"2023-11-08T08:21:18.809545+0000","last_clean":"2023-11-08T08:21:18.809545+0000","last_became_active":"2023-11-08T08:21:17.775776+0000","last_became_peered":"2023-11-08T08:21:17.775776+0000","last_unstale":"2023-11-08T08:21:18.809545+0000","last_undegraded":"2023-11-08T08:21:18.809545+0000","last_fullsized":"2023-11-08T08:21:18.809545+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:49:36.507723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812525+0000","last_change":"2023-11-08T08:21:17.768621+0000","last_active":"2023-11-08T08:21:18.812525+0000","last_peered":"2023-11-08T08:21:18.812525+0000","last_clean":"2023-11-08T08:21:18.812525+0000","last_became_active":"2023-11-08T08:21:17.767297+0000","last_became_peered":"2023-11-08T08:21:17.767297+0000","last_unstale":"2023-11-08T08:21:18.812525+0000","last_undegraded":"2023-11-08T08:21:18.812525+0000","last_fullsized":"2023-11-08T08:21:18.812525+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:22:02.813495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.811712+0000","last_change":"2023-11-08T08:21:17.760411+0000","last_active":"2023-11-08T08:21:18.811712+0000","last_peered":"2023-11-08T08:21:18.811712+0000","last_clean":"2023-11-08T08:21:18.811712+0000","last_became_active":"2023-11-08T08:21:17.760120+0000","last_became_peered":"2023-11-08T08:21:17.760120+0000","last_unstale":"2023-11-08T08:21:18.811712+0000","last_undegraded":"2023-11-08T08:21:18.811712+0000","last_fullsized":"2023-11-08T08:21:18.811712+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:32:15.406882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.754202+0000","last_change":"2023-11-08T08:21:17.761461+0000","last_active":"2023-11-08T08:21:18.754202+0000","last_peered":"2023-11-08T08:21:18.754202+0000","last_clean":"2023-11-08T08:21:18.754202+0000","last_became_active":"2023-11-08T08:21:17.761185+0000","last_became_peered":"2023-11-08T08:21:17.761185+0000","last_unstale":"2023-11-08T08:21:18.754202+0000","last_undegraded":"2023-11-08T08:21:18.754202+0000","last_fullsized":"2023-11-08T08:21:18.754202+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:50:54.435794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813044+0000","last_change":"2023-11-08T08:21:17.777439+0000","last_active":"2023-11-08T08:21:18.813044+0000","last_peered":"2023-11-08T08:21:18.813044+0000","last_clean":"2023-11-08T08:21:18.813044+0000","last_became_active":"2023-11-08T08:21:17.777120+0000","last_became_peered":"2023-11-08T08:21:17.777120+0000","last_unstale":"2023-11-08T08:21:18.813044+0000","last_undegraded":"2023-11-08T08:21:18.813044+0000","last_fullsized":"2023-11-08T08:21:18.813044+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:22:02.813495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753064+0000","last_change":"2023-11-08T08:21:17.761393+0000","last_active":"2023-11-08T08:21:18.753064+0000","last_peered":"2023-11-08T08:21:18.753064+0000","last_clean":"2023-11-08T08:21:18.753064+0000","last_became_active":"2023-11-08T08:21:17.761079+0000","last_became_peered":"2023-11-08T08:21:17.761079+0000","last_unstale":"2023-11-08T08:21:18.753064+0000","last_undegraded":"2023-11-08T08:21:18.753064+0000","last_fullsized":"2023-11-08T08:21:18.753064+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:30:19.324907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.754046+0000","last_change":"2023-11-08T08:21:17.772056+0000","last_active":"2023-11-08T08:21:18.754046+0000","last_peered":"2023-11-08T08:21:18.754046+0000","last_clean":"2023-11-08T08:21:18.754046+0000","last_became_active":"2023-11-08T08:21:17.771820+0000","last_became_peered":"2023-11-08T08:21:17.771820+0000","last_unstale":"2023-11-08T08:21:18.754046+0000","last_undegraded":"2023-11-08T08:21:18.754046+0000","last_fullsized":"2023-11-08T08:21:18.754046+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:10:09.581328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809686+0000","last_change":"2023-11-08T08:21:17.785604+0000","last_active":"2023-11-08T08:21:18.809686+0000","last_peered":"2023-11-08T08:21:18.809686+0000","last_clean":"2023-11-08T08:21:18.809686+0000","last_became_active":"2023-11-08T08:21:17.785324+0000","last_became_peered":"2023-11-08T08:21:17.785324+0000","last_unstale":"2023-11-08T08:21:18.809686+0000","last_undegraded":"2023-11-08T08:21:18.809686+0000","last_fullsized":"2023-11-08T08:21:18.809686+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:30:19.324907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813151+0000","last_change":"2023-11-08T08:21:17.771048+0000","last_active":"2023-11-08T08:21:18.813151+0000","last_peered":"2023-11-08T08:21:18.813151+0000","last_clean":"2023-11-08T08:21:18.813151+0000","last_became_active":"2023-11-08T08:21:17.770768+0000","last_became_peered":"2023-11-08T08:21:17.770768+0000","last_unstale":"2023-11-08T08:21:18.813151+0000","last_undegraded":"2023-11-08T08:21:18.813151+0000","last_fullsized":"2023-11-08T08:21:18.813151+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:15:05.006165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753390+0000","last_change":"2023-11-08T08:21:17.762157+0000","last_active":"2023-11-08T08:21:18.753390+0000","last_peered":"2023-11-08T08:21:18.753390+0000","last_clean":"2023-11-08T08:21:18.753390+0000","last_became_active":"2023-11-08T08:21:17.761876+0000","last_became_peered":"2023-11-08T08:21:17.761876+0000","last_unstale":"2023-11-08T08:21:18.753390+0000","last_undegraded":"2023-11-08T08:21:18.753390+0000","last_fullsized":"2023-11-08T08:21:18.753390+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:27:40.888088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.752241+0000","last_change":"2023-11-08T08:21:17.774106+0000","last_active":"2023-11-08T08:21:18.752241+0000","last_peered":"2023-11-08T08:21:18.752241+0000","last_clean":"2023-11-08T08:21:18.752241+0000","last_became_active":"2023-11-08T08:21:17.773626+0000","last_became_peered":"2023-11-08T08:21:17.773626+0000","last_unstale":"2023-11-08T08:21:18.752241+0000","last_undegraded":"2023-11-08T08:21:18.752241+0000","last_fullsized":"2023-11-08T08:21:18.752241+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:40:39.325330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753445+0000","last_change":"2023-11-08T08:21:17.766739+0000","last_active":"2023-11-08T08:21:18.753445+0000","last_peered":"2023-11-08T08:21:18.753445+0000","last_clean":"2023-11-08T08:21:18.753445+0000","last_became_active":"2023-11-08T08:21:17.762207+0000","last_became_peered":"2023-11-08T08:21:17.762207+0000","last_unstale":"2023-11-08T08:21:18.753445+0000","last_undegraded":"2023-11-08T08:21:18.753445+0000","last_fullsized":"2023-11-08T08:21:18.753445+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:54:22.302215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753511+0000","last_change":"2023-11-08T08:21:17.284548+0000","last_active":"2023-11-08T08:21:18.753511+0000","last_peered":"2023-11-08T08:21:18.753511+0000","last_clean":"2023-11-08T08:21:18.753511+0000","last_became_active":"2023-11-08T08:21:17.284197+0000","last_became_peered":"2023-11-08T08:21:17.284197+0000","last_unstale":"2023-11-08T08:21:18.753511+0000","last_undegraded":"2023-11-08T08:21:18.753511+0000","last_fullsized":"2023-11-08T08:21:18.753511+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:22:01.799995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.808679+0000","last_change":"2023-11-08T08:21:17.763462+0000","last_active":"2023-11-08T08:21:18.808679+0000","last_peered":"2023-11-08T08:21:18.808679+0000","last_clean":"2023-11-08T08:21:18.808679+0000","last_became_active":"2023-11-08T08:21:17.763150+0000","last_became_peered":"2023-11-08T08:21:17.763150+0000","last_unstale":"2023-11-08T08:21:18.808679+0000","last_undegraded":"2023-11-08T08:21:18.808679+0000","last_fullsized":"2023-11-08T08:21:18.808679+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:14:54.711453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"56'1","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809967+0000","last_change":"2023-11-08T08:21:17.287812+0000","last_active":"2023-11-08T08:21:18.809967+0000","last_peered":"2023-11-08T08:21:18.809967+0000","last_clean":"2023-11-08T08:21:18.809967+0000","last_became_active":"2023-11-08T08:21:17.287547+0000","last_became_peered":"2023-11-08T08:21:17.287547+0000","last_unstale":"2023-11-08T08:21:18.809967+0000","last_undegraded":"2023-11-08T08:21:18.809967+0000","last_fullsized":"2023-11-08T08:21:18.809967+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:44:40.357032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753059+0000","last_change":"2023-11-08T08:21:17.762739+0000","last_active":"2023-11-08T08:21:18.753059+0000","last_peered":"2023-11-08T08:21:18.753059+0000","last_clean":"2023-11-08T08:21:18.753059+0000","last_became_active":"2023-11-08T08:21:17.762471+0000","last_became_peered":"2023-11-08T08:21:17.762471+0000","last_unstale":"2023-11-08T08:21:18.753059+0000","last_undegraded":"2023-11-08T08:21:18.753059+0000","last_fullsized":"2023-11-08T08:21:18.753059+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:18:18.548903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753275+0000","last_change":"2023-11-08T08:21:17.281557+0000","last_active":"2023-11-08T08:21:18.753275+0000","last_peered":"2023-11-08T08:21:18.753275+0000","last_clean":"2023-11-08T08:21:18.753275+0000","last_became_active":"2023-11-08T08:21:17.281265+0000","last_became_peered":"2023-11-08T08:21:17.281265+0000","last_unstale":"2023-11-08T08:21:18.753275+0000","last_undegraded":"2023-11-08T08:21:18.753275+0000","last_fullsized":"2023-11-08T08:21:18.753275+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:00:35.071027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753380+0000","last_change":"2023-11-08T08:21:17.777307+0000","last_active":"2023-11-08T08:21:18.753380+0000","last_peered":"2023-11-08T08:21:18.753380+0000","last_clean":"2023-11-08T08:21:18.753380+0000","last_became_active":"2023-11-08T08:21:17.777037+0000","last_became_peered":"2023-11-08T08:21:17.777037+0000","last_unstale":"2023-11-08T08:21:18.753380+0000","last_undegraded":"2023-11-08T08:21:18.753380+0000","last_fullsized":"2023-11-08T08:21:18.753380+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:39:20.550673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"56'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812321+0000","last_change":"2023-11-08T08:21:17.290426+0000","last_active":"2023-11-08T08:21:18.812321+0000","last_peered":"2023-11-08T08:21:18.812321+0000","last_clean":"2023-11-08T08:21:18.812321+0000","last_became_active":"2023-11-08T08:21:17.290113+0000","last_became_peered":"2023-11-08T08:21:17.290113+0000","last_unstale":"2023-11-08T08:21:18.812321+0000","last_undegraded":"2023-11-08T08:21:18.812321+0000","last_fullsized":"2023-11-08T08:21:18.812321+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:46.050979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809816+0000","last_change":"2023-11-08T08:21:17.776014+0000","last_active":"2023-11-08T08:21:18.809816+0000","last_peered":"2023-11-08T08:21:18.809816+0000","last_clean":"2023-11-08T08:21:18.809816+0000","last_became_active":"2023-11-08T08:21:17.775433+0000","last_became_peered":"2023-11-08T08:21:17.775433+0000","last_unstale":"2023-11-08T08:21:18.809816+0000","last_undegraded":"2023-11-08T08:21:18.809816+0000","last_fullsized":"2023-11-08T08:21:18.809816+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:07:42.164520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.755251+0000","last_change":"2023-11-08T08:21:17.302570+0000","last_active":"2023-11-08T08:21:18.755251+0000","last_peered":"2023-11-08T08:21:18.755251+0000","last_clean":"2023-11-08T08:21:18.755251+0000","last_became_active":"2023-11-08T08:21:17.302288+0000","last_became_peered":"2023-11-08T08:21:17.302288+0000","last_unstale":"2023-11-08T08:21:18.755251+0000","last_undegraded":"2023-11-08T08:21:18.755251+0000","last_fullsized":"2023-11-08T08:21:18.755251+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:00:15.086010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809501+0000","last_change":"2023-11-08T08:21:17.784037+0000","last_active":"2023-11-08T08:21:18.809501+0000","last_peered":"2023-11-08T08:21:18.809501+0000","last_clean":"2023-11-08T08:21:18.809501+0000","last_became_active":"2023-11-08T08:21:17.779861+0000","last_became_peered":"2023-11-08T08:21:17.779861+0000","last_unstale":"2023-11-08T08:21:18.809501+0000","last_undegraded":"2023-11-08T08:21:18.809501+0000","last_fullsized":"2023-11-08T08:21:18.809501+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:53:38.208291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.752714+0000","last_change":"2023-11-08T08:21:17.289310+0000","last_active":"2023-11-08T08:21:18.752714+0000","last_peered":"2023-11-08T08:21:18.752714+0000","last_clean":"2023-11-08T08:21:18.752714+0000","last_became_active":"2023-11-08T08:21:17.289009+0000","last_became_peered":"2023-11-08T08:21:17.289009+0000","last_unstale":"2023-11-08T08:21:18.752714+0000","last_undegraded":"2023-11-08T08:21:18.752714+0000","last_fullsized":"2023-11-08T08:21:18.752714+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:00:15.086010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812775+0000","last_change":"2023-11-08T08:21:17.767414+0000","last_active":"2023-11-08T08:21:18.812775+0000","last_peered":"2023-11-08T08:21:18.812775+0000","last_clean":"2023-11-08T08:21:18.812775+0000","last_became_active":"2023-11-08T08:21:17.766859+0000","last_became_peered":"2023-11-08T08:21:17.766859+0000","last_unstale":"2023-11-08T08:21:18.812775+0000","last_undegraded":"2023-11-08T08:21:18.812775+0000","last_fullsized":"2023-11-08T08:21:18.812775+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:27:03.140038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812621+0000","last_change":"2023-11-08T08:21:17.290817+0000","last_active":"2023-11-08T08:21:18.812621+0000","last_peered":"2023-11-08T08:21:18.812621+0000","last_clean":"2023-11-08T08:21:18.812621+0000","last_became_active":"2023-11-08T08:21:17.290538+0000","last_became_peered":"2023-11-08T08:21:17.290538+0000","last_unstale":"2023-11-08T08:21:18.812621+0000","last_undegraded":"2023-11-08T08:21:18.812621+0000","last_fullsized":"2023-11-08T08:21:18.812621+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:09:29.890451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809539+0000","last_change":"2023-11-08T08:21:17.773741+0000","last_active":"2023-11-08T08:21:18.809539+0000","last_peered":"2023-11-08T08:21:18.809539+0000","last_clean":"2023-11-08T08:21:18.809539+0000","last_became_active":"2023-11-08T08:21:17.773489+0000","last_became_peered":"2023-11-08T08:21:17.773489+0000","last_unstale":"2023-11-08T08:21:18.809539+0000","last_undegraded":"2023-11-08T08:21:18.809539+0000","last_fullsized":"2023-11-08T08:21:18.809539+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:28:38.025842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.754733+0000","last_change":"2023-11-08T08:21:17.292281+0000","last_active":"2023-11-08T08:21:18.754733+0000","last_peered":"2023-11-08T08:21:18.754733+0000","last_clean":"2023-11-08T08:21:18.754733+0000","last_became_active":"2023-11-08T08:21:17.291983+0000","last_became_peered":"2023-11-08T08:21:17.291983+0000","last_unstale":"2023-11-08T08:21:18.754733+0000","last_undegraded":"2023-11-08T08:21:18.754733+0000","last_fullsized":"2023-11-08T08:21:18.754733+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:22:59.290723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.755350+0000","last_change":"2023-11-08T08:21:17.770902+0000","last_active":"2023-11-08T08:21:18.755350+0000","last_peered":"2023-11-08T08:21:18.755350+0000","last_clean":"2023-11-08T08:21:18.755350+0000","last_became_active":"2023-11-08T08:21:17.769289+0000","last_became_peered":"2023-11-08T08:21:17.769289+0000","last_unstale":"2023-11-08T08:21:18.755350+0000","last_undegraded":"2023-11-08T08:21:18.755350+0000","last_fullsized":"2023-11-08T08:21:18.755350+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:28:38.025842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.814593+0000","last_change":"2023-11-08T08:21:17.289374+0000","last_active":"2023-11-08T08:21:18.814593+0000","last_peered":"2023-11-08T08:21:18.814593+0000","last_clean":"2023-11-08T08:21:18.814593+0000","last_became_active":"2023-11-08T08:21:17.289090+0000","last_became_peered":"2023-11-08T08:21:17.289090+0000","last_unstale":"2023-11-08T08:21:18.814593+0000","last_undegraded":"2023-11-08T08:21:18.814593+0000","last_fullsized":"2023-11-08T08:21:18.814593+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:14:53.697954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.814512+0000","last_change":"2023-11-08T08:21:17.763076+0000","last_active":"2023-11-08T08:21:18.814512+0000","last_peered":"2023-11-08T08:21:18.814512+0000","last_clean":"2023-11-08T08:21:18.814512+0000","last_became_active":"2023-11-08T08:21:17.762785+0000","last_became_peered":"2023-11-08T08:21:17.762785+0000","last_unstale":"2023-11-08T08:21:18.814512+0000","last_undegraded":"2023-11-08T08:21:18.814512+0000","last_fullsized":"2023-11-08T08:21:18.814512+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:55:51.112712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.810260+0000","last_change":"2023-11-08T08:21:17.299611+0000","last_active":"2023-11-08T08:21:18.810260+0000","last_peered":"2023-11-08T08:21:18.810260+0000","last_clean":"2023-11-08T08:21:18.810260+0000","last_became_active":"2023-11-08T08:21:17.299303+0000","last_became_peered":"2023-11-08T08:21:17.299303+0000","last_unstale":"2023-11-08T08:21:18.810260+0000","last_undegraded":"2023-11-08T08:21:18.810260+0000","last_fullsized":"2023-11-08T08:21:18.810260+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:22:01.799995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.808195+0000","last_change":"2023-11-08T08:21:17.776807+0000","last_active":"2023-11-08T08:21:18.808195+0000","last_peered":"2023-11-08T08:21:18.808195+0000","last_clean":"2023-11-08T08:21:18.808195+0000","last_became_active":"2023-11-08T08:21:17.774585+0000","last_became_peered":"2023-11-08T08:21:17.774585+0000","last_unstale":"2023-11-08T08:21:18.808195+0000","last_undegraded":"2023-11-08T08:21:18.808195+0000","last_fullsized":"2023-11-08T08:21:18.808195+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:58:12.422909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"56'3","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.814223+0000","last_change":"2023-11-08T08:21:17.299245+0000","last_active":"2023-11-08T08:21:18.814223+0000","last_peered":"2023-11-08T08:21:18.814223+0000","last_clean":"2023-11-08T08:21:18.814223+0000","last_became_active":"2023-11-08T08:21:17.298868+0000","last_became_peered":"2023-11-08T08:21:17.298868+0000","last_unstale":"2023-11-08T08:21:18.814223+0000","last_undegraded":"2023-11-08T08:21:18.814223+0000","last_fullsized":"2023-11-08T08:21:18.814223+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:32:14.393382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813973+0000","last_change":"2023-11-08T08:21:17.777569+0000","last_active":"2023-11-08T08:21:18.813973+0000","last_peered":"2023-11-08T08:21:18.813973+0000","last_clean":"2023-11-08T08:21:18.813973+0000","last_became_active":"2023-11-08T08:21:17.777269+0000","last_became_peered":"2023-11-08T08:21:17.777269+0000","last_unstale":"2023-11-08T08:21:18.813973+0000","last_undegraded":"2023-11-08T08:21:18.813973+0000","last_fullsized":"2023-11-08T08:21:18.813973+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:12:08.899976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.755300+0000","last_change":"2023-11-08T08:21:17.293406+0000","last_active":"2023-11-08T08:21:18.755300+0000","last_peered":"2023-11-08T08:21:18.755300+0000","last_clean":"2023-11-08T08:21:18.755300+0000","last_became_active":"2023-11-08T08:21:17.292530+0000","last_became_peered":"2023-11-08T08:21:17.292530+0000","last_unstale":"2023-11-08T08:21:18.755300+0000","last_undegraded":"2023-11-08T08:21:18.755300+0000","last_fullsized":"2023-11-08T08:21:18.755300+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:22:01.799995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812524+0000","last_change":"2023-11-08T08:21:17.771134+0000","last_active":"2023-11-08T08:21:18.812524+0000","last_peered":"2023-11-08T08:21:18.812524+0000","last_clean":"2023-11-08T08:21:18.812524+0000","last_became_active":"2023-11-08T08:21:17.770882+0000","last_became_peered":"2023-11-08T08:21:17.770882+0000","last_unstale":"2023-11-08T08:21:18.812524+0000","last_undegraded":"2023-11-08T08:21:18.812524+0000","last_fullsized":"2023-11-08T08:21:18.812524+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:36:37.378597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"56'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809593+0000","last_change":"2023-11-08T08:21:17.302611+0000","last_active":"2023-11-08T08:21:18.809593+0000","last_peered":"2023-11-08T08:21:18.809593+0000","last_clean":"2023-11-08T08:21:18.809593+0000","last_became_active":"2023-11-08T08:21:17.302309+0000","last_became_peered":"2023-11-08T08:21:17.302309+0000","last_unstale":"2023-11-08T08:21:18.809593+0000","last_undegraded":"2023-11-08T08:21:18.809593+0000","last_fullsized":"2023-11-08T08:21:18.809593+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:00:35.071027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813457+0000","last_change":"2023-11-08T08:21:17.777423+0000","last_active":"2023-11-08T08:21:18.813457+0000","last_peered":"2023-11-08T08:21:18.813457+0000","last_clean":"2023-11-08T08:21:18.813457+0000","last_became_active":"2023-11-08T08:21:17.776987+0000","last_became_peered":"2023-11-08T08:21:17.776987+0000","last_unstale":"2023-11-08T08:21:18.813457+0000","last_undegraded":"2023-11-08T08:21:18.813457+0000","last_fullsized":"2023-11-08T08:21:18.813457+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:44:41.370532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813247+0000","last_change":"2023-11-08T08:21:17.288246+0000","last_active":"2023-11-08T08:21:18.813247+0000","last_peered":"2023-11-08T08:21:18.813247+0000","last_clean":"2023-11-08T08:21:18.813247+0000","last_became_active":"2023-11-08T08:21:17.287961+0000","last_became_peered":"2023-11-08T08:21:17.287961+0000","last_unstale":"2023-11-08T08:21:18.813247+0000","last_undegraded":"2023-11-08T08:21:18.813247+0000","last_fullsized":"2023-11-08T08:21:18.813247+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:52:49.330954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.754710+0000","last_change":"2023-11-08T08:21:17.770103+0000","last_active":"2023-11-08T08:21:18.754710+0000","last_peered":"2023-11-08T08:21:18.754710+0000","last_clean":"2023-11-08T08:21:18.754710+0000","last_became_active":"2023-11-08T08:21:17.769763+0000","last_became_peered":"2023-11-08T08:21:17.769763+0000","last_unstale":"2023-11-08T08:21:18.754710+0000","last_undegraded":"2023-11-08T08:21:18.754710+0000","last_fullsized":"2023-11-08T08:21:18.754710+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:30:04.897468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812793+0000","last_change":"2023-11-08T08:21:17.292207+0000","last_active":"2023-11-08T08:21:18.812793+0000","last_peered":"2023-11-08T08:21:18.812793+0000","last_clean":"2023-11-08T08:21:18.812793+0000","last_became_active":"2023-11-08T08:21:17.291975+0000","last_became_peered":"2023-11-08T08:21:17.291975+0000","last_unstale":"2023-11-08T08:21:18.812793+0000","last_undegraded":"2023-11-08T08:21:18.812793+0000","last_fullsized":"2023-11-08T08:21:18.812793+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:55:55.253541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812726+0000","last_change":"2023-11-08T08:21:17.767546+0000","last_active":"2023-11-08T08:21:18.812726+0000","last_peered":"2023-11-08T08:21:18.812726+0000","last_clean":"2023-11-08T08:21:18.812726+0000","last_became_active":"2023-11-08T08:21:17.767191+0000","last_became_peered":"2023-11-08T08:21:17.767191+0000","last_unstale":"2023-11-08T08:21:18.812726+0000","last_undegraded":"2023-11-08T08:21:18.812726+0000","last_fullsized":"2023-11-08T08:21:18.812726+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:00:36.084527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"56'8","reported_seq":19,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812048+0000","last_change":"2023-11-08T08:21:17.291277+0000","last_active":"2023-11-08T08:21:18.812048+0000","last_peered":"2023-11-08T08:21:18.812048+0000","last_clean":"2023-11-08T08:21:18.812048+0000","last_became_active":"2023-11-08T08:21:17.290945+0000","last_became_peered":"2023-11-08T08:21:17.290945+0000","last_unstale":"2023-11-08T08:21:18.812048+0000","last_undegraded":"2023-11-08T08:21:18.812048+0000","last_fullsized":"2023-11-08T08:21:18.812048+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:27:02.126538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813277+0000","last_change":"2023-11-08T08:21:17.783141+0000","last_active":"2023-11-08T08:21:18.813277+0000","last_peered":"2023-11-08T08:21:18.813277+0000","last_clean":"2023-11-08T08:21:18.813277+0000","last_became_active":"2023-11-08T08:21:17.782750+0000","last_became_peered":"2023-11-08T08:21:17.782750+0000","last_unstale":"2023-11-08T08:21:18.813277+0000","last_undegraded":"2023-11-08T08:21:18.813277+0000","last_fullsized":"2023-11-08T08:21:18.813277+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:47:35.532963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"56'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.808768+0000","last_change":"2023-11-08T08:21:17.286590+0000","last_active":"2023-11-08T08:21:18.808768+0000","last_peered":"2023-11-08T08:21:18.808768+0000","last_clean":"2023-11-08T08:21:18.808768+0000","last_became_active":"2023-11-08T08:21:17.286273+0000","last_became_peered":"2023-11-08T08:21:17.286273+0000","last_unstale":"2023-11-08T08:21:18.808768+0000","last_undegraded":"2023-11-08T08:21:18.808768+0000","last_fullsized":"2023-11-08T08:21:18.808768+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:40:30.132469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812906+0000","last_change":"2023-11-08T08:21:17.770666+0000","last_active":"2023-11-08T08:21:18.812906+0000","last_peered":"2023-11-08T08:21:18.812906+0000","last_clean":"2023-11-08T08:21:18.812906+0000","last_became_active":"2023-11-08T08:21:17.767891+0000","last_became_peered":"2023-11-08T08:21:17.767891+0000","last_unstale":"2023-11-08T08:21:18.812906+0000","last_undegraded":"2023-11-08T08:21:18.812906+0000","last_fullsized":"2023-11-08T08:21:18.812906+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:00:36.084527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753572+0000","last_change":"2023-11-08T08:21:17.292014+0000","last_active":"2023-11-08T08:21:18.753572+0000","last_peered":"2023-11-08T08:21:18.753572+0000","last_clean":"2023-11-08T08:21:18.753572+0000","last_became_active":"2023-11-08T08:21:17.291696+0000","last_became_peered":"2023-11-08T08:21:17.291696+0000","last_unstale":"2023-11-08T08:21:18.753572+0000","last_undegraded":"2023-11-08T08:21:18.753572+0000","last_fullsized":"2023-11-08T08:21:18.753572+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:13:07.575998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.808404+0000","last_change":"2023-11-08T08:21:17.767063+0000","last_active":"2023-11-08T08:21:18.808404+0000","last_peered":"2023-11-08T08:21:18.808404+0000","last_clean":"2023-11-08T08:21:18.808404+0000","last_became_active":"2023-11-08T08:21:17.766672+0000","last_became_peered":"2023-11-08T08:21:17.766672+0000","last_unstale":"2023-11-08T08:21:18.808404+0000","last_undegraded":"2023-11-08T08:21:18.808404+0000","last_fullsized":"2023-11-08T08:21:18.808404+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:18:24.957381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813879+0000","last_change":"2023-11-08T08:21:17.294933+0000","last_active":"2023-11-08T08:21:18.813879+0000","last_peered":"2023-11-08T08:21:18.813879+0000","last_clean":"2023-11-08T08:21:18.813879+0000","last_became_active":"2023-11-08T08:21:17.294633+0000","last_became_peered":"2023-11-08T08:21:17.294633+0000","last_unstale":"2023-11-08T08:21:18.813879+0000","last_undegraded":"2023-11-08T08:21:18.813879+0000","last_fullsized":"2023-11-08T08:21:18.813879+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:41:17.888191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753173+0000","last_change":"2023-11-08T08:21:17.775361+0000","last_active":"2023-11-08T08:21:18.753173+0000","last_peered":"2023-11-08T08:21:18.753173+0000","last_clean":"2023-11-08T08:21:18.753173+0000","last_became_active":"2023-11-08T08:21:17.775101+0000","last_became_peered":"2023-11-08T08:21:17.775101+0000","last_unstale":"2023-11-08T08:21:18.753173+0000","last_undegraded":"2023-11-08T08:21:18.753173+0000","last_fullsized":"2023-11-08T08:21:18.753173+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:28:52.114179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812441+0000","last_change":"2023-11-08T08:21:17.291899+0000","last_active":"2023-11-08T08:21:18.812441+0000","last_peered":"2023-11-08T08:21:18.812441+0000","last_clean":"2023-11-08T08:21:18.812441+0000","last_became_active":"2023-11-08T08:21:17.291210+0000","last_became_peered":"2023-11-08T08:21:17.291210+0000","last_unstale":"2023-11-08T08:21:18.812441+0000","last_undegraded":"2023-11-08T08:21:18.812441+0000","last_fullsized":"2023-11-08T08:21:18.812441+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:06:57.770400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.810089+0000","last_change":"2023-11-08T08:21:17.769525+0000","last_active":"2023-11-08T08:21:18.810089+0000","last_peered":"2023-11-08T08:21:18.810089+0000","last_clean":"2023-11-08T08:21:18.810089+0000","last_became_active":"2023-11-08T08:21:17.769228+0000","last_became_peered":"2023-11-08T08:21:17.769228+0000","last_unstale":"2023-11-08T08:21:18.810089+0000","last_undegraded":"2023-11-08T08:21:18.810089+0000","last_fullsized":"2023-11-08T08:21:18.810089+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:13:54.410850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.807845+0000","last_change":"2023-11-08T08:21:17.283845+0000","last_active":"2023-11-08T08:21:18.807845+0000","last_peered":"2023-11-08T08:21:18.807845+0000","last_clean":"2023-11-08T08:21:18.807845+0000","last_became_active":"2023-11-08T08:21:17.283085+0000","last_became_peered":"2023-11-08T08:21:17.283085+0000","last_unstale":"2023-11-08T08:21:18.807845+0000","last_undegraded":"2023-11-08T08:21:18.807845+0000","last_fullsized":"2023-11-08T08:21:18.807845+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:10:08.567828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813882+0000","last_change":"2023-11-08T08:21:17.777223+0000","last_active":"2023-11-08T08:21:18.813882+0000","last_peered":"2023-11-08T08:21:18.813882+0000","last_clean":"2023-11-08T08:21:18.813882+0000","last_became_active":"2023-11-08T08:21:17.776641+0000","last_became_peered":"2023-11-08T08:21:17.776641+0000","last_unstale":"2023-11-08T08:21:18.813882+0000","last_undegraded":"2023-11-08T08:21:18.813882+0000","last_fullsized":"2023-11-08T08:21:18.813882+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:13:08.589498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.754998+0000","last_change":"2023-11-08T08:21:17.290352+0000","last_active":"2023-11-08T08:21:18.754998+0000","last_peered":"2023-11-08T08:21:18.754998+0000","last_clean":"2023-11-08T08:21:18.754998+0000","last_became_active":"2023-11-08T08:21:17.289568+0000","last_became_peered":"2023-11-08T08:21:17.289568+0000","last_unstale":"2023-11-08T08:21:18.754998+0000","last_undegraded":"2023-11-08T08:21:18.754998+0000","last_fullsized":"2023-11-08T08:21:18.754998+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:13:07.575998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.807399+0000","last_change":"2023-11-08T08:21:17.775092+0000","last_active":"2023-11-08T08:21:18.807399+0000","last_peered":"2023-11-08T08:21:18.807399+0000","last_clean":"2023-11-08T08:21:18.807399+0000","last_became_active":"2023-11-08T08:21:17.774817+0000","last_became_peered":"2023-11-08T08:21:17.774817+0000","last_unstale":"2023-11-08T08:21:18.807399+0000","last_undegraded":"2023-11-08T08:21:18.807399+0000","last_fullsized":"2023-11-08T08:21:18.807399+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:44:46.041614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.807768+0000","last_change":"2023-11-08T08:21:17.287571+0000","last_active":"2023-11-08T08:21:18.807768+0000","last_peered":"2023-11-08T08:21:18.807768+0000","last_clean":"2023-11-08T08:21:18.807768+0000","last_became_active":"2023-11-08T08:21:17.287071+0000","last_became_peered":"2023-11-08T08:21:17.287071+0000","last_unstale":"2023-11-08T08:21:18.807768+0000","last_undegraded":"2023-11-08T08:21:18.807768+0000","last_fullsized":"2023-11-08T08:21:18.807768+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:59:09.027422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809443+0000","last_change":"2023-11-08T08:21:17.777907+0000","last_active":"2023-11-08T08:21:18.809443+0000","last_peered":"2023-11-08T08:21:18.809443+0000","last_clean":"2023-11-08T08:21:18.809443+0000","last_became_active":"2023-11-08T08:21:17.777571+0000","last_became_peered":"2023-11-08T08:21:17.777571+0000","last_unstale":"2023-11-08T08:21:18.809443+0000","last_undegraded":"2023-11-08T08:21:18.809443+0000","last_fullsized":"2023-11-08T08:21:18.809443+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:28:52.114179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809269+0000","last_change":"2023-11-08T08:21:17.291335+0000","last_active":"2023-11-08T08:21:18.809269+0000","last_peered":"2023-11-08T08:21:18.809269+0000","last_clean":"2023-11-08T08:21:18.809269+0000","last_became_active":"2023-11-08T08:21:17.291030+0000","last_became_peered":"2023-11-08T08:21:17.291030+0000","last_unstale":"2023-11-08T08:21:18.809269+0000","last_undegraded":"2023-11-08T08:21:18.809269+0000","last_fullsized":"2023-11-08T08:21:18.809269+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:15:03.992666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.810089+0000","last_change":"2023-11-08T08:21:17.779348+0000","last_active":"2023-11-08T08:21:18.810089+0000","last_peered":"2023-11-08T08:21:18.810089+0000","last_clean":"2023-11-08T08:21:18.810089+0000","last_became_active":"2023-11-08T08:21:17.779100+0000","last_became_peered":"2023-11-08T08:21:17.779100+0000","last_unstale":"2023-11-08T08:21:18.810089+0000","last_undegraded":"2023-11-08T08:21:18.810089+0000","last_fullsized":"2023-11-08T08:21:18.810089+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:02:19.326469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"56'1","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813142+0000","last_change":"2023-11-08T08:21:17.291536+0000","last_active":"2023-11-08T08:21:18.813142+0000","last_peered":"2023-11-08T08:21:18.813142+0000","last_clean":"2023-11-08T08:21:18.813142+0000","last_became_active":"2023-11-08T08:21:17.291241+0000","last_became_peered":"2023-11-08T08:21:17.291241+0000","last_unstale":"2023-11-08T08:21:18.813142+0000","last_undegraded":"2023-11-08T08:21:18.813142+0000","last_fullsized":"2023-11-08T08:21:18.813142+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:24:43.804674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-08T08:21:17.746955+0000","last_change":"2023-11-08T08:18:58.771310+0000","last_active":"2023-11-08T08:21:17.746955+0000","last_peered":"2023-11-08T08:21:17.746955+0000","last_clean":"2023-11-08T08:21:17.746955+0000","last_became_active":"2023-11-08T08:18:58.770891+0000","last_became_peered":"2023-11-08T08:18:58.770891+0000","last_unstale":"2023-11-08T08:21:17.746955+0000","last_undegraded":"2023-11-08T08:21:17.746955+0000","last_fullsized":"2023-11-08T08:21:17.746955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.814094+0000","last_change":"2023-11-08T08:21:17.775920+0000","last_active":"2023-11-08T08:21:18.814094+0000","last_peered":"2023-11-08T08:21:18.814094+0000","last_clean":"2023-11-08T08:21:18.814094+0000","last_became_active":"2023-11-08T08:21:17.771601+0000","last_became_peered":"2023-11-08T08:21:17.771601+0000","last_unstale":"2023-11-08T08:21:18.814094+0000","last_undegraded":"2023-11-08T08:21:18.814094+0000","last_fullsized":"2023-11-08T08:21:18.814094+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:10:57.252073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753180+0000","last_change":"2023-11-08T08:21:17.287063+0000","last_active":"2023-11-08T08:21:18.753180+0000","last_peered":"2023-11-08T08:21:18.753180+0000","last_clean":"2023-11-08T08:21:18.753180+0000","last_became_active":"2023-11-08T08:21:17.286181+0000","last_became_peered":"2023-11-08T08:21:17.286181+0000","last_unstale":"2023-11-08T08:21:18.753180+0000","last_undegraded":"2023-11-08T08:21:18.753180+0000","last_fullsized":"2023-11-08T08:21:18.753180+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:22:59.290723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-08T08:21:17.754057+0000","last_change":"2023-11-08T08:18:58.770483+0000","last_active":"2023-11-08T08:21:17.754057+0000","last_peered":"2023-11-08T08:21:17.754057+0000","last_clean":"2023-11-08T08:21:17.754057+0000","last_became_active":"2023-11-08T08:18:58.770040+0000","last_became_peered":"2023-11-08T08:18:58.770040+0000","last_unstale":"2023-11-08T08:21:17.754057+0000","last_undegraded":"2023-11-08T08:21:17.754057+0000","last_fullsized":"2023-11-08T08:21:17.754057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.814487+0000","last_change":"2023-11-08T08:21:17.777849+0000","last_active":"2023-11-08T08:21:18.814487+0000","last_peered":"2023-11-08T08:21:18.814487+0000","last_clean":"2023-11-08T08:21:18.814487+0000","last_became_active":"2023-11-08T08:21:17.777549+0000","last_became_peered":"2023-11-08T08:21:17.777549+0000","last_unstale":"2023-11-08T08:21:18.814487+0000","last_undegraded":"2023-11-08T08:21:18.814487+0000","last_fullsized":"2023-11-08T08:21:18.814487+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:40:34.435351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"56'1","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.810425+0000","last_change":"2023-11-08T08:21:17.300526+0000","last_active":"2023-11-08T08:21:18.810425+0000","last_peered":"2023-11-08T08:21:18.810425+0000","last_clean":"2023-11-08T08:21:18.810425+0000","last_became_active":"2023-11-08T08:21:17.298791+0000","last_became_peered":"2023-11-08T08:21:17.298791+0000","last_unstale":"2023-11-08T08:21:18.810425+0000","last_undegraded":"2023-11-08T08:21:18.810425+0000","last_fullsized":"2023-11-08T08:21:18.810425+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:12:07.886476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-08T08:21:17.753273+0000","last_change":"2023-11-08T08:18:58.766848+0000","last_active":"2023-11-08T08:21:17.753273+0000","last_peered":"2023-11-08T08:21:17.753273+0000","last_clean":"2023-11-08T08:21:17.753273+0000","last_became_active":"2023-11-08T08:18:58.766378+0000","last_became_peered":"2023-11-08T08:18:58.766378+0000","last_unstale":"2023-11-08T08:21:17.753273+0000","last_undegraded":"2023-11-08T08:21:17.753273+0000","last_fullsized":"2023-11-08T08:21:17.753273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812767+0000","last_change":"2023-11-08T08:21:17.772916+0000","last_active":"2023-11-08T08:21:18.812767+0000","last_peered":"2023-11-08T08:21:18.812767+0000","last_clean":"2023-11-08T08:21:18.812767+0000","last_became_active":"2023-11-08T08:21:17.772642+0000","last_became_peered":"2023-11-08T08:21:17.772642+0000","last_unstale":"2023-11-08T08:21:18.812767+0000","last_undegraded":"2023-11-08T08:21:18.812767+0000","last_fullsized":"2023-11-08T08:21:18.812767+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:00:16.099510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753592+0000","last_change":"2023-11-08T08:21:17.298893+0000","last_active":"2023-11-08T08:21:18.753592+0000","last_peered":"2023-11-08T08:21:18.753592+0000","last_clean":"2023-11-08T08:21:18.753592+0000","last_became_active":"2023-11-08T08:21:17.298621+0000","last_became_peered":"2023-11-08T08:21:17.298621+0000","last_unstale":"2023-11-08T08:21:18.753592+0000","last_undegraded":"2023-11-08T08:21:18.753592+0000","last_fullsized":"2023-11-08T08:21:18.753592+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:36:36.365097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-08T08:21:17.755150+0000","last_change":"2023-11-08T08:18:58.769764+0000","last_active":"2023-11-08T08:21:17.755150+0000","last_peered":"2023-11-08T08:21:17.755150+0000","last_clean":"2023-11-08T08:21:17.755150+0000","last_became_active":"2023-11-08T08:18:58.769357+0000","last_became_peered":"2023-11-08T08:18:58.769357+0000","last_unstale":"2023-11-08T08:21:17.755150+0000","last_undegraded":"2023-11-08T08:21:17.755150+0000","last_fullsized":"2023-11-08T08:21:17.755150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:13.748094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.752433+0000","last_change":"2023-11-08T08:21:17.753174+0000","last_active":"2023-11-08T08:21:18.752433+0000","last_peered":"2023-11-08T08:21:18.752433+0000","last_clean":"2023-11-08T08:21:18.752433+0000","last_became_active":"2023-11-08T08:21:17.752894+0000","last_became_peered":"2023-11-08T08:21:17.752894+0000","last_unstale":"2023-11-08T08:21:18.752433+0000","last_undegraded":"2023-11-08T08:21:18.752433+0000","last_fullsized":"2023-11-08T08:21:18.752433+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:19:00.004764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812056+0000","last_change":"2023-11-08T08:21:17.294130+0000","last_active":"2023-11-08T08:21:18.812056+0000","last_peered":"2023-11-08T08:21:18.812056+0000","last_clean":"2023-11-08T08:21:18.812056+0000","last_became_active":"2023-11-08T08:21:17.293726+0000","last_became_peered":"2023-11-08T08:21:17.293726+0000","last_unstale":"2023-11-08T08:21:18.812056+0000","last_undegraded":"2023-11-08T08:21:18.812056+0000","last_fullsized":"2023-11-08T08:21:18.812056+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:33:03.450167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-08T08:21:17.755188+0000","last_change":"2023-11-08T08:18:58.771460+0000","last_active":"2023-11-08T08:21:17.755188+0000","last_peered":"2023-11-08T08:21:17.755188+0000","last_clean":"2023-11-08T08:21:17.755188+0000","last_became_active":"2023-11-08T08:18:58.770648+0000","last_became_peered":"2023-11-08T08:18:58.770648+0000","last_unstale":"2023-11-08T08:21:17.755188+0000","last_undegraded":"2023-11-08T08:21:17.755188+0000","last_fullsized":"2023-11-08T08:21:17.755188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812848+0000","last_change":"2023-11-08T08:21:17.772227+0000","last_active":"2023-11-08T08:21:18.812848+0000","last_peered":"2023-11-08T08:21:18.812848+0000","last_clean":"2023-11-08T08:21:18.812848+0000","last_became_active":"2023-11-08T08:21:17.771928+0000","last_became_peered":"2023-11-08T08:21:17.771928+0000","last_unstale":"2023-11-08T08:21:18.812848+0000","last_undegraded":"2023-11-08T08:21:18.812848+0000","last_fullsized":"2023-11-08T08:21:18.812848+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:23:00.304223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"56'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809772+0000","last_change":"2023-11-08T08:21:17.296624+0000","last_active":"2023-11-08T08:21:18.809772+0000","last_peered":"2023-11-08T08:21:18.809772+0000","last_clean":"2023-11-08T08:21:18.809772+0000","last_became_active":"2023-11-08T08:21:17.296357+0000","last_became_peered":"2023-11-08T08:21:17.296357+0000","last_unstale":"2023-11-08T08:21:18.809772+0000","last_undegraded":"2023-11-08T08:21:18.809772+0000","last_fullsized":"2023-11-08T08:21:18.809772+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:40:33.421851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-08T08:21:17.745102+0000","last_change":"2023-11-08T08:18:58.973732+0000","last_active":"2023-11-08T08:21:17.745102+0000","last_peered":"2023-11-08T08:21:17.745102+0000","last_clean":"2023-11-08T08:21:17.745102+0000","last_became_active":"2023-11-08T08:18:58.973007+0000","last_became_peered":"2023-11-08T08:18:58.973007+0000","last_unstale":"2023-11-08T08:21:17.745102+0000","last_undegraded":"2023-11-08T08:21:17.745102+0000","last_fullsized":"2023-11-08T08:21:17.745102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.752940+0000","last_change":"2023-11-08T08:21:17.768129+0000","last_active":"2023-11-08T08:21:18.752940+0000","last_peered":"2023-11-08T08:21:18.752940+0000","last_clean":"2023-11-08T08:21:18.752940+0000","last_became_active":"2023-11-08T08:21:17.762620+0000","last_became_peered":"2023-11-08T08:21:17.762620+0000","last_unstale":"2023-11-08T08:21:18.752940+0000","last_undegraded":"2023-11-08T08:21:18.752940+0000","last_fullsized":"2023-11-08T08:21:18.752940+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:13:54.410850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.807880+0000","last_change":"2023-11-08T08:21:17.301161+0000","last_active":"2023-11-08T08:21:18.807880+0000","last_peered":"2023-11-08T08:21:18.807880+0000","last_clean":"2023-11-08T08:21:18.807880+0000","last_became_active":"2023-11-08T08:21:17.300579+0000","last_became_peered":"2023-11-08T08:21:17.300579+0000","last_unstale":"2023-11-08T08:21:18.807880+0000","last_undegraded":"2023-11-08T08:21:18.807880+0000","last_fullsized":"2023-11-08T08:21:18.807880+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:27:39.874588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-08T08:21:17.750488+0000","last_change":"2023-11-08T08:18:58.770136+0000","last_active":"2023-11-08T08:21:17.750488+0000","last_peered":"2023-11-08T08:21:17.750488+0000","last_clean":"2023-11-08T08:21:17.750488+0000","last_became_active":"2023-11-08T08:18:58.769797+0000","last_became_peered":"2023-11-08T08:18:58.769797+0000","last_unstale":"2023-11-08T08:21:17.750488+0000","last_undegraded":"2023-11-08T08:21:17.750488+0000","last_fullsized":"2023-11-08T08:21:17.750488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:29:12.107128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809085+0000","last_change":"2023-11-08T08:21:17.764312+0000","last_active":"2023-11-08T08:21:18.809085+0000","last_peered":"2023-11-08T08:21:18.809085+0000","last_clean":"2023-11-08T08:21:18.809085+0000","last_became_active":"2023-11-08T08:21:17.764041+0000","last_became_peered":"2023-11-08T08:21:17.764041+0000","last_unstale":"2023-11-08T08:21:18.809085+0000","last_undegraded":"2023-11-08T08:21:18.809085+0000","last_fullsized":"2023-11-08T08:21:18.809085+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:26:37.331809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813377+0000","last_change":"2023-11-08T08:21:17.296472+0000","last_active":"2023-11-08T08:21:18.813377+0000","last_peered":"2023-11-08T08:21:18.813377+0000","last_clean":"2023-11-08T08:21:18.813377+0000","last_became_active":"2023-11-08T08:21:17.296049+0000","last_became_peered":"2023-11-08T08:21:17.296049+0000","last_unstale":"2023-11-08T08:21:18.813377+0000","last_undegraded":"2023-11-08T08:21:18.813377+0000","last_fullsized":"2023-11-08T08:21:18.813377+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:50:52.603382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-08T08:21:17.763256+0000","last_change":"2023-11-08T08:18:58.777140+0000","last_active":"2023-11-08T08:21:17.763256+0000","last_peered":"2023-11-08T08:21:17.763256+0000","last_clean":"2023-11-08T08:21:17.763256+0000","last_became_active":"2023-11-08T08:18:58.776834+0000","last_became_peered":"2023-11-08T08:18:58.776834+0000","last_unstale":"2023-11-08T08:21:17.763256+0000","last_undegraded":"2023-11-08T08:21:17.763256+0000","last_fullsized":"2023-11-08T08:21:17.763256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":196,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-08T08:21:17.751636+0000","last_change":"2023-11-08T08:18:53.780089+0000","last_active":"2023-11-08T08:21:17.751636+0000","last_peered":"2023-11-08T08:21:17.751636+0000","last_clean":"2023-11-08T08:21:17.751636+0000","last_became_active":"2023-11-08T08:18:53.779533+0000","last_became_peered":"2023-11-08T08:18:53.779533+0000","last_unstale":"2023-11-08T08:21:17.751636+0000","last_undegraded":"2023-11-08T08:21:17.751636+0000","last_fullsized":"2023-11-08T08:21:17.751636+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_clean_scrub_stamp":"2023-11-08T08:18:52.744693+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:58:24.877070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10412,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361428,"statfs":{"total":96636764160,"available":96626102272,"internally_reserved":0,"allocated":1282048,"data_stored":904236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.0349999999999999,"15min":1.012},"min":{"1min":0.63300000000000001,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.169,"15min":1.1619999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":1.147}]},{"osd":1,"last update":"Wed Nov 8 08:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1879999999999999,"15min":1.194},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.484,"5min":1.972,"15min":1.972},"last":1.107},{"interface":"front","average":{"1min":1.248,"5min":1.3109999999999999,"15min":1.321},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.5980000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.1779999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.393,"15min":1.425},"min":{"1min":0.77000000000000002,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.514,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.373,"15min":1.419},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.5720000000000001,"5min":4.9089999999999998,"15min":4.9089999999999998},"last":1.3220000000000001}]},{"osd":3,"last update":"Wed Nov 8 08:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.256,"15min":1.262},"min":{"1min":0.94499999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5109999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.3049999999999999,"15min":1.3120000000000001},"min":{"1min":0.872,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.474,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":1.357}]},{"osd":4,"last update":"Wed Nov 8 08:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2629999999999999,"15min":1.2729999999999999},"min":{"1min":0.94099999999999995,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.431,"15min":1.4370000000000001},"min":{"1min":1.0800000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.7170000000000001,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":1.292}]},{"osd":5,"last update":"Wed Nov 8 08:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.375,"15min":1.389},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.627,"5min":2.73,"15min":2.73},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3380000000000001,"15min":1.3440000000000001},"min":{"1min":0.95399999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.7310000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.218}]},{"osd":6,"last update":"Wed Nov 8 08:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.4450000000000001,"15min":1.456},"min":{"1min":1.1120000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.665,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.258},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.4379999999999999,"15min":1.454},"min":{"1min":0.94099999999999995,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.6870000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.2370000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9184,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362656,"statfs":{"total":96636764160,"available":96627359744,"internally_reserved":0,"allocated":679936,"data_stored":313708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.1639999999999999,"15min":1.1870000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.4350000000000001,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0960000000000001,"15min":1.1020000000000001},"min":{"1min":0.65500000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.4930000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.1879999999999999}]},{"osd":1,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.1990000000000001,"15min":1.1910000000000001},"min":{"1min":0.81299999999999994,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6970000000000001,"5min":1.782,"15min":1.782},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.22,"5min":1.208,"15min":1.206},"min":{"1min":0.76700000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5740000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.1100000000000001}]},{"osd":2,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.175,"15min":1.1839999999999999},"min":{"1min":0.79800000000000004,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.7589999999999999,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.079,"5min":1.262,"15min":1.292},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.4990000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.238}]},{"osd":3,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.278,"15min":1.2889999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.605,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.87},{"interface":"front","average":{"1min":1.331,"5min":1.399,"15min":1.4099999999999999},"min":{"1min":1.0880000000000001,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.6539999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.052}]},{"osd":4,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.248,"15min":1.264},"min":{"1min":0.79100000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.5149999999999999,"5min":2.02,"15min":2.02},"last":1.008},{"interface":"front","average":{"1min":1.28,"5min":1.208,"15min":1.196},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.6619999999999999,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.29}]},{"osd":5,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.2609999999999999,"15min":1.2390000000000001},"min":{"1min":1.153,"5min":0.88,"15min":0.88},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3759999999999999,"15min":1.375},"min":{"1min":1.024,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.6599999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.173}]},{"osd":7,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.389,"15min":1.3879999999999999},"min":{"1min":1.129,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.714,"5min":3.133,"15min":3.133},"last":1.363},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.448,"15min":1.4399999999999999},"min":{"1min":1.004,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.8560000000000001,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.4910000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9256,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362584,"statfs":{"total":96636764160,"available":96627286016,"internally_reserved":0,"allocated":688128,"data_stored":314344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.123,"15min":1.129},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":1.246},{"interface":"front","average":{"1min":1.198,"5min":1.234,"15min":1.24},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.4730000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.1739999999999999}]},{"osd":1,"last update":"Wed Nov 8 08:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.1359999999999999,"15min":1.1419999999999999},"min":{"1min":0.86699999999999999,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.427,"5min":2.012,"15min":2.012},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1799999999999999,"15min":1.1970000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.4450000000000001,"5min":1.667,"15min":1.667},"last":1.2889999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1990000000000001,"15min":1.2010000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.121,"5min":1.214,"15min":1.23},"min":{"1min":0.77700000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.7190000000000001,"5min":1.796,"15min":1.796},"last":1.0409999999999999}]},{"osd":3,"last update":"Wed Nov 8 08:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.333,"15min":1.345},"min":{"1min":1.0009999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":1.4930000000000001,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.3160000000000001,"15min":1.335},"min":{"1min":0.88900000000000001,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.5009999999999999,"5min":3.7999999999999998,"15min":3.7999999999999998},"last":1.1459999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.1839999999999999,"15min":1.21},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.409,"5min":1.629,"15min":1.629},"last":1.085},{"interface":"front","average":{"1min":1.167,"5min":1.325,"15min":1.351},"min":{"1min":0.89300000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.536,"5min":3.073,"15min":3.073},"last":1.1060000000000001}]},{"osd":6,"last update":"Wed Nov 8 08:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.3280000000000001,"15min":1.3580000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.46,"5min":3.8650000000000002,"15min":3.8650000000000002},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.262,"5min":1.373,"15min":1.391},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.587,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.99299999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.478,"15min":1.498},"min":{"1min":1.089,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":1.631,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.278},{"interface":"front","average":{"1min":1.294,"5min":1.474,"15min":1.504},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.605,"5min":3.9369999999999998,"15min":3.9369999999999998},"last":0.95399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9452,"kb_used_data":676,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362388,"statfs":{"total":96636764160,"available":96627085312,"internally_reserved":0,"allocated":692224,"data_stored":315256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.147,"15min":1.145},"min":{"1min":0.82699999999999996,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.171,"5min":1.2250000000000001,"15min":1.234},"min":{"1min":0.76100000000000001,"5min":0.747,"15min":0.747},"max":{"1min":1.5029999999999999,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":1.079}]},{"osd":1,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.079,"15min":1.089},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.3460000000000001,"5min":1.99,"15min":1.99},"last":1.139},{"interface":"front","average":{"1min":1.159,"5min":1.2090000000000001,"15min":1.2170000000000001},"min":{"1min":0.77800000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.546,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.97099999999999997}]},{"osd":2,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.127,"15min":1.1259999999999999},"min":{"1min":0.90600000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.4490000000000001,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.177,"5min":1.256,"15min":1.2689999999999999},"min":{"1min":0.82099999999999995,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.7350000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.1890000000000001}]},{"osd":3,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.2010000000000001,"15min":1.202},"min":{"1min":0.73099999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.5760000000000001,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":1.306},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2390000000000001,"15min":1.2490000000000001},"min":{"1min":0.91900000000000004,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5429999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":0.92600000000000005}]},{"osd":5,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2909999999999999,"15min":1.2989999999999999},"min":{"1min":0.89500000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.8180000000000001,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.264,"15min":1.2609999999999999},"min":{"1min":0.93700000000000006,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.6830000000000001,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":1.002}]},{"osd":6,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2809999999999999,"15min":1.2869999999999999},"min":{"1min":0.82699999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5329999999999999,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3700000000000001,"15min":1.373},"min":{"1min":1.0269999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.7769999999999999,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.4219999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.4359999999999999,"15min":1.4419999999999999},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.772,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.419,"5min":1.4830000000000001,"15min":1.4930000000000001},"min":{"1min":1.1699999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.71,"5min":2.399,"15min":2.399},"last":1.452}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9384,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362456,"statfs":{"total":96636764160,"available":96627154944,"internally_reserved":0,"allocated":688128,"data_stored":313764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.91300000000000003,"15min":0.91500000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.94899999999999995,"15min":0.94799999999999995},"min":{"1min":0.65800000000000003,"5min":0.626,"15min":0.626},"max":{"1min":1.331,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.081}]},{"osd":1,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.93200000000000005,"15min":0.94099999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.125,"15min":1.1319999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.373,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.105}]},{"osd":2,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1200000000000001,"15min":1.1140000000000001},"min":{"1min":0.74099999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.135,"5min":1.2110000000000001,"15min":1.224},"min":{"1min":0.85499999999999998,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.7929999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.04}]},{"osd":4,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.2809999999999999,"15min":1.3100000000000001},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.528,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.153},{"interface":"front","average":{"1min":1.27,"5min":1.3859999999999999,"15min":1.405},"min":{"1min":0.88800000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.496,"5min":2.27,"15min":2.27},"last":1.208}]},{"osd":5,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.4379999999999999,"15min":1.45},"min":{"1min":1.0069999999999999,"5min":0.998,"15min":0.998},"max":{"1min":1.645,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.4219999999999999,"15min":1.4359999999999999},"min":{"1min":1.0349999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.694,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.246}]},{"osd":6,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.4410000000000001,"15min":1.482},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.786,"5min":2.008,"15min":2.008},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.405,"5min":1.528,"15min":1.548},"min":{"1min":1.028,"5min":1.028,"15min":1.028},"max":{"1min":1.77,"5min":2.145,"15min":2.145},"last":1.341}]},{"osd":7,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.583,"15min":1.609},"min":{"1min":1.155,"5min":1.155,"15min":1.155},"max":{"1min":1.819,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":1.524},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.5589999999999999,"15min":1.601},"min":{"1min":1.03,"5min":1.03,"15min":1.03},"max":{"1min":1.7210000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.3160000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9192,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362648,"statfs":{"total":96636764160,"available":96627351552,"internally_reserved":0,"allocated":688128,"data_stored":313764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.92300000000000004,"15min":0.94499999999999995},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.997,"15min":1.0069999999999999},"min":{"1min":0.626,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6100000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.95099999999999996}]},{"osd":1,"last update":"Wed Nov 8 08:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.0900000000000001,"15min":1.115},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.4810000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.014},{"interface":"front","average":{"1min":1.034,"5min":1.03,"15min":1.0289999999999999},"min":{"1min":0.65400000000000003,"5min":0.505,"15min":0.505},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.149}]},{"osd":3,"last update":"Wed Nov 8 08:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.0169999999999999,"15min":1.0209999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.5189999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.3160000000000001,"15min":1.3580000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5620000000000001,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":1.2969999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:20:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.448,"15min":1.486},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.504,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":1.45},{"interface":"front","average":{"1min":1.248,"5min":1.3560000000000001,"15min":1.3740000000000001},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.8260000000000001,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.516}]},{"osd":5,"last update":"Wed Nov 8 08:20:59 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.417,"15min":1.4430000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.9179999999999999,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.708},{"interface":"front","average":{"1min":1.351,"5min":1.355,"15min":1.355},"min":{"1min":0.93999999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.3720000000000001}]},{"osd":6,"last update":"Wed Nov 8 08:20:59 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4339999999999999,"15min":1.4370000000000001},"min":{"1min":0.91400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.8169999999999999,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.5549999999999999,"15min":1.5960000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.6020000000000001,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":1.546}]},{"osd":7,"last update":"Wed Nov 8 08:20:59 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.4670000000000001,"15min":1.484},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.9750000000000001,"5min":2.867,"15min":2.867},"last":1.409},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4910000000000001,"15min":1.492},"min":{"1min":1.012,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.02,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.6599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9252,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362588,"statfs":{"total":96636764160,"available":96627290112,"internally_reserved":0,"allocated":684032,"data_stored":313798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.89100000000000001,"15min":0.89900000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.204,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0449999999999999,"15min":1.0489999999999999},"min":{"1min":0.68100000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3080000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.214}]},{"osd":2,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.97999999999999998,"15min":0.99399999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.101,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.1679999999999999,"15min":1.194},"min":{"1min":0.76000000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.425,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.044}]},{"osd":3,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.109,"15min":1.1120000000000001},"min":{"1min":0.70199999999999996,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3260000000000001,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.1060000000000001,"15min":1.125},"min":{"1min":0.56599999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.8260000000000001,"5min":1.972,"15min":1.972},"last":0.78800000000000003}]},{"osd":4,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.232,"15min":1.236},"min":{"1min":0.77700000000000002,"5min":0.746,"15min":0.746},"max":{"1min":1.639,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3340000000000001,"15min":1.3520000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.554,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":1.28}]},{"osd":5,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.331,"15min":1.341},"min":{"1min":0.94099999999999995,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.694,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.26,"5min":1.3400000000000001,"15min":1.353},"min":{"1min":0.81200000000000006,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.73,"5min":2.48,"15min":2.48},"last":1.258}]},{"osd":6,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.4350000000000001,"15min":1.4490000000000001},"min":{"1min":1.034,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.671,"5min":3.149,"15min":3.149},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.474,"15min":1.482},"min":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":1.7729999999999999,"5min":2.169,"15min":2.169},"last":1.4219999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.5589999999999999,"15min":1.5740000000000001},"min":{"1min":1.145,"5min":1.1020000000000001,"15min":1.1020000000000001},"max":{"1min":1.8720000000000001,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.476,"15min":1.48},"min":{"1min":0.89800000000000002,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.0030000000000001,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.3680000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10548,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361292,"statfs":{"total":96636764160,"available":96625963008,"internally_reserved":0,"allocated":1290240,"data_stored":906190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 08:20:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.94799999999999995,"15min":0.97399999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.153,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.94599999999999995,"15min":0.96699999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.236,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.93700000000000006}]},{"osd":2,"last update":"Wed Nov 8 08:20:53 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.995,"15min":1.0089999999999999},"min":{"1min":0.59099999999999997,"5min":0.497,"15min":0.497},"max":{"1min":1.282,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1020000000000001,"15min":1.1180000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3320000000000001,"5min":1.528,"15min":1.528},"last":1.5920000000000001}]},{"osd":3,"last update":"Wed Nov 8 08:20:53 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.1339999999999999,"15min":1.1679999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3480000000000001,"5min":1.964,"15min":1.964},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.044,"5min":1.1830000000000001,"15min":1.206},"min":{"1min":0.67900000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.343,"5min":1.786,"15min":1.786},"last":1.4139999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:20:53 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.49,"15min":1.5289999999999999},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.6559999999999999,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.909},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.4590000000000001,"15min":1.4870000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.6839999999999999,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.657}]},{"osd":5,"last update":"Wed Nov 8 08:20:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.4870000000000001,"15min":1.52},"min":{"1min":0.999,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.6990000000000001,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.958},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.621,"15min":1.663},"min":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"max":{"1min":1.7430000000000001,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.766}]},{"osd":6,"last update":"Wed Nov 8 08:20:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.512,"15min":1.544},"min":{"1min":0.96799999999999997,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.5780000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.498},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.5629999999999999,"15min":1.5920000000000001},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":1.7430000000000001,"5min":2.048,"15min":2.048},"last":1.8480000000000001}]},{"osd":7,"last update":"Wed Nov 8 08:20:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.6240000000000001,"15min":1.6639999999999999},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":1.708,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.6659999999999999,"15min":1.7050000000000001},"min":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"max":{"1min":1.681,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":2.0070000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T08:21:22.108 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T08:21:22.223 INFO:tasks.ceph.mds.a.smithi040.stderr:2023-11-08T08:21:22.216+0000 7efe52bf5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T08:21:22.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.320+0000 7f78997ef700 1 Processor -- start 2023-11-08T08:21:22.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.320+0000 7f78997ef700 1 -- start start 2023-11-08T08:21:22.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.324+0000 7f78997ef700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7894158e50 0x7f78940c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:22.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.324+0000 7f78997ef700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7894069bf0 0x7f789406c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:22.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.324+0000 7f78997ef700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f789415ceb0 con 0x7f789406c930 2023-11-08T08:21:22.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.324+0000 7f78997ef700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f789406cdb0 con 0x7f7894069bf0 2023-11-08T08:21:22.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.324+0000 7f78997ef700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789415d190 con 0x7f7894158e50 2023-11-08T08:21:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.324+0000 7f788b7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7894069bf0 0x7f789406c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.324+0000 7f7898fee700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f789406c930 0x7f7894070900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33988/0 (socket says 172.21.15.40:33988) 2023-11-08T08:21:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.324+0000 7f7898fee700 1 -- 172.21.15.40:0/2430018726 learned_addr learned my addr 172.21.15.40:0/2430018726 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.324+0000 7f7893fff700 1 --2- 172.21.15.40:0/2430018726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7894158e50 0x7f78940c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.324+0000 7f788b7fe700 1 -- 172.21.15.40:0/2430018726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7894158e50 msgr2=0x7f78940c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.324+0000 7f788b7fe700 1 --2- 172.21.15.40:0/2430018726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7894158e50 0x7f78940c9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.324+0000 7f788b7fe700 1 -- 172.21.15.40:0/2430018726 >> v1:172.21.15.40:6789/0 conn(0x7f789406c930 legacy=0x7f7894070900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.324+0000 7f788b7fe700 1 -- 172.21.15.40:0/2430018726 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789415d470 con 0x7f7894069bf0 2023-11-08T08:21:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.324+0000 7f788b7fe700 1 --2- 172.21.15.40:0/2430018726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7894069bf0 0x7f789406c3f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f787c009850 tx=0x7f787c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdcf8c2412bde5e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:22.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.324+0000 7f78937fe700 1 -- 172.21.15.40:0/2430018726 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c00f8a0 con 0x7f7894069bf0 2023-11-08T08:21:22.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.324+0000 7f78937fe700 1 -- 172.21.15.40:0/2430018726 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787c00fa00 con 0x7f7894069bf0 2023-11-08T08:21:22.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.324+0000 7f78937fe700 1 -- 172.21.15.40:0/2430018726 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c00fdb0 con 0x7f7894069bf0 2023-11-08T08:21:22.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.328+0000 7f78997ef700 1 -- 172.21.15.40:0/2430018726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7894069bf0 msgr2=0x7f789406c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:22.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.328+0000 7f78997ef700 1 --2- 172.21.15.40:0/2430018726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7894069bf0 0x7f789406c3f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f787c009850 tx=0x7f787c009b60 comp rx=0 tx=0).stop 2023-11-08T08:21:22.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.328+0000 7f78997ef700 1 -- 172.21.15.40:0/2430018726 shutdown_connections 2023-11-08T08:21:22.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.328+0000 7f78997ef700 1 --2- 172.21.15.40:0/2430018726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7894069bf0 0x7f789406c3f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:22.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.328+0000 7f78997ef700 1 --2- 172.21.15.40:0/2430018726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7894158e50 0x7f78940c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:22.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.328+0000 7f78997ef700 1 -- 172.21.15.40:0/2430018726 >> 172.21.15.40:0/2430018726 conn(0x7f78940b0a20 msgr2=0x7f78940b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:22.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.328+0000 7f78997ef700 1 -- 172.21.15.40:0/2430018726 shutdown_connections 2023-11-08T08:21:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.328+0000 7f78997ef700 1 -- 172.21.15.40:0/2430018726 wait complete. 2023-11-08T08:21:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.328+0000 7f78997ef700 1 Processor -- start 2023-11-08T08:21:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.328+0000 7f78997ef700 1 -- start start 2023-11-08T08:21:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f78997ef700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7894069bf0 0x7f7894156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f78997ef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f789406c930 0x7f78941571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f78997ef700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7894158e50 0x7f7894154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f78997ef700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78941565d0 con 0x7f789406c930 2023-11-08T08:21:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f78997ef700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7894156010 con 0x7f7894158e50 2023-11-08T08:21:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f78997ef700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78941562f0 con 0x7f7894069bf0 2023-11-08T08:21:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f788b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f789406c930 0x7f78941571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f788b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f789406c930 0x7f78941571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51496/0 (socket says 172.21.15.40:51496) 2023-11-08T08:21:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f788b7fe700 1 -- 172.21.15.40:0/2383614330 learned_addr learned my addr 172.21.15.40:0/2383614330 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:22.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f7893fff700 1 --2- 172.21.15.40:0/2383614330 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7894069bf0 0x7f7894156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:22.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f7898fee700 1 --2- 172.21.15.40:0/2383614330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7894158e50 0x7f7894154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:22.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f788b7fe700 1 -- 172.21.15.40:0/2383614330 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7894069bf0 msgr2=0x7f7894156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f788b7fe700 1 --2- 172.21.15.40:0/2383614330 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7894069bf0 0x7f7894156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f788b7fe700 1 -- 172.21.15.40:0/2383614330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7894158e50 msgr2=0x7f7894154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f788b7fe700 1 --2- 172.21.15.40:0/2383614330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7894158e50 0x7f7894154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f788b7fe700 1 -- 172.21.15.40:0/2383614330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c0094b0 con 0x7f789406c930 2023-11-08T08:21:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f788b7fe700 1 --2- 172.21.15.40:0/2383614330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f789406c930 0x7f78941571b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f787c0090b0 tx=0x7f787c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:22.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f78917fa700 1 -- 172.21.15.40:0/2383614330 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c0038c0 con 0x7f789406c930 2023-11-08T08:21:22.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f78917fa700 1 -- 172.21.15.40:0/2383614330 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787c003a40 con 0x7f789406c930 2023-11-08T08:21:22.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f78997ef700 1 -- 172.21.15.40:0/2383614330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789406a040 con 0x7f789406c930 2023-11-08T08:21:22.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f78917fa700 1 -- 172.21.15.40:0/2383614330 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c003df0 con 0x7f789406c930 2023-11-08T08:21:22.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.332+0000 7f78997ef700 1 -- 172.21.15.40:0/2383614330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789415d750 con 0x7f789406c930 2023-11-08T08:21:22.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.336+0000 7f78997ef700 1 -- 172.21.15.40:0/2383614330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78940ccf10 con 0x7f789406c930 2023-11-08T08:21:22.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.336+0000 7f78917fa700 1 -- 172.21.15.40:0/2383614330 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f787c019400 con 0x7f789406c930 2023-11-08T08:21:22.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.340+0000 7f78917fa700 1 --2- 172.21.15.40:0/2383614330 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7874064ee0 0x7f7874067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:22.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.340+0000 7f78917fa700 1 -- 172.21.15.40:0/2383614330 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f787c082fb0 con 0x7f789406c930 2023-11-08T08:21:22.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.340+0000 7f7893fff700 1 --2- 172.21.15.40:0/2383614330 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7874064ee0 0x7f7874067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:22.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.340+0000 7f7893fff700 1 --2- 172.21.15.40:0/2383614330 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7874064ee0 0x7f7874067390 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f78940cab60 tx=0x7f788400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:22.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.344+0000 7f78917fa700 1 -- 172.21.15.40:0/2383614330 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f787c050ec0 con 0x7f789406c930 2023-11-08T08:21:22.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.520+0000 7f78997ef700 1 -- 172.21.15.40:0/2383614330 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7894073cb0 con 0x7f7874064ee0 2023-11-08T08:21:22.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.536+0000 7f78917fa700 1 -- 172.21.15.40:0/2383614330 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308752 (secure 0 0 0) 0x7f7894073cb0 con 0x7f7874064ee0 2023-11-08T08:21:22.546 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:22.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.544+0000 7f78997ef700 1 -- 172.21.15.40:0/2383614330 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7874064ee0 msgr2=0x7f7874067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:22.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.544+0000 7f78997ef700 1 --2- 172.21.15.40:0/2383614330 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7874064ee0 0x7f7874067390 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f78940cab60 tx=0x7f788400b040 comp rx=0 tx=0).stop 2023-11-08T08:21:22.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.544+0000 7f78997ef700 1 -- 172.21.15.40:0/2383614330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f789406c930 msgr2=0x7f78941571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:22.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.544+0000 7f78997ef700 1 --2- 172.21.15.40:0/2383614330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f789406c930 0x7f78941571b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f787c0090b0 tx=0x7f787c004770 comp rx=0 tx=0).stop 2023-11-08T08:21:22.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.544+0000 7f78997ef700 1 -- 172.21.15.40:0/2383614330 shutdown_connections 2023-11-08T08:21:22.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.544+0000 7f78997ef700 1 --2- 172.21.15.40:0/2383614330 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7874064ee0 0x7f7874067390 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:22.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.544+0000 7f78997ef700 1 --2- 172.21.15.40:0/2383614330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7894158e50 0x7f7894154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:22.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.544+0000 7f78997ef700 1 --2- 172.21.15.40:0/2383614330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f789406c930 0x7f78941571b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:22.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.544+0000 7f78997ef700 1 --2- 172.21.15.40:0/2383614330 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7894069bf0 0x7f7894156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:22.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.544+0000 7f78997ef700 1 -- 172.21.15.40:0/2383614330 >> 172.21.15.40:0/2383614330 conn(0x7f78940b0a20 msgr2=0x7f78940cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:22.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.544+0000 7f78997ef700 1 -- 172.21.15.40:0/2383614330 shutdown_connections 2023-11-08T08:21:22.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.544+0000 7f78997ef700 1 -- 172.21.15.40:0/2383614330 wait complete. 2023-11-08T08:21:22.555 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-11-08T08:21:22.575 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2023-11-08T08:21:21.932804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":76680,"kb_used_data":6536,"kb_used_omap":0,"kb_used_meta":70080,"kb_avail":754898040,"statfs":{"total":773094113280,"available":773015592960,"internally_reserved":0,"allocated":6692864,"data_stored":3695060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":71761920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.674823"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.754199+0000","last_change":"2023-11-08T08:21:17.774658+0000","last_active":"2023-11-08T08:21:18.754199+0000","last_peered":"2023-11-08T08:21:18.754199+0000","last_clean":"2023-11-08T08:21:18.754199+0000","last_became_active":"2023-11-08T08:21:17.774405+0000","last_became_peered":"2023-11-08T08:21:17.774405+0000","last_unstale":"2023-11-08T08:21:18.754199+0000","last_undegraded":"2023-11-08T08:21:18.754199+0000","last_fullsized":"2023-11-08T08:21:18.754199+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:50:53.616882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809742+0000","last_change":"2023-11-08T08:21:17.778052+0000","last_active":"2023-11-08T08:21:18.809742+0000","last_peered":"2023-11-08T08:21:18.809742+0000","last_clean":"2023-11-08T08:21:18.809742+0000","last_became_active":"2023-11-08T08:21:17.777801+0000","last_became_peered":"2023-11-08T08:21:17.777801+0000","last_unstale":"2023-11-08T08:21:18.809742+0000","last_undegraded":"2023-11-08T08:21:18.809742+0000","last_fullsized":"2023-11-08T08:21:18.809742+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:02:46.560811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813846+0000","last_change":"2023-11-08T08:21:17.775681+0000","last_active":"2023-11-08T08:21:18.813846+0000","last_peered":"2023-11-08T08:21:18.813846+0000","last_clean":"2023-11-08T08:21:18.813846+0000","last_became_active":"2023-11-08T08:21:17.771243+0000","last_became_peered":"2023-11-08T08:21:17.771243+0000","last_unstale":"2023-11-08T08:21:18.813846+0000","last_undegraded":"2023-11-08T08:21:18.813846+0000","last_fullsized":"2023-11-08T08:21:18.813846+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:50:29.721556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812505+0000","last_change":"2023-11-08T08:21:17.771572+0000","last_active":"2023-11-08T08:21:18.812505+0000","last_peered":"2023-11-08T08:21:18.812505+0000","last_clean":"2023-11-08T08:21:18.812505+0000","last_became_active":"2023-11-08T08:21:17.771307+0000","last_became_peered":"2023-11-08T08:21:17.771307+0000","last_unstale":"2023-11-08T08:21:18.812505+0000","last_undegraded":"2023-11-08T08:21:18.812505+0000","last_fullsized":"2023-11-08T08:21:18.812505+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:03:09.691918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753856+0000","last_change":"2023-11-08T08:21:17.772356+0000","last_active":"2023-11-08T08:21:18.753856+0000","last_peered":"2023-11-08T08:21:18.753856+0000","last_clean":"2023-11-08T08:21:18.753856+0000","last_became_active":"2023-11-08T08:21:17.771165+0000","last_became_peered":"2023-11-08T08:21:17.771165+0000","last_unstale":"2023-11-08T08:21:18.753856+0000","last_undegraded":"2023-11-08T08:21:18.753856+0000","last_fullsized":"2023-11-08T08:21:18.753856+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:59:38.074076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.752732+0000","last_change":"2023-11-08T08:21:17.775859+0000","last_active":"2023-11-08T08:21:18.752732+0000","last_peered":"2023-11-08T08:21:18.752732+0000","last_clean":"2023-11-08T08:21:18.752732+0000","last_became_active":"2023-11-08T08:21:17.775611+0000","last_became_peered":"2023-11-08T08:21:17.775611+0000","last_unstale":"2023-11-08T08:21:18.752732+0000","last_undegraded":"2023-11-08T08:21:18.752732+0000","last_fullsized":"2023-11-08T08:21:18.752732+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:53:38.208291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.754987+0000","last_change":"2023-11-08T08:21:17.776518+0000","last_active":"2023-11-08T08:21:18.754987+0000","last_peered":"2023-11-08T08:21:18.754987+0000","last_clean":"2023-11-08T08:21:18.754987+0000","last_became_active":"2023-11-08T08:21:17.776229+0000","last_became_peered":"2023-11-08T08:21:17.776229+0000","last_unstale":"2023-11-08T08:21:18.754987+0000","last_undegraded":"2023-11-08T08:21:18.754987+0000","last_fullsized":"2023-11-08T08:21:18.754987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:19:00.004764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809201+0000","last_change":"2023-11-08T08:21:17.765200+0000","last_active":"2023-11-08T08:21:18.809201+0000","last_peered":"2023-11-08T08:21:18.809201+0000","last_clean":"2023-11-08T08:21:18.809201+0000","last_became_active":"2023-11-08T08:21:17.764941+0000","last_became_peered":"2023-11-08T08:21:17.764941+0000","last_unstale":"2023-11-08T08:21:18.809201+0000","last_undegraded":"2023-11-08T08:21:18.809201+0000","last_fullsized":"2023-11-08T08:21:18.809201+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:39:20.550673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.755114+0000","last_change":"2023-11-08T08:21:17.765151+0000","last_active":"2023-11-08T08:21:18.755114+0000","last_peered":"2023-11-08T08:21:18.755114+0000","last_clean":"2023-11-08T08:21:18.755114+0000","last_became_active":"2023-11-08T08:21:17.762399+0000","last_became_peered":"2023-11-08T08:21:17.762399+0000","last_unstale":"2023-11-08T08:21:18.755114+0000","last_undegraded":"2023-11-08T08:21:18.755114+0000","last_fullsized":"2023-11-08T08:21:18.755114+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:54:22.302215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.808407+0000","last_change":"2023-11-08T08:21:17.773195+0000","last_active":"2023-11-08T08:21:18.808407+0000","last_peered":"2023-11-08T08:21:18.808407+0000","last_clean":"2023-11-08T08:21:18.808407+0000","last_became_active":"2023-11-08T08:21:17.772925+0000","last_became_peered":"2023-11-08T08:21:17.772925+0000","last_unstale":"2023-11-08T08:21:18.808407+0000","last_undegraded":"2023-11-08T08:21:18.808407+0000","last_fullsized":"2023-11-08T08:21:18.808407+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:32:15.406882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.808110+0000","last_change":"2023-11-08T08:21:17.773953+0000","last_active":"2023-11-08T08:21:18.808110+0000","last_peered":"2023-11-08T08:21:18.808110+0000","last_clean":"2023-11-08T08:21:18.808110+0000","last_became_active":"2023-11-08T08:21:17.773694+0000","last_became_peered":"2023-11-08T08:21:17.773694+0000","last_unstale":"2023-11-08T08:21:18.808110+0000","last_undegraded":"2023-11-08T08:21:18.808110+0000","last_fullsized":"2023-11-08T08:21:18.808110+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:27:03.140038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812189+0000","last_change":"2023-11-08T08:21:17.770468+0000","last_active":"2023-11-08T08:21:18.812189+0000","last_peered":"2023-11-08T08:21:18.812189+0000","last_clean":"2023-11-08T08:21:18.812189+0000","last_became_active":"2023-11-08T08:21:17.767802+0000","last_became_peered":"2023-11-08T08:21:17.767802+0000","last_unstale":"2023-11-08T08:21:18.812189+0000","last_undegraded":"2023-11-08T08:21:18.812189+0000","last_fullsized":"2023-11-08T08:21:18.812189+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:16:37.763141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813638+0000","last_change":"2023-11-08T08:21:17.756611+0000","last_active":"2023-11-08T08:21:18.813638+0000","last_peered":"2023-11-08T08:21:18.813638+0000","last_clean":"2023-11-08T08:21:18.813638+0000","last_became_active":"2023-11-08T08:21:17.756239+0000","last_became_peered":"2023-11-08T08:21:17.756239+0000","last_unstale":"2023-11-08T08:21:18.813638+0000","last_undegraded":"2023-11-08T08:21:18.813638+0000","last_fullsized":"2023-11-08T08:21:18.813638+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:14:54.711453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.755105+0000","last_change":"2023-11-08T08:21:17.766210+0000","last_active":"2023-11-08T08:21:18.755105+0000","last_peered":"2023-11-08T08:21:18.755105+0000","last_clean":"2023-11-08T08:21:18.755105+0000","last_became_active":"2023-11-08T08:21:17.765915+0000","last_became_peered":"2023-11-08T08:21:17.765915+0000","last_unstale":"2023-11-08T08:21:18.755105+0000","last_undegraded":"2023-11-08T08:21:18.755105+0000","last_fullsized":"2023-11-08T08:21:18.755105+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:53:39.019269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.755144+0000","last_change":"2023-11-08T08:21:17.773028+0000","last_active":"2023-11-08T08:21:18.755144+0000","last_peered":"2023-11-08T08:21:18.755144+0000","last_clean":"2023-11-08T08:21:18.755144+0000","last_became_active":"2023-11-08T08:21:17.772766+0000","last_became_peered":"2023-11-08T08:21:17.772766+0000","last_unstale":"2023-11-08T08:21:18.755144+0000","last_undegraded":"2023-11-08T08:21:18.755144+0000","last_fullsized":"2023-11-08T08:21:18.755144+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:02:19.326469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.752244+0000","last_change":"2023-11-08T08:21:17.761867+0000","last_active":"2023-11-08T08:21:18.752244+0000","last_peered":"2023-11-08T08:21:18.752244+0000","last_clean":"2023-11-08T08:21:18.752244+0000","last_became_active":"2023-11-08T08:21:17.761389+0000","last_became_peered":"2023-11-08T08:21:17.761389+0000","last_unstale":"2023-11-08T08:21:18.752244+0000","last_undegraded":"2023-11-08T08:21:18.752244+0000","last_fullsized":"2023-11-08T08:21:18.752244+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:02:46.560811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.810297+0000","last_change":"2023-11-08T08:21:17.778277+0000","last_active":"2023-11-08T08:21:18.810297+0000","last_peered":"2023-11-08T08:21:18.810297+0000","last_clean":"2023-11-08T08:21:18.810297+0000","last_became_active":"2023-11-08T08:21:17.777595+0000","last_became_peered":"2023-11-08T08:21:17.777595+0000","last_unstale":"2023-11-08T08:21:18.810297+0000","last_undegraded":"2023-11-08T08:21:18.810297+0000","last_fullsized":"2023-11-08T08:21:18.810297+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:18:18.548903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.814453+0000","last_change":"2023-11-08T08:21:17.776225+0000","last_active":"2023-11-08T08:21:18.814453+0000","last_peered":"2023-11-08T08:21:18.814453+0000","last_clean":"2023-11-08T08:21:18.814453+0000","last_became_active":"2023-11-08T08:21:17.775934+0000","last_became_peered":"2023-11-08T08:21:17.775934+0000","last_unstale":"2023-11-08T08:21:18.814453+0000","last_undegraded":"2023-11-08T08:21:18.814453+0000","last_fullsized":"2023-11-08T08:21:18.814453+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:33:04.463667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753962+0000","last_change":"2023-11-08T08:21:17.774429+0000","last_active":"2023-11-08T08:21:18.753962+0000","last_peered":"2023-11-08T08:21:18.753962+0000","last_clean":"2023-11-08T08:21:18.753962+0000","last_became_active":"2023-11-08T08:21:17.774122+0000","last_became_peered":"2023-11-08T08:21:17.774122+0000","last_unstale":"2023-11-08T08:21:18.753962+0000","last_undegraded":"2023-11-08T08:21:18.753962+0000","last_fullsized":"2023-11-08T08:21:18.753962+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:40:31.145969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809814+0000","last_change":"2023-11-08T08:21:17.783562+0000","last_active":"2023-11-08T08:21:18.809814+0000","last_peered":"2023-11-08T08:21:18.809814+0000","last_clean":"2023-11-08T08:21:18.809814+0000","last_became_active":"2023-11-08T08:21:17.783251+0000","last_became_peered":"2023-11-08T08:21:17.783251+0000","last_unstale":"2023-11-08T08:21:18.809814+0000","last_undegraded":"2023-11-08T08:21:18.809814+0000","last_fullsized":"2023-11-08T08:21:18.809814+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:03:09.691918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813300+0000","last_change":"2023-11-08T08:21:17.784886+0000","last_active":"2023-11-08T08:21:18.813300+0000","last_peered":"2023-11-08T08:21:18.813300+0000","last_clean":"2023-11-08T08:21:18.813300+0000","last_became_active":"2023-11-08T08:21:17.783682+0000","last_became_peered":"2023-11-08T08:21:17.783682+0000","last_unstale":"2023-11-08T08:21:18.813300+0000","last_undegraded":"2023-11-08T08:21:18.813300+0000","last_fullsized":"2023-11-08T08:21:18.813300+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:05:45.264423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809545+0000","last_change":"2023-11-08T08:21:17.776227+0000","last_active":"2023-11-08T08:21:18.809545+0000","last_peered":"2023-11-08T08:21:18.809545+0000","last_clean":"2023-11-08T08:21:18.809545+0000","last_became_active":"2023-11-08T08:21:17.775776+0000","last_became_peered":"2023-11-08T08:21:17.775776+0000","last_unstale":"2023-11-08T08:21:18.809545+0000","last_undegraded":"2023-11-08T08:21:18.809545+0000","last_fullsized":"2023-11-08T08:21:18.809545+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:49:36.507723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812525+0000","last_change":"2023-11-08T08:21:17.768621+0000","last_active":"2023-11-08T08:21:18.812525+0000","last_peered":"2023-11-08T08:21:18.812525+0000","last_clean":"2023-11-08T08:21:18.812525+0000","last_became_active":"2023-11-08T08:21:17.767297+0000","last_became_peered":"2023-11-08T08:21:17.767297+0000","last_unstale":"2023-11-08T08:21:18.812525+0000","last_undegraded":"2023-11-08T08:21:18.812525+0000","last_fullsized":"2023-11-08T08:21:18.812525+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:22:02.813495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.811712+0000","last_change":"2023-11-08T08:21:17.760411+0000","last_active":"2023-11-08T08:21:18.811712+0000","last_peered":"2023-11-08T08:21:18.811712+0000","last_clean":"2023-11-08T08:21:18.811712+0000","last_became_active":"2023-11-08T08:21:17.760120+0000","last_became_peered":"2023-11-08T08:21:17.760120+0000","last_unstale":"2023-11-08T08:21:18.811712+0000","last_undegraded":"2023-11-08T08:21:18.811712+0000","last_fullsized":"2023-11-08T08:21:18.811712+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:32:15.406882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.754202+0000","last_change":"2023-11-08T08:21:17.761461+0000","last_active":"2023-11-08T08:21:18.754202+0000","last_peered":"2023-11-08T08:21:18.754202+0000","last_clean":"2023-11-08T08:21:18.754202+0000","last_became_active":"2023-11-08T08:21:17.761185+0000","last_became_peered":"2023-11-08T08:21:17.761185+0000","last_unstale":"2023-11-08T08:21:18.754202+0000","last_undegraded":"2023-11-08T08:21:18.754202+0000","last_fullsized":"2023-11-08T08:21:18.754202+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:50:54.435794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813044+0000","last_change":"2023-11-08T08:21:17.777439+0000","last_active":"2023-11-08T08:21:18.813044+0000","last_peered":"2023-11-08T08:21:18.813044+0000","last_clean":"2023-11-08T08:21:18.813044+0000","last_became_active":"2023-11-08T08:21:17.777120+0000","last_became_peered":"2023-11-08T08:21:17.777120+0000","last_unstale":"2023-11-08T08:21:18.813044+0000","last_undegraded":"2023-11-08T08:21:18.813044+0000","last_fullsized":"2023-11-08T08:21:18.813044+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:22:02.813495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753064+0000","last_change":"2023-11-08T08:21:17.761393+0000","last_active":"2023-11-08T08:21:18.753064+0000","last_peered":"2023-11-08T08:21:18.753064+0000","last_clean":"2023-11-08T08:21:18.753064+0000","last_became_active":"2023-11-08T08:21:17.761079+0000","last_became_peered":"2023-11-08T08:21:17.761079+0000","last_unstale":"2023-11-08T08:21:18.753064+0000","last_undegraded":"2023-11-08T08:21:18.753064+0000","last_fullsized":"2023-11-08T08:21:18.753064+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:30:19.324907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.754046+0000","last_change":"2023-11-08T08:21:17.772056+0000","last_active":"2023-11-08T08:21:18.754046+0000","last_peered":"2023-11-08T08:21:18.754046+0000","last_clean":"2023-11-08T08:21:18.754046+0000","last_became_active":"2023-11-08T08:21:17.771820+0000","last_became_peered":"2023-11-08T08:21:17.771820+0000","last_unstale":"2023-11-08T08:21:18.754046+0000","last_undegraded":"2023-11-08T08:21:18.754046+0000","last_fullsized":"2023-11-08T08:21:18.754046+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:10:09.581328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809686+0000","last_change":"2023-11-08T08:21:17.785604+0000","last_active":"2023-11-08T08:21:18.809686+0000","last_peered":"2023-11-08T08:21:18.809686+0000","last_clean":"2023-11-08T08:21:18.809686+0000","last_became_active":"2023-11-08T08:21:17.785324+0000","last_became_peered":"2023-11-08T08:21:17.785324+0000","last_unstale":"2023-11-08T08:21:18.809686+0000","last_undegraded":"2023-11-08T08:21:18.809686+0000","last_fullsized":"2023-11-08T08:21:18.809686+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:30:19.324907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813151+0000","last_change":"2023-11-08T08:21:17.771048+0000","last_active":"2023-11-08T08:21:18.813151+0000","last_peered":"2023-11-08T08:21:18.813151+0000","last_clean":"2023-11-08T08:21:18.813151+0000","last_became_active":"2023-11-08T08:21:17.770768+0000","last_became_peered":"2023-11-08T08:21:17.770768+0000","last_unstale":"2023-11-08T08:21:18.813151+0000","last_undegraded":"2023-11-08T08:21:18.813151+0000","last_fullsized":"2023-11-08T08:21:18.813151+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:15:05.006165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753390+0000","last_change":"2023-11-08T08:21:17.762157+0000","last_active":"2023-11-08T08:21:18.753390+0000","last_peered":"2023-11-08T08:21:18.753390+0000","last_clean":"2023-11-08T08:21:18.753390+0000","last_became_active":"2023-11-08T08:21:17.761876+0000","last_became_peered":"2023-11-08T08:21:17.761876+0000","last_unstale":"2023-11-08T08:21:18.753390+0000","last_undegraded":"2023-11-08T08:21:18.753390+0000","last_fullsized":"2023-11-08T08:21:18.753390+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:27:40.888088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.752241+0000","last_change":"2023-11-08T08:21:17.774106+0000","last_active":"2023-11-08T08:21:18.752241+0000","last_peered":"2023-11-08T08:21:18.752241+0000","last_clean":"2023-11-08T08:21:18.752241+0000","last_became_active":"2023-11-08T08:21:17.773626+0000","last_became_peered":"2023-11-08T08:21:17.773626+0000","last_unstale":"2023-11-08T08:21:18.752241+0000","last_undegraded":"2023-11-08T08:21:18.752241+0000","last_fullsized":"2023-11-08T08:21:18.752241+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:40:39.325330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753445+0000","last_change":"2023-11-08T08:21:17.766739+0000","last_active":"2023-11-08T08:21:18.753445+0000","last_peered":"2023-11-08T08:21:18.753445+0000","last_clean":"2023-11-08T08:21:18.753445+0000","last_became_active":"2023-11-08T08:21:17.762207+0000","last_became_peered":"2023-11-08T08:21:17.762207+0000","last_unstale":"2023-11-08T08:21:18.753445+0000","last_undegraded":"2023-11-08T08:21:18.753445+0000","last_fullsized":"2023-11-08T08:21:18.753445+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:54:22.302215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753511+0000","last_change":"2023-11-08T08:21:17.284548+0000","last_active":"2023-11-08T08:21:18.753511+0000","last_peered":"2023-11-08T08:21:18.753511+0000","last_clean":"2023-11-08T08:21:18.753511+0000","last_became_active":"2023-11-08T08:21:17.284197+0000","last_became_peered":"2023-11-08T08:21:17.284197+0000","last_unstale":"2023-11-08T08:21:18.753511+0000","last_undegraded":"2023-11-08T08:21:18.753511+0000","last_fullsized":"2023-11-08T08:21:18.753511+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:22:01.799995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.808679+0000","last_change":"2023-11-08T08:21:17.763462+0000","last_active":"2023-11-08T08:21:18.808679+0000","last_peered":"2023-11-08T08:21:18.808679+0000","last_clean":"2023-11-08T08:21:18.808679+0000","last_became_active":"2023-11-08T08:21:17.763150+0000","last_became_peered":"2023-11-08T08:21:17.763150+0000","last_unstale":"2023-11-08T08:21:18.808679+0000","last_undegraded":"2023-11-08T08:21:18.808679+0000","last_fullsized":"2023-11-08T08:21:18.808679+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:14:54.711453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"56'1","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809967+0000","last_change":"2023-11-08T08:21:17.287812+0000","last_active":"2023-11-08T08:21:18.809967+0000","last_peered":"2023-11-08T08:21:18.809967+0000","last_clean":"2023-11-08T08:21:18.809967+0000","last_became_active":"2023-11-08T08:21:17.287547+0000","last_became_peered":"2023-11-08T08:21:17.287547+0000","last_unstale":"2023-11-08T08:21:18.809967+0000","last_undegraded":"2023-11-08T08:21:18.809967+0000","last_fullsized":"2023-11-08T08:21:18.809967+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:44:40.357032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753059+0000","last_change":"2023-11-08T08:21:17.762739+0000","last_active":"2023-11-08T08:21:18.753059+0000","last_peered":"2023-11-08T08:21:18.753059+0000","last_clean":"2023-11-08T08:21:18.753059+0000","last_became_active":"2023-11-08T08:21:17.762471+0000","last_became_peered":"2023-11-08T08:21:17.762471+0000","last_unstale":"2023-11-08T08:21:18.753059+0000","last_undegraded":"2023-11-08T08:21:18.753059+0000","last_fullsized":"2023-11-08T08:21:18.753059+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:18:18.548903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753275+0000","last_change":"2023-11-08T08:21:17.281557+0000","last_active":"2023-11-08T08:21:18.753275+0000","last_peered":"2023-11-08T08:21:18.753275+0000","last_clean":"2023-11-08T08:21:18.753275+0000","last_became_active":"2023-11-08T08:21:17.281265+0000","last_became_peered":"2023-11-08T08:21:17.281265+0000","last_unstale":"2023-11-08T08:21:18.753275+0000","last_undegraded":"2023-11-08T08:21:18.753275+0000","last_fullsized":"2023-11-08T08:21:18.753275+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:00:35.071027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753380+0000","last_change":"2023-11-08T08:21:17.777307+0000","last_active":"2023-11-08T08:21:18.753380+0000","last_peered":"2023-11-08T08:21:18.753380+0000","last_clean":"2023-11-08T08:21:18.753380+0000","last_became_active":"2023-11-08T08:21:17.777037+0000","last_became_peered":"2023-11-08T08:21:17.777037+0000","last_unstale":"2023-11-08T08:21:18.753380+0000","last_undegraded":"2023-11-08T08:21:18.753380+0000","last_fullsized":"2023-11-08T08:21:18.753380+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:39:20.550673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"56'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812321+0000","last_change":"2023-11-08T08:21:17.290426+0000","last_active":"2023-11-08T08:21:18.812321+0000","last_peered":"2023-11-08T08:21:18.812321+0000","last_clean":"2023-11-08T08:21:18.812321+0000","last_became_active":"2023-11-08T08:21:17.290113+0000","last_became_peered":"2023-11-08T08:21:17.290113+0000","last_unstale":"2023-11-08T08:21:18.812321+0000","last_undegraded":"2023-11-08T08:21:18.812321+0000","last_fullsized":"2023-11-08T08:21:18.812321+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:46.050979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809816+0000","last_change":"2023-11-08T08:21:17.776014+0000","last_active":"2023-11-08T08:21:18.809816+0000","last_peered":"2023-11-08T08:21:18.809816+0000","last_clean":"2023-11-08T08:21:18.809816+0000","last_became_active":"2023-11-08T08:21:17.775433+0000","last_became_peered":"2023-11-08T08:21:17.775433+0000","last_unstale":"2023-11-08T08:21:18.809816+0000","last_undegraded":"2023-11-08T08:21:18.809816+0000","last_fullsized":"2023-11-08T08:21:18.809816+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:07:42.164520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.755251+0000","last_change":"2023-11-08T08:21:17.302570+0000","last_active":"2023-11-08T08:21:18.755251+0000","last_peered":"2023-11-08T08:21:18.755251+0000","last_clean":"2023-11-08T08:21:18.755251+0000","last_became_active":"2023-11-08T08:21:17.302288+0000","last_became_peered":"2023-11-08T08:21:17.302288+0000","last_unstale":"2023-11-08T08:21:18.755251+0000","last_undegraded":"2023-11-08T08:21:18.755251+0000","last_fullsized":"2023-11-08T08:21:18.755251+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:00:15.086010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809501+0000","last_change":"2023-11-08T08:21:17.784037+0000","last_active":"2023-11-08T08:21:18.809501+0000","last_peered":"2023-11-08T08:21:18.809501+0000","last_clean":"2023-11-08T08:21:18.809501+0000","last_became_active":"2023-11-08T08:21:17.779861+0000","last_became_peered":"2023-11-08T08:21:17.779861+0000","last_unstale":"2023-11-08T08:21:18.809501+0000","last_undegraded":"2023-11-08T08:21:18.809501+0000","last_fullsized":"2023-11-08T08:21:18.809501+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:53:38.208291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.752714+0000","last_change":"2023-11-08T08:21:17.289310+0000","last_active":"2023-11-08T08:21:18.752714+0000","last_peered":"2023-11-08T08:21:18.752714+0000","last_clean":"2023-11-08T08:21:18.752714+0000","last_became_active":"2023-11-08T08:21:17.289009+0000","last_became_peered":"2023-11-08T08:21:17.289009+0000","last_unstale":"2023-11-08T08:21:18.752714+0000","last_undegraded":"2023-11-08T08:21:18.752714+0000","last_fullsized":"2023-11-08T08:21:18.752714+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:00:15.086010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812775+0000","last_change":"2023-11-08T08:21:17.767414+0000","last_active":"2023-11-08T08:21:18.812775+0000","last_peered":"2023-11-08T08:21:18.812775+0000","last_clean":"2023-11-08T08:21:18.812775+0000","last_became_active":"2023-11-08T08:21:17.766859+0000","last_became_peered":"2023-11-08T08:21:17.766859+0000","last_unstale":"2023-11-08T08:21:18.812775+0000","last_undegraded":"2023-11-08T08:21:18.812775+0000","last_fullsized":"2023-11-08T08:21:18.812775+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:27:03.140038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812621+0000","last_change":"2023-11-08T08:21:17.290817+0000","last_active":"2023-11-08T08:21:18.812621+0000","last_peered":"2023-11-08T08:21:18.812621+0000","last_clean":"2023-11-08T08:21:18.812621+0000","last_became_active":"2023-11-08T08:21:17.290538+0000","last_became_peered":"2023-11-08T08:21:17.290538+0000","last_unstale":"2023-11-08T08:21:18.812621+0000","last_undegraded":"2023-11-08T08:21:18.812621+0000","last_fullsized":"2023-11-08T08:21:18.812621+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:09:29.890451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809539+0000","last_change":"2023-11-08T08:21:17.773741+0000","last_active":"2023-11-08T08:21:18.809539+0000","last_peered":"2023-11-08T08:21:18.809539+0000","last_clean":"2023-11-08T08:21:18.809539+0000","last_became_active":"2023-11-08T08:21:17.773489+0000","last_became_peered":"2023-11-08T08:21:17.773489+0000","last_unstale":"2023-11-08T08:21:18.809539+0000","last_undegraded":"2023-11-08T08:21:18.809539+0000","last_fullsized":"2023-11-08T08:21:18.809539+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:28:38.025842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.754733+0000","last_change":"2023-11-08T08:21:17.292281+0000","last_active":"2023-11-08T08:21:18.754733+0000","last_peered":"2023-11-08T08:21:18.754733+0000","last_clean":"2023-11-08T08:21:18.754733+0000","last_became_active":"2023-11-08T08:21:17.291983+0000","last_became_peered":"2023-11-08T08:21:17.291983+0000","last_unstale":"2023-11-08T08:21:18.754733+0000","last_undegraded":"2023-11-08T08:21:18.754733+0000","last_fullsized":"2023-11-08T08:21:18.754733+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:22:59.290723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.755350+0000","last_change":"2023-11-08T08:21:17.770902+0000","last_active":"2023-11-08T08:21:18.755350+0000","last_peered":"2023-11-08T08:21:18.755350+0000","last_clean":"2023-11-08T08:21:18.755350+0000","last_became_active":"2023-11-08T08:21:17.769289+0000","last_became_peered":"2023-11-08T08:21:17.769289+0000","last_unstale":"2023-11-08T08:21:18.755350+0000","last_undegraded":"2023-11-08T08:21:18.755350+0000","last_fullsized":"2023-11-08T08:21:18.755350+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:28:38.025842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.814593+0000","last_change":"2023-11-08T08:21:17.289374+0000","last_active":"2023-11-08T08:21:18.814593+0000","last_peered":"2023-11-08T08:21:18.814593+0000","last_clean":"2023-11-08T08:21:18.814593+0000","last_became_active":"2023-11-08T08:21:17.289090+0000","last_became_peered":"2023-11-08T08:21:17.289090+0000","last_unstale":"2023-11-08T08:21:18.814593+0000","last_undegraded":"2023-11-08T08:21:18.814593+0000","last_fullsized":"2023-11-08T08:21:18.814593+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:14:53.697954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.814512+0000","last_change":"2023-11-08T08:21:17.763076+0000","last_active":"2023-11-08T08:21:18.814512+0000","last_peered":"2023-11-08T08:21:18.814512+0000","last_clean":"2023-11-08T08:21:18.814512+0000","last_became_active":"2023-11-08T08:21:17.762785+0000","last_became_peered":"2023-11-08T08:21:17.762785+0000","last_unstale":"2023-11-08T08:21:18.814512+0000","last_undegraded":"2023-11-08T08:21:18.814512+0000","last_fullsized":"2023-11-08T08:21:18.814512+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:55:51.112712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.810260+0000","last_change":"2023-11-08T08:21:17.299611+0000","last_active":"2023-11-08T08:21:18.810260+0000","last_peered":"2023-11-08T08:21:18.810260+0000","last_clean":"2023-11-08T08:21:18.810260+0000","last_became_active":"2023-11-08T08:21:17.299303+0000","last_became_peered":"2023-11-08T08:21:17.299303+0000","last_unstale":"2023-11-08T08:21:18.810260+0000","last_undegraded":"2023-11-08T08:21:18.810260+0000","last_fullsized":"2023-11-08T08:21:18.810260+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:22:01.799995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.808195+0000","last_change":"2023-11-08T08:21:17.776807+0000","last_active":"2023-11-08T08:21:18.808195+0000","last_peered":"2023-11-08T08:21:18.808195+0000","last_clean":"2023-11-08T08:21:18.808195+0000","last_became_active":"2023-11-08T08:21:17.774585+0000","last_became_peered":"2023-11-08T08:21:17.774585+0000","last_unstale":"2023-11-08T08:21:18.808195+0000","last_undegraded":"2023-11-08T08:21:18.808195+0000","last_fullsized":"2023-11-08T08:21:18.808195+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:58:12.422909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"56'3","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.814223+0000","last_change":"2023-11-08T08:21:17.299245+0000","last_active":"2023-11-08T08:21:18.814223+0000","last_peered":"2023-11-08T08:21:18.814223+0000","last_clean":"2023-11-08T08:21:18.814223+0000","last_became_active":"2023-11-08T08:21:17.298868+0000","last_became_peered":"2023-11-08T08:21:17.298868+0000","last_unstale":"2023-11-08T08:21:18.814223+0000","last_undegraded":"2023-11-08T08:21:18.814223+0000","last_fullsized":"2023-11-08T08:21:18.814223+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:32:14.393382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813973+0000","last_change":"2023-11-08T08:21:17.777569+0000","last_active":"2023-11-08T08:21:18.813973+0000","last_peered":"2023-11-08T08:21:18.813973+0000","last_clean":"2023-11-08T08:21:18.813973+0000","last_became_active":"2023-11-08T08:21:17.777269+0000","last_became_peered":"2023-11-08T08:21:17.777269+0000","last_unstale":"2023-11-08T08:21:18.813973+0000","last_undegraded":"2023-11-08T08:21:18.813973+0000","last_fullsized":"2023-11-08T08:21:18.813973+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:12:08.899976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.755300+0000","last_change":"2023-11-08T08:21:17.293406+0000","last_active":"2023-11-08T08:21:18.755300+0000","last_peered":"2023-11-08T08:21:18.755300+0000","last_clean":"2023-11-08T08:21:18.755300+0000","last_became_active":"2023-11-08T08:21:17.292530+0000","last_became_peered":"2023-11-08T08:21:17.292530+0000","last_unstale":"2023-11-08T08:21:18.755300+0000","last_undegraded":"2023-11-08T08:21:18.755300+0000","last_fullsized":"2023-11-08T08:21:18.755300+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:22:01.799995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812524+0000","last_change":"2023-11-08T08:21:17.771134+0000","last_active":"2023-11-08T08:21:18.812524+0000","last_peered":"2023-11-08T08:21:18.812524+0000","last_clean":"2023-11-08T08:21:18.812524+0000","last_became_active":"2023-11-08T08:21:17.770882+0000","last_became_peered":"2023-11-08T08:21:17.770882+0000","last_unstale":"2023-11-08T08:21:18.812524+0000","last_undegraded":"2023-11-08T08:21:18.812524+0000","last_fullsized":"2023-11-08T08:21:18.812524+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:36:37.378597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"56'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809593+0000","last_change":"2023-11-08T08:21:17.302611+0000","last_active":"2023-11-08T08:21:18.809593+0000","last_peered":"2023-11-08T08:21:18.809593+0000","last_clean":"2023-11-08T08:21:18.809593+0000","last_became_active":"2023-11-08T08:21:17.302309+0000","last_became_peered":"2023-11-08T08:21:17.302309+0000","last_unstale":"2023-11-08T08:21:18.809593+0000","last_undegraded":"2023-11-08T08:21:18.809593+0000","last_fullsized":"2023-11-08T08:21:18.809593+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:00:35.071027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813457+0000","last_change":"2023-11-08T08:21:17.777423+0000","last_active":"2023-11-08T08:21:18.813457+0000","last_peered":"2023-11-08T08:21:18.813457+0000","last_clean":"2023-11-08T08:21:18.813457+0000","last_became_active":"2023-11-08T08:21:17.776987+0000","last_became_peered":"2023-11-08T08:21:17.776987+0000","last_unstale":"2023-11-08T08:21:18.813457+0000","last_undegraded":"2023-11-08T08:21:18.813457+0000","last_fullsized":"2023-11-08T08:21:18.813457+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:44:41.370532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813247+0000","last_change":"2023-11-08T08:21:17.288246+0000","last_active":"2023-11-08T08:21:18.813247+0000","last_peered":"2023-11-08T08:21:18.813247+0000","last_clean":"2023-11-08T08:21:18.813247+0000","last_became_active":"2023-11-08T08:21:17.287961+0000","last_became_peered":"2023-11-08T08:21:17.287961+0000","last_unstale":"2023-11-08T08:21:18.813247+0000","last_undegraded":"2023-11-08T08:21:18.813247+0000","last_fullsized":"2023-11-08T08:21:18.813247+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:52:49.330954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.754710+0000","last_change":"2023-11-08T08:21:17.770103+0000","last_active":"2023-11-08T08:21:18.754710+0000","last_peered":"2023-11-08T08:21:18.754710+0000","last_clean":"2023-11-08T08:21:18.754710+0000","last_became_active":"2023-11-08T08:21:17.769763+0000","last_became_peered":"2023-11-08T08:21:17.769763+0000","last_unstale":"2023-11-08T08:21:18.754710+0000","last_undegraded":"2023-11-08T08:21:18.754710+0000","last_fullsized":"2023-11-08T08:21:18.754710+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:30:04.897468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812793+0000","last_change":"2023-11-08T08:21:17.292207+0000","last_active":"2023-11-08T08:21:18.812793+0000","last_peered":"2023-11-08T08:21:18.812793+0000","last_clean":"2023-11-08T08:21:18.812793+0000","last_became_active":"2023-11-08T08:21:17.291975+0000","last_became_peered":"2023-11-08T08:21:17.291975+0000","last_unstale":"2023-11-08T08:21:18.812793+0000","last_undegraded":"2023-11-08T08:21:18.812793+0000","last_fullsized":"2023-11-08T08:21:18.812793+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:55:55.253541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812726+0000","last_change":"2023-11-08T08:21:17.767546+0000","last_active":"2023-11-08T08:21:18.812726+0000","last_peered":"2023-11-08T08:21:18.812726+0000","last_clean":"2023-11-08T08:21:18.812726+0000","last_became_active":"2023-11-08T08:21:17.767191+0000","last_became_peered":"2023-11-08T08:21:17.767191+0000","last_unstale":"2023-11-08T08:21:18.812726+0000","last_undegraded":"2023-11-08T08:21:18.812726+0000","last_fullsized":"2023-11-08T08:21:18.812726+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:00:36.084527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"56'8","reported_seq":19,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812048+0000","last_change":"2023-11-08T08:21:17.291277+0000","last_active":"2023-11-08T08:21:18.812048+0000","last_peered":"2023-11-08T08:21:18.812048+0000","last_clean":"2023-11-08T08:21:18.812048+0000","last_became_active":"2023-11-08T08:21:17.290945+0000","last_became_peered":"2023-11-08T08:21:17.290945+0000","last_unstale":"2023-11-08T08:21:18.812048+0000","last_undegraded":"2023-11-08T08:21:18.812048+0000","last_fullsized":"2023-11-08T08:21:18.812048+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:27:02.126538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813277+0000","last_change":"2023-11-08T08:21:17.783141+0000","last_active":"2023-11-08T08:21:18.813277+0000","last_peered":"2023-11-08T08:21:18.813277+0000","last_clean":"2023-11-08T08:21:18.813277+0000","last_became_active":"2023-11-08T08:21:17.782750+0000","last_became_peered":"2023-11-08T08:21:17.782750+0000","last_unstale":"2023-11-08T08:21:18.813277+0000","last_undegraded":"2023-11-08T08:21:18.813277+0000","last_fullsized":"2023-11-08T08:21:18.813277+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:47:35.532963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"56'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.808768+0000","last_change":"2023-11-08T08:21:17.286590+0000","last_active":"2023-11-08T08:21:18.808768+0000","last_peered":"2023-11-08T08:21:18.808768+0000","last_clean":"2023-11-08T08:21:18.808768+0000","last_became_active":"2023-11-08T08:21:17.286273+0000","last_became_peered":"2023-11-08T08:21:17.286273+0000","last_unstale":"2023-11-08T08:21:18.808768+0000","last_undegraded":"2023-11-08T08:21:18.808768+0000","last_fullsized":"2023-11-08T08:21:18.808768+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:40:30.132469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812906+0000","last_change":"2023-11-08T08:21:17.770666+0000","last_active":"2023-11-08T08:21:18.812906+0000","last_peered":"2023-11-08T08:21:18.812906+0000","last_clean":"2023-11-08T08:21:18.812906+0000","last_became_active":"2023-11-08T08:21:17.767891+0000","last_became_peered":"2023-11-08T08:21:17.767891+0000","last_unstale":"2023-11-08T08:21:18.812906+0000","last_undegraded":"2023-11-08T08:21:18.812906+0000","last_fullsized":"2023-11-08T08:21:18.812906+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:00:36.084527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753572+0000","last_change":"2023-11-08T08:21:17.292014+0000","last_active":"2023-11-08T08:21:18.753572+0000","last_peered":"2023-11-08T08:21:18.753572+0000","last_clean":"2023-11-08T08:21:18.753572+0000","last_became_active":"2023-11-08T08:21:17.291696+0000","last_became_peered":"2023-11-08T08:21:17.291696+0000","last_unstale":"2023-11-08T08:21:18.753572+0000","last_undegraded":"2023-11-08T08:21:18.753572+0000","last_fullsized":"2023-11-08T08:21:18.753572+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:13:07.575998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.808404+0000","last_change":"2023-11-08T08:21:17.767063+0000","last_active":"2023-11-08T08:21:18.808404+0000","last_peered":"2023-11-08T08:21:18.808404+0000","last_clean":"2023-11-08T08:21:18.808404+0000","last_became_active":"2023-11-08T08:21:17.766672+0000","last_became_peered":"2023-11-08T08:21:17.766672+0000","last_unstale":"2023-11-08T08:21:18.808404+0000","last_undegraded":"2023-11-08T08:21:18.808404+0000","last_fullsized":"2023-11-08T08:21:18.808404+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:18:24.957381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813879+0000","last_change":"2023-11-08T08:21:17.294933+0000","last_active":"2023-11-08T08:21:18.813879+0000","last_peered":"2023-11-08T08:21:18.813879+0000","last_clean":"2023-11-08T08:21:18.813879+0000","last_became_active":"2023-11-08T08:21:17.294633+0000","last_became_peered":"2023-11-08T08:21:17.294633+0000","last_unstale":"2023-11-08T08:21:18.813879+0000","last_undegraded":"2023-11-08T08:21:18.813879+0000","last_fullsized":"2023-11-08T08:21:18.813879+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:41:17.888191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753173+0000","last_change":"2023-11-08T08:21:17.775361+0000","last_active":"2023-11-08T08:21:18.753173+0000","last_peered":"2023-11-08T08:21:18.753173+0000","last_clean":"2023-11-08T08:21:18.753173+0000","last_became_active":"2023-11-08T08:21:17.775101+0000","last_became_peered":"2023-11-08T08:21:17.775101+0000","last_unstale":"2023-11-08T08:21:18.753173+0000","last_undegraded":"2023-11-08T08:21:18.753173+0000","last_fullsized":"2023-11-08T08:21:18.753173+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:28:52.114179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812441+0000","last_change":"2023-11-08T08:21:17.291899+0000","last_active":"2023-11-08T08:21:18.812441+0000","last_peered":"2023-11-08T08:21:18.812441+0000","last_clean":"2023-11-08T08:21:18.812441+0000","last_became_active":"2023-11-08T08:21:17.291210+0000","last_became_peered":"2023-11-08T08:21:17.291210+0000","last_unstale":"2023-11-08T08:21:18.812441+0000","last_undegraded":"2023-11-08T08:21:18.812441+0000","last_fullsized":"2023-11-08T08:21:18.812441+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:06:57.770400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.810089+0000","last_change":"2023-11-08T08:21:17.769525+0000","last_active":"2023-11-08T08:21:18.810089+0000","last_peered":"2023-11-08T08:21:18.810089+0000","last_clean":"2023-11-08T08:21:18.810089+0000","last_became_active":"2023-11-08T08:21:17.769228+0000","last_became_peered":"2023-11-08T08:21:17.769228+0000","last_unstale":"2023-11-08T08:21:18.810089+0000","last_undegraded":"2023-11-08T08:21:18.810089+0000","last_fullsized":"2023-11-08T08:21:18.810089+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:13:54.410850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.807845+0000","last_change":"2023-11-08T08:21:17.283845+0000","last_active":"2023-11-08T08:21:18.807845+0000","last_peered":"2023-11-08T08:21:18.807845+0000","last_clean":"2023-11-08T08:21:18.807845+0000","last_became_active":"2023-11-08T08:21:17.283085+0000","last_became_peered":"2023-11-08T08:21:17.283085+0000","last_unstale":"2023-11-08T08:21:18.807845+0000","last_undegraded":"2023-11-08T08:21:18.807845+0000","last_fullsized":"2023-11-08T08:21:18.807845+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:10:08.567828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813882+0000","last_change":"2023-11-08T08:21:17.777223+0000","last_active":"2023-11-08T08:21:18.813882+0000","last_peered":"2023-11-08T08:21:18.813882+0000","last_clean":"2023-11-08T08:21:18.813882+0000","last_became_active":"2023-11-08T08:21:17.776641+0000","last_became_peered":"2023-11-08T08:21:17.776641+0000","last_unstale":"2023-11-08T08:21:18.813882+0000","last_undegraded":"2023-11-08T08:21:18.813882+0000","last_fullsized":"2023-11-08T08:21:18.813882+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:13:08.589498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.754998+0000","last_change":"2023-11-08T08:21:17.290352+0000","last_active":"2023-11-08T08:21:18.754998+0000","last_peered":"2023-11-08T08:21:18.754998+0000","last_clean":"2023-11-08T08:21:18.754998+0000","last_became_active":"2023-11-08T08:21:17.289568+0000","last_became_peered":"2023-11-08T08:21:17.289568+0000","last_unstale":"2023-11-08T08:21:18.754998+0000","last_undegraded":"2023-11-08T08:21:18.754998+0000","last_fullsized":"2023-11-08T08:21:18.754998+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:13:07.575998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.807399+0000","last_change":"2023-11-08T08:21:17.775092+0000","last_active":"2023-11-08T08:21:18.807399+0000","last_peered":"2023-11-08T08:21:18.807399+0000","last_clean":"2023-11-08T08:21:18.807399+0000","last_became_active":"2023-11-08T08:21:17.774817+0000","last_became_peered":"2023-11-08T08:21:17.774817+0000","last_unstale":"2023-11-08T08:21:18.807399+0000","last_undegraded":"2023-11-08T08:21:18.807399+0000","last_fullsized":"2023-11-08T08:21:18.807399+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:44:46.041614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.807768+0000","last_change":"2023-11-08T08:21:17.287571+0000","last_active":"2023-11-08T08:21:18.807768+0000","last_peered":"2023-11-08T08:21:18.807768+0000","last_clean":"2023-11-08T08:21:18.807768+0000","last_became_active":"2023-11-08T08:21:17.287071+0000","last_became_peered":"2023-11-08T08:21:17.287071+0000","last_unstale":"2023-11-08T08:21:18.807768+0000","last_undegraded":"2023-11-08T08:21:18.807768+0000","last_fullsized":"2023-11-08T08:21:18.807768+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:59:09.027422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809443+0000","last_change":"2023-11-08T08:21:17.777907+0000","last_active":"2023-11-08T08:21:18.809443+0000","last_peered":"2023-11-08T08:21:18.809443+0000","last_clean":"2023-11-08T08:21:18.809443+0000","last_became_active":"2023-11-08T08:21:17.777571+0000","last_became_peered":"2023-11-08T08:21:17.777571+0000","last_unstale":"2023-11-08T08:21:18.809443+0000","last_undegraded":"2023-11-08T08:21:18.809443+0000","last_fullsized":"2023-11-08T08:21:18.809443+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:28:52.114179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809269+0000","last_change":"2023-11-08T08:21:17.291335+0000","last_active":"2023-11-08T08:21:18.809269+0000","last_peered":"2023-11-08T08:21:18.809269+0000","last_clean":"2023-11-08T08:21:18.809269+0000","last_became_active":"2023-11-08T08:21:17.291030+0000","last_became_peered":"2023-11-08T08:21:17.291030+0000","last_unstale":"2023-11-08T08:21:18.809269+0000","last_undegraded":"2023-11-08T08:21:18.809269+0000","last_fullsized":"2023-11-08T08:21:18.809269+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:15:03.992666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.810089+0000","last_change":"2023-11-08T08:21:17.779348+0000","last_active":"2023-11-08T08:21:18.810089+0000","last_peered":"2023-11-08T08:21:18.810089+0000","last_clean":"2023-11-08T08:21:18.810089+0000","last_became_active":"2023-11-08T08:21:17.779100+0000","last_became_peered":"2023-11-08T08:21:17.779100+0000","last_unstale":"2023-11-08T08:21:18.810089+0000","last_undegraded":"2023-11-08T08:21:18.810089+0000","last_fullsized":"2023-11-08T08:21:18.810089+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:02:19.326469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"56'1","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813142+0000","last_change":"2023-11-08T08:21:17.291536+0000","last_active":"2023-11-08T08:21:18.813142+0000","last_peered":"2023-11-08T08:21:18.813142+0000","last_clean":"2023-11-08T08:21:18.813142+0000","last_became_active":"2023-11-08T08:21:17.291241+0000","last_became_peered":"2023-11-08T08:21:17.291241+0000","last_unstale":"2023-11-08T08:21:18.813142+0000","last_undegraded":"2023-11-08T08:21:18.813142+0000","last_fullsized":"2023-11-08T08:21:18.813142+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:24:43.804674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-08T08:21:17.746955+0000","last_change":"2023-11-08T08:18:58.771310+0000","last_active":"2023-11-08T08:21:17.746955+0000","last_peered":"2023-11-08T08:21:17.746955+0000","last_clean":"2023-11-08T08:21:17.746955+0000","last_became_active":"2023-11-08T08:18:58.770891+0000","last_became_peered":"2023-11-08T08:18:58.770891+0000","last_unstale":"2023-11-08T08:21:17.746955+0000","last_undegraded":"2023-11-08T08:21:17.746955+0000","last_fullsized":"2023-11-08T08:21:17.746955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.814094+0000","last_change":"2023-11-08T08:21:17.775920+0000","last_active":"2023-11-08T08:21:18.814094+0000","last_peered":"2023-11-08T08:21:18.814094+0000","last_clean":"2023-11-08T08:21:18.814094+0000","last_became_active":"2023-11-08T08:21:17.771601+0000","last_became_peered":"2023-11-08T08:21:17.771601+0000","last_unstale":"2023-11-08T08:21:18.814094+0000","last_undegraded":"2023-11-08T08:21:18.814094+0000","last_fullsized":"2023-11-08T08:21:18.814094+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:10:57.252073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753180+0000","last_change":"2023-11-08T08:21:17.287063+0000","last_active":"2023-11-08T08:21:18.753180+0000","last_peered":"2023-11-08T08:21:18.753180+0000","last_clean":"2023-11-08T08:21:18.753180+0000","last_became_active":"2023-11-08T08:21:17.286181+0000","last_became_peered":"2023-11-08T08:21:17.286181+0000","last_unstale":"2023-11-08T08:21:18.753180+0000","last_undegraded":"2023-11-08T08:21:18.753180+0000","last_fullsized":"2023-11-08T08:21:18.753180+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:22:59.290723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-08T08:21:17.754057+0000","last_change":"2023-11-08T08:18:58.770483+0000","last_active":"2023-11-08T08:21:17.754057+0000","last_peered":"2023-11-08T08:21:17.754057+0000","last_clean":"2023-11-08T08:21:17.754057+0000","last_became_active":"2023-11-08T08:18:58.770040+0000","last_became_peered":"2023-11-08T08:18:58.770040+0000","last_unstale":"2023-11-08T08:21:17.754057+0000","last_undegraded":"2023-11-08T08:21:17.754057+0000","last_fullsized":"2023-11-08T08:21:17.754057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.814487+0000","last_change":"2023-11-08T08:21:17.777849+0000","last_active":"2023-11-08T08:21:18.814487+0000","last_peered":"2023-11-08T08:21:18.814487+0000","last_clean":"2023-11-08T08:21:18.814487+0000","last_became_active":"2023-11-08T08:21:17.777549+0000","last_became_peered":"2023-11-08T08:21:17.777549+0000","last_unstale":"2023-11-08T08:21:18.814487+0000","last_undegraded":"2023-11-08T08:21:18.814487+0000","last_fullsized":"2023-11-08T08:21:18.814487+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:40:34.435351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"56'1","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.810425+0000","last_change":"2023-11-08T08:21:17.300526+0000","last_active":"2023-11-08T08:21:18.810425+0000","last_peered":"2023-11-08T08:21:18.810425+0000","last_clean":"2023-11-08T08:21:18.810425+0000","last_became_active":"2023-11-08T08:21:17.298791+0000","last_became_peered":"2023-11-08T08:21:17.298791+0000","last_unstale":"2023-11-08T08:21:18.810425+0000","last_undegraded":"2023-11-08T08:21:18.810425+0000","last_fullsized":"2023-11-08T08:21:18.810425+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:12:07.886476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-08T08:21:17.753273+0000","last_change":"2023-11-08T08:18:58.766848+0000","last_active":"2023-11-08T08:21:17.753273+0000","last_peered":"2023-11-08T08:21:17.753273+0000","last_clean":"2023-11-08T08:21:17.753273+0000","last_became_active":"2023-11-08T08:18:58.766378+0000","last_became_peered":"2023-11-08T08:18:58.766378+0000","last_unstale":"2023-11-08T08:21:17.753273+0000","last_undegraded":"2023-11-08T08:21:17.753273+0000","last_fullsized":"2023-11-08T08:21:17.753273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812767+0000","last_change":"2023-11-08T08:21:17.772916+0000","last_active":"2023-11-08T08:21:18.812767+0000","last_peered":"2023-11-08T08:21:18.812767+0000","last_clean":"2023-11-08T08:21:18.812767+0000","last_became_active":"2023-11-08T08:21:17.772642+0000","last_became_peered":"2023-11-08T08:21:17.772642+0000","last_unstale":"2023-11-08T08:21:18.812767+0000","last_undegraded":"2023-11-08T08:21:18.812767+0000","last_fullsized":"2023-11-08T08:21:18.812767+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:00:16.099510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.753592+0000","last_change":"2023-11-08T08:21:17.298893+0000","last_active":"2023-11-08T08:21:18.753592+0000","last_peered":"2023-11-08T08:21:18.753592+0000","last_clean":"2023-11-08T08:21:18.753592+0000","last_became_active":"2023-11-08T08:21:17.298621+0000","last_became_peered":"2023-11-08T08:21:17.298621+0000","last_unstale":"2023-11-08T08:21:18.753592+0000","last_undegraded":"2023-11-08T08:21:18.753592+0000","last_fullsized":"2023-11-08T08:21:18.753592+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:36:36.365097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-08T08:21:17.755150+0000","last_change":"2023-11-08T08:18:58.769764+0000","last_active":"2023-11-08T08:21:17.755150+0000","last_peered":"2023-11-08T08:21:17.755150+0000","last_clean":"2023-11-08T08:21:17.755150+0000","last_became_active":"2023-11-08T08:18:58.769357+0000","last_became_peered":"2023-11-08T08:18:58.769357+0000","last_unstale":"2023-11-08T08:21:17.755150+0000","last_undegraded":"2023-11-08T08:21:17.755150+0000","last_fullsized":"2023-11-08T08:21:17.755150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:13.748094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.752433+0000","last_change":"2023-11-08T08:21:17.753174+0000","last_active":"2023-11-08T08:21:18.752433+0000","last_peered":"2023-11-08T08:21:18.752433+0000","last_clean":"2023-11-08T08:21:18.752433+0000","last_became_active":"2023-11-08T08:21:17.752894+0000","last_became_peered":"2023-11-08T08:21:17.752894+0000","last_unstale":"2023-11-08T08:21:18.752433+0000","last_undegraded":"2023-11-08T08:21:18.752433+0000","last_fullsized":"2023-11-08T08:21:18.752433+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:19:00.004764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812056+0000","last_change":"2023-11-08T08:21:17.294130+0000","last_active":"2023-11-08T08:21:18.812056+0000","last_peered":"2023-11-08T08:21:18.812056+0000","last_clean":"2023-11-08T08:21:18.812056+0000","last_became_active":"2023-11-08T08:21:17.293726+0000","last_became_peered":"2023-11-08T08:21:17.293726+0000","last_unstale":"2023-11-08T08:21:18.812056+0000","last_undegraded":"2023-11-08T08:21:18.812056+0000","last_fullsized":"2023-11-08T08:21:18.812056+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:33:03.450167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-08T08:21:17.755188+0000","last_change":"2023-11-08T08:18:58.771460+0000","last_active":"2023-11-08T08:21:17.755188+0000","last_peered":"2023-11-08T08:21:17.755188+0000","last_clean":"2023-11-08T08:21:17.755188+0000","last_became_active":"2023-11-08T08:18:58.770648+0000","last_became_peered":"2023-11-08T08:18:58.770648+0000","last_unstale":"2023-11-08T08:21:17.755188+0000","last_undegraded":"2023-11-08T08:21:17.755188+0000","last_fullsized":"2023-11-08T08:21:17.755188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.812848+0000","last_change":"2023-11-08T08:21:17.772227+0000","last_active":"2023-11-08T08:21:18.812848+0000","last_peered":"2023-11-08T08:21:18.812848+0000","last_clean":"2023-11-08T08:21:18.812848+0000","last_became_active":"2023-11-08T08:21:17.771928+0000","last_became_peered":"2023-11-08T08:21:17.771928+0000","last_unstale":"2023-11-08T08:21:18.812848+0000","last_undegraded":"2023-11-08T08:21:18.812848+0000","last_fullsized":"2023-11-08T08:21:18.812848+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:23:00.304223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"56'2","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809772+0000","last_change":"2023-11-08T08:21:17.296624+0000","last_active":"2023-11-08T08:21:18.809772+0000","last_peered":"2023-11-08T08:21:18.809772+0000","last_clean":"2023-11-08T08:21:18.809772+0000","last_became_active":"2023-11-08T08:21:17.296357+0000","last_became_peered":"2023-11-08T08:21:17.296357+0000","last_unstale":"2023-11-08T08:21:18.809772+0000","last_undegraded":"2023-11-08T08:21:18.809772+0000","last_fullsized":"2023-11-08T08:21:18.809772+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:40:33.421851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-08T08:21:17.745102+0000","last_change":"2023-11-08T08:18:58.973732+0000","last_active":"2023-11-08T08:21:17.745102+0000","last_peered":"2023-11-08T08:21:17.745102+0000","last_clean":"2023-11-08T08:21:17.745102+0000","last_became_active":"2023-11-08T08:18:58.973007+0000","last_became_peered":"2023-11-08T08:18:58.973007+0000","last_unstale":"2023-11-08T08:21:17.745102+0000","last_undegraded":"2023-11-08T08:21:17.745102+0000","last_fullsized":"2023-11-08T08:21:17.745102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.752940+0000","last_change":"2023-11-08T08:21:17.768129+0000","last_active":"2023-11-08T08:21:18.752940+0000","last_peered":"2023-11-08T08:21:18.752940+0000","last_clean":"2023-11-08T08:21:18.752940+0000","last_became_active":"2023-11-08T08:21:17.762620+0000","last_became_peered":"2023-11-08T08:21:17.762620+0000","last_unstale":"2023-11-08T08:21:18.752940+0000","last_undegraded":"2023-11-08T08:21:18.752940+0000","last_fullsized":"2023-11-08T08:21:18.752940+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:13:54.410850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.807880+0000","last_change":"2023-11-08T08:21:17.301161+0000","last_active":"2023-11-08T08:21:18.807880+0000","last_peered":"2023-11-08T08:21:18.807880+0000","last_clean":"2023-11-08T08:21:18.807880+0000","last_became_active":"2023-11-08T08:21:17.300579+0000","last_became_peered":"2023-11-08T08:21:17.300579+0000","last_unstale":"2023-11-08T08:21:18.807880+0000","last_undegraded":"2023-11-08T08:21:18.807880+0000","last_fullsized":"2023-11-08T08:21:18.807880+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:27:39.874588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-08T08:21:17.750488+0000","last_change":"2023-11-08T08:18:58.770136+0000","last_active":"2023-11-08T08:21:17.750488+0000","last_peered":"2023-11-08T08:21:17.750488+0000","last_clean":"2023-11-08T08:21:17.750488+0000","last_became_active":"2023-11-08T08:18:58.769797+0000","last_became_peered":"2023-11-08T08:18:58.769797+0000","last_unstale":"2023-11-08T08:21:17.750488+0000","last_undegraded":"2023-11-08T08:21:17.750488+0000","last_fullsized":"2023-11-08T08:21:17.750488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:29:12.107128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.809085+0000","last_change":"2023-11-08T08:21:17.764312+0000","last_active":"2023-11-08T08:21:18.809085+0000","last_peered":"2023-11-08T08:21:18.809085+0000","last_clean":"2023-11-08T08:21:18.809085+0000","last_became_active":"2023-11-08T08:21:17.764041+0000","last_became_peered":"2023-11-08T08:21:17.764041+0000","last_unstale":"2023-11-08T08:21:18.809085+0000","last_undegraded":"2023-11-08T08:21:18.809085+0000","last_fullsized":"2023-11-08T08:21:18.809085+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:17.252515+0000","last_clean_scrub_stamp":"2023-11-08T08:21:17.252515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:26:37.331809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T08:21:18.813377+0000","last_change":"2023-11-08T08:21:17.296472+0000","last_active":"2023-11-08T08:21:18.813377+0000","last_peered":"2023-11-08T08:21:18.813377+0000","last_clean":"2023-11-08T08:21:18.813377+0000","last_became_active":"2023-11-08T08:21:17.296049+0000","last_became_peered":"2023-11-08T08:21:17.296049+0000","last_unstale":"2023-11-08T08:21:18.813377+0000","last_undegraded":"2023-11-08T08:21:18.813377+0000","last_fullsized":"2023-11-08T08:21:18.813377+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:21:16.239016+0000","last_clean_scrub_stamp":"2023-11-08T08:21:16.239016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:50:52.603382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":42,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-08T08:21:17.763256+0000","last_change":"2023-11-08T08:18:58.777140+0000","last_active":"2023-11-08T08:21:17.763256+0000","last_peered":"2023-11-08T08:21:17.763256+0000","last_clean":"2023-11-08T08:21:17.763256+0000","last_became_active":"2023-11-08T08:18:58.776834+0000","last_became_peered":"2023-11-08T08:18:58.776834+0000","last_unstale":"2023-11-08T08:21:17.763256+0000","last_undegraded":"2023-11-08T08:21:17.763256+0000","last_fullsized":"2023-11-08T08:21:17.763256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":196,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-08T08:21:17.751636+0000","last_change":"2023-11-08T08:18:53.780089+0000","last_active":"2023-11-08T08:21:17.751636+0000","last_peered":"2023-11-08T08:21:17.751636+0000","last_clean":"2023-11-08T08:21:17.751636+0000","last_became_active":"2023-11-08T08:18:53.779533+0000","last_became_peered":"2023-11-08T08:18:53.779533+0000","last_unstale":"2023-11-08T08:21:17.751636+0000","last_undegraded":"2023-11-08T08:21:17.751636+0000","last_fullsized":"2023-11-08T08:21:17.751636+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_clean_scrub_stamp":"2023-11-08T08:18:52.744693+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:58:24.877070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10412,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361428,"statfs":{"total":96636764160,"available":96626102272,"internally_reserved":0,"allocated":1282048,"data_stored":904236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.0349999999999999,"15min":1.012},"min":{"1min":0.63300000000000001,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.169,"15min":1.1619999999999999},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":1.147}]},{"osd":1,"last update":"Wed Nov 8 08:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1879999999999999,"15min":1.194},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.484,"5min":1.972,"15min":1.972},"last":1.107},{"interface":"front","average":{"1min":1.248,"5min":1.3109999999999999,"15min":1.321},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.5980000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.1779999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.393,"15min":1.425},"min":{"1min":0.77000000000000002,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.514,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.373,"15min":1.419},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.5720000000000001,"5min":4.9089999999999998,"15min":4.9089999999999998},"last":1.3220000000000001}]},{"osd":3,"last update":"Wed Nov 8 08:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.256,"15min":1.262},"min":{"1min":0.94499999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5109999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.3049999999999999,"15min":1.3120000000000001},"min":{"1min":0.872,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.474,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":1.357}]},{"osd":4,"last update":"Wed Nov 8 08:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2629999999999999,"15min":1.2729999999999999},"min":{"1min":0.94099999999999995,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.431,"15min":1.4370000000000001},"min":{"1min":1.0800000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.7170000000000001,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":1.292}]},{"osd":5,"last update":"Wed Nov 8 08:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.375,"15min":1.389},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.627,"5min":2.73,"15min":2.73},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3380000000000001,"15min":1.3440000000000001},"min":{"1min":0.95399999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.7310000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.218}]},{"osd":6,"last update":"Wed Nov 8 08:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.4450000000000001,"15min":1.456},"min":{"1min":1.1120000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.665,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.258},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.4379999999999999,"15min":1.454},"min":{"1min":0.94099999999999995,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.6870000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.2370000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9184,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362656,"statfs":{"total":96636764160,"available":96627359744,"internally_reserved":0,"allocated":679936,"data_stored":313708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.1639999999999999,"15min":1.1870000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.4350000000000001,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0960000000000001,"15min":1.1020000000000001},"min":{"1min":0.65500000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.4930000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.1879999999999999}]},{"osd":1,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.1990000000000001,"15min":1.1910000000000001},"min":{"1min":0.81299999999999994,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6970000000000001,"5min":1.782,"15min":1.782},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.22,"5min":1.208,"15min":1.206},"min":{"1min":0.76700000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.5740000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.1100000000000001}]},{"osd":2,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.175,"15min":1.1839999999999999},"min":{"1min":0.79800000000000004,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.7589999999999999,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.079,"5min":1.262,"15min":1.292},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.4990000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.238}]},{"osd":3,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.278,"15min":1.2889999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.605,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.87},{"interface":"front","average":{"1min":1.331,"5min":1.399,"15min":1.4099999999999999},"min":{"1min":1.0880000000000001,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.6539999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.052}]},{"osd":4,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.248,"15min":1.264},"min":{"1min":0.79100000000000004,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.5149999999999999,"5min":2.02,"15min":2.02},"last":1.008},{"interface":"front","average":{"1min":1.28,"5min":1.208,"15min":1.196},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.6619999999999999,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.29}]},{"osd":5,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.2609999999999999,"15min":1.2390000000000001},"min":{"1min":1.153,"5min":0.88,"15min":0.88},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3759999999999999,"15min":1.375},"min":{"1min":1.024,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.6599999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.173}]},{"osd":7,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.389,"15min":1.3879999999999999},"min":{"1min":1.129,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.714,"5min":3.133,"15min":3.133},"last":1.363},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.448,"15min":1.4399999999999999},"min":{"1min":1.004,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.8560000000000001,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.4910000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9256,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362584,"statfs":{"total":96636764160,"available":96627286016,"internally_reserved":0,"allocated":688128,"data_stored":314344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.123,"15min":1.129},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":1.246},{"interface":"front","average":{"1min":1.198,"5min":1.234,"15min":1.24},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.4730000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.1739999999999999}]},{"osd":1,"last update":"Wed Nov 8 08:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.1359999999999999,"15min":1.1419999999999999},"min":{"1min":0.86699999999999999,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.427,"5min":2.012,"15min":2.012},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1799999999999999,"15min":1.1970000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.4450000000000001,"5min":1.667,"15min":1.667},"last":1.2889999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1990000000000001,"15min":1.2010000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.121,"5min":1.214,"15min":1.23},"min":{"1min":0.77700000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.7190000000000001,"5min":1.796,"15min":1.796},"last":1.0409999999999999}]},{"osd":3,"last update":"Wed Nov 8 08:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.333,"15min":1.345},"min":{"1min":1.0009999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":1.4930000000000001,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.3160000000000001,"15min":1.335},"min":{"1min":0.88900000000000001,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.5009999999999999,"5min":3.7999999999999998,"15min":3.7999999999999998},"last":1.1459999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.1839999999999999,"15min":1.21},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.409,"5min":1.629,"15min":1.629},"last":1.085},{"interface":"front","average":{"1min":1.167,"5min":1.325,"15min":1.351},"min":{"1min":0.89300000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.536,"5min":3.073,"15min":3.073},"last":1.1060000000000001}]},{"osd":6,"last update":"Wed Nov 8 08:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.3280000000000001,"15min":1.3580000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.46,"5min":3.8650000000000002,"15min":3.8650000000000002},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.262,"5min":1.373,"15min":1.391},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.587,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.99299999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.478,"15min":1.498},"min":{"1min":1.089,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":1.631,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.278},{"interface":"front","average":{"1min":1.294,"5min":1.474,"15min":1.504},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.605,"5min":3.9369999999999998,"15min":3.9369999999999998},"last":0.95399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9452,"kb_used_data":676,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362388,"statfs":{"total":96636764160,"available":96627085312,"internally_reserved":0,"allocated":692224,"data_stored":315256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.147,"15min":1.145},"min":{"1min":0.82699999999999996,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.171,"5min":1.2250000000000001,"15min":1.234},"min":{"1min":0.76100000000000001,"5min":0.747,"15min":0.747},"max":{"1min":1.5029999999999999,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":1.079}]},{"osd":1,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.079,"15min":1.089},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.3460000000000001,"5min":1.99,"15min":1.99},"last":1.139},{"interface":"front","average":{"1min":1.159,"5min":1.2090000000000001,"15min":1.2170000000000001},"min":{"1min":0.77800000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.546,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.97099999999999997}]},{"osd":2,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.127,"15min":1.1259999999999999},"min":{"1min":0.90600000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.4490000000000001,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.177,"5min":1.256,"15min":1.2689999999999999},"min":{"1min":0.82099999999999995,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.7350000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.1890000000000001}]},{"osd":3,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.2010000000000001,"15min":1.202},"min":{"1min":0.73099999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.5760000000000001,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":1.306},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2390000000000001,"15min":1.2490000000000001},"min":{"1min":0.91900000000000004,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.5429999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":0.92600000000000005}]},{"osd":5,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2909999999999999,"15min":1.2989999999999999},"min":{"1min":0.89500000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.8180000000000001,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.264,"15min":1.2609999999999999},"min":{"1min":0.93700000000000006,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.6830000000000001,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":1.002}]},{"osd":6,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2809999999999999,"15min":1.2869999999999999},"min":{"1min":0.82699999999999996,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5329999999999999,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3700000000000001,"15min":1.373},"min":{"1min":1.0269999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.7769999999999999,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.4219999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.4359999999999999,"15min":1.4419999999999999},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.772,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.419,"5min":1.4830000000000001,"15min":1.4930000000000001},"min":{"1min":1.1699999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.71,"5min":2.399,"15min":2.399},"last":1.452}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9384,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362456,"statfs":{"total":96636764160,"available":96627154944,"internally_reserved":0,"allocated":688128,"data_stored":313764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.91300000000000003,"15min":0.91500000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.94899999999999995,"15min":0.94799999999999995},"min":{"1min":0.65800000000000003,"5min":0.626,"15min":0.626},"max":{"1min":1.331,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.081}]},{"osd":1,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.93200000000000005,"15min":0.94099999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.125,"15min":1.1319999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.373,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.105}]},{"osd":2,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1200000000000001,"15min":1.1140000000000001},"min":{"1min":0.74099999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.135,"5min":1.2110000000000001,"15min":1.224},"min":{"1min":0.85499999999999998,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.7929999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.04}]},{"osd":4,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.2809999999999999,"15min":1.3100000000000001},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.528,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.153},{"interface":"front","average":{"1min":1.27,"5min":1.3859999999999999,"15min":1.405},"min":{"1min":0.88800000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.496,"5min":2.27,"15min":2.27},"last":1.208}]},{"osd":5,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.4379999999999999,"15min":1.45},"min":{"1min":1.0069999999999999,"5min":0.998,"15min":0.998},"max":{"1min":1.645,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.4219999999999999,"15min":1.4359999999999999},"min":{"1min":1.0349999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.694,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.246}]},{"osd":6,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.4410000000000001,"15min":1.482},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.786,"5min":2.008,"15min":2.008},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.405,"5min":1.528,"15min":1.548},"min":{"1min":1.028,"5min":1.028,"15min":1.028},"max":{"1min":1.77,"5min":2.145,"15min":2.145},"last":1.341}]},{"osd":7,"last update":"Wed Nov 8 08:20:51 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.583,"15min":1.609},"min":{"1min":1.155,"5min":1.155,"15min":1.155},"max":{"1min":1.819,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":1.524},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.5589999999999999,"15min":1.601},"min":{"1min":1.03,"5min":1.03,"15min":1.03},"max":{"1min":1.7210000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.3160000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9192,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362648,"statfs":{"total":96636764160,"available":96627351552,"internally_reserved":0,"allocated":688128,"data_stored":313764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.92300000000000004,"15min":0.94499999999999995},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.997,"15min":1.0069999999999999},"min":{"1min":0.626,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6100000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.95099999999999996}]},{"osd":1,"last update":"Wed Nov 8 08:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.0900000000000001,"15min":1.115},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.4810000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.014},{"interface":"front","average":{"1min":1.034,"5min":1.03,"15min":1.0289999999999999},"min":{"1min":0.65400000000000003,"5min":0.505,"15min":0.505},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.149}]},{"osd":3,"last update":"Wed Nov 8 08:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.0169999999999999,"15min":1.0209999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.5189999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.3160000000000001,"15min":1.3580000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5620000000000001,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":1.2969999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:20:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.448,"15min":1.486},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.504,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":1.45},{"interface":"front","average":{"1min":1.248,"5min":1.3560000000000001,"15min":1.3740000000000001},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.8260000000000001,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.516}]},{"osd":5,"last update":"Wed Nov 8 08:20:59 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.417,"15min":1.4430000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.9179999999999999,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.708},{"interface":"front","average":{"1min":1.351,"5min":1.355,"15min":1.355},"min":{"1min":0.93999999999999995,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.3720000000000001}]},{"osd":6,"last update":"Wed Nov 8 08:20:59 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4339999999999999,"15min":1.4370000000000001},"min":{"1min":0.91400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.8169999999999999,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.5549999999999999,"15min":1.5960000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.6020000000000001,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":1.546}]},{"osd":7,"last update":"Wed Nov 8 08:20:59 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.4670000000000001,"15min":1.484},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.9750000000000001,"5min":2.867,"15min":2.867},"last":1.409},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4910000000000001,"15min":1.492},"min":{"1min":1.012,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.02,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.6599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9252,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362588,"statfs":{"total":96636764160,"available":96627290112,"internally_reserved":0,"allocated":684032,"data_stored":313798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.89100000000000001,"15min":0.89900000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.204,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0449999999999999,"15min":1.0489999999999999},"min":{"1min":0.68100000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3080000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.214}]},{"osd":2,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.97999999999999998,"15min":0.99399999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.101,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.1679999999999999,"15min":1.194},"min":{"1min":0.76000000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.425,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.044}]},{"osd":3,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.109,"15min":1.1120000000000001},"min":{"1min":0.70199999999999996,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3260000000000001,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.1060000000000001,"15min":1.125},"min":{"1min":0.56599999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.8260000000000001,"5min":1.972,"15min":1.972},"last":0.78800000000000003}]},{"osd":4,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.232,"15min":1.236},"min":{"1min":0.77700000000000002,"5min":0.746,"15min":0.746},"max":{"1min":1.639,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3340000000000001,"15min":1.3520000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.554,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":1.28}]},{"osd":5,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.331,"15min":1.341},"min":{"1min":0.94099999999999995,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.694,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.26,"5min":1.3400000000000001,"15min":1.353},"min":{"1min":0.81200000000000006,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.73,"5min":2.48,"15min":2.48},"last":1.258}]},{"osd":6,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.4350000000000001,"15min":1.4490000000000001},"min":{"1min":1.034,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.671,"5min":3.149,"15min":3.149},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.474,"15min":1.482},"min":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":1.7729999999999999,"5min":2.169,"15min":2.169},"last":1.4219999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.5589999999999999,"15min":1.5740000000000001},"min":{"1min":1.145,"5min":1.1020000000000001,"15min":1.1020000000000001},"max":{"1min":1.8720000000000001,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.476,"15min":1.48},"min":{"1min":0.89800000000000002,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.0030000000000001,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.3680000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10548,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361292,"statfs":{"total":96636764160,"available":96625963008,"internally_reserved":0,"allocated":1290240,"data_stored":906190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 08:20:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.94799999999999995,"15min":0.97399999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.153,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.94599999999999995,"15min":0.96699999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.236,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.93700000000000006}]},{"osd":2,"last update":"Wed Nov 8 08:20:53 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.995,"15min":1.0089999999999999},"min":{"1min":0.59099999999999997,"5min":0.497,"15min":0.497},"max":{"1min":1.282,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1020000000000001,"15min":1.1180000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3320000000000001,"5min":1.528,"15min":1.528},"last":1.5920000000000001}]},{"osd":3,"last update":"Wed Nov 8 08:20:53 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.1339999999999999,"15min":1.1679999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3480000000000001,"5min":1.964,"15min":1.964},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.044,"5min":1.1830000000000001,"15min":1.206},"min":{"1min":0.67900000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.343,"5min":1.786,"15min":1.786},"last":1.4139999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:20:53 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.49,"15min":1.5289999999999999},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.6559999999999999,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.909},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.4590000000000001,"15min":1.4870000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.6839999999999999,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.657}]},{"osd":5,"last update":"Wed Nov 8 08:20:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.4870000000000001,"15min":1.52},"min":{"1min":0.999,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":1.6990000000000001,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.958},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.621,"15min":1.663},"min":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"max":{"1min":1.7430000000000001,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.766}]},{"osd":6,"last update":"Wed Nov 8 08:20:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.512,"15min":1.544},"min":{"1min":0.96799999999999997,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.5780000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.498},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.5629999999999999,"15min":1.5920000000000001},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":1.7430000000000001,"5min":2.048,"15min":2.048},"last":1.8480000000000001}]},{"osd":7,"last update":"Wed Nov 8 08:20:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.6240000000000001,"15min":1.6639999999999999},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":1.708,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.6659999999999999,"15min":1.7050000000000001},"min":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"max":{"1min":1.681,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":2.0070000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T08:21:22.582 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T08:21:22.583 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-08T08:21:22.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.824+0000 7f3491ae7700 1 Processor -- start 2023-11-08T08:21:22.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.824+0000 7f3491ae7700 1 -- start start 2023-11-08T08:21:22.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.828+0000 7f3491ae7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f348c0b5410 0x7f348c0b2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:22.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.828+0000 7f3491ae7700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f348c0ce260 0x7f348c06ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:22.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.828+0000 7f3491ae7700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348c0ce6b0 con 0x7f348c06d340 2023-11-08T08:21:22.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.828+0000 7f3491ae7700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c06aa50 con 0x7f348c0b5410 2023-11-08T08:21:22.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.828+0000 7f3491ae7700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c0b57e0 con 0x7f348c0ce260 2023-11-08T08:21:22.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.828+0000 7f348b7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f348c0b5410 0x7f348c0b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:22.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.828+0000 7f348b7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f348c0b5410 0x7f348c0b2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51512/0 (socket says 172.21.15.40:51512) 2023-11-08T08:21:22.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.828+0000 7f348b7fe700 1 -- 172.21.15.40:0/3673814251 learned_addr learned my addr 172.21.15.40:0/3673814251 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:22.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.828+0000 7f348affd700 1 --2- 172.21.15.40:0/3673814251 >> v2:172.21.15.169:3300/0 conn(0x7f348c0ce260 0x7f348c06ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:22.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.828+0000 7f348b7fe700 1 -- 172.21.15.40:0/3673814251 >> v2:172.21.15.169:3300/0 conn(0x7f348c0ce260 msgr2=0x7f348c06ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:22.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.828+0000 7f348b7fe700 1 --2- 172.21.15.40:0/3673814251 >> v2:172.21.15.169:3300/0 conn(0x7f348c0ce260 0x7f348c06ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:22.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.828+0000 7f348b7fe700 1 -- 172.21.15.40:0/3673814251 >> v1:172.21.15.169:6789/0 conn(0x7f348c06d340 legacy=0x7f348c069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:22.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.828+0000 7f348b7fe700 1 -- 172.21.15.40:0/3673814251 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c15cb00 con 0x7f348c0b5410 2023-11-08T08:21:22.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.828+0000 7f348b7fe700 1 --2- 172.21.15.40:0/3673814251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348c0b5410 0x7f348c0b2260 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f348c073640 tx=0x7f347c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4550fa6479bd1dc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:22.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.828+0000 7f348a7fc700 1 -- 172.21.15.40:0/3673814251 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c011830 con 0x7f348c0b5410 2023-11-08T08:21:22.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.832+0000 7f348a7fc700 1 -- 172.21.15.40:0/3673814251 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347c011990 con 0x7f348c0b5410 2023-11-08T08:21:22.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.832+0000 7f348a7fc700 1 -- 172.21.15.40:0/3673814251 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c011d20 con 0x7f348c0b5410 2023-11-08T08:21:22.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.832+0000 7f3491ae7700 1 -- 172.21.15.40:0/3673814251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348c0b5410 msgr2=0x7f348c0b2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:22.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.832+0000 7f3491ae7700 1 --2- 172.21.15.40:0/3673814251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348c0b5410 0x7f348c0b2260 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f348c073640 tx=0x7f347c009bf0 comp rx=0 tx=0).stop 2023-11-08T08:21:22.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.832+0000 7f3491ae7700 1 -- 172.21.15.40:0/3673814251 shutdown_connections 2023-11-08T08:21:22.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.832+0000 7f3491ae7700 1 --2- 172.21.15.40:0/3673814251 >> v2:172.21.15.169:3300/0 conn(0x7f348c0ce260 0x7f348c06ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:22.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.832+0000 7f3491ae7700 1 --2- 172.21.15.40:0/3673814251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348c0b5410 0x7f348c0b2260 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:22.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.832+0000 7f3491ae7700 1 -- 172.21.15.40:0/3673814251 >> 172.21.15.40:0/3673814251 conn(0x7f348c0d3990 msgr2=0x7f348c0cf780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:22.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.832+0000 7f3491ae7700 1 -- 172.21.15.40:0/3673814251 shutdown_connections 2023-11-08T08:21:22.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.832+0000 7f3491ae7700 1 -- 172.21.15.40:0/3673814251 wait complete. 2023-11-08T08:21:22.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.836+0000 7f3491ae7700 1 Processor -- start 2023-11-08T08:21:22.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.836+0000 7f3491ae7700 1 -- start start 2023-11-08T08:21:22.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.836+0000 7f3491ae7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c06d340 0x7f348c1f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:22.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.836+0000 7f3491ae7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348c0b5410 0x7f348c1ef620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:22.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.836+0000 7f3491ae7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348c0ce260 0x7f348c1f3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:22.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.836+0000 7f3491ae7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c15cd40 con 0x7f348c0ce260 2023-11-08T08:21:22.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.836+0000 7f3491ae7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c1f3160 con 0x7f348c06d340 2023-11-08T08:21:22.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.836+0000 7f3491ae7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c15cfc0 con 0x7f348c0b5410 2023-11-08T08:21:22.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.836+0000 7f348bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348c0ce260 0x7f348c1f3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:22.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.836+0000 7f348bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348c0ce260 0x7f348c1f3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51524/0 (socket says 172.21.15.40:51524) 2023-11-08T08:21:22.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.836+0000 7f348bfff700 1 -- 172.21.15.40:0/1162555271 learned_addr learned my addr 172.21.15.40:0/1162555271 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:22.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.836+0000 7f348b7fe700 1 --2- 172.21.15.40:0/1162555271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c06d340 0x7f348c1f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:22.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.836+0000 7f348affd700 1 --2- 172.21.15.40:0/1162555271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348c0b5410 0x7f348c1ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:22.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.840+0000 7f348bfff700 1 -- 172.21.15.40:0/1162555271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348c0b5410 msgr2=0x7f348c1ef620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:22.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.840+0000 7f348bfff700 1 --2- 172.21.15.40:0/1162555271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348c0b5410 0x7f348c1ef620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:22.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.840+0000 7f348bfff700 1 -- 172.21.15.40:0/1162555271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c06d340 msgr2=0x7f348c1f2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:22.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.840+0000 7f348bfff700 1 --2- 172.21.15.40:0/1162555271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c06d340 0x7f348c1f2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:22.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.840+0000 7f348bfff700 1 -- 172.21.15.40:0/1162555271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c009570 con 0x7f348c0ce260 2023-11-08T08:21:22.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.840+0000 7f348bfff700 1 --2- 172.21.15.40:0/1162555271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348c0ce260 0x7f348c1f3360 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f348400cc50 tx=0x7f348400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:22.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.840+0000 7f3473fff700 1 -- 172.21.15.40:0/1162555271 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484003c20 con 0x7f348c0ce260 2023-11-08T08:21:22.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.840+0000 7f3473fff700 1 -- 172.21.15.40:0/1162555271 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3484003d80 con 0x7f348c0ce260 2023-11-08T08:21:22.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.840+0000 7f3491ae7700 1 -- 172.21.15.40:0/1162555271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f348c06c040 con 0x7f348c0ce260 2023-11-08T08:21:22.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.840+0000 7f3473fff700 1 -- 172.21.15.40:0/1162555271 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348400aab0 con 0x7f348c0ce260 2023-11-08T08:21:22.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.840+0000 7f3491ae7700 1 -- 172.21.15.40:0/1162555271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348c0ce8e0 con 0x7f348c0ce260 2023-11-08T08:21:22.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.840+0000 7f3491ae7700 1 -- 172.21.15.40:0/1162555271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348c06c320 con 0x7f348c0ce260 2023-11-08T08:21:22.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.840+0000 7f3473fff700 1 -- 172.21.15.40:0/1162555271 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f3484010050 con 0x7f348c0ce260 2023-11-08T08:21:22.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.844+0000 7f3473fff700 1 --2- 172.21.15.40:0/1162555271 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3478064ee0 0x7f3478067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:22.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.844+0000 7f3473fff700 1 -- 172.21.15.40:0/1162555271 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3484081f30 con 0x7f348c0ce260 2023-11-08T08:21:22.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.844+0000 7f348b7fe700 1 --2- 172.21.15.40:0/1162555271 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3478064ee0 0x7f3478067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:22.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.844+0000 7f348b7fe700 1 --2- 172.21.15.40:0/1162555271 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3478064ee0 0x7f3478067390 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f347c000c00 tx=0x7f347c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:22.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:22.848+0000 7f3473fff700 1 -- 172.21.15.40:0/1162555271 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f348404fe20 con 0x7f348c0ce260 2023-11-08T08:21:23.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.084+0000 7f3491ae7700 1 -- 172.21.15.40:0/1162555271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f348c0726c0 con 0x7f348c0ce260 2023-11-08T08:21:23.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.092+0000 7f3473fff700 1 -- 172.21.15.40:0/1162555271 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 184+0+0 (secure 0 0 0) 0x7f3484053440 con 0x7f348c0ce260 2023-11-08T08:21:23.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.100+0000 7f3491ae7700 1 -- 172.21.15.40:0/1162555271 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3478064ee0 msgr2=0x7f3478067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:23.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.100+0000 7f3491ae7700 1 --2- 172.21.15.40:0/1162555271 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3478064ee0 0x7f3478067390 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f347c000c00 tx=0x7f347c008030 comp rx=0 tx=0).stop 2023-11-08T08:21:23.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.100+0000 7f3491ae7700 1 -- 172.21.15.40:0/1162555271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348c0ce260 msgr2=0x7f348c1f3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:23.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.100+0000 7f3491ae7700 1 --2- 172.21.15.40:0/1162555271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348c0ce260 0x7f348c1f3360 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f348400cc50 tx=0x7f348400cf60 comp rx=0 tx=0).stop 2023-11-08T08:21:23.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.100+0000 7f3491ae7700 1 -- 172.21.15.40:0/1162555271 shutdown_connections 2023-11-08T08:21:23.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.100+0000 7f3491ae7700 1 --2- 172.21.15.40:0/1162555271 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3478064ee0 0x7f3478067390 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:23.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.100+0000 7f3491ae7700 1 --2- 172.21.15.40:0/1162555271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348c0ce260 0x7f348c1f3360 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:23.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.100+0000 7f3491ae7700 1 --2- 172.21.15.40:0/1162555271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f348c0b5410 0x7f348c1ef620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:23.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.100+0000 7f3491ae7700 1 --2- 172.21.15.40:0/1162555271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f348c06d340 0x7f348c1f2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:23.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.100+0000 7f3491ae7700 1 -- 172.21.15.40:0/1162555271 >> 172.21.15.40:0/1162555271 conn(0x7f348c0d3990 msgr2=0x7f348c0b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:23.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.100+0000 7f3491ae7700 1 -- 172.21.15.40:0/1162555271 shutdown_connections 2023-11-08T08:21:23.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.100+0000 7f3491ae7700 1 -- 172.21.15.40:0/1162555271 wait complete. 2023-11-08T08:21:23.108 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2023-11-08T08:21:23.129 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:21:23.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.304+0000 7f4deafe8700 1 Processor -- start 2023-11-08T08:21:23.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.304+0000 7f4deafe8700 1 -- start start 2023-11-08T08:21:23.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4deafe8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4de4071aa0 0x7f4de406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:23.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4deafe8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4de4069e00 0x7f4de406a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:23.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4deafe8700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4de406c600 con 0x7f4de415c640 2023-11-08T08:21:23.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4deafe8700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4de406cef0 con 0x7f4de4069e00 2023-11-08T08:21:23.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4deafe8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de40d38f0 con 0x7f4de4071aa0 2023-11-08T08:21:23.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4dea7e7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4de4069e00 0x7f4de406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:23.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4dea7e7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4de4069e00 0x7f4de406a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51530/0 (socket says 172.21.15.40:51530) 2023-11-08T08:21:23.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4dea7e7700 1 -- 172.21.15.40:0/4247629399 learned_addr learned my addr 172.21.15.40:0/4247629399 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:23.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4de97e5700 1 --2- 172.21.15.40:0/4247629399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4de4071aa0 0x7f4de406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:23.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4dea7e7700 1 -- 172.21.15.40:0/4247629399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4de4071aa0 msgr2=0x7f4de406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:23.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4dea7e7700 1 --2- 172.21.15.40:0/4247629399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4de4071aa0 0x7f4de406d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:23.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4dea7e7700 1 -- 172.21.15.40:0/4247629399 >> v1:172.21.15.169:6789/0 conn(0x7f4de415c640 legacy=0x7f4de40d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:23.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4dea7e7700 1 -- 172.21.15.40:0/4247629399 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de40722f0 con 0x7f4de4069e00 2023-11-08T08:21:23.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4dea7e7700 1 --2- 172.21.15.40:0/4247629399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4de4069e00 0x7f4de406a280 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc009e50 tx=0x7f4ddc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbe26c9bad36643 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:23.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4de8fe4700 1 -- 172.21.15.40:0/4247629399 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc00e070 con 0x7f4de4069e00 2023-11-08T08:21:23.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4de8fe4700 1 -- 172.21.15.40:0/4247629399 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ddc00b9c0 con 0x7f4de4069e00 2023-11-08T08:21:23.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4de8fe4700 1 -- 172.21.15.40:0/4247629399 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc00bd70 con 0x7f4de4069e00 2023-11-08T08:21:23.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4deafe8700 1 -- 172.21.15.40:0/4247629399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4de4069e00 msgr2=0x7f4de406a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:23.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4deafe8700 1 --2- 172.21.15.40:0/4247629399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4de4069e00 0x7f4de406a280 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc009e50 tx=0x7f4ddc00b330 comp rx=0 tx=0).stop 2023-11-08T08:21:23.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4deafe8700 1 -- 172.21.15.40:0/4247629399 shutdown_connections 2023-11-08T08:21:23.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4deafe8700 1 --2- 172.21.15.40:0/4247629399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4de4071aa0 0x7f4de406d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:23.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4deafe8700 1 --2- 172.21.15.40:0/4247629399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4de4069e00 0x7f4de406a280 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:23.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.308+0000 7f4deafe8700 1 -- 172.21.15.40:0/4247629399 >> 172.21.15.40:0/4247629399 conn(0x7f4de40b6520 msgr2=0x7f4de4071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:23.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.312+0000 7f4deafe8700 1 -- 172.21.15.40:0/4247629399 shutdown_connections 2023-11-08T08:21:23.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.312+0000 7f4deafe8700 1 -- 172.21.15.40:0/4247629399 wait complete. 2023-11-08T08:21:23.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.312+0000 7f4deafe8700 1 Processor -- start 2023-11-08T08:21:23.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.312+0000 7f4deafe8700 1 -- start start 2023-11-08T08:21:23.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4deafe8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4de4069e00 0x7f4de40cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:23.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4deafe8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4de4071aa0 0x7f4de40c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:23.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4deafe8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4de415c640 0x7f4de40c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:23.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4deafe8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de40b1860 con 0x7f4de415c640 2023-11-08T08:21:23.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4deafe8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de415dd50 con 0x7f4de4071aa0 2023-11-08T08:21:23.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4deafe8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de40ca6b0 con 0x7f4de4069e00 2023-11-08T08:21:23.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4dea7e7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4de415c640 0x7f4de40c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:23.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4dea7e7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4de415c640 0x7f4de40c64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51544/0 (socket says 172.21.15.40:51544) 2023-11-08T08:21:23.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4dea7e7700 1 -- 172.21.15.40:0/2845144642 learned_addr learned my addr 172.21.15.40:0/2845144642 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:23.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4de9fe6700 1 --2- 172.21.15.40:0/2845144642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4de4069e00 0x7f4de40cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:23.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4de97e5700 1 --2- 172.21.15.40:0/2845144642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4de4071aa0 0x7f4de40c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:23.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4dea7e7700 1 -- 172.21.15.40:0/2845144642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4de4069e00 msgr2=0x7f4de40cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:23.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4dea7e7700 1 --2- 172.21.15.40:0/2845144642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4de4069e00 0x7f4de40cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:23.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4dea7e7700 1 -- 172.21.15.40:0/2845144642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4de4071aa0 msgr2=0x7f4de40c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:23.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4dea7e7700 1 --2- 172.21.15.40:0/2845144642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4de4071aa0 0x7f4de40c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:23.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4dea7e7700 1 -- 172.21.15.40:0/2845144642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc009b30 con 0x7f4de415c640 2023-11-08T08:21:23.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4dea7e7700 1 --2- 172.21.15.40:0/2845144642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4de415c640 0x7f4de40c64c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4de406f090 tx=0x7f4ddc004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:23.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4dda7fc700 1 -- 172.21.15.40:0/2845144642 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc004960 con 0x7f4de415c640 2023-11-08T08:21:23.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4dda7fc700 1 -- 172.21.15.40:0/2845144642 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ddc0099b0 con 0x7f4de415c640 2023-11-08T08:21:23.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4deafe8700 1 -- 172.21.15.40:0/2845144642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de4072010 con 0x7f4de415c640 2023-11-08T08:21:23.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4dda7fc700 1 -- 172.21.15.40:0/2845144642 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc01b670 con 0x7f4de415c640 2023-11-08T08:21:23.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.316+0000 7f4deafe8700 1 -- 172.21.15.40:0/2845144642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de415ca10 con 0x7f4de415c640 2023-11-08T08:21:23.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.320+0000 7f4deafe8700 1 -- 172.21.15.40:0/2845144642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de406c830 con 0x7f4de415c640 2023-11-08T08:21:23.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.320+0000 7f4dda7fc700 1 -- 172.21.15.40:0/2845144642 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4ddc02b070 con 0x7f4de415c640 2023-11-08T08:21:23.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.324+0000 7f4dda7fc700 1 --2- 172.21.15.40:0/2845144642 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4dc8064f30 0x7f4dc80673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:23.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.324+0000 7f4dda7fc700 1 -- 172.21.15.40:0/2845144642 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4ddc0838c0 con 0x7f4de415c640 2023-11-08T08:21:23.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.324+0000 7f4de9fe6700 1 --2- 172.21.15.40:0/2845144642 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4dc8064f30 0x7f4dc80673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:23.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.324+0000 7f4de9fe6700 1 --2- 172.21.15.40:0/2845144642 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4dc8064f30 0x7f4dc80673e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4de0000f60 tx=0x7f4de0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:23.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.328+0000 7f4dda7fc700 1 -- 172.21.15.40:0/2845144642 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4ddc0518b0 con 0x7f4de415c640 2023-11-08T08:21:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.536+0000 7f4deafe8700 1 -- 172.21.15.40:0/2845144642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4de4073cb0 con 0x7f4de415c640 2023-11-08T08:21:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.540+0000 7f4dda7fc700 1 -- 172.21.15.40:0/2845144642 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f4ddc054ed0 con 0x7f4de415c640 2023-11-08T08:21:23.545 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:23.545 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1549516710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1549516710},{"type":"v1","addr":"172.21.15.169:6837","nonce":1549516710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5728,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/912870125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":912870125},{"type":"v1","addr":"172.21.15.169:6835","nonce":912870125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5772,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1045606427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1045606427},{"type":"v1","addr":"172.21.15.40:6834","nonce":1045606427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:21:17.730002+0000","modified":"2023-11-08T08:21:20.752976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2471311394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2471311394},{"type":"v1","addr":"172.21.15.40:6835","nonce":2471311394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T08:21:23.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.540+0000 7f4deafe8700 1 -- 172.21.15.40:0/2845144642 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4dc8064f30 msgr2=0x7f4dc80673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:23.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.540+0000 7f4deafe8700 1 --2- 172.21.15.40:0/2845144642 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4dc8064f30 0x7f4dc80673e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4de0000f60 tx=0x7f4de0009040 comp rx=0 tx=0).stop 2023-11-08T08:21:23.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.540+0000 7f4deafe8700 1 -- 172.21.15.40:0/2845144642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4de415c640 msgr2=0x7f4de40c64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:23.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.540+0000 7f4deafe8700 1 --2- 172.21.15.40:0/2845144642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4de415c640 0x7f4de40c64c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4de406f090 tx=0x7f4ddc004af0 comp rx=0 tx=0).stop 2023-11-08T08:21:23.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.540+0000 7f4deafe8700 1 -- 172.21.15.40:0/2845144642 shutdown_connections 2023-11-08T08:21:23.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.540+0000 7f4deafe8700 1 --2- 172.21.15.40:0/2845144642 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4dc8064f30 0x7f4dc80673e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:23.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.540+0000 7f4deafe8700 1 --2- 172.21.15.40:0/2845144642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4de415c640 0x7f4de40c64c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:23.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.540+0000 7f4deafe8700 1 --2- 172.21.15.40:0/2845144642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4de4071aa0 0x7f4de40c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:23.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.540+0000 7f4deafe8700 1 --2- 172.21.15.40:0/2845144642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4de4069e00 0x7f4de40cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:23.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.540+0000 7f4deafe8700 1 -- 172.21.15.40:0/2845144642 >> 172.21.15.40:0/2845144642 conn(0x7f4de40b6520 msgr2=0x7f4de40d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.540+0000 7f4deafe8700 1 -- 172.21.15.40:0/2845144642 shutdown_connections 2023-11-08T08:21:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:23.540+0000 7f4deafe8700 1 -- 172.21.15.40:0/2845144642 wait complete. 2023-11-08T08:21:23.550 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 25 2023-11-08T08:21:23.971 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T08:21:17.730002+0000', 'modified': '2023-11-08T08:21:20.752976+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5775}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5775': {'gid': 5775, 'name': 'a', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6835/2471311394', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6833', 'nonce': 2471311394}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 2471311394}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T08:21:23.972 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T08:21:23.972 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-08T08:21:24.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.156+0000 7f9ca3466700 1 Processor -- start 2023-11-08T08:21:24.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.156+0000 7f9ca3466700 1 -- start start 2023-11-08T08:21:24.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.160+0000 7f9ca3466700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9c9c070790 0x7f9c9c167110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.160+0000 7f9ca3466700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c9c0adf90 con 0x7f9c9c15c1a0 2023-11-08T08:21:24.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.160+0000 7f9ca3466700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c9c0ce370 con 0x7f9c9c0cdeb0 2023-11-08T08:21:24.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.160+0000 7f9ca3466700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c9c069bf0 con 0x7f9c9c070790 2023-11-08T08:21:24.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.160+0000 7f9ca2464700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9c9c15c1a0 0x7f9c9c06c8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33990/0 (socket says 172.21.15.40:33990) 2023-11-08T08:21:24.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.160+0000 7f9ca2464700 1 -- 172.21.15.40:0/2534218276 learned_addr learned my addr 172.21.15.40:0/2534218276 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:24.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.160+0000 7f9ca2c65700 1 --2- 172.21.15.40:0/2534218276 >> v2:172.21.15.40:3300/0 conn(0x7f9c9c070790 0x7f9c9c167110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.160+0000 7f9ca1462700 1 -- 172.21.15.40:0/2534218276 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1485097603 0 0) 0x7f9c9c0adf90 con 0x7f9c9c15c1a0 2023-11-08T08:21:24.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.160+0000 7f9ca1462700 1 -- 172.21.15.40:0/2534218276 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c880035d0 con 0x7f9c9c15c1a0 2023-11-08T08:21:24.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.160+0000 7f9ca1462700 1 -- 172.21.15.40:0/2534218276 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3832866402 0 0) 0x7f9c9c0ce370 con 0x7f9c9c0cdeb0 2023-11-08T08:21:24.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.160+0000 7f9ca1462700 1 -- 172.21.15.40:0/2534218276 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c9c0adf90 con 0x7f9c9c0cdeb0 2023-11-08T08:21:24.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.160+0000 7f9ca2c65700 1 -- 172.21.15.40:0/2534218276 >> v1:172.21.15.169:6789/0 conn(0x7f9c9c0cdeb0 legacy=0x7f9c9c070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:24.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.160+0000 7f9ca2c65700 1 -- 172.21.15.40:0/2534218276 >> v1:172.21.15.40:6789/0 conn(0x7f9c9c15c1a0 legacy=0x7f9c9c06c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:24.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.160+0000 7f9ca2c65700 1 -- 172.21.15.40:0/2534218276 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9c0cc4a0 con 0x7f9c9c070790 2023-11-08T08:21:24.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.164+0000 7f9ca2c65700 1 --2- 172.21.15.40:0/2534218276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c070790 0x7f9c9c167110 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0ae180 tx=0x7f9c9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99dcdfc680992ab server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:24.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.164+0000 7f9ca1462700 1 -- 172.21.15.40:0/2534218276 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c94016070 con 0x7f9c9c070790 2023-11-08T08:21:24.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.164+0000 7f9ca1462700 1 -- 172.21.15.40:0/2534218276 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9400bb20 con 0x7f9c9c070790 2023-11-08T08:21:24.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.164+0000 7f9ca1462700 1 -- 172.21.15.40:0/2534218276 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9400e070 con 0x7f9c9c070790 2023-11-08T08:21:24.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.164+0000 7f9ca3466700 1 -- 172.21.15.40:0/2534218276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c070790 msgr2=0x7f9c9c167110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:24.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.164+0000 7f9ca3466700 1 --2- 172.21.15.40:0/2534218276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c070790 0x7f9c9c167110 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0ae180 tx=0x7f9c9400b330 comp rx=0 tx=0).stop 2023-11-08T08:21:24.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.164+0000 7f9ca3466700 1 -- 172.21.15.40:0/2534218276 shutdown_connections 2023-11-08T08:21:24.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.164+0000 7f9ca3466700 1 --2- 172.21.15.40:0/2534218276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c070790 0x7f9c9c167110 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.164+0000 7f9ca3466700 1 -- 172.21.15.40:0/2534218276 >> 172.21.15.40:0/2534218276 conn(0x7f9c9c0b0f80 msgr2=0x7f9c9c0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:24.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.164+0000 7f9ca3466700 1 -- 172.21.15.40:0/2534218276 shutdown_connections 2023-11-08T08:21:24.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.164+0000 7f9ca3466700 1 -- 172.21.15.40:0/2534218276 wait complete. 2023-11-08T08:21:24.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.168+0000 7f9ca3466700 1 Processor -- start 2023-11-08T08:21:24.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.168+0000 7f9ca3466700 1 -- start start 2023-11-08T08:21:24.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.168+0000 7f9ca3466700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c9c070790 0x7f9c9c154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.168+0000 7f9ca3466700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c9c0cdeb0 0x7f9c9c159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.168+0000 7f9ca3466700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c15c1a0 0x7f9c9c158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.168+0000 7f9ca3466700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c15b310 con 0x7f9c9c15c1a0 2023-11-08T08:21:24.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.168+0000 7f9ca3466700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c15a690 con 0x7f9c9c0cdeb0 2023-11-08T08:21:24.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.168+0000 7f9ca3466700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c15a970 con 0x7f9c9c070790 2023-11-08T08:21:24.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.168+0000 7f9ca2c65700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c15c1a0 0x7f9c9c158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.168+0000 7f9ca2c65700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c15c1a0 0x7f9c9c158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51566/0 (socket says 172.21.15.40:51566) 2023-11-08T08:21:24.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.168+0000 7f9ca2c65700 1 -- 172.21.15.40:0/796817655 learned_addr learned my addr 172.21.15.40:0/796817655 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:24.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.168+0000 7f9ca1c63700 1 --2- 172.21.15.40:0/796817655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c9c0cdeb0 0x7f9c9c159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.172+0000 7f9ca2464700 1 --2- 172.21.15.40:0/796817655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c9c070790 0x7f9c9c154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.172+0000 7f9ca2c65700 1 -- 172.21.15.40:0/796817655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c9c070790 msgr2=0x7f9c9c154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:24.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.172+0000 7f9ca2c65700 1 --2- 172.21.15.40:0/796817655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c9c070790 0x7f9c9c154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.172+0000 7f9ca2c65700 1 -- 172.21.15.40:0/796817655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c9c0cdeb0 msgr2=0x7f9c9c159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:24.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.172+0000 7f9ca2c65700 1 --2- 172.21.15.40:0/796817655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c9c0cdeb0 0x7f9c9c159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.172+0000 7f9ca2c65700 1 -- 172.21.15.40:0/796817655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c94009a70 con 0x7f9c9c15c1a0 2023-11-08T08:21:24.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.172+0000 7f9ca2c65700 1 --2- 172.21.15.40:0/796817655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c15c1a0 0x7f9c9c158430 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9c94014040 tx=0x7f9c94004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:24.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.172+0000 7f9c92ffd700 1 -- 172.21.15.40:0/796817655 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c94016040 con 0x7f9c9c15c1a0 2023-11-08T08:21:24.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.172+0000 7f9c92ffd700 1 -- 172.21.15.40:0/796817655 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c94004930 con 0x7f9c9c15c1a0 2023-11-08T08:21:24.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.172+0000 7f9ca3466700 1 -- 172.21.15.40:0/796817655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9c15c570 con 0x7f9c9c15c1a0 2023-11-08T08:21:24.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.172+0000 7f9c92ffd700 1 -- 172.21.15.40:0/796817655 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9401b680 con 0x7f9c9c15c1a0 2023-11-08T08:21:24.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.172+0000 7f9ca3466700 1 -- 172.21.15.40:0/796817655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9c0cc780 con 0x7f9c9c15c1a0 2023-11-08T08:21:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.176+0000 7f9c92ffd700 1 -- 172.21.15.40:0/796817655 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f9c94023030 con 0x7f9c9c15c1a0 2023-11-08T08:21:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.176+0000 7f9c92ffd700 1 --2- 172.21.15.40:0/796817655 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9c80064ee0 0x7f9c80067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.176+0000 7f9c92ffd700 1 -- 172.21.15.40:0/796817655 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9c94083540 con 0x7f9c9c15c1a0 2023-11-08T08:21:24.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.176+0000 7f9ca2464700 1 --2- 172.21.15.40:0/796817655 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9c80064ee0 0x7f9c80067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.180+0000 7f9ca2464700 1 --2- 172.21.15.40:0/796817655 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9c80064ee0 0x7f9c80067390 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9c980011e0 tx=0x7f9c98009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:24.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.180+0000 7f9ca794e740 1 Processor -- start 2023-11-08T08:21:24.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.180+0000 7f9ca794e740 1 -- start start 2023-11-08T08:21:24.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.180+0000 7f9ca794e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2917ae0 0x2bc7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.180+0000 7f9ca794e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2917710 0x2bc8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.180+0000 7f9ca794e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2962790 0x2bc8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.180+0000 7f9ca794e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a86ae0 con 0x2962790 2023-11-08T08:21:24.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.180+0000 7f9ca794e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x2af6bd0 con 0x2917710 2023-11-08T08:21:24.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.180+0000 7f9ca794e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x2aefed0 con 0x2917ae0 2023-11-08T08:21:24.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.180+0000 7f9ca2c65700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2962790 0x2bc8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.180+0000 7f9ca2c65700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2962790 0x2bc8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51568/0 (socket says 172.21.15.40:51568) 2023-11-08T08:21:24.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.180+0000 7f9ca2c65700 1 -- 172.21.15.40:0/2059449334 learned_addr learned my addr 172.21.15.40:0/2059449334 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:24.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.180+0000 7f9ca2464700 1 --2- 172.21.15.40:0/2059449334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2917ae0 0x2bc7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.180+0000 7f9ca1c63700 1 --2- 172.21.15.40:0/2059449334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2917710 0x2bc8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.184+0000 7f9ca2c65700 1 -- 172.21.15.40:0/2059449334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2917ae0 msgr2=0x2bc7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:24.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.184+0000 7f9ca2c65700 1 --2- 172.21.15.40:0/2059449334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2917ae0 0x2bc7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.184+0000 7f9ca2c65700 1 -- 172.21.15.40:0/2059449334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2917710 msgr2=0x2bc8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:24.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.184+0000 7f9ca2c65700 1 --2- 172.21.15.40:0/2059449334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2917710 0x2bc8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.184+0000 7f9ca2c65700 1 -- 172.21.15.40:0/2059449334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2b15cc0 con 0x2962790 2023-11-08T08:21:24.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.184+0000 7f9ca2c65700 1 --2- 172.21.15.40:0/2059449334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2962790 0x2bc8880 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9c94054b80 tx=0x7f9c94054bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de9fabbe0f31bde server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:24.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.184+0000 7f9c7ffff700 1 -- 172.21.15.40:0/2059449334 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c94019e00 con 0x2962790 2023-11-08T08:21:24.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.184+0000 7f9c7ffff700 1 -- 172.21.15.40:0/2059449334 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c94053370 con 0x2962790 2023-11-08T08:21:24.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.184+0000 7f9c7ffff700 1 -- 172.21.15.40:0/2059449334 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c940832c0 con 0x2962790 2023-11-08T08:21:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.184+0000 7f9ca794e740 1 -- 172.21.15.40:0/2059449334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2962790 msgr2=0x2bc8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.184+0000 7f9ca794e740 1 --2- 172.21.15.40:0/2059449334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2962790 0x2bc8880 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9c94054b80 tx=0x7f9c94054bb0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.184+0000 7f9ca794e740 1 -- 172.21.15.40:0/2059449334 shutdown_connections 2023-11-08T08:21:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.184+0000 7f9ca794e740 1 --2- 172.21.15.40:0/2059449334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2962790 0x2bc8880 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.184+0000 7f9ca794e740 1 --2- 172.21.15.40:0/2059449334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2917710 0x2bc8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.184+0000 7f9ca794e740 1 --2- 172.21.15.40:0/2059449334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2917ae0 0x2bc7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.184+0000 7f9ca794e740 1 -- 172.21.15.40:0/2059449334 >> 172.21.15.40:0/2059449334 conn(0x2917eb0 msgr2=0x2bb0700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:24.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.184+0000 7f9ca794e740 1 -- 172.21.15.40:0/2059449334 shutdown_connections 2023-11-08T08:21:24.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.184+0000 7f9ca794e740 1 -- 172.21.15.40:0/2059449334 wait complete. 2023-11-08T08:21:24.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.184+0000 7f9ca794e740 1 -- start start 2023-11-08T08:21:24.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.184+0000 7f9ca794e740 1 Processor -- start 2023-11-08T08:21:24.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9ca794e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2917710 0x2bc55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9ca794e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2917ae0 0x2b842b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9ca794e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2962790 0x2b847f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9ca794e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x299cb20 con 0x2917ae0 2023-11-08T08:21:24.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9ca794e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x2bc5b30 con 0x2917710 2023-11-08T08:21:24.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9ca794e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x2b84ff0 con 0x2962790 2023-11-08T08:21:24.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9ca794e740 10 client.0 _ll_register_callbacks cb 0x2b4c0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:21:24.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9ca1c63700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2917ae0 0x2b842b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9ca1c63700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2917ae0 0x2b842b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51574/0 (socket says 172.21.15.40:51574) 2023-11-08T08:21:24.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9ca1c63700 1 -- 172.21.15.40:0/2159001420 learned_addr learned my addr 172.21.15.40:0/2159001420 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:24.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9ca2464700 1 --2- 172.21.15.40:0/2159001420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2917710 0x2bc55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9ca2c65700 1 --2- 172.21.15.40:0/2159001420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2962790 0x2b847f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9ca1c63700 1 -- 172.21.15.40:0/2159001420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2962790 msgr2=0x2b847f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:24.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9ca1c63700 1 --2- 172.21.15.40:0/2159001420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2962790 0x2b847f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9ca1c63700 1 -- 172.21.15.40:0/2159001420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2917710 msgr2=0x2bc55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:24.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9ca1c63700 1 --2- 172.21.15.40:0/2159001420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2917710 0x2bc55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9ca1c63700 1 -- 172.21.15.40:0/2159001420 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9c0cc780 con 0x2917ae0 2023-11-08T08:21:24.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9ca1c63700 1 --2- 172.21.15.40:0/2159001420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2917ae0 0x2b842b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c0010e0 tx=0x7f9c8c0127d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daac8a6500b520fd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:24.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9c7d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:21:24.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9c7d7fa700 1 -- 172.21.15.40:0/2159001420 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c0050d0 con 0x2917ae0 2023-11-08T08:21:24.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9c7d7fa700 1 -- 172.21.15.40:0/2159001420 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9c8c005230 con 0x2917ae0 2023-11-08T08:21:24.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.188+0000 7f9ca794e740 1 -- 172.21.15.40:0/2159001420 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2ae8c70 con 0x2917ae0 2023-11-08T08:21:24.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.192+0000 7f9c7d7fa700 1 -- 172.21.15.40:0/2159001420 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c8c0053f0 con 0x2917ae0 2023-11-08T08:21:24.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.192+0000 7f9ca794e740 10 client.5940 fetch_fsmap learned FSMap version 25 2023-11-08T08:21:24.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.192+0000 7f9ca794e740 1 -- 172.21.15.40:0/2159001420 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x29639e0 con 0x2917ae0 2023-11-08T08:21:24.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.192+0000 7f9c7d7fa700 1 -- 172.21.15.40:0/2159001420 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2ae8c70 con 0x2917ae0 2023-11-08T08:21:24.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.192+0000 7f9c7d7fa700 1 -- 172.21.15.40:0/2159001420 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9c8c01e3f0 con 0x2917ae0 2023-11-08T08:21:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.192+0000 7f9ca794e740 10 client.5940 fetch_fsmap finished waiting for FSMap version 25 2023-11-08T08:21:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.192+0000 7f9ca794e740 10 client.5940 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-08T08:21:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.192+0000 7f9ca794e740 20 client.5940 populate_metadata read hostname 'smithi040' 2023-11-08T08:21:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.192+0000 7f9ca794e740 1 --2- 172.21.15.40:0/2159001420 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x2b866d0 0x2b895e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.192+0000 7f9ca794e740 4 client.5940 mds_command: new command op to 5775 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-08T08:21:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.192+0000 7f9ca794e740 1 -- 172.21.15.40:0/2159001420 --> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x2994260 con 0x2b866d0 2023-11-08T08:21:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.192+0000 7f9ca2464700 1 --2- 172.21.15.40:0/2159001420 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x2b866d0 0x2b895e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.192+0000 7f9ca2464700 1 --2- 172.21.15.40:0/2159001420 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x2b866d0 0x2b895e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5bd3a8c5daca051 server_cookie=8bf79678279590b in_seq=0 out_seq=0 2023-11-08T08:21:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.192+0000 7f9c7d7fa700 10 client.5940 ms_handle_connect on v2:172.21.15.40:6833/2471311394 2023-11-08T08:21:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.196+0000 7f9c7d7fa700 1 -- 172.21.15.40:0/2159001420 <== mds.0 v2:172.21.15.40:6833/2471311394 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x2994260 con 0x2b866d0 2023-11-08T08:21:24.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.196+0000 7f9c7d7fa700 10 client.5940 handle_command_reply: tid=0 2023-11-08T08:21:24.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.196+0000 7f9ca794e740 1 client.5940 shutdown 2023-11-08T08:21:24.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.196+0000 7f9ca794e740 1 -- 172.21.15.40:0/2159001420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2917ae0 msgr2=0x2b842b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:24.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.196+0000 7f9ca794e740 1 --2- 172.21.15.40:0/2159001420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2917ae0 0x2b842b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c0010e0 tx=0x7f9c8c0127d0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.196+0000 7f9ca794e740 1 -- 172.21.15.40:0/2159001420 shutdown_connections 2023-11-08T08:21:24.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.196+0000 7f9ca794e740 1 --2- 172.21.15.40:0/2159001420 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x2b866d0 0x2b895e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.196+0000 7f9ca794e740 1 --2- 172.21.15.40:0/2159001420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2962790 0x2b847f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.196+0000 7f9ca794e740 1 --2- 172.21.15.40:0/2159001420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2917ae0 0x2b842b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.196+0000 7f9ca794e740 1 --2- 172.21.15.40:0/2159001420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2917710 0x2bc55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.196+0000 7f9c7d7fa700 0 client.5940 ms_handle_reset on v2:172.21.15.40:6833/2471311394 2023-11-08T08:21:24.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.196+0000 7f9ca794e740 1 -- 172.21.15.40:0/2159001420 >> 172.21.15.40:0/2159001420 conn(0x2917eb0 msgr2=0x2bb06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.196+0000 7f9ca794e740 1 -- 172.21.15.40:0/2159001420 shutdown_connections 2023-11-08T08:21:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.196+0000 7f9ca794e740 1 -- 172.21.15.40:0/2159001420 wait complete. 2023-11-08T08:21:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.196+0000 7f9ca794e740 20 client.5940 trim_cache size 0 max 16384 2023-11-08T08:21:24.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.216+0000 7f9ca794e740 1 Processor -- start 2023-11-08T08:21:24.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.216+0000 7f9ca794e740 1 -- start start 2023-11-08T08:21:24.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.216+0000 7f9ca794e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2917710 0x2bc39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.216+0000 7f9ca794e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2917ae0 0x2bc3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.216+0000 7f9ca794e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2962790 0x2b82e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.216+0000 7f9ca794e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x2aeed60 con 0x2917ae0 2023-11-08T08:21:24.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.216+0000 7f9ca794e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a86ae0 con 0x2962790 2023-11-08T08:21:24.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.216+0000 7f9ca794e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x2ac5de0 con 0x2917710 2023-11-08T08:21:24.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.216+0000 7f9ca1c63700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2917ae0 0x2bc3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.216+0000 7f9ca2464700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2917710 0x2bc39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.216+0000 7f9ca2464700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2917710 0x2bc39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:39644/0 (socket says 172.21.15.40:39644) 2023-11-08T08:21:24.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.216+0000 7f9ca2464700 1 -- 172.21.15.40:0/573325983 learned_addr learned my addr 172.21.15.40:0/573325983 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:24.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.220+0000 7f9ca2c65700 1 --2- 172.21.15.40:0/573325983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2962790 0x2b82e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.220+0000 7f9ca2464700 1 -- 172.21.15.40:0/573325983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2962790 msgr2=0x2b82e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:24.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.220+0000 7f9ca2464700 1 --2- 172.21.15.40:0/573325983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2962790 0x2b82e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.220+0000 7f9ca2464700 1 -- 172.21.15.40:0/573325983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2917ae0 msgr2=0x2bc3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:24.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.220+0000 7f9ca2464700 1 --2- 172.21.15.40:0/573325983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2917ae0 0x2bc3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.220+0000 7f9ca2464700 1 -- 172.21.15.40:0/573325983 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2ab68a0 con 0x2917710 2023-11-08T08:21:24.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.220+0000 7f9ca2c65700 1 --2- 172.21.15.40:0/573325983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2962790 0x2b82e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:24.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.220+0000 7f9ca2464700 1 --2- 172.21.15.40:0/573325983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2917710 0x2bc39c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x2b86e90 tx=0x7f9c98012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cf0a3c2215d5411 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:24.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.220+0000 7f9c90ff9700 1 -- 172.21.15.40:0/573325983 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c980075b0 con 0x2917710 2023-11-08T08:21:24.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.220+0000 7f9c90ff9700 1 -- 172.21.15.40:0/573325983 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c98007710 con 0x2917710 2023-11-08T08:21:24.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.220+0000 7f9c90ff9700 1 -- 172.21.15.40:0/573325983 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c980262b0 con 0x2917710 2023-11-08T08:21:24.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.220+0000 7f9ca794e740 1 -- 172.21.15.40:0/573325983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2917710 msgr2=0x2bc39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:24.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.220+0000 7f9ca794e740 1 --2- 172.21.15.40:0/573325983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2917710 0x2bc39c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x2b86e90 tx=0x7f9c98012000 comp rx=0 tx=0).stop 2023-11-08T08:21:24.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.220+0000 7f9ca794e740 1 -- 172.21.15.40:0/573325983 shutdown_connections 2023-11-08T08:21:24.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.220+0000 7f9ca794e740 1 --2- 172.21.15.40:0/573325983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2962790 0x2b82e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.220+0000 7f9ca794e740 1 --2- 172.21.15.40:0/573325983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2917ae0 0x2bc3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.220+0000 7f9ca794e740 1 --2- 172.21.15.40:0/573325983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2917710 0x2bc39c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.220+0000 7f9ca794e740 1 -- 172.21.15.40:0/573325983 >> 172.21.15.40:0/573325983 conn(0x2917eb0 msgr2=0x2bde320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:24.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.220+0000 7f9ca794e740 1 -- 172.21.15.40:0/573325983 shutdown_connections 2023-11-08T08:21:24.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.220+0000 7f9ca794e740 1 -- 172.21.15.40:0/573325983 wait complete. 2023-11-08T08:21:24.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.224+0000 7f9ca794e740 1 -- start start 2023-11-08T08:21:24.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.224+0000 7f9ca794e740 1 Processor -- start 2023-11-08T08:21:24.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.224+0000 7f9ca794e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2917710 0x2c2abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.224+0000 7f9ca794e740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2917ae0 0x2bc40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.224+0000 7f9ca794e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2962790 0x2bc4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.224+0000 7f9ca794e740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x2aed2d0 con 0x2917ae0 2023-11-08T08:21:24.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.224+0000 7f9ca794e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a86130 con 0x2917710 2023-11-08T08:21:24.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.224+0000 7f9ca794e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x2af2a30 con 0x2962790 2023-11-08T08:21:24.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.224+0000 7f9ca794e740 10 client.0 _ll_register_callbacks cb 0x2a9c2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:21:24.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.224+0000 7f9ca1c63700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2917ae0 0x2bc40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.224+0000 7f9ca1c63700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2917ae0 0x2bc40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51590/0 (socket says 172.21.15.40:51590) 2023-11-08T08:21:24.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.224+0000 7f9ca1c63700 1 -- 172.21.15.40:0/1709054673 learned_addr learned my addr 172.21.15.40:0/1709054673 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:24.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.224+0000 7f9ca2464700 1 --2- 172.21.15.40:0/1709054673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2917710 0x2c2abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.224+0000 7f9ca2c65700 1 --2- 172.21.15.40:0/1709054673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2962790 0x2bc4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9ca1c63700 1 -- 172.21.15.40:0/1709054673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2962790 msgr2=0x2bc4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:24.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9ca1c63700 1 --2- 172.21.15.40:0/1709054673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2962790 0x2bc4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9ca1c63700 1 -- 172.21.15.40:0/1709054673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2917710 msgr2=0x2c2abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:24.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9ca1c63700 1 --2- 172.21.15.40:0/1709054673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2917710 0x2c2abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9ca1c63700 1 -- 172.21.15.40:0/1709054673 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9800f040 con 0x2917ae0 2023-11-08T08:21:24.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9ca1c63700 1 --2- 172.21.15.40:0/1709054673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2917ae0 0x2bc40d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c01e650 tx=0x7f9c8c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4f09a08cc853b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:24.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9c7d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:21:24.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9c7d7fa700 1 -- 172.21.15.40:0/1709054673 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c0054e0 con 0x2917ae0 2023-11-08T08:21:24.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9c7d7fa700 1 -- 172.21.15.40:0/1709054673 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9c8c01e950 con 0x2917ae0 2023-11-08T08:21:24.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9ca794e740 1 -- 172.21.15.40:0/1709054673 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2ae8f50 con 0x2917ae0 2023-11-08T08:21:24.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9c7d7fa700 1 -- 172.21.15.40:0/1709054673 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c8c012b20 con 0x2917ae0 2023-11-08T08:21:24.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9ca794e740 10 client.5946 fetch_fsmap learned FSMap version 25 2023-11-08T08:21:24.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9ca794e740 1 -- 172.21.15.40:0/1709054673 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x2a9c380 con 0x2917ae0 2023-11-08T08:21:24.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9c7d7fa700 1 -- 172.21.15.40:0/1709054673 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2ae8f50 con 0x2917ae0 2023-11-08T08:21:24.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9c7d7fa700 1 -- 172.21.15.40:0/1709054673 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9c8c006880 con 0x2917ae0 2023-11-08T08:21:24.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9ca794e740 10 client.5946 fetch_fsmap finished waiting for FSMap version 25 2023-11-08T08:21:24.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9ca794e740 10 client.5946 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-08T08:21:24.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9ca794e740 20 client.5946 populate_metadata read hostname 'smithi040' 2023-11-08T08:21:24.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9ca794e740 1 --2- 172.21.15.40:0/1709054673 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x2be33d0 0x2be4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9ca794e740 4 client.5946 mds_command: new command op to 5775 tid=0[{"prefix": "status"}] 2023-11-08T08:21:24.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9ca794e740 1 -- 172.21.15.40:0/1709054673 --> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] -- command(tid 0: {"prefix": "status"}) v1 -- 0x2925260 con 0x2be33d0 2023-11-08T08:21:24.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9ca2464700 1 --2- 172.21.15.40:0/1709054673 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x2be33d0 0x2be4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9c7d7fa700 10 client.5946 ms_handle_connect on v2:172.21.15.40:6833/2471311394 2023-11-08T08:21:24.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.228+0000 7f9ca2464700 1 --2- 172.21.15.40:0/1709054673 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x2be33d0 0x2be4af0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3eabfcda2045fa4 server_cookie=9fd3f1c0a7fc1447 in_seq=0 out_seq=0 2023-11-08T08:21:24.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9c7d7fa700 1 -- 172.21.15.40:0/1709054673 <== mds.0 v2:172.21.15.40:6833/2471311394 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x2925260 con 0x2be33d0 2023-11-08T08:21:24.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9c7d7fa700 10 client.5946 handle_command_reply: tid=0 2023-11-08T08:21:24.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca794e740 1 client.5946 shutdown 2023-11-08T08:21:24.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca794e740 1 -- 172.21.15.40:0/1709054673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2917ae0 msgr2=0x2bc40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:24.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca794e740 1 --2- 172.21.15.40:0/1709054673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2917ae0 0x2bc40d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c01e650 tx=0x7f9c8c015000 comp rx=0 tx=0).stop 2023-11-08T08:21:24.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca794e740 1 -- 172.21.15.40:0/1709054673 shutdown_connections 2023-11-08T08:21:24.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca794e740 1 --2- 172.21.15.40:0/1709054673 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x2be33d0 0x2be4af0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca794e740 1 --2- 172.21.15.40:0/1709054673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2962790 0x2bc4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca794e740 1 --2- 172.21.15.40:0/1709054673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2917ae0 0x2bc40d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9c7d7fa700 0 client.5946 ms_handle_reset on v2:172.21.15.40:6833/2471311394 2023-11-08T08:21:24.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca794e740 1 --2- 172.21.15.40:0/1709054673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2917710 0x2c2abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca794e740 1 -- 172.21.15.40:0/1709054673 >> 172.21.15.40:0/1709054673 conn(0x2917eb0 msgr2=0x2bc39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:24.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca794e740 1 -- 172.21.15.40:0/1709054673 shutdown_connections 2023-11-08T08:21:24.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca794e740 1 -- 172.21.15.40:0/1709054673 wait complete. 2023-11-08T08:21:24.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca794e740 20 client.5946 trim_cache size 0 max 16384 2023-11-08T08:21:24.247 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:21:24.247 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "54ed45f4-d5b0-41c3-8c3b-b5a18b844f25", 2023-11-08T08:21:24.247 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2023-11-08T08:21:24.248 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 5775, 2023-11-08T08:21:24.248 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-11-08T08:21:24.248 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-11-08T08:21:24.248 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-11-08T08:21:24.248 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 6.4993611629999997, 2023-11-08T08:21:24.249 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 25, 2023-11-08T08:21:24.249 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 57, 2023-11-08T08:21:24.249 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 57, 2023-11-08T08:21:24.249 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 12.697773419000001 2023-11-08T08:21:24.249 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:21:24.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca3466700 1 -- 172.21.15.40:0/796817655 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9c80064ee0 msgr2=0x7f9c80067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:24.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca3466700 1 --2- 172.21.15.40:0/796817655 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9c80064ee0 0x7f9c80067390 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9c980011e0 tx=0x7f9c98009040 comp rx=0 tx=0).stop 2023-11-08T08:21:24.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca3466700 1 -- 172.21.15.40:0/796817655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c15c1a0 msgr2=0x7f9c9c158430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:24.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca3466700 1 --2- 172.21.15.40:0/796817655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c15c1a0 0x7f9c9c158430 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9c94014040 tx=0x7f9c94004af0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca3466700 1 -- 172.21.15.40:0/796817655 shutdown_connections 2023-11-08T08:21:24.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca3466700 1 --2- 172.21.15.40:0/796817655 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9c80064ee0 0x7f9c80067390 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca3466700 1 --2- 172.21.15.40:0/796817655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c9c15c1a0 0x7f9c9c158430 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca3466700 1 --2- 172.21.15.40:0/796817655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c9c0cdeb0 0x7f9c9c159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca3466700 1 --2- 172.21.15.40:0/796817655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c9c070790 0x7f9c9c154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca3466700 1 -- 172.21.15.40:0/796817655 >> 172.21.15.40:0/796817655 conn(0x7f9c9c0b0f80 msgr2=0x7f9c9c1605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:24.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca3466700 1 -- 172.21.15.40:0/796817655 shutdown_connections 2023-11-08T08:21:24.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.232+0000 7f9ca3466700 1 -- 172.21.15.40:0/796817655 wait complete. 2023-11-08T08:21:24.262 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T08:21:24.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.440+0000 7f4022db1700 1 Processor -- start 2023-11-08T08:21:24.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.444+0000 7f4022db1700 1 -- start start 2023-11-08T08:21:24.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.444+0000 7f4022db1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f401c163cb0 0x7f401c164cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4022db1700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f401c0cdfb0 0x7f401c06cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4022db1700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f401c06d2f0 0x7f401c06bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4022db1700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f401c0cc290 con 0x7f401c163cb0 2023-11-08T08:21:24.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4022db1700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f401c06d770 con 0x7f401c06d2f0 2023-11-08T08:21:24.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4022db1700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c164080 con 0x7f401c0cdfb0 2023-11-08T08:21:24.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4020b4d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f401c163cb0 0x7f401c164cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4020b4d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f401c163cb0 0x7f401c164cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51596/0 (socket says 172.21.15.40:51596) 2023-11-08T08:21:24.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4020b4d700 1 -- 172.21.15.40:0/3188731579 learned_addr learned my addr 172.21.15.40:0/3188731579 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:24.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f402134e700 1 --2- 172.21.15.40:0/3188731579 >> v2:172.21.15.169:3300/0 conn(0x7f401c06d2f0 0x7f401c06bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4013fff700 1 --2- 172.21.15.40:0/3188731579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f401c0cdfb0 0x7f401c06cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4013fff700 1 -- 172.21.15.40:0/3188731579 >> v2:172.21.15.169:3300/0 conn(0x7f401c06d2f0 msgr2=0x7f401c06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:24.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4013fff700 1 --2- 172.21.15.40:0/3188731579 >> v2:172.21.15.169:3300/0 conn(0x7f401c06d2f0 0x7f401c06bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4013fff700 1 -- 172.21.15.40:0/3188731579 >> v2:172.21.15.40:3300/0 conn(0x7f401c163cb0 msgr2=0x7f401c164cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:24.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4013fff700 1 --2- 172.21.15.40:0/3188731579 >> v2:172.21.15.40:3300/0 conn(0x7f401c163cb0 0x7f401c164cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4013fff700 1 -- 172.21.15.40:0/3188731579 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c069bf0 con 0x7f401c0cdfb0 2023-11-08T08:21:24.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4020b4d700 1 --2- 172.21.15.40:0/3188731579 >> v2:172.21.15.40:3300/0 conn(0x7f401c163cb0 0x7f401c164cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:21:24.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4013fff700 1 --2- 172.21.15.40:0/3188731579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f401c0cdfb0 0x7f401c06cdb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f401c06f370 tx=0x7f400c009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efb2abd737987927 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:24.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f40137fe700 1 -- 172.21.15.40:0/3188731579 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c011830 con 0x7f401c0cdfb0 2023-11-08T08:21:24.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f40137fe700 1 -- 172.21.15.40:0/3188731579 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400c011990 con 0x7f401c0cdfb0 2023-11-08T08:21:24.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f40137fe700 1 -- 172.21.15.40:0/3188731579 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c011d20 con 0x7f401c0cdfb0 2023-11-08T08:21:24.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4022db1700 1 -- 172.21.15.40:0/3188731579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f401c0cdfb0 msgr2=0x7f401c06cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:24.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4022db1700 1 --2- 172.21.15.40:0/3188731579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f401c0cdfb0 0x7f401c06cdb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f401c06f370 tx=0x7f400c009b30 comp rx=0 tx=0).stop 2023-11-08T08:21:24.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4022db1700 1 -- 172.21.15.40:0/3188731579 shutdown_connections 2023-11-08T08:21:24.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4022db1700 1 --2- 172.21.15.40:0/3188731579 >> v2:172.21.15.169:3300/0 conn(0x7f401c06d2f0 0x7f401c06bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4022db1700 1 --2- 172.21.15.40:0/3188731579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f401c0cdfb0 0x7f401c06cdb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4022db1700 1 --2- 172.21.15.40:0/3188731579 >> v2:172.21.15.40:3300/0 conn(0x7f401c163cb0 0x7f401c164cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4022db1700 1 -- 172.21.15.40:0/3188731579 >> 172.21.15.40:0/3188731579 conn(0x7f401c0cf350 msgr2=0x7f401c0b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:24.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4022db1700 1 -- 172.21.15.40:0/3188731579 shutdown_connections 2023-11-08T08:21:24.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.448+0000 7f4022db1700 1 -- 172.21.15.40:0/3188731579 wait complete. 2023-11-08T08:21:24.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.452+0000 7f4022db1700 1 Processor -- start 2023-11-08T08:21:24.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.452+0000 7f4022db1700 1 -- start start 2023-11-08T08:21:24.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.452+0000 7f4022db1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f401c06d2f0 0x7f401c14c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.452+0000 7f4022db1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f401c0cdfb0 0x7f401c14c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.452+0000 7f4022db1700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f401c163cb0 0x7f401c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.452+0000 7f4022db1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0afc60 con 0x7f401c0cdfb0 2023-11-08T08:21:24.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.452+0000 7f4022db1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0af050 con 0x7f401c06d2f0 2023-11-08T08:21:24.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.452+0000 7f4022db1700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0af330 con 0x7f401c163cb0 2023-11-08T08:21:24.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.456+0000 7f4013fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f401c0cdfb0 0x7f401c14c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.456+0000 7f402134e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f401c163cb0 0x7f401c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.456+0000 7f4013fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f401c0cdfb0 0x7f401c14c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51604/0 (socket says 172.21.15.40:51604) 2023-11-08T08:21:24.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.456+0000 7f4013fff700 1 -- 172.21.15.40:0/3864541543 learned_addr learned my addr 172.21.15.40:0/3864541543 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:24.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.456+0000 7f4020b4d700 1 --2- 172.21.15.40:0/3864541543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f401c06d2f0 0x7f401c14c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.456+0000 7f402134e700 1 -- 172.21.15.40:0/3864541543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f401c06d2f0 msgr2=0x7f401c14c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:24.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.456+0000 7f402134e700 1 --2- 172.21.15.40:0/3864541543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f401c06d2f0 0x7f401c14c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.456+0000 7f402134e700 1 -- 172.21.15.40:0/3864541543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f401c0cdfb0 msgr2=0x7f401c14c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:24.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.456+0000 7f402134e700 1 --2- 172.21.15.40:0/3864541543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f401c0cdfb0 0x7f401c14c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.456+0000 7f402134e700 1 -- 172.21.15.40:0/3864541543 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4018009b30 con 0x7f401c163cb0 2023-11-08T08:21:24.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.456+0000 7f4013fff700 1 --2- 172.21.15.40:0/3864541543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f401c0cdfb0 0x7f401c14c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:21:24.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.456+0000 7f402134e700 1 --2- 172.21.15.40:0/3864541543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f401c163cb0 0x7f401c0b3260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f401400e7f0 tx=0x7f401400eb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:24.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.456+0000 7f40117fa700 1 -- 172.21.15.40:0/3864541543 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4014013680 con 0x7f401c163cb0 2023-11-08T08:21:24.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.456+0000 7f40117fa700 1 -- 172.21.15.40:0/3864541543 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4014013800 con 0x7f401c163cb0 2023-11-08T08:21:24.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.456+0000 7f40117fa700 1 -- 172.21.15.40:0/3864541543 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4014013bb0 con 0x7f401c163cb0 2023-11-08T08:21:24.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.456+0000 7f4022db1700 1 -- 172.21.15.40:0/3864541543 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400c0094b0 con 0x7f401c163cb0 2023-11-08T08:21:24.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.456+0000 7f4022db1700 1 -- 172.21.15.40:0/3864541543 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401c0cc4a0 con 0x7f401c163cb0 2023-11-08T08:21:24.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.460+0000 7f4022db1700 1 -- 172.21.15.40:0/3864541543 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f401c1651f0 con 0x7f401c163cb0 2023-11-08T08:21:24.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.460+0000 7f40117fa700 1 -- 172.21.15.40:0/3864541543 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4014019030 con 0x7f401c163cb0 2023-11-08T08:21:24.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.460+0000 7f40117fa700 1 --2- 172.21.15.40:0/3864541543 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4008064ee0 0x7f4008067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:24.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.460+0000 7f40117fa700 1 -- 172.21.15.40:0/3864541543 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f40140819c0 con 0x7f401c163cb0 2023-11-08T08:21:24.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.460+0000 7f4020b4d700 1 --2- 172.21.15.40:0/3864541543 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4008064ee0 0x7f4008067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:24.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.464+0000 7f4020b4d700 1 --2- 172.21.15.40:0/3864541543 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4008064ee0 0x7f4008067390 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f40180072c0 tx=0x7f401800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:24.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.468+0000 7f40117fa700 1 -- 172.21.15.40:0/3864541543 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f401404f930 con 0x7f401c163cb0 2023-11-08T08:21:24.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.680+0000 7f4022db1700 1 -- 172.21.15.40:0/3864541543 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f401c0cc780 con 0x7f401c163cb0 2023-11-08T08:21:24.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.680+0000 7f40117fa700 1 -- 172.21.15.40:0/3864541543 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4014052f50 con 0x7f401c163cb0 2023-11-08T08:21:24.687 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T08:21:24.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.684+0000 7f4022db1700 1 -- 172.21.15.40:0/3864541543 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4008064ee0 msgr2=0x7f4008067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:24.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.684+0000 7f4022db1700 1 --2- 172.21.15.40:0/3864541543 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4008064ee0 0x7f4008067390 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f40180072c0 tx=0x7f401800b040 comp rx=0 tx=0).stop 2023-11-08T08:21:24.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.684+0000 7f4022db1700 1 -- 172.21.15.40:0/3864541543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f401c163cb0 msgr2=0x7f401c0b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:24.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.684+0000 7f4022db1700 1 --2- 172.21.15.40:0/3864541543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f401c163cb0 0x7f401c0b3260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f401400e7f0 tx=0x7f401400eb00 comp rx=0 tx=0).stop 2023-11-08T08:21:24.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.684+0000 7f4022db1700 1 -- 172.21.15.40:0/3864541543 shutdown_connections 2023-11-08T08:21:24.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.684+0000 7f4022db1700 1 --2- 172.21.15.40:0/3864541543 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4008064ee0 0x7f4008067390 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.684+0000 7f4022db1700 1 --2- 172.21.15.40:0/3864541543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f401c163cb0 0x7f401c0b3260 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.684+0000 7f4022db1700 1 --2- 172.21.15.40:0/3864541543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f401c0cdfb0 0x7f401c14c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.684+0000 7f4022db1700 1 --2- 172.21.15.40:0/3864541543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f401c06d2f0 0x7f401c14c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:24.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.684+0000 7f4022db1700 1 -- 172.21.15.40:0/3864541543 >> 172.21.15.40:0/3864541543 conn(0x7f401c0cf350 msgr2=0x7f401c162de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:24.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.684+0000 7f4022db1700 1 -- 172.21.15.40:0/3864541543 shutdown_connections 2023-11-08T08:21:24.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:24.684+0000 7f4022db1700 1 -- 172.21.15.40:0/3864541543 wait complete. 2023-11-08T08:21:24.715 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-08T08:21:24.715 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T08:21:24.716 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-11-08T08:21:24.716 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T08:21:24.716 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T08:21:24.717 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-08T08:21:24.717 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-08T08:21:24.717 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T08:21:24.718 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T08:21:24.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:21:24.718 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-11-08T08:21:24.734 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T08:21:24.734 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T08:21:24.734 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T08:21:24.735 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:21:24.735 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-11-08T08:21:24.735 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:21:24.735 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:21:24.735 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:21:24.736 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2023-11-08T08:21:24.736 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:21:24.736 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:21:24.736 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2023-11-08T08:21:24.736 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T08:21:24.736 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-11-08T08:21:24.737 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2023-11-08T08:21:24.737 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-08T08:21:24.737 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42236sec preferred_lft 42236sec 2023-11-08T08:21:24.737 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link 2023-11-08T08:21:24.737 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:21:24.738 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:21:24.738 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-11-08T08:21:24.738 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2023-11-08T08:21:24.738 INFO:teuthology.orchestra.run.smithi040.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-08T08:21:24.739 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 02:42:94:e4:3d:2e brd ff:ff:ff:ff:ff:ff 2023-11-08T08:21:24.739 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-08T08:21:24.739 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:21:24.739 INFO:teuthology.orchestra.run.smithi040.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T08:21:24.739 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0e:a3:2a:89:6f:47 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:21:24.739 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T08:21:24.740 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:21:24.740 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a4cc:6bff:fe07:287b/64 scope link 2023-11-08T08:21:24.740 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:21:24.740 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T08:21:24.740 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0e:a3:2a:89:6f:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T08:21:24.741 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ca3:2aff:fe89:6f47/64 scope link 2023-11-08T08:21:24.741 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:21:24.741 INFO:teuthology.orchestra.run.smithi040.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T08:21:24.741 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 72:b3:2e:65:5a:02 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T08:21:24.742 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::70b3:2eff:fe65:5a02/64 scope link 2023-11-08T08:21:24.742 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:21:24.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:21:24.743 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T08:21:24.829 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-11-08T08:21:24.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:21:24.835 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-11-08T08:21:24.899 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-11-08T08:21:24.900 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T08:21:24.900 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-08T08:21:24.900 INFO:teuthology.orchestra.run.smithi040.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-08T08:21:24.900 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-08T08:21:24.901 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T08:21:24.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:21:24.902 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-11-08T08:21:24.903 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-11-08T08:21:24.903 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-08T08:21:24.903 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-08T08:21:24.904 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-08T08:21:24.904 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-11-08T08:21:25.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:21:25.040 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-11-08T08:21:25.058 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T08:21:25.058 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T08:21:25.059 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T08:21:25.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:21:25.060 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2023-11-08T08:21:25.137 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T08:21:25.137 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T08:21:25.203 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T08:21:25.204 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T08:21:25.204 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T08:21:25.264 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-08T08:21:25.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:21:25.265 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:21:25.401 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:21:25.401 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T08:21:25.425 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:21:25.425 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T08:21:25.425 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-08T08:21:25.425 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@54ed45f4-d5b0-41c3-8c3b-b5a18b844f25.cephfs=/ 2023-11-08T08:21:25.426 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2023-11-08T08:21:25.432 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T08:21:25.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:21:25.433 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T08:21:25.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:21:25.467 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T08:21:25.550 INFO:teuthology.orchestra.run.smithi040.stdout:client5955 2023-11-08T08:21:25.551 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:21:25.551 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:21:25.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:21:25.612 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2023-11-08T08:21:25.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:21:25.692 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2023-11-08T08:21:25.763 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-08T08:21:25.763 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-11-08T08:21:25.764 DEBUG:teuthology.orchestra.run.smithi040:> import stat 2023-11-08T08:21:25.764 DEBUG:teuthology.orchestra.run.smithi040:> 2023-11-08T08:21:25.764 DEBUG:teuthology.orchestra.run.smithi040:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2023-11-08T08:21:25.765 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-11-08T08:21:25.901 INFO:teuthology.orchestra.run.smithi040.stdout:1099511627777 2023-11-08T08:21:26.091 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:21:26.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.324+0000 7fa90d85e700 1 Processor -- start 2023-11-08T08:21:26.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.324+0000 7fa90d85e700 1 -- start start 2023-11-08T08:21:26.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.324+0000 7fa90d85e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa908163cb0 0x7fa908164cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:26.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.324+0000 7fa90d85e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa9080cdfb0 0x7fa90806cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:26.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.324+0000 7fa90d85e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa90806d2f0 0x7fa90806bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:26.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.324+0000 7fa90d85e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9080cc290 con 0x7fa9080cdfb0 2023-11-08T08:21:26.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.324+0000 7fa90d85e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa90806d770 con 0x7fa90806d2f0 2023-11-08T08:21:26.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.324+0000 7fa90d85e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa908164080 con 0x7fa908163cb0 2023-11-08T08:21:26.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.324+0000 7fa8fffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa9080cdfb0 0x7fa90806cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:26.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.324+0000 7fa8fffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa9080cdfb0 0x7fa90806cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51606/0 (socket says 172.21.15.40:51606) 2023-11-08T08:21:26.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.324+0000 7fa8fffff700 1 -- 172.21.15.40:0/3646140972 learned_addr learned my addr 172.21.15.40:0/3646140972 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:26.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa9077fe700 1 --2- 172.21.15.40:0/3646140972 >> v2:172.21.15.169:3300/0 conn(0x7fa90806d2f0 0x7fa90806bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:26.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa906ffd700 1 --2- 172.21.15.40:0/3646140972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa908163cb0 0x7fa908164cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:26.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa8fffff700 1 -- 172.21.15.40:0/3646140972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa908163cb0 msgr2=0x7fa908164cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:26.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa8fffff700 1 --2- 172.21.15.40:0/3646140972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa908163cb0 0x7fa908164cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa8fffff700 1 -- 172.21.15.40:0/3646140972 >> v2:172.21.15.169:3300/0 conn(0x7fa90806d2f0 msgr2=0x7fa90806bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:26.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa8fffff700 1 --2- 172.21.15.40:0/3646140972 >> v2:172.21.15.169:3300/0 conn(0x7fa90806d2f0 0x7fa90806bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa8fffff700 1 -- 172.21.15.40:0/3646140972 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa908069bf0 con 0x7fa9080cdfb0 2023-11-08T08:21:26.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa9077fe700 1 --2- 172.21.15.40:0/3646140972 >> v2:172.21.15.169:3300/0 conn(0x7fa90806d2f0 0x7fa90806bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:26.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa8fffff700 1 --2- 172.21.15.40:0/3646140972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9080cdfb0 0x7fa90806cdb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa90806f370 tx=0x7fa8f0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17367cad97cab7c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:26.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa9067fc700 1 -- 172.21.15.40:0/3646140972 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f000b8a0 con 0x7fa9080cdfb0 2023-11-08T08:21:26.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa9067fc700 1 -- 172.21.15.40:0/3646140972 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f000ba00 con 0x7fa9080cdfb0 2023-11-08T08:21:26.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa9067fc700 1 -- 172.21.15.40:0/3646140972 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f000bd90 con 0x7fa9080cdfb0 2023-11-08T08:21:26.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa90d85e700 1 -- 172.21.15.40:0/3646140972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9080cdfb0 msgr2=0x7fa90806cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:26.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa90d85e700 1 --2- 172.21.15.40:0/3646140972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9080cdfb0 0x7fa90806cdb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa90806f370 tx=0x7fa8f0009bf0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa90d85e700 1 -- 172.21.15.40:0/3646140972 shutdown_connections 2023-11-08T08:21:26.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa90d85e700 1 --2- 172.21.15.40:0/3646140972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9080cdfb0 0x7fa90806cdb0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa90d85e700 1 --2- 172.21.15.40:0/3646140972 >> v2:172.21.15.169:3300/0 conn(0x7fa90806d2f0 0x7fa90806bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa90d85e700 1 --2- 172.21.15.40:0/3646140972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa908163cb0 0x7fa908164cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa90d85e700 1 -- 172.21.15.40:0/3646140972 >> 172.21.15.40:0/3646140972 conn(0x7fa9080cf350 msgr2=0x7fa9080b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:26.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa90d85e700 1 -- 172.21.15.40:0/3646140972 shutdown_connections 2023-11-08T08:21:26.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.328+0000 7fa90d85e700 1 -- 172.21.15.40:0/3646140972 wait complete. 2023-11-08T08:21:26.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.332+0000 7fa90d85e700 1 Processor -- start 2023-11-08T08:21:26.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.332+0000 7fa90d85e700 1 -- start start 2023-11-08T08:21:26.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.332+0000 7fa90d85e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa90806d2f0 0x7fa90815e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:26.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.332+0000 7fa90d85e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9080cdfb0 0x7fa9080c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:26.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.332+0000 7fa90d85e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa908163cb0 0x7fa9080b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:26.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.332+0000 7fa90d85e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90815e630 con 0x7fa908163cb0 2023-11-08T08:21:26.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.332+0000 7fa90d85e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080c2080 con 0x7fa9080cdfb0 2023-11-08T08:21:26.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.332+0000 7fa90d85e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080b14f0 con 0x7fa90806d2f0 2023-11-08T08:21:26.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.332+0000 7fa9077fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa908163cb0 0x7fa9080b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:26.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.332+0000 7fa9077fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa908163cb0 0x7fa9080b3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51622/0 (socket says 172.21.15.40:51622) 2023-11-08T08:21:26.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.332+0000 7fa9077fe700 1 -- 172.21.15.40:0/1250891603 learned_addr learned my addr 172.21.15.40:0/1250891603 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:26.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.332+0000 7fa8fffff700 1 --2- 172.21.15.40:0/1250891603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9080cdfb0 0x7fa9080c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:26.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.332+0000 7fa906ffd700 1 --2- 172.21.15.40:0/1250891603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa90806d2f0 0x7fa90815e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:26.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.332+0000 7fa9077fe700 1 -- 172.21.15.40:0/1250891603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa90806d2f0 msgr2=0x7fa90815e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:26.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.332+0000 7fa9077fe700 1 --2- 172.21.15.40:0/1250891603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa90806d2f0 0x7fa90815e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.332+0000 7fa9077fe700 1 -- 172.21.15.40:0/1250891603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9080cdfb0 msgr2=0x7fa9080c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:26.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.332+0000 7fa9077fe700 1 --2- 172.21.15.40:0/1250891603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9080cdfb0 0x7fa9080c7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.332+0000 7fa9077fe700 1 -- 172.21.15.40:0/1250891603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f0009570 con 0x7fa908163cb0 2023-11-08T08:21:26.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.336+0000 7fa9077fe700 1 --2- 172.21.15.40:0/1250891603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa908163cb0 0x7fa9080b3980 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa900007e70 tx=0x7fa90000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:26.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.336+0000 7fa904ff9700 1 -- 172.21.15.40:0/1250891603 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90000dd20 con 0x7fa908163cb0 2023-11-08T08:21:26.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.336+0000 7fa904ff9700 1 -- 172.21.15.40:0/1250891603 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90000f040 con 0x7fa908163cb0 2023-11-08T08:21:26.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.336+0000 7fa904ff9700 1 -- 172.21.15.40:0/1250891603 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90000dea0 con 0x7fa908163cb0 2023-11-08T08:21:26.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.336+0000 7fa90d85e700 1 -- 172.21.15.40:0/1250891603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9080cc4a0 con 0x7fa908163cb0 2023-11-08T08:21:26.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.336+0000 7fa90d85e700 1 -- 172.21.15.40:0/1250891603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9081651f0 con 0x7fa908163cb0 2023-11-08T08:21:26.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.336+0000 7fa90d85e700 1 -- 172.21.15.40:0/1250891603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9080cc780 con 0x7fa908163cb0 2023-11-08T08:21:26.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.340+0000 7fa904ff9700 1 -- 172.21.15.40:0/1250891603 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa90001c030 con 0x7fa908163cb0 2023-11-08T08:21:26.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.340+0000 7fa904ff9700 1 --2- 172.21.15.40:0/1250891603 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa8e8064ee0 0x7fa8e8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:26.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.340+0000 7fa904ff9700 1 -- 172.21.15.40:0/1250891603 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa900081e70 con 0x7fa908163cb0 2023-11-08T08:21:26.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.340+0000 7fa906ffd700 1 --2- 172.21.15.40:0/1250891603 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa8e8064ee0 0x7fa8e8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:26.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.340+0000 7fa906ffd700 1 --2- 172.21.15.40:0/1250891603 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa8e8064ee0 0x7fa8e8067390 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009980 tx=0x7fa8f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:26.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.348+0000 7fa904ff9700 1 -- 172.21.15.40:0/1250891603 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa90004fed0 con 0x7fa908163cb0 2023-11-08T08:21:26.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.560+0000 7fa90d85e700 1 -- 172.21.15.40:0/1250891603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9080726c0 con 0x7fa908163cb0 2023-11-08T08:21:26.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.560+0000 7fa904ff9700 1 -- 172.21.15.40:0/1250891603 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4651 (secure 0 0 0) 0x7fa9000534f0 con 0x7fa908163cb0 2023-11-08T08:21:26.568 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:26.568 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1549516710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1549516710},{"type":"v1","addr":"172.21.15.169:6837","nonce":1549516710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5728,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/912870125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":912870125},{"type":"v1","addr":"172.21.15.169:6835","nonce":912870125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5772,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1045606427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1045606427},{"type":"v1","addr":"172.21.15.40:6834","nonce":1045606427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:21:17.730002+0000","modified":"2023-11-08T08:21:20.752976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2471311394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2471311394},{"type":"v1","addr":"172.21.15.40:6835","nonce":2471311394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T08:21:26.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.568+0000 7fa90d85e700 1 -- 172.21.15.40:0/1250891603 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa8e8064ee0 msgr2=0x7fa8e8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:26.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.568+0000 7fa90d85e700 1 --2- 172.21.15.40:0/1250891603 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa8e8064ee0 0x7fa8e8067390 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009980 tx=0x7fa8f8008040 comp rx=0 tx=0).stop 2023-11-08T08:21:26.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.568+0000 7fa90d85e700 1 -- 172.21.15.40:0/1250891603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa908163cb0 msgr2=0x7fa9080b3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:26.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.568+0000 7fa90d85e700 1 --2- 172.21.15.40:0/1250891603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa908163cb0 0x7fa9080b3980 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa900007e70 tx=0x7fa90000d3b0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.568+0000 7fa90d85e700 1 -- 172.21.15.40:0/1250891603 shutdown_connections 2023-11-08T08:21:26.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.568+0000 7fa90d85e700 1 --2- 172.21.15.40:0/1250891603 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa8e8064ee0 0x7fa8e8067390 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.568+0000 7fa90d85e700 1 --2- 172.21.15.40:0/1250891603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa908163cb0 0x7fa9080b3980 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.568+0000 7fa90d85e700 1 --2- 172.21.15.40:0/1250891603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9080cdfb0 0x7fa9080c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.568+0000 7fa90d85e700 1 --2- 172.21.15.40:0/1250891603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa90806d2f0 0x7fa90815e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.568+0000 7fa90d85e700 1 -- 172.21.15.40:0/1250891603 >> 172.21.15.40:0/1250891603 conn(0x7fa9080cf350 msgr2=0x7fa90806a540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:26.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.568+0000 7fa90d85e700 1 -- 172.21.15.40:0/1250891603 shutdown_connections 2023-11-08T08:21:26.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.568+0000 7fa90d85e700 1 -- 172.21.15.40:0/1250891603 wait complete. 2023-11-08T08:21:26.576 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 25 2023-11-08T08:21:26.595 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-08T08:21:26.638 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:21:26.651 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:21:26.801 INFO:teuthology.orchestra.run.smithi040.stdout:{"message":"","return_code":0} 2023-11-08T08:21:26.801 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-08T08:21:26.802 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-08T08:21:26.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.864+0000 7f7f85da7140 1 Processor -- start 2023-11-08T08:21:26.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.864+0000 7f7f85da7140 1 -- start start 2023-11-08T08:21:26.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.868+0000 7f7f85da7140 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d9d009700 0x564d9d0394a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:26.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.868+0000 7f7f85da7140 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564d9d03c980 0x564d9d0cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:26.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.868+0000 7f7f85da7140 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564d9d0c7830 0x564d9d030460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:26.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.868+0000 7f7f85da7140 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d9cf31240 con 0x564d9d009700 2023-11-08T08:21:26.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.868+0000 7f7f85da7140 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d9d0302d0 con 0x564d9d0c7830 2023-11-08T08:21:26.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.868+0000 7f7f85da7140 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d9d0cf620 con 0x564d9d03c980 2023-11-08T08:21:26.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.868+0000 7f7f838dc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d9d009700 0x564d9d0394a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:26.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.868+0000 7f7f838dc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d9d009700 0x564d9d0394a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51638/0 (socket says 172.21.15.40:51638) 2023-11-08T08:21:26.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.868+0000 7f7f838dc700 1 -- 172.21.15.40:0/2964275813 learned_addr learned my addr 172.21.15.40:0/2964275813 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:26.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.868+0000 7f7f830db700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564d9d03c980 0x564d9d0cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:26.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.868+0000 7f7f840dd700 1 --2- 172.21.15.40:0/2964275813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564d9d0c7830 0x564d9d030460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:26.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.868+0000 7f7f838dc700 1 -- 172.21.15.40:0/2964275813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564d9d03c980 msgr2=0x564d9d0cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:26.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.868+0000 7f7f838dc700 1 --2- 172.21.15.40:0/2964275813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564d9d03c980 0x564d9d0cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.868+0000 7f7f838dc700 1 -- 172.21.15.40:0/2964275813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564d9d0c7830 msgr2=0x564d9d030460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:26.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.868+0000 7f7f838dc700 1 --2- 172.21.15.40:0/2964275813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564d9d0c7830 0x564d9d030460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.868+0000 7f7f838dc700 1 -- 172.21.15.40:0/2964275813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f74005de0 con 0x564d9d009700 2023-11-08T08:21:26.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.868+0000 7f7f838dc700 1 --2- 172.21.15.40:0/2964275813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d9d009700 0x564d9d0394a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7f74015040 tx=0x7f7f7400d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca09bd2c62a1d418 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:26.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.868+0000 7f7f820d9700 1 -- 172.21.15.40:0/2964275813 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74008670 con 0x564d9d009700 2023-11-08T08:21:26.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.868+0000 7f7f820d9700 1 -- 172.21.15.40:0/2964275813 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f74008b40 con 0x564d9d009700 2023-11-08T08:21:26.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.868+0000 7f7f820d9700 1 -- 172.21.15.40:0/2964275813 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74012450 con 0x564d9d009700 2023-11-08T08:21:26.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.872+0000 7f7f85da7140 1 -- 172.21.15.40:0/2964275813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d9d009700 msgr2=0x564d9d0394a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:26.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.872+0000 7f7f85da7140 1 --2- 172.21.15.40:0/2964275813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d9d009700 0x564d9d0394a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7f74015040 tx=0x7f7f7400d000 comp rx=0 tx=0).stop 2023-11-08T08:21:26.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.872+0000 7f7f85da7140 1 -- 172.21.15.40:0/2964275813 shutdown_connections 2023-11-08T08:21:26.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.872+0000 7f7f85da7140 1 --2- 172.21.15.40:0/2964275813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564d9d0c7830 0x564d9d030460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.872+0000 7f7f85da7140 1 --2- 172.21.15.40:0/2964275813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564d9d03c980 0x564d9d0cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.872+0000 7f7f85da7140 1 --2- 172.21.15.40:0/2964275813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d9d009700 0x564d9d0394a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.872+0000 7f7f85da7140 1 -- 172.21.15.40:0/2964275813 >> 172.21.15.40:0/2964275813 conn(0x564d9d00fc30 msgr2=0x564d9d0c7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:26.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.872+0000 7f7f85da7140 1 -- 172.21.15.40:0/2964275813 shutdown_connections 2023-11-08T08:21:26.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.872+0000 7f7f85da7140 1 -- 172.21.15.40:0/2964275813 wait complete. 2023-11-08T08:21:26.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.872+0000 7f7f85da7140 1 Processor -- start 2023-11-08T08:21:26.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.872+0000 7f7f85da7140 1 -- start start 2023-11-08T08:21:26.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f85da7140 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564d9d03c980 0x564d9d0b7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:26.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f85da7140 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d9d0c7830 0x564d9d0575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:26.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f85da7140 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564d9d0b7950 0x564d9d057b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:26.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f85da7140 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d9d0b7ea0 con 0x564d9d0c7830 2023-11-08T08:21:26.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f85da7140 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d9d0b8140 con 0x564d9d0b7950 2023-11-08T08:21:26.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f85da7140 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d9d0b83e0 con 0x564d9d03c980 2023-11-08T08:21:26.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f830db700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d9d0c7830 0x564d9d0575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:26.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f830db700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d9d0c7830 0x564d9d0575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51646/0 (socket says 172.21.15.40:51646) 2023-11-08T08:21:26.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f830db700 1 -- 172.21.15.40:0/955953464 learned_addr learned my addr 172.21.15.40:0/955953464 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:26.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f838dc700 1 --2- 172.21.15.40:0/955953464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564d9d03c980 0x564d9d0b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:26.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f840dd700 1 --2- 172.21.15.40:0/955953464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564d9d0b7950 0x564d9d057b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:26.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f830db700 1 -- 172.21.15.40:0/955953464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564d9d03c980 msgr2=0x564d9d0b7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:26.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f830db700 1 --2- 172.21.15.40:0/955953464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564d9d03c980 0x564d9d0b7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f830db700 1 -- 172.21.15.40:0/955953464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564d9d0b7950 msgr2=0x564d9d057b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:26.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f830db700 1 --2- 172.21.15.40:0/955953464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564d9d0b7950 0x564d9d057b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f830db700 1 -- 172.21.15.40:0/955953464 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d9cf63760 con 0x564d9d0c7830 2023-11-08T08:21:26.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f838dc700 1 --2- 172.21.15.40:0/955953464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564d9d03c980 0x564d9d0b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:26.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f830db700 1 --2- 172.21.15.40:0/955953464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d9d0c7830 0x564d9d0575c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7f78009fa0 tx=0x7f7f78008790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:26.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f810d7700 1 -- 172.21.15.40:0/955953464 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f78011040 con 0x564d9d0c7830 2023-11-08T08:21:26.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f810d7700 1 -- 172.21.15.40:0/955953464 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f78008e90 con 0x564d9d0c7830 2023-11-08T08:21:26.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f810d7700 1 -- 172.21.15.40:0/955953464 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f78006990 con 0x564d9d0c7830 2023-11-08T08:21:26.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f85da7140 1 -- 172.21.15.40:0/955953464 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564d9cf49390 con 0x564d9d0c7830 2023-11-08T08:21:26.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.876+0000 7f7f85da7140 1 -- 172.21.15.40:0/955953464 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d9cf3c8b0 con 0x564d9d0c7830 2023-11-08T08:21:26.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.880+0000 7f7f810d7700 1 -- 172.21.15.40:0/955953464 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7f7801d030 con 0x564d9d0c7830 2023-11-08T08:21:26.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.880+0000 7f7f810d7700 1 --2- 172.21.15.40:0/955953464 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7f70063810 0x7f7f70065cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:26.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.880+0000 7f7f810d7700 1 -- 172.21.15.40:0/955953464 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7f78080810 con 0x564d9d0c7830 2023-11-08T08:21:26.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.880+0000 7f7f838dc700 1 --2- 172.21.15.40:0/955953464 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7f70063810 0x7f7f70065cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:26.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.880+0000 7f7f85da7140 1 --2- 172.21.15.40:0/955953464 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x564d9d0b8cc0 0x564d9d0b90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:26.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.880+0000 7f7f85da7140 1 -- 172.21.15.40:0/955953464 --> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57) v8 -- 0x564d9d0b9600 con 0x564d9d0b8cc0 2023-11-08T08:21:26.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.884+0000 7f7f840dd700 1 --2- 172.21.15.40:0/955953464 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x564d9d0b8cc0 0x564d9d0b90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:26.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.884+0000 7f7f838dc700 1 --2- 172.21.15.40:0/955953464 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7f70063810 0x7f7f70065cc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400c010 tx=0x7f7f74010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:26.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.884+0000 7f7f840dd700 1 --2- 172.21.15.40:0/955953464 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x564d9d0b8cc0 0x564d9d0b90c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:26.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.884+0000 7f7f840dd700 1 -- 172.21.15.40:0/955953464 <== osd.5 v2:172.21.15.169:6809/35793 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f7f7c0017c0 con 0x564d9d0b8cc0 2023-11-08T08:21:26.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.884+0000 7f7f85da7140 1 -- 172.21.15.40:0/955953464 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x564d9d0b8cc0 msgr2=0x564d9d0b90c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:26.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.884+0000 7f7f85da7140 1 --2- 172.21.15.40:0/955953464 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x564d9d0b8cc0 0x564d9d0b90c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:26.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.884+0000 7f7f85da7140 1 -- 172.21.15.40:0/955953464 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7f70063810 msgr2=0x7f7f70065cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:26.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.884+0000 7f7f85da7140 1 --2- 172.21.15.40:0/955953464 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7f70063810 0x7f7f70065cc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400c010 tx=0x7f7f74010000 comp rx=0 tx=0).stop 2023-11-08T08:21:26.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.884+0000 7f7f85da7140 1 -- 172.21.15.40:0/955953464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d9d0c7830 msgr2=0x564d9d0575c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:26.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.884+0000 7f7f85da7140 1 --2- 172.21.15.40:0/955953464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d9d0c7830 0x564d9d0575c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7f78009fa0 tx=0x7f7f78008790 comp rx=0 tx=0).stop 2023-11-08T08:21:26.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.884+0000 7f7f840dd700 1 -- 172.21.15.40:0/955953464 reap_dead start 2023-11-08T08:21:26.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.884+0000 7f7f85da7140 1 -- 172.21.15.40:0/955953464 shutdown_connections 2023-11-08T08:21:26.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.884+0000 7f7f85da7140 1 -- 172.21.15.40:0/955953464 >> 172.21.15.40:0/955953464 conn(0x564d9d00fc30 msgr2=0x564d9d0c7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:26.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.884+0000 7f7f85da7140 1 -- 172.21.15.40:0/955953464 shutdown_connections 2023-11-08T08:21:26.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:26.884+0000 7f7f85da7140 1 -- 172.21.15.40:0/955953464 wait complete. 2023-11-08T08:21:26.899 INFO:teuthology.orchestra.run.smithi040.stdout:Yalphaparent_a 2023-11-08T08:21:26.900 DEBUG:teuthology.orchestra.run.smithi040:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-08T08:21:26.988 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:21:26.988 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511627777, 2023-11-08T08:21:26.989 INFO:teuthology.orchestra.run.smithi040.stdout: "ancestors": [ 2023-11-08T08:21:26.989 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:26.989 INFO:teuthology.orchestra.run.smithi040.stdout: "dirino": 1099511627776, 2023-11-08T08:21:26.989 INFO:teuthology.orchestra.run.smithi040.stdout: "dname": "alpha", 2023-11-08T08:21:26.989 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 2 2023-11-08T08:21:26.990 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:26.990 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:26.990 INFO:teuthology.orchestra.run.smithi040.stdout: "dirino": 1, 2023-11-08T08:21:26.991 INFO:teuthology.orchestra.run.smithi040.stdout: "dname": "parent_a", 2023-11-08T08:21:26.991 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 4 2023-11-08T08:21:26.991 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:26.991 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-11-08T08:21:26.991 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 9, 2023-11-08T08:21:26.992 INFO:teuthology.orchestra.run.smithi040.stdout: "old_pools": [] 2023-11-08T08:21:26.992 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:21:26.992 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:26.993 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2023-11-08T08:21:27.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.104+0000 7f1d7e4b6140 1 Processor -- start 2023-11-08T08:21:27.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.104+0000 7f1d7e4b6140 1 -- start start 2023-11-08T08:21:27.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.104+0000 7f1d7e4b6140 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af9fac8710 0x55af9fb8a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.104+0000 7f1d7e4b6140 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af9fafb990 0x55af9fb88ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.104+0000 7f1d7e4b6140 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55af9fb86840 0x55af9fc28fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.104+0000 7f1d7e4b6140 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9faf6650 con 0x55af9fb86840 2023-11-08T08:21:27.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.104+0000 7f1d7e4b6140 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9fb8a690 con 0x55af9fafb990 2023-11-08T08:21:27.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.104+0000 7f1d7e4b6140 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af9faf8740 con 0x55af9fac8710 2023-11-08T08:21:27.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.104+0000 7f1d7c7ec700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55af9fb86840 0x55af9fc28fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.104+0000 7f1d7c7ec700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55af9fb86840 0x55af9fc28fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51662/0 (socket says 172.21.15.40:51662) 2023-11-08T08:21:27.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.104+0000 7f1d7c7ec700 1 -- 172.21.15.40:0/597184338 learned_addr learned my addr 172.21.15.40:0/597184338 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:27.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.104+0000 7f1d7bfeb700 1 --2- 172.21.15.40:0/597184338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af9fac8710 0x55af9fb8a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.104+0000 7f1d7b7ea700 1 --2- 172.21.15.40:0/597184338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af9fafb990 0x55af9fb88ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.104+0000 7f1d7c7ec700 1 -- 172.21.15.40:0/597184338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af9fac8710 msgr2=0x55af9fb8a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.104+0000 7f1d7c7ec700 1 --2- 172.21.15.40:0/597184338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af9fac8710 0x55af9fb8a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.104+0000 7f1d7c7ec700 1 -- 172.21.15.40:0/597184338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af9fafb990 msgr2=0x55af9fb88ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.104+0000 7f1d7c7ec700 1 --2- 172.21.15.40:0/597184338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af9fafb990 0x55af9fb88ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.104+0000 7f1d7c7ec700 1 -- 172.21.15.40:0/597184338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d64005d20 con 0x55af9fb86840 2023-11-08T08:21:27.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.104+0000 7f1d7bfeb700 1 --2- 172.21.15.40:0/597184338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af9fac8710 0x55af9fb8a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:21:27.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.104+0000 7f1d7b7ea700 1 --2- 172.21.15.40:0/597184338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af9fafb990 0x55af9fb88ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:21:27.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.108+0000 7f1d7c7ec700 1 --2- 172.21.15.40:0/597184338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55af9fb86840 0x55af9fc28fe0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x55af9fad11d0 tx=0x7f1d7400d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ad601bfe4d6716 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.108+0000 7f1d7a7e8700 1 -- 172.21.15.40:0/597184338 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d74008690 con 0x55af9fb86840 2023-11-08T08:21:27.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.108+0000 7f1d7a7e8700 1 -- 172.21.15.40:0/597184338 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d74005430 con 0x55af9fb86840 2023-11-08T08:21:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.108+0000 7f1d7a7e8700 1 -- 172.21.15.40:0/597184338 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d74008b60 con 0x55af9fb86840 2023-11-08T08:21:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.108+0000 7f1d7e4b6140 1 -- 172.21.15.40:0/597184338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55af9fb86840 msgr2=0x55af9fc28fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.108+0000 7f1d7e4b6140 1 --2- 172.21.15.40:0/597184338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55af9fb86840 0x55af9fc28fe0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x55af9fad11d0 tx=0x7f1d7400d000 comp rx=0 tx=0).stop 2023-11-08T08:21:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.108+0000 7f1d7e4b6140 1 -- 172.21.15.40:0/597184338 shutdown_connections 2023-11-08T08:21:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.108+0000 7f1d7e4b6140 1 --2- 172.21.15.40:0/597184338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55af9fb86840 0x55af9fc28fe0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.108+0000 7f1d7e4b6140 1 --2- 172.21.15.40:0/597184338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af9fafb990 0x55af9fb88ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.108+0000 7f1d7e4b6140 1 --2- 172.21.15.40:0/597184338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af9fac8710 0x55af9fb8a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.108+0000 7f1d7e4b6140 1 -- 172.21.15.40:0/597184338 >> 172.21.15.40:0/597184338 conn(0x55af9facec40 msgr2=0x55af9fb14db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.108+0000 7f1d7e4b6140 1 -- 172.21.15.40:0/597184338 shutdown_connections 2023-11-08T08:21:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.108+0000 7f1d7e4b6140 1 -- 172.21.15.40:0/597184338 wait complete. 2023-11-08T08:21:27.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.108+0000 7f1d7e4b6140 1 Processor -- start 2023-11-08T08:21:27.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.108+0000 7f1d7e4b6140 1 -- start start 2023-11-08T08:21:27.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d7e4b6140 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55af9fac8710 0x55af9fba1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d7e4b6140 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af9fafb990 0x55af9fc2f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d7e4b6140 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af9fb86840 0x55af9faf0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d7e4b6140 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9fc300d0 con 0x55af9fac8710 2023-11-08T08:21:27.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d7e4b6140 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af9fc30370 con 0x55af9fafb990 2023-11-08T08:21:27.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d7e4b6140 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af9fc30610 con 0x55af9fb86840 2023-11-08T08:21:27.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d7bfeb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55af9fac8710 0x55af9fba1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d7bfeb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55af9fac8710 0x55af9fba1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51674/0 (socket says 172.21.15.40:51674) 2023-11-08T08:21:27.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d7bfeb700 1 -- 172.21.15.40:0/1172189162 learned_addr learned my addr 172.21.15.40:0/1172189162 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:27.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d7b7ea700 1 --2- 172.21.15.40:0/1172189162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af9fafb990 0x55af9fc2f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d7c7ec700 1 --2- 172.21.15.40:0/1172189162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af9fb86840 0x55af9faf0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d7bfeb700 1 -- 172.21.15.40:0/1172189162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af9fb86840 msgr2=0x55af9faf0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d7bfeb700 1 --2- 172.21.15.40:0/1172189162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af9fb86840 0x55af9faf0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d7bfeb700 1 -- 172.21.15.40:0/1172189162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af9fafb990 msgr2=0x55af9fc2f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d7bfeb700 1 --2- 172.21.15.40:0/1172189162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af9fafb990 0x55af9fc2f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d7bfeb700 1 -- 172.21.15.40:0/1172189162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d64009530 con 0x55af9fac8710 2023-11-08T08:21:27.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d7b7ea700 1 --2- 172.21.15.40:0/1172189162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af9fafb990 0x55af9fc2f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:21:27.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d7bfeb700 1 --2- 172.21.15.40:0/1172189162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55af9fac8710 0x55af9fba1ef0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00cfd0 tx=0x7f1d6c00b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d797e6700 1 -- 172.21.15.40:0/1172189162 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c011040 con 0x55af9fac8710 2023-11-08T08:21:27.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d797e6700 1 -- 172.21.15.40:0/1172189162 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c016680 con 0x55af9fac8710 2023-11-08T08:21:27.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d797e6700 1 -- 172.21.15.40:0/1172189162 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c016a30 con 0x55af9fac8710 2023-11-08T08:21:27.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.112+0000 7f1d7e4b6140 1 -- 172.21.15.40:0/1172189162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55af9f9fb520 con 0x55af9fac8710 2023-11-08T08:21:27.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.116+0000 7f1d7e4b6140 1 -- 172.21.15.40:0/1172189162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af9fa08390 con 0x55af9fac8710 2023-11-08T08:21:27.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.116+0000 7f1d797e6700 1 -- 172.21.15.40:0/1172189162 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1d6c01d030 con 0x55af9fac8710 2023-11-08T08:21:27.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.120+0000 7f1d797e6700 1 --2- 172.21.15.40:0/1172189162 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d5c063810 0x7f1d5c065cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.120+0000 7f1d797e6700 1 -- 172.21.15.40:0/1172189162 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1d6c0807f0 con 0x55af9fac8710 2023-11-08T08:21:27.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.120+0000 7f1d7e4b6140 1 --2- 172.21.15.40:0/1172189162 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x55af9fc2fc50 0x55af9fc31500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.120+0000 7f1d7e4b6140 1 -- 172.21.15.40:0/1172189162 --> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57) v8 -- 0x55af9fba2430 con 0x55af9fc2fc50 2023-11-08T08:21:27.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.120+0000 7f1d7b7ea700 1 --2- 172.21.15.40:0/1172189162 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d5c063810 0x7f1d5c065cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.120+0000 7f1d7c7ec700 1 --2- 172.21.15.40:0/1172189162 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x55af9fc2fc50 0x55af9fc31500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.120+0000 7f1d7b7ea700 1 --2- 172.21.15.40:0/1172189162 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d5c063810 0x7f1d5c065cc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009500 tx=0x7f1d6400c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.120+0000 7f1d7c7ec700 1 --2- 172.21.15.40:0/1172189162 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x55af9fc2fc50 0x55af9fc31500 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.124+0000 7f1d7c7ec700 1 -- 172.21.15.40:0/1172189162 <== osd.5 v2:172.21.15.169:6809/35793 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v57'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1d74008b40 con 0x55af9fc2fc50 2023-11-08T08:21:27.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.124+0000 7f1d7e4b6140 1 -- 172.21.15.40:0/1172189162 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x55af9fc2fc50 msgr2=0x55af9fc31500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.124+0000 7f1d7e4b6140 1 --2- 172.21.15.40:0/1172189162 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x55af9fc2fc50 0x55af9fc31500 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.124+0000 7f1d7e4b6140 1 -- 172.21.15.40:0/1172189162 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d5c063810 msgr2=0x7f1d5c065cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.124+0000 7f1d7e4b6140 1 --2- 172.21.15.40:0/1172189162 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d5c063810 0x7f1d5c065cc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009500 tx=0x7f1d6400c040 comp rx=0 tx=0).stop 2023-11-08T08:21:27.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.124+0000 7f1d7e4b6140 1 -- 172.21.15.40:0/1172189162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55af9fac8710 msgr2=0x55af9fba1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.124+0000 7f1d7e4b6140 1 --2- 172.21.15.40:0/1172189162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55af9fac8710 0x55af9fba1ef0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00cfd0 tx=0x7f1d6c00b7a0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.124+0000 7f1d7c7ec700 1 -- 172.21.15.40:0/1172189162 reap_dead start 2023-11-08T08:21:27.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.124+0000 7f1d7e4b6140 1 -- 172.21.15.40:0/1172189162 shutdown_connections 2023-11-08T08:21:27.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.124+0000 7f1d7e4b6140 1 -- 172.21.15.40:0/1172189162 >> 172.21.15.40:0/1172189162 conn(0x55af9facec40 msgr2=0x55af9fba3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:27.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.124+0000 7f1d7e4b6140 1 -- 172.21.15.40:0/1172189162 shutdown_connections 2023-11-08T08:21:27.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.124+0000 7f1d7e4b6140 1 -- 172.21.15.40:0/1172189162 wait complete. 2023-11-08T08:21:27.139 DEBUG:teuthology.orchestra.run.smithi040:> sudo daemon-helper kill ceph --cluster ceph -w 2023-11-08T08:21:27.146 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2023-11-08T08:21:27.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.328+0000 7fce0a377700 1 Processor -- start 2023-11-08T08:21:27.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.328+0000 7fce0a377700 1 -- start start 2023-11-08T08:21:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce0a377700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fce0415c6e0 0x7fce04071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce0a377700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fce0406a510 0x7fce0406c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce0a377700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce0415cab0 con 0x7fce04071fe0 2023-11-08T08:21:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce0a377700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce040d2800 con 0x7fce0406a510 2023-11-08T08:21:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce0a377700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce040702d0 con 0x7fce0415c6e0 2023-11-08T08:21:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce09b76700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fce0406a510 0x7fce0406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce09b76700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fce0406a510 0x7fce0406c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56198/0 (socket says 172.21.15.40:56198) 2023-11-08T08:21:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce09b76700 1 -- 172.21.15.40:0/1179469295 learned_addr learned my addr 172.21.15.40:0/1179469295 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:27.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce09375700 1 --2- 172.21.15.40:0/1179469295 >> v2:172.21.15.169:3300/0 conn(0x7fce0415c6e0 0x7fce04071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce09b76700 1 -- 172.21.15.40:0/1179469295 >> v2:172.21.15.169:3300/0 conn(0x7fce0415c6e0 msgr2=0x7fce04071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce09b76700 1 --2- 172.21.15.40:0/1179469295 >> v2:172.21.15.169:3300/0 conn(0x7fce0415c6e0 0x7fce04071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce09b76700 1 -- 172.21.15.40:0/1179469295 >> v1:172.21.15.169:6789/0 conn(0x7fce04071fe0 legacy=0x7fce04069e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce09b76700 1 -- 172.21.15.40:0/1179469295 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce040d2a10 con 0x7fce0406a510 2023-11-08T08:21:27.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce09b76700 1 --2- 172.21.15.40:0/1179469295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0406a510 0x7fce0406c630 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fce0406e290 tx=0x7fcdfc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc9f25707643ca6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fcdfbfff700 1 -- 172.21.15.40:0/1179469295 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc00e070 con 0x7fce0406a510 2023-11-08T08:21:27.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fcdfbfff700 1 -- 172.21.15.40:0/1179469295 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc003a40 con 0x7fce0406a510 2023-11-08T08:21:27.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fcdfbfff700 1 -- 172.21.15.40:0/1179469295 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc003df0 con 0x7fce0406a510 2023-11-08T08:21:27.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce0a377700 1 -- 172.21.15.40:0/1179469295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0406a510 msgr2=0x7fce0406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce0a377700 1 --2- 172.21.15.40:0/1179469295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0406a510 0x7fce0406c630 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fce0406e290 tx=0x7fcdfc00b330 comp rx=0 tx=0).stop 2023-11-08T08:21:27.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce0a377700 1 -- 172.21.15.40:0/1179469295 shutdown_connections 2023-11-08T08:21:27.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce0a377700 1 --2- 172.21.15.40:0/1179469295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0406a510 0x7fce0406c630 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce0a377700 1 --2- 172.21.15.40:0/1179469295 >> v2:172.21.15.169:3300/0 conn(0x7fce0415c6e0 0x7fce04071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce0a377700 1 -- 172.21.15.40:0/1179469295 >> 172.21.15.40:0/1179469295 conn(0x7fce040b65b0 msgr2=0x7fce04071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:27.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce0a377700 1 -- 172.21.15.40:0/1179469295 shutdown_connections 2023-11-08T08:21:27.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.332+0000 7fce0a377700 1 -- 172.21.15.40:0/1179469295 wait complete. 2023-11-08T08:21:27.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.336+0000 7fce0a377700 1 Processor -- start 2023-11-08T08:21:27.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.336+0000 7fce0a377700 1 -- start start 2023-11-08T08:21:27.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.336+0000 7fce0a377700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0406a510 0x7fce040cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.336+0000 7fce0a377700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce04071fe0 0x7fce040cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.336+0000 7fce0a377700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce0415c6e0 0x7fce041e7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.336+0000 7fce0a377700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce040ccb50 con 0x7fce0406a510 2023-11-08T08:21:27.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.336+0000 7fce0a377700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce04158d60 con 0x7fce04071fe0 2023-11-08T08:21:27.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.336+0000 7fce0a377700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce040cbf50 con 0x7fce0415c6e0 2023-11-08T08:21:27.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.336+0000 7fce09375700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0406a510 0x7fce040cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.336+0000 7fce09375700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0406a510 0x7fce040cb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56212/0 (socket says 172.21.15.40:56212) 2023-11-08T08:21:27.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.336+0000 7fce09375700 1 -- 172.21.15.40:0/1862245849 learned_addr learned my addr 172.21.15.40:0/1862245849 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:27.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.340+0000 7fce09b76700 1 --2- 172.21.15.40:0/1862245849 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce0415c6e0 0x7fce041e7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.340+0000 7fce08b74700 1 --2- 172.21.15.40:0/1862245849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce04071fe0 0x7fce040cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.340+0000 7fce09375700 1 -- 172.21.15.40:0/1862245849 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce0415c6e0 msgr2=0x7fce041e7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.340+0000 7fce09375700 1 --2- 172.21.15.40:0/1862245849 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce0415c6e0 0x7fce041e7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.340+0000 7fce09375700 1 -- 172.21.15.40:0/1862245849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce04071fe0 msgr2=0x7fce040cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.340+0000 7fce09375700 1 --2- 172.21.15.40:0/1862245849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce04071fe0 0x7fce040cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.340+0000 7fce09375700 1 -- 172.21.15.40:0/1862245849 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc009b30 con 0x7fce0406a510 2023-11-08T08:21:27.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.340+0000 7fce08b74700 1 --2- 172.21.15.40:0/1862245849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce04071fe0 0x7fce040cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:21:27.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.340+0000 7fce09375700 1 --2- 172.21.15.40:0/1862245849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0406a510 0x7fce040cb660 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fce0000e790 tx=0x7fce0000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.340+0000 7fcdf9ffb700 1 -- 172.21.15.40:0/1862245849 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce00013610 con 0x7fce0406a510 2023-11-08T08:21:27.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.340+0000 7fcdf9ffb700 1 -- 172.21.15.40:0/1862245849 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce00013770 con 0x7fce0406a510 2023-11-08T08:21:27.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.340+0000 7fce0a377700 1 -- 172.21.15.40:0/1862245849 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce040d2fd0 con 0x7fce0406a510 2023-11-08T08:21:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.340+0000 7fcdf9ffb700 1 -- 172.21.15.40:0/1862245849 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce00013ae0 con 0x7fce0406a510 2023-11-08T08:21:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.340+0000 7fce0a377700 1 -- 172.21.15.40:0/1862245849 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce040d2cf0 con 0x7fce0406a510 2023-11-08T08:21:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.344+0000 7fcdf9ffb700 1 -- 172.21.15.40:0/1862245849 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fce00023070 con 0x7fce0406a510 2023-11-08T08:21:27.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.344+0000 7fcdf9ffb700 1 --2- 172.21.15.40:0/1862245849 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcde8064f10 0x7fcde80673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.344+0000 7fcdf9ffb700 1 -- 172.21.15.40:0/1862245849 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fce00081d10 con 0x7fce0406a510 2023-11-08T08:21:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.344+0000 7fce08b74700 1 --2- 172.21.15.40:0/1862245849 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcde8064f10 0x7fcde80673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.344+0000 7fce0e861740 1 Processor -- start 2023-11-08T08:21:27.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.344+0000 7fce0e861740 1 -- start start 2023-11-08T08:21:27.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce08b74700 1 --2- 172.21.15.40:0/1862245849 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcde8064f10 0x7fcde80673c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fce040cbba0 tx=0x7fcdf4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce0e861740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1effd80 0x21add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce0e861740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1eff9b0 0x21b6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce0e861740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1f4aa30 0x21b7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce0e861740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x206ec80 con 0x1f4aa30 2023-11-08T08:21:27.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce0e861740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x20ded60 con 0x1effd80 2023-11-08T08:21:27.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce0e861740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x20d8060 con 0x1eff9b0 2023-11-08T08:21:27.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce09b76700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1f4aa30 0x21b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce09b76700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1f4aa30 0x21b7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56214/0 (socket says 172.21.15.40:56214) 2023-11-08T08:21:27.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce09b76700 1 -- 172.21.15.40:0/3103393342 learned_addr learned my addr 172.21.15.40:0/3103393342 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:27.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce09375700 1 --2- 172.21.15.40:0/3103393342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1effd80 0x21add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce08b74700 1 --2- 172.21.15.40:0/3103393342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1eff9b0 0x21b6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce09b76700 1 -- 172.21.15.40:0/3103393342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1eff9b0 msgr2=0x21b6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce09b76700 1 --2- 172.21.15.40:0/3103393342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1eff9b0 0x21b6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce09b76700 1 -- 172.21.15.40:0/3103393342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1effd80 msgr2=0x21add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce09b76700 1 --2- 172.21.15.40:0/3103393342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1effd80 0x21add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce09b76700 1 -- 172.21.15.40:0/3103393342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x20fd4e0 con 0x1f4aa30 2023-11-08T08:21:27.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce08b74700 1 --2- 172.21.15.40:0/3103393342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1eff9b0 0x21b6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:27.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce09b76700 1 --2- 172.21.15.40:0/3103393342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1f4aa30 0x21b7340 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fce0406ecf0 tx=0x7fcdfc010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b37db1bf34714da server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fcdeeffd700 1 -- 172.21.15.40:0/3103393342 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc00b980 con 0x1f4aa30 2023-11-08T08:21:27.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fcdeeffd700 1 -- 172.21.15.40:0/3103393342 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc00bae0 con 0x1f4aa30 2023-11-08T08:21:27.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fcdeeffd700 1 -- 172.21.15.40:0/3103393342 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc00be90 con 0x1f4aa30 2023-11-08T08:21:27.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7ff8d759e700 1 Processor -- start 2023-11-08T08:21:27.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7ff8d759e700 1 -- start start 2023-11-08T08:21:27.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce0e861740 1 -- 172.21.15.40:0/3103393342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1f4aa30 msgr2=0x21b7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.348+0000 7fce0e861740 1 --2- 172.21.15.40:0/3103393342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1f4aa30 0x21b7340 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fce0406ecf0 tx=0x7fcdfc010000 comp rx=0 tx=0).stop 2023-11-08T08:21:27.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce0e861740 1 -- 172.21.15.40:0/3103393342 shutdown_connections 2023-11-08T08:21:27.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce0e861740 1 --2- 172.21.15.40:0/3103393342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1f4aa30 0x21b7340 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce0e861740 1 --2- 172.21.15.40:0/3103393342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1eff9b0 0x21b6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce0e861740 1 --2- 172.21.15.40:0/3103393342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1effd80 0x21add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce0e861740 1 -- 172.21.15.40:0/3103393342 >> 172.21.15.40:0/3103393342 conn(0x1f00150 msgr2=0x2198350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:27.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce0e861740 1 -- 172.21.15.40:0/3103393342 shutdown_connections 2023-11-08T08:21:27.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce0e861740 1 -- 172.21.15.40:0/3103393342 wait complete. 2023-11-08T08:21:27.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce0e861740 1 -- start start 2023-11-08T08:21:27.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce0e861740 1 Processor -- start 2023-11-08T08:21:27.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7ff8d759e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff8d8071600 0x7ff8d80719d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7ff8d759e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff8d80b1730 0x7ff8d80b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7ff8d759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8d80b4930 0x7ff8d806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7ff8d759e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d80b1eb0 con 0x7ff8d8071600 2023-11-08T08:21:27.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7ff8d759e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d806ac60 con 0x7ff8d80b1730 2023-11-08T08:21:27.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7ff8d759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d80b4ee0 con 0x7ff8d80b4930 2023-11-08T08:21:27.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7ff8d659c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff8d8071600 0x7ff8d80719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7ff8d659c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff8d8071600 0x7ff8d80719d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56220/0 (socket says 172.21.15.40:56220) 2023-11-08T08:21:27.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7ff8d659c700 1 -- 172.21.15.40:0/817875979 learned_addr learned my addr 172.21.15.40:0/817875979 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:27.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7ff8d5d9b700 1 --2- 172.21.15.40:0/817875979 >> v2:172.21.15.169:3300/0 conn(0x7ff8d80b1730 0x7ff8d80b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7ff8d6d9d700 1 --2- 172.21.15.40:0/817875979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8d80b4930 0x7ff8d806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7ff8d659c700 1 -- 172.21.15.40:0/817875979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8d80b4930 msgr2=0x7ff8d806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7ff8d659c700 1 --2- 172.21.15.40:0/817875979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8d80b4930 0x7ff8d806c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7ff8d659c700 1 -- 172.21.15.40:0/817875979 >> v2:172.21.15.169:3300/0 conn(0x7ff8d80b1730 msgr2=0x7ff8d80b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7ff8d659c700 1 --2- 172.21.15.40:0/817875979 >> v2:172.21.15.169:3300/0 conn(0x7ff8d80b1730 0x7ff8d80b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7ff8d659c700 1 -- 172.21.15.40:0/817875979 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d80b5370 con 0x7ff8d8071600 2023-11-08T08:21:27.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce0e861740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1eff9b0 0x21bf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce0e861740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1effd80 0x21bc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce0e861740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1f4aa30 0x2170400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce0e861740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f84dc0 con 0x1eff9b0 2023-11-08T08:21:27.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce0e861740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x216f4e0 con 0x1f4aa30 2023-11-08T08:21:27.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce0e861740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x216f780 con 0x1effd80 2023-11-08T08:21:27.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce0e861740 10 client.0 _ll_register_callbacks cb 0x2133b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:21:27.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7ff8d659c700 1 --2- 172.21.15.40:0/817875979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8d8071600 0x7ff8d80719d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8066b40 tx=0x7ff8c8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68982cee1b32963 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce09375700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1eff9b0 0x21bf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce09375700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1eff9b0 0x21bf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56228/0 (socket says 172.21.15.40:56228) 2023-11-08T08:21:27.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce09375700 1 -- 172.21.15.40:0/3893420024 learned_addr learned my addr 172.21.15.40:0/3893420024 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce08b74700 1 --2- 172.21.15.40:0/3893420024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1effd80 0x21bc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7ff8d559a700 1 -- 172.21.15.40:0/817875979 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c800b8a0 con 0x7ff8d8071600 2023-11-08T08:21:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7ff8d559a700 1 -- 172.21.15.40:0/817875979 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c800ba00 con 0x7ff8d8071600 2023-11-08T08:21:27.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7ff8d559a700 1 -- 172.21.15.40:0/817875979 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c800bd90 con 0x7ff8d8071600 2023-11-08T08:21:27.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce09b76700 1 --2- 172.21.15.40:0/3893420024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1f4aa30 0x2170400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce09375700 1 -- 172.21.15.40:0/3893420024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1effd80 msgr2=0x21bc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce09375700 1 --2- 172.21.15.40:0/3893420024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1effd80 0x21bc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce09375700 1 -- 172.21.15.40:0/3893420024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1f4aa30 msgr2=0x2170400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce09375700 1 --2- 172.21.15.40:0/3893420024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1f4aa30 0x2170400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.352+0000 7fce09375700 1 -- 172.21.15.40:0/3893420024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce040d2a10 con 0x1eff9b0 2023-11-08T08:21:27.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce09375700 1 --2- 172.21.15.40:0/3893420024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1eff9b0 0x21bf390 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fce0004fdb0 tx=0x7fce0005b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2660d5c8a564101f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fcde3fff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:21:27.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7ff8d759e700 1 -- 172.21.15.40:0/817875979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8d8071600 msgr2=0x7ff8d80719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7ff8d759e700 1 --2- 172.21.15.40:0/817875979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8d8071600 0x7ff8d80719d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8066b40 tx=0x7ff8c8009bf0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fcde3fff700 1 -- 172.21.15.40:0/3893420024 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce00005a40 con 0x1eff9b0 2023-11-08T08:21:27.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fcde3fff700 1 -- 172.21.15.40:0/3893420024 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fce00005ba0 con 0x1eff9b0 2023-11-08T08:21:27.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce0e861740 1 -- 172.21.15.40:0/3893420024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x20d10e0 con 0x1eff9b0 2023-11-08T08:21:27.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fcde3fff700 1 -- 172.21.15.40:0/3893420024 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce00005d60 con 0x1eff9b0 2023-11-08T08:21:27.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce0e861740 10 client.5994 fetch_fsmap learned FSMap version 25 2023-11-08T08:21:27.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce0e861740 1 -- 172.21.15.40:0/3893420024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x1f4bc80 con 0x1eff9b0 2023-11-08T08:21:27.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fcde3fff700 1 -- 172.21.15.40:0/3893420024 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x20d10e0 con 0x1eff9b0 2023-11-08T08:21:27.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7ff8d759e700 1 -- 172.21.15.40:0/817875979 shutdown_connections 2023-11-08T08:21:27.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7ff8d759e700 1 --2- 172.21.15.40:0/817875979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8d80b4930 0x7ff8d806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7ff8d759e700 1 --2- 172.21.15.40:0/817875979 >> v2:172.21.15.169:3300/0 conn(0x7ff8d80b1730 0x7ff8d80b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7ff8d759e700 1 --2- 172.21.15.40:0/817875979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8d8071600 0x7ff8d80719d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7ff8d759e700 1 -- 172.21.15.40:0/817875979 >> 172.21.15.40:0/817875979 conn(0x7ff8d80b64e0 msgr2=0x7ff8d80d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:27.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fcde3fff700 1 -- 172.21.15.40:0/3893420024 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fce000545c0 con 0x1eff9b0 2023-11-08T08:21:27.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7ff8d759e700 1 -- 172.21.15.40:0/817875979 shutdown_connections 2023-11-08T08:21:27.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce0e861740 10 client.5994 fetch_fsmap finished waiting for FSMap version 25 2023-11-08T08:21:27.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce0e861740 10 client.5994 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-08T08:21:27.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce0e861740 20 client.5994 populate_metadata read hostname 'smithi040' 2023-11-08T08:21:27.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce0e861740 1 --2- 172.21.15.40:0/3893420024 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x216d3c0 0x216e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce0e861740 4 client.5994 mds_command: new command op to 5775 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-08T08:21:27.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce0e861740 1 -- 172.21.15.40:0/3893420024 --> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x20d0e00 con 0x216d3c0 2023-11-08T08:21:27.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7ff8d759e700 1 -- 172.21.15.40:0/817875979 wait complete. 2023-11-08T08:21:27.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce08b74700 1 --2- 172.21.15.40:0/3893420024 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x216d3c0 0x216e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce08b74700 1 --2- 172.21.15.40:0/3893420024 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x216d3c0 0x216e8f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=591edc9a8dd0362c server_cookie=8f860ff116eb4d6d in_seq=0 out_seq=0 2023-11-08T08:21:27.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fcde3fff700 10 client.5994 ms_handle_connect on v2:172.21.15.40:6833/2471311394 2023-11-08T08:21:27.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fcde3fff700 1 -- 172.21.15.40:0/3893420024 <== mds.0 v2:172.21.15.40:6833/2471311394 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x20d0e00 con 0x216d3c0 2023-11-08T08:21:27.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fcde3fff700 10 client.5994 handle_command_reply: tid=0 2023-11-08T08:21:27.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce0e861740 1 client.5994 shutdown 2023-11-08T08:21:27.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce0e861740 1 -- 172.21.15.40:0/3893420024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1eff9b0 msgr2=0x21bf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce0e861740 1 --2- 172.21.15.40:0/3893420024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1eff9b0 0x21bf390 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fce0004fdb0 tx=0x7fce0005b820 comp rx=0 tx=0).stop 2023-11-08T08:21:27.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce0e861740 1 -- 172.21.15.40:0/3893420024 shutdown_connections 2023-11-08T08:21:27.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce0e861740 1 --2- 172.21.15.40:0/3893420024 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x216d3c0 0x216e8f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce0e861740 1 --2- 172.21.15.40:0/3893420024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1f4aa30 0x2170400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce0e861740 1 --2- 172.21.15.40:0/3893420024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1effd80 0x21bc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fcde3fff700 0 client.5994 ms_handle_reset on v2:172.21.15.40:6833/2471311394 2023-11-08T08:21:27.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce0e861740 1 --2- 172.21.15.40:0/3893420024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1eff9b0 0x21bf390 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.356+0000 7fce0e861740 1 -- 172.21.15.40:0/3893420024 >> 172.21.15.40:0/3893420024 conn(0x1f00150 msgr2=0x21715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:27.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8d759e700 1 Processor -- start 2023-11-08T08:21:27.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7fce0e861740 1 -- 172.21.15.40:0/3893420024 shutdown_connections 2023-11-08T08:21:27.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7fce0e861740 1 -- 172.21.15.40:0/3893420024 wait complete. 2023-11-08T08:21:27.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8d759e700 1 -- start start 2023-11-08T08:21:27.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7fce0e861740 20 client.5994 trim_cache size 0 max 16384 2023-11-08T08:21:27.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8d759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8d8071600 0x7ff8d815dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8d759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8d80b1730 0x7ff8d815e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8d759e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8d80b4930 0x7ff8d809f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8d759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d809c670 con 0x7ff8d8071600 2023-11-08T08:21:27.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8d759e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d81e5340 con 0x7ff8d80b4930 2023-11-08T08:21:27.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8d759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d81e5620 con 0x7ff8d80b1730 2023-11-08T08:21:27.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8d659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8d8071600 0x7ff8d815dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8d659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8d8071600 0x7ff8d815dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56232/0 (socket says 172.21.15.40:56232) 2023-11-08T08:21:27.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8d659c700 1 -- 172.21.15.40:0/2398014643 learned_addr learned my addr 172.21.15.40:0/2398014643 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:27.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8d6d9d700 1 --2- 172.21.15.40:0/2398014643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8d80b4930 0x7ff8d809f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8d659c700 1 -- 172.21.15.40:0/2398014643 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8d80b1730 msgr2=0x7ff8d815e160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-08T08:21:27.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8d659c700 1 --2- 172.21.15.40:0/2398014643 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8d80b1730 0x7ff8d815e160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8d659c700 1 -- 172.21.15.40:0/2398014643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8d80b4930 msgr2=0x7ff8d809f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8d659c700 1 --2- 172.21.15.40:0/2398014643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8d80b4930 0x7ff8d809f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8d659c700 1 -- 172.21.15.40:0/2398014643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c8009570 con 0x7ff8d8071600 2023-11-08T08:21:27.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8d659c700 1 --2- 172.21.15.40:0/2398014643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8d8071600 0x7ff8d815dad0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8000c00 tx=0x7ff8c8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8c6ffd700 1 -- 172.21.15.40:0/2398014643 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c80038c0 con 0x7ff8d8071600 2023-11-08T08:21:27.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8c6ffd700 1 -- 172.21.15.40:0/2398014643 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c801e070 con 0x7ff8d8071600 2023-11-08T08:21:27.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8d759e700 1 -- 172.21.15.40:0/2398014643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d806a6a0 con 0x7ff8d8071600 2023-11-08T08:21:27.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8c6ffd700 1 -- 172.21.15.40:0/2398014643 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8003c80 con 0x7ff8d8071600 2023-11-08T08:21:27.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.360+0000 7ff8d759e700 1 -- 172.21.15.40:0/2398014643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d8072210 con 0x7ff8d8071600 2023-11-08T08:21:27.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.364+0000 7ff8d759e700 1 -- 172.21.15.40:0/2398014643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff8d806a980 con 0x7ff8d8071600 2023-11-08T08:21:27.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.364+0000 7ff8c6ffd700 1 -- 172.21.15.40:0/2398014643 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff8c800f050 con 0x7ff8d8071600 2023-11-08T08:21:27.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.368+0000 7ff8c6ffd700 1 --2- 172.21.15.40:0/2398014643 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff8b8064ee0 0x7ff8b8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.368+0000 7ff8c6ffd700 1 -- 172.21.15.40:0/2398014643 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff8c8020070 con 0x7ff8d8071600 2023-11-08T08:21:27.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.368+0000 7ff8c6ffd700 1 -- 172.21.15.40:0/2398014643 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+502 (secure 0 0 0) 0x7ff8c80842b0 con 0x7ff8d8071600 2023-11-08T08:21:27.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.368+0000 7ff8d759e700 1 -- 172.21.15.40:0/2398014643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7ff8d804ee90 con 0x7ff8d8071600 2023-11-08T08:21:27.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.368+0000 7ff8d5d9b700 1 --2- 172.21.15.40:0/2398014643 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff8b8064ee0 0x7ff8b8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.368+0000 7ff8d5d9b700 1 --2- 172.21.15.40:0/2398014643 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff8b8064ee0 0x7ff8b8067390 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff8d81f4cd0 tx=0x7ff8cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.380+0000 7fce0e861740 1 Processor -- start 2023-11-08T08:21:27.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.380+0000 7fce0e861740 1 -- start start 2023-11-08T08:21:27.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.380+0000 7fce0e861740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1eff9b0 0x21c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.380+0000 7fce0e861740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1effd80 0x2181320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.380+0000 7fce0e861740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1f4aa30 0x2181860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.380+0000 7fce0e861740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x20dc060 con 0x1effd80 2023-11-08T08:21:27.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.380+0000 7fce0e861740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x20d2400 con 0x1f4aa30 2023-11-08T08:21:27.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.380+0000 7fce0e861740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x2094c60 con 0x1eff9b0 2023-11-08T08:21:27.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.380+0000 7fce09b76700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1f4aa30 0x2181860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.380+0000 7fce09375700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1eff9b0 0x21c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.380+0000 7fce09b76700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1f4aa30 0x2181860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:38682/0 (socket says 172.21.15.40:38682) 2023-11-08T08:21:27.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.380+0000 7fce09b76700 1 -- 172.21.15.40:0/2927998297 learned_addr learned my addr 172.21.15.40:0/2927998297 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:27.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.380+0000 7fce08b74700 1 --2- 172.21.15.40:0/2927998297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1effd80 0x2181320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.384+0000 7fce09375700 1 -- 172.21.15.40:0/2927998297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1f4aa30 msgr2=0x2181860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.384+0000 7fce09375700 1 --2- 172.21.15.40:0/2927998297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1f4aa30 0x2181860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.384+0000 7fce09375700 1 -- 172.21.15.40:0/2927998297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1effd80 msgr2=0x2181320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.384+0000 7fce09375700 1 --2- 172.21.15.40:0/2927998297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1effd80 0x2181320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.384+0000 7fce09375700 1 -- 172.21.15.40:0/2927998297 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce040716f0 con 0x1eff9b0 2023-11-08T08:21:27.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.384+0000 7fce08b74700 1 --2- 172.21.15.40:0/2927998297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1effd80 0x2181320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:27.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.384+0000 7fce09375700 1 --2- 172.21.15.40:0/2927998297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1eff9b0 0x21c5d00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fce0000e760 tx=0x7fce00068000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e56f4d88471a8d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.384+0000 7fcdef7fe700 1 -- 172.21.15.40:0/2927998297 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce000800a0 con 0x1eff9b0 2023-11-08T08:21:27.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.384+0000 7fcdef7fe700 1 -- 172.21.15.40:0/2927998297 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0004f980 con 0x1eff9b0 2023-11-08T08:21:27.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.384+0000 7fcdef7fe700 1 -- 172.21.15.40:0/2927998297 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0004fbf0 con 0x1eff9b0 2023-11-08T08:21:27.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.384+0000 7fce0e861740 1 -- 172.21.15.40:0/2927998297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1eff9b0 msgr2=0x21c5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.384+0000 7fce0e861740 1 --2- 172.21.15.40:0/2927998297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1eff9b0 0x21c5d00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fce0000e760 tx=0x7fce00068000 comp rx=0 tx=0).stop 2023-11-08T08:21:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.384+0000 7fce0e861740 1 -- 172.21.15.40:0/2927998297 shutdown_connections 2023-11-08T08:21:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.384+0000 7fce0e861740 1 --2- 172.21.15.40:0/2927998297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1f4aa30 0x2181860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.384+0000 7fce0e861740 1 --2- 172.21.15.40:0/2927998297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1effd80 0x2181320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.384+0000 7fce0e861740 1 --2- 172.21.15.40:0/2927998297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1eff9b0 0x21c5d00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.384+0000 7fce0e861740 1 -- 172.21.15.40:0/2927998297 >> 172.21.15.40:0/2927998297 conn(0x1f00150 msgr2=0x21715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:27.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.384+0000 7fce0e861740 1 -- 172.21.15.40:0/2927998297 shutdown_connections 2023-11-08T08:21:27.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.384+0000 7fce0e861740 1 -- 172.21.15.40:0/2927998297 wait complete. 2023-11-08T08:21:27.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.384+0000 7fce0e861740 1 -- start start 2023-11-08T08:21:27.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce0e861740 1 Processor -- start 2023-11-08T08:21:27.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce0e861740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1eff9b0 0x21974d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce0e861740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1effd80 0x2181980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce0e861740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1f4aa30 0x2201c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce0e861740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x20dabc0 con 0x1effd80 2023-11-08T08:21:27.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce0e861740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x20d6ef0 con 0x1f4aa30 2023-11-08T08:21:27.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce0e861740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x2089290 con 0x1eff9b0 2023-11-08T08:21:27.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce0e861740 10 client.0 _ll_register_callbacks cb 0x20fe910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:21:27.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce08b74700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1effd80 0x2181980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce08b74700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1effd80 0x2181980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56260/0 (socket says 172.21.15.40:56260) 2023-11-08T08:21:27.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce08b74700 1 -- 172.21.15.40:0/2319245732 learned_addr learned my addr 172.21.15.40:0/2319245732 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:27.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce09375700 1 --2- 172.21.15.40:0/2319245732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1eff9b0 0x21974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce09b76700 1 --2- 172.21.15.40:0/2319245732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1f4aa30 0x2201c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce08b74700 1 -- 172.21.15.40:0/2319245732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1eff9b0 msgr2=0x21974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce08b74700 1 --2- 172.21.15.40:0/2319245732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1eff9b0 0x21974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce08b74700 1 -- 172.21.15.40:0/2319245732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1f4aa30 msgr2=0x2201c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce08b74700 1 --2- 172.21.15.40:0/2319245732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1f4aa30 0x2201c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce08b74700 1 -- 172.21.15.40:0/2319245732 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1f4bc80 con 0x1effd80 2023-11-08T08:21:27.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce09b76700 1 --2- 172.21.15.40:0/2319245732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1f4aa30 0x2201c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:21:27.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce08b74700 1 --2- 172.21.15.40:0/2319245732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1effd80 0x2181980 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4014c20 tx=0x7fcdf4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4528254a08b0f2ed server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fcde3fff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:21:27.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fcde3fff700 1 -- 172.21.15.40:0/2319245732 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4023b70 con 0x1effd80 2023-11-08T08:21:27.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fcde3fff700 1 -- 172.21.15.40:0/2319245732 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fcdf4031d90 con 0x1effd80 2023-11-08T08:21:27.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.388+0000 7fce0e861740 1 -- 172.21.15.40:0/2319245732 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1f7c500 con 0x1effd80 2023-11-08T08:21:27.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.392+0000 7fcde3fff700 1 -- 172.21.15.40:0/2319245732 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdf4023cd0 con 0x1effd80 2023-11-08T08:21:27.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.392+0000 7fce0e861740 10 client.6003 fetch_fsmap learned FSMap version 25 2023-11-08T08:21:27.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.392+0000 7fce0e861740 1 -- 172.21.15.40:0/2319245732 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x2084bb0 con 0x1effd80 2023-11-08T08:21:27.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.392+0000 7fcde3fff700 1 -- 172.21.15.40:0/2319245732 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1f7c500 con 0x1effd80 2023-11-08T08:21:27.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.392+0000 7fcde3fff700 1 -- 172.21.15.40:0/2319245732 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fcdf4005020 con 0x1effd80 2023-11-08T08:21:27.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.392+0000 7fce0e861740 10 client.6003 fetch_fsmap finished waiting for FSMap version 25 2023-11-08T08:21:27.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.392+0000 7fce0e861740 10 client.6003 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-08T08:21:27.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.392+0000 7fce0e861740 20 client.6003 populate_metadata read hostname 'smithi040' 2023-11-08T08:21:27.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.392+0000 7fce0e861740 1 --2- 172.21.15.40:0/2319245732 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x2202150 0x2192360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.392+0000 7fce0e861740 4 client.6003 mds_command: new command op to 5775 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-11-08T08:21:27.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.392+0000 7fce0e861740 1 -- 172.21.15.40:0/2319245732 --> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x1f0d500 con 0x2202150 2023-11-08T08:21:27.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.392+0000 7fce09375700 1 --2- 172.21.15.40:0/2319245732 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x2202150 0x2192360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.392+0000 7fce09375700 1 --2- 172.21.15.40:0/2319245732 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x2202150 0x2192360 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47f76bdde9b91897 server_cookie=ff95cbab08897a61 in_seq=0 out_seq=0 2023-11-08T08:21:27.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.392+0000 7fcde3fff700 10 client.6003 ms_handle_connect on v2:172.21.15.40:6833/2471311394 2023-11-08T08:21:27.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fcde3fff700 1 -- 172.21.15.40:0/2319245732 <== mds.0 v2:172.21.15.40:6833/2471311394 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x1f0d500 con 0x2202150 2023-11-08T08:21:27.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fcde3fff700 10 client.6003 handle_command_reply: tid=0 2023-11-08T08:21:27.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0e861740 1 client.6003 shutdown 2023-11-08T08:21:27.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0e861740 1 -- 172.21.15.40:0/2319245732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1effd80 msgr2=0x2181980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0e861740 1 --2- 172.21.15.40:0/2319245732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1effd80 0x2181980 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4014c20 tx=0x7fcdf4010000 comp rx=0 tx=0).stop 2023-11-08T08:21:27.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0e861740 1 -- 172.21.15.40:0/2319245732 shutdown_connections 2023-11-08T08:21:27.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0e861740 1 --2- 172.21.15.40:0/2319245732 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x2202150 0x2192360 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0e861740 1 --2- 172.21.15.40:0/2319245732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1f4aa30 0x2201c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0e861740 1 --2- 172.21.15.40:0/2319245732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1effd80 0x2181980 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0e861740 1 --2- 172.21.15.40:0/2319245732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1eff9b0 0x21974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0e861740 1 -- 172.21.15.40:0/2319245732 >> 172.21.15.40:0/2319245732 conn(0x1f00150 msgr2=0x21bc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:27.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fcde3fff700 0 client.6003 ms_handle_reset on v2:172.21.15.40:6833/2471311394 2023-11-08T08:21:27.416 INFO:tasks.ceph.mds.a.smithi040.stderr:2023-11-08T08:21:27.396+0000 7efe4f3ee700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5955=0-4194304@1} caps={5955=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5955 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55ebd23d5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T08:21:27.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0e861740 1 -- 172.21.15.40:0/2319245732 shutdown_connections 2023-11-08T08:21:27.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0e861740 1 -- 172.21.15.40:0/2319245732 wait complete. 2023-11-08T08:21:27.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0e861740 20 client.6003 trim_cache size 0 max 16384 2023-11-08T08:21:27.417 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:21:27.418 INFO:teuthology.orchestra.run.smithi040.stdout: "return_code": 0, 2023-11-08T08:21:27.418 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_tag": "4541f6e1-31da-419f-97de-c48ef4872ec8", 2023-11-08T08:21:27.418 INFO:teuthology.orchestra.run.smithi040.stdout: "mode": "asynchronous" 2023-11-08T08:21:27.418 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:21:27.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0a377700 1 -- 172.21.15.40:0/1862245849 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcde8064f10 msgr2=0x7fcde80673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0a377700 1 --2- 172.21.15.40:0/1862245849 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcde8064f10 0x7fcde80673c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fce040cbba0 tx=0x7fcdf4009040 comp rx=0 tx=0).stop 2023-11-08T08:21:27.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0a377700 1 -- 172.21.15.40:0/1862245849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0406a510 msgr2=0x7fce040cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0a377700 1 --2- 172.21.15.40:0/1862245849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0406a510 0x7fce040cb660 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fce0000e790 tx=0x7fce0000eb50 comp rx=0 tx=0).stop 2023-11-08T08:21:27.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0a377700 1 -- 172.21.15.40:0/1862245849 shutdown_connections 2023-11-08T08:21:27.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0a377700 1 --2- 172.21.15.40:0/1862245849 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcde8064f10 0x7fcde80673c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0a377700 1 --2- 172.21.15.40:0/1862245849 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce0415c6e0 0x7fce041e7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0a377700 1 --2- 172.21.15.40:0/1862245849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce04071fe0 0x7fce040cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0a377700 1 --2- 172.21.15.40:0/1862245849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0406a510 0x7fce040cb660 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0a377700 1 -- 172.21.15.40:0/1862245849 >> 172.21.15.40:0/1862245849 conn(0x7fce040b65b0 msgr2=0x7fce040d0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:27.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0a377700 1 -- 172.21.15.40:0/1862245849 shutdown_connections 2023-11-08T08:21:27.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.396+0000 7fce0a377700 1 -- 172.21.15.40:0/1862245849 wait complete. 2023-11-08T08:21:27.424 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-08T08:21:27.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.604+0000 7ff2dd012700 1 Processor -- start 2023-11-08T08:21:27.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.604+0000 7ff2dd012700 1 -- start start 2023-11-08T08:21:27.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.608+0000 7ff2dd012700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff2d80cd720 0x7ff2d815c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.608+0000 7ff2dd012700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2d815cf10 0x7ff2d8156340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.608+0000 7ff2dd012700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d815d350 con 0x7ff2d80aed90 2023-11-08T08:21:27.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.608+0000 7ff2dd012700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d80cdc20 con 0x7ff2d80cd720 2023-11-08T08:21:27.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.608+0000 7ff2dd012700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d80cdf00 con 0x7ff2d815cf10 2023-11-08T08:21:27.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.608+0000 7ff2d7fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff2d80aed90 0x7ff2d806c050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54382/0 (socket says 172.21.15.40:54382) 2023-11-08T08:21:27.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.608+0000 7ff2d7fff700 1 -- 172.21.15.40:0/2192821728 learned_addr learned my addr 172.21.15.40:0/2192821728 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:27.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.608+0000 7ff2dc811700 1 --2- 172.21.15.40:0/2192821728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2d815cf10 0x7ff2d8156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.608+0000 7ff2d77fe700 1 --2- 172.21.15.40:0/2192821728 >> v2:172.21.15.169:3300/0 conn(0x7ff2d80cd720 0x7ff2d815c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.608+0000 7ff2dc811700 1 -- 172.21.15.40:0/2192821728 >> v2:172.21.15.169:3300/0 conn(0x7ff2d80cd720 msgr2=0x7ff2d815c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.608+0000 7ff2dc811700 1 --2- 172.21.15.40:0/2192821728 >> v2:172.21.15.169:3300/0 conn(0x7ff2d80cd720 0x7ff2d815c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.608+0000 7ff2dc811700 1 -- 172.21.15.40:0/2192821728 >> v1:172.21.15.40:6789/0 conn(0x7ff2d80aed90 legacy=0x7ff2d806c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.608+0000 7ff2d6ffd700 1 -- 172.21.15.40:0/2192821728 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 20605497 0 0) 0x7ff2d815d350 con 0x7ff2d80aed90 2023-11-08T08:21:27.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.608+0000 7ff2dc811700 1 -- 172.21.15.40:0/2192821728 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d80ce1e0 con 0x7ff2d815cf10 2023-11-08T08:21:27.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.608+0000 7ff2dc811700 1 --2- 172.21.15.40:0/2192821728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2d815cf10 0x7ff2d8156340 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009850 tx=0x7ff2d0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34348d93fba461a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.612+0000 7ff2d6ffd700 1 -- 172.21.15.40:0/2192821728 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0011830 con 0x7ff2d815cf10 2023-11-08T08:21:27.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.612+0000 7ff2d6ffd700 1 -- 172.21.15.40:0/2192821728 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d0011990 con 0x7ff2d815cf10 2023-11-08T08:21:27.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.612+0000 7ff2d6ffd700 1 -- 172.21.15.40:0/2192821728 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0011d40 con 0x7ff2d815cf10 2023-11-08T08:21:27.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.612+0000 7ff2dd012700 1 -- 172.21.15.40:0/2192821728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2d815cf10 msgr2=0x7ff2d8156340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.612+0000 7ff2dd012700 1 --2- 172.21.15.40:0/2192821728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2d815cf10 0x7ff2d8156340 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009850 tx=0x7ff2d0009b60 comp rx=0 tx=0).stop 2023-11-08T08:21:27.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.612+0000 7ff2dd012700 1 -- 172.21.15.40:0/2192821728 shutdown_connections 2023-11-08T08:21:27.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.612+0000 7ff2dd012700 1 --2- 172.21.15.40:0/2192821728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2d815cf10 0x7ff2d8156340 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.612+0000 7ff2dd012700 1 --2- 172.21.15.40:0/2192821728 >> v2:172.21.15.169:3300/0 conn(0x7ff2d80cd720 0x7ff2d815c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.612+0000 7ff2dd012700 1 -- 172.21.15.40:0/2192821728 >> 172.21.15.40:0/2192821728 conn(0x7ff2d80d03b0 msgr2=0x7ff2d80cc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:27.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.612+0000 7ff2dd012700 1 -- 172.21.15.40:0/2192821728 shutdown_connections 2023-11-08T08:21:27.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.612+0000 7ff2dd012700 1 -- 172.21.15.40:0/2192821728 wait complete. 2023-11-08T08:21:27.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.616+0000 7ff2dd012700 1 Processor -- start 2023-11-08T08:21:27.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.616+0000 7ff2dd012700 1 -- start start 2023-11-08T08:21:27.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.616+0000 7ff2dd012700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2d80aed90 0x7ff2d80c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.616+0000 7ff2dd012700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2d80cd720 0x7ff2d81546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.616+0000 7ff2dd012700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d815cf10 0x7ff2d80c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.616+0000 7ff2dd012700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d8153620 con 0x7ff2d815cf10 2023-11-08T08:21:27.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.616+0000 7ff2dd012700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d8153870 con 0x7ff2d80aed90 2023-11-08T08:21:27.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.616+0000 7ff2dd012700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d8153b50 con 0x7ff2d80cd720 2023-11-08T08:21:27.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.616+0000 7ff2dc811700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d815cf10 0x7ff2d80c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.616+0000 7ff2dc811700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d815cf10 0x7ff2d80c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56272/0 (socket says 172.21.15.40:56272) 2023-11-08T08:21:27.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.616+0000 7ff2dc811700 1 -- 172.21.15.40:0/74933325 learned_addr learned my addr 172.21.15.40:0/74933325 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:27.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.616+0000 7ff2d77fe700 1 --2- 172.21.15.40:0/74933325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2d80cd720 0x7ff2d81546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.616+0000 7ff2d7fff700 1 --2- 172.21.15.40:0/74933325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2d80aed90 0x7ff2d80c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.616+0000 7ff2dc811700 1 -- 172.21.15.40:0/74933325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2d80cd720 msgr2=0x7ff2d81546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.616+0000 7ff2dc811700 1 --2- 172.21.15.40:0/74933325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2d80cd720 0x7ff2d81546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.616+0000 7ff2dc811700 1 -- 172.21.15.40:0/74933325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2d80aed90 msgr2=0x7ff2d80c6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.616+0000 7ff2dc811700 1 --2- 172.21.15.40:0/74933325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2d80aed90 0x7ff2d80c6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.616+0000 7ff2dc811700 1 -- 172.21.15.40:0/74933325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d00094b0 con 0x7ff2d815cf10 2023-11-08T08:21:27.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.616+0000 7ff2dc811700 1 --2- 172.21.15.40:0/74933325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d815cf10 0x7ff2d80c83c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0014040 tx=0x7ff2d0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.620+0000 7ff2d4ff9700 1 -- 172.21.15.40:0/74933325 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d00198c0 con 0x7ff2d815cf10 2023-11-08T08:21:27.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.620+0000 7ff2d4ff9700 1 -- 172.21.15.40:0/74933325 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d001e070 con 0x7ff2d815cf10 2023-11-08T08:21:27.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.620+0000 7ff2dd012700 1 -- 172.21.15.40:0/74933325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d80ce7a0 con 0x7ff2d815cf10 2023-11-08T08:21:27.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.620+0000 7ff2d4ff9700 1 -- 172.21.15.40:0/74933325 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0019c60 con 0x7ff2d815cf10 2023-11-08T08:21:27.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.620+0000 7ff2dd012700 1 -- 172.21.15.40:0/74933325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d80ce4c0 con 0x7ff2d815cf10 2023-11-08T08:21:27.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.624+0000 7ff2d4ff9700 1 -- 172.21.15.40:0/74933325 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff2d000f050 con 0x7ff2d815cf10 2023-11-08T08:21:27.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.624+0000 7ff2d4ff9700 1 --2- 172.21.15.40:0/74933325 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff2c4064ee0 0x7ff2c4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.624+0000 7ff2d4ff9700 1 -- 172.21.15.40:0/74933325 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff2d0082ff0 con 0x7ff2d815cf10 2023-11-08T08:21:27.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.624+0000 7ff2d7fff700 1 --2- 172.21.15.40:0/74933325 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff2c4064ee0 0x7ff2c4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.628+0000 7ff2d7fff700 1 --2- 172.21.15.40:0/74933325 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff2c4064ee0 0x7ff2c4067390 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80012e0 tx=0x7ff2c8001b70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.628+0000 7ff2e14fd740 1 Processor -- start 2023-11-08T08:21:27.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.628+0000 7ff2e14fd740 1 -- start start 2023-11-08T08:21:27.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.628+0000 7ff2e14fd740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604c10 0x289b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.628+0000 7ff2e14fd740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2604840 0x28c0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.628+0000 7ff2e14fd740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x264f8c0 0x28c1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.628+0000 7ff2e14fd740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x2773d20 con 0x264f8c0 2023-11-08T08:21:27.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.628+0000 7ff2e14fd740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x27e3d60 con 0x2604840 2023-11-08T08:21:27.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.628+0000 7ff2e14fd740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x27dd060 con 0x2604c10 2023-11-08T08:21:27.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.628+0000 7ff2dc811700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x264f8c0 0x28c1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.628+0000 7ff2dc811700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x264f8c0 0x28c1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56278/0 (socket says 172.21.15.40:56278) 2023-11-08T08:21:27.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.628+0000 7ff2dc811700 1 -- 172.21.15.40:0/1277469964 learned_addr learned my addr 172.21.15.40:0/1277469964 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.628+0000 7ff2d77fe700 1 --2- 172.21.15.40:0/1277469964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2604840 0x28c0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.628+0000 7ff2d7fff700 1 --2- 172.21.15.40:0/1277469964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604c10 0x289b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.632+0000 7ff2dc811700 1 -- 172.21.15.40:0/1277469964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604c10 msgr2=0x289b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.632+0000 7ff2dc811700 1 --2- 172.21.15.40:0/1277469964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604c10 0x289b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.632+0000 7ff2dc811700 1 -- 172.21.15.40:0/1277469964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2604840 msgr2=0x28c0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.632+0000 7ff2dc811700 1 --2- 172.21.15.40:0/1277469964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2604840 0x28c0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.632+0000 7ff2dc811700 1 -- 172.21.15.40:0/1277469964 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x28030a0 con 0x264f8c0 2023-11-08T08:21:27.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.632+0000 7ff2dc811700 1 --2- 172.21.15.40:0/1277469964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x264f8c0 0x28c1200 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0006cf0 tx=0x7ff2d0054b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeebe0d2c43d74f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.632+0000 7ff2bdffb700 1 -- 172.21.15.40:0/1277469964 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0052e60 con 0x264f8c0 2023-11-08T08:21:27.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.632+0000 7ff2bdffb700 1 -- 172.21.15.40:0/1277469964 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d0054150 con 0x264f8c0 2023-11-08T08:21:27.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.632+0000 7ff2bdffb700 1 -- 172.21.15.40:0/1277469964 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d005c860 con 0x264f8c0 2023-11-08T08:21:27.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.632+0000 7ff2e14fd740 1 -- 172.21.15.40:0/1277469964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x264f8c0 msgr2=0x28c1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.632+0000 7ff2e14fd740 1 --2- 172.21.15.40:0/1277469964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x264f8c0 0x28c1200 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0006cf0 tx=0x7ff2d0054b00 comp rx=0 tx=0).stop 2023-11-08T08:21:27.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.632+0000 7ff2e14fd740 1 -- 172.21.15.40:0/1277469964 shutdown_connections 2023-11-08T08:21:27.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.632+0000 7ff2e14fd740 1 --2- 172.21.15.40:0/1277469964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x264f8c0 0x28c1200 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.632+0000 7ff2e14fd740 1 --2- 172.21.15.40:0/1277469964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2604840 0x28c0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.632+0000 7ff2e14fd740 1 --2- 172.21.15.40:0/1277469964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604c10 0x289b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.632+0000 7ff2e14fd740 1 -- 172.21.15.40:0/1277469964 >> 172.21.15.40:0/1277469964 conn(0x2604fe0 msgr2=0x289db80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:27.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.632+0000 7ff2e14fd740 1 -- 172.21.15.40:0/1277469964 shutdown_connections 2023-11-08T08:21:27.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.632+0000 7ff2e14fd740 1 -- 172.21.15.40:0/1277469964 wait complete. 2023-11-08T08:21:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.632+0000 7ff2e14fd740 1 -- start start 2023-11-08T08:21:27.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.632+0000 7ff2e14fd740 1 Processor -- start 2023-11-08T08:21:27.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2e14fd740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604840 0x28ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2e14fd740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2604c10 0x28ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2e14fd740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x264f8c0 0x28cc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2e14fd740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x2689c50 con 0x264f8c0 2023-11-08T08:21:27.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2e14fd740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x28aadf0 con 0x2604c10 2023-11-08T08:21:27.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2e14fd740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x28ab090 con 0x2604840 2023-11-08T08:21:27.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2e14fd740 10 client.0 _ll_register_callbacks cb 0x27d69a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:21:27.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2dc811700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x264f8c0 0x28cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2dc811700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x264f8c0 0x28cc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56294/0 (socket says 172.21.15.40:56294) 2023-11-08T08:21:27.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2dc811700 1 -- 172.21.15.40:0/2644597882 learned_addr learned my addr 172.21.15.40:0/2644597882 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:27.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2d7fff700 1 --2- 172.21.15.40:0/2644597882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604840 0x28ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2d77fe700 1 --2- 172.21.15.40:0/2644597882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2604c10 0x28ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2dc811700 1 -- 172.21.15.40:0/2644597882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604840 msgr2=0x28ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2dc811700 1 --2- 172.21.15.40:0/2644597882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604840 0x28ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2dc811700 1 -- 172.21.15.40:0/2644597882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2604c10 msgr2=0x28ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2dc811700 1 --2- 172.21.15.40:0/2644597882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2604c10 0x28ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2dc811700 1 -- 172.21.15.40:0/2644597882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d80ce4c0 con 0x264f8c0 2023-11-08T08:21:27.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2dc811700 1 --2- 172.21.15.40:0/2644597882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x264f8c0 0x28cc850 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0006e50 tx=0x7ff2d0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671247c9a6068eb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:21:27.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2b6ffd700 1 -- 172.21.15.40:0/2644597882 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d005ca60 con 0x264f8c0 2023-11-08T08:21:27.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2b6ffd700 1 -- 172.21.15.40:0/2644597882 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff2d0058bc0 con 0x264f8c0 2023-11-08T08:21:27.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2e14fd740 1 -- 172.21.15.40:0/2644597882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x27d60e0 con 0x264f8c0 2023-11-08T08:21:27.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2b6ffd700 1 -- 172.21.15.40:0/2644597882 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2d0058dc0 con 0x264f8c0 2023-11-08T08:21:27.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2e14fd740 10 client.6015 fetch_fsmap learned FSMap version 25 2023-11-08T08:21:27.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2e14fd740 1 -- 172.21.15.40:0/2644597882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x2650b10 con 0x264f8c0 2023-11-08T08:21:27.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2b6ffd700 1 -- 172.21.15.40:0/2644597882 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x27d60e0 con 0x264f8c0 2023-11-08T08:21:27.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2b6ffd700 1 -- 172.21.15.40:0/2644597882 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff2d0050a50 con 0x264f8c0 2023-11-08T08:21:27.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2e14fd740 10 client.6015 fetch_fsmap finished waiting for FSMap version 25 2023-11-08T08:21:27.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2e14fd740 10 client.6015 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-08T08:21:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2e14fd740 20 client.6015 populate_metadata read hostname 'smithi040' 2023-11-08T08:21:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2e14fd740 1 --2- 172.21.15.40:0/2644597882 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x28763e0 0x28a90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2e14fd740 4 client.6015 mds_command: new command op to 5775 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-08T08:21:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.636+0000 7ff2e14fd740 1 -- 172.21.15.40:0/2644597882 --> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x27d5e00 con 0x28763e0 2023-11-08T08:21:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.640+0000 7ff2d7fff700 1 --2- 172.21.15.40:0/2644597882 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x28763e0 0x28a90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.640+0000 7ff2d7fff700 1 --2- 172.21.15.40:0/2644597882 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x28763e0 0x28a90c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f0ff30b23fe15e server_cookie=77aaf1a7111767c in_seq=0 out_seq=0 2023-11-08T08:21:27.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.640+0000 7ff2b6ffd700 10 client.6015 ms_handle_connect on v2:172.21.15.40:6833/2471311394 2023-11-08T08:21:27.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.640+0000 7ff2b6ffd700 1 -- 172.21.15.40:0/2644597882 <== mds.0 v2:172.21.15.40:6833/2471311394 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x27d5e00 con 0x28763e0 2023-11-08T08:21:27.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.640+0000 7ff2b6ffd700 10 client.6015 handle_command_reply: tid=0 2023-11-08T08:21:27.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.640+0000 7ff2e14fd740 1 client.6015 shutdown 2023-11-08T08:21:27.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.640+0000 7ff2e14fd740 1 -- 172.21.15.40:0/2644597882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x264f8c0 msgr2=0x28cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.640+0000 7ff2e14fd740 1 --2- 172.21.15.40:0/2644597882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x264f8c0 0x28cc850 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0006e50 tx=0x7ff2d0015000 comp rx=0 tx=0).stop 2023-11-08T08:21:27.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.640+0000 7ff2e14fd740 1 -- 172.21.15.40:0/2644597882 shutdown_connections 2023-11-08T08:21:27.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.640+0000 7ff2e14fd740 1 --2- 172.21.15.40:0/2644597882 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x28763e0 0x28a90c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.640+0000 7ff2e14fd740 1 --2- 172.21.15.40:0/2644597882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x264f8c0 0x28cc850 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.640+0000 7ff2e14fd740 1 --2- 172.21.15.40:0/2644597882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x2604c10 0x28ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.640+0000 7ff2e14fd740 1 --2- 172.21.15.40:0/2644597882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604840 0x28ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.640+0000 7ff2e14fd740 1 -- 172.21.15.40:0/2644597882 >> 172.21.15.40:0/2644597882 conn(0x2604fe0 msgr2=0x28734b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:27.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.640+0000 7ff2b6ffd700 0 client.6015 ms_handle_reset on v2:172.21.15.40:6833/2471311394 2023-11-08T08:21:27.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.640+0000 7ff2e14fd740 1 -- 172.21.15.40:0/2644597882 shutdown_connections 2023-11-08T08:21:27.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.640+0000 7ff2e14fd740 1 -- 172.21.15.40:0/2644597882 wait complete. 2023-11-08T08:21:27.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.640+0000 7ff2e14fd740 20 client.6015 trim_cache size 0 max 16384 2023-11-08T08:21:27.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.664+0000 7ff2e14fd740 1 Processor -- start 2023-11-08T08:21:27.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.664+0000 7ff2e14fd740 1 -- start start 2023-11-08T08:21:27.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.664+0000 7ff2e14fd740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2604840 0x28cb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.664+0000 7ff2e14fd740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604c10 0x28cb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.664+0000 7ff2e14fd740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x264f8c0 0x28cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.664+0000 7ff2e14fd740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x2773200 con 0x2604840 2023-11-08T08:21:27.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.664+0000 7ff2e14fd740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x27e08d0 con 0x264f8c0 2023-11-08T08:21:27.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.664+0000 7ff2e14fd740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x27dc870 con 0x2604c10 2023-11-08T08:21:27.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.664+0000 7ff2d7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2604840 0x28cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.664+0000 7ff2d7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2604840 0x28cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56304/0 (socket says 172.21.15.40:56304) 2023-11-08T08:21:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.664+0000 7ff2d7fff700 1 -- 172.21.15.40:0/188608728 learned_addr learned my addr 172.21.15.40:0/188608728 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:27.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.664+0000 7ff2dc811700 1 --2- 172.21.15.40:0/188608728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x264f8c0 0x28cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.664+0000 7ff2d77fe700 1 --2- 172.21.15.40:0/188608728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604c10 0x28cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.664+0000 7ff2d7fff700 1 -- 172.21.15.40:0/188608728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604c10 msgr2=0x28cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.664+0000 7ff2d7fff700 1 --2- 172.21.15.40:0/188608728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604c10 0x28cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.668+0000 7ff2d7fff700 1 -- 172.21.15.40:0/188608728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x264f8c0 msgr2=0x28cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.668+0000 7ff2d7fff700 1 --2- 172.21.15.40:0/188608728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x264f8c0 0x28cbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.668+0000 7ff2d7fff700 1 -- 172.21.15.40:0/188608728 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d8071bb0 con 0x2604840 2023-11-08T08:21:27.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.668+0000 7ff2d77fe700 1 --2- 172.21.15.40:0/188608728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604c10 0x28cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:27.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.668+0000 7ff2d7fff700 1 --2- 172.21.15.40:0/188608728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2604840 0x28cb410 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8154fc0 tx=0x7ff2c8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2850166ea3ffbf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.668+0000 7ff2be7fc700 1 -- 172.21.15.40:0/188608728 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8013ea0 con 0x2604840 2023-11-08T08:21:27.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.668+0000 7ff2be7fc700 1 -- 172.21.15.40:0/188608728 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c802b070 con 0x2604840 2023-11-08T08:21:27.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.668+0000 7ff2be7fc700 1 -- 172.21.15.40:0/188608728 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8022630 con 0x2604840 2023-11-08T08:21:27.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.668+0000 7ff2e14fd740 1 -- 172.21.15.40:0/188608728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2604840 msgr2=0x28cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.668+0000 7ff2e14fd740 1 --2- 172.21.15.40:0/188608728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2604840 0x28cb410 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8154fc0 tx=0x7ff2c8013000 comp rx=0 tx=0).stop 2023-11-08T08:21:27.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.668+0000 7ff2e14fd740 1 -- 172.21.15.40:0/188608728 shutdown_connections 2023-11-08T08:21:27.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.668+0000 7ff2e14fd740 1 --2- 172.21.15.40:0/188608728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x264f8c0 0x28cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.668+0000 7ff2e14fd740 1 --2- 172.21.15.40:0/188608728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604c10 0x28cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.668+0000 7ff2e14fd740 1 --2- 172.21.15.40:0/188608728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2604840 0x28cb410 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.668+0000 7ff2e14fd740 1 -- 172.21.15.40:0/188608728 >> 172.21.15.40:0/188608728 conn(0x2604fe0 msgr2=0x28d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:27.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.668+0000 7ff2e14fd740 1 -- 172.21.15.40:0/188608728 shutdown_connections 2023-11-08T08:21:27.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.668+0000 7ff2e14fd740 1 -- 172.21.15.40:0/188608728 wait complete. 2023-11-08T08:21:27.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.668+0000 7ff2e14fd740 1 -- start start 2023-11-08T08:21:27.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.668+0000 7ff2e14fd740 1 Processor -- start 2023-11-08T08:21:27.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2e14fd740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604840 0x2920060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2e14fd740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2604c10 0x289d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2e14fd740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x264f8c0 0x289d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2e14fd740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x28ccb30 con 0x2604c10 2023-11-08T08:21:27.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2e14fd740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x29205a0 con 0x264f8c0 2023-11-08T08:21:27.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2e14fd740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x28ad730 con 0x2604840 2023-11-08T08:21:27.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2e14fd740 10 client.0 _ll_register_callbacks cb 0x7ff2d00549d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:21:27.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2d77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2604c10 0x289d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2d77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2604c10 0x289d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56320/0 (socket says 172.21.15.40:56320) 2023-11-08T08:21:27.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2d77fe700 1 -- 172.21.15.40:0/4056140240 learned_addr learned my addr 172.21.15.40:0/4056140240 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:27.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2d7fff700 1 --2- 172.21.15.40:0/4056140240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604840 0x2920060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2dc811700 1 --2- 172.21.15.40:0/4056140240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x264f8c0 0x289d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2d77fe700 1 -- 172.21.15.40:0/4056140240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604840 msgr2=0x2920060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2d77fe700 1 --2- 172.21.15.40:0/4056140240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604840 0x2920060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2d77fe700 1 -- 172.21.15.40:0/4056140240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x264f8c0 msgr2=0x289d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:27.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2d77fe700 1 --2- 172.21.15.40:0/4056140240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x264f8c0 0x289d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2d77fe700 1 -- 172.21.15.40:0/4056140240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c8013870 con 0x2604c10 2023-11-08T08:21:27.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2d77fe700 1 --2- 172.21.15.40:0/4056140240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2604c10 0x289d060 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc017a60 tx=0x7ff2cc017a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3344db6f0e31f2f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:27.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:21:27.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2b6ffd700 1 -- 172.21.15.40:0/4056140240 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc01c070 con 0x2604c10 2023-11-08T08:21:27.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2b6ffd700 1 -- 172.21.15.40:0/4056140240 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff2cc022d70 con 0x2604c10 2023-11-08T08:21:27.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2e14fd740 1 -- 172.21.15.40:0/4056140240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2d0054150 con 0x2604c10 2023-11-08T08:21:27.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2b6ffd700 1 -- 172.21.15.40:0/4056140240 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2cc004300 con 0x2604c10 2023-11-08T08:21:27.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2e14fd740 10 client.6021 fetch_fsmap learned FSMap version 25 2023-11-08T08:21:27.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.672+0000 7ff2e14fd740 1 -- 172.21.15.40:0/4056140240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x2789af0 con 0x2604c10 2023-11-08T08:21:27.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2b6ffd700 1 -- 172.21.15.40:0/4056140240 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d0054150 con 0x2604c10 2023-11-08T08:21:27.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2b6ffd700 1 -- 172.21.15.40:0/4056140240 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff2cc0036c0 con 0x2604c10 2023-11-08T08:21:27.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2e14fd740 10 client.6021 fetch_fsmap finished waiting for FSMap version 25 2023-11-08T08:21:27.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2e14fd740 10 client.6021 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-08T08:21:27.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2e14fd740 20 client.6021 populate_metadata read hostname 'smithi040' 2023-11-08T08:21:27.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2e14fd740 1 --2- 172.21.15.40:0/4056140240 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x28748e0 0x2874ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:27.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2e14fd740 4 client.6021 mds_command: new command op to 5775 tid=0[{"prefix": "scrub status"}] 2023-11-08T08:21:27.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2e14fd740 1 -- 172.21.15.40:0/4056140240 --> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x2681390 con 0x28748e0 2023-11-08T08:21:27.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2d7fff700 1 --2- 172.21.15.40:0/4056140240 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x28748e0 0x2874ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:27.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2d7fff700 1 --2- 172.21.15.40:0/4056140240 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x28748e0 0x2874ce0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e1bfac065695a75 server_cookie=d69850b079551121 in_seq=0 out_seq=0 2023-11-08T08:21:27.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2b6ffd700 10 client.6021 ms_handle_connect on v2:172.21.15.40:6833/2471311394 2023-11-08T08:21:27.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2b6ffd700 1 -- 172.21.15.40:0/4056140240 <== mds.0 v2:172.21.15.40:6833/2471311394 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x2681390 con 0x28748e0 2023-11-08T08:21:27.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2b6ffd700 10 client.6021 handle_command_reply: tid=0 2023-11-08T08:21:27.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2e14fd740 1 client.6021 shutdown 2023-11-08T08:21:27.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2e14fd740 1 -- 172.21.15.40:0/4056140240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2604c10 msgr2=0x289d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2e14fd740 1 --2- 172.21.15.40:0/4056140240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2604c10 0x289d060 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc017a60 tx=0x7ff2cc017a90 comp rx=0 tx=0).stop 2023-11-08T08:21:27.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2e14fd740 1 -- 172.21.15.40:0/4056140240 shutdown_connections 2023-11-08T08:21:27.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2e14fd740 1 --2- 172.21.15.40:0/4056140240 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x28748e0 0x2874ce0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2e14fd740 1 --2- 172.21.15.40:0/4056140240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x264f8c0 0x289d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2e14fd740 1 --2- 172.21.15.40:0/4056140240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x2604c10 0x289d060 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2e14fd740 1 --2- 172.21.15.40:0/4056140240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x2604840 0x2920060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2b6ffd700 0 client.6021 ms_handle_reset on v2:172.21.15.40:6833/2471311394 2023-11-08T08:21:27.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2e14fd740 1 -- 172.21.15.40:0/4056140240 >> 172.21.15.40:0/4056140240 conn(0x2604fe0 msgr2=0x28d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:27.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2e14fd740 1 -- 172.21.15.40:0/4056140240 shutdown_connections 2023-11-08T08:21:27.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2e14fd740 1 -- 172.21.15.40:0/4056140240 wait complete. 2023-11-08T08:21:27.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.676+0000 7ff2e14fd740 20 client.6021 trim_cache size 0 max 16384 2023-11-08T08:21:27.698 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:21:27.698 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-08T08:21:27.699 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-11-08T08:21:27.699 INFO:teuthology.orchestra.run.smithi040.stdout: "4541f6e1-31da-419f-97de-c48ef4872ec8": { 2023-11-08T08:21:27.699 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-11-08T08:21:27.699 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "4541f6e1-31da-419f-97de-c48ef4872ec8", 2023-11-08T08:21:27.700 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,repair" 2023-11-08T08:21:27.700 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:27.700 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:27.700 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:21:27.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.680+0000 7ff2dd012700 1 -- 172.21.15.40:0/74933325 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff2c4064ee0 msgr2=0x7ff2c4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.680+0000 7ff2dd012700 1 --2- 172.21.15.40:0/74933325 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff2c4064ee0 0x7ff2c4067390 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80012e0 tx=0x7ff2c8001b70 comp rx=0 tx=0).stop 2023-11-08T08:21:27.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.680+0000 7ff2dd012700 1 -- 172.21.15.40:0/74933325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d815cf10 msgr2=0x7ff2d80c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:27.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.680+0000 7ff2dd012700 1 --2- 172.21.15.40:0/74933325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d815cf10 0x7ff2d80c83c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0014040 tx=0x7ff2d0004770 comp rx=0 tx=0).stop 2023-11-08T08:21:27.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.680+0000 7ff2dd012700 1 -- 172.21.15.40:0/74933325 shutdown_connections 2023-11-08T08:21:27.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.680+0000 7ff2dd012700 1 --2- 172.21.15.40:0/74933325 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff2c4064ee0 0x7ff2c4067390 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.680+0000 7ff2dd012700 1 --2- 172.21.15.40:0/74933325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d815cf10 0x7ff2d80c83c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.680+0000 7ff2dd012700 1 --2- 172.21.15.40:0/74933325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2d80cd720 0x7ff2d81546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.680+0000 7ff2dd012700 1 --2- 172.21.15.40:0/74933325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2d80aed90 0x7ff2d80c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:27.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.680+0000 7ff2dd012700 1 -- 172.21.15.40:0/74933325 >> 172.21.15.40:0/74933325 conn(0x7ff2d80d03b0 msgr2=0x7ff2d8071000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:27.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.680+0000 7ff2dd012700 1 -- 172.21.15.40:0/74933325 shutdown_connections 2023-11-08T08:21:27.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:27.680+0000 7ff2dd012700 1 -- 172.21.15.40:0/74933325 wait complete. 2023-11-08T08:21:27.709 INFO:tasks.cephfs.filesystem:scrub status for tag:4541f6e1-31da-419f-97de-c48ef4872ec8 - {'path': '/', 'tag': '4541f6e1-31da-419f-97de-c48ef4872ec8', 'options': 'recursive,repair'} 2023-11-08T08:21:28.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:28.000+0000 7ff8c6ffd700 1 -- 172.21.15.40:0/2398014643 <== mon.0 v2:172.21.15.40:3300/0 7 ==== log(6 entries from seq 1 at 2023-11-08T08:21:27.398607+0000) v1 ==== 1078+0+0 (secure 0 0 0) 0x7ff8c802d350 con 0x7ff8d8071600 2023-11-08T08:21:28.008 INFO:teuthology.orchestra.run.smithi040.stdout: cluster: 2023-11-08T08:21:28.008 INFO:teuthology.orchestra.run.smithi040.stdout: id: 54ed45f4-d5b0-41c3-8c3b-b5a18b844f25 2023-11-08T08:21:28.008 INFO:teuthology.orchestra.run.smithi040.stdout: health: HEALTH_OK 2023-11-08T08:21:28.009 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:28.009 INFO:teuthology.orchestra.run.smithi040.stdout: services: 2023-11-08T08:21:28.009 INFO:teuthology.orchestra.run.smithi040.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2023-11-08T08:21:28.009 INFO:teuthology.orchestra.run.smithi040.stdout: mgr: x(active, since 2m), standbys: y 2023-11-08T08:21:28.010 INFO:teuthology.orchestra.run.smithi040.stdout: mds: 1/1 daemons up, 3 standby 2023-11-08T08:21:28.010 INFO:teuthology.orchestra.run.smithi040.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2023-11-08T08:21:28.010 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:28.010 INFO:teuthology.orchestra.run.smithi040.stdout: data: 2023-11-08T08:21:28.011 INFO:teuthology.orchestra.run.smithi040.stdout: volumes: 1/1 healthy 2023-11-08T08:21:28.011 INFO:teuthology.orchestra.run.smithi040.stdout: pools: 4 pools, 105 pgs 2023-11-08T08:21:28.011 INFO:teuthology.orchestra.run.smithi040.stdout: objects: 23 objects, 579 KiB 2023-11-08T08:21:28.011 INFO:teuthology.orchestra.run.smithi040.stdout: usage: 75 MiB used, 720 GiB / 720 GiB avail 2023-11-08T08:21:28.011 INFO:teuthology.orchestra.run.smithi040.stdout: pgs: 105 active+clean 2023-11-08T08:21:28.012 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:28.012 INFO:teuthology.orchestra.run.smithi040.stdout: io: 2023-11-08T08:21:28.012 INFO:teuthology.orchestra.run.smithi040.stdout: client: 1.7 KiB/s wr, 0 op/s rd, 5 op/s wr 2023-11-08T08:21:28.012 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:28.012 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:28.013 INFO:teuthology.orchestra.run.smithi040.stdout:2023-11-08T08:21:27.398607+0000 mds.a [INF] scrub queued for path: / 2023-11-08T08:21:28.013 INFO:teuthology.orchestra.run.smithi040.stdout:2023-11-08T08:21:27.398617+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2023-11-08T08:21:28.013 INFO:teuthology.orchestra.run.smithi040.stdout:2023-11-08T08:21:27.398629+0000 mds.a [INF] scrub summary: active paths [/] 2023-11-08T08:21:28.013 INFO:teuthology.orchestra.run.smithi040.stdout:2023-11-08T08:21:27.400807+0000 mds.a [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2023-11-08T08:21:28.013 INFO:teuthology.orchestra.run.smithi040.stdout:2023-11-08T08:21:27.400819+0000 mds.a [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2023-11-08T08:21:28.014 INFO:teuthology.orchestra.run.smithi040.stdout:2023-11-08T08:21:27.401325+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2023-11-08T08:21:32.828 INFO:teuthology.orchestra.run.smithi040.stdout:2023-11-08T08:21:31.541561+0000 mds.a [INF] scrub summary: idle 2023-11-08T08:21:32.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:32.820+0000 7ff8c6ffd700 1 -- 172.21.15.40:0/2398014643 <== mon.0 v2:172.21.15.40:3300/0 8 ==== log(1 entries from seq 7 at 2023-11-08T08:21:31.541561+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7ff8c805d790 con 0x7ff8d8071600 2023-11-08T08:21:56.735 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:21:56.744 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:21:57.710 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-08T08:21:57.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.895+0000 7f99efe25700 1 Processor -- start 2023-11-08T08:21:57.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.895+0000 7f99efe25700 1 -- start start 2023-11-08T08:21:57.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.899+0000 7f99efe25700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f99e815c700 0x7f99e80702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.899+0000 7f99efe25700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f99e80b2130 0x7f99e8070830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.899+0000 7f99efe25700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99e80b1be0 0x7f99e806c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.899+0000 7f99efe25700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e80cd360 con 0x7f99e815c700 2023-11-08T08:21:57.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.899+0000 7f99efe25700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e8160190 con 0x7f99e80b2130 2023-11-08T08:21:57.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.899+0000 7f99efe25700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e8160470 con 0x7f99e80b1be0 2023-11-08T08:21:57.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.899+0000 7f99edbc1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f99e815c700 0x7f99e80702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.899+0000 7f99edbc1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f99e815c700 0x7f99e80702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56318/0 (socket says 172.21.15.40:56318) 2023-11-08T08:21:57.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.899+0000 7f99edbc1700 1 -- 172.21.15.40:0/3763622875 learned_addr learned my addr 172.21.15.40:0/3763622875 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:57.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.899+0000 7f99ed3c0700 1 --2- 172.21.15.40:0/3763622875 >> v2:172.21.15.169:3300/0 conn(0x7f99e80b2130 0x7f99e8070830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.899+0000 7f99ee3c2700 1 --2- 172.21.15.40:0/3763622875 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99e80b1be0 0x7f99e806c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.903+0000 7f99edbc1700 1 -- 172.21.15.40:0/3763622875 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99e80b1be0 msgr2=0x7f99e806c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:57.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.903+0000 7f99edbc1700 1 --2- 172.21.15.40:0/3763622875 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99e80b1be0 0x7f99e806c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.903+0000 7f99edbc1700 1 -- 172.21.15.40:0/3763622875 >> v2:172.21.15.169:3300/0 conn(0x7f99e80b2130 msgr2=0x7f99e8070830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:57.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.903+0000 7f99edbc1700 1 --2- 172.21.15.40:0/3763622875 >> v2:172.21.15.169:3300/0 conn(0x7f99e80b2130 0x7f99e8070830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.903+0000 7f99edbc1700 1 -- 172.21.15.40:0/3763622875 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e8160750 con 0x7f99e815c700 2023-11-08T08:21:57.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.903+0000 7f99edbc1700 1 --2- 172.21.15.40:0/3763622875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99e815c700 0x7f99e80702f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f99e80700f0 tx=0x7f99e4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734716b6b33149e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:57.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.903+0000 7f99ecbbf700 1 -- 172.21.15.40:0/3763622875 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4011830 con 0x7f99e815c700 2023-11-08T08:21:57.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.903+0000 7f99ecbbf700 1 -- 172.21.15.40:0/3763622875 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e4011990 con 0x7f99e815c700 2023-11-08T08:21:57.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.903+0000 7f99ecbbf700 1 -- 172.21.15.40:0/3763622875 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4011d20 con 0x7f99e815c700 2023-11-08T08:21:57.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.903+0000 7f99efe25700 1 -- 172.21.15.40:0/3763622875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99e815c700 msgr2=0x7f99e80702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:57.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.903+0000 7f99efe25700 1 --2- 172.21.15.40:0/3763622875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99e815c700 0x7f99e80702f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f99e80700f0 tx=0x7f99e4009bf0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.903+0000 7f99efe25700 1 -- 172.21.15.40:0/3763622875 shutdown_connections 2023-11-08T08:21:57.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.903+0000 7f99efe25700 1 --2- 172.21.15.40:0/3763622875 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99e80b1be0 0x7f99e806c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.903+0000 7f99efe25700 1 --2- 172.21.15.40:0/3763622875 >> v2:172.21.15.169:3300/0 conn(0x7f99e80b2130 0x7f99e8070830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.903+0000 7f99efe25700 1 --2- 172.21.15.40:0/3763622875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99e815c700 0x7f99e80702f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.903+0000 7f99efe25700 1 -- 172.21.15.40:0/3763622875 >> 172.21.15.40:0/3763622875 conn(0x7f99e80d2d10 msgr2=0x7f99e80d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:57.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.903+0000 7f99efe25700 1 -- 172.21.15.40:0/3763622875 shutdown_connections 2023-11-08T08:21:57.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.903+0000 7f99efe25700 1 -- 172.21.15.40:0/3763622875 wait complete. 2023-11-08T08:21:57.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.907+0000 7f99efe25700 1 Processor -- start 2023-11-08T08:21:57.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99efe25700 1 -- start start 2023-11-08T08:21:57.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99efe25700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99e80b1be0 0x7f99e80cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99efe25700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99e80b2130 0x7f99e815d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99efe25700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99e815c700 0x7f99e80c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99efe25700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e80cb080 con 0x7f99e80b1be0 2023-11-08T08:21:57.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99efe25700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e815dac0 con 0x7f99e815c700 2023-11-08T08:21:57.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99efe25700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e815e2b0 con 0x7f99e80b2130 2023-11-08T08:21:57.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99edbc1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99e80b1be0 0x7f99e80cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99edbc1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99e80b1be0 0x7f99e80cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56334/0 (socket says 172.21.15.40:56334) 2023-11-08T08:21:57.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99edbc1700 1 -- 172.21.15.40:0/1087695655 learned_addr learned my addr 172.21.15.40:0/1087695655 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:57.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99ee3c2700 1 --2- 172.21.15.40:0/1087695655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99e815c700 0x7f99e80c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99ed3c0700 1 --2- 172.21.15.40:0/1087695655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99e80b2130 0x7f99e815d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99edbc1700 1 -- 172.21.15.40:0/1087695655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99e80b2130 msgr2=0x7f99e815d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:57.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99edbc1700 1 --2- 172.21.15.40:0/1087695655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99e80b2130 0x7f99e815d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99edbc1700 1 -- 172.21.15.40:0/1087695655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99e815c700 msgr2=0x7f99e80c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:57.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99edbc1700 1 --2- 172.21.15.40:0/1087695655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99e815c700 0x7f99e80c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99edbc1700 1 -- 172.21.15.40:0/1087695655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e4009570 con 0x7f99e80b1be0 2023-11-08T08:21:57.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99ed3c0700 1 --2- 172.21.15.40:0/1087695655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99e80b2130 0x7f99e815d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:57.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99ee3c2700 1 --2- 172.21.15.40:0/1087695655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99e815c700 0x7f99e80c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:57.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99edbc1700 1 --2- 172.21.15.40:0/1087695655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99e80b1be0 0x7f99e80cab40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f99e4000c00 tx=0x7f99e4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:57.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99de7fc700 1 -- 172.21.15.40:0/1087695655 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4004610 con 0x7f99e80b1be0 2023-11-08T08:21:57.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99de7fc700 1 -- 172.21.15.40:0/1087695655 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e401e070 con 0x7f99e80b1be0 2023-11-08T08:21:57.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99efe25700 1 -- 172.21.15.40:0/1087695655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e806b530 con 0x7f99e80b1be0 2023-11-08T08:21:57.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99de7fc700 1 -- 172.21.15.40:0/1087695655 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e40038e0 con 0x7f99e80b1be0 2023-11-08T08:21:57.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.911+0000 7f99efe25700 1 -- 172.21.15.40:0/1087695655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e8160a30 con 0x7f99e80b1be0 2023-11-08T08:21:57.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.915+0000 7f99de7fc700 1 -- 172.21.15.40:0/1087695655 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f99e400f050 con 0x7f99e80b1be0 2023-11-08T08:21:57.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.919+0000 7f99de7fc700 1 --2- 172.21.15.40:0/1087695655 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f99cc064ee0 0x7f99cc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.919+0000 7f99de7fc700 1 -- 172.21.15.40:0/1087695655 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f99e4082d20 con 0x7f99e80b1be0 2023-11-08T08:21:57.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.919+0000 7f99ed3c0700 1 --2- 172.21.15.40:0/1087695655 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f99cc064ee0 0x7f99cc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.919+0000 7f99ed3c0700 1 --2- 172.21.15.40:0/1087695655 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f99cc064ee0 0x7f99cc067390 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f99e80c6690 tx=0x7f99d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:57.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99f30b3740 1 Processor -- start 2023-11-08T08:21:57.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99f30b3740 1 -- start start 2023-11-08T08:21:57.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99f30b3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7c10 0x106eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99f30b3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7840 0x106f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99f30b3740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xe028c0 0x106b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99f30b3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0xf26d20 con 0xdb7840 2023-11-08T08:21:57.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99f30b3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0xf96d60 con 0xdb7c10 2023-11-08T08:21:57.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99f30b3740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0xf90060 con 0xe028c0 2023-11-08T08:21:57.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99ed3c0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7840 0x106f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99ed3c0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7840 0x106f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56340/0 (socket says 172.21.15.40:56340) 2023-11-08T08:21:57.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99ed3c0700 1 -- 172.21.15.40:0/1781708141 learned_addr learned my addr 172.21.15.40:0/1781708141 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:57.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99edbc1700 1 --2- 172.21.15.40:0/1781708141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7c10 0x106eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99ee3c2700 1 --2- 172.21.15.40:0/1781708141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xe028c0 0x106b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99ed3c0700 1 -- 172.21.15.40:0/1781708141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xe028c0 msgr2=0x106b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:57.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99ed3c0700 1 --2- 172.21.15.40:0/1781708141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xe028c0 0x106b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99ed3c0700 1 -- 172.21.15.40:0/1781708141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7c10 msgr2=0x106eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:57.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99ed3c0700 1 --2- 172.21.15.40:0/1781708141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7c10 0x106eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99ed3c0700 1 -- 172.21.15.40:0/1781708141 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xfb60a0 con 0xdb7840 2023-11-08T08:21:57.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99edbc1700 1 --2- 172.21.15.40:0/1781708141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7c10 0x106eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:57.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99ed3c0700 1 --2- 172.21.15.40:0/1781708141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7840 0x106f0a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f99d8004480 tx=0x7f99d8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffe92abc4a84271 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:57.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99cb7fe700 1 -- 172.21.15.40:0/1781708141 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d80044b0 con 0xdb7840 2023-11-08T08:21:57.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99cb7fe700 1 -- 172.21.15.40:0/1781708141 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d80036c0 con 0xdb7840 2023-11-08T08:21:57.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.923+0000 7f99cb7fe700 1 -- 172.21.15.40:0/1781708141 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d8003a30 con 0xdb7840 2023-11-08T08:21:57.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99f30b3740 1 -- 172.21.15.40:0/1781708141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7840 msgr2=0x106f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:57.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99f30b3740 1 --2- 172.21.15.40:0/1781708141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7840 0x106f0a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f99d8004480 tx=0x7f99d8014000 comp rx=0 tx=0).stop 2023-11-08T08:21:57.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99f30b3740 1 -- 172.21.15.40:0/1781708141 shutdown_connections 2023-11-08T08:21:57.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99f30b3740 1 --2- 172.21.15.40:0/1781708141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xe028c0 0x106b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99f30b3740 1 --2- 172.21.15.40:0/1781708141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7840 0x106f0a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99f30b3740 1 --2- 172.21.15.40:0/1781708141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7c10 0x106eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99f30b3740 1 -- 172.21.15.40:0/1781708141 >> 172.21.15.40:0/1781708141 conn(0xdb7fe0 msgr2=0x1050b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:57.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99f30b3740 1 -- 172.21.15.40:0/1781708141 shutdown_connections 2023-11-08T08:21:57.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99f30b3740 1 -- 172.21.15.40:0/1781708141 wait complete. 2023-11-08T08:21:57.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99f30b3740 1 -- start start 2023-11-08T08:21:57.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99f30b3740 1 Processor -- start 2023-11-08T08:21:57.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99f30b3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7840 0x1050ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99f30b3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7c10 0x1051db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99f30b3740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xe028c0 0x1026ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99f30b3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0xe3cc50 con 0xdb7c10 2023-11-08T08:21:57.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99f30b3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x104db70 con 0xdb7840 2023-11-08T08:21:57.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99f30b3740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1051010 con 0xe028c0 2023-11-08T08:21:57.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99f30b3740 10 client.0 _ll_register_callbacks cb 0xf899a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:21:57.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99ed3c0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7c10 0x1051db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99ee3c2700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xe028c0 0x1026ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99ed3c0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7c10 0x1051db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56354/0 (socket says 172.21.15.40:56354) 2023-11-08T08:21:57.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99ed3c0700 1 -- 172.21.15.40:0/867984461 learned_addr learned my addr 172.21.15.40:0/867984461 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:57.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.927+0000 7f99edbc1700 1 --2- 172.21.15.40:0/867984461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7840 0x1050ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99ee3c2700 1 -- 172.21.15.40:0/867984461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7840 msgr2=0x1050ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:57.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99ee3c2700 1 --2- 172.21.15.40:0/867984461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7840 0x1050ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99ee3c2700 1 -- 172.21.15.40:0/867984461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7c10 msgr2=0x1051db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:57.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99ee3c2700 1 --2- 172.21.15.40:0/867984461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7c10 0x1051db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99ee3c2700 1 -- 172.21.15.40:0/867984461 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d8004130 con 0xe028c0 2023-11-08T08:21:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99ee3c2700 1 --2- 172.21.15.40:0/867984461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xe028c0 0x1026ce0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f99e0007e40 tx=0x7f99e0007e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a69ce9d2f5054816 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.169:3301/0 2023-11-08T08:21:57.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99c8ff9700 1 -- 172.21.15.40:0/867984461 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0012b40 con 0xe028c0 2023-11-08T08:21:57.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99c8ff9700 1 -- 172.21.15.40:0/867984461 <== mon.2 v2:172.21.15.169:3301/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f99e0012ca0 con 0xe028c0 2023-11-08T08:21:57.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99f30b3740 1 -- 172.21.15.40:0/867984461 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xf890e0 con 0xe028c0 2023-11-08T08:21:57.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99c8ff9700 1 -- 172.21.15.40:0/867984461 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99e0012e60 con 0xe028c0 2023-11-08T08:21:57.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99f30b3740 10 client.5468 fetch_fsmap learned FSMap version 25 2023-11-08T08:21:57.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99f30b3740 1 -- 172.21.15.40:0/867984461 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0xe03b10 con 0xe028c0 2023-11-08T08:21:57.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99c8ff9700 1 -- 172.21.15.40:0/867984461 <== mon.2 v2:172.21.15.169:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xf890e0 con 0xe028c0 2023-11-08T08:21:57.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99c8ff9700 1 -- 172.21.15.40:0/867984461 <== mon.2 v2:172.21.15.169:3301/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f99e0014040 con 0xe028c0 2023-11-08T08:21:57.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99f30b3740 10 client.5468 fetch_fsmap finished waiting for FSMap version 25 2023-11-08T08:21:57.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99f30b3740 10 client.5468 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-08T08:21:57.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99f30b3740 20 client.5468 populate_metadata read hostname 'smithi040' 2023-11-08T08:21:57.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99f30b3740 1 --2- 172.21.15.40:0/867984461 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x104a0e0 0x1027220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99f30b3740 4 client.5468 mds_command: new command op to 5775 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-08T08:21:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99f30b3740 1 -- 172.21.15.40:0/867984461 --> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xf88e00 con 0x104a0e0 2023-11-08T08:21:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.931+0000 7f99edbc1700 1 --2- 172.21.15.40:0/867984461 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x104a0e0 0x1027220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.935+0000 7f99edbc1700 1 --2- 172.21.15.40:0/867984461 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x104a0e0 0x1027220 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9886d12113febe89 server_cookie=f61c7f7b397cdac9 in_seq=0 out_seq=0 2023-11-08T08:21:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.935+0000 7f99c8ff9700 10 client.5468 ms_handle_connect on v2:172.21.15.40:6833/2471311394 2023-11-08T08:21:57.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.935+0000 7f99c8ff9700 1 -- 172.21.15.40:0/867984461 <== mds.0 v2:172.21.15.40:6833/2471311394 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0xf88e00 con 0x104a0e0 2023-11-08T08:21:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.935+0000 7f99c8ff9700 10 client.5468 handle_command_reply: tid=0 2023-11-08T08:21:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.935+0000 7f99f30b3740 1 client.5468 shutdown 2023-11-08T08:21:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.935+0000 7f99f30b3740 1 -- 172.21.15.40:0/867984461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xe028c0 msgr2=0x1026ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.935+0000 7f99f30b3740 1 --2- 172.21.15.40:0/867984461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xe028c0 0x1026ce0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f99e0007e40 tx=0x7f99e0007e70 comp rx=0 tx=0).stop 2023-11-08T08:21:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.935+0000 7f99f30b3740 1 -- 172.21.15.40:0/867984461 shutdown_connections 2023-11-08T08:21:57.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.935+0000 7f99f30b3740 1 --2- 172.21.15.40:0/867984461 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0x104a0e0 0x1027220 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.935+0000 7f99f30b3740 1 --2- 172.21.15.40:0/867984461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xe028c0 0x1026ce0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.935+0000 7f99f30b3740 1 --2- 172.21.15.40:0/867984461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7c10 0x1051db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.935+0000 7f99f30b3740 1 --2- 172.21.15.40:0/867984461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7840 0x1050ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.935+0000 7f99f30b3740 1 -- 172.21.15.40:0/867984461 >> 172.21.15.40:0/867984461 conn(0xdb7fe0 msgr2=0x104c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:57.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.935+0000 7f99c8ff9700 0 client.5468 ms_handle_reset on v2:172.21.15.40:6833/2471311394 2023-11-08T08:21:57.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.935+0000 7f99f30b3740 1 -- 172.21.15.40:0/867984461 shutdown_connections 2023-11-08T08:21:57.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.935+0000 7f99f30b3740 1 -- 172.21.15.40:0/867984461 wait complete. 2023-11-08T08:21:57.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.935+0000 7f99f30b3740 20 client.5468 trim_cache size 0 max 16384 2023-11-08T08:21:57.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.955+0000 7f99f30b3740 1 Processor -- start 2023-11-08T08:21:57.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.955+0000 7f99f30b3740 1 -- start start 2023-11-08T08:21:57.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99f30b3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7fe0 0x10cc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99f30b3740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xdb7840 0x104cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99f30b3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7c10 0x104d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99f30b3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0xf8eef0 con 0xdb7fe0 2023-11-08T08:21:57.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99f30b3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0xf92bc0 con 0xdb7c10 2023-11-08T08:21:57.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99f30b3740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0xf55440 con 0xdb7840 2023-11-08T08:21:57.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99edbc1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7fe0 0x10cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99edbc1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7fe0 0x10cc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56356/0 (socket says 172.21.15.40:56356) 2023-11-08T08:21:57.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99edbc1700 1 -- 172.21.15.40:0/3250730513 learned_addr learned my addr 172.21.15.40:0/3250730513 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:57.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99ed3c0700 1 --2- 172.21.15.40:0/3250730513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xdb7840 0x104cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99ee3c2700 1 --2- 172.21.15.40:0/3250730513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7c10 0x104d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99edbc1700 1 -- 172.21.15.40:0/3250730513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xdb7840 msgr2=0x104cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:57.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99edbc1700 1 --2- 172.21.15.40:0/3250730513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xdb7840 0x104cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99edbc1700 1 -- 172.21.15.40:0/3250730513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7c10 msgr2=0x104d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:57.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99edbc1700 1 --2- 172.21.15.40:0/3250730513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7c10 0x104d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99edbc1700 1 -- 172.21.15.40:0/3250730513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e80b3d80 con 0xdb7fe0 2023-11-08T08:21:57.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99ee3c2700 1 --2- 172.21.15.40:0/3250730513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7c10 0x104d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:57.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99edbc1700 1 --2- 172.21.15.40:0/3250730513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7fe0 0x10cc020 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f99e400a010 tx=0x7f99e4006da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdad22446c97c50d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:57.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99cbfff700 1 -- 172.21.15.40:0/3250730513 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4052cf0 con 0xdb7fe0 2023-11-08T08:21:57.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99cbfff700 1 -- 172.21.15.40:0/3250730513 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e407f070 con 0xdb7fe0 2023-11-08T08:21:57.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.959+0000 7f99cbfff700 1 -- 172.21.15.40:0/3250730513 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4054060 con 0xdb7fe0 2023-11-08T08:21:57.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.963+0000 7f99f30b3740 1 -- 172.21.15.40:0/3250730513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7fe0 msgr2=0x10cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:57.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.963+0000 7f99f30b3740 1 --2- 172.21.15.40:0/3250730513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7fe0 0x10cc020 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f99e400a010 tx=0x7f99e4006da0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.963+0000 7f99f30b3740 1 -- 172.21.15.40:0/3250730513 shutdown_connections 2023-11-08T08:21:57.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.963+0000 7f99f30b3740 1 --2- 172.21.15.40:0/3250730513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7c10 0x104d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.963+0000 7f99f30b3740 1 --2- 172.21.15.40:0/3250730513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xdb7840 0x104cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.963+0000 7f99f30b3740 1 --2- 172.21.15.40:0/3250730513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7fe0 0x10cc020 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.963+0000 7f99f30b3740 1 -- 172.21.15.40:0/3250730513 >> 172.21.15.40:0/3250730513 conn(0x1050c40 msgr2=0x104c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:57.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.963+0000 7f99f30b3740 1 -- 172.21.15.40:0/3250730513 shutdown_connections 2023-11-08T08:21:57.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.963+0000 7f99f30b3740 1 -- 172.21.15.40:0/3250730513 wait complete. 2023-11-08T08:21:57.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.963+0000 7f99f30b3740 1 -- start start 2023-11-08T08:21:57.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.963+0000 7f99f30b3740 1 Processor -- start 2023-11-08T08:21:57.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99f30b3740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7840 0x1081300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99f30b3740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xdb7c10 0x1081840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99f30b3740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7fe0 0x1047820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99f30b3740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0xf5fba0 con 0xdb7840 2023-11-08T08:21:57.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99f30b3740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1048bb0 con 0xdb7fe0 2023-11-08T08:21:57.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99f30b3740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1049860 con 0xdb7c10 2023-11-08T08:21:57.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99f30b3740 10 client.0 _ll_register_callbacks cb 0x7f99d8019040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:21:57.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99edbc1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7840 0x1081300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99edbc1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7840 0x1081300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56366/0 (socket says 172.21.15.40:56366) 2023-11-08T08:21:57.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99edbc1700 1 -- 172.21.15.40:0/763115606 learned_addr learned my addr 172.21.15.40:0/763115606 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:57.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99ed3c0700 1 --2- 172.21.15.40:0/763115606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xdb7c10 0x1081840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99ee3c2700 1 --2- 172.21.15.40:0/763115606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7fe0 0x1047820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99edbc1700 1 -- 172.21.15.40:0/763115606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xdb7c10 msgr2=0x1081840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:57.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99edbc1700 1 --2- 172.21.15.40:0/763115606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xdb7c10 0x1081840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99edbc1700 1 -- 172.21.15.40:0/763115606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7fe0 msgr2=0x1047820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:57.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99edbc1700 1 --2- 172.21.15.40:0/763115606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7fe0 0x1047820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99edbc1700 1 -- 172.21.15.40:0/763115606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e8160a30 con 0xdb7840 2023-11-08T08:21:57.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99edbc1700 1 --2- 172.21.15.40:0/763115606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7840 0x1081300 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f99e4054650 tx=0x7f99e405a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af8d74e621c69b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:57.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:21:57.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99c8ff9700 1 -- 172.21.15.40:0/763115606 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e405c550 con 0xdb7840 2023-11-08T08:21:57.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99c8ff9700 1 -- 172.21.15.40:0/763115606 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f99e405c6b0 con 0xdb7840 2023-11-08T08:21:57.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99f30b3740 1 -- 172.21.15.40:0/763115606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99d80036c0 con 0xdb7840 2023-11-08T08:21:57.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99c8ff9700 1 -- 172.21.15.40:0/763115606 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99e407b1f0 con 0xdb7840 2023-11-08T08:21:57.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99f30b3740 10 client.6039 fetch_fsmap learned FSMap version 25 2023-11-08T08:21:57.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99f30b3740 1 -- 172.21.15.40:0/763115606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0xf3caf0 con 0xdb7840 2023-11-08T08:21:57.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99c8ff9700 1 -- 172.21.15.40:0/763115606 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d80036c0 con 0xdb7840 2023-11-08T08:21:57.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99c8ff9700 1 -- 172.21.15.40:0/763115606 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f99e4050740 con 0xdb7840 2023-11-08T08:21:57.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99f30b3740 10 client.6039 fetch_fsmap finished waiting for FSMap version 25 2023-11-08T08:21:57.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99f30b3740 10 client.6039 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-08T08:21:57.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99f30b3740 20 client.6039 populate_metadata read hostname 'smithi040' 2023-11-08T08:21:57.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99f30b3740 1 --2- 172.21.15.40:0/763115606 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0xe028c0 0x1047d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:57.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99f30b3740 4 client.6039 mds_command: new command op to 5775 tid=0[{"prefix": "scrub status"}] 2023-11-08T08:21:57.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.967+0000 7f99f30b3740 1 -- 172.21.15.40:0/763115606 --> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0xe34390 con 0xe028c0 2023-11-08T08:21:57.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99ed3c0700 1 --2- 172.21.15.40:0/763115606 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0xe028c0 0x1047d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:57.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99ed3c0700 1 --2- 172.21.15.40:0/763115606 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0xe028c0 0x1047d60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37f70454949f3aa1 server_cookie=c300490a8378d50c in_seq=0 out_seq=0 2023-11-08T08:21:57.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99c8ff9700 10 client.6039 ms_handle_connect on v2:172.21.15.40:6833/2471311394 2023-11-08T08:21:57.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99c8ff9700 1 -- 172.21.15.40:0/763115606 <== mds.0 v2:172.21.15.40:6833/2471311394 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0xe34390 con 0xe028c0 2023-11-08T08:21:57.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99c8ff9700 10 client.6039 handle_command_reply: tid=0 2023-11-08T08:21:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99f30b3740 1 client.6039 shutdown 2023-11-08T08:21:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99f30b3740 1 -- 172.21.15.40:0/763115606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7840 msgr2=0x1081300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99f30b3740 1 --2- 172.21.15.40:0/763115606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7840 0x1081300 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f99e4054650 tx=0x7f99e405a040 comp rx=0 tx=0).stop 2023-11-08T08:21:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99f30b3740 1 -- 172.21.15.40:0/763115606 shutdown_connections 2023-11-08T08:21:57.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99f30b3740 1 --2- 172.21.15.40:0/763115606 >> [v2:172.21.15.40:6833/2471311394,v1:172.21.15.40:6835/2471311394] conn(0xe028c0 0x1047d60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99f30b3740 1 --2- 172.21.15.40:0/763115606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xdb7fe0 0x1047820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99f30b3740 1 --2- 172.21.15.40:0/763115606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xdb7c10 0x1081840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99c8ff9700 0 client.6039 ms_handle_reset on v2:172.21.15.40:6833/2471311394 2023-11-08T08:21:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99f30b3740 1 --2- 172.21.15.40:0/763115606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xdb7840 0x1081300 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99f30b3740 1 -- 172.21.15.40:0/763115606 >> 172.21.15.40:0/763115606 conn(0x1050c40 msgr2=0x10cb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:57.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99f30b3740 1 -- 172.21.15.40:0/763115606 shutdown_connections 2023-11-08T08:21:57.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99f30b3740 1 -- 172.21.15.40:0/763115606 wait complete. 2023-11-08T08:21:57.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99f30b3740 20 client.6039 trim_cache size 0 max 16384 2023-11-08T08:21:57.988 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:21:57.989 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "no active scrubs running", 2023-11-08T08:21:57.989 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": {} 2023-11-08T08:21:57.989 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:21:57.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99efe25700 1 -- 172.21.15.40:0/1087695655 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f99cc064ee0 msgr2=0x7f99cc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:57.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99efe25700 1 --2- 172.21.15.40:0/1087695655 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f99cc064ee0 0x7f99cc067390 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f99e80c6690 tx=0x7f99d8008040 comp rx=0 tx=0).stop 2023-11-08T08:21:57.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99efe25700 1 -- 172.21.15.40:0/1087695655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99e80b1be0 msgr2=0x7f99e80cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:57.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.971+0000 7f99efe25700 1 --2- 172.21.15.40:0/1087695655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99e80b1be0 0x7f99e80cab40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f99e4000c00 tx=0x7f99e4004770 comp rx=0 tx=0).stop 2023-11-08T08:21:57.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.975+0000 7f99efe25700 1 -- 172.21.15.40:0/1087695655 shutdown_connections 2023-11-08T08:21:57.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.975+0000 7f99efe25700 1 --2- 172.21.15.40:0/1087695655 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f99cc064ee0 0x7f99cc067390 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.975+0000 7f99efe25700 1 --2- 172.21.15.40:0/1087695655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99e815c700 0x7f99e80c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.975+0000 7f99efe25700 1 --2- 172.21.15.40:0/1087695655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99e80b2130 0x7f99e815d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.975+0000 7f99efe25700 1 --2- 172.21.15.40:0/1087695655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99e80b1be0 0x7f99e80cab40 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.975+0000 7f99efe25700 1 -- 172.21.15.40:0/1087695655 >> 172.21.15.40:0/1087695655 conn(0x7f99e80d2d10 msgr2=0x7f99e80b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.975+0000 7f99efe25700 1 -- 172.21.15.40:0/1087695655 shutdown_connections 2023-11-08T08:21:57.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:57.975+0000 7f99efe25700 1 -- 172.21.15.40:0/1087695655 wait complete. 2023-11-08T08:21:58.005 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-08T08:21:58.106 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:21:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.295+0000 7f86cd197700 1 Processor -- start 2023-11-08T08:21:58.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.295+0000 7f86cd197700 1 -- start start 2023-11-08T08:21:58.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.295+0000 7f86cd197700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86c80b1640 0x7f86c8070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:58.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.295+0000 7f86cd197700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86c80c9040 0x7f86c80705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:58.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.295+0000 7f86cd197700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c815a550 con 0x7f86c80b43a0 2023-11-08T08:21:58.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.295+0000 7f86cd197700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c806b0a0 con 0x7f86c80c9040 2023-11-08T08:21:58.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.295+0000 7f86cd197700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c80caac0 con 0x7f86c80b1640 2023-11-08T08:21:58.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86cc996700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86c80c9040 0x7f86c80705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:58.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86cc996700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86c80c9040 0x7f86c80705c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56380/0 (socket says 172.21.15.40:56380) 2023-11-08T08:21:58.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86cc996700 1 -- 172.21.15.40:0/2454901492 learned_addr learned my addr 172.21.15.40:0/2454901492 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86c77fe700 1 --2- 172.21.15.40:0/2454901492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86c80b1640 0x7f86c8070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86cc996700 1 -- 172.21.15.40:0/2454901492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86c80b1640 msgr2=0x7f86c8070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86cc996700 1 --2- 172.21.15.40:0/2454901492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86c80b1640 0x7f86c8070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86cc996700 1 -- 172.21.15.40:0/2454901492 >> v1:172.21.15.169:6789/0 conn(0x7f86c80b43a0 legacy=0x7f86c815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:58.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86cc996700 1 -- 172.21.15.40:0/2454901492 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c80cada0 con 0x7f86c80c9040 2023-11-08T08:21:58.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86c77fe700 1 --2- 172.21.15.40:0/2454901492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86c80b1640 0x7f86c8070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:58.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86cc996700 1 --2- 172.21.15.40:0/2454901492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80c9040 0x7f86c80705c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009e50 tx=0x7f86c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991658ecaff96415 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:58.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86c6ffd700 1 -- 172.21.15.40:0/2454901492 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c000e070 con 0x7f86c80c9040 2023-11-08T08:21:58.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86c6ffd700 1 -- 172.21.15.40:0/2454901492 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c000b9c0 con 0x7f86c80c9040 2023-11-08T08:21:58.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86c6ffd700 1 -- 172.21.15.40:0/2454901492 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c000bd70 con 0x7f86c80c9040 2023-11-08T08:21:58.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86cd197700 1 -- 172.21.15.40:0/2454901492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80c9040 msgr2=0x7f86c80705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:58.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86cd197700 1 --2- 172.21.15.40:0/2454901492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80c9040 0x7f86c80705c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009e50 tx=0x7f86c000b330 comp rx=0 tx=0).stop 2023-11-08T08:21:58.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86cd197700 1 -- 172.21.15.40:0/2454901492 shutdown_connections 2023-11-08T08:21:58.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86cd197700 1 --2- 172.21.15.40:0/2454901492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86c80b1640 0x7f86c8070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:58.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86cd197700 1 --2- 172.21.15.40:0/2454901492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80c9040 0x7f86c80705c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:58.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86cd197700 1 -- 172.21.15.40:0/2454901492 >> 172.21.15.40:0/2454901492 conn(0x7f86c80b0f80 msgr2=0x7f86c80ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:58.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86cd197700 1 -- 172.21.15.40:0/2454901492 shutdown_connections 2023-11-08T08:21:58.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.299+0000 7f86cd197700 1 -- 172.21.15.40:0/2454901492 wait complete. 2023-11-08T08:21:58.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.307+0000 7f86cd197700 1 Processor -- start 2023-11-08T08:21:58.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.307+0000 7f86cd197700 1 -- start start 2023-11-08T08:21:58.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.307+0000 7f86cd197700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86c80b1640 0x7f86c80c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:58.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.307+0000 7f86cd197700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86c80b43a0 0x7f86c814c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:58.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.307+0000 7f86cd197700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80c9040 0x7f86c80ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:58.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.307+0000 7f86cd197700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c80c6810 con 0x7f86c80c9040 2023-11-08T08:21:58.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.307+0000 7f86cd197700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c80c86e0 con 0x7f86c80b43a0 2023-11-08T08:21:58.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.307+0000 7f86cd197700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c80c5ae0 con 0x7f86c80b1640 2023-11-08T08:21:58.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.307+0000 7f86cc996700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80c9040 0x7f86c80ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:58.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.307+0000 7f86cc996700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80c9040 0x7f86c80ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56386/0 (socket says 172.21.15.40:56386) 2023-11-08T08:21:58.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.307+0000 7f86cc996700 1 -- 172.21.15.40:0/3238554034 learned_addr learned my addr 172.21.15.40:0/3238554034 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:58.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.311+0000 7f86c7fff700 1 --2- 172.21.15.40:0/3238554034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86c80b1640 0x7f86c80c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.311+0000 7f86c77fe700 1 --2- 172.21.15.40:0/3238554034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86c80b43a0 0x7f86c814c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.311+0000 7f86cc996700 1 -- 172.21.15.40:0/3238554034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86c80b1640 msgr2=0x7f86c80c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.311+0000 7f86cc996700 1 --2- 172.21.15.40:0/3238554034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86c80b1640 0x7f86c80c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.311+0000 7f86cc996700 1 -- 172.21.15.40:0/3238554034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86c80b43a0 msgr2=0x7f86c814c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.311+0000 7f86cc996700 1 --2- 172.21.15.40:0/3238554034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86c80b43a0 0x7f86c814c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:58.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.311+0000 7f86cc996700 1 -- 172.21.15.40:0/3238554034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c0009b30 con 0x7f86c80c9040 2023-11-08T08:21:58.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.311+0000 7f86cc996700 1 --2- 172.21.15.40:0/3238554034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80c9040 0x7f86c80ae290 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f86c8072eb0 tx=0x7f86c0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:58.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.311+0000 7f86c4ff9700 1 -- 172.21.15.40:0/3238554034 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0004800 con 0x7f86c80c9040 2023-11-08T08:21:58.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.311+0000 7f86c4ff9700 1 -- 172.21.15.40:0/3238554034 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c00092e0 con 0x7f86c80c9040 2023-11-08T08:21:58.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.311+0000 7f86cd197700 1 -- 172.21.15.40:0/3238554034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c806b5d0 con 0x7f86c80c9040 2023-11-08T08:21:58.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.311+0000 7f86c4ff9700 1 -- 172.21.15.40:0/3238554034 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c001c650 con 0x7f86c80c9040 2023-11-08T08:21:58.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.311+0000 7f86cd197700 1 -- 172.21.15.40:0/3238554034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c806b2f0 con 0x7f86c80c9040 2023-11-08T08:21:58.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.315+0000 7f86cd197700 1 -- 172.21.15.40:0/3238554034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c80ca7b0 con 0x7f86c80c9040 2023-11-08T08:21:58.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.315+0000 7f86c4ff9700 1 -- 172.21.15.40:0/3238554034 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f86c002b070 con 0x7f86c80c9040 2023-11-08T08:21:58.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.315+0000 7f86c4ff9700 1 --2- 172.21.15.40:0/3238554034 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f86b4064ee0 0x7f86b4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:58.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.315+0000 7f86c4ff9700 1 -- 172.21.15.40:0/3238554034 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f86c0083700 con 0x7f86c80c9040 2023-11-08T08:21:58.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.319+0000 7f86c7fff700 1 --2- 172.21.15.40:0/3238554034 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f86b4064ee0 0x7f86b4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:58.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.319+0000 7f86c7fff700 1 --2- 172.21.15.40:0/3238554034 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f86b4064ee0 0x7f86b4067390 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f86b8000f60 tx=0x7f86b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:58.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.323+0000 7f86c4ff9700 1 -- 172.21.15.40:0/3238554034 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f86c0051670 con 0x7f86c80c9040 2023-11-08T08:21:58.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.535+0000 7f86cd197700 1 -- 172.21.15.40:0/3238554034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86c8073cb0 con 0x7f86c80c9040 2023-11-08T08:21:58.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.535+0000 7f86c4ff9700 1 -- 172.21.15.40:0/3238554034 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f86c0054c90 con 0x7f86c80c9040 2023-11-08T08:21:58.544 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:58.544 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1549516710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1549516710},{"type":"v1","addr":"172.21.15.169:6837","nonce":1549516710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5728,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/912870125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":912870125},{"type":"v1","addr":"172.21.15.169:6835","nonce":912870125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5772,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1045606427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1045606427},{"type":"v1","addr":"172.21.15.40:6834","nonce":1045606427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:21:17.730002+0000","modified":"2023-11-08T08:21:20.752976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2471311394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2471311394},{"type":"v1","addr":"172.21.15.40:6835","nonce":2471311394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T08:21:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.539+0000 7f86cd197700 1 -- 172.21.15.40:0/3238554034 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f86b4064ee0 msgr2=0x7f86b4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.539+0000 7f86cd197700 1 --2- 172.21.15.40:0/3238554034 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f86b4064ee0 0x7f86b4067390 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f86b8000f60 tx=0x7f86b8009040 comp rx=0 tx=0).stop 2023-11-08T08:21:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.539+0000 7f86cd197700 1 -- 172.21.15.40:0/3238554034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80c9040 msgr2=0x7f86c80ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.539+0000 7f86cd197700 1 --2- 172.21.15.40:0/3238554034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80c9040 0x7f86c80ae290 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f86c8072eb0 tx=0x7f86c0004af0 comp rx=0 tx=0).stop 2023-11-08T08:21:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.539+0000 7f86cd197700 1 -- 172.21.15.40:0/3238554034 shutdown_connections 2023-11-08T08:21:58.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.539+0000 7f86cd197700 1 --2- 172.21.15.40:0/3238554034 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f86b4064ee0 0x7f86b4067390 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:58.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.539+0000 7f86cd197700 1 --2- 172.21.15.40:0/3238554034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80c9040 0x7f86c80ae290 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:58.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.539+0000 7f86cd197700 1 --2- 172.21.15.40:0/3238554034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86c80b43a0 0x7f86c814c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:58.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.539+0000 7f86cd197700 1 --2- 172.21.15.40:0/3238554034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86c80b1640 0x7f86c80c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:58.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.539+0000 7f86cd197700 1 -- 172.21.15.40:0/3238554034 >> 172.21.15.40:0/3238554034 conn(0x7f86c80b0f80 msgr2=0x7f86c8159bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:58.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.539+0000 7f86cd197700 1 -- 172.21.15.40:0/3238554034 shutdown_connections 2023-11-08T08:21:58.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.539+0000 7f86cd197700 1 -- 172.21.15.40:0/3238554034 wait complete. 2023-11-08T08:21:58.550 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 25 2023-11-08T08:21:58.568 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-08T08:21:58.773 INFO:teuthology.orchestra.run.smithi040.stdout:{"message":"","return_code":0} 2023-11-08T08:21:58.774 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-08T08:21:58.774 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-08T08:21:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.835+0000 7f99aae95140 1 Processor -- start 2023-11-08T08:21:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.835+0000 7f99aae95140 1 -- start start 2023-11-08T08:21:58.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99aae95140 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x565155544700 0x565155571790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99aae95140 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565155577980 0x565155592840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99aae95140 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565155590f80 0x56515556cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99aae95140 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651554d9180 con 0x565155544700 2023-11-08T08:21:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99aae95140 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515546c240 con 0x565155590f80 2023-11-08T08:21:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99aae95140 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x56515556e670 con 0x565155577980 2023-11-08T08:21:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99a89ca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x565155544700 0x565155571790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99a89ca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x565155544700 0x565155571790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56392/0 (socket says 172.21.15.40:56392) 2023-11-08T08:21:58.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99a89ca700 1 -- 172.21.15.40:0/2335117983 learned_addr learned my addr 172.21.15.40:0/2335117983 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99a89ca700 1 -- 172.21.15.40:0/2335117983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565155577980 msgr2=0x565155592840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99a91cb700 1 --2- 172.21.15.40:0/2335117983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565155590f80 0x56515556cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99a89ca700 1 --2- 172.21.15.40:0/2335117983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565155577980 0x565155592840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99a89ca700 1 -- 172.21.15.40:0/2335117983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565155590f80 msgr2=0x56515556cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99a89ca700 1 --2- 172.21.15.40:0/2335117983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565155590f80 0x56515556cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99a89ca700 1 -- 172.21.15.40:0/2335117983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9990005d20 con 0x565155544700 2023-11-08T08:21:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99a91cb700 1 --2- 172.21.15.40:0/2335117983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565155590f80 0x56515556cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:21:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99a89ca700 1 --2- 172.21.15.40:0/2335117983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x565155544700 0x565155571790 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9998001d40 tx=0x7f999800b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb11be0fc7663fc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:58.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99a37fe700 1 -- 172.21.15.40:0/2335117983 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999800be60 con 0x565155544700 2023-11-08T08:21:58.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99a37fe700 1 -- 172.21.15.40:0/2335117983 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999800e470 con 0x565155544700 2023-11-08T08:21:58.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99a37fe700 1 -- 172.21.15.40:0/2335117983 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999800e800 con 0x565155544700 2023-11-08T08:21:58.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99aae95140 1 -- 172.21.15.40:0/2335117983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x565155544700 msgr2=0x565155571790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:58.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99aae95140 1 --2- 172.21.15.40:0/2335117983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x565155544700 0x565155571790 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9998001d40 tx=0x7f999800b5d0 comp rx=0 tx=0).stop 2023-11-08T08:21:58.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99aae95140 1 -- 172.21.15.40:0/2335117983 shutdown_connections 2023-11-08T08:21:58.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99aae95140 1 --2- 172.21.15.40:0/2335117983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565155590f80 0x56515556cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:58.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99aae95140 1 --2- 172.21.15.40:0/2335117983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565155577980 0x565155592840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:58.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99aae95140 1 --2- 172.21.15.40:0/2335117983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x565155544700 0x565155571790 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:58.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99aae95140 1 -- 172.21.15.40:0/2335117983 >> 172.21.15.40:0/2335117983 conn(0x56515554ac30 msgr2=0x56515561f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:58.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99aae95140 1 -- 172.21.15.40:0/2335117983 shutdown_connections 2023-11-08T08:21:58.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.839+0000 7f99aae95140 1 -- 172.21.15.40:0/2335117983 wait complete. 2023-11-08T08:21:58.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.843+0000 7f99aae95140 1 Processor -- start 2023-11-08T08:21:58.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.843+0000 7f99aae95140 1 -- start start 2023-11-08T08:21:58.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.843+0000 7f99aae95140 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565155544700 0x56515560c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:58.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.843+0000 7f99aae95140 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x565155577980 0x5651556b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:58.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.843+0000 7f99aae95140 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565155590f80 0x5651556b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:58.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.843+0000 7f99aae95140 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515561ea40 con 0x565155577980 2023-11-08T08:21:58.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.843+0000 7f99aae95140 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515560b200 con 0x565155590f80 2023-11-08T08:21:58.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.843+0000 7f99aae95140 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x56515560b4a0 con 0x565155544700 2023-11-08T08:21:58.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.843+0000 7f99a01c9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x565155577980 0x5651556b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:58.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.843+0000 7f99a01c9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x565155577980 0x5651556b6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56404/0 (socket says 172.21.15.40:56404) 2023-11-08T08:21:58.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.843+0000 7f99a89ca700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565155544700 0x56515560c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:58.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.843+0000 7f99a01c9700 1 -- 172.21.15.40:0/1329064471 learned_addr learned my addr 172.21.15.40:0/1329064471 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:58.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.843+0000 7f99a91cb700 1 --2- 172.21.15.40:0/1329064471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565155590f80 0x5651556b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:58.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.843+0000 7f99a01c9700 1 -- 172.21.15.40:0/1329064471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565155544700 msgr2=0x56515560c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:58.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.843+0000 7f99a01c9700 1 --2- 172.21.15.40:0/1329064471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565155544700 0x56515560c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:58.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.843+0000 7f99a01c9700 1 -- 172.21.15.40:0/1329064471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565155590f80 msgr2=0x5651556b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:58.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.843+0000 7f99a01c9700 1 --2- 172.21.15.40:0/1329064471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565155590f80 0x5651556b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:58.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.843+0000 7f99a01c9700 1 -- 172.21.15.40:0/1329064471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9994001e10 con 0x565155577980 2023-11-08T08:21:58.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.843+0000 7f99a89ca700 1 --2- 172.21.15.40:0/1329064471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565155544700 0x56515560c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:21:58.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.847+0000 7f99a01c9700 1 --2- 172.21.15.40:0/1329064471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x565155577980 0x5651556b6500 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f999000a440 tx=0x7f9990005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:58.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.847+0000 7f99a27fc700 1 -- 172.21.15.40:0/1329064471 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99900054e0 con 0x565155577980 2023-11-08T08:21:58.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.847+0000 7f99a27fc700 1 -- 172.21.15.40:0/1329064471 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999001e860 con 0x565155577980 2023-11-08T08:21:58.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.847+0000 7f99a27fc700 1 -- 172.21.15.40:0/1329064471 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999001ec10 con 0x565155577980 2023-11-08T08:21:58.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.847+0000 7f99aae95140 1 -- 172.21.15.40:0/1329064471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5651554778b0 con 0x565155577980 2023-11-08T08:21:58.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.847+0000 7f99aae95140 1 -- 172.21.15.40:0/1329064471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651554845d0 con 0x565155577980 2023-11-08T08:21:58.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.847+0000 7f99a27fc700 1 -- 172.21.15.40:0/1329064471 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f999001ed70 con 0x565155577980 2023-11-08T08:21:58.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.851+0000 7f99a27fc700 1 --2- 172.21.15.40:0/1329064471 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9988063810 0x7f9988065cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:58.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.851+0000 7f99a27fc700 1 -- 172.21.15.40:0/1329064471 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f999000ae40 con 0x565155577980 2023-11-08T08:21:58.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.851+0000 7f99aae95140 1 --2- 172.21.15.40:0/1329064471 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x565155549f30 0x565155609790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:58.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.851+0000 7f99aae95140 1 -- 172.21.15.40:0/1329064471 --> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57) v8 -- 0x56515554a330 con 0x565155549f30 2023-11-08T08:21:58.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.851+0000 7f99a89ca700 1 --2- 172.21.15.40:0/1329064471 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9988063810 0x7f9988065cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:58.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.851+0000 7f99a91cb700 1 --2- 172.21.15.40:0/1329064471 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x565155549f30 0x565155609790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:58.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.851+0000 7f99a89ca700 1 --2- 172.21.15.40:0/1329064471 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9988063810 0x7f9988065cc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9998000c00 tx=0x7f999801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:58.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.851+0000 7f99a91cb700 1 --2- 172.21.15.40:0/1329064471 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x565155549f30 0x565155609790 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:58.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.855+0000 7f99a91cb700 1 -- 172.21.15.40:0/1329064471 <== osd.5 v2:172.21.15.169:6809/35793 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f99a4003450 con 0x565155549f30 2023-11-08T08:21:58.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.855+0000 7f99aae95140 1 -- 172.21.15.40:0/1329064471 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x565155549f30 msgr2=0x565155609790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:58.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.855+0000 7f99aae95140 1 --2- 172.21.15.40:0/1329064471 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x565155549f30 0x565155609790 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:58.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.855+0000 7f99aae95140 1 -- 172.21.15.40:0/1329064471 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9988063810 msgr2=0x7f9988065cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:58.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.855+0000 7f99aae95140 1 --2- 172.21.15.40:0/1329064471 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9988063810 0x7f9988065cc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9998000c00 tx=0x7f999801c040 comp rx=0 tx=0).stop 2023-11-08T08:21:58.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.855+0000 7f99aae95140 1 -- 172.21.15.40:0/1329064471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x565155577980 msgr2=0x5651556b6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:58.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.855+0000 7f99aae95140 1 --2- 172.21.15.40:0/1329064471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x565155577980 0x5651556b6500 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f999000a440 tx=0x7f9990005a00 comp rx=0 tx=0).stop 2023-11-08T08:21:58.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.855+0000 7f99a91cb700 1 -- 172.21.15.40:0/1329064471 reap_dead start 2023-11-08T08:21:58.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.855+0000 7f99aae95140 1 -- 172.21.15.40:0/1329064471 shutdown_connections 2023-11-08T08:21:58.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.855+0000 7f99aae95140 1 -- 172.21.15.40:0/1329064471 >> 172.21.15.40:0/1329064471 conn(0x56515554ac30 msgr2=0x565155571d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:58.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.855+0000 7f99aae95140 1 -- 172.21.15.40:0/1329064471 shutdown_connections 2023-11-08T08:21:58.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:58.855+0000 7f99aae95140 1 -- 172.21.15.40:0/1329064471 wait complete. 2023-11-08T08:21:58.869 INFO:teuthology.orchestra.run.smithi040.stdout:Yalphaparent_a 2023-11-08T08:21:58.869 DEBUG:teuthology.orchestra.run.smithi040:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-08T08:21:58.960 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:21:58.960 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511627777, 2023-11-08T08:21:58.960 INFO:teuthology.orchestra.run.smithi040.stdout: "ancestors": [ 2023-11-08T08:21:58.961 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:58.961 INFO:teuthology.orchestra.run.smithi040.stdout: "dirino": 1099511627776, 2023-11-08T08:21:58.961 INFO:teuthology.orchestra.run.smithi040.stdout: "dname": "alpha", 2023-11-08T08:21:58.961 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 2 2023-11-08T08:21:58.961 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:21:58.962 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:21:58.962 INFO:teuthology.orchestra.run.smithi040.stdout: "dirino": 1, 2023-11-08T08:21:58.962 INFO:teuthology.orchestra.run.smithi040.stdout: "dname": "parent_a", 2023-11-08T08:21:58.962 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 4 2023-11-08T08:21:58.962 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:21:58.963 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-11-08T08:21:58.963 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 9, 2023-11-08T08:21:58.963 INFO:teuthology.orchestra.run.smithi040.stdout: "old_pools": [] 2023-11-08T08:21:58.963 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:21:58.963 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:58.968 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T08:21:58.969 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T08:21:59.060 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T08:21:59.103 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T08:21:59.104 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:21:59.104 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:21:59.126 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:21:59.126 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:21:59.190 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:21:59.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.415+0000 7fce2288a700 1 Processor -- start 2023-11-08T08:21:59.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.415+0000 7fce2288a700 1 -- start start 2023-11-08T08:21:59.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.419+0000 7fce2288a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fce1c0c8d10 0x7fce1c0cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:59.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.419+0000 7fce2288a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fce1c069d60 0x7fce1c06a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:59.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.419+0000 7fce2288a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce1c15a850 con 0x7fce1c15a480 2023-11-08T08:21:59.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.419+0000 7fce2288a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce1c0ce060 con 0x7fce1c0c8d10 2023-11-08T08:21:59.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.419+0000 7fce2288a700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce1c0ce300 con 0x7fce1c069d60 2023-11-08T08:21:59.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.419+0000 7fce1bfff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fce1c15a480 0x7fce1c0cc290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41838/0 (socket says 172.21.15.40:41838) 2023-11-08T08:21:59.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.419+0000 7fce1bfff700 1 -- 172.21.15.40:0/2678476054 learned_addr learned my addr 172.21.15.40:0/2678476054 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:21:59.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.419+0000 7fce1b7fe700 1 --2- 172.21.15.40:0/2678476054 >> v2:172.21.15.40:3300/0 conn(0x7fce1c0c8d10 0x7fce1c0cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:59.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.419+0000 7fce20e27700 1 --2- 172.21.15.40:0/2678476054 >> v2:172.21.15.169:3300/0 conn(0x7fce1c069d60 0x7fce1c06a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:59.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.419+0000 7fce1affd700 1 -- 172.21.15.40:0/2678476054 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954752847 0 0) 0x7fce1c15a850 con 0x7fce1c15a480 2023-11-08T08:21:59.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.419+0000 7fce1affd700 1 -- 172.21.15.40:0/2678476054 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce040035d0 con 0x7fce1c15a480 2023-11-08T08:21:59.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.419+0000 7fce1b7fe700 1 -- 172.21.15.40:0/2678476054 >> v2:172.21.15.169:3300/0 conn(0x7fce1c069d60 msgr2=0x7fce1c06a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:59.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.419+0000 7fce1b7fe700 1 --2- 172.21.15.40:0/2678476054 >> v2:172.21.15.169:3300/0 conn(0x7fce1c069d60 0x7fce1c06a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:59.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.419+0000 7fce1b7fe700 1 -- 172.21.15.40:0/2678476054 >> v1:172.21.15.40:6789/0 conn(0x7fce1c15a480 legacy=0x7fce1c0cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:59.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.419+0000 7fce1b7fe700 1 -- 172.21.15.40:0/2678476054 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1c0cada0 con 0x7fce1c0c8d10 2023-11-08T08:21:59.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.423+0000 7fce1b7fe700 1 --2- 172.21.15.40:0/2678476054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce1c0c8d10 0x7fce1c0cc9a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0c9340 tx=0x7fce10009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b1ca8a740b1e25 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:59.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.423+0000 7fce1affd700 1 -- 172.21.15.40:0/2678476054 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce10011830 con 0x7fce1c0c8d10 2023-11-08T08:21:59.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.423+0000 7fce1affd700 1 -- 172.21.15.40:0/2678476054 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce10011990 con 0x7fce1c0c8d10 2023-11-08T08:21:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.423+0000 7fce1affd700 1 -- 172.21.15.40:0/2678476054 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce10011d20 con 0x7fce1c0c8d10 2023-11-08T08:21:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.423+0000 7fce2288a700 1 -- 172.21.15.40:0/2678476054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce1c0c8d10 msgr2=0x7fce1c0cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.423+0000 7fce2288a700 1 --2- 172.21.15.40:0/2678476054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce1c0c8d10 0x7fce1c0cc9a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0c9340 tx=0x7fce10009b30 comp rx=0 tx=0).stop 2023-11-08T08:21:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.423+0000 7fce2288a700 1 -- 172.21.15.40:0/2678476054 shutdown_connections 2023-11-08T08:21:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.423+0000 7fce2288a700 1 --2- 172.21.15.40:0/2678476054 >> v2:172.21.15.169:3300/0 conn(0x7fce1c069d60 0x7fce1c06a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.423+0000 7fce2288a700 1 --2- 172.21.15.40:0/2678476054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce1c0c8d10 0x7fce1c0cc9a0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:59.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.423+0000 7fce2288a700 1 -- 172.21.15.40:0/2678476054 >> 172.21.15.40:0/2678476054 conn(0x7fce1c0cf350 msgr2=0x7fce1c0b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:59.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.423+0000 7fce2288a700 1 -- 172.21.15.40:0/2678476054 shutdown_connections 2023-11-08T08:21:59.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.423+0000 7fce2288a700 1 -- 172.21.15.40:0/2678476054 wait complete. 2023-11-08T08:21:59.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.427+0000 7fce2288a700 1 Processor -- start 2023-11-08T08:21:59.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.427+0000 7fce2288a700 1 -- start start 2023-11-08T08:21:59.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.427+0000 7fce2288a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce1c069d60 0x7fce1c0c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:59.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.427+0000 7fce2288a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce1c0c8d10 0x7fce1c0b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:59.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.427+0000 7fce2288a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce1c15a480 0x7fce1c15cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:59.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.427+0000 7fce2288a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c15b1f0 con 0x7fce1c069d60 2023-11-08T08:21:59.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.427+0000 7fce2288a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c0c6990 con 0x7fce1c0c8d10 2023-11-08T08:21:59.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.427+0000 7fce2288a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c0b3b40 con 0x7fce1c15a480 2023-11-08T08:21:59.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.427+0000 7fce1bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce1c069d60 0x7fce1c0c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:59.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.427+0000 7fce1bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce1c069d60 0x7fce1c0c6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56422/0 (socket says 172.21.15.40:56422) 2023-11-08T08:21:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.427+0000 7fce1bfff700 1 -- 172.21.15.40:0/3092959255 learned_addr learned my addr 172.21.15.40:0/3092959255 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.427+0000 7fce1b7fe700 1 --2- 172.21.15.40:0/3092959255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce1c0c8d10 0x7fce1c0b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.427+0000 7fce20e27700 1 --2- 172.21.15.40:0/3092959255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce1c15a480 0x7fce1c15cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:59.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.427+0000 7fce1bfff700 1 -- 172.21.15.40:0/3092959255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce1c15a480 msgr2=0x7fce1c15cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:59.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.427+0000 7fce1bfff700 1 --2- 172.21.15.40:0/3092959255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce1c15a480 0x7fce1c15cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.427+0000 7fce1bfff700 1 -- 172.21.15.40:0/3092959255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce1c0c8d10 msgr2=0x7fce1c0b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.427+0000 7fce1bfff700 1 --2- 172.21.15.40:0/3092959255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce1c0c8d10 0x7fce1c0b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.427+0000 7fce1bfff700 1 -- 172.21.15.40:0/3092959255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce100094b0 con 0x7fce1c069d60 2023-11-08T08:21:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.427+0000 7fce1bfff700 1 --2- 172.21.15.40:0/3092959255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce1c069d60 0x7fce1c0c6450 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00c4d0 tx=0x7fce0c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.431+0000 7fce18ff9700 1 -- 172.21.15.40:0/3092959255 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c005d90 con 0x7fce1c069d60 2023-11-08T08:21:59.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.431+0000 7fce18ff9700 1 -- 172.21.15.40:0/3092959255 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0c0070d0 con 0x7fce1c069d60 2023-11-08T08:21:59.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.431+0000 7fce2288a700 1 -- 172.21.15.40:0/3092959255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1c0add50 con 0x7fce1c069d60 2023-11-08T08:21:59.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.431+0000 7fce18ff9700 1 -- 172.21.15.40:0/3092959255 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c007480 con 0x7fce1c069d60 2023-11-08T08:21:59.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.431+0000 7fce2288a700 1 -- 172.21.15.40:0/3092959255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1c0caa30 con 0x7fce1c069d60 2023-11-08T08:21:59.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.431+0000 7fce2288a700 1 -- 172.21.15.40:0/3092959255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1c0ca6e0 con 0x7fce1c069d60 2023-11-08T08:21:59.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.435+0000 7fce18ff9700 1 -- 172.21.15.40:0/3092959255 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fce0c019030 con 0x7fce1c069d60 2023-11-08T08:21:59.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.435+0000 7fce18ff9700 1 --2- 172.21.15.40:0/3092959255 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fce08064ee0 0x7fce08067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:59.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.435+0000 7fce18ff9700 1 -- 172.21.15.40:0/3092959255 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fce0c081350 con 0x7fce1c069d60 2023-11-08T08:21:59.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.435+0000 7fce1b7fe700 1 --2- 172.21.15.40:0/3092959255 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fce08064ee0 0x7fce08067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:59.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.435+0000 7fce1b7fe700 1 --2- 172.21.15.40:0/3092959255 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fce08064ee0 0x7fce08067390 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fce10000c00 tx=0x7fce10008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:59.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.439+0000 7fce18ff9700 1 -- 172.21.15.40:0/3092959255 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fce0c04f3b0 con 0x7fce1c069d60 2023-11-08T08:21:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.619+0000 7fce2288a700 1 -- 172.21.15.40:0/3092959255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce1c0726c0 con 0x7fce1c069d60 2023-11-08T08:21:59.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.619+0000 7fce18ff9700 1 -- 172.21.15.40:0/3092959255 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+18084 (secure 0 0 0) 0x7fce0c0529d0 con 0x7fce1c069d60 2023-11-08T08:21:59.629 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:21:59.629 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:21:18.736039+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:21:15.464908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-08T08:21:16.755373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:21:59.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.623+0000 7fce2288a700 1 -- 172.21.15.40:0/3092959255 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fce08064ee0 msgr2=0x7fce08067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:59.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.623+0000 7fce2288a700 1 --2- 172.21.15.40:0/3092959255 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fce08064ee0 0x7fce08067390 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fce10000c00 tx=0x7fce10008030 comp rx=0 tx=0).stop 2023-11-08T08:21:59.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.623+0000 7fce2288a700 1 -- 172.21.15.40:0/3092959255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce1c069d60 msgr2=0x7fce1c0c6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:59.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.623+0000 7fce2288a700 1 --2- 172.21.15.40:0/3092959255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce1c069d60 0x7fce1c0c6450 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00c4d0 tx=0x7fce0c00c7e0 comp rx=0 tx=0).stop 2023-11-08T08:21:59.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.623+0000 7fce2288a700 1 -- 172.21.15.40:0/3092959255 shutdown_connections 2023-11-08T08:21:59.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.623+0000 7fce2288a700 1 --2- 172.21.15.40:0/3092959255 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fce08064ee0 0x7fce08067390 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:59.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.623+0000 7fce2288a700 1 --2- 172.21.15.40:0/3092959255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce1c15a480 0x7fce1c15cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:59.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.623+0000 7fce2288a700 1 --2- 172.21.15.40:0/3092959255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce1c0c8d10 0x7fce1c0b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:59.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.623+0000 7fce2288a700 1 --2- 172.21.15.40:0/3092959255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce1c069d60 0x7fce1c0c6450 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:59.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.623+0000 7fce2288a700 1 -- 172.21.15.40:0/3092959255 >> 172.21.15.40:0/3092959255 conn(0x7fce1c0cf350 msgr2=0x7fce1c06a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:59.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.623+0000 7fce2288a700 1 -- 172.21.15.40:0/3092959255 shutdown_connections 2023-11-08T08:21:59.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.623+0000 7fce2288a700 1 -- 172.21.15.40:0/3092959255 wait complete. 2023-11-08T08:21:59.659 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T08:21:15.464908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T08:21:16.755373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T08:21:59.660 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:21:59.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.835+0000 7fb1b17b7700 1 Processor -- start 2023-11-08T08:21:59.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.835+0000 7fb1b17b7700 1 -- start start 2023-11-08T08:21:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.839+0000 7fb1b17b7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb1ac0d1d00 0x7fb1ac0d20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.839+0000 7fb1b17b7700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb1ac160920 0x7fb1ac164230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.839+0000 7fb1b17b7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1ac0d0540 0x7fb1ac164770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.839+0000 7fb1b17b7700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1ac071020 con 0x7fb1ac0d1d00 2023-11-08T08:21:59.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.839+0000 7fb1b17b7700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1ac0d2610 con 0x7fb1ac160920 2023-11-08T08:21:59.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.839+0000 7fb1b17b7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac070410 con 0x7fb1ac0d0540 2023-11-08T08:21:59.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.839+0000 7fb1abfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb1ac0d1d00 0x7fb1ac0d20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:59.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.839+0000 7fb1a37fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb1ac160920 0x7fb1ac164230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:59.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.839+0000 7fb1b0fb6700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1ac0d0540 0x7fb1ac164770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:59.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.839+0000 7fb1a37fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb1ac160920 0x7fb1ac164230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33936/0 (socket says 172.21.15.40:33936) 2023-11-08T08:21:59.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.839+0000 7fb1abfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb1ac0d1d00 0x7fb1ac0d20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56426/0 (socket says 172.21.15.40:56426) 2023-11-08T08:21:59.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.839+0000 7fb1a37fe700 1 -- 172.21.15.40:0/3094190396 learned_addr learned my addr 172.21.15.40:0/3094190396 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:59.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.839+0000 7fb1a37fe700 1 -- 172.21.15.40:0/3094190396 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1ac0d0540 msgr2=0x7fb1ac164770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:59.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.839+0000 7fb1a37fe700 1 --2- 172.21.15.40:0/3094190396 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1ac0d0540 0x7fb1ac164770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:59.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.839+0000 7fb1a37fe700 1 -- 172.21.15.40:0/3094190396 >> v2:172.21.15.40:3300/0 conn(0x7fb1ac0d1d00 msgr2=0x7fb1ac0d20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:21:59.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.839+0000 7fb1a37fe700 1 --2- 172.21.15.40:0/3094190396 >> v2:172.21.15.40:3300/0 conn(0x7fb1ac0d1d00 0x7fb1ac0d20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:59.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.839+0000 7fb1a37fe700 1 -- 172.21.15.40:0/3094190396 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ac0706f0 con 0x7fb1ac160920 2023-11-08T08:21:59.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.839+0000 7fb1abfff700 1 --2- 172.21.15.40:0/3094190396 >> v2:172.21.15.40:3300/0 conn(0x7fb1ac0d1d00 0x7fb1ac0d20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:21:59.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.839+0000 7fb1a37fe700 1 --2- 172.21.15.40:0/3094190396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1ac160920 0x7fb1ac164230 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac164190 tx=0x7fb194009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8ba94f0e0292b57 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:59.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.843+0000 7fb1ab7fe700 1 -- 172.21.15.40:0/3094190396 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb194005e50 con 0x7fb1ac160920 2023-11-08T08:21:59.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.843+0000 7fb1ab7fe700 1 -- 172.21.15.40:0/3094190396 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb194006030 con 0x7fb1ac160920 2023-11-08T08:21:59.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.843+0000 7fb1ab7fe700 1 -- 172.21.15.40:0/3094190396 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1940063c0 con 0x7fb1ac160920 2023-11-08T08:21:59.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.843+0000 7fb1b17b7700 1 -- 172.21.15.40:0/3094190396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1ac160920 msgr2=0x7fb1ac164230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:59.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.843+0000 7fb1b17b7700 1 --2- 172.21.15.40:0/3094190396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1ac160920 0x7fb1ac164230 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac164190 tx=0x7fb194009bf0 comp rx=0 tx=0).stop 2023-11-08T08:21:59.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.843+0000 7fb1b17b7700 1 -- 172.21.15.40:0/3094190396 shutdown_connections 2023-11-08T08:21:59.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.843+0000 7fb1b17b7700 1 --2- 172.21.15.40:0/3094190396 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1ac0d0540 0x7fb1ac164770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:59.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.843+0000 7fb1b17b7700 1 --2- 172.21.15.40:0/3094190396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1ac160920 0x7fb1ac164230 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:59.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.843+0000 7fb1b17b7700 1 --2- 172.21.15.40:0/3094190396 >> v2:172.21.15.40:3300/0 conn(0x7fb1ac0d1d00 0x7fb1ac0d20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:59.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.843+0000 7fb1b17b7700 1 -- 172.21.15.40:0/3094190396 >> 172.21.15.40:0/3094190396 conn(0x7fb1ac0d3680 msgr2=0x7fb1ac0cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:21:59.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.843+0000 7fb1b17b7700 1 -- 172.21.15.40:0/3094190396 shutdown_connections 2023-11-08T08:21:59.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.843+0000 7fb1b17b7700 1 -- 172.21.15.40:0/3094190396 wait complete. 2023-11-08T08:21:59.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.843+0000 7fb1b17b7700 1 Processor -- start 2023-11-08T08:21:59.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.847+0000 7fb1b17b7700 1 -- start start 2023-11-08T08:21:59.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.847+0000 7fb1b17b7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1ac0d0540 0x7fb1ac0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:59.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.847+0000 7fb1b17b7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1ac0d1d00 0x7fb1ac14de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:59.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.847+0000 7fb1b17b7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1ac160920 0x7fb1ac0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:59.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.847+0000 7fb1b17b7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac14e360 con 0x7fb1ac0d1d00 2023-11-08T08:21:59.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.847+0000 7fb1b17b7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac0cc670 con 0x7fb1ac0d0540 2023-11-08T08:21:59.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.847+0000 7fb1b17b7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac0cb120 con 0x7fb1ac160920 2023-11-08T08:21:59.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.847+0000 7fb1a37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1ac0d1d00 0x7fb1ac14de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:59.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.847+0000 7fb1a37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1ac0d1d00 0x7fb1ac14de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56440/0 (socket says 172.21.15.40:56440) 2023-11-08T08:21:59.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.847+0000 7fb1a37fe700 1 -- 172.21.15.40:0/1234406321 learned_addr learned my addr 172.21.15.40:0/1234406321 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:21:59.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.847+0000 7fb1abfff700 1 --2- 172.21.15.40:0/1234406321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1ac0d0540 0x7fb1ac0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:59.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.847+0000 7fb1b0fb6700 1 --2- 172.21.15.40:0/1234406321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1ac160920 0x7fb1ac0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:59.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.847+0000 7fb1a37fe700 1 -- 172.21.15.40:0/1234406321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1ac160920 msgr2=0x7fb1ac0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:59.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.847+0000 7fb1a37fe700 1 --2- 172.21.15.40:0/1234406321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1ac160920 0x7fb1ac0cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:59.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.847+0000 7fb1a37fe700 1 -- 172.21.15.40:0/1234406321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1ac0d0540 msgr2=0x7fb1ac0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:21:59.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.847+0000 7fb1a37fe700 1 --2- 172.21.15.40:0/1234406321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1ac0d0540 0x7fb1ac0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:21:59.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.847+0000 7fb1a37fe700 1 -- 172.21.15.40:0/1234406321 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c009a70 con 0x7fb1ac0d1d00 2023-11-08T08:21:59.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.847+0000 7fb1b0fb6700 1 --2- 172.21.15.40:0/1234406321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1ac160920 0x7fb1ac0cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:21:59.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.847+0000 7fb1a37fe700 1 --2- 172.21.15.40:0/1234406321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1ac0d1d00 0x7fb1ac14de20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb194009c30 tx=0x7fb194004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:59.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.847+0000 7fb1a97fa700 1 -- 172.21.15.40:0/1234406321 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb194004050 con 0x7fb1ac0d1d00 2023-11-08T08:21:59.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.851+0000 7fb1a97fa700 1 -- 172.21.15.40:0/1234406321 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1940041d0 con 0x7fb1ac0d1d00 2023-11-08T08:21:59.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.851+0000 7fb1a97fa700 1 -- 172.21.15.40:0/1234406321 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb194004580 con 0x7fb1ac0d1d00 2023-11-08T08:21:59.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.851+0000 7fb1b17b7700 1 -- 172.21.15.40:0/1234406321 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb194009520 con 0x7fb1ac0d1d00 2023-11-08T08:21:59.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.851+0000 7fb1b17b7700 1 -- 172.21.15.40:0/1234406321 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1ac070cb0 con 0x7fb1ac0d1d00 2023-11-08T08:21:59.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.851+0000 7fb1b17b7700 1 -- 172.21.15.40:0/1234406321 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1ac0709d0 con 0x7fb1ac0d1d00 2023-11-08T08:21:59.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.851+0000 7fb1a97fa700 1 -- 172.21.15.40:0/1234406321 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fb19400f050 con 0x7fb1ac0d1d00 2023-11-08T08:21:59.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.855+0000 7fb1a97fa700 1 --2- 172.21.15.40:0/1234406321 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb18c064ee0 0x7fb18c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:21:59.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.855+0000 7fb1a97fa700 1 -- 172.21.15.40:0/1234406321 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb1940839a0 con 0x7fb1ac0d1d00 2023-11-08T08:21:59.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.855+0000 7fb1abfff700 1 --2- 172.21.15.40:0/1234406321 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb18c064ee0 0x7fb18c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:21:59.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.855+0000 7fb1abfff700 1 --2- 172.21.15.40:0/1234406321 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb18c064ee0 0x7fb18c067390 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009fd0 tx=0x7fb19c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:21:59.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:21:59.859+0000 7fb1a97fa700 1 -- 172.21.15.40:0/1234406321 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fb1940518b0 con 0x7fb1ac0d1d00 2023-11-08T08:22:00.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.035+0000 7fb1b17b7700 1 -- 172.21.15.40:0/1234406321 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1ac06cf30 con 0x7fb1ac0d1d00 2023-11-08T08:22:00.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.035+0000 7fb1a97fa700 1 -- 172.21.15.40:0/1234406321 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb194054ed0 con 0x7fb1ac0d1d00 2023-11-08T08:22:00.043 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:22:00.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.039+0000 7fb1b17b7700 1 -- 172.21.15.40:0/1234406321 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb18c064ee0 msgr2=0x7fb18c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:00.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.039+0000 7fb1b17b7700 1 --2- 172.21.15.40:0/1234406321 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb18c064ee0 0x7fb18c067390 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009fd0 tx=0x7fb19c00b040 comp rx=0 tx=0).stop 2023-11-08T08:22:00.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.039+0000 7fb1b17b7700 1 -- 172.21.15.40:0/1234406321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1ac0d1d00 msgr2=0x7fb1ac14de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:00.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.039+0000 7fb1b17b7700 1 --2- 172.21.15.40:0/1234406321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1ac0d1d00 0x7fb1ac14de20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb194009c30 tx=0x7fb194004770 comp rx=0 tx=0).stop 2023-11-08T08:22:00.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.039+0000 7fb1b17b7700 1 -- 172.21.15.40:0/1234406321 shutdown_connections 2023-11-08T08:22:00.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.039+0000 7fb1b17b7700 1 --2- 172.21.15.40:0/1234406321 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb18c064ee0 0x7fb18c067390 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.039+0000 7fb1b17b7700 1 --2- 172.21.15.40:0/1234406321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1ac160920 0x7fb1ac0cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.039+0000 7fb1b17b7700 1 --2- 172.21.15.40:0/1234406321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1ac0d1d00 0x7fb1ac14de20 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.039+0000 7fb1b17b7700 1 --2- 172.21.15.40:0/1234406321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1ac0d0540 0x7fb1ac0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.039+0000 7fb1b17b7700 1 -- 172.21.15.40:0/1234406321 >> 172.21.15.40:0/1234406321 conn(0x7fb1ac0d3680 msgr2=0x7fb1ac06add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:00.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.039+0000 7fb1b17b7700 1 -- 172.21.15.40:0/1234406321 shutdown_connections 2023-11-08T08:22:00.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.039+0000 7fb1b17b7700 1 -- 172.21.15.40:0/1234406321 wait complete. 2023-11-08T08:22:00.067 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:22:00.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.239+0000 7ff4a555e700 1 Processor -- start 2023-11-08T08:22:00.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.239+0000 7ff4a555e700 1 -- start start 2023-11-08T08:22:00.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.239+0000 7ff4a555e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a0154670 0x7ff4a00afa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:00.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.239+0000 7ff4a555e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff4a015a480 0x7ff4a00ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:00.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.239+0000 7ff4a555e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff4a00ce920 0x7ff4a015dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:00.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.239+0000 7ff4a555e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a015db10 con 0x7ff4a015a480 2023-11-08T08:22:00.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.239+0000 7ff4a555e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a015aa40 con 0x7ff4a00ce920 2023-11-08T08:22:00.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.239+0000 7ff4a555e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a015ad20 con 0x7ff4a0154670 2023-11-08T08:22:00.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.239+0000 7ff49e7fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff4a015a480 0x7ff4a00ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:00.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.239+0000 7ff49e7fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff4a015a480 0x7ff4a00ce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56448/0 (socket says 172.21.15.40:56448) 2023-11-08T08:22:00.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.239+0000 7ff49e7fc700 1 -- 172.21.15.40:0/2247034081 learned_addr learned my addr 172.21.15.40:0/2247034081 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:00.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.239+0000 7ff49effd700 1 --2- 172.21.15.40:0/2247034081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a0154670 0x7ff4a00afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:00.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.239+0000 7ff49f7fe700 1 --2- 172.21.15.40:0/2247034081 >> v2:172.21.15.169:3300/0 conn(0x7ff4a00ce920 0x7ff4a015dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:00.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.239+0000 7ff49e7fc700 1 -- 172.21.15.40:0/2247034081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a0154670 msgr2=0x7ff4a00afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:00.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.239+0000 7ff49e7fc700 1 --2- 172.21.15.40:0/2247034081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a0154670 0x7ff4a00afa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.239+0000 7ff49e7fc700 1 -- 172.21.15.40:0/2247034081 >> v2:172.21.15.169:3300/0 conn(0x7ff4a00ce920 msgr2=0x7ff4a015dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:00.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.239+0000 7ff49e7fc700 1 --2- 172.21.15.40:0/2247034081 >> v2:172.21.15.169:3300/0 conn(0x7ff4a00ce920 0x7ff4a015dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.239+0000 7ff49e7fc700 1 -- 172.21.15.40:0/2247034081 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a015b000 con 0x7ff4a015a480 2023-11-08T08:22:00.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.239+0000 7ff49e7fc700 1 --2- 172.21.15.40:0/2247034081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4a015a480 0x7ff4a00ce3e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff4a00cd240 tx=0x7ff494009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ad7a5117bd212c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:00.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.243+0000 7ff49dffb700 1 -- 172.21.15.40:0/2247034081 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49400b8a0 con 0x7ff4a015a480 2023-11-08T08:22:00.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.243+0000 7ff49dffb700 1 -- 172.21.15.40:0/2247034081 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49400ba00 con 0x7ff4a015a480 2023-11-08T08:22:00.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.243+0000 7ff49dffb700 1 -- 172.21.15.40:0/2247034081 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49400bd90 con 0x7ff4a015a480 2023-11-08T08:22:00.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.243+0000 7ff4a555e700 1 -- 172.21.15.40:0/2247034081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4a015a480 msgr2=0x7ff4a00ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:00.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.243+0000 7ff4a555e700 1 --2- 172.21.15.40:0/2247034081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4a015a480 0x7ff4a00ce3e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff4a00cd240 tx=0x7ff494009bf0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.243+0000 7ff4a555e700 1 -- 172.21.15.40:0/2247034081 shutdown_connections 2023-11-08T08:22:00.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.243+0000 7ff4a555e700 1 --2- 172.21.15.40:0/2247034081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4a015a480 0x7ff4a00ce3e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.243+0000 7ff4a555e700 1 --2- 172.21.15.40:0/2247034081 >> v2:172.21.15.169:3300/0 conn(0x7ff4a00ce920 0x7ff4a015dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.243+0000 7ff4a555e700 1 --2- 172.21.15.40:0/2247034081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a0154670 0x7ff4a00afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.243+0000 7ff4a555e700 1 -- 172.21.15.40:0/2247034081 >> 172.21.15.40:0/2247034081 conn(0x7ff4a00cd290 msgr2=0x7ff4a00cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:00.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.243+0000 7ff4a555e700 1 -- 172.21.15.40:0/2247034081 shutdown_connections 2023-11-08T08:22:00.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.243+0000 7ff4a555e700 1 -- 172.21.15.40:0/2247034081 wait complete. 2023-11-08T08:22:00.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.247+0000 7ff4a555e700 1 Processor -- start 2023-11-08T08:22:00.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.247+0000 7ff4a555e700 1 -- start start 2023-11-08T08:22:00.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.247+0000 7ff4a555e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4a00ce920 0x7ff4a0151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:00.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.247+0000 7ff4a555e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a0154670 0x7ff4a00c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:00.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.247+0000 7ff4a555e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4a015a480 0x7ff4a015b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:00.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.247+0000 7ff4a555e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a00b3990 con 0x7ff4a00ce920 2023-11-08T08:22:00.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.247+0000 7ff4a555e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a00b3640 con 0x7ff4a015a480 2023-11-08T08:22:00.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.247+0000 7ff4a555e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a00caf20 con 0x7ff4a0154670 2023-11-08T08:22:00.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.247+0000 7ff49effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4a00ce920 0x7ff4a0151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:00.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.247+0000 7ff49effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4a00ce920 0x7ff4a0151990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56458/0 (socket says 172.21.15.40:56458) 2023-11-08T08:22:00.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.247+0000 7ff49effd700 1 -- 172.21.15.40:0/3152480529 learned_addr learned my addr 172.21.15.40:0/3152480529 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:00.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.247+0000 7ff49e7fc700 1 --2- 172.21.15.40:0/3152480529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a0154670 0x7ff4a00c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:00.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.247+0000 7ff49f7fe700 1 --2- 172.21.15.40:0/3152480529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4a015a480 0x7ff4a015b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:00.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.247+0000 7ff49effd700 1 -- 172.21.15.40:0/3152480529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a0154670 msgr2=0x7ff4a00c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:00.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.247+0000 7ff49effd700 1 --2- 172.21.15.40:0/3152480529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a0154670 0x7ff4a00c8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.247+0000 7ff49effd700 1 -- 172.21.15.40:0/3152480529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4a015a480 msgr2=0x7ff4a015b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:00.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.247+0000 7ff49effd700 1 --2- 172.21.15.40:0/3152480529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4a015a480 0x7ff4a015b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.247+0000 7ff49effd700 1 -- 172.21.15.40:0/3152480529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff494009570 con 0x7ff4a00ce920 2023-11-08T08:22:00.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.247+0000 7ff49e7fc700 1 --2- 172.21.15.40:0/3152480529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a0154670 0x7ff4a00c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:22:00.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.247+0000 7ff49effd700 1 --2- 172.21.15.40:0/3152480529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4a00ce920 0x7ff4a0151990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff49000e7c0 tx=0x7ff49000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:00.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.251+0000 7ff4877fe700 1 -- 172.21.15.40:0/3152480529 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490013610 con 0x7ff4a00ce920 2023-11-08T08:22:00.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.251+0000 7ff4877fe700 1 -- 172.21.15.40:0/3152480529 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff490013770 con 0x7ff4a00ce920 2023-11-08T08:22:00.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.251+0000 7ff4a555e700 1 -- 172.21.15.40:0/3152480529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a015b2e0 con 0x7ff4a00ce920 2023-11-08T08:22:00.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.251+0000 7ff4877fe700 1 -- 172.21.15.40:0/3152480529 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490013ae0 con 0x7ff4a00ce920 2023-11-08T08:22:00.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.251+0000 7ff4a555e700 1 -- 172.21.15.40:0/3152480529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a00affb0 con 0x7ff4a00ce920 2023-11-08T08:22:00.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.251+0000 7ff4a555e700 1 -- 172.21.15.40:0/3152480529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a00b3c00 con 0x7ff4a00ce920 2023-11-08T08:22:00.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.251+0000 7ff4877fe700 1 -- 172.21.15.40:0/3152480529 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff490023070 con 0x7ff4a00ce920 2023-11-08T08:22:00.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.255+0000 7ff4877fe700 1 --2- 172.21.15.40:0/3152480529 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff48c064ee0 0x7ff48c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:00.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.255+0000 7ff4877fe700 1 -- 172.21.15.40:0/3152480529 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff490081d30 con 0x7ff4a00ce920 2023-11-08T08:22:00.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.255+0000 7ff49e7fc700 1 --2- 172.21.15.40:0/3152480529 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff48c064ee0 0x7ff48c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:00.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.255+0000 7ff49e7fc700 1 --2- 172.21.15.40:0/3152480529 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff48c064ee0 0x7ff48c067390 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff494014040 tx=0x7ff494008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.259+0000 7ff4877fe700 1 -- 172.21.15.40:0/3152480529 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ff49004fca0 con 0x7ff4a00ce920 2023-11-08T08:22:00.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.431+0000 7ff4a555e700 1 -- 172.21.15.40:0/3152480529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff4a0073d20 con 0x7ff4a00ce920 2023-11-08T08:22:00.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.435+0000 7ff4877fe700 1 -- 172.21.15.40:0/3152480529 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v57) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff4900532c0 con 0x7ff4a00ce920 2023-11-08T08:22:00.442 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:22:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.439+0000 7ff4a555e700 1 -- 172.21.15.40:0/3152480529 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff48c064ee0 msgr2=0x7ff48c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.439+0000 7ff4a555e700 1 --2- 172.21.15.40:0/3152480529 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff48c064ee0 0x7ff48c067390 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff494014040 tx=0x7ff494008030 comp rx=0 tx=0).stop 2023-11-08T08:22:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.439+0000 7ff4a555e700 1 -- 172.21.15.40:0/3152480529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4a00ce920 msgr2=0x7ff4a0151990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:00.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.439+0000 7ff4a555e700 1 --2- 172.21.15.40:0/3152480529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4a00ce920 0x7ff4a0151990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff49000e7c0 tx=0x7ff49000eb80 comp rx=0 tx=0).stop 2023-11-08T08:22:00.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.439+0000 7ff4a555e700 1 -- 172.21.15.40:0/3152480529 shutdown_connections 2023-11-08T08:22:00.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.439+0000 7ff4a555e700 1 --2- 172.21.15.40:0/3152480529 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff48c064ee0 0x7ff48c067390 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.439+0000 7ff4a555e700 1 --2- 172.21.15.40:0/3152480529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4a015a480 0x7ff4a015b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.439+0000 7ff4a555e700 1 --2- 172.21.15.40:0/3152480529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4a0154670 0x7ff4a00c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.439+0000 7ff4a555e700 1 --2- 172.21.15.40:0/3152480529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4a00ce920 0x7ff4a0151990 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.439+0000 7ff4a555e700 1 -- 172.21.15.40:0/3152480529 >> 172.21.15.40:0/3152480529 conn(0x7ff4a00cd290 msgr2=0x7ff4a006c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.439+0000 7ff4a555e700 1 -- 172.21.15.40:0/3152480529 shutdown_connections 2023-11-08T08:22:00.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.439+0000 7ff4a555e700 1 -- 172.21.15.40:0/3152480529 wait complete. 2023-11-08T08:22:00.467 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T08:22:00.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.639+0000 7f4f4b124700 1 Processor -- start 2023-11-08T08:22:00.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.639+0000 7f4f4b124700 1 -- start start 2023-11-08T08:22:00.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.643+0000 7f4f4b124700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4f4415c410 0x7f4f44160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:00.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.643+0000 7f4f4b124700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f440cf2a0 0x7f4f44069c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:00.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.643+0000 7f4f4b124700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f440ca740 con 0x7f4f4406a1a0 2023-11-08T08:22:00.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.643+0000 7f4f4b124700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f44161020 con 0x7f4f4415c410 2023-11-08T08:22:00.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.643+0000 7f4f4b124700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f440ca9e0 con 0x7f4f440cf2a0 2023-11-08T08:22:00.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.643+0000 7f4f49921700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f440cf2a0 0x7f4f44069c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:00.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.643+0000 7f4f4a122700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4f4415c410 0x7f4f44160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:00.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.643+0000 7f4f49921700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f440cf2a0 0x7f4f44069c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:40664/0 (socket says 172.21.15.40:40664) 2023-11-08T08:22:00.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.647+0000 7f4f49921700 1 -- 172.21.15.40:0/960552166 learned_addr learned my addr 172.21.15.40:0/960552166 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:00.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.647+0000 7f4f49921700 1 -- 172.21.15.40:0/960552166 >> v2:172.21.15.169:3300/0 conn(0x7f4f4415c410 msgr2=0x7f4f44160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:00.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.647+0000 7f4f49921700 1 --2- 172.21.15.40:0/960552166 >> v2:172.21.15.169:3300/0 conn(0x7f4f4415c410 0x7f4f44160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.647+0000 7f4f49921700 1 -- 172.21.15.40:0/960552166 >> v1:172.21.15.169:6789/0 conn(0x7f4f4406a1a0 legacy=0x7f4f440b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:00.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.647+0000 7f4f49921700 1 -- 172.21.15.40:0/960552166 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f440cacc0 con 0x7f4f440cf2a0 2023-11-08T08:22:00.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.647+0000 7f4f49921700 1 --2- 172.21.15.40:0/960552166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f440cf2a0 0x7f4f44069c60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009910 tx=0x7f4f34009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2cc43c255bb3461 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:00.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.647+0000 7f4f49120700 1 -- 172.21.15.40:0/960552166 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f34011830 con 0x7f4f440cf2a0 2023-11-08T08:22:00.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.647+0000 7f4f49120700 1 -- 172.21.15.40:0/960552166 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f34011990 con 0x7f4f440cf2a0 2023-11-08T08:22:00.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.647+0000 7f4f49120700 1 -- 172.21.15.40:0/960552166 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f34011d40 con 0x7f4f440cf2a0 2023-11-08T08:22:00.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.647+0000 7f4f4b124700 1 -- 172.21.15.40:0/960552166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f440cf2a0 msgr2=0x7f4f44069c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:00.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.647+0000 7f4f4b124700 1 --2- 172.21.15.40:0/960552166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f440cf2a0 0x7f4f44069c60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009910 tx=0x7f4f34009c20 comp rx=0 tx=0).stop 2023-11-08T08:22:00.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.647+0000 7f4f4b124700 1 -- 172.21.15.40:0/960552166 shutdown_connections 2023-11-08T08:22:00.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.647+0000 7f4f4b124700 1 --2- 172.21.15.40:0/960552166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f440cf2a0 0x7f4f44069c60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.647+0000 7f4f4b124700 1 --2- 172.21.15.40:0/960552166 >> v2:172.21.15.169:3300/0 conn(0x7f4f4415c410 0x7f4f44160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.647+0000 7f4f4b124700 1 -- 172.21.15.40:0/960552166 >> 172.21.15.40:0/960552166 conn(0x7f4f440cc240 msgr2=0x7f4f440cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:00.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.647+0000 7f4f4b124700 1 -- 172.21.15.40:0/960552166 shutdown_connections 2023-11-08T08:22:00.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.647+0000 7f4f4b124700 1 -- 172.21.15.40:0/960552166 wait complete. 2023-11-08T08:22:00.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.651+0000 7f4f4b124700 1 Processor -- start 2023-11-08T08:22:00.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.651+0000 7f4f4b124700 1 -- start start 2023-11-08T08:22:00.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.651+0000 7f4f4b124700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f4406a1a0 0x7f4f441f1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:00.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.651+0000 7f4f4b124700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f440cf2a0 0x7f4f440b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:00.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.651+0000 7f4f4b124700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f4415c410 0x7f4f440b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:00.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.651+0000 7f4f4b124700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f44156780 con 0x7f4f4406a1a0 2023-11-08T08:22:00.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.651+0000 7f4f4b124700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f441f6060 con 0x7f4f440cf2a0 2023-11-08T08:22:00.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.651+0000 7f4f4b124700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f440c8350 con 0x7f4f4415c410 2023-11-08T08:22:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.651+0000 7f4f4a122700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f4406a1a0 0x7f4f441f1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.651+0000 7f4f49921700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f440cf2a0 0x7f4f440b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.651+0000 7f4f4a122700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f4406a1a0 0x7f4f441f1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56470/0 (socket says 172.21.15.40:56470) 2023-11-08T08:22:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.651+0000 7f4f4a122700 1 -- 172.21.15.40:0/2881783710 learned_addr learned my addr 172.21.15.40:0/2881783710 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:00.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.651+0000 7f4f4a923700 1 --2- 172.21.15.40:0/2881783710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f4415c410 0x7f4f440b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:00.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.655+0000 7f4f49921700 1 -- 172.21.15.40:0/2881783710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f4415c410 msgr2=0x7f4f440b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:00.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.655+0000 7f4f49921700 1 --2- 172.21.15.40:0/2881783710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f4415c410 0x7f4f440b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.655+0000 7f4f49921700 1 -- 172.21.15.40:0/2881783710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f4406a1a0 msgr2=0x7f4f441f1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:00.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.655+0000 7f4f49921700 1 --2- 172.21.15.40:0/2881783710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f4406a1a0 0x7f4f441f1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.655+0000 7f4f49921700 1 -- 172.21.15.40:0/2881783710 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f34009570 con 0x7f4f440cf2a0 2023-11-08T08:22:00.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.655+0000 7f4f4a122700 1 --2- 172.21.15.40:0/2881783710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f4406a1a0 0x7f4f441f1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:22:00.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.655+0000 7f4f49921700 1 --2- 172.21.15.40:0/2881783710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f440cf2a0 0x7f4f440b2610 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4f34016000 tx=0x7f4f34004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:00.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.655+0000 7f4f3affd700 1 -- 172.21.15.40:0/2881783710 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f34004510 con 0x7f4f440cf2a0 2023-11-08T08:22:00.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.655+0000 7f4f3affd700 1 -- 172.21.15.40:0/2881783710 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f340036c0 con 0x7f4f440cf2a0 2023-11-08T08:22:00.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.655+0000 7f4f4b124700 1 -- 172.21.15.40:0/2881783710 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f440cde70 con 0x7f4f440cf2a0 2023-11-08T08:22:00.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.655+0000 7f4f3affd700 1 -- 172.21.15.40:0/2881783710 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f34003a70 con 0x7f4f440cf2a0 2023-11-08T08:22:00.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.655+0000 7f4f4b124700 1 -- 172.21.15.40:0/2881783710 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f440cdb90 con 0x7f4f440cf2a0 2023-11-08T08:22:00.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.655+0000 7f4f4b124700 1 -- 172.21.15.40:0/2881783710 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f4415c7e0 con 0x7f4f440cf2a0 2023-11-08T08:22:00.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.659+0000 7f4f3affd700 1 -- 172.21.15.40:0/2881783710 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4f3402d070 con 0x7f4f440cf2a0 2023-11-08T08:22:00.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.659+0000 7f4f3affd700 1 --2- 172.21.15.40:0/2881783710 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4f28064ee0 0x7f4f28067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:00.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.659+0000 7f4f3affd700 1 -- 172.21.15.40:0/2881783710 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4f34021030 con 0x7f4f440cf2a0 2023-11-08T08:22:00.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.663+0000 7f4f4a122700 1 --2- 172.21.15.40:0/2881783710 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4f28064ee0 0x7f4f28067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:00.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.663+0000 7f4f4a122700 1 --2- 172.21.15.40:0/2881783710 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4f28064ee0 0x7f4f28067390 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4f400112c0 tx=0x7f4f40009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:00.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.667+0000 7f4f3affd700 1 -- 172.21.15.40:0/2881783710 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4f340519d0 con 0x7f4f440cf2a0 2023-11-08T08:22:00.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.843+0000 7f4f4b124700 1 -- 172.21.15.40:0/2881783710 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4f44073d20 con 0x7f4f440cf2a0 2023-11-08T08:22:00.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.843+0000 7f4f3affd700 1 -- 172.21.15.40:0/2881783710 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4f34015020 con 0x7f4f440cf2a0 2023-11-08T08:22:00.851 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:22:00.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.847+0000 7f4f4b124700 1 -- 172.21.15.40:0/2881783710 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4f28064ee0 msgr2=0x7f4f28067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:00.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.847+0000 7f4f4b124700 1 --2- 172.21.15.40:0/2881783710 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4f28064ee0 0x7f4f28067390 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4f400112c0 tx=0x7f4f40009240 comp rx=0 tx=0).stop 2023-11-08T08:22:00.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.847+0000 7f4f4b124700 1 -- 172.21.15.40:0/2881783710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f440cf2a0 msgr2=0x7f4f440b2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:00.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.847+0000 7f4f4b124700 1 --2- 172.21.15.40:0/2881783710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f440cf2a0 0x7f4f440b2610 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4f34016000 tx=0x7f4f34004770 comp rx=0 tx=0).stop 2023-11-08T08:22:00.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.847+0000 7f4f4b124700 1 -- 172.21.15.40:0/2881783710 shutdown_connections 2023-11-08T08:22:00.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.847+0000 7f4f4b124700 1 --2- 172.21.15.40:0/2881783710 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4f28064ee0 0x7f4f28067390 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.847+0000 7f4f4b124700 1 --2- 172.21.15.40:0/2881783710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f4415c410 0x7f4f440b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.847+0000 7f4f4b124700 1 --2- 172.21.15.40:0/2881783710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f440cf2a0 0x7f4f440b2610 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.847+0000 7f4f4b124700 1 --2- 172.21.15.40:0/2881783710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f4406a1a0 0x7f4f441f1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:00.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.847+0000 7f4f4b124700 1 -- 172.21.15.40:0/2881783710 >> 172.21.15.40:0/2881783710 conn(0x7f4f440cc240 msgr2=0x7f4f440af530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:00.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.847+0000 7f4f4b124700 1 -- 172.21.15.40:0/2881783710 shutdown_connections 2023-11-08T08:22:00.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:00.847+0000 7f4f4b124700 1 -- 172.21.15.40:0/2881783710 wait complete. 2023-11-08T08:22:00.875 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T08:22:01.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.059+0000 7f23b88d5700 1 Processor -- start 2023-11-08T08:22:01.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.059+0000 7f23b88d5700 1 -- start start 2023-11-08T08:22:01.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.063+0000 7f23b88d5700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f23b415c150 0x7f23b40b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:01.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.063+0000 7f23b88d5700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23b406ff10 0x7f23b4070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:01.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.063+0000 7f23b88d5700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b415c520 con 0x7f23b415fc30 2023-11-08T08:22:01.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.063+0000 7f23b88d5700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b40b4240 con 0x7f23b415c150 2023-11-08T08:22:01.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.063+0000 7f23b88d5700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40708a0 con 0x7f23b406ff10 2023-11-08T08:22:01.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.063+0000 7f23b3fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f23b415fc30 0x7f23b41600b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41854/0 (socket says 172.21.15.40:41854) 2023-11-08T08:22:01.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.063+0000 7f23b3fff700 1 -- 172.21.15.40:0/939745277 learned_addr learned my addr 172.21.15.40:0/939745277 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:01.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.063+0000 7f23b2ffd700 1 --2- 172.21.15.40:0/939745277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23b406ff10 0x7f23b4070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:01.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.063+0000 7f23b37fe700 1 --2- 172.21.15.40:0/939745277 >> v2:172.21.15.169:3300/0 conn(0x7f23b415c150 0x7f23b40b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:01.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b27fc700 1 -- 172.21.15.40:0/939745277 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1660620517 0 0) 0x7f23b415c520 con 0x7f23b415fc30 2023-11-08T08:22:01.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b27fc700 1 -- 172.21.15.40:0/939745277 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f239c0035d0 con 0x7f23b415fc30 2023-11-08T08:22:01.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b27fc700 1 -- 172.21.15.40:0/939745277 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3390665458 0 0) 0x7f239c0035d0 con 0x7f23b415fc30 2023-11-08T08:22:01.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b27fc700 1 -- 172.21.15.40:0/939745277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23b406ff10 msgr2=0x7f23b4070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:01.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b27fc700 1 --2- 172.21.15.40:0/939745277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23b406ff10 0x7f23b4070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b27fc700 1 -- 172.21.15.40:0/939745277 >> v2:172.21.15.169:3300/0 conn(0x7f23b415c150 msgr2=0x7f23b40b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:01.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b27fc700 1 --2- 172.21.15.40:0/939745277 >> v2:172.21.15.169:3300/0 conn(0x7f23b415c150 0x7f23b40b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b27fc700 1 -- 172.21.15.40:0/939745277 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b406ac00 con 0x7f23b415fc30 2023-11-08T08:22:01.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b27fc700 1 -- 172.21.15.40:0/939745277 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f23ac0019f0 con 0x7f23b415fc30 2023-11-08T08:22:01.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b27fc700 1 -- 172.21.15.40:0/939745277 >> v1:172.21.15.40:6789/0 conn(0x7f23b415fc30 legacy=0x7f23b41600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:01.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b27fc700 1 --2- 172.21.15.40:0/939745277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f239c003a20 0x7f239c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:01.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b27fc700 1 --2- 172.21.15.40:0/939745277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f239c004400 0x7f239c0048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:01.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b27fc700 1 --2- 172.21.15.40:0/939745277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f239c004f40 0x7f239c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:01.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b27fc700 1 -- 172.21.15.40:0/939745277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40b4240 con 0x7f239c004400 2023-11-08T08:22:01.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b27fc700 1 -- 172.21.15.40:0/939745277 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40708a0 con 0x7f239c004f40 2023-11-08T08:22:01.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b27fc700 1 -- 172.21.15.40:0/939745277 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239c00bf70 con 0x7f239c003a20 2023-11-08T08:22:01.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b2ffd700 1 --2- 172.21.15.40:0/939745277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23b406ff10 0x7f23b4070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:22:01.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b37fe700 1 --2- 172.21.15.40:0/939745277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f239c003a20 0x7f239c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:01.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b2ffd700 1 --2- 172.21.15.40:0/939745277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f239c004400 0x7f239c0048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:01.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b3fff700 1 --2- 172.21.15.40:0/939745277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f239c004f40 0x7f239c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:01.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b2ffd700 1 -- 172.21.15.40:0/939745277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f239c003a20 msgr2=0x7f239c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:01.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b2ffd700 1 --2- 172.21.15.40:0/939745277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f239c003a20 0x7f239c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b2ffd700 1 -- 172.21.15.40:0/939745277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f239c004f40 msgr2=0x7f239c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:01.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b2ffd700 1 --2- 172.21.15.40:0/939745277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f239c004f40 0x7f239c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b2ffd700 1 -- 172.21.15.40:0/939745277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239c0035d0 con 0x7f239c004400 2023-11-08T08:22:01.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b2ffd700 1 --2- 172.21.15.40:0/939745277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f239c004400 0x7f239c0048d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f23b406ba10 tx=0x7f239c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c583bc15816afc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:01.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b27fc700 1 -- 172.21.15.40:0/939745277 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8004030 con 0x7f239c004400 2023-11-08T08:22:01.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b27fc700 1 -- 172.21.15.40:0/939745277 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a8004190 con 0x7f239c004400 2023-11-08T08:22:01.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b27fc700 1 -- 172.21.15.40:0/939745277 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8004540 con 0x7f239c004400 2023-11-08T08:22:01.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b88d5700 1 -- 172.21.15.40:0/939745277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f239c004400 msgr2=0x7f239c0048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:01.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b88d5700 1 --2- 172.21.15.40:0/939745277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f239c004400 0x7f239c0048d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f23b406ba10 tx=0x7f239c0058a0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.067+0000 7f23b3fff700 1 -- 172.21.15.40:0/939745277 reap_dead start 2023-11-08T08:22:01.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.071+0000 7f23b88d5700 1 -- 172.21.15.40:0/939745277 shutdown_connections 2023-11-08T08:22:01.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.071+0000 7f23b88d5700 1 -- 172.21.15.40:0/939745277 >> 172.21.15.40:0/939745277 conn(0x7f23b40d3680 msgr2=0x7f23b40cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:01.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.071+0000 7f23b88d5700 1 -- 172.21.15.40:0/939745277 shutdown_connections 2023-11-08T08:22:01.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.071+0000 7f23b88d5700 1 -- 172.21.15.40:0/939745277 wait complete. 2023-11-08T08:22:01.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.071+0000 7f23b88d5700 1 Processor -- start 2023-11-08T08:22:01.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.071+0000 7f23b88d5700 1 -- start start 2023-11-08T08:22:01.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f23b88d5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f239c004400 0x7f23b41f1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:01.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f23b88d5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23b4149af0 0x7f23b4146030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:01.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f23b88d5700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23b4146570 0x7f23b4146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:01.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f23b88d5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b415e6f0 con 0x7f23b4149af0 2023-11-08T08:22:01.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f23b88d5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b4143ce0 con 0x7f239c004400 2023-11-08T08:22:01.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f23b88d5700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b415e940 con 0x7f23b4146570 2023-11-08T08:22:01.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f23b2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23b4149af0 0x7f23b4146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:01.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f23b2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23b4149af0 0x7f23b4146030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56492/0 (socket says 172.21.15.40:56492) 2023-11-08T08:22:01.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f23b2ffd700 1 -- 172.21.15.40:0/2117368544 learned_addr learned my addr 172.21.15.40:0/2117368544 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:01.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f23b37fe700 1 --2- 172.21.15.40:0/2117368544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f239c004400 0x7f23b41f1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:01.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f23b3fff700 1 --2- 172.21.15.40:0/2117368544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23b4146570 0x7f23b4146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:01.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f23b2ffd700 1 -- 172.21.15.40:0/2117368544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23b4146570 msgr2=0x7f23b4146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:01.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f23b2ffd700 1 --2- 172.21.15.40:0/2117368544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23b4146570 0x7f23b4146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f23b2ffd700 1 -- 172.21.15.40:0/2117368544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f239c004400 msgr2=0x7f23b41f1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:01.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f23b2ffd700 1 --2- 172.21.15.40:0/2117368544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f239c004400 0x7f23b41f1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f23b2ffd700 1 -- 172.21.15.40:0/2117368544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b40d2450 con 0x7f23b4149af0 2023-11-08T08:22:01.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f23b37fe700 1 --2- 172.21.15.40:0/2117368544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f239c004400 0x7f23b41f1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:22:01.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f23b2ffd700 1 --2- 172.21.15.40:0/2117368544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23b4149af0 0x7f23b4146030 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009600 tx=0x7f23a80047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:01.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f239bfff700 1 -- 172.21.15.40:0/2117368544 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8004030 con 0x7f23b4149af0 2023-11-08T08:22:01.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f239bfff700 1 -- 172.21.15.40:0/2117368544 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a8005c60 con 0x7f23b4149af0 2023-11-08T08:22:01.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f239bfff700 1 -- 172.21.15.40:0/2117368544 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a800e6c0 con 0x7f23b4149af0 2023-11-08T08:22:01.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f23b88d5700 1 -- 172.21.15.40:0/2117368544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b406cc50 con 0x7f23b4149af0 2023-11-08T08:22:01.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.075+0000 7f23b88d5700 1 -- 172.21.15.40:0/2117368544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b40d2730 con 0x7f23b4149af0 2023-11-08T08:22:01.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.079+0000 7f23b88d5700 1 -- 172.21.15.40:0/2117368544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b4073d20 con 0x7f23b4149af0 2023-11-08T08:22:01.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.079+0000 7f239bfff700 1 -- 172.21.15.40:0/2117368544 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f23a802d070 con 0x7f23b4149af0 2023-11-08T08:22:01.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.079+0000 7f239bfff700 1 --2- 172.21.15.40:0/2117368544 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f23a0064f10 0x7f23a00673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:01.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.079+0000 7f239bfff700 1 -- 172.21.15.40:0/2117368544 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f23a8000e40 con 0x7f23b4149af0 2023-11-08T08:22:01.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.079+0000 7f23b37fe700 1 --2- 172.21.15.40:0/2117368544 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f23a0064f10 0x7f23a00673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:01.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.083+0000 7f23b37fe700 1 --2- 172.21.15.40:0/2117368544 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f23a0064f10 0x7f23a00673c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f23a40144c0 tx=0x7f23a4009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:01.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.087+0000 7f239bfff700 1 -- 172.21.15.40:0/2117368544 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f23a8051590 con 0x7f23b4149af0 2023-11-08T08:22:01.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.263+0000 7f23b88d5700 1 -- 172.21.15.40:0/2117368544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f23b40d0150 con 0x7f23b4149af0 2023-11-08T08:22:01.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.263+0000 7f239bfff700 1 -- 172.21.15.40:0/2117368544 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7f23a80170a0 con 0x7f23b4149af0 2023-11-08T08:22:01.271 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-11-08T08:22:01.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.267+0000 7f23b88d5700 1 -- 172.21.15.40:0/2117368544 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f23a0064f10 msgr2=0x7f23a00673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:01.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.267+0000 7f23b88d5700 1 --2- 172.21.15.40:0/2117368544 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f23a0064f10 0x7f23a00673c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f23a40144c0 tx=0x7f23a4009240 comp rx=0 tx=0).stop 2023-11-08T08:22:01.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.267+0000 7f23b88d5700 1 -- 172.21.15.40:0/2117368544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23b4149af0 msgr2=0x7f23b4146030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:01.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.267+0000 7f23b88d5700 1 --2- 172.21.15.40:0/2117368544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23b4149af0 0x7f23b4146030 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009600 tx=0x7f23a80047e0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.267+0000 7f23b88d5700 1 -- 172.21.15.40:0/2117368544 shutdown_connections 2023-11-08T08:22:01.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.267+0000 7f23b88d5700 1 --2- 172.21.15.40:0/2117368544 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f23a0064f10 0x7f23a00673c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.267+0000 7f23b88d5700 1 --2- 172.21.15.40:0/2117368544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23b4146570 0x7f23b4146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.267+0000 7f23b88d5700 1 --2- 172.21.15.40:0/2117368544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23b4149af0 0x7f23b4146030 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.267+0000 7f23b88d5700 1 --2- 172.21.15.40:0/2117368544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f239c004400 0x7f23b41f1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.267+0000 7f23b88d5700 1 -- 172.21.15.40:0/2117368544 >> 172.21.15.40:0/2117368544 conn(0x7f23b40d3680 msgr2=0x7f23b40c8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:01.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.267+0000 7f23b88d5700 1 -- 172.21.15.40:0/2117368544 shutdown_connections 2023-11-08T08:22:01.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.267+0000 7f23b88d5700 1 -- 172.21.15.40:0/2117368544 wait complete. 2023-11-08T08:22:01.300 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T08:22:01.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.487+0000 7f2cd9956700 1 Processor -- start 2023-11-08T08:22:01.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.491+0000 7f2cd9956700 1 -- start start 2023-11-08T08:22:01.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.491+0000 7f2cd9956700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2cd41534a0 0x7f2cd415d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:01.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.491+0000 7f2cd9956700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2cd4160920 0x7f2cd40b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:01.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.491+0000 7f2cd9956700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd415d080 con 0x7f2cd40b6030 2023-11-08T08:22:01.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.491+0000 7f2cd9956700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd4153870 con 0x7f2cd4160920 2023-11-08T08:22:01.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.491+0000 7f2cd9956700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd40b6430 con 0x7f2cd41534a0 2023-11-08T08:22:01.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.491+0000 7f2ccbfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2cd4160920 0x7f2cd40b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:01.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.491+0000 7f2ccbfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2cd4160920 0x7f2cd40b5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56500/0 (socket says 172.21.15.40:56500) 2023-11-08T08:22:01.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.491+0000 7f2ccbfff700 1 -- 172.21.15.40:0/2088658058 learned_addr learned my addr 172.21.15.40:0/2088658058 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:01.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.491+0000 7f2cd2ffd700 1 --2- 172.21.15.40:0/2088658058 >> v2:172.21.15.169:3300/0 conn(0x7f2cd41534a0 0x7f2cd415d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:01.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.491+0000 7f2ccbfff700 1 -- 172.21.15.40:0/2088658058 >> v2:172.21.15.169:3300/0 conn(0x7f2cd41534a0 msgr2=0x7f2cd415d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:01.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.491+0000 7f2ccbfff700 1 --2- 172.21.15.40:0/2088658058 >> v2:172.21.15.169:3300/0 conn(0x7f2cd41534a0 0x7f2cd415d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.491+0000 7f2ccbfff700 1 -- 172.21.15.40:0/2088658058 >> v1:172.21.15.169:6789/0 conn(0x7f2cd40b6030 legacy=0x7f2cd4070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:01.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.491+0000 7f2ccbfff700 1 -- 172.21.15.40:0/2088658058 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd40b53c0 con 0x7f2cd4160920 2023-11-08T08:22:01.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.491+0000 7f2ccbfff700 1 --2- 172.21.15.40:0/2088658058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd4160920 0x7f2cd40b5af0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2cd406ff60 tx=0x7f2cbc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b827098db52f3722 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:01.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.495+0000 7f2cd27fc700 1 -- 172.21.15.40:0/2088658058 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc00b8a0 con 0x7f2cd4160920 2023-11-08T08:22:01.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.495+0000 7f2cd27fc700 1 -- 172.21.15.40:0/2088658058 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cbc00ba00 con 0x7f2cd4160920 2023-11-08T08:22:01.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.495+0000 7f2cd27fc700 1 -- 172.21.15.40:0/2088658058 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc00bd90 con 0x7f2cd4160920 2023-11-08T08:22:01.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.495+0000 7f2cd9956700 1 -- 172.21.15.40:0/2088658058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd4160920 msgr2=0x7f2cd40b5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:01.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.495+0000 7f2cd9956700 1 --2- 172.21.15.40:0/2088658058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd4160920 0x7f2cd40b5af0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2cd406ff60 tx=0x7f2cbc009bf0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.495+0000 7f2cd9956700 1 -- 172.21.15.40:0/2088658058 shutdown_connections 2023-11-08T08:22:01.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.495+0000 7f2cd9956700 1 --2- 172.21.15.40:0/2088658058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd4160920 0x7f2cd40b5af0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.495+0000 7f2cd9956700 1 --2- 172.21.15.40:0/2088658058 >> v2:172.21.15.169:3300/0 conn(0x7f2cd41534a0 0x7f2cd415d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.495+0000 7f2cd9956700 1 -- 172.21.15.40:0/2088658058 >> 172.21.15.40:0/2088658058 conn(0x7f2cd406bfc0 msgr2=0x7f2cd406c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:01.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.495+0000 7f2cd9956700 1 -- 172.21.15.40:0/2088658058 shutdown_connections 2023-11-08T08:22:01.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.495+0000 7f2cd9956700 1 -- 172.21.15.40:0/2088658058 wait complete. 2023-11-08T08:22:01.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.499+0000 7f2cd9956700 1 Processor -- start 2023-11-08T08:22:01.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.499+0000 7f2cd9956700 1 -- start start 2023-11-08T08:22:01.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.499+0000 7f2cd9956700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd40b6030 0x7f2cd41519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:01.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.499+0000 7f2cd9956700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cd41534a0 0x7f2cd40b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:01.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.499+0000 7f2cd9956700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd4160920 0x7f2cd40b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:01.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.499+0000 7f2cd9956700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd4153b70 con 0x7f2cd4160920 2023-11-08T08:22:01.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.499+0000 7f2cd9956700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd4153e50 con 0x7f2cd40b6030 2023-11-08T08:22:01.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.499+0000 7f2cd9956700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd4154130 con 0x7f2cd41534a0 2023-11-08T08:22:01.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.499+0000 7f2cd37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd4160920 0x7f2cd40b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:01.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.499+0000 7f2cd37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd4160920 0x7f2cd40b45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56516/0 (socket says 172.21.15.40:56516) 2023-11-08T08:22:01.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.499+0000 7f2cd37fe700 1 -- 172.21.15.40:0/626781524 learned_addr learned my addr 172.21.15.40:0/626781524 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:01.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.499+0000 7f2ccbfff700 1 --2- 172.21.15.40:0/626781524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cd41534a0 0x7f2cd40b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:01.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.499+0000 7f2cd2ffd700 1 --2- 172.21.15.40:0/626781524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd40b6030 0x7f2cd41519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:01.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.499+0000 7f2cd37fe700 1 -- 172.21.15.40:0/626781524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cd41534a0 msgr2=0x7f2cd40b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:01.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.499+0000 7f2cd37fe700 1 --2- 172.21.15.40:0/626781524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cd41534a0 0x7f2cd40b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.499+0000 7f2cd37fe700 1 -- 172.21.15.40:0/626781524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd40b6030 msgr2=0x7f2cd41519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:01.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.499+0000 7f2cd37fe700 1 --2- 172.21.15.40:0/626781524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd40b6030 0x7f2cd41519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.499+0000 7f2cd37fe700 1 -- 172.21.15.40:0/626781524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc009570 con 0x7f2cd4160920 2023-11-08T08:22:01.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.499+0000 7f2cd2ffd700 1 --2- 172.21.15.40:0/626781524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd40b6030 0x7f2cd41519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:22:01.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.499+0000 7f2cd37fe700 1 --2- 172.21.15.40:0/626781524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd4160920 0x7f2cd40b45a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00cc50 tx=0x7f2ccc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:01.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.503+0000 7f2cd0ff9700 1 -- 172.21.15.40:0/626781524 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc003c20 con 0x7f2cd4160920 2023-11-08T08:22:01.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.503+0000 7f2cd0ff9700 1 -- 172.21.15.40:0/626781524 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc003d80 con 0x7f2cd4160920 2023-11-08T08:22:01.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.503+0000 7f2cd9956700 1 -- 172.21.15.40:0/626781524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd415ca30 con 0x7f2cd4160920 2023-11-08T08:22:01.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.503+0000 7f2cd0ff9700 1 -- 172.21.15.40:0/626781524 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc00ab60 con 0x7f2cd4160920 2023-11-08T08:22:01.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.503+0000 7f2cd9956700 1 -- 172.21.15.40:0/626781524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd40b56a0 con 0x7f2cd4160920 2023-11-08T08:22:01.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.503+0000 7f2cd9956700 1 -- 172.21.15.40:0/626781524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd415cd10 con 0x7f2cd4160920 2023-11-08T08:22:01.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.503+0000 7f2cd0ff9700 1 -- 172.21.15.40:0/626781524 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f2ccc010050 con 0x7f2cd4160920 2023-11-08T08:22:01.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.507+0000 7f2cd0ff9700 1 --2- 172.21.15.40:0/626781524 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2cb4064ee0 0x7f2cb4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:01.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.507+0000 7f2cd0ff9700 1 -- 172.21.15.40:0/626781524 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2ccc081ed0 con 0x7f2cd4160920 2023-11-08T08:22:01.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.507+0000 7f2cd2ffd700 1 --2- 172.21.15.40:0/626781524 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2cb4064ee0 0x7f2cb4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:01.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.507+0000 7f2cd2ffd700 1 --2- 172.21.15.40:0/626781524 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2cb4064ee0 0x7f2cb4067390 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009a10 tx=0x7f2cc4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:01.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.511+0000 7f2cd0ff9700 1 -- 172.21.15.40:0/626781524 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f2ccc04ff30 con 0x7f2cd4160920 2023-11-08T08:22:01.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.695+0000 7f2cd9956700 1 -- 172.21.15.40:0/626781524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2cd40726c0 con 0x7f2cd4160920 2023-11-08T08:22:01.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.699+0000 7f2cd0ff9700 1 -- 172.21.15.40:0/626781524 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v57) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2ccc053550 con 0x7f2cd4160920 2023-11-08T08:22:01.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.699+0000 7f2cd9956700 1 -- 172.21.15.40:0/626781524 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2cb4064ee0 msgr2=0x7f2cb4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:01.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.699+0000 7f2cd9956700 1 --2- 172.21.15.40:0/626781524 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2cb4064ee0 0x7f2cb4067390 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009a10 tx=0x7f2cc4008040 comp rx=0 tx=0).stop 2023-11-08T08:22:01.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.699+0000 7f2cd9956700 1 -- 172.21.15.40:0/626781524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd4160920 msgr2=0x7f2cd40b45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:01.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.699+0000 7f2cd9956700 1 --2- 172.21.15.40:0/626781524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd4160920 0x7f2cd40b45a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00cc50 tx=0x7f2ccc00cf60 comp rx=0 tx=0).stop 2023-11-08T08:22:01.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.699+0000 7f2cd9956700 1 -- 172.21.15.40:0/626781524 shutdown_connections 2023-11-08T08:22:01.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.699+0000 7f2cd9956700 1 --2- 172.21.15.40:0/626781524 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2cb4064ee0 0x7f2cb4067390 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.699+0000 7f2cd9956700 1 --2- 172.21.15.40:0/626781524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2cd4160920 0x7f2cd40b45a0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.699+0000 7f2cd9956700 1 --2- 172.21.15.40:0/626781524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cd41534a0 0x7f2cd40b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.699+0000 7f2cd9956700 1 --2- 172.21.15.40:0/626781524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2cd40b6030 0x7f2cd41519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:01.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.699+0000 7f2cd9956700 1 -- 172.21.15.40:0/626781524 >> 172.21.15.40:0/626781524 conn(0x7f2cd406bfc0 msgr2=0x7f2cd415fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:01.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.699+0000 7f2cd9956700 1 -- 172.21.15.40:0/626781524 shutdown_connections 2023-11-08T08:22:01.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:01.699+0000 7f2cd9956700 1 -- 172.21.15.40:0/626781524 wait complete. 2023-11-08T08:22:01.711 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-11-08T08:22:01.731 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-08T08:22:01.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:22:01.732 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-08T08:22:01.804 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-11-08T08:22:01.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:22:01.805 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T08:22:01.830 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:22:01.830 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:22:01.843 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352316k,nr_inodes=4088079,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1804 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T08:22:01.843 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:22:01.844 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:22:01.888 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16355412k,nr_inodes=4088853,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6645 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T08:22:01.888 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:22:01.888 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:22:01.909 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352316k,nr_inodes=4088079,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1804 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T08:22:01.910 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:22:02.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.103+0000 7fd57759e700 1 Processor -- start 2023-11-08T08:22:02.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.103+0000 7fd57759e700 1 -- start start 2023-11-08T08:22:02.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.107+0000 7fd57759e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd578070790 0x7fd578167110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:02.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.107+0000 7fd57759e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5780adf90 con 0x7fd5780cdeb0 2023-11-08T08:22:02.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.107+0000 7fd57759e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5780ce370 con 0x7fd57815c1a0 2023-11-08T08:22:02.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.107+0000 7fd57759e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd578069bf0 con 0x7fd578070790 2023-11-08T08:22:02.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.107+0000 7fd575d9b700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd5780cdeb0 0x7fd578070080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41866/0 (socket says 172.21.15.40:41866) 2023-11-08T08:22:02.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.107+0000 7fd575d9b700 1 -- 172.21.15.40:0/2359111227 learned_addr learned my addr 172.21.15.40:0/2359111227 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:02.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.107+0000 7fd576d9d700 1 --2- 172.21.15.40:0/2359111227 >> v2:172.21.15.40:3300/0 conn(0x7fd578070790 0x7fd578167110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:02.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.107+0000 7fd57559a700 1 -- 172.21.15.40:0/2359111227 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4170901221 0 0) 0x7fd5780adf90 con 0x7fd5780cdeb0 2023-11-08T08:22:02.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.107+0000 7fd57559a700 1 -- 172.21.15.40:0/2359111227 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5600035d0 con 0x7fd5780cdeb0 2023-11-08T08:22:02.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.107+0000 7fd57559a700 1 -- 172.21.15.40:0/2359111227 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1118712786 0 0) 0x7fd5780ce370 con 0x7fd57815c1a0 2023-11-08T08:22:02.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.107+0000 7fd57559a700 1 -- 172.21.15.40:0/2359111227 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5780adf90 con 0x7fd57815c1a0 2023-11-08T08:22:02.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.107+0000 7fd57559a700 1 -- 172.21.15.40:0/2359111227 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1155001896 0 0) 0x7fd5600035d0 con 0x7fd5780cdeb0 2023-11-08T08:22:02.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.107+0000 7fd576d9d700 1 -- 172.21.15.40:0/2359111227 >> v1:172.21.15.169:6789/0 conn(0x7fd57815c1a0 legacy=0x7fd57806c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:02.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.107+0000 7fd576d9d700 1 -- 172.21.15.40:0/2359111227 >> v1:172.21.15.40:6789/0 conn(0x7fd5780cdeb0 legacy=0x7fd578070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:02.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.107+0000 7fd576d9d700 1 -- 172.21.15.40:0/2359111227 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5780cc4a0 con 0x7fd578070790 2023-11-08T08:22:02.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.111+0000 7fd576d9d700 1 --2- 172.21.15.40:0/2359111227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd578070790 0x7fd578167110 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd5780ae180 tx=0x7fd57000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7636074f515d8cfb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:02.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.111+0000 7fd57559a700 1 -- 172.21.15.40:0/2359111227 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57000e070 con 0x7fd578070790 2023-11-08T08:22:02.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.111+0000 7fd57559a700 1 -- 172.21.15.40:0/2359111227 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd570003a40 con 0x7fd578070790 2023-11-08T08:22:02.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.111+0000 7fd57559a700 1 -- 172.21.15.40:0/2359111227 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c001670 con 0x7fd578070790 2023-11-08T08:22:02.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.111+0000 7fd57759e700 1 -- 172.21.15.40:0/2359111227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd578070790 msgr2=0x7fd578167110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:02.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.111+0000 7fd57759e700 1 --2- 172.21.15.40:0/2359111227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd578070790 0x7fd578167110 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd5780ae180 tx=0x7fd57000b330 comp rx=0 tx=0).stop 2023-11-08T08:22:02.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.111+0000 7fd57759e700 1 -- 172.21.15.40:0/2359111227 shutdown_connections 2023-11-08T08:22:02.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.111+0000 7fd57759e700 1 --2- 172.21.15.40:0/2359111227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd578070790 0x7fd578167110 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:02.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.111+0000 7fd57759e700 1 -- 172.21.15.40:0/2359111227 >> 172.21.15.40:0/2359111227 conn(0x7fd5780b0f80 msgr2=0x7fd5780ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:02.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.111+0000 7fd57759e700 1 -- 172.21.15.40:0/2359111227 shutdown_connections 2023-11-08T08:22:02.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.111+0000 7fd57759e700 1 -- 172.21.15.40:0/2359111227 wait complete. 2023-11-08T08:22:02.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.115+0000 7fd57759e700 1 Processor -- start 2023-11-08T08:22:02.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.115+0000 7fd57759e700 1 -- start start 2023-11-08T08:22:02.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.115+0000 7fd57759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd578070790 0x7fd5780c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:02.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.115+0000 7fd57759e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5780cdeb0 0x7fd5781566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:02.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.115+0000 7fd57759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd57815c1a0 0x7fd578154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:02.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.115+0000 7fd57759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd578159790 con 0x7fd578070790 2023-11-08T08:22:02.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.115+0000 7fd57759e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780c6930 con 0x7fd5780cdeb0 2023-11-08T08:22:02.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.115+0000 7fd57759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd578157160 con 0x7fd57815c1a0 2023-11-08T08:22:02.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.115+0000 7fd57659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd578070790 0x7fd5780c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:02.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.115+0000 7fd57659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd578070790 0x7fd5780c6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56526/0 (socket says 172.21.15.40:56526) 2023-11-08T08:22:02.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.115+0000 7fd57659c700 1 -- 172.21.15.40:0/2044808480 learned_addr learned my addr 172.21.15.40:0/2044808480 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:02.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.119+0000 7fd575d9b700 1 --2- 172.21.15.40:0/2044808480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5780cdeb0 0x7fd5781566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:02.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.119+0000 7fd576d9d700 1 --2- 172.21.15.40:0/2044808480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd57815c1a0 0x7fd578154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:02.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.119+0000 7fd57659c700 1 -- 172.21.15.40:0/2044808480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd57815c1a0 msgr2=0x7fd578154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:02.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.119+0000 7fd57659c700 1 --2- 172.21.15.40:0/2044808480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd57815c1a0 0x7fd578154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:02.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.119+0000 7fd57659c700 1 -- 172.21.15.40:0/2044808480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5780cdeb0 msgr2=0x7fd5781566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:02.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.119+0000 7fd57659c700 1 --2- 172.21.15.40:0/2044808480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5780cdeb0 0x7fd5781566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:02.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.119+0000 7fd57659c700 1 -- 172.21.15.40:0/2044808480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd570009a70 con 0x7fd578070790 2023-11-08T08:22:02.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.119+0000 7fd575d9b700 1 --2- 172.21.15.40:0/2044808480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5780cdeb0 0x7fd5781566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:22:02.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.119+0000 7fd57659c700 1 --2- 172.21.15.40:0/2044808480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd578070790 0x7fd5780c6230 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd56800c490 tx=0x7fd56800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:02.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.119+0000 7fd566ffd700 1 -- 172.21.15.40:0/2044808480 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5680052d0 con 0x7fd578070790 2023-11-08T08:22:02.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.119+0000 7fd566ffd700 1 -- 172.21.15.40:0/2044808480 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd568005450 con 0x7fd578070790 2023-11-08T08:22:02.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.119+0000 7fd57759e700 1 -- 172.21.15.40:0/2044808480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57815c570 con 0x7fd578070790 2023-11-08T08:22:02.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.119+0000 7fd566ffd700 1 -- 172.21.15.40:0/2044808480 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5680038e0 con 0x7fd578070790 2023-11-08T08:22:02.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.119+0000 7fd57759e700 1 -- 172.21.15.40:0/2044808480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5780cc780 con 0x7fd578070790 2023-11-08T08:22:02.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.119+0000 7fd57759e700 1 -- 172.21.15.40:0/2044808480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57806b020 con 0x7fd578070790 2023-11-08T08:22:02.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.123+0000 7fd566ffd700 1 -- 172.21.15.40:0/2044808480 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd568019030 con 0x7fd578070790 2023-11-08T08:22:02.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.123+0000 7fd566ffd700 1 --2- 172.21.15.40:0/2044808480 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd558064f10 0x7fd5580673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:02.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.123+0000 7fd566ffd700 1 -- 172.21.15.40:0/2044808480 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd568081440 con 0x7fd578070790 2023-11-08T08:22:02.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.123+0000 7fd575d9b700 1 --2- 172.21.15.40:0/2044808480 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd558064f10 0x7fd5580673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:02.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.127+0000 7fd575d9b700 1 --2- 172.21.15.40:0/2044808480 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd558064f10 0x7fd5580673c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd578160440 tx=0x7fd56c001c80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:02.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.131+0000 7fd566ffd700 1 -- 172.21.15.40:0/2044808480 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd56804f3b0 con 0x7fd578070790 2023-11-08T08:22:02.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.343+0000 7fd57759e700 1 -- 172.21.15.40:0/2044808480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd57804e1c0 con 0x7fd578070790 2023-11-08T08:22:02.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.343+0000 7fd566ffd700 1 -- 172.21.15.40:0/2044808480 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4651 (secure 0 0 0) 0x7fd5680529d0 con 0x7fd578070790 2023-11-08T08:22:02.351 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:22:02.351 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1549516710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1549516710},{"type":"v1","addr":"172.21.15.169:6837","nonce":1549516710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5728,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/912870125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":912870125},{"type":"v1","addr":"172.21.15.169:6835","nonce":912870125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5772,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1045606427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1045606427},{"type":"v1","addr":"172.21.15.40:6834","nonce":1045606427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:21:17.730002+0000","modified":"2023-11-08T08:21:20.752976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2471311394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2471311394},{"type":"v1","addr":"172.21.15.40:6835","nonce":2471311394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T08:22:02.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.347+0000 7fd57759e700 1 -- 172.21.15.40:0/2044808480 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd558064f10 msgr2=0x7fd5580673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:02.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.347+0000 7fd57759e700 1 --2- 172.21.15.40:0/2044808480 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd558064f10 0x7fd5580673c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd578160440 tx=0x7fd56c001c80 comp rx=0 tx=0).stop 2023-11-08T08:22:02.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.347+0000 7fd57759e700 1 -- 172.21.15.40:0/2044808480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd578070790 msgr2=0x7fd5780c6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:02.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.347+0000 7fd57759e700 1 --2- 172.21.15.40:0/2044808480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd578070790 0x7fd5780c6230 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd56800c490 tx=0x7fd56800c850 comp rx=0 tx=0).stop 2023-11-08T08:22:02.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.347+0000 7fd57759e700 1 -- 172.21.15.40:0/2044808480 shutdown_connections 2023-11-08T08:22:02.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.347+0000 7fd57759e700 1 --2- 172.21.15.40:0/2044808480 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd558064f10 0x7fd5580673c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:02.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.347+0000 7fd57759e700 1 --2- 172.21.15.40:0/2044808480 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd57815c1a0 0x7fd578154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:02.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.347+0000 7fd57759e700 1 --2- 172.21.15.40:0/2044808480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5780cdeb0 0x7fd5781566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:02.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.347+0000 7fd57759e700 1 --2- 172.21.15.40:0/2044808480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd578070790 0x7fd5780c6230 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:02.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.347+0000 7fd57759e700 1 -- 172.21.15.40:0/2044808480 >> 172.21.15.40:0/2044808480 conn(0x7fd5780b0f80 msgr2=0x7fd578158b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:02.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.347+0000 7fd57759e700 1 -- 172.21.15.40:0/2044808480 shutdown_connections 2023-11-08T08:22:02.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.347+0000 7fd57759e700 1 -- 172.21.15.40:0/2044808480 wait complete. 2023-11-08T08:22:02.356 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 25 2023-11-08T08:22:02.375 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:22:02.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.547+0000 7fa84759e700 1 Processor -- start 2023-11-08T08:22:02.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.547+0000 7fa84759e700 1 -- start start 2023-11-08T08:22:02.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.551+0000 7fa84759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa848071aa0 0x7fa84806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:02.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.551+0000 7fa84759e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84806c600 con 0x7fa84815c640 2023-11-08T08:22:02.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.551+0000 7fa84759e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8480722f0 con 0x7fa848069e00 2023-11-08T08:22:02.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.551+0000 7fa84759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84806cef0 con 0x7fa848071aa0 2023-11-08T08:22:02.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.551+0000 7fa84659c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa84815c640 0x7fa8480d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41874/0 (socket says 172.21.15.40:41874) 2023-11-08T08:22:02.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.551+0000 7fa84659c700 1 -- 172.21.15.40:0/2774957711 learned_addr learned my addr 172.21.15.40:0/2774957711 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:02.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.551+0000 7fa83dd9b700 1 --2- 172.21.15.40:0/2774957711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa848071aa0 0x7fa84806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:02.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.551+0000 7fa845d9b700 1 -- 172.21.15.40:0/2774957711 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873723819 0 0) 0x7fa84806c600 con 0x7fa84815c640 2023-11-08T08:22:02.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.551+0000 7fa845d9b700 1 -- 172.21.15.40:0/2774957711 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8340035d0 con 0x7fa84815c640 2023-11-08T08:22:02.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.551+0000 7fa83dd9b700 1 -- 172.21.15.40:0/2774957711 >> v1:172.21.15.169:6789/0 conn(0x7fa848069e00 legacy=0x7fa8480d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:02.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.551+0000 7fa845d9b700 1 -- 172.21.15.40:0/2774957711 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567351855 0 0) 0x7fa8480722f0 con 0x7fa848069e00 2023-11-08T08:22:02.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.551+0000 7fa83dd9b700 1 -- 172.21.15.40:0/2774957711 >> v1:172.21.15.40:6789/0 conn(0x7fa84815c640 legacy=0x7fa8480d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:02.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.551+0000 7fa83dd9b700 1 -- 172.21.15.40:0/2774957711 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84815ca10 con 0x7fa848071aa0 2023-11-08T08:22:02.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.551+0000 7fa83dd9b700 1 --2- 172.21.15.40:0/2774957711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa848071aa0 0x7fa84806d310 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa830009880 tx=0x7fa830009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e70077958357d0cd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:02.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.551+0000 7fa845d9b700 1 -- 172.21.15.40:0/2774957711 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830011830 con 0x7fa848071aa0 2023-11-08T08:22:02.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.551+0000 7fa845d9b700 1 -- 172.21.15.40:0/2774957711 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa830011990 con 0x7fa848071aa0 2023-11-08T08:22:02.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.551+0000 7fa845d9b700 1 -- 172.21.15.40:0/2774957711 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830011d20 con 0x7fa848071aa0 2023-11-08T08:22:02.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.555+0000 7fa84759e700 1 -- 172.21.15.40:0/2774957711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa848071aa0 msgr2=0x7fa84806d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:02.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.555+0000 7fa84759e700 1 --2- 172.21.15.40:0/2774957711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa848071aa0 0x7fa84806d310 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa830009880 tx=0x7fa830009b90 comp rx=0 tx=0).stop 2023-11-08T08:22:02.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.555+0000 7fa84759e700 1 -- 172.21.15.40:0/2774957711 shutdown_connections 2023-11-08T08:22:02.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.555+0000 7fa84759e700 1 --2- 172.21.15.40:0/2774957711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa848071aa0 0x7fa84806d310 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:02.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.555+0000 7fa84759e700 1 -- 172.21.15.40:0/2774957711 >> 172.21.15.40:0/2774957711 conn(0x7fa8480b6520 msgr2=0x7fa848071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:02.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.555+0000 7fa84759e700 1 -- 172.21.15.40:0/2774957711 shutdown_connections 2023-11-08T08:22:02.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.555+0000 7fa84759e700 1 -- 172.21.15.40:0/2774957711 wait complete. 2023-11-08T08:22:02.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.555+0000 7fa84759e700 1 Processor -- start 2023-11-08T08:22:02.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa84759e700 1 -- start start 2023-11-08T08:22:02.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa84759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa848069e00 0x7fa8481f1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:02.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa84759e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa848071aa0 0x7fa8480cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:02.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa84759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa84815c640 0x7fa8480cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:02.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa84759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8480d38f0 con 0x7fa84815c640 2023-11-08T08:22:02.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa84759e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa848156a10 con 0x7fa848071aa0 2023-11-08T08:22:02.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa84759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8480c9650 con 0x7fa848069e00 2023-11-08T08:22:02.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa846d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa84815c640 0x7fa8480cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:02.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa846d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa84815c640 0x7fa8480cc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56540/0 (socket says 172.21.15.40:56540) 2023-11-08T08:22:02.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa846d9d700 1 -- 172.21.15.40:0/2069459270 learned_addr learned my addr 172.21.15.40:0/2069459270 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:02.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa83dd9b700 1 --2- 172.21.15.40:0/2069459270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa848071aa0 0x7fa8480cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:02.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa84659c700 1 --2- 172.21.15.40:0/2069459270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa848069e00 0x7fa8481f1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:02.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa846d9d700 1 -- 172.21.15.40:0/2069459270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa848069e00 msgr2=0x7fa8481f1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:02.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa846d9d700 1 --2- 172.21.15.40:0/2069459270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa848069e00 0x7fa8481f1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:02.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa846d9d700 1 -- 172.21.15.40:0/2069459270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa848071aa0 msgr2=0x7fa8480cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:02.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa846d9d700 1 --2- 172.21.15.40:0/2069459270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa848071aa0 0x7fa8480cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:02.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa846d9d700 1 -- 172.21.15.40:0/2069459270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8300094b0 con 0x7fa84815c640 2023-11-08T08:22:02.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa846d9d700 1 --2- 172.21.15.40:0/2069459270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa84815c640 0x7fa8480cc190 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa84000cc40 tx=0x7fa84000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:02.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa83f7fe700 1 -- 172.21.15.40:0/2069459270 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa840003e50 con 0x7fa84815c640 2023-11-08T08:22:02.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa83f7fe700 1 -- 172.21.15.40:0/2069459270 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84000a670 con 0x7fa84815c640 2023-11-08T08:22:02.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa84759e700 1 -- 172.21.15.40:0/2069459270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84806c830 con 0x7fa84815c640 2023-11-08T08:22:02.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.559+0000 7fa83f7fe700 1 -- 172.21.15.40:0/2069459270 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84000aa20 con 0x7fa84815c640 2023-11-08T08:22:02.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.563+0000 7fa84759e700 1 -- 172.21.15.40:0/2069459270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa848072010 con 0x7fa84815c640 2023-11-08T08:22:02.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.563+0000 7fa84759e700 1 -- 172.21.15.40:0/2069459270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84806cb10 con 0x7fa84815c640 2023-11-08T08:22:02.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.563+0000 7fa83f7fe700 1 -- 172.21.15.40:0/2069459270 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa840019030 con 0x7fa84815c640 2023-11-08T08:22:02.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.567+0000 7fa83f7fe700 1 --2- 172.21.15.40:0/2069459270 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa828064ee0 0x7fa828067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:02.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.567+0000 7fa83f7fe700 1 -- 172.21.15.40:0/2069459270 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa840081920 con 0x7fa84815c640 2023-11-08T08:22:02.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.567+0000 7fa84659c700 1 --2- 172.21.15.40:0/2069459270 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa828064ee0 0x7fa828067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:02.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.567+0000 7fa84659c700 1 --2- 172.21.15.40:0/2069459270 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa828064ee0 0x7fa828067390 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa838007fd0 tx=0x7fa838009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:02.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.571+0000 7fa83f7fe700 1 -- 172.21.15.40:0/2069459270 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa84004f890 con 0x7fa84815c640 2023-11-08T08:22:02.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.743+0000 7fa84759e700 1 -- 172.21.15.40:0/2069459270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8480b2510 con 0x7fa84815c640 2023-11-08T08:22:02.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.747+0000 7fa83f7fe700 1 -- 172.21.15.40:0/2069459270 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+18084 (secure 0 0 0) 0x7fa840052eb0 con 0x7fa84815c640 2023-11-08T08:22:02.756 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:22:02.757 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:21:18.736039+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:21:15.464908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-08T08:21:16.755373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:22:02.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.751+0000 7fa84759e700 1 -- 172.21.15.40:0/2069459270 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa828064ee0 msgr2=0x7fa828067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:02.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.751+0000 7fa84759e700 1 --2- 172.21.15.40:0/2069459270 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa828064ee0 0x7fa828067390 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa838007fd0 tx=0x7fa838009040 comp rx=0 tx=0).stop 2023-11-08T08:22:02.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.751+0000 7fa84759e700 1 -- 172.21.15.40:0/2069459270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa84815c640 msgr2=0x7fa8480cc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:02.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.751+0000 7fa84759e700 1 --2- 172.21.15.40:0/2069459270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa84815c640 0x7fa8480cc190 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa84000cc40 tx=0x7fa84000cf50 comp rx=0 tx=0).stop 2023-11-08T08:22:02.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.751+0000 7fa84759e700 1 -- 172.21.15.40:0/2069459270 shutdown_connections 2023-11-08T08:22:02.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.751+0000 7fa84759e700 1 --2- 172.21.15.40:0/2069459270 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa828064ee0 0x7fa828067390 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:02.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.751+0000 7fa84759e700 1 --2- 172.21.15.40:0/2069459270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa84815c640 0x7fa8480cc190 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:02.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.751+0000 7fa84759e700 1 --2- 172.21.15.40:0/2069459270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa848071aa0 0x7fa8480cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:02.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.751+0000 7fa84759e700 1 --2- 172.21.15.40:0/2069459270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa848069e00 0x7fa8481f1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:02.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.751+0000 7fa84759e700 1 -- 172.21.15.40:0/2069459270 >> 172.21.15.40:0/2069459270 conn(0x7fa8480b6520 msgr2=0x7fa84806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:02.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.751+0000 7fa84759e700 1 -- 172.21.15.40:0/2069459270 shutdown_connections 2023-11-08T08:22:02.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.751+0000 7fa84759e700 1 -- 172.21.15.40:0/2069459270 wait complete. 2023-11-08T08:22:02.785 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T08:21:15.464908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T08:21:16.755373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T08:22:02.786 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:22:02.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.971+0000 7fc7fa918700 1 Processor -- start 2023-11-08T08:22:02.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.971+0000 7fc7fa918700 1 -- start start 2023-11-08T08:22:02.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.975+0000 7fc7fa918700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc7f40cdc50 0x7fc7f4069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:02.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.975+0000 7fc7fa918700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f415e560 con 0x7fc7f406ff10 2023-11-08T08:22:02.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.975+0000 7fc7fa918700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f406be80 con 0x7fc7f415e190 2023-11-08T08:22:02.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.975+0000 7fc7fa918700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f40ce0a0 con 0x7fc7f40cdc50 2023-11-08T08:22:02.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.975+0000 7fc7fa117700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc7f406ff10 0x7fc7f406c310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41886/0 (socket says 172.21.15.40:41886) 2023-11-08T08:22:02.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.975+0000 7fc7fa117700 1 -- 172.21.15.40:0/4102385020 learned_addr learned my addr 172.21.15.40:0/4102385020 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:02.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.975+0000 7fc7f9115700 1 --2- 172.21.15.40:0/4102385020 >> v2:172.21.15.169:3300/0 conn(0x7fc7f40cdc50 0x7fc7f4069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:02.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.975+0000 7fc7f8914700 1 -- 172.21.15.40:0/4102385020 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3468671420 0 0) 0x7fc7ec001490 con 0x7fc7f406ff10 2023-11-08T08:22:02.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.975+0000 7fc7f8914700 1 -- 172.21.15.40:0/4102385020 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7e00035d0 con 0x7fc7f406ff10 2023-11-08T08:22:02.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.975+0000 7fc7f9115700 1 -- 172.21.15.40:0/4102385020 >> v1:172.21.15.169:6789/0 conn(0x7fc7f415e190 legacy=0x7fc7f4161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:02.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.975+0000 7fc7f8914700 1 -- 172.21.15.40:0/4102385020 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307134945 0 0) 0x7fc7f406be80 con 0x7fc7f415e190 2023-11-08T08:22:02.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.975+0000 7fc7f9115700 1 -- 172.21.15.40:0/4102385020 >> v1:172.21.15.40:6789/0 conn(0x7fc7f406ff10 legacy=0x7fc7f406c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:02.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.975+0000 7fc7f9115700 1 -- 172.21.15.40:0/4102385020 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f40ce330 con 0x7fc7f40cdc50 2023-11-08T08:22:02.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.975+0000 7fc7f9115700 1 --2- 172.21.15.40:0/4102385020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7f40cdc50 0x7fc7f4069bf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009850 tx=0x7fc7e4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f54ed292be8de541 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:02.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.979+0000 7fc7f8914700 1 -- 172.21.15.40:0/4102385020 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4011830 con 0x7fc7f40cdc50 2023-11-08T08:22:02.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.979+0000 7fc7f8914700 1 -- 172.21.15.40:0/4102385020 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e4011990 con 0x7fc7f40cdc50 2023-11-08T08:22:02.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.979+0000 7fc7f8914700 1 -- 172.21.15.40:0/4102385020 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4011d40 con 0x7fc7f40cdc50 2023-11-08T08:22:02.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.979+0000 7fc7fa918700 1 -- 172.21.15.40:0/4102385020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7f40cdc50 msgr2=0x7fc7f4069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:02.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.979+0000 7fc7fa918700 1 --2- 172.21.15.40:0/4102385020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7f40cdc50 0x7fc7f4069bf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009850 tx=0x7fc7e4009b60 comp rx=0 tx=0).stop 2023-11-08T08:22:02.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.979+0000 7fc7fa918700 1 -- 172.21.15.40:0/4102385020 shutdown_connections 2023-11-08T08:22:02.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.979+0000 7fc7fa918700 1 --2- 172.21.15.40:0/4102385020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7f40cdc50 0x7fc7f4069bf0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:02.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.979+0000 7fc7fa918700 1 -- 172.21.15.40:0/4102385020 >> 172.21.15.40:0/4102385020 conn(0x7fc7f40cf280 msgr2=0x7fc7f40cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:02.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.979+0000 7fc7fa918700 1 -- 172.21.15.40:0/4102385020 shutdown_connections 2023-11-08T08:22:02.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.979+0000 7fc7fa918700 1 -- 172.21.15.40:0/4102385020 wait complete. 2023-11-08T08:22:02.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7fa918700 1 Processor -- start 2023-11-08T08:22:02.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7fa918700 1 -- start start 2023-11-08T08:22:02.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7fa918700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7f406ff10 0x7fc7f415b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:02.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7fa918700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7f40cdc50 0x7fc7f4154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:02.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7fa918700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7f415e190 0x7fc7f40b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:02.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7fa918700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f4154b60 con 0x7fc7f40cdc50 2023-11-08T08:22:02.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7fa918700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f40b23a0 con 0x7fc7f406ff10 2023-11-08T08:22:02.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7fa918700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f4156700 con 0x7fc7f415e190 2023-11-08T08:22:02.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7f9115700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7f40cdc50 0x7fc7f4154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:02.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7f9115700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7f40cdc50 0x7fc7f4154620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56552/0 (socket says 172.21.15.40:56552) 2023-11-08T08:22:02.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7f9115700 1 -- 172.21.15.40:0/3732141827 learned_addr learned my addr 172.21.15.40:0/3732141827 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:02.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7f9916700 1 --2- 172.21.15.40:0/3732141827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7f406ff10 0x7fc7f415b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:02.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7fa117700 1 --2- 172.21.15.40:0/3732141827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7f415e190 0x7fc7f40b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:02.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7f9115700 1 -- 172.21.15.40:0/3732141827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7f415e190 msgr2=0x7fc7f40b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:02.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7f9115700 1 --2- 172.21.15.40:0/3732141827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7f415e190 0x7fc7f40b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:02.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7f9115700 1 -- 172.21.15.40:0/3732141827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7f406ff10 msgr2=0x7fc7f415b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:02.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7f9115700 1 --2- 172.21.15.40:0/3732141827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7f406ff10 0x7fc7f415b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:02.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7f9115700 1 -- 172.21.15.40:0/3732141827 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e40094b0 con 0x7fc7f40cdc50 2023-11-08T08:22:02.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7fa117700 1 --2- 172.21.15.40:0/3732141827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7f415e190 0x7fc7f40b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:22:02.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7f9115700 1 --2- 172.21.15.40:0/3732141827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7f40cdc50 0x7fc7f4154620 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009eb0 tx=0x7fc7e4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:02.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7ea7fc700 1 -- 172.21.15.40:0/3732141827 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e401aba0 con 0x7fc7f40cdc50 2023-11-08T08:22:02.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7ea7fc700 1 -- 172.21.15.40:0/3732141827 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e401f070 con 0x7fc7f40cdc50 2023-11-08T08:22:02.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7fa918700 1 -- 172.21.15.40:0/3732141827 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f40afc70 con 0x7fc7f40cdc50 2023-11-08T08:22:02.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7ea7fc700 1 -- 172.21.15.40:0/3732141827 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e401b430 con 0x7fc7f40cdc50 2023-11-08T08:22:02.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.983+0000 7fc7fa918700 1 -- 172.21.15.40:0/3732141827 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f40af990 con 0x7fc7f40cdc50 2023-11-08T08:22:02.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.987+0000 7fc7fa918700 1 -- 172.21.15.40:0/3732141827 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f40aff50 con 0x7fc7f40cdc50 2023-11-08T08:22:02.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.987+0000 7fc7ea7fc700 1 -- 172.21.15.40:0/3732141827 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc7e400f050 con 0x7fc7f40cdc50 2023-11-08T08:22:02.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.991+0000 7fc7ea7fc700 1 --2- 172.21.15.40:0/3732141827 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc7d8064f30 0x7fc7d80673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:02.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.991+0000 7fc7ea7fc700 1 -- 172.21.15.40:0/3732141827 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc7e40831b0 con 0x7fc7f40cdc50 2023-11-08T08:22:02.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.991+0000 7fc7f9916700 1 --2- 172.21.15.40:0/3732141827 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc7d8064f30 0x7fc7d80673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:02.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.991+0000 7fc7f9916700 1 --2- 172.21.15.40:0/3732141827 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc7d8064f30 0x7fc7d80673e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0000f60 tx=0x7fc7f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:03.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:02.995+0000 7fc7ea7fc700 1 -- 172.21.15.40:0/3732141827 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc7e40511a0 con 0x7fc7f40cdc50 2023-11-08T08:22:03.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.167+0000 7fc7fa918700 1 -- 172.21.15.40:0/3732141827 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc7f4073d20 con 0x7fc7f40cdc50 2023-11-08T08:22:03.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.167+0000 7fc7ea7fc700 1 -- 172.21.15.40:0/3732141827 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc7e40210a0 con 0x7fc7f40cdc50 2023-11-08T08:22:03.176 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:22:03.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.171+0000 7fc7fa918700 1 -- 172.21.15.40:0/3732141827 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc7d8064f30 msgr2=0x7fc7d80673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:03.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.171+0000 7fc7fa918700 1 --2- 172.21.15.40:0/3732141827 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc7d8064f30 0x7fc7d80673e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0000f60 tx=0x7fc7f0009040 comp rx=0 tx=0).stop 2023-11-08T08:22:03.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.171+0000 7fc7fa918700 1 -- 172.21.15.40:0/3732141827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7f40cdc50 msgr2=0x7fc7f4154620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:03.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.171+0000 7fc7fa918700 1 --2- 172.21.15.40:0/3732141827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7f40cdc50 0x7fc7f4154620 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009eb0 tx=0x7fc7e4004770 comp rx=0 tx=0).stop 2023-11-08T08:22:03.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.171+0000 7fc7fa918700 1 -- 172.21.15.40:0/3732141827 shutdown_connections 2023-11-08T08:22:03.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.171+0000 7fc7fa918700 1 --2- 172.21.15.40:0/3732141827 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc7d8064f30 0x7fc7d80673e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:03.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.171+0000 7fc7fa918700 1 --2- 172.21.15.40:0/3732141827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7f415e190 0x7fc7f40b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:03.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.171+0000 7fc7fa918700 1 --2- 172.21.15.40:0/3732141827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7f40cdc50 0x7fc7f4154620 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:03.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.171+0000 7fc7fa918700 1 --2- 172.21.15.40:0/3732141827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7f406ff10 0x7fc7f415b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:03.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.171+0000 7fc7fa918700 1 -- 172.21.15.40:0/3732141827 >> 172.21.15.40:0/3732141827 conn(0x7fc7f40cf280 msgr2=0x7fc7f40c3500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:03.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.171+0000 7fc7fa918700 1 -- 172.21.15.40:0/3732141827 shutdown_connections 2023-11-08T08:22:03.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.171+0000 7fc7fa918700 1 -- 172.21.15.40:0/3732141827 wait complete. 2023-11-08T08:22:03.202 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:22:03.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.375+0000 7f0c314fa700 1 Processor -- start 2023-11-08T08:22:03.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.375+0000 7f0c314fa700 1 -- start start 2023-11-08T08:22:03.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c314fa700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c2c070a00 0x7f0c2c069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:03.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c314fa700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2c0cd360 con 0x7f0c2c0b2130 2023-11-08T08:22:03.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c314fa700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2c160190 con 0x7f0c2c15c700 2023-11-08T08:22:03.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c314fa700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c160470 con 0x7f0c2c070a00 2023-11-08T08:22:03.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2b7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0c2c0b2130 0x7f0c2c06a130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41888/0 (socket says 172.21.15.40:41888) 2023-11-08T08:22:03.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2b7fe700 1 -- 172.21.15.40:0/473646834 learned_addr learned my addr 172.21.15.40:0/473646834 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:03.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c23fff700 1 --2- 172.21.15.40:0/473646834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c2c070a00 0x7f0c2c069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:03.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2a7fc700 1 -- 172.21.15.40:0/473646834 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259017356 0 0) 0x7f0c2c0cd360 con 0x7f0c2c0b2130 2023-11-08T08:22:03.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2a7fc700 1 -- 172.21.15.40:0/473646834 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c180035d0 con 0x7f0c2c0b2130 2023-11-08T08:22:03.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2a7fc700 1 -- 172.21.15.40:0/473646834 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2052028049 0 0) 0x7f0c2c160190 con 0x7f0c2c15c700 2023-11-08T08:22:03.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2a7fc700 1 -- 172.21.15.40:0/473646834 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c2c0cd360 con 0x7f0c2c15c700 2023-11-08T08:22:03.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2a7fc700 1 -- 172.21.15.40:0/473646834 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 786695666 0 0) 0x7f0c180035d0 con 0x7f0c2c0b2130 2023-11-08T08:22:03.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2a7fc700 1 -- 172.21.15.40:0/473646834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c2c070a00 msgr2=0x7f0c2c069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2a7fc700 1 --2- 172.21.15.40:0/473646834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c2c070a00 0x7f0c2c069bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2a7fc700 1 -- 172.21.15.40:0/473646834 >> v1:172.21.15.169:6789/0 conn(0x7f0c2c15c700 legacy=0x7f0c2c0702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2a7fc700 1 -- 172.21.15.40:0/473646834 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c160750 con 0x7f0c2c0b2130 2023-11-08T08:22:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2a7fc700 1 -- 172.21.15.40:0/473646834 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f0c240019f0 con 0x7f0c2c0b2130 2023-11-08T08:22:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2a7fc700 1 -- 172.21.15.40:0/473646834 >> v1:172.21.15.40:6789/0 conn(0x7f0c2c0b2130 legacy=0x7f0c2c06a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:03.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2a7fc700 1 --2- 172.21.15.40:0/473646834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c2c070a00 0x7f0c18003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:03.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2a7fc700 1 --2- 172.21.15.40:0/473646834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c18004140 0x7f0c18004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:03.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2a7fc700 1 --2- 172.21.15.40:0/473646834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c18004db0 0x7f0c18005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:03.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2a7fc700 1 -- 172.21.15.40:0/473646834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c160470 con 0x7f0c18004db0 2023-11-08T08:22:03.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2a7fc700 1 -- 172.21.15.40:0/473646834 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1800bae0 con 0x7f0c18004140 2023-11-08T08:22:03.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2a7fc700 1 -- 172.21.15.40:0/473646834 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1800bca0 con 0x7f0c2c070a00 2023-11-08T08:22:03.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2b7fe700 1 --2- 172.21.15.40:0/473646834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c18004db0 0x7f0c18005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:03.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c23fff700 1 --2- 172.21.15.40:0/473646834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c18004140 0x7f0c18004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:03.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.379+0000 7f0c2affd700 1 --2- 172.21.15.40:0/473646834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c2c070a00 0x7f0c18003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:03.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.383+0000 7f0c2b7fe700 1 -- 172.21.15.40:0/473646834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c2c070a00 msgr2=0x7f0c18003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:03.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.383+0000 7f0c2b7fe700 1 --2- 172.21.15.40:0/473646834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c2c070a00 0x7f0c18003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:03.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.383+0000 7f0c2b7fe700 1 -- 172.21.15.40:0/473646834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c18004140 msgr2=0x7f0c18004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:03.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.383+0000 7f0c2b7fe700 1 --2- 172.21.15.40:0/473646834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c18004140 0x7f0c18004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:03.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.383+0000 7f0c2b7fe700 1 -- 172.21.15.40:0/473646834 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c180035d0 con 0x7f0c18004db0 2023-11-08T08:22:03.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.383+0000 7f0c2b7fe700 1 --2- 172.21.15.40:0/473646834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c18004db0 0x7f0c18005220 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0c24002230 tx=0x7f0c180058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432e6d142d3e0f4a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:03.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.383+0000 7f0c2a7fc700 1 -- 172.21.15.40:0/473646834 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c240072d0 con 0x7f0c18004db0 2023-11-08T08:22:03.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.383+0000 7f0c2a7fc700 1 -- 172.21.15.40:0/473646834 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c24007430 con 0x7f0c18004db0 2023-11-08T08:22:03.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.383+0000 7f0c2a7fc700 1 -- 172.21.15.40:0/473646834 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c240077a0 con 0x7f0c18004db0 2023-11-08T08:22:03.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.383+0000 7f0c314fa700 1 -- 172.21.15.40:0/473646834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c18004db0 msgr2=0x7f0c18005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:03.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.383+0000 7f0c314fa700 1 --2- 172.21.15.40:0/473646834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c18004db0 0x7f0c18005220 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0c24002230 tx=0x7f0c180058a0 comp rx=0 tx=0).stop 2023-11-08T08:22:03.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.383+0000 7f0c2b7fe700 1 -- 172.21.15.40:0/473646834 reap_dead start 2023-11-08T08:22:03.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.383+0000 7f0c314fa700 1 -- 172.21.15.40:0/473646834 shutdown_connections 2023-11-08T08:22:03.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.383+0000 7f0c314fa700 1 -- 172.21.15.40:0/473646834 >> 172.21.15.40:0/473646834 conn(0x7f0c2c0d2d10 msgr2=0x7f0c2c0d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:03.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.383+0000 7f0c314fa700 1 -- 172.21.15.40:0/473646834 shutdown_connections 2023-11-08T08:22:03.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.383+0000 7f0c314fa700 1 -- 172.21.15.40:0/473646834 wait complete. 2023-11-08T08:22:03.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.387+0000 7f0c314fa700 1 Processor -- start 2023-11-08T08:22:03.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.387+0000 7f0c314fa700 1 -- start start 2023-11-08T08:22:03.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.387+0000 7f0c314fa700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c18004db0 0x7f0c2c15eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:03.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.387+0000 7f0c314fa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c2c1567a0 0x7f0c2c15f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:03.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.387+0000 7f0c314fa700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c2c0d08c0 0x7f0c2c0d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:03.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.387+0000 7f0c314fa700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c0d1270 con 0x7f0c2c1567a0 2023-11-08T08:22:03.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.387+0000 7f0c314fa700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c151ad0 con 0x7f0c18004db0 2023-11-08T08:22:03.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.387+0000 7f0c314fa700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c1559e0 con 0x7f0c2c0d08c0 2023-11-08T08:22:03.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.387+0000 7f0c23fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c2c1567a0 0x7f0c2c15f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:03.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.387+0000 7f0c23fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c2c1567a0 0x7f0c2c15f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56578/0 (socket says 172.21.15.40:56578) 2023-11-08T08:22:03.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.387+0000 7f0c23fff700 1 -- 172.21.15.40:0/2265969335 learned_addr learned my addr 172.21.15.40:0/2265969335 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:03.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.391+0000 7f0c2affd700 1 --2- 172.21.15.40:0/2265969335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c18004db0 0x7f0c2c15eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:03.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.391+0000 7f0c2b7fe700 1 --2- 172.21.15.40:0/2265969335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c2c0d08c0 0x7f0c2c0d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:03.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.391+0000 7f0c23fff700 1 -- 172.21.15.40:0/2265969335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c2c0d08c0 msgr2=0x7f0c2c0d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.391+0000 7f0c23fff700 1 --2- 172.21.15.40:0/2265969335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c2c0d08c0 0x7f0c2c0d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.391+0000 7f0c23fff700 1 -- 172.21.15.40:0/2265969335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c18004db0 msgr2=0x7f0c2c15eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.391+0000 7f0c23fff700 1 --2- 172.21.15.40:0/2265969335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c18004db0 0x7f0c2c15eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.391+0000 7f0c23fff700 1 -- 172.21.15.40:0/2265969335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c06b530 con 0x7f0c2c1567a0 2023-11-08T08:22:03.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.391+0000 7f0c23fff700 1 --2- 172.21.15.40:0/2265969335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c2c1567a0 0x7f0c2c15f3f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c0119d0 tx=0x7f0c1c011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:03.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.391+0000 7f0c237fe700 1 -- 172.21.15.40:0/2265969335 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c009640 con 0x7f0c2c1567a0 2023-11-08T08:22:03.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.391+0000 7f0c237fe700 1 -- 172.21.15.40:0/2265969335 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1c004240 con 0x7f0c2c1567a0 2023-11-08T08:22:03.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.391+0000 7f0c237fe700 1 -- 172.21.15.40:0/2265969335 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0045d0 con 0x7f0c2c1567a0 2023-11-08T08:22:03.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.391+0000 7f0c314fa700 1 -- 172.21.15.40:0/2265969335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2c160a30 con 0x7f0c2c1567a0 2023-11-08T08:22:03.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.391+0000 7f0c314fa700 1 -- 172.21.15.40:0/2265969335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2c0b1be0 con 0x7f0c2c1567a0 2023-11-08T08:22:03.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.391+0000 7f0c314fa700 1 -- 172.21.15.40:0/2265969335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c2c0b8d00 con 0x7f0c2c1567a0 2023-11-08T08:22:03.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.395+0000 7f0c237fe700 1 -- 172.21.15.40:0/2265969335 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f0c1c025070 con 0x7f0c2c1567a0 2023-11-08T08:22:03.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.395+0000 7f0c237fe700 1 --2- 172.21.15.40:0/2265969335 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0c0c064ee0 0x7f0c0c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:03.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.395+0000 7f0c237fe700 1 -- 172.21.15.40:0/2265969335 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0c1c0824d0 con 0x7f0c2c1567a0 2023-11-08T08:22:03.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.395+0000 7f0c2affd700 1 --2- 172.21.15.40:0/2265969335 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0c0c064ee0 0x7f0c0c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:03.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.395+0000 7f0c2affd700 1 --2- 172.21.15.40:0/2265969335 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0c0c064ee0 0x7f0c0c067390 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0c14007fd0 tx=0x7f0c14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:03.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.403+0000 7f0c237fe700 1 -- 172.21.15.40:0/2265969335 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f0c1c050440 con 0x7f0c2c1567a0 2023-11-08T08:22:03.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.575+0000 7f0c314fa700 1 -- 172.21.15.40:0/2265969335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0c2c0726c0 con 0x7f0c2c1567a0 2023-11-08T08:22:03.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.575+0000 7f0c237fe700 1 -- 172.21.15.40:0/2265969335 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v57) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0c1c0190a0 con 0x7f0c2c1567a0 2023-11-08T08:22:03.584 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:22:03.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.579+0000 7f0c314fa700 1 -- 172.21.15.40:0/2265969335 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0c0c064ee0 msgr2=0x7f0c0c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:03.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.579+0000 7f0c314fa700 1 --2- 172.21.15.40:0/2265969335 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0c0c064ee0 0x7f0c0c067390 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0c14007fd0 tx=0x7f0c14009040 comp rx=0 tx=0).stop 2023-11-08T08:22:03.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.579+0000 7f0c314fa700 1 -- 172.21.15.40:0/2265969335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c2c1567a0 msgr2=0x7f0c2c15f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:03.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.579+0000 7f0c314fa700 1 --2- 172.21.15.40:0/2265969335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c2c1567a0 0x7f0c2c15f3f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c0119d0 tx=0x7f0c1c011ce0 comp rx=0 tx=0).stop 2023-11-08T08:22:03.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.579+0000 7f0c314fa700 1 -- 172.21.15.40:0/2265969335 shutdown_connections 2023-11-08T08:22:03.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.579+0000 7f0c314fa700 1 --2- 172.21.15.40:0/2265969335 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0c0c064ee0 0x7f0c0c067390 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:03.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.579+0000 7f0c314fa700 1 --2- 172.21.15.40:0/2265969335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c2c0d08c0 0x7f0c2c0d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:03.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.579+0000 7f0c314fa700 1 --2- 172.21.15.40:0/2265969335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c2c1567a0 0x7f0c2c15f3f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:03.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.579+0000 7f0c314fa700 1 --2- 172.21.15.40:0/2265969335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c18004db0 0x7f0c2c15eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:03.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.579+0000 7f0c314fa700 1 -- 172.21.15.40:0/2265969335 >> 172.21.15.40:0/2265969335 conn(0x7f0c2c0d2d10 msgr2=0x7f0c2c0702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:03.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.579+0000 7f0c314fa700 1 -- 172.21.15.40:0/2265969335 shutdown_connections 2023-11-08T08:22:03.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.579+0000 7f0c314fa700 1 -- 172.21.15.40:0/2265969335 wait complete. 2023-11-08T08:22:03.609 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T08:22:03.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.787+0000 7f669359e700 1 Processor -- start 2023-11-08T08:22:03.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.787+0000 7f669359e700 1 -- start start 2023-11-08T08:22:03.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.791+0000 7f669359e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f66940b3d00 0x7f669406a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:03.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.791+0000 7f669359e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669415c520 con 0x7f669415c150 2023-11-08T08:22:03.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.791+0000 7f669359e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669406cc50 con 0x7f6694070620 2023-11-08T08:22:03.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.791+0000 7f669359e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66940d2450 con 0x7f66940b3d00 2023-11-08T08:22:03.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.791+0000 7f669259c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f669415c150 0x7f669406ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41902/0 (socket says 172.21.15.40:41902) 2023-11-08T08:22:03.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.791+0000 7f669259c700 1 -- 172.21.15.40:0/3677903581 learned_addr learned my addr 172.21.15.40:0/3677903581 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:03.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.791+0000 7f6692d9d700 1 --2- 172.21.15.40:0/3677903581 >> v2:172.21.15.169:3300/0 conn(0x7f66940b3d00 0x7f669406a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:03.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.795+0000 7f669159a700 1 -- 172.21.15.40:0/3677903581 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581029441 0 0) 0x7f669415c520 con 0x7f669415c150 2023-11-08T08:22:03.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.795+0000 7f669159a700 1 -- 172.21.15.40:0/3677903581 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f667c0035d0 con 0x7f669415c150 2023-11-08T08:22:03.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.795+0000 7f6692d9d700 1 -- 172.21.15.40:0/3677903581 >> v1:172.21.15.169:6789/0 conn(0x7f6694070620 legacy=0x7f6694069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:03.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.795+0000 7f6692d9d700 1 -- 172.21.15.40:0/3677903581 >> v1:172.21.15.40:6789/0 conn(0x7f669415c150 legacy=0x7f669406ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:03.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.795+0000 7f6692d9d700 1 -- 172.21.15.40:0/3677903581 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66940d2730 con 0x7f66940b3d00 2023-11-08T08:22:03.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.795+0000 7f6692d9d700 1 --2- 172.21.15.40:0/3677903581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66940b3d00 0x7f669406a300 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f668c009d90 tx=0x7f668c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ff4670c2d624614 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:03.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.795+0000 7f669159a700 1 -- 172.21.15.40:0/3677903581 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c00e070 con 0x7f66940b3d00 2023-11-08T08:22:03.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.795+0000 7f669159a700 1 -- 172.21.15.40:0/3677903581 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668c00bb20 con 0x7f66940b3d00 2023-11-08T08:22:03.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.795+0000 7f669159a700 1 -- 172.21.15.40:0/3677903581 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c00be90 con 0x7f66940b3d00 2023-11-08T08:22:03.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.795+0000 7f669359e700 1 -- 172.21.15.40:0/3677903581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66940b3d00 msgr2=0x7f669406a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:03.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.795+0000 7f669359e700 1 --2- 172.21.15.40:0/3677903581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66940b3d00 0x7f669406a300 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f668c009d90 tx=0x7f668c00b330 comp rx=0 tx=0).stop 2023-11-08T08:22:03.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.795+0000 7f669359e700 1 -- 172.21.15.40:0/3677903581 shutdown_connections 2023-11-08T08:22:03.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.795+0000 7f669359e700 1 --2- 172.21.15.40:0/3677903581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66940b3d00 0x7f669406a300 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:03.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.795+0000 7f669359e700 1 -- 172.21.15.40:0/3677903581 >> 172.21.15.40:0/3677903581 conn(0x7f66940d3680 msgr2=0x7f66940cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:03.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.795+0000 7f669359e700 1 -- 172.21.15.40:0/3677903581 shutdown_connections 2023-11-08T08:22:03.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.795+0000 7f669359e700 1 -- 172.21.15.40:0/3677903581 wait complete. 2023-11-08T08:22:03.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.799+0000 7f669359e700 1 Processor -- start 2023-11-08T08:22:03.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.803+0000 7f669359e700 1 -- start start 2023-11-08T08:22:03.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.803+0000 7f669359e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6694070620 0x7f66941571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:03.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.803+0000 7f669359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66940b3d00 0x7f66940c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:03.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.803+0000 7f669359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f669415c150 0x7f66940ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:03.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.803+0000 7f669359e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669414bd60 con 0x7f669415c150 2023-11-08T08:22:03.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.803+0000 7f669359e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66941f50e0 con 0x7f6694070620 2023-11-08T08:22:03.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.803+0000 7f669359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66941f53a0 con 0x7f66940b3d00 2023-11-08T08:22:03.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.803+0000 7f6692d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f669415c150 0x7f66940ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:03.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.803+0000 7f6692d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f669415c150 0x7f66940ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56592/0 (socket says 172.21.15.40:56592) 2023-11-08T08:22:03.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.803+0000 7f669259c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6694070620 0x7f66941571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:03.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.803+0000 7f6692d9d700 1 -- 172.21.15.40:0/4272830300 learned_addr learned my addr 172.21.15.40:0/4272830300 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:03.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.803+0000 7f6691d9b700 1 --2- 172.21.15.40:0/4272830300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66940b3d00 0x7f66940c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:03.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.803+0000 7f669259c700 1 -- 172.21.15.40:0/4272830300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66940b3d00 msgr2=0x7f66940c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:03.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.803+0000 7f669259c700 1 --2- 172.21.15.40:0/4272830300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66940b3d00 0x7f66940c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:03.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.803+0000 7f669259c700 1 -- 172.21.15.40:0/4272830300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f669415c150 msgr2=0x7f66940ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:03.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.803+0000 7f669259c700 1 --2- 172.21.15.40:0/4272830300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f669415c150 0x7f66940ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:03.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.803+0000 7f669259c700 1 -- 172.21.15.40:0/4272830300 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c009a70 con 0x7f6694070620 2023-11-08T08:22:03.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.803+0000 7f6691d9b700 1 --2- 172.21.15.40:0/4272830300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66940b3d00 0x7f66940c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:22:03.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.803+0000 7f669259c700 1 --2- 172.21.15.40:0/4272830300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6694070620 0x7f66941571b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f668400c410 tx=0x7f668400c720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:03.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.803+0000 7f6682ffd700 1 -- 172.21.15.40:0/4272830300 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684005d90 con 0x7f6694070620 2023-11-08T08:22:03.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.807+0000 7f6682ffd700 1 -- 172.21.15.40:0/4272830300 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66840070d0 con 0x7f6694070620 2023-11-08T08:22:03.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.807+0000 7f669359e700 1 -- 172.21.15.40:0/4272830300 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66940d2140 con 0x7f6694070620 2023-11-08T08:22:03.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.807+0000 7f6682ffd700 1 -- 172.21.15.40:0/4272830300 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684007440 con 0x7f6694070620 2023-11-08T08:22:03.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.807+0000 7f669359e700 1 -- 172.21.15.40:0/4272830300 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66940d05e0 con 0x7f6694070620 2023-11-08T08:22:03.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.807+0000 7f669359e700 1 -- 172.21.15.40:0/4272830300 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6694070cd0 con 0x7f6694070620 2023-11-08T08:22:03.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.811+0000 7f6682ffd700 1 -- 172.21.15.40:0/4272830300 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f6684010070 con 0x7f6694070620 2023-11-08T08:22:03.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.811+0000 7f6682ffd700 1 --2- 172.21.15.40:0/4272830300 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6674064ee0 0x7f6674067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:03.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.811+0000 7f6682ffd700 1 -- 172.21.15.40:0/4272830300 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6684080e90 con 0x7f6694070620 2023-11-08T08:22:03.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.811+0000 7f6691d9b700 1 --2- 172.21.15.40:0/4272830300 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6674064ee0 0x7f6674067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:03.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.815+0000 7f6691d9b700 1 --2- 172.21.15.40:0/4272830300 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6674064ee0 0x7f6674067390 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f669404e340 tx=0x7f6688009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:03.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.819+0000 7f6682ffd700 1 -- 172.21.15.40:0/4272830300 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f6684049400 con 0x7f6694070620 2023-11-08T08:22:04.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.999+0000 7f669359e700 1 -- 172.21.15.40:0/4272830300 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6694073d20 con 0x7f6694070620 2023-11-08T08:22:04.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.999+0000 7f6682ffd700 1 -- 172.21.15.40:0/4272830300 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) v1 ==== 104+0+11 (secure 0 0 0) 0x7f668404eeb0 con 0x7f6694070620 2023-11-08T08:22:04.006 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:22:04.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.999+0000 7f669359e700 1 -- 172.21.15.40:0/4272830300 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6674064ee0 msgr2=0x7f6674067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:04.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.999+0000 7f669359e700 1 --2- 172.21.15.40:0/4272830300 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6674064ee0 0x7f6674067390 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f669404e340 tx=0x7f6688009040 comp rx=0 tx=0).stop 2023-11-08T08:22:04.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.999+0000 7f669359e700 1 -- 172.21.15.40:0/4272830300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6694070620 msgr2=0x7f66941571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:04.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:03.999+0000 7f669359e700 1 --2- 172.21.15.40:0/4272830300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6694070620 0x7f66941571b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f668400c410 tx=0x7f668400c720 comp rx=0 tx=0).stop 2023-11-08T08:22:04.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.003+0000 7f669359e700 1 -- 172.21.15.40:0/4272830300 shutdown_connections 2023-11-08T08:22:04.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.003+0000 7f669359e700 1 --2- 172.21.15.40:0/4272830300 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6674064ee0 0x7f6674067390 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.003+0000 7f669359e700 1 --2- 172.21.15.40:0/4272830300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f669415c150 0x7f66940ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.003+0000 7f669359e700 1 --2- 172.21.15.40:0/4272830300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66940b3d00 0x7f66940c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.003+0000 7f669359e700 1 --2- 172.21.15.40:0/4272830300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6694070620 0x7f66941571b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.003+0000 7f669359e700 1 -- 172.21.15.40:0/4272830300 >> 172.21.15.40:0/4272830300 conn(0x7f66940d3680 msgr2=0x7f669406ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:04.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.003+0000 7f669359e700 1 -- 172.21.15.40:0/4272830300 shutdown_connections 2023-11-08T08:22:04.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.003+0000 7f669359e700 1 -- 172.21.15.40:0/4272830300 wait complete. 2023-11-08T08:22:04.029 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T08:22:04.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.199+0000 7fc0d951e700 1 Processor -- start 2023-11-08T08:22:04.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.199+0000 7fc0d951e700 1 -- start start 2023-11-08T08:22:04.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.203+0000 7fc0d951e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0d4159cd0 0x7fc0d415d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:04.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.203+0000 7fc0d951e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc0d406ff10 0x7fc0d4070390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:04.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.203+0000 7fc0d951e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0d415d650 con 0x7fc0d40cc1a0 2023-11-08T08:22:04.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.203+0000 7fc0d951e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0d406be80 con 0x7fc0d4159cd0 2023-11-08T08:22:04.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.203+0000 7fc0d951e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0d40708d0 con 0x7fc0d406ff10 2023-11-08T08:22:04.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.203+0000 7fc0d2ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0d4159cd0 0x7fc0d415d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:04.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.203+0000 7fc0d2ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0d4159cd0 0x7fc0d415d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56600/0 (socket says 172.21.15.40:56600) 2023-11-08T08:22:04.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.203+0000 7fc0d2ffd700 1 -- 172.21.15.40:0/2779703487 learned_addr learned my addr 172.21.15.40:0/2779703487 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:04.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.203+0000 7fc0d37fe700 1 --2- 172.21.15.40:0/2779703487 >> v2:172.21.15.169:3300/0 conn(0x7fc0d406ff10 0x7fc0d4070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:04.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.203+0000 7fc0d2ffd700 1 -- 172.21.15.40:0/2779703487 >> v2:172.21.15.169:3300/0 conn(0x7fc0d406ff10 msgr2=0x7fc0d4070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:04.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.203+0000 7fc0d2ffd700 1 --2- 172.21.15.40:0/2779703487 >> v2:172.21.15.169:3300/0 conn(0x7fc0d406ff10 0x7fc0d4070390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.203+0000 7fc0d2ffd700 1 -- 172.21.15.40:0/2779703487 >> v1:172.21.15.169:6789/0 conn(0x7fc0d40cc1a0 legacy=0x7fc0d406c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:04.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.203+0000 7fc0d2ffd700 1 -- 172.21.15.40:0/2779703487 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d415a0a0 con 0x7fc0d4159cd0 2023-11-08T08:22:04.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.203+0000 7fc0d27fc700 1 -- 172.21.15.40:0/2779703487 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1542222457 0 0) 0x7fc0d415d650 con 0x7fc0d40cc1a0 2023-11-08T08:22:04.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.207+0000 7fc0d2ffd700 1 --2- 172.21.15.40:0/2779703487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0d4159cd0 0x7fc0d415d110 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc0d415c6e0 tx=0x7fc0c4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dceb698a55ceae server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:04.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.207+0000 7fc0d27fc700 1 -- 172.21.15.40:0/2779703487 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4011830 con 0x7fc0d4159cd0 2023-11-08T08:22:04.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.207+0000 7fc0d27fc700 1 -- 172.21.15.40:0/2779703487 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c4011990 con 0x7fc0d4159cd0 2023-11-08T08:22:04.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.207+0000 7fc0d27fc700 1 -- 172.21.15.40:0/2779703487 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4011d20 con 0x7fc0d4159cd0 2023-11-08T08:22:04.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.207+0000 7fc0d951e700 1 -- 172.21.15.40:0/2779703487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0d4159cd0 msgr2=0x7fc0d415d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:04.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.207+0000 7fc0d951e700 1 --2- 172.21.15.40:0/2779703487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0d4159cd0 0x7fc0d415d110 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc0d415c6e0 tx=0x7fc0c4009bf0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.207+0000 7fc0d951e700 1 -- 172.21.15.40:0/2779703487 shutdown_connections 2023-11-08T08:22:04.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.207+0000 7fc0d951e700 1 --2- 172.21.15.40:0/2779703487 >> v2:172.21.15.169:3300/0 conn(0x7fc0d406ff10 0x7fc0d4070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.207+0000 7fc0d951e700 1 --2- 172.21.15.40:0/2779703487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0d4159cd0 0x7fc0d415d110 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.207+0000 7fc0d951e700 1 -- 172.21.15.40:0/2779703487 >> 172.21.15.40:0/2779703487 conn(0x7fc0d40cf280 msgr2=0x7fc0d40cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:04.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.207+0000 7fc0d951e700 1 -- 172.21.15.40:0/2779703487 shutdown_connections 2023-11-08T08:22:04.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.207+0000 7fc0d951e700 1 -- 172.21.15.40:0/2779703487 wait complete. 2023-11-08T08:22:04.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.211+0000 7fc0d951e700 1 Processor -- start 2023-11-08T08:22:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.211+0000 7fc0d951e700 1 -- start start 2023-11-08T08:22:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.211+0000 7fc0d951e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0d406ff10 0x7fc0d414a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.211+0000 7fc0d951e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0d40cc1a0 0x7fc0d414fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.211+0000 7fc0d951e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0d4159cd0 0x7fc0d41550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.211+0000 7fc0d951e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d41f9180 con 0x7fc0d40cc1a0 2023-11-08T08:22:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.211+0000 7fc0d951e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d414f280 con 0x7fc0d4159cd0 2023-11-08T08:22:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.211+0000 7fc0d951e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d4151460 con 0x7fc0d406ff10 2023-11-08T08:22:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.211+0000 7fc0ca7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0d40cc1a0 0x7fc0d414fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.211+0000 7fc0ca7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0d40cc1a0 0x7fc0d414fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56616/0 (socket says 172.21.15.40:56616) 2023-11-08T08:22:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.211+0000 7fc0ca7fc700 1 -- 172.21.15.40:0/3413712618 learned_addr learned my addr 172.21.15.40:0/3413712618 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.211+0000 7fc0d37fe700 1 --2- 172.21.15.40:0/3413712618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0d4159cd0 0x7fc0d41550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:04.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.211+0000 7fc0d2ffd700 1 --2- 172.21.15.40:0/3413712618 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0d406ff10 0x7fc0d414a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:04.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.215+0000 7fc0ca7fc700 1 -- 172.21.15.40:0/3413712618 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0d406ff10 msgr2=0x7fc0d414a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:04.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.215+0000 7fc0ca7fc700 1 --2- 172.21.15.40:0/3413712618 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0d406ff10 0x7fc0d414a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.215+0000 7fc0ca7fc700 1 -- 172.21.15.40:0/3413712618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0d4159cd0 msgr2=0x7fc0d41550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:04.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.215+0000 7fc0ca7fc700 1 --2- 172.21.15.40:0/3413712618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0d4159cd0 0x7fc0d41550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.215+0000 7fc0ca7fc700 1 -- 172.21.15.40:0/3413712618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c4009570 con 0x7fc0d40cc1a0 2023-11-08T08:22:04.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.215+0000 7fc0d37fe700 1 --2- 172.21.15.40:0/3413712618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0d4159cd0 0x7fc0d41550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:22:04.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.215+0000 7fc0ca7fc700 1 --2- 172.21.15.40:0/3413712618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0d40cc1a0 0x7fc0d414fa60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc001f90 tx=0x7fc0bc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:04.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.215+0000 7fc0cbfff700 1 -- 172.21.15.40:0/3413712618 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc003e50 con 0x7fc0d40cc1a0 2023-11-08T08:22:04.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.215+0000 7fc0cbfff700 1 -- 172.21.15.40:0/3413712618 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc00a990 con 0x7fc0d40cc1a0 2023-11-08T08:22:04.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.215+0000 7fc0cbfff700 1 -- 172.21.15.40:0/3413712618 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc00ad40 con 0x7fc0d40cc1a0 2023-11-08T08:22:04.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.215+0000 7fc0d951e700 1 -- 172.21.15.40:0/3413712618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d40ce260 con 0x7fc0d40cc1a0 2023-11-08T08:22:04.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.215+0000 7fc0d951e700 1 -- 172.21.15.40:0/3413712618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d40cdf80 con 0x7fc0d40cc1a0 2023-11-08T08:22:04.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.215+0000 7fc0d951e700 1 -- 172.21.15.40:0/3413712618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d406c090 con 0x7fc0d40cc1a0 2023-11-08T08:22:04.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.219+0000 7fc0cbfff700 1 -- 172.21.15.40:0/3413712618 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc0bc019030 con 0x7fc0d40cc1a0 2023-11-08T08:22:04.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.219+0000 7fc0cbfff700 1 --2- 172.21.15.40:0/3413712618 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc0b4064ee0 0x7fc0b4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:04.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.219+0000 7fc0cbfff700 1 -- 172.21.15.40:0/3413712618 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc0bc081be0 con 0x7fc0d40cc1a0 2023-11-08T08:22:04.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.219+0000 7fc0d2ffd700 1 --2- 172.21.15.40:0/3413712618 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc0b4064ee0 0x7fc0b4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:04.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.223+0000 7fc0d2ffd700 1 --2- 172.21.15.40:0/3413712618 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc0b4064ee0 0x7fc0b4067390 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4000c00 tx=0x7fc0c4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:04.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.223+0000 7fc0cbfff700 1 -- 172.21.15.40:0/3413712618 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc0bc04fb50 con 0x7fc0d40cc1a0 2023-11-08T08:22:04.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.395+0000 7fc0d951e700 1 -- 172.21.15.40:0/3413712618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc0d40cca80 con 0x7fc0d40cc1a0 2023-11-08T08:22:04.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.399+0000 7fc0cbfff700 1 -- 172.21.15.40:0/3413712618 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc0bc0170a0 con 0x7fc0d40cc1a0 2023-11-08T08:22:04.405 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-11-08T08:22:04.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.403+0000 7fc0d951e700 1 -- 172.21.15.40:0/3413712618 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc0b4064ee0 msgr2=0x7fc0b4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:04.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.403+0000 7fc0d951e700 1 --2- 172.21.15.40:0/3413712618 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc0b4064ee0 0x7fc0b4067390 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4000c00 tx=0x7fc0c4008030 comp rx=0 tx=0).stop 2023-11-08T08:22:04.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.403+0000 7fc0d951e700 1 -- 172.21.15.40:0/3413712618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0d40cc1a0 msgr2=0x7fc0d414fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:04.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.403+0000 7fc0d951e700 1 --2- 172.21.15.40:0/3413712618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0d40cc1a0 0x7fc0d414fa60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc001f90 tx=0x7fc0bc00a340 comp rx=0 tx=0).stop 2023-11-08T08:22:04.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.403+0000 7fc0d951e700 1 -- 172.21.15.40:0/3413712618 shutdown_connections 2023-11-08T08:22:04.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.403+0000 7fc0d951e700 1 --2- 172.21.15.40:0/3413712618 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc0b4064ee0 0x7fc0b4067390 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.403+0000 7fc0d951e700 1 --2- 172.21.15.40:0/3413712618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0d4159cd0 0x7fc0d41550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.403+0000 7fc0d951e700 1 --2- 172.21.15.40:0/3413712618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0d40cc1a0 0x7fc0d414fa60 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.403+0000 7fc0d951e700 1 --2- 172.21.15.40:0/3413712618 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0d406ff10 0x7fc0d414a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.403+0000 7fc0d951e700 1 -- 172.21.15.40:0/3413712618 >> 172.21.15.40:0/3413712618 conn(0x7fc0d40cf280 msgr2=0x7fc0d4070a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:04.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.403+0000 7fc0d951e700 1 -- 172.21.15.40:0/3413712618 shutdown_connections 2023-11-08T08:22:04.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.403+0000 7fc0d951e700 1 -- 172.21.15.40:0/3413712618 wait complete. 2023-11-08T08:22:04.437 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:22:04.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.607+0000 7f51f0537700 1 Processor -- start 2023-11-08T08:22:04.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.607+0000 7f51f0537700 1 -- start start 2023-11-08T08:22:04.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.611+0000 7f51f0537700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f51e8163cb0 0x7f51e8164cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:04.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.611+0000 7f51f0537700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f51e806d4c0 0x7f51e806a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:04.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.611+0000 7f51f0537700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e8164080 con 0x7f51e80cdfb0 2023-11-08T08:22:04.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.611+0000 7f51f0537700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e80cc290 con 0x7f51e806d4c0 2023-11-08T08:22:04.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.611+0000 7f51f0537700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e8069bf0 con 0x7f51e8163cb0 2023-11-08T08:22:04.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.611+0000 7f51edad2700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f51e80cdfb0 0x7f51e806cdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41910/0 (socket says 172.21.15.40:41910) 2023-11-08T08:22:04.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.611+0000 7f51edad2700 1 -- 172.21.15.40:0/2697827396 learned_addr learned my addr 172.21.15.40:0/2697827396 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:04.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.611+0000 7f51eead4700 1 --2- 172.21.15.40:0/2697827396 >> v2:172.21.15.40:3300/0 conn(0x7f51e806d4c0 0x7f51e806a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:04.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.611+0000 7f51ee2d3700 1 --2- 172.21.15.40:0/2697827396 >> v2:172.21.15.169:3300/0 conn(0x7f51e8163cb0 0x7f51e8164cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:04.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.611+0000 7f51ed2d1700 1 -- 172.21.15.40:0/2697827396 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322104720 0 0) 0x7f51e8164080 con 0x7f51e80cdfb0 2023-11-08T08:22:04.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.611+0000 7f51ed2d1700 1 -- 172.21.15.40:0/2697827396 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51d40035d0 con 0x7f51e80cdfb0 2023-11-08T08:22:04.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.611+0000 7f51eead4700 1 -- 172.21.15.40:0/2697827396 >> v2:172.21.15.169:3300/0 conn(0x7f51e8163cb0 msgr2=0x7f51e8164cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:04.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.611+0000 7f51eead4700 1 --2- 172.21.15.40:0/2697827396 >> v2:172.21.15.169:3300/0 conn(0x7f51e8163cb0 0x7f51e8164cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.611+0000 7f51eead4700 1 -- 172.21.15.40:0/2697827396 >> v1:172.21.15.40:6789/0 conn(0x7f51e80cdfb0 legacy=0x7f51e806cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:04.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.611+0000 7f51eead4700 1 -- 172.21.15.40:0/2697827396 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e81651f0 con 0x7f51e806d4c0 2023-11-08T08:22:04.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.611+0000 7f51ee2d3700 1 --2- 172.21.15.40:0/2697827396 >> v2:172.21.15.169:3300/0 conn(0x7f51e8163cb0 0x7f51e8164cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:22:04.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.611+0000 7f51eead4700 1 --2- 172.21.15.40:0/2697827396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51e806d4c0 0x7f51e806a640 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f51e806fbe0 tx=0x7f51e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e1a96620eb27c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:04.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.611+0000 7f51ed2d1700 1 -- 172.21.15.40:0/2697827396 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e000e070 con 0x7f51e806d4c0 2023-11-08T08:22:04.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.611+0000 7f51ed2d1700 1 -- 172.21.15.40:0/2697827396 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e000bb20 con 0x7f51e806d4c0 2023-11-08T08:22:04.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.615+0000 7f51ed2d1700 1 -- 172.21.15.40:0/2697827396 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e0012430 con 0x7f51e806d4c0 2023-11-08T08:22:04.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.615+0000 7f51f0537700 1 -- 172.21.15.40:0/2697827396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51e806d4c0 msgr2=0x7f51e806a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:04.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.615+0000 7f51f0537700 1 --2- 172.21.15.40:0/2697827396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51e806d4c0 0x7f51e806a640 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f51e806fbe0 tx=0x7f51e000b330 comp rx=0 tx=0).stop 2023-11-08T08:22:04.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.615+0000 7f51f0537700 1 -- 172.21.15.40:0/2697827396 shutdown_connections 2023-11-08T08:22:04.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.615+0000 7f51f0537700 1 --2- 172.21.15.40:0/2697827396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51e806d4c0 0x7f51e806a640 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.615+0000 7f51f0537700 1 --2- 172.21.15.40:0/2697827396 >> v2:172.21.15.169:3300/0 conn(0x7f51e8163cb0 0x7f51e8164cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.615+0000 7f51f0537700 1 -- 172.21.15.40:0/2697827396 >> 172.21.15.40:0/2697827396 conn(0x7f51e80cf350 msgr2=0x7f51e80b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:04.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.615+0000 7f51f0537700 1 -- 172.21.15.40:0/2697827396 shutdown_connections 2023-11-08T08:22:04.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.615+0000 7f51f0537700 1 -- 172.21.15.40:0/2697827396 wait complete. 2023-11-08T08:22:04.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.615+0000 7f51f0537700 1 Processor -- start 2023-11-08T08:22:04.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.615+0000 7f51f0537700 1 -- start start 2023-11-08T08:22:04.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51f0537700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51e806d4c0 0x7f51e80c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:04.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51f0537700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51e80cdfb0 0x7f51e806cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:04.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51f0537700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51e8163cb0 0x7f51e806a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:04.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51f0537700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e81f9d40 con 0x7f51e8163cb0 2023-11-08T08:22:04.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51f0537700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e81f90c0 con 0x7f51e806d4c0 2023-11-08T08:22:04.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51f0537700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e81f93a0 con 0x7f51e80cdfb0 2023-11-08T08:22:04.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51eead4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51e8163cb0 0x7f51e806a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:04.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51eead4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51e8163cb0 0x7f51e806a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56642/0 (socket says 172.21.15.40:56642) 2023-11-08T08:22:04.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51eead4700 1 -- 172.21.15.40:0/2841217384 learned_addr learned my addr 172.21.15.40:0/2841217384 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:04.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51edad2700 1 --2- 172.21.15.40:0/2841217384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51e80cdfb0 0x7f51e806cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51ee2d3700 1 --2- 172.21.15.40:0/2841217384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51e806d4c0 0x7f51e80c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:04.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51eead4700 1 -- 172.21.15.40:0/2841217384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51e80cdfb0 msgr2=0x7f51e806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:04.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51eead4700 1 --2- 172.21.15.40:0/2841217384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51e80cdfb0 0x7f51e806cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51eead4700 1 -- 172.21.15.40:0/2841217384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51e806d4c0 msgr2=0x7f51e80c4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51eead4700 1 --2- 172.21.15.40:0/2841217384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51e806d4c0 0x7f51e80c4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51eead4700 1 -- 172.21.15.40:0/2841217384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e40094b0 con 0x7f51e8163cb0 2023-11-08T08:22:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51eead4700 1 --2- 172.21.15.40:0/2841217384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51e8163cb0 0x7f51e806a540 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f51e0014040 tx=0x7f51e0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51deffd700 1 -- 172.21.15.40:0/2841217384 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e0012430 con 0x7f51e8163cb0 2023-11-08T08:22:04.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51deffd700 1 -- 172.21.15.40:0/2841217384 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e0020070 con 0x7f51e8163cb0 2023-11-08T08:22:04.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51f0537700 1 -- 172.21.15.40:0/2841217384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e0009a70 con 0x7f51e8163cb0 2023-11-08T08:22:04.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51deffd700 1 -- 172.21.15.40:0/2841217384 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e0004280 con 0x7f51e8163cb0 2023-11-08T08:22:04.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.619+0000 7f51f0537700 1 -- 172.21.15.40:0/2841217384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e80cc780 con 0x7f51e8163cb0 2023-11-08T08:22:04.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.623+0000 7f51f0537700 1 -- 172.21.15.40:0/2841217384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e80cc4a0 con 0x7f51e8163cb0 2023-11-08T08:22:04.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.623+0000 7f51deffd700 1 -- 172.21.15.40:0/2841217384 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f51e00043e0 con 0x7f51e8163cb0 2023-11-08T08:22:04.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.627+0000 7f51deffd700 1 --2- 172.21.15.40:0/2841217384 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f51cc064ee0 0x7f51cc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:04.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.627+0000 7f51deffd700 1 -- 172.21.15.40:0/2841217384 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f51e0022070 con 0x7f51e8163cb0 2023-11-08T08:22:04.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.627+0000 7f51ee2d3700 1 --2- 172.21.15.40:0/2841217384 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f51cc064ee0 0x7f51cc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:04.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.627+0000 7f51ee2d3700 1 --2- 172.21.15.40:0/2841217384 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f51cc064ee0 0x7f51cc067390 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009c70 tx=0x7f51e4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:04.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.631+0000 7f51deffd700 1 -- 172.21.15.40:0/2841217384 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f51e0052410 con 0x7f51e8163cb0 2023-11-08T08:22:04.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.847+0000 7f51f0537700 1 -- 172.21.15.40:0/2841217384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51e8069e40 con 0x7f51e8163cb0 2023-11-08T08:22:04.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.847+0000 7f51deffd700 1 -- 172.21.15.40:0/2841217384 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f51e0055a30 con 0x7f51e8163cb0 2023-11-08T08:22:04.856 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:22:04.856 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1549516710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1549516710},{"type":"v1","addr":"172.21.15.169:6837","nonce":1549516710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5728,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/912870125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":912870125},{"type":"v1","addr":"172.21.15.169:6835","nonce":912870125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5772,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1045606427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1045606427},{"type":"v1","addr":"172.21.15.40:6834","nonce":1045606427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:21:17.730002+0000","modified":"2023-11-08T08:21:20.752976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2471311394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2471311394},{"type":"v1","addr":"172.21.15.40:6835","nonce":2471311394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T08:22:04.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.851+0000 7f51f0537700 1 -- 172.21.15.40:0/2841217384 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f51cc064ee0 msgr2=0x7f51cc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:04.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.851+0000 7f51f0537700 1 --2- 172.21.15.40:0/2841217384 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f51cc064ee0 0x7f51cc067390 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009c70 tx=0x7f51e4008040 comp rx=0 tx=0).stop 2023-11-08T08:22:04.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.851+0000 7f51f0537700 1 -- 172.21.15.40:0/2841217384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51e8163cb0 msgr2=0x7f51e806a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:04.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.851+0000 7f51f0537700 1 --2- 172.21.15.40:0/2841217384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51e8163cb0 0x7f51e806a540 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f51e0014040 tx=0x7f51e0004af0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.851+0000 7f51f0537700 1 -- 172.21.15.40:0/2841217384 shutdown_connections 2023-11-08T08:22:04.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.851+0000 7f51f0537700 1 --2- 172.21.15.40:0/2841217384 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f51cc064ee0 0x7f51cc067390 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.851+0000 7f51f0537700 1 --2- 172.21.15.40:0/2841217384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51e8163cb0 0x7f51e806a540 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.851+0000 7f51f0537700 1 --2- 172.21.15.40:0/2841217384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51e80cdfb0 0x7f51e806cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.851+0000 7f51f0537700 1 --2- 172.21.15.40:0/2841217384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51e806d4c0 0x7f51e80c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:04.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.851+0000 7f51f0537700 1 -- 172.21.15.40:0/2841217384 >> 172.21.15.40:0/2841217384 conn(0x7f51e80cf350 msgr2=0x7f51e80c9720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:04.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.851+0000 7f51f0537700 1 -- 172.21.15.40:0/2841217384 shutdown_connections 2023-11-08T08:22:04.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:04.851+0000 7f51f0537700 1 -- 172.21.15.40:0/2841217384 wait complete. 2023-11-08T08:22:04.861 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 25 2023-11-08T08:22:04.883 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:22:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.055+0000 7f817759e700 1 Processor -- start 2023-11-08T08:22:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.055+0000 7f817759e700 1 -- start start 2023-11-08T08:22:05.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f817759e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8178071aa0 0x7f817806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:05.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f817759e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8178069e00 0x7f817806a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:05.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f817759e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f817806c600 con 0x7f817815c640 2023-11-08T08:22:05.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f817759e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f817806cef0 con 0x7f8178069e00 2023-11-08T08:22:05.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f817759e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81780d38f0 con 0x7f8178071aa0 2023-11-08T08:22:05.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f817659c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f817815c640 0x7f81780d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41924/0 (socket says 172.21.15.40:41924) 2023-11-08T08:22:05.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f817659c700 1 -- 172.21.15.40:0/3790556360 learned_addr learned my addr 172.21.15.40:0/3790556360 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:05.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f8176d9d700 1 --2- 172.21.15.40:0/3790556360 >> v2:172.21.15.40:3300/0 conn(0x7f8178069e00 0x7f817806a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:05.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f816dd9b700 1 --2- 172.21.15.40:0/3790556360 >> v2:172.21.15.169:3300/0 conn(0x7f8178071aa0 0x7f817806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:05.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f8175d9b700 1 -- 172.21.15.40:0/3790556360 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703393665 0 0) 0x7f817806c600 con 0x7f817815c640 2023-11-08T08:22:05.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f8175d9b700 1 -- 172.21.15.40:0/3790556360 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81640035d0 con 0x7f817815c640 2023-11-08T08:22:05.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f8175d9b700 1 -- 172.21.15.40:0/3790556360 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4014388600 0 0) 0x7f81640035d0 con 0x7f817815c640 2023-11-08T08:22:05.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f8176d9d700 1 -- 172.21.15.40:0/3790556360 >> v2:172.21.15.169:3300/0 conn(0x7f8178071aa0 msgr2=0x7f817806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:05.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f8176d9d700 1 --2- 172.21.15.40:0/3790556360 >> v2:172.21.15.169:3300/0 conn(0x7f8178071aa0 0x7f817806d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f8176d9d700 1 -- 172.21.15.40:0/3790556360 >> v1:172.21.15.40:6789/0 conn(0x7f817815c640 legacy=0x7f81780d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:05.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f8176d9d700 1 -- 172.21.15.40:0/3790556360 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81780722f0 con 0x7f8178069e00 2023-11-08T08:22:05.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f816dd9b700 1 --2- 172.21.15.40:0/3790556360 >> v2:172.21.15.169:3300/0 conn(0x7f8178071aa0 0x7f817806d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:22:05.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f8176d9d700 1 --2- 172.21.15.40:0/3790556360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8178069e00 0x7f817806a280 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8178070e70 tx=0x7f817000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87f62c80461b040 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:05.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f8175d9b700 1 -- 172.21.15.40:0/3790556360 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817000e070 con 0x7f8178069e00 2023-11-08T08:22:05.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f8175d9b700 1 -- 172.21.15.40:0/3790556360 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8170003a40 con 0x7f8178069e00 2023-11-08T08:22:05.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.059+0000 7f8175d9b700 1 -- 172.21.15.40:0/3790556360 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8168001670 con 0x7f8178069e00 2023-11-08T08:22:05.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.063+0000 7f817759e700 1 -- 172.21.15.40:0/3790556360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8178069e00 msgr2=0x7f817806a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:05.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.063+0000 7f817759e700 1 --2- 172.21.15.40:0/3790556360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8178069e00 0x7f817806a280 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8178070e70 tx=0x7f817000b330 comp rx=0 tx=0).stop 2023-11-08T08:22:05.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.063+0000 7f817759e700 1 -- 172.21.15.40:0/3790556360 shutdown_connections 2023-11-08T08:22:05.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.063+0000 7f817759e700 1 --2- 172.21.15.40:0/3790556360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8178069e00 0x7f817806a280 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.063+0000 7f817759e700 1 --2- 172.21.15.40:0/3790556360 >> v2:172.21.15.169:3300/0 conn(0x7f8178071aa0 0x7f817806d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.063+0000 7f817759e700 1 -- 172.21.15.40:0/3790556360 >> 172.21.15.40:0/3790556360 conn(0x7f81780b6520 msgr2=0x7f8178071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:05.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.063+0000 7f817759e700 1 -- 172.21.15.40:0/3790556360 shutdown_connections 2023-11-08T08:22:05.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.063+0000 7f817759e700 1 -- 172.21.15.40:0/3790556360 wait complete. 2023-11-08T08:22:05.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.063+0000 7f817759e700 1 Processor -- start 2023-11-08T08:22:05.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.067+0000 7f817759e700 1 -- start start 2023-11-08T08:22:05.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.067+0000 7f817759e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8178069e00 0x7f81781e3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:05.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.067+0000 7f817759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8178071aa0 0x7f81781e3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:05.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.067+0000 7f817759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f817815c640 0x7f81780c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:05.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.067+0000 7f817759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81780c7230 con 0x7f8178071aa0 2023-11-08T08:22:05.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.067+0000 7f817759e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81780c65f0 con 0x7f8178069e00 2023-11-08T08:22:05.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.067+0000 7f817759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81780c6890 con 0x7f817815c640 2023-11-08T08:22:05.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.067+0000 7f816dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8178071aa0 0x7f81781e3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:05.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.067+0000 7f816dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8178071aa0 0x7f81781e3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56662/0 (socket says 172.21.15.40:56662) 2023-11-08T08:22:05.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.067+0000 7f816dd9b700 1 -- 172.21.15.40:0/874015344 learned_addr learned my addr 172.21.15.40:0/874015344 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:05.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.067+0000 7f817659c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8178069e00 0x7f81781e3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:05.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.067+0000 7f8176d9d700 1 --2- 172.21.15.40:0/874015344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f817815c640 0x7f81780c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:05.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.067+0000 7f816dd9b700 1 -- 172.21.15.40:0/874015344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f817815c640 msgr2=0x7f81780c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:05.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.067+0000 7f816dd9b700 1 --2- 172.21.15.40:0/874015344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f817815c640 0x7f81780c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.067+0000 7f816dd9b700 1 -- 172.21.15.40:0/874015344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8178069e00 msgr2=0x7f81781e3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:05.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.067+0000 7f816dd9b700 1 --2- 172.21.15.40:0/874015344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8178069e00 0x7f81781e3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.067+0000 7f816dd9b700 1 -- 172.21.15.40:0/874015344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81600094b0 con 0x7f8178071aa0 2023-11-08T08:22:05.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.067+0000 7f816dd9b700 1 --2- 172.21.15.40:0/874015344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8178071aa0 0x7f81781e3e30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f816000ebe0 tx=0x7f816000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:05.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.067+0000 7f816f7fe700 1 -- 172.21.15.40:0/874015344 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8160009770 con 0x7f8178071aa0 2023-11-08T08:22:05.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.067+0000 7f816f7fe700 1 -- 172.21.15.40:0/874015344 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816000c850 con 0x7f8178071aa0 2023-11-08T08:22:05.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.071+0000 7f816f7fe700 1 -- 172.21.15.40:0/874015344 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816000cbe0 con 0x7f8178071aa0 2023-11-08T08:22:05.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.071+0000 7f817759e700 1 -- 172.21.15.40:0/874015344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8170009a70 con 0x7f8178071aa0 2023-11-08T08:22:05.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.071+0000 7f817759e700 1 -- 172.21.15.40:0/874015344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8178072010 con 0x7f8178071aa0 2023-11-08T08:22:05.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.071+0000 7f817759e700 1 -- 172.21.15.40:0/874015344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f817815ca10 con 0x7f8178071aa0 2023-11-08T08:22:05.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.075+0000 7f816f7fe700 1 -- 172.21.15.40:0/874015344 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f8160019030 con 0x7f8178071aa0 2023-11-08T08:22:05.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.075+0000 7f816f7fe700 1 --2- 172.21.15.40:0/874015344 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8158064ee0 0x7f8158067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:05.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.075+0000 7f816f7fe700 1 -- 172.21.15.40:0/874015344 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8160081e80 con 0x7f8178071aa0 2023-11-08T08:22:05.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.075+0000 7f817659c700 1 --2- 172.21.15.40:0/874015344 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8158064ee0 0x7f8158067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:05.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.075+0000 7f817659c700 1 --2- 172.21.15.40:0/874015344 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8158064ee0 0x7f8158067390 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8168000f60 tx=0x7f8168001230 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:05.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.083+0000 7f816f7fe700 1 -- 172.21.15.40:0/874015344 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f816004fee0 con 0x7f8178071aa0 2023-11-08T08:22:05.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.255+0000 7f817759e700 1 -- 172.21.15.40:0/874015344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f817806c830 con 0x7f8178071aa0 2023-11-08T08:22:05.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.259+0000 7f816f7fe700 1 -- 172.21.15.40:0/874015344 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+18084 (secure 0 0 0) 0x7f8160053500 con 0x7f8178071aa0 2023-11-08T08:22:05.266 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:22:05.266 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:21:18.736039+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:21:15.464908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-08T08:21:16.755373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:22:05.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.259+0000 7f817759e700 1 -- 172.21.15.40:0/874015344 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8158064ee0 msgr2=0x7f8158067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:05.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.259+0000 7f817759e700 1 --2- 172.21.15.40:0/874015344 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8158064ee0 0x7f8158067390 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8168000f60 tx=0x7f8168001230 comp rx=0 tx=0).stop 2023-11-08T08:22:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.259+0000 7f817759e700 1 -- 172.21.15.40:0/874015344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8178071aa0 msgr2=0x7f81781e3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.259+0000 7f817759e700 1 --2- 172.21.15.40:0/874015344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8178071aa0 0x7f81781e3e30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f816000ebe0 tx=0x7f816000eef0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.259+0000 7f817759e700 1 -- 172.21.15.40:0/874015344 shutdown_connections 2023-11-08T08:22:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.259+0000 7f817759e700 1 --2- 172.21.15.40:0/874015344 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8158064ee0 0x7f8158067390 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.259+0000 7f817759e700 1 --2- 172.21.15.40:0/874015344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f817815c640 0x7f81780c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.259+0000 7f817759e700 1 --2- 172.21.15.40:0/874015344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8178071aa0 0x7f81781e3e30 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.259+0000 7f817759e700 1 --2- 172.21.15.40:0/874015344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8178069e00 0x7f81781e3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.259+0000 7f817759e700 1 -- 172.21.15.40:0/874015344 >> 172.21.15.40:0/874015344 conn(0x7f81780b6520 msgr2=0x7f81780d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:05.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.259+0000 7f817759e700 1 -- 172.21.15.40:0/874015344 shutdown_connections 2023-11-08T08:22:05.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.259+0000 7f817759e700 1 -- 172.21.15.40:0/874015344 wait complete. 2023-11-08T08:22:05.289 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T08:22:05.290 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:22:05.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.455+0000 7f924de61700 1 Processor -- start 2023-11-08T08:22:05.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.455+0000 7f924de61700 1 -- start start 2023-11-08T08:22:05.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.459+0000 7f924de61700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f924815c640 0x7f92480d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:05.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.459+0000 7f924de61700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9248071aa0 0x7f924806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:05.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.459+0000 7f924de61700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f924806c600 con 0x7f9248069e00 2023-11-08T08:22:05.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.459+0000 7f924de61700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f924806cef0 con 0x7f924815c640 2023-11-08T08:22:05.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.459+0000 7f924de61700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92480722f0 con 0x7f9248071aa0 2023-11-08T08:22:05.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.459+0000 7f924ce5f700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f924815c640 0x7f92480d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:05.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.459+0000 7f923ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9248071aa0 0x7f924806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:05.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.459+0000 7f924ce5f700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f924815c640 0x7f92480d31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:34180/0 (socket says 172.21.15.40:34180) 2023-11-08T08:22:05.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.459+0000 7f924ce5f700 1 -- 172.21.15.40:0/293189333 learned_addr learned my addr 172.21.15.40:0/293189333 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:05.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.459+0000 7f923ffff700 1 -- 172.21.15.40:0/293189333 >> v2:172.21.15.169:3300/0 conn(0x7f924815c640 msgr2=0x7f92480d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:05.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.459+0000 7f923ffff700 1 --2- 172.21.15.40:0/293189333 >> v2:172.21.15.169:3300/0 conn(0x7f924815c640 0x7f92480d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.459+0000 7f923ffff700 1 -- 172.21.15.40:0/293189333 >> v1:172.21.15.169:6789/0 conn(0x7f9248069e00 legacy=0x7f92480d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:05.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.459+0000 7f923ffff700 1 -- 172.21.15.40:0/293189333 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924815ca10 con 0x7f9248071aa0 2023-11-08T08:22:05.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.459+0000 7f923ffff700 1 --2- 172.21.15.40:0/293189333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9248071aa0 0x7f924806d310 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9238009850 tx=0x7f9238009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e311914a20709871 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:05.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.463+0000 7f923f7fe700 1 -- 172.21.15.40:0/293189333 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9238011830 con 0x7f9248071aa0 2023-11-08T08:22:05.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.463+0000 7f923f7fe700 1 -- 172.21.15.40:0/293189333 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9238011990 con 0x7f9248071aa0 2023-11-08T08:22:05.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.463+0000 7f923f7fe700 1 -- 172.21.15.40:0/293189333 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9238011d40 con 0x7f9248071aa0 2023-11-08T08:22:05.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.463+0000 7f924de61700 1 -- 172.21.15.40:0/293189333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9248071aa0 msgr2=0x7f924806d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:05.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.463+0000 7f924de61700 1 --2- 172.21.15.40:0/293189333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9248071aa0 0x7f924806d310 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9238009850 tx=0x7f9238009b60 comp rx=0 tx=0).stop 2023-11-08T08:22:05.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.463+0000 7f924de61700 1 -- 172.21.15.40:0/293189333 shutdown_connections 2023-11-08T08:22:05.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.463+0000 7f924de61700 1 --2- 172.21.15.40:0/293189333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9248071aa0 0x7f924806d310 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.463+0000 7f924de61700 1 --2- 172.21.15.40:0/293189333 >> v2:172.21.15.169:3300/0 conn(0x7f924815c640 0x7f92480d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.463+0000 7f924de61700 1 -- 172.21.15.40:0/293189333 >> 172.21.15.40:0/293189333 conn(0x7f92480b6520 msgr2=0x7f9248071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:05.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.463+0000 7f924de61700 1 -- 172.21.15.40:0/293189333 shutdown_connections 2023-11-08T08:22:05.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.463+0000 7f924de61700 1 -- 172.21.15.40:0/293189333 wait complete. 2023-11-08T08:22:05.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.467+0000 7f924de61700 1 Processor -- start 2023-11-08T08:22:05.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.467+0000 7f924de61700 1 -- start start 2023-11-08T08:22:05.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.467+0000 7f924de61700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9248069e00 0x7f92481f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:05.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.467+0000 7f924de61700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9248071aa0 0x7f92480b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:05.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.467+0000 7f924de61700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f924815c640 0x7f92480b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:05.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.467+0000 7f924de61700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924809c2b0 con 0x7f9248071aa0 2023-11-08T08:22:05.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.467+0000 7f924de61700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92480b2bb0 con 0x7f9248069e00 2023-11-08T08:22:05.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.467+0000 7f924de61700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92480b28d0 con 0x7f924815c640 2023-11-08T08:22:05.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.471+0000 7f923ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9248071aa0 0x7f92480b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:05.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.471+0000 7f923ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9248071aa0 0x7f92480b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56664/0 (socket says 172.21.15.40:56664) 2023-11-08T08:22:05.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.471+0000 7f923ffff700 1 -- 172.21.15.40:0/1819213667 learned_addr learned my addr 172.21.15.40:0/1819213667 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:05.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.471+0000 7f924ce5f700 1 --2- 172.21.15.40:0/1819213667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9248069e00 0x7f92481f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:05.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.471+0000 7f924d660700 1 --2- 172.21.15.40:0/1819213667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f924815c640 0x7f92480b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:05.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.471+0000 7f923ffff700 1 -- 172.21.15.40:0/1819213667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f924815c640 msgr2=0x7f92480b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:05.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.471+0000 7f923ffff700 1 --2- 172.21.15.40:0/1819213667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f924815c640 0x7f92480b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.471+0000 7f923ffff700 1 -- 172.21.15.40:0/1819213667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9248069e00 msgr2=0x7f92481f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:05.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.471+0000 7f923ffff700 1 --2- 172.21.15.40:0/1819213667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9248069e00 0x7f92481f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.471+0000 7f923ffff700 1 -- 172.21.15.40:0/1819213667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92380094b0 con 0x7f9248071aa0 2023-11-08T08:22:05.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.471+0000 7f923ffff700 1 --2- 172.21.15.40:0/1819213667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9248071aa0 0x7f92480b3ff0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f92380097a0 tx=0x7f9238004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:05.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.471+0000 7f923d7fa700 1 -- 172.21.15.40:0/1819213667 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9238004450 con 0x7f9248071aa0 2023-11-08T08:22:05.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.471+0000 7f923d7fa700 1 -- 172.21.15.40:0/1819213667 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923801e070 con 0x7f9248071aa0 2023-11-08T08:22:05.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.471+0000 7f923d7fa700 1 -- 172.21.15.40:0/1819213667 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9238003900 con 0x7f9248071aa0 2023-11-08T08:22:05.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.471+0000 7f924de61700 1 -- 172.21.15.40:0/1819213667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924806c830 con 0x7f9248071aa0 2023-11-08T08:22:05.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.471+0000 7f924de61700 1 -- 172.21.15.40:0/1819213667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9248072010 con 0x7f9248071aa0 2023-11-08T08:22:05.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.475+0000 7f924de61700 1 -- 172.21.15.40:0/1819213667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f924806cb10 con 0x7f9248071aa0 2023-11-08T08:22:05.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.475+0000 7f923d7fa700 1 -- 172.21.15.40:0/1819213667 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f923802a020 con 0x7f9248071aa0 2023-11-08T08:22:05.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.475+0000 7f923d7fa700 1 --2- 172.21.15.40:0/1819213667 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9234064ee0 0x7f9234067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:05.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.475+0000 7f923d7fa700 1 -- 172.21.15.40:0/1819213667 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9238082e80 con 0x7f9248071aa0 2023-11-08T08:22:05.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.479+0000 7f924ce5f700 1 --2- 172.21.15.40:0/1819213667 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9234064ee0 0x7f9234067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:05.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.479+0000 7f924ce5f700 1 --2- 172.21.15.40:0/1819213667 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9234064ee0 0x7f9234067390 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f92440113e0 tx=0x7f924400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:05.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.483+0000 7f923d7fa700 1 -- 172.21.15.40:0/1819213667 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f9238050df0 con 0x7f9248071aa0 2023-11-08T08:22:05.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.695+0000 7f924de61700 1 -- 172.21.15.40:0/1819213667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f924815e630 con 0x7f9248071aa0 2023-11-08T08:22:05.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.695+0000 7f923d7fa700 1 -- 172.21.15.40:0/1819213667 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f924815e630 con 0x7f9248071aa0 2023-11-08T08:22:05.703 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:22:05.703 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1549516710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1549516710},{"type":"v1","addr":"172.21.15.169:6837","nonce":1549516710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5728,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/912870125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":912870125},{"type":"v1","addr":"172.21.15.169:6835","nonce":912870125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5772,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1045606427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1045606427},{"type":"v1","addr":"172.21.15.40:6834","nonce":1045606427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:21:17.730002+0000","modified":"2023-11-08T08:21:20.752976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2471311394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2471311394},{"type":"v1","addr":"172.21.15.40:6835","nonce":2471311394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T08:22:05.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.699+0000 7f924de61700 1 -- 172.21.15.40:0/1819213667 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9234064ee0 msgr2=0x7f9234067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:05.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.699+0000 7f924de61700 1 --2- 172.21.15.40:0/1819213667 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9234064ee0 0x7f9234067390 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f92440113e0 tx=0x7f924400b040 comp rx=0 tx=0).stop 2023-11-08T08:22:05.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.699+0000 7f924de61700 1 -- 172.21.15.40:0/1819213667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9248071aa0 msgr2=0x7f92480b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:05.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.699+0000 7f924de61700 1 --2- 172.21.15.40:0/1819213667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9248071aa0 0x7f92480b3ff0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f92380097a0 tx=0x7f9238004770 comp rx=0 tx=0).stop 2023-11-08T08:22:05.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.699+0000 7f924de61700 1 -- 172.21.15.40:0/1819213667 shutdown_connections 2023-11-08T08:22:05.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.699+0000 7f924de61700 1 --2- 172.21.15.40:0/1819213667 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9234064ee0 0x7f9234067390 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.699+0000 7f924de61700 1 --2- 172.21.15.40:0/1819213667 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f924815c640 0x7f92480b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.699+0000 7f924de61700 1 --2- 172.21.15.40:0/1819213667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9248071aa0 0x7f92480b3ff0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.699+0000 7f924de61700 1 --2- 172.21.15.40:0/1819213667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9248069e00 0x7f92481f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.699+0000 7f924de61700 1 -- 172.21.15.40:0/1819213667 >> 172.21.15.40:0/1819213667 conn(0x7f92480b6520 msgr2=0x7f92480d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:05.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.699+0000 7f924de61700 1 -- 172.21.15.40:0/1819213667 shutdown_connections 2023-11-08T08:22:05.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.699+0000 7f924de61700 1 -- 172.21.15.40:0/1819213667 wait complete. 2023-11-08T08:22:05.708 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 25 2023-11-08T08:22:05.727 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:22:05.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.903+0000 7f05876ee700 1 Processor -- start 2023-11-08T08:22:05.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.903+0000 7f05876ee700 1 -- start start 2023-11-08T08:22:05.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.907+0000 7f05876ee700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f05800b6200 0x7f0580070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:05.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.907+0000 7f05876ee700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f058015d080 con 0x7f058015d510 2023-11-08T08:22:05.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.907+0000 7f05876ee700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0580153870 con 0x7f05801534a0 2023-11-08T08:22:05.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.907+0000 7f05876ee700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f058015d970 con 0x7f05800b6200 2023-11-08T08:22:05.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.907+0000 7f0584c89700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f058015d510 0x7f05800b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41932/0 (socket says 172.21.15.40:41932) 2023-11-08T08:22:05.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.907+0000 7f0584c89700 1 -- 172.21.15.40:0/298945323 learned_addr learned my addr 172.21.15.40:0/298945323 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:05.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.907+0000 7f0585c8b700 1 --2- 172.21.15.40:0/298945323 >> v2:172.21.15.40:3300/0 conn(0x7f05800b6200 0x7f0580070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:05.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.907+0000 7f0577fff700 1 -- 172.21.15.40:0/298945323 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615694917 0 0) 0x7f058015d080 con 0x7f058015d510 2023-11-08T08:22:05.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.907+0000 7f0577fff700 1 -- 172.21.15.40:0/298945323 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f056c0035d0 con 0x7f058015d510 2023-11-08T08:22:05.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.907+0000 7f0577fff700 1 -- 172.21.15.40:0/298945323 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 629669947 0 0) 0x7f0580153870 con 0x7f05801534a0 2023-11-08T08:22:05.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.907+0000 7f0577fff700 1 -- 172.21.15.40:0/298945323 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f058015d080 con 0x7f05801534a0 2023-11-08T08:22:05.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.907+0000 7f0585c8b700 1 -- 172.21.15.40:0/298945323 >> v1:172.21.15.169:6789/0 conn(0x7f05801534a0 legacy=0x7f0580160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:05.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.907+0000 7f0585c8b700 1 -- 172.21.15.40:0/298945323 >> v1:172.21.15.40:6789/0 conn(0x7f058015d510 legacy=0x7f05800b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:05.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.907+0000 7f0585c8b700 1 -- 172.21.15.40:0/298945323 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05800b53c0 con 0x7f05800b6200 2023-11-08T08:22:05.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.907+0000 7f0585c8b700 1 --2- 172.21.15.40:0/298945323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05800b6200 0x7f0580070630 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0578009850 tx=0x7f0578009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc4cd6805fff739 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:05.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.911+0000 7f0577fff700 1 -- 172.21.15.40:0/298945323 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0578011830 con 0x7f05800b6200 2023-11-08T08:22:05.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.911+0000 7f0577fff700 1 -- 172.21.15.40:0/298945323 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0578011990 con 0x7f05800b6200 2023-11-08T08:22:05.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.911+0000 7f0577fff700 1 -- 172.21.15.40:0/298945323 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0578011d40 con 0x7f05800b6200 2023-11-08T08:22:05.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.911+0000 7f05876ee700 1 -- 172.21.15.40:0/298945323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05800b6200 msgr2=0x7f0580070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:05.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.911+0000 7f05876ee700 1 --2- 172.21.15.40:0/298945323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05800b6200 0x7f0580070630 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0578009850 tx=0x7f0578009b60 comp rx=0 tx=0).stop 2023-11-08T08:22:05.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.911+0000 7f05876ee700 1 -- 172.21.15.40:0/298945323 shutdown_connections 2023-11-08T08:22:05.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.911+0000 7f05876ee700 1 --2- 172.21.15.40:0/298945323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05800b6200 0x7f0580070630 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.911+0000 7f05876ee700 1 -- 172.21.15.40:0/298945323 >> 172.21.15.40:0/298945323 conn(0x7f058006bfc0 msgr2=0x7f058006c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:05.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.911+0000 7f05876ee700 1 -- 172.21.15.40:0/298945323 shutdown_connections 2023-11-08T08:22:05.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.911+0000 7f05876ee700 1 -- 172.21.15.40:0/298945323 wait complete. 2023-11-08T08:22:05.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.911+0000 7f05876ee700 1 Processor -- start 2023-11-08T08:22:05.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.915+0000 7f05876ee700 1 -- start start 2023-11-08T08:22:05.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.915+0000 7f05876ee700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05800b6200 0x7f05801f13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:05.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.915+0000 7f05876ee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05801534a0 0x7f058014c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:05.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.915+0000 7f05876ee700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f058015d510 0x7f05801f5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:05.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.915+0000 7f05876ee700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0580154a50 con 0x7f05801534a0 2023-11-08T08:22:05.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.915+0000 7f05876ee700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0580154d30 con 0x7f058015d510 2023-11-08T08:22:05.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.915+0000 7f05876ee700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058014f1b0 con 0x7f05800b6200 2023-11-08T08:22:05.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.915+0000 7f0584c89700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05801534a0 0x7f058014c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:05.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.915+0000 7f0584c89700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05801534a0 0x7f058014c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56692/0 (socket says 172.21.15.40:56692) 2023-11-08T08:22:05.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.915+0000 7f0584c89700 1 -- 172.21.15.40:0/2594509053 learned_addr learned my addr 172.21.15.40:0/2594509053 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:05.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.915+0000 7f058548a700 1 --2- 172.21.15.40:0/2594509053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05800b6200 0x7f05801f13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:05.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.915+0000 7f0585c8b700 1 --2- 172.21.15.40:0/2594509053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f058015d510 0x7f05801f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:05.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.915+0000 7f0585c8b700 1 -- 172.21.15.40:0/2594509053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05800b6200 msgr2=0x7f05801f13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:05.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.915+0000 7f0585c8b700 1 --2- 172.21.15.40:0/2594509053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05800b6200 0x7f05801f13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.915+0000 7f0585c8b700 1 -- 172.21.15.40:0/2594509053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05801534a0 msgr2=0x7f058014c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:05.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.915+0000 7f0585c8b700 1 --2- 172.21.15.40:0/2594509053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05801534a0 0x7f058014c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:05.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.915+0000 7f0585c8b700 1 -- 172.21.15.40:0/2594509053 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05780094b0 con 0x7f058015d510 2023-11-08T08:22:05.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.915+0000 7f0584c89700 1 --2- 172.21.15.40:0/2594509053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05801534a0 0x7f058014c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:22:05.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.915+0000 7f0585c8b700 1 --2- 172.21.15.40:0/2594509053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f058015d510 0x7f05801f5100 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0578016000 tx=0x7f0578004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:05.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.919+0000 7f0575ffb700 1 -- 172.21.15.40:0/2594509053 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057801b840 con 0x7f058015d510 2023-11-08T08:22:05.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.919+0000 7f0575ffb700 1 -- 172.21.15.40:0/2594509053 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057801f070 con 0x7f058015d510 2023-11-08T08:22:05.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.919+0000 7f0575ffb700 1 -- 172.21.15.40:0/2594509053 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057801bc00 con 0x7f058015d510 2023-11-08T08:22:05.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.919+0000 7f05876ee700 1 -- 172.21.15.40:0/2594509053 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f058015ca30 con 0x7f058015d510 2023-11-08T08:22:05.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.919+0000 7f05876ee700 1 -- 172.21.15.40:0/2594509053 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05800b56a0 con 0x7f058015d510 2023-11-08T08:22:05.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.919+0000 7f05876ee700 1 -- 172.21.15.40:0/2594509053 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f058015cd10 con 0x7f058015d510 2023-11-08T08:22:05.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.923+0000 7f0575ffb700 1 -- 172.21.15.40:0/2594509053 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f057800f050 con 0x7f058015d510 2023-11-08T08:22:05.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.923+0000 7f0575ffb700 1 --2- 172.21.15.40:0/2594509053 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0564064f30 0x7f05640673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:05.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.923+0000 7f0575ffb700 1 -- 172.21.15.40:0/2594509053 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0578021070 con 0x7f058015d510 2023-11-08T08:22:05.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.923+0000 7f058548a700 1 --2- 172.21.15.40:0/2594509053 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0564064f30 0x7f05640673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:05.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.927+0000 7f058548a700 1 --2- 172.21.15.40:0/2594509053 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0564064f30 0x7f05640673e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f057c001560 tx=0x7f057c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:05.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:05.931+0000 7f0575ffb700 1 -- 172.21.15.40:0/2594509053 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f0578051c30 con 0x7f058015d510 2023-11-08T08:22:06.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.143+0000 7f05876ee700 1 -- 172.21.15.40:0/2594509053 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05800726c0 con 0x7f058015d510 2023-11-08T08:22:06.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.143+0000 7f0575ffb700 1 -- 172.21.15.40:0/2594509053 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f0578015090 con 0x7f058015d510 2023-11-08T08:22:06.151 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:22:06.152 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1549516710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1549516710},{"type":"v1","addr":"172.21.15.169:6837","nonce":1549516710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5728,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/912870125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":912870125},{"type":"v1","addr":"172.21.15.169:6835","nonce":912870125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5772,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1045606427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1045606427},{"type":"v1","addr":"172.21.15.40:6834","nonce":1045606427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:21:17.730002+0000","modified":"2023-11-08T08:21:20.752976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2471311394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2471311394},{"type":"v1","addr":"172.21.15.40:6835","nonce":2471311394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T08:22:06.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.147+0000 7f05876ee700 1 -- 172.21.15.40:0/2594509053 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0564064f30 msgr2=0x7f05640673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:06.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.147+0000 7f05876ee700 1 --2- 172.21.15.40:0/2594509053 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0564064f30 0x7f05640673e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f057c001560 tx=0x7f057c009040 comp rx=0 tx=0).stop 2023-11-08T08:22:06.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.147+0000 7f05876ee700 1 -- 172.21.15.40:0/2594509053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f058015d510 msgr2=0x7f05801f5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:06.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.147+0000 7f05876ee700 1 --2- 172.21.15.40:0/2594509053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f058015d510 0x7f05801f5100 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0578016000 tx=0x7f0578004770 comp rx=0 tx=0).stop 2023-11-08T08:22:06.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.147+0000 7f05876ee700 1 -- 172.21.15.40:0/2594509053 shutdown_connections 2023-11-08T08:22:06.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.147+0000 7f05876ee700 1 --2- 172.21.15.40:0/2594509053 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f0564064f30 0x7f05640673e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:06.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.147+0000 7f05876ee700 1 --2- 172.21.15.40:0/2594509053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f058015d510 0x7f05801f5100 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:06.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.147+0000 7f05876ee700 1 --2- 172.21.15.40:0/2594509053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05801534a0 0x7f058014c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:06.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.147+0000 7f05876ee700 1 --2- 172.21.15.40:0/2594509053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05800b6200 0x7f05801f13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:06.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.147+0000 7f05876ee700 1 -- 172.21.15.40:0/2594509053 >> 172.21.15.40:0/2594509053 conn(0x7f058006bfc0 msgr2=0x7f05800d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:06.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.147+0000 7f05876ee700 1 -- 172.21.15.40:0/2594509053 shutdown_connections 2023-11-08T08:22:06.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.147+0000 7f05876ee700 1 -- 172.21.15.40:0/2594509053 wait complete. 2023-11-08T08:22:06.158 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 25 2023-11-08T08:22:06.178 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:22:06.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.351+0000 7f6d2b59e700 1 Processor -- start 2023-11-08T08:22:06.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.351+0000 7f6d2b59e700 1 -- start start 2023-11-08T08:22:06.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d2b59e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6d2c070080 0x7f6d2c0704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:06.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d2b59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2c164250 0x7f6d2c1646d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:06.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d2b59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d2c0ad6c0 con 0x7f6d2c160710 2023-11-08T08:22:06.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d2b59e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d2c06b1c0 con 0x7f6d2c070080 2023-11-08T08:22:06.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d2b59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c0adc60 con 0x7f6d2c164250 2023-11-08T08:22:06.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d2a59c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6d2c160710 0x7f6d2c0cc320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41946/0 (socket says 172.21.15.40:41946) 2023-11-08T08:22:06.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d2a59c700 1 -- 172.21.15.40:0/1959757792 learned_addr learned my addr 172.21.15.40:0/1959757792 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:06.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d21d9b700 1 --2- 172.21.15.40:0/1959757792 >> v2:172.21.15.169:3300/0 conn(0x7f6d2c070080 0x7f6d2c0704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:06.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d2ad9d700 1 --2- 172.21.15.40:0/1959757792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2c164250 0x7f6d2c1646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:06.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d29d9b700 1 -- 172.21.15.40:0/1959757792 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2859717158 0 0) 0x7f6d2c0ad6c0 con 0x7f6d2c160710 2023-11-08T08:22:06.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d29d9b700 1 -- 172.21.15.40:0/1959757792 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d180035d0 con 0x7f6d2c160710 2023-11-08T08:22:06.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d29d9b700 1 -- 172.21.15.40:0/1959757792 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1185065031 0 0) 0x7f6d180035d0 con 0x7f6d2c160710 2023-11-08T08:22:06.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d29d9b700 1 -- 172.21.15.40:0/1959757792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2c164250 msgr2=0x7f6d2c1646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:06.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d29d9b700 1 --2- 172.21.15.40:0/1959757792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2c164250 0x7f6d2c1646d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:06.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d29d9b700 1 -- 172.21.15.40:0/1959757792 >> v2:172.21.15.169:3300/0 conn(0x7f6d2c070080 msgr2=0x7f6d2c0704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:06.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d29d9b700 1 --2- 172.21.15.40:0/1959757792 >> v2:172.21.15.169:3300/0 conn(0x7f6d2c070080 0x7f6d2c0704b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:06.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d29d9b700 1 -- 172.21.15.40:0/1959757792 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2c0709f0 con 0x7f6d2c160710 2023-11-08T08:22:06.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d29d9b700 1 -- 172.21.15.40:0/1959757792 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f6d1c001670 con 0x7f6d2c160710 2023-11-08T08:22:06.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d29d9b700 1 -- 172.21.15.40:0/1959757792 >> v1:172.21.15.40:6789/0 conn(0x7f6d2c160710 legacy=0x7f6d2c0cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:06.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d29d9b700 1 --2- 172.21.15.40:0/1959757792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d18003a20 0x7f6d18003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:06.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d29d9b700 1 --2- 172.21.15.40:0/1959757792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d18004450 0x7f6d18004910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:06.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d29d9b700 1 --2- 172.21.15.40:0/1959757792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2c164250 0x7f6d18005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:06.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d29d9b700 1 -- 172.21.15.40:0/1959757792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c06b1c0 con 0x7f6d18003a20 2023-11-08T08:22:06.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d29d9b700 1 -- 172.21.15.40:0/1959757792 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c0adc60 con 0x7f6d18004450 2023-11-08T08:22:06.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d29d9b700 1 -- 172.21.15.40:0/1959757792 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1800ba30 con 0x7f6d2c164250 2023-11-08T08:22:06.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d2a59c700 1 --2- 172.21.15.40:0/1959757792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d18003a20 0x7f6d18003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:06.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d21d9b700 1 --2- 172.21.15.40:0/1959757792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d18004450 0x7f6d18004910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:06.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d2a59c700 1 -- 172.21.15.40:0/1959757792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2c164250 msgr2=0x7f6d18005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:06.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d2a59c700 1 --2- 172.21.15.40:0/1959757792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2c164250 0x7f6d18005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:06.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d2a59c700 1 -- 172.21.15.40:0/1959757792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d18004450 msgr2=0x7f6d18004910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:06.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.355+0000 7f6d2a59c700 1 --2- 172.21.15.40:0/1959757792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d18004450 0x7f6d18004910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:06.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.359+0000 7f6d2a59c700 1 -- 172.21.15.40:0/1959757792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d180035d0 con 0x7f6d18003a20 2023-11-08T08:22:06.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.359+0000 7f6d2a59c700 1 --2- 172.21.15.40:0/1959757792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d18003a20 0x7f6d18003e30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00c320 tx=0x7f6d180058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99847d632c789247 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:06.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.359+0000 7f6d29d9b700 1 -- 172.21.15.40:0/1959757792 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c00ce60 con 0x7f6d18003a20 2023-11-08T08:22:06.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.359+0000 7f6d29d9b700 1 -- 172.21.15.40:0/1959757792 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1c005980 con 0x7f6d18003a20 2023-11-08T08:22:06.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.359+0000 7f6d29d9b700 1 -- 172.21.15.40:0/1959757792 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c005d10 con 0x7f6d18003a20 2023-11-08T08:22:06.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.359+0000 7f6d2b59e700 1 -- 172.21.15.40:0/1959757792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d18003a20 msgr2=0x7f6d18003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:06.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.359+0000 7f6d2b59e700 1 --2- 172.21.15.40:0/1959757792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d18003a20 0x7f6d18003e30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00c320 tx=0x7f6d180058a0 comp rx=0 tx=0).stop 2023-11-08T08:22:06.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.359+0000 7f6d2ad9d700 1 -- 172.21.15.40:0/1959757792 reap_dead start 2023-11-08T08:22:06.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.359+0000 7f6d2b59e700 1 -- 172.21.15.40:0/1959757792 shutdown_connections 2023-11-08T08:22:06.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.359+0000 7f6d2b59e700 1 -- 172.21.15.40:0/1959757792 >> 172.21.15.40:0/1959757792 conn(0x7f6d2c0b0f80 msgr2=0x7f6d2c0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:06.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.359+0000 7f6d2b59e700 1 -- 172.21.15.40:0/1959757792 shutdown_connections 2023-11-08T08:22:06.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.359+0000 7f6d2b59e700 1 -- 172.21.15.40:0/1959757792 wait complete. 2023-11-08T08:22:06.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.363+0000 7f6d2b59e700 1 Processor -- start 2023-11-08T08:22:06.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.363+0000 7f6d2b59e700 1 -- start start 2023-11-08T08:22:06.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.363+0000 7f6d2b59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d18003a20 0x7f6d2c159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:06.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.363+0000 7f6d2b59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d2c15f5a0 0x7f6d2c0c2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:06.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.363+0000 7f6d2b59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2c0b2690 0x7f6d2c0b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:06.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.363+0000 7f6d2b59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c0b3040 con 0x7f6d18003a20 2023-11-08T08:22:06.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.363+0000 7f6d2b59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c15d910 con 0x7f6d2c15f5a0 2023-11-08T08:22:06.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.363+0000 7f6d2b59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c15a020 con 0x7f6d2c0b2690 2023-11-08T08:22:06.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.363+0000 7f6d2a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d18003a20 0x7f6d2c159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:06.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.367+0000 7f6d2a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d18003a20 0x7f6d2c159ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56718/0 (socket says 172.21.15.40:56718) 2023-11-08T08:22:06.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.367+0000 7f6d2a59c700 1 -- 172.21.15.40:0/3348248404 learned_addr learned my addr 172.21.15.40:0/3348248404 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:06.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.367+0000 7f6d21d9b700 1 --2- 172.21.15.40:0/3348248404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d2c15f5a0 0x7f6d2c0c2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:06.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.367+0000 7f6d2ad9d700 1 --2- 172.21.15.40:0/3348248404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2c0b2690 0x7f6d2c0b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:06.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.367+0000 7f6d2a59c700 1 -- 172.21.15.40:0/3348248404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2c0b2690 msgr2=0x7f6d2c0b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:06.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.367+0000 7f6d2a59c700 1 --2- 172.21.15.40:0/3348248404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2c0b2690 0x7f6d2c0b2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:06.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.367+0000 7f6d2a59c700 1 -- 172.21.15.40:0/3348248404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d2c15f5a0 msgr2=0x7f6d2c0c2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:06.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.367+0000 7f6d2a59c700 1 --2- 172.21.15.40:0/3348248404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d2c15f5a0 0x7f6d2c0c2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:06.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.367+0000 7f6d2a59c700 1 -- 172.21.15.40:0/3348248404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2c06b3d0 con 0x7f6d18003a20 2023-11-08T08:22:06.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.367+0000 7f6d2ad9d700 1 --2- 172.21.15.40:0/3348248404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2c0b2690 0x7f6d2c0b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:22:06.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.367+0000 7f6d2a59c700 1 --2- 172.21.15.40:0/3348248404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d18003a20 0x7f6d2c159ae0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00c910 tx=0x7f6d1c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:06.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.367+0000 7f6d237fe700 1 -- 172.21.15.40:0/3348248404 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c005e90 con 0x7f6d18003a20 2023-11-08T08:22:06.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.367+0000 7f6d237fe700 1 -- 172.21.15.40:0/3348248404 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1c001680 con 0x7f6d18003a20 2023-11-08T08:22:06.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.367+0000 7f6d2b59e700 1 -- 172.21.15.40:0/3348248404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d2c0cca30 con 0x7f6d18003a20 2023-11-08T08:22:06.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.367+0000 7f6d237fe700 1 -- 172.21.15.40:0/3348248404 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c004cc0 con 0x7f6d18003a20 2023-11-08T08:22:06.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.367+0000 7f6d2b59e700 1 -- 172.21.15.40:0/3348248404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d2c06b6b0 con 0x7f6d18003a20 2023-11-08T08:22:06.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.367+0000 7f6d2b59e700 1 -- 172.21.15.40:0/3348248404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d2c073cb0 con 0x7f6d18003a20 2023-11-08T08:22:06.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.371+0000 7f6d237fe700 1 -- 172.21.15.40:0/3348248404 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f6d1c010070 con 0x7f6d18003a20 2023-11-08T08:22:06.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.371+0000 7f6d237fe700 1 --2- 172.21.15.40:0/3348248404 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6d0c064ee0 0x7f6d0c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:06.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.371+0000 7f6d237fe700 1 -- 172.21.15.40:0/3348248404 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6d1c082d90 con 0x7f6d18003a20 2023-11-08T08:22:06.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.371+0000 7f6d21d9b700 1 --2- 172.21.15.40:0/3348248404 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6d0c064ee0 0x7f6d0c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:06.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.375+0000 7f6d21d9b700 1 --2- 172.21.15.40:0/3348248404 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6d0c064ee0 0x7f6d0c067390 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c163cc0 tx=0x7f6d14008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:06.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.379+0000 7f6d237fe700 1 -- 172.21.15.40:0/3348248404 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f6d1c050dc0 con 0x7f6d18003a20 2023-11-08T08:22:06.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.555+0000 7f6d2b59e700 1 -- 172.21.15.40:0/3348248404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d2c15bed0 con 0x7f6d18003a20 2023-11-08T08:22:06.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.555+0000 7f6d237fe700 1 -- 172.21.15.40:0/3348248404 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+18084 (secure 0 0 0) 0x7f6d1c0543e0 con 0x7f6d18003a20 2023-11-08T08:22:06.565 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:22:06.565 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:21:18.736039+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:21:15.464908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-08T08:21:16.755373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:22:06.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.559+0000 7f6d2b59e700 1 -- 172.21.15.40:0/3348248404 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6d0c064ee0 msgr2=0x7f6d0c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:06.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.559+0000 7f6d2b59e700 1 --2- 172.21.15.40:0/3348248404 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6d0c064ee0 0x7f6d0c067390 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c163cc0 tx=0x7f6d14008040 comp rx=0 tx=0).stop 2023-11-08T08:22:06.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.559+0000 7f6d2b59e700 1 -- 172.21.15.40:0/3348248404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d18003a20 msgr2=0x7f6d2c159ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:06.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.559+0000 7f6d2b59e700 1 --2- 172.21.15.40:0/3348248404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d18003a20 0x7f6d2c159ae0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00c910 tx=0x7f6d1c001a00 comp rx=0 tx=0).stop 2023-11-08T08:22:06.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.559+0000 7f6d2b59e700 1 -- 172.21.15.40:0/3348248404 shutdown_connections 2023-11-08T08:22:06.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.559+0000 7f6d2b59e700 1 --2- 172.21.15.40:0/3348248404 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6d0c064ee0 0x7f6d0c067390 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:06.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.559+0000 7f6d2b59e700 1 --2- 172.21.15.40:0/3348248404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2c0b2690 0x7f6d2c0b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:06.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.559+0000 7f6d2b59e700 1 --2- 172.21.15.40:0/3348248404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d2c15f5a0 0x7f6d2c0c2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:06.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.559+0000 7f6d2b59e700 1 --2- 172.21.15.40:0/3348248404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d18003a20 0x7f6d2c159ae0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.559+0000 7f6d2b59e700 1 -- 172.21.15.40:0/3348248404 >> 172.21.15.40:0/3348248404 conn(0x7f6d2c0b0f80 msgr2=0x7f6d2c15ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.559+0000 7f6d2b59e700 1 -- 172.21.15.40:0/3348248404 shutdown_connections 2023-11-08T08:22:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.559+0000 7f6d2b59e700 1 -- 172.21.15.40:0/3348248404 wait complete. 2023-11-08T08:22:06.591 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T08:22:06.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.767+0000 7ff278a8c700 1 Processor -- start 2023-11-08T08:22:06.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.767+0000 7ff278a8c700 1 -- start start 2023-11-08T08:22:06.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.771+0000 7ff278a8c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff274070080 0x7ff2740704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:06.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.771+0000 7ff278a8c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2740ad6c0 con 0x7ff274164250 2023-11-08T08:22:06.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.771+0000 7ff278a8c700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2740adc60 con 0x7ff274160710 2023-11-08T08:22:06.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.771+0000 7ff278a8c700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff27406b1c0 con 0x7ff274070080 2023-11-08T08:22:06.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.771+0000 7ff26bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff274070080 0x7ff2740704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:06.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.771+0000 7ff26bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff274070080 0x7ff2740704b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56734/0 (socket says 172.21.15.40:56734) 2023-11-08T08:22:06.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.771+0000 7ff26bfff700 1 -- 172.21.15.40:0/955379000 learned_addr learned my addr 172.21.15.40:0/955379000 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:06.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.771+0000 7ff273fff700 1 --1- 172.21.15.40:0/955379000 >> v1:172.21.15.40:6789/0 conn(0x7ff274164250 0x7ff2741646b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41956/0 (socket says 172.21.15.40:41956) 2023-11-08T08:22:06.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.771+0000 7ff2737fe700 1 --1- 172.21.15.40:0/955379000 >> v1:172.21.15.169:6789/0 conn(0x7ff274160710 0x7ff2740cc320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:37890/0 (socket says 172.21.15.40:37890) 2023-11-08T08:22:06.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.771+0000 7ff26bfff700 1 -- 172.21.15.40:0/955379000 >> v1:172.21.15.169:6789/0 conn(0x7ff274160710 legacy=0x7ff2740cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:06.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.771+0000 7ff26bfff700 1 -- 172.21.15.40:0/955379000 >> v1:172.21.15.40:6789/0 conn(0x7ff274164250 legacy=0x7ff2741646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:06.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.771+0000 7ff26bfff700 1 -- 172.21.15.40:0/955379000 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2740709f0 con 0x7ff274070080 2023-11-08T08:22:06.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.771+0000 7ff26bfff700 1 --2- 172.21.15.40:0/955379000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff274070080 0x7ff2740704b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009910 tx=0x7ff25c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13187cc70d25e18 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:06.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.771+0000 7ff272ffd700 1 -- 172.21.15.40:0/955379000 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c00ba60 con 0x7ff274070080 2023-11-08T08:22:06.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.771+0000 7ff272ffd700 1 -- 172.21.15.40:0/955379000 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25c00bbc0 con 0x7ff274070080 2023-11-08T08:22:06.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.771+0000 7ff272ffd700 1 -- 172.21.15.40:0/955379000 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c009e60 con 0x7ff274070080 2023-11-08T08:22:06.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.775+0000 7ff278a8c700 1 -- 172.21.15.40:0/955379000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff274070080 msgr2=0x7ff2740704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:06.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.775+0000 7ff278a8c700 1 --2- 172.21.15.40:0/955379000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff274070080 0x7ff2740704b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009910 tx=0x7ff25c009c20 comp rx=0 tx=0).stop 2023-11-08T08:22:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.775+0000 7ff278a8c700 1 -- 172.21.15.40:0/955379000 shutdown_connections 2023-11-08T08:22:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.775+0000 7ff278a8c700 1 --2- 172.21.15.40:0/955379000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff274070080 0x7ff2740704b0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.775+0000 7ff278a8c700 1 -- 172.21.15.40:0/955379000 >> 172.21.15.40:0/955379000 conn(0x7ff2740b0f80 msgr2=0x7ff2740ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:06.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.775+0000 7ff278a8c700 1 -- 172.21.15.40:0/955379000 shutdown_connections 2023-11-08T08:22:06.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.775+0000 7ff278a8c700 1 -- 172.21.15.40:0/955379000 wait complete. 2023-11-08T08:22:06.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.775+0000 7ff278a8c700 1 Processor -- start 2023-11-08T08:22:06.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.775+0000 7ff278a8c700 1 -- start start 2023-11-08T08:22:06.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff278a8c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff274070080 0x7ff2741614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:06.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff278a8c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff274160710 0x7ff27415ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:06.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff278a8c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff274164250 0x7ff2740c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:06.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff278a8c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27415e240 con 0x7ff274160710 2023-11-08T08:22:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff278a8c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2741f61b0 con 0x7ff274164250 2023-11-08T08:22:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff278a8c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2741620b0 con 0x7ff274070080 2023-11-08T08:22:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff26bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff274160710 0x7ff27415ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff26bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff274160710 0x7ff27415ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56744/0 (socket says 172.21.15.40:56744) 2023-11-08T08:22:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff26bfff700 1 -- 172.21.15.40:0/2819475855 learned_addr learned my addr 172.21.15.40:0/2819475855 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:06.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff2737fe700 1 --2- 172.21.15.40:0/2819475855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff274070080 0x7ff2741614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:06.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff273fff700 1 --2- 172.21.15.40:0/2819475855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff274164250 0x7ff2740c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:06.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff26bfff700 1 -- 172.21.15.40:0/2819475855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff274070080 msgr2=0x7ff2741614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:06.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff26bfff700 1 --2- 172.21.15.40:0/2819475855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff274070080 0x7ff2741614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:06.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff26bfff700 1 -- 172.21.15.40:0/2819475855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff274164250 msgr2=0x7ff2740c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:06.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff26bfff700 1 --2- 172.21.15.40:0/2819475855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff274164250 0x7ff2740c39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:06.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff26bfff700 1 -- 172.21.15.40:0/2819475855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c009570 con 0x7ff274160710 2023-11-08T08:22:06.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff273fff700 1 --2- 172.21.15.40:0/2819475855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff274164250 0x7ff2740c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:22:06.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff26bfff700 1 --2- 172.21.15.40:0/2819475855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff274160710 0x7ff27415ae40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff25c000c00 tx=0x7ff25c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:06.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff270ff9700 1 -- 172.21.15.40:0/2819475855 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c004550 con 0x7ff274160710 2023-11-08T08:22:06.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff270ff9700 1 -- 172.21.15.40:0/2819475855 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25c01f070 con 0x7ff274160710 2023-11-08T08:22:06.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff270ff9700 1 -- 172.21.15.40:0/2819475855 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c003900 con 0x7ff274160710 2023-11-08T08:22:06.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff278a8c700 1 -- 172.21.15.40:0/2819475855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27406b3d0 con 0x7ff274160710 2023-11-08T08:22:06.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.779+0000 7ff278a8c700 1 -- 172.21.15.40:0/2819475855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2740cca30 con 0x7ff274160710 2023-11-08T08:22:06.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.783+0000 7ff278a8c700 1 -- 172.21.15.40:0/2819475855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff27406b6b0 con 0x7ff274160710 2023-11-08T08:22:06.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.783+0000 7ff270ff9700 1 -- 172.21.15.40:0/2819475855 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff25c00f050 con 0x7ff274160710 2023-11-08T08:22:06.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.783+0000 7ff270ff9700 1 --2- 172.21.15.40:0/2819475855 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff254064f10 0x7ff2540673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:06.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.783+0000 7ff270ff9700 1 -- 172.21.15.40:0/2819475855 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff25c021070 con 0x7ff274160710 2023-11-08T08:22:06.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.787+0000 7ff2737fe700 1 --2- 172.21.15.40:0/2819475855 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff254064f10 0x7ff2540673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:06.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.787+0000 7ff2737fe700 1 --2- 172.21.15.40:0/2819475855 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff254064f10 0x7ff2540673c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff264000f60 tx=0x7ff264007ef0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:06.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:06.791+0000 7ff270ff9700 1 -- 172.21.15.40:0/2819475855 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ff25c051930 con 0x7ff274160710 2023-11-08T08:22:07.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:07.003+0000 7ff278a8c700 1 -- 172.21.15.40:0/2819475855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff274162220 con 0x7ff274160710 2023-11-08T08:22:07.839 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-11-08T08:22:07.835+0000 7f513abf9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T08:22:07.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:07.839+0000 7ff270ff9700 1 -- 172.21.15.40:0/2819475855 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff25c054f50 con 0x7ff274160710 2023-11-08T08:22:07.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:07.843+0000 7ff278a8c700 1 -- 172.21.15.40:0/2819475855 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff254064f10 msgr2=0x7ff2540673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:07.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:07.843+0000 7ff278a8c700 1 --2- 172.21.15.40:0/2819475855 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff254064f10 0x7ff2540673c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff264000f60 tx=0x7ff264007ef0 comp rx=0 tx=0).stop 2023-11-08T08:22:07.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:07.843+0000 7ff278a8c700 1 -- 172.21.15.40:0/2819475855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff274160710 msgr2=0x7ff27415ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:07.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:07.843+0000 7ff278a8c700 1 --2- 172.21.15.40:0/2819475855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff274160710 0x7ff27415ae40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff25c000c00 tx=0x7ff25c004770 comp rx=0 tx=0).stop 2023-11-08T08:22:07.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:07.843+0000 7ff278a8c700 1 -- 172.21.15.40:0/2819475855 shutdown_connections 2023-11-08T08:22:07.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:07.843+0000 7ff278a8c700 1 --2- 172.21.15.40:0/2819475855 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff254064f10 0x7ff2540673c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:07.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:07.843+0000 7ff278a8c700 1 --2- 172.21.15.40:0/2819475855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff274164250 0x7ff2740c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:07.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:07.843+0000 7ff278a8c700 1 --2- 172.21.15.40:0/2819475855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff274160710 0x7ff27415ae40 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:07.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:07.843+0000 7ff278a8c700 1 --2- 172.21.15.40:0/2819475855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff274070080 0x7ff2741614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:07.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:07.843+0000 7ff278a8c700 1 -- 172.21.15.40:0/2819475855 >> 172.21.15.40:0/2819475855 conn(0x7ff2740b0f80 msgr2=0x7ff2740c4930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:07.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:07.843+0000 7ff278a8c700 1 -- 172.21.15.40:0/2819475855 shutdown_connections 2023-11-08T08:22:07.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:07.843+0000 7ff278a8c700 1 -- 172.21.15.40:0/2819475855 wait complete. 2023-11-08T08:22:07.855 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T08:22:07.863 INFO:tasks.ceph.mds.a.smithi040.stderr: -5483> 2023-11-08T08:21:22.216+0000 7efe52bf5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T08:22:07.869 INFO:tasks.ceph.mds.a.smithi040.stderr: -3016> 2023-11-08T08:21:27.396+0000 7efe4f3ee700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5955=0-4194304@1} caps={5955=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5955 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55ebd23d5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T08:22:07.879 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T08:22:07.929 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2023-11-08T08:22:08.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.059+0000 7f3115a16700 1 Processor -- start 2023-11-08T08:22:08.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.059+0000 7f3115a16700 1 -- start start 2023-11-08T08:22:08.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.059+0000 7f3115a16700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f31100b43a0 0x7f311015e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:08.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.059+0000 7f3115a16700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31100b1640 0x7f3110070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:08.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.059+0000 7f3115a16700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f31100c9040 0x7f31100705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:08.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.059+0000 7f3115a16700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f311015a550 con 0x7f31100b43a0 2023-11-08T08:22:08.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.059+0000 7f3115a16700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f311006b0a0 con 0x7f31100c9040 2023-11-08T08:22:08.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.059+0000 7f3115a16700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31100caac0 con 0x7f31100b1640 2023-11-08T08:22:08.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f310effd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f31100b43a0 0x7f311015e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:08.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f310e7fc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31100b1640 0x7f3110070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:08.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f310f7fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f31100c9040 0x7f31100705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:08.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f310e7fc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31100b1640 0x7f3110070080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:55554/0 (socket says 172.21.15.40:55554) 2023-11-08T08:22:08.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f310e7fc700 1 -- 172.21.15.40:0/3996304654 learned_addr learned my addr 172.21.15.40:0/3996304654 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:08.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f310f7fe700 1 -- 172.21.15.40:0/3996304654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31100b1640 msgr2=0x7f3110070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:08.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f310f7fe700 1 --2- 172.21.15.40:0/3996304654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31100b1640 0x7f3110070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:08.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f310f7fe700 1 -- 172.21.15.40:0/3996304654 >> v2:172.21.15.40:3300/0 conn(0x7f31100b43a0 msgr2=0x7f311015e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:08.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f310f7fe700 1 --2- 172.21.15.40:0/3996304654 >> v2:172.21.15.40:3300/0 conn(0x7f31100b43a0 0x7f311015e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:08.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f310f7fe700 1 -- 172.21.15.40:0/3996304654 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31100cada0 con 0x7f31100c9040 2023-11-08T08:22:08.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f310effd700 1 --2- 172.21.15.40:0/3996304654 >> v2:172.21.15.40:3300/0 conn(0x7f31100b43a0 0x7f311015e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:22:08.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f310f7fe700 1 --2- 172.21.15.40:0/3996304654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31100c9040 0x7f31100705c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f31100c9f60 tx=0x7f310800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9801603624a41ab8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:08.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f310dffb700 1 -- 172.21.15.40:0/3996304654 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310800e070 con 0x7f31100c9040 2023-11-08T08:22:08.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f310dffb700 1 -- 172.21.15.40:0/3996304654 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310800bb20 con 0x7f31100c9040 2023-11-08T08:22:08.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f310dffb700 1 -- 172.21.15.40:0/3996304654 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108012430 con 0x7f31100c9040 2023-11-08T08:22:08.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f3115a16700 1 -- 172.21.15.40:0/3996304654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31100c9040 msgr2=0x7f31100705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:08.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f3115a16700 1 --2- 172.21.15.40:0/3996304654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31100c9040 0x7f31100705c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f31100c9f60 tx=0x7f310800b330 comp rx=0 tx=0).stop 2023-11-08T08:22:08.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f3115a16700 1 -- 172.21.15.40:0/3996304654 shutdown_connections 2023-11-08T08:22:08.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f3115a16700 1 --2- 172.21.15.40:0/3996304654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31100c9040 0x7f31100705c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:08.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f3115a16700 1 --2- 172.21.15.40:0/3996304654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31100b1640 0x7f3110070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:08.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f3115a16700 1 --2- 172.21.15.40:0/3996304654 >> v2:172.21.15.40:3300/0 conn(0x7f31100b43a0 0x7f311015e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:08.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f3115a16700 1 -- 172.21.15.40:0/3996304654 >> 172.21.15.40:0/3996304654 conn(0x7f31100b0f80 msgr2=0x7f31100ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:08.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f3115a16700 1 -- 172.21.15.40:0/3996304654 shutdown_connections 2023-11-08T08:22:08.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.063+0000 7f3115a16700 1 -- 172.21.15.40:0/3996304654 wait complete. 2023-11-08T08:22:08.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.067+0000 7f3115a16700 1 Processor -- start 2023-11-08T08:22:08.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.067+0000 7f3115a16700 1 -- start start 2023-11-08T08:22:08.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.067+0000 7f3115a16700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31100b1640 0x7f311006a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:08.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.067+0000 7f3115a16700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31100b43a0 0x7f31100c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:08.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.067+0000 7f3115a16700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31100c9040 0x7f31100c2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:08.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.067+0000 7f3115a16700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311014aee0 con 0x7f31100b43a0 2023-11-08T08:22:08.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.067+0000 7f3115a16700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31100c2f60 con 0x7f31100b1640 2023-11-08T08:22:08.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.067+0000 7f3115a16700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31100acb50 con 0x7f31100c9040 2023-11-08T08:22:08.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.067+0000 7f310e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31100b43a0 0x7f31100c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:08.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.067+0000 7f310e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31100b43a0 0x7f31100c24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34570/0 (socket says 172.21.15.40:34570) 2023-11-08T08:22:08.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.071+0000 7f310e7fc700 1 -- 172.21.15.40:0/2499606846 learned_addr learned my addr 172.21.15.40:0/2499606846 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:08.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.071+0000 7f310f7fe700 1 --2- 172.21.15.40:0/2499606846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31100c9040 0x7f31100c2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:08.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.071+0000 7f310effd700 1 --2- 172.21.15.40:0/2499606846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31100b1640 0x7f311006a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:08.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.071+0000 7f310e7fc700 1 -- 172.21.15.40:0/2499606846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31100c9040 msgr2=0x7f31100c2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:08.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.071+0000 7f310e7fc700 1 --2- 172.21.15.40:0/2499606846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31100c9040 0x7f31100c2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:08.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.071+0000 7f310e7fc700 1 -- 172.21.15.40:0/2499606846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31100b1640 msgr2=0x7f311006a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:08.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.071+0000 7f310e7fc700 1 --2- 172.21.15.40:0/2499606846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31100b1640 0x7f311006a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:08.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.071+0000 7f310e7fc700 1 -- 172.21.15.40:0/2499606846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31000094b0 con 0x7f31100b43a0 2023-11-08T08:22:08.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.071+0000 7f310e7fc700 1 --2- 172.21.15.40:0/2499606846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31100b43a0 0x7f31100c24e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f310400e8b0 tx=0x7f310400ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:08.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.071+0000 7f30f7fff700 1 -- 172.21.15.40:0/2499606846 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104013680 con 0x7f31100b43a0 2023-11-08T08:22:08.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.071+0000 7f30f7fff700 1 -- 172.21.15.40:0/2499606846 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31040137e0 con 0x7f31100b43a0 2023-11-08T08:22:08.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.071+0000 7f30f7fff700 1 -- 172.21.15.40:0/2499606846 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104013b50 con 0x7f31100b43a0 2023-11-08T08:22:08.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.071+0000 7f3115a16700 1 -- 172.21.15.40:0/2499606846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3108009a70 con 0x7f31100b43a0 2023-11-08T08:22:08.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.071+0000 7f3115a16700 1 -- 172.21.15.40:0/2499606846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311006b5d0 con 0x7f31100b43a0 2023-11-08T08:22:08.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.075+0000 7f3115a16700 1 -- 172.21.15.40:0/2499606846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311006b2f0 con 0x7f31100b43a0 2023-11-08T08:22:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.079+0000 7f30f7fff700 1 -- 172.21.15.40:0/2499606846 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f3104019030 con 0x7f31100b43a0 2023-11-08T08:22:08.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.083+0000 7f30f7fff700 1 --2- 172.21.15.40:0/2499606846 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f30fc064ee0 0x7f30fc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:08.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.083+0000 7f30f7fff700 1 -- 172.21.15.40:0/2499606846 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f3104082640 con 0x7f31100b43a0 2023-11-08T08:22:08.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.083+0000 7f310effd700 1 --2- 172.21.15.40:0/2499606846 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f30fc064ee0 0x7f30fc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:08.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.083+0000 7f310effd700 1 --2- 172.21.15.40:0/2499606846 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f30fc064ee0 0x7f30fc067390 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3100009d80 tx=0x7f3100009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:08.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.087+0000 7f30f7fff700 1 -- 172.21.15.40:0/2499606846 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f3104050570 con 0x7f31100b43a0 2023-11-08T08:22:08.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.303+0000 7f3115a16700 1 -- 172.21.15.40:0/2499606846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f311014b150 con 0x7f31100b43a0 2023-11-08T08:22:08.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.843+0000 7f30f7fff700 1 -- 172.21.15.40:0/2499606846 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3104013cb0 con 0x7f31100b43a0 2023-11-08T08:22:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.851+0000 7f3115a16700 1 -- 172.21.15.40:0/2499606846 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f30fc064ee0 msgr2=0x7f30fc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.851+0000 7f3115a16700 1 --2- 172.21.15.40:0/2499606846 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f30fc064ee0 0x7f30fc067390 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3100009d80 tx=0x7f3100009240 comp rx=0 tx=0).stop 2023-11-08T08:22:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.851+0000 7f3115a16700 1 -- 172.21.15.40:0/2499606846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31100b43a0 msgr2=0x7f31100c24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.851+0000 7f3115a16700 1 --2- 172.21.15.40:0/2499606846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31100b43a0 0x7f31100c24e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f310400e8b0 tx=0x7f310400ebc0 comp rx=0 tx=0).stop 2023-11-08T08:22:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.851+0000 7f3115a16700 1 -- 172.21.15.40:0/2499606846 shutdown_connections 2023-11-08T08:22:08.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.851+0000 7f3115a16700 1 --2- 172.21.15.40:0/2499606846 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f30fc064ee0 0x7f30fc067390 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:08.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.851+0000 7f3115a16700 1 --2- 172.21.15.40:0/2499606846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31100c9040 0x7f31100c2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:08.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.851+0000 7f3115a16700 1 --2- 172.21.15.40:0/2499606846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31100b43a0 0x7f31100c24e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:08.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.851+0000 7f3115a16700 1 --2- 172.21.15.40:0/2499606846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31100b1640 0x7f311006a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:08.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.851+0000 7f3115a16700 1 -- 172.21.15.40:0/2499606846 >> 172.21.15.40:0/2499606846 conn(0x7f31100b0f80 msgr2=0x7f31100ab860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:08.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.851+0000 7f3115a16700 1 -- 172.21.15.40:0/2499606846 shutdown_connections 2023-11-08T08:22:08.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:08.851+0000 7f3115a16700 1 -- 172.21.15.40:0/2499606846 wait complete. 2023-11-08T08:22:08.884 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T08:22:09.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.119+0000 7fded0691700 1 Processor -- start 2023-11-08T08:22:09.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.119+0000 7fded0691700 1 -- start start 2023-11-08T08:22:09.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.123+0000 7fded0691700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdec815c400 0x7fdec8160900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:09.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.123+0000 7fded0691700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdec80b1900 0x7fdec80b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:09.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.123+0000 7fded0691700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdec8069bf0 0x7fdec806a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:09.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.123+0000 7fded0691700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec815c7d0 con 0x7fdec8069bf0 2023-11-08T08:22:09.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.123+0000 7fded0691700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec80cbba0 con 0x7fdec80b1900 2023-11-08T08:22:09.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.123+0000 7fded0691700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec80b2290 con 0x7fdec815c400 2023-11-08T08:22:09.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.123+0000 7fdecdc2c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdec80b1900 0x7fdec80b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:09.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.123+0000 7fdecdc2c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdec80b1900 0x7fdec80b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33046/0 (socket says 172.21.15.40:33046) 2023-11-08T08:22:09.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.123+0000 7fdecdc2c700 1 -- 172.21.15.40:0/2484649159 learned_addr learned my addr 172.21.15.40:0/2484649159 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:09.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.123+0000 7fdecec2e700 1 --2- 172.21.15.40:0/2484649159 >> v2:172.21.15.40:3300/0 conn(0x7fdec8069bf0 0x7fdec806a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:09.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.123+0000 7fdece42d700 1 --2- 172.21.15.40:0/2484649159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdec815c400 0x7fdec8160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:09.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.123+0000 7fdecdc2c700 1 -- 172.21.15.40:0/2484649159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdec815c400 msgr2=0x7fdec8160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:09.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.123+0000 7fdecdc2c700 1 --2- 172.21.15.40:0/2484649159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdec815c400 0x7fdec8160900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:09.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.123+0000 7fdecdc2c700 1 -- 172.21.15.40:0/2484649159 >> v2:172.21.15.40:3300/0 conn(0x7fdec8069bf0 msgr2=0x7fdec806a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:09.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.123+0000 7fdecdc2c700 1 --2- 172.21.15.40:0/2484649159 >> v2:172.21.15.40:3300/0 conn(0x7fdec8069bf0 0x7fdec806a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:09.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.123+0000 7fdecdc2c700 1 -- 172.21.15.40:0/2484649159 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec8160e40 con 0x7fdec80b1900 2023-11-08T08:22:09.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.123+0000 7fdecdc2c700 1 --2- 172.21.15.40:0/2484649159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdec80b1900 0x7fdec80b1d50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdec806cfd0 tx=0x7fdeb8009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93c73f132ce1579e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:09.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.123+0000 7fdecd42b700 1 -- 172.21.15.40:0/2484649159 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8011830 con 0x7fdec80b1900 2023-11-08T08:22:09.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.123+0000 7fdecd42b700 1 -- 172.21.15.40:0/2484649159 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb8011990 con 0x7fdec80b1900 2023-11-08T08:22:09.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.123+0000 7fdecd42b700 1 -- 172.21.15.40:0/2484649159 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8011d20 con 0x7fdec80b1900 2023-11-08T08:22:09.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.127+0000 7fded0691700 1 -- 172.21.15.40:0/2484649159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdec80b1900 msgr2=0x7fdec80b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:09.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.127+0000 7fded0691700 1 --2- 172.21.15.40:0/2484649159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdec80b1900 0x7fdec80b1d50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdec806cfd0 tx=0x7fdeb8009bf0 comp rx=0 tx=0).stop 2023-11-08T08:22:09.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.127+0000 7fded0691700 1 -- 172.21.15.40:0/2484649159 shutdown_connections 2023-11-08T08:22:09.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.127+0000 7fded0691700 1 --2- 172.21.15.40:0/2484649159 >> v2:172.21.15.40:3300/0 conn(0x7fdec8069bf0 0x7fdec806a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:09.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.127+0000 7fded0691700 1 --2- 172.21.15.40:0/2484649159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdec80b1900 0x7fdec80b1d50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:09.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.127+0000 7fded0691700 1 --2- 172.21.15.40:0/2484649159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdec815c400 0x7fdec8160900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:09.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.127+0000 7fded0691700 1 -- 172.21.15.40:0/2484649159 >> 172.21.15.40:0/2484649159 conn(0x7fdec806bfc0 msgr2=0x7fdec806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:09.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.127+0000 7fded0691700 1 -- 172.21.15.40:0/2484649159 shutdown_connections 2023-11-08T08:22:09.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.127+0000 7fded0691700 1 -- 172.21.15.40:0/2484649159 wait complete. 2023-11-08T08:22:09.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.127+0000 7fded0691700 1 Processor -- start 2023-11-08T08:22:09.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fded0691700 1 -- start start 2023-11-08T08:22:09.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fded0691700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdec8069bf0 0x7fdec81f1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:09.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fded0691700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdec80b1900 0x7fdec80cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:09.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fded0691700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdec815c400 0x7fdec80cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:09.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fded0691700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec814ee60 con 0x7fdec8069bf0 2023-11-08T08:22:09.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fded0691700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec81f1a70 con 0x7fdec80b1900 2023-11-08T08:22:09.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fded0691700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec8150e60 con 0x7fdec815c400 2023-11-08T08:22:09.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fdece42d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdec8069bf0 0x7fdec81f1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:09.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fdece42d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdec8069bf0 0x7fdec81f1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34578/0 (socket says 172.21.15.40:34578) 2023-11-08T08:22:09.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fdece42d700 1 -- 172.21.15.40:0/831915903 learned_addr learned my addr 172.21.15.40:0/831915903 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:09.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fdecdc2c700 1 --2- 172.21.15.40:0/831915903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdec80b1900 0x7fdec80cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:09.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fdecec2e700 1 --2- 172.21.15.40:0/831915903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdec815c400 0x7fdec80cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:09.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fdece42d700 1 -- 172.21.15.40:0/831915903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdec815c400 msgr2=0x7fdec80cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:09.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fdece42d700 1 --2- 172.21.15.40:0/831915903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdec815c400 0x7fdec80cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:09.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fdece42d700 1 -- 172.21.15.40:0/831915903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdec80b1900 msgr2=0x7fdec80cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:09.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fdece42d700 1 --2- 172.21.15.40:0/831915903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdec80b1900 0x7fdec80cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:09.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fdece42d700 1 -- 172.21.15.40:0/831915903 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb8009570 con 0x7fdec8069bf0 2023-11-08T08:22:09.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fdecdc2c700 1 --2- 172.21.15.40:0/831915903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdec80b1900 0x7fdec80cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:22:09.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fdece42d700 1 --2- 172.21.15.40:0/831915903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdec8069bf0 0x7fdec81f1530 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdec400e760 tx=0x7fdec400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:09.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fdebeffd700 1 -- 172.21.15.40:0/831915903 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec4013610 con 0x7fdec8069bf0 2023-11-08T08:22:09.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fdebeffd700 1 -- 172.21.15.40:0/831915903 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec4013770 con 0x7fdec8069bf0 2023-11-08T08:22:09.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fdebeffd700 1 -- 172.21.15.40:0/831915903 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec4013ae0 con 0x7fdec8069bf0 2023-11-08T08:22:09.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fded0691700 1 -- 172.21.15.40:0/831915903 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec80cb5e0 con 0x7fdec8069bf0 2023-11-08T08:22:09.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.131+0000 7fded0691700 1 -- 172.21.15.40:0/831915903 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec80cb300 con 0x7fdec8069bf0 2023-11-08T08:22:09.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.135+0000 7fded0691700 1 -- 172.21.15.40:0/831915903 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec80cb8c0 con 0x7fdec8069bf0 2023-11-08T08:22:09.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.135+0000 7fdebeffd700 1 -- 172.21.15.40:0/831915903 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fdec4023070 con 0x7fdec8069bf0 2023-11-08T08:22:09.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.139+0000 7fdebeffd700 1 --2- 172.21.15.40:0/831915903 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdeac064ee0 0x7fdeac067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:09.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.139+0000 7fdebeffd700 1 -- 172.21.15.40:0/831915903 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6468+0+0 (secure 0 0 0) 0x7fdec4081d90 con 0x7fdec8069bf0 2023-11-08T08:22:09.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.139+0000 7fdecdc2c700 1 --2- 172.21.15.40:0/831915903 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdeac064ee0 0x7fdeac067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:09.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.139+0000 7fdecdc2c700 1 --2- 172.21.15.40:0/831915903 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdeac064ee0 0x7fdeac067390 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8000c00 tx=0x7fdeb8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:09.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.143+0000 7fdebeffd700 1 -- 172.21.15.40:0/831915903 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fdec404fca0 con 0x7fdec8069bf0 2023-11-08T08:22:09.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:09.319+0000 7fded0691700 1 -- 172.21.15.40:0/831915903 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdec80726c0 con 0x7fdec8069bf0 2023-11-08T08:22:10.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.095+0000 7fdebeffd700 1 -- 172.21.15.40:0/831915903 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v59) v1 ==== 181+0+0 (secure 0 0 0) 0x7fdec40532c0 con 0x7fdec8069bf0 2023-11-08T08:22:10.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.099+0000 7fded0691700 1 -- 172.21.15.40:0/831915903 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdeac064ee0 msgr2=0x7fdeac067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:10.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.099+0000 7fded0691700 1 --2- 172.21.15.40:0/831915903 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdeac064ee0 0x7fdeac067390 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8000c00 tx=0x7fdeb8008030 comp rx=0 tx=0).stop 2023-11-08T08:22:10.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.099+0000 7fded0691700 1 -- 172.21.15.40:0/831915903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdec8069bf0 msgr2=0x7fdec81f1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:10.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.099+0000 7fded0691700 1 --2- 172.21.15.40:0/831915903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdec8069bf0 0x7fdec81f1530 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdec400e760 tx=0x7fdec400eb20 comp rx=0 tx=0).stop 2023-11-08T08:22:10.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.103+0000 7fded0691700 1 -- 172.21.15.40:0/831915903 shutdown_connections 2023-11-08T08:22:10.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.103+0000 7fded0691700 1 --2- 172.21.15.40:0/831915903 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdeac064ee0 0x7fdeac067390 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:10.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.103+0000 7fded0691700 1 --2- 172.21.15.40:0/831915903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdec815c400 0x7fdec80cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:10.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.103+0000 7fded0691700 1 --2- 172.21.15.40:0/831915903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdec80b1900 0x7fdec80cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:10.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.103+0000 7fded0691700 1 --2- 172.21.15.40:0/831915903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdec8069bf0 0x7fdec81f1530 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:10.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.103+0000 7fded0691700 1 -- 172.21.15.40:0/831915903 >> 172.21.15.40:0/831915903 conn(0x7fdec806bfc0 msgr2=0x7fdec8156470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:10.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.103+0000 7fded0691700 1 -- 172.21.15.40:0/831915903 shutdown_connections 2023-11-08T08:22:10.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.103+0000 7fded0691700 1 -- 172.21.15.40:0/831915903 wait complete. 2023-11-08T08:22:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2023-11-08T08:22:10.137 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T08:22:10.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.307+0000 7f4d41fc3700 1 Processor -- start 2023-11-08T08:22:10.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.307+0000 7f4d41fc3700 1 -- start start 2023-11-08T08:22:10.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.311+0000 7f4d41fc3700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d3c15c5f0 0x7f4d3c0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:10.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.311+0000 7f4d41fc3700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4d3c160ba0 0x7f4d3c0b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:10.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.311+0000 7f4d41fc3700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d3c0c9c10 con 0x7f4d3c0cc060 2023-11-08T08:22:10.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.311+0000 7f4d41fc3700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d3c0b55c0 con 0x7f4d3c160ba0 2023-11-08T08:22:10.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.311+0000 7f4d41fc3700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c0cde60 con 0x7f4d3c15c5f0 2023-11-08T08:22:10.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.311+0000 7f4d417c2700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4d3c160ba0 0x7f4d3c0b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:10.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.311+0000 7f4d33fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d3c15c5f0 0x7f4d3c0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:10.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.311+0000 7f4d417c2700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4d3c160ba0 0x7f4d3c0b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33058/0 (socket says 172.21.15.40:33058) 2023-11-08T08:22:10.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.311+0000 7f4d417c2700 1 -- 172.21.15.40:0/1376678075 learned_addr learned my addr 172.21.15.40:0/1376678075 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:10.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.311+0000 7f4d40fc1700 1 --1- 172.21.15.40:0/1376678075 >> v1:172.21.15.169:6789/0 conn(0x7f4d3c0cc060 0x7f4d3c0cc430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:42920/0 (socket says 172.21.15.40:42920) 2023-11-08T08:22:10.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.311+0000 7f4d33fff700 1 -- 172.21.15.40:0/1376678075 >> v2:172.21.15.169:3300/0 conn(0x7f4d3c160ba0 msgr2=0x7f4d3c0b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:10.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.311+0000 7f4d33fff700 1 --2- 172.21.15.40:0/1376678075 >> v2:172.21.15.169:3300/0 conn(0x7f4d3c160ba0 0x7f4d3c0b10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:10.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.311+0000 7f4d33fff700 1 -- 172.21.15.40:0/1376678075 >> v1:172.21.15.169:6789/0 conn(0x7f4d3c0cc060 legacy=0x7f4d3c0cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:10.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.311+0000 7f4d33fff700 1 -- 172.21.15.40:0/1376678075 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c0b4800 con 0x7f4d3c15c5f0 2023-11-08T08:22:10.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.315+0000 7f4d33fff700 1 --2- 172.21.15.40:0/1376678075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d3c15c5f0 0x7f4d3c0cd550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009850 tx=0x7f4d2c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ee3c80c7d37a10b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:10.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.315+0000 7f4d337fe700 1 -- 172.21.15.40:0/1376678075 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c011830 con 0x7f4d3c15c5f0 2023-11-08T08:22:10.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.315+0000 7f4d337fe700 1 -- 172.21.15.40:0/1376678075 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2c011990 con 0x7f4d3c15c5f0 2023-11-08T08:22:10.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.315+0000 7f4d337fe700 1 -- 172.21.15.40:0/1376678075 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c011d40 con 0x7f4d3c15c5f0 2023-11-08T08:22:10.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.315+0000 7f4d41fc3700 1 -- 172.21.15.40:0/1376678075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d3c15c5f0 msgr2=0x7f4d3c0cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:10.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.315+0000 7f4d41fc3700 1 --2- 172.21.15.40:0/1376678075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d3c15c5f0 0x7f4d3c0cd550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009850 tx=0x7f4d2c009b60 comp rx=0 tx=0).stop 2023-11-08T08:22:10.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.315+0000 7f4d41fc3700 1 -- 172.21.15.40:0/1376678075 shutdown_connections 2023-11-08T08:22:10.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.315+0000 7f4d41fc3700 1 --2- 172.21.15.40:0/1376678075 >> v2:172.21.15.169:3300/0 conn(0x7f4d3c160ba0 0x7f4d3c0b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:10.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.315+0000 7f4d41fc3700 1 --2- 172.21.15.40:0/1376678075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d3c15c5f0 0x7f4d3c0cd550 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:10.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.315+0000 7f4d41fc3700 1 -- 172.21.15.40:0/1376678075 >> 172.21.15.40:0/1376678075 conn(0x7f4d3c06e100 msgr2=0x7f4d3c06e500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:10.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.315+0000 7f4d41fc3700 1 -- 172.21.15.40:0/1376678075 shutdown_connections 2023-11-08T08:22:10.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.315+0000 7f4d41fc3700 1 -- 172.21.15.40:0/1376678075 wait complete. 2023-11-08T08:22:10.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.319+0000 7f4d41fc3700 1 Processor -- start 2023-11-08T08:22:10.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.319+0000 7f4d41fc3700 1 -- start start 2023-11-08T08:22:10.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.319+0000 7f4d41fc3700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d3c0cc060 0x7f4d3c1f1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:10.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.319+0000 7f4d41fc3700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d3c15c5f0 0x7f4d3c0c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:10.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.319+0000 7f4d41fc3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3c160ba0 0x7f4d3c0ad320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:10.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.319+0000 7f4d41fc3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c1f1c70 con 0x7f4d3c160ba0 2023-11-08T08:22:10.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.319+0000 7f4d41fc3700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c1f58b0 con 0x7f4d3c15c5f0 2023-11-08T08:22:10.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.319+0000 7f4d41fc3700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c1f5b90 con 0x7f4d3c0cc060 2023-11-08T08:22:10.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.319+0000 7f4d417c2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3c160ba0 0x7f4d3c0ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:10.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.319+0000 7f4d417c2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3c160ba0 0x7f4d3c0ad320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34590/0 (socket says 172.21.15.40:34590) 2023-11-08T08:22:10.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.319+0000 7f4d417c2700 1 -- 172.21.15.40:0/2378455338 learned_addr learned my addr 172.21.15.40:0/2378455338 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:10.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.319+0000 7f4d40fc1700 1 --2- 172.21.15.40:0/2378455338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d3c0cc060 0x7f4d3c1f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:10.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.323+0000 7f4d33fff700 1 --2- 172.21.15.40:0/2378455338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d3c15c5f0 0x7f4d3c0c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:10.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.323+0000 7f4d417c2700 1 -- 172.21.15.40:0/2378455338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d3c0cc060 msgr2=0x7f4d3c1f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:10.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.323+0000 7f4d417c2700 1 --2- 172.21.15.40:0/2378455338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d3c0cc060 0x7f4d3c1f1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:10.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.323+0000 7f4d417c2700 1 -- 172.21.15.40:0/2378455338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d3c15c5f0 msgr2=0x7f4d3c0c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:10.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.323+0000 7f4d417c2700 1 --2- 172.21.15.40:0/2378455338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d3c15c5f0 0x7f4d3c0c31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:10.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.323+0000 7f4d417c2700 1 -- 172.21.15.40:0/2378455338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2c0094b0 con 0x7f4d3c160ba0 2023-11-08T08:22:10.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.323+0000 7f4d417c2700 1 --2- 172.21.15.40:0/2378455338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3c160ba0 0x7f4d3c0ad320 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400efc0 tx=0x7f4d3400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:10.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.323+0000 7f4d317fa700 1 -- 172.21.15.40:0/2378455338 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d34003c40 con 0x7f4d3c160ba0 2023-11-08T08:22:10.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.323+0000 7f4d317fa700 1 -- 172.21.15.40:0/2378455338 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3400f040 con 0x7f4d3c160ba0 2023-11-08T08:22:10.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.323+0000 7f4d317fa700 1 -- 172.21.15.40:0/2378455338 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d34009e90 con 0x7f4d3c160ba0 2023-11-08T08:22:10.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.323+0000 7f4d41fc3700 1 -- 172.21.15.40:0/2378455338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3c0b4dc0 con 0x7f4d3c160ba0 2023-11-08T08:22:10.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.323+0000 7f4d41fc3700 1 -- 172.21.15.40:0/2378455338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3c0b4ae0 con 0x7f4d3c160ba0 2023-11-08T08:22:10.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.323+0000 7f4d41fc3700 1 -- 172.21.15.40:0/2378455338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3c0b50a0 con 0x7f4d3c160ba0 2023-11-08T08:22:10.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.327+0000 7f4d317fa700 1 -- 172.21.15.40:0/2378455338 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4d3400c930 con 0x7f4d3c160ba0 2023-11-08T08:22:10.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.327+0000 7f4d317fa700 1 --2- 172.21.15.40:0/2378455338 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4d28064f10 0x7f4d280673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:10.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.327+0000 7f4d317fa700 1 -- 172.21.15.40:0/2378455338 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6030+0+0 (secure 0 0 0) 0x7f4d340824b0 con 0x7f4d3c160ba0 2023-11-08T08:22:10.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.327+0000 7f4d40fc1700 1 --2- 172.21.15.40:0/2378455338 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4d28064f10 0x7f4d280673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:10.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.331+0000 7f4d40fc1700 1 --2- 172.21.15.40:0/2378455338 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4d28064f10 0x7f4d280673c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4d38000f60 tx=0x7f4d38007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:10.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.331+0000 7f4d317fa700 1 -- 172.21.15.40:0/2378455338 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4d340504f0 con 0x7f4d3c160ba0 2023-11-08T08:22:10.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:10.515+0000 7f4d41fc3700 1 -- 172.21.15.40:0/2378455338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4d3c073cb0 con 0x7f4d3c160ba0 2023-11-08T08:22:11.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.103+0000 7f4d317fa700 1 -- 172.21.15.40:0/2378455338 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v60) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4d34053b10 con 0x7f4d3c160ba0 2023-11-08T08:22:11.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.111+0000 7f4d41fc3700 1 -- 172.21.15.40:0/2378455338 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4d28064f10 msgr2=0x7f4d280673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:11.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.111+0000 7f4d41fc3700 1 --2- 172.21.15.40:0/2378455338 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4d28064f10 0x7f4d280673c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4d38000f60 tx=0x7f4d38007d30 comp rx=0 tx=0).stop 2023-11-08T08:22:11.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.111+0000 7f4d41fc3700 1 -- 172.21.15.40:0/2378455338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3c160ba0 msgr2=0x7f4d3c0ad320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:11.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.111+0000 7f4d41fc3700 1 --2- 172.21.15.40:0/2378455338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3c160ba0 0x7f4d3c0ad320 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400efc0 tx=0x7f4d3400c3a0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.111+0000 7f4d41fc3700 1 -- 172.21.15.40:0/2378455338 shutdown_connections 2023-11-08T08:22:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.111+0000 7f4d41fc3700 1 --2- 172.21.15.40:0/2378455338 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4d28064f10 0x7f4d280673c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.111+0000 7f4d41fc3700 1 --2- 172.21.15.40:0/2378455338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d3c160ba0 0x7f4d3c0ad320 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.111+0000 7f4d41fc3700 1 --2- 172.21.15.40:0/2378455338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d3c15c5f0 0x7f4d3c0c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.111+0000 7f4d41fc3700 1 --2- 172.21.15.40:0/2378455338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d3c0cc060 0x7f4d3c1f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.111+0000 7f4d41fc3700 1 -- 172.21.15.40:0/2378455338 >> 172.21.15.40:0/2378455338 conn(0x7f4d3c06e100 msgr2=0x7f4d3c0b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:11.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.119+0000 7f4d41fc3700 1 -- 172.21.15.40:0/2378455338 shutdown_connections 2023-11-08T08:22:11.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.119+0000 7f4d41fc3700 1 -- 172.21.15.40:0/2378455338 wait complete. 2023-11-08T08:22:11.128 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2023-11-08T08:22:11.149 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-08T08:22:11.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.335+0000 7f9742570700 1 Processor -- start 2023-11-08T08:22:11.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.335+0000 7f9742570700 1 -- start start 2023-11-08T08:22:11.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9742570700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c0704b0 0x7f973c164960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:11.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9742570700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c0ad6c0 con 0x7f973c160710 2023-11-08T08:22:11.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9742570700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c0adc60 con 0x7f973c070080 2023-11-08T08:22:11.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9742570700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c06b1c0 con 0x7f973c0704b0 2023-11-08T08:22:11.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f974156e700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f973c160710 0x7f973c0cc320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50924/0 (socket says 172.21.15.40:50924) 2023-11-08T08:22:11.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f974156e700 1 -- 172.21.15.40:0/2644211491 learned_addr learned my addr 172.21.15.40:0/2644211491 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:11.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9741d6f700 1 --2- 172.21.15.40:0/2644211491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c0704b0 0x7f973c164960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:11.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9733fff700 1 -- 172.21.15.40:0/2644211491 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3841382486 0 0) 0x7f973c0ad6c0 con 0x7f973c160710 2023-11-08T08:22:11.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9733fff700 1 -- 172.21.15.40:0/2644211491 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97280035d0 con 0x7f973c160710 2023-11-08T08:22:11.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9733fff700 1 -- 172.21.15.40:0/2644211491 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3359797507 0 0) 0x7f97280035d0 con 0x7f973c160710 2023-11-08T08:22:11.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9733fff700 1 -- 172.21.15.40:0/2644211491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c0704b0 msgr2=0x7f973c164960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:11.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9733fff700 1 --2- 172.21.15.40:0/2644211491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c0704b0 0x7f973c164960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9733fff700 1 -- 172.21.15.40:0/2644211491 >> v1:172.21.15.169:6789/0 conn(0x7f973c070080 legacy=0x7f973c164250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:11.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9733fff700 1 -- 172.21.15.40:0/2644211491 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c0cca30 con 0x7f973c160710 2023-11-08T08:22:11.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9733fff700 1 -- 172.21.15.40:0/2644211491 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f9738001670 con 0x7f973c160710 2023-11-08T08:22:11.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9733fff700 1 -- 172.21.15.40:0/2644211491 >> v1:172.21.15.40:6789/0 conn(0x7f973c160710 legacy=0x7f973c0cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:11.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9733fff700 1 --2- 172.21.15.40:0/2644211491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9728003a50 0x7f9728003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:11.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9733fff700 1 --2- 172.21.15.40:0/2644211491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c0704b0 0x7f9728004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:11.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9733fff700 1 --2- 172.21.15.40:0/2644211491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9728004db0 0x7f9728005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:11.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9733fff700 1 -- 172.21.15.40:0/2644211491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c06b1c0 con 0x7f9728003a50 2023-11-08T08:22:11.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9733fff700 1 -- 172.21.15.40:0/2644211491 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972800bae0 con 0x7f9728004db0 2023-11-08T08:22:11.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9733fff700 1 -- 172.21.15.40:0/2644211491 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f972800bc20 con 0x7f973c0704b0 2023-11-08T08:22:11.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f974156e700 1 --2- 172.21.15.40:0/2644211491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9728003a50 0x7f9728003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:11.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9741d6f700 1 --2- 172.21.15.40:0/2644211491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9728004db0 0x7f9728005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:11.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.339+0000 7f9740d6d700 1 --2- 172.21.15.40:0/2644211491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c0704b0 0x7f9728004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:11.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.343+0000 7f974156e700 1 -- 172.21.15.40:0/2644211491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c0704b0 msgr2=0x7f9728004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:11.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.343+0000 7f974156e700 1 --2- 172.21.15.40:0/2644211491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c0704b0 0x7f9728004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.343+0000 7f974156e700 1 -- 172.21.15.40:0/2644211491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9728004db0 msgr2=0x7f9728005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:11.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.343+0000 7f974156e700 1 --2- 172.21.15.40:0/2644211491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9728004db0 0x7f9728005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.343+0000 7f974156e700 1 -- 172.21.15.40:0/2644211491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97280035d0 con 0x7f9728003a50 2023-11-08T08:22:11.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.343+0000 7f9741d6f700 1 --2- 172.21.15.40:0/2644211491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9728004db0 0x7f9728005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:22:11.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.343+0000 7f974156e700 1 --2- 172.21.15.40:0/2644211491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9728003a50 0x7f9728003e20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f973800c2c0 tx=0x7f97280058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275c81c1b724321b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:11.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.343+0000 7f9733fff700 1 -- 172.21.15.40:0/2644211491 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738007b90 con 0x7f9728003a50 2023-11-08T08:22:11.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.343+0000 7f9733fff700 1 -- 172.21.15.40:0/2644211491 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973800ce40 con 0x7f9728003a50 2023-11-08T08:22:11.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.343+0000 7f9733fff700 1 -- 172.21.15.40:0/2644211491 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738005bc0 con 0x7f9728003a50 2023-11-08T08:22:11.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.343+0000 7f9742570700 1 -- 172.21.15.40:0/2644211491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9728003a50 msgr2=0x7f9728003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:11.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.343+0000 7f9742570700 1 --2- 172.21.15.40:0/2644211491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9728003a50 0x7f9728003e20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f973800c2c0 tx=0x7f97280058a0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.343+0000 7f9741d6f700 1 -- 172.21.15.40:0/2644211491 reap_dead start 2023-11-08T08:22:11.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.343+0000 7f9742570700 1 -- 172.21.15.40:0/2644211491 shutdown_connections 2023-11-08T08:22:11.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.343+0000 7f9742570700 1 -- 172.21.15.40:0/2644211491 >> 172.21.15.40:0/2644211491 conn(0x7f973c0b0f80 msgr2=0x7f973c0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:11.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.343+0000 7f9742570700 1 -- 172.21.15.40:0/2644211491 shutdown_connections 2023-11-08T08:22:11.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.343+0000 7f9742570700 1 -- 172.21.15.40:0/2644211491 wait complete. 2023-11-08T08:22:11.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.347+0000 7f9742570700 1 Processor -- start 2023-11-08T08:22:11.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.347+0000 7f9742570700 1 -- start start 2023-11-08T08:22:11.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.347+0000 7f9742570700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9728003a50 0x7f973c0b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:11.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.347+0000 7f9742570700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c0c2280 0x7f973c1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:11.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.347+0000 7f9742570700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f973c1ea540 0x7f973c1ea9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:11.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.347+0000 7f9742570700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c1eaef0 con 0x7f973c1ea540 2023-11-08T08:22:11.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.347+0000 7f9742570700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0c3290 con 0x7f9728003a50 2023-11-08T08:22:11.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.347+0000 7f9742570700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0c29f0 con 0x7f973c0c2280 2023-11-08T08:22:11.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.347+0000 7f9741d6f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f973c1ea540 0x7f973c1ea9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:11.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.347+0000 7f9741d6f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f973c1ea540 0x7f973c1ea9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34610/0 (socket says 172.21.15.40:34610) 2023-11-08T08:22:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.347+0000 7f9741d6f700 1 -- 172.21.15.40:0/113410722 learned_addr learned my addr 172.21.15.40:0/113410722 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.351+0000 7f9740d6d700 1 --2- 172.21.15.40:0/113410722 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c0c2280 0x7f973c1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.351+0000 7f974156e700 1 --2- 172.21.15.40:0/113410722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9728003a50 0x7f973c0b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:11.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.351+0000 7f9741d6f700 1 -- 172.21.15.40:0/113410722 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c0c2280 msgr2=0x7f973c1565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:11.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.351+0000 7f9741d6f700 1 --2- 172.21.15.40:0/113410722 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c0c2280 0x7f973c1565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.351+0000 7f9741d6f700 1 -- 172.21.15.40:0/113410722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9728003a50 msgr2=0x7f973c0b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:11.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.351+0000 7f9741d6f700 1 --2- 172.21.15.40:0/113410722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9728003a50 0x7f973c0b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.351+0000 7f9741d6f700 1 -- 172.21.15.40:0/113410722 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c06b6b0 con 0x7f973c1ea540 2023-11-08T08:22:11.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.351+0000 7f9741d6f700 1 --2- 172.21.15.40:0/113410722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f973c1ea540 0x7f973c1ea9b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f973c06e590 tx=0x7f973400f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.351+0000 7f9731ffb700 1 -- 172.21.15.40:0/113410722 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973400e040 con 0x7f973c1ea540 2023-11-08T08:22:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.351+0000 7f9731ffb700 1 -- 172.21.15.40:0/113410722 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973400f040 con 0x7f973c1ea540 2023-11-08T08:22:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.351+0000 7f9742570700 1 -- 172.21.15.40:0/113410722 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973c06b3d0 con 0x7f973c1ea540 2023-11-08T08:22:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.351+0000 7f9731ffb700 1 -- 172.21.15.40:0/113410722 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734003a40 con 0x7f973c1ea540 2023-11-08T08:22:11.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.351+0000 7f9742570700 1 -- 172.21.15.40:0/113410722 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c0ad8f0 con 0x7f973c1ea540 2023-11-08T08:22:11.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.355+0000 7f9742570700 1 -- 172.21.15.40:0/113410722 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973c073cb0 con 0x7f973c1ea540 2023-11-08T08:22:11.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.355+0000 7f9731ffb700 1 -- 172.21.15.40:0/113410722 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f9734024070 con 0x7f973c1ea540 2023-11-08T08:22:11.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.355+0000 7f9731ffb700 1 --2- 172.21.15.40:0/113410722 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9720064f10 0x7f97200673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:11.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.355+0000 7f9731ffb700 1 -- 172.21.15.40:0/113410722 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f9734081f20 con 0x7f973c1ea540 2023-11-08T08:22:11.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.355+0000 7f974156e700 1 --2- 172.21.15.40:0/113410722 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9720064f10 0x7f97200673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:11.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.359+0000 7f974156e700 1 --2- 172.21.15.40:0/113410722 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9720064f10 0x7f97200673c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9738015040 tx=0x7f9738007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:11.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.363+0000 7f9731ffb700 1 -- 172.21.15.40:0/113410722 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f9734050220 con 0x7f973c1ea540 2023-11-08T08:22:11.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.543+0000 7f9742570700 1 -- 172.21.15.40:0/113410722 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f973c06afa0 con 0x7f973c1ea540 2023-11-08T08:22:11.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.551+0000 7f9731ffb700 1 -- 172.21.15.40:0/113410722 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9734053840 con 0x7f973c1ea540 2023-11-08T08:22:11.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.555+0000 7f9742570700 1 -- 172.21.15.40:0/113410722 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9720064f10 msgr2=0x7f97200673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:11.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.555+0000 7f9742570700 1 --2- 172.21.15.40:0/113410722 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9720064f10 0x7f97200673c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9738015040 tx=0x7f9738007000 comp rx=0 tx=0).stop 2023-11-08T08:22:11.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.555+0000 7f9742570700 1 -- 172.21.15.40:0/113410722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f973c1ea540 msgr2=0x7f973c1ea9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:11.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.555+0000 7f9742570700 1 --2- 172.21.15.40:0/113410722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f973c1ea540 0x7f973c1ea9b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f973c06e590 tx=0x7f973400f490 comp rx=0 tx=0).stop 2023-11-08T08:22:11.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.555+0000 7f9742570700 1 -- 172.21.15.40:0/113410722 shutdown_connections 2023-11-08T08:22:11.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.555+0000 7f9742570700 1 --2- 172.21.15.40:0/113410722 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9720064f10 0x7f97200673c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.555+0000 7f9742570700 1 --2- 172.21.15.40:0/113410722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f973c1ea540 0x7f973c1ea9b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.555+0000 7f9742570700 1 --2- 172.21.15.40:0/113410722 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f973c0c2280 0x7f973c1565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.555+0000 7f9742570700 1 --2- 172.21.15.40:0/113410722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9728003a50 0x7f973c0b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.555+0000 7f9742570700 1 -- 172.21.15.40:0/113410722 >> 172.21.15.40:0/113410722 conn(0x7f973c0b0f80 msgr2=0x7f973c162af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:11.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.555+0000 7f9742570700 1 -- 172.21.15.40:0/113410722 shutdown_connections 2023-11-08T08:22:11.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.555+0000 7f9742570700 1 -- 172.21.15.40:0/113410722 wait complete. 2023-11-08T08:22:11.585 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-08T08:22:11.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.755+0000 7f08edd24700 1 Processor -- start 2023-11-08T08:22:11.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.759+0000 7f08edd24700 1 -- start start 2023-11-08T08:22:11.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.759+0000 7f08edd24700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e8160710 0x7f08e80cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:11.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.759+0000 7f08edd24700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e80ad6c0 con 0x7f08e80cc860 2023-11-08T08:22:11.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.759+0000 7f08edd24700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e80adc60 con 0x7f08e8070790 2023-11-08T08:22:11.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.759+0000 7f08edd24700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e806b1c0 con 0x7f08e8160710 2023-11-08T08:22:11.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.759+0000 7f08dffff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f08e80cc860 0x7f08e8070080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50934/0 (socket says 172.21.15.40:50934) 2023-11-08T08:22:11.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.759+0000 7f08dffff700 1 -- 172.21.15.40:0/2911934565 learned_addr learned my addr 172.21.15.40:0/2911934565 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:11.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08ecd22700 1 --2- 172.21.15.40:0/2911934565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e8160710 0x7f08e80cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:11.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08df7fe700 1 -- 172.21.15.40:0/2911934565 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3938716143 0 0) 0x7f08e80ad6c0 con 0x7f08e80cc860 2023-11-08T08:22:11.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08df7fe700 1 -- 172.21.15.40:0/2911934565 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08d00035d0 con 0x7f08e80cc860 2023-11-08T08:22:11.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08df7fe700 1 -- 172.21.15.40:0/2911934565 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 445970182 0 0) 0x7f08e80adc60 con 0x7f08e8070790 2023-11-08T08:22:11.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08df7fe700 1 -- 172.21.15.40:0/2911934565 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08e80ad6c0 con 0x7f08e8070790 2023-11-08T08:22:11.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08df7fe700 1 -- 172.21.15.40:0/2911934565 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1942643363 0 0) 0x7f08d00035d0 con 0x7f08e80cc860 2023-11-08T08:22:11.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08df7fe700 1 -- 172.21.15.40:0/2911934565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e8160710 msgr2=0x7f08e80cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:11.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08df7fe700 1 --2- 172.21.15.40:0/2911934565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e8160710 0x7f08e80cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08df7fe700 1 -- 172.21.15.40:0/2911934565 >> v1:172.21.15.169:6789/0 conn(0x7f08e8070790 legacy=0x7f08e8164250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:11.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08df7fe700 1 -- 172.21.15.40:0/2911934565 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e806b3d0 con 0x7f08e80cc860 2023-11-08T08:22:11.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08df7fe700 1 -- 172.21.15.40:0/2911934565 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f08d8001670 con 0x7f08e80cc860 2023-11-08T08:22:11.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08df7fe700 1 -- 172.21.15.40:0/2911934565 >> v1:172.21.15.40:6789/0 conn(0x7f08e80cc860 legacy=0x7f08e8070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:11.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08df7fe700 1 --2- 172.21.15.40:0/2911934565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d0003be0 0x7f08d0003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:11.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08df7fe700 1 --2- 172.21.15.40:0/2911934565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08d0004870 0x7f08d0004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:11.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08df7fe700 1 --2- 172.21.15.40:0/2911934565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e8160710 0x7f08d0005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:11.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08df7fe700 1 -- 172.21.15.40:0/2911934565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e806b1c0 con 0x7f08d0003be0 2023-11-08T08:22:11.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08df7fe700 1 -- 172.21.15.40:0/2911934565 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d000bae0 con 0x7f08d0004870 2023-11-08T08:22:11.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08df7fe700 1 -- 172.21.15.40:0/2911934565 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d000bc50 con 0x7f08e8160710 2023-11-08T08:22:11.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08ecd22700 1 --2- 172.21.15.40:0/2911934565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d0003be0 0x7f08d0003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:11.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08dffff700 1 --2- 172.21.15.40:0/2911934565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08d0004870 0x7f08d0004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:11.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08ed523700 1 --2- 172.21.15.40:0/2911934565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e8160710 0x7f08d0005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:11.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08ecd22700 1 -- 172.21.15.40:0/2911934565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e8160710 msgr2=0x7f08d0005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:11.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08ecd22700 1 --2- 172.21.15.40:0/2911934565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e8160710 0x7f08d0005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08ecd22700 1 -- 172.21.15.40:0/2911934565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08d0004870 msgr2=0x7f08d0004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:11.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08ecd22700 1 --2- 172.21.15.40:0/2911934565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08d0004870 0x7f08d0004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08ecd22700 1 -- 172.21.15.40:0/2911934565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d00035d0 con 0x7f08d0003be0 2023-11-08T08:22:11.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.763+0000 7f08ed523700 1 --2- 172.21.15.40:0/2911934565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e8160710 0x7f08d0005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:22:11.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.767+0000 7f08ecd22700 1 --2- 172.21.15.40:0/2911934565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d0003be0 0x7f08d0003fd0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009b70 tx=0x7f08d00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93e22f817de7303 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:11.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.767+0000 7f08df7fe700 1 -- 172.21.15.40:0/2911934565 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e400e6c0 con 0x7f08d0003be0 2023-11-08T08:22:11.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.767+0000 7f08df7fe700 1 -- 172.21.15.40:0/2911934565 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e400e820 con 0x7f08d0003be0 2023-11-08T08:22:11.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.767+0000 7f08df7fe700 1 -- 172.21.15.40:0/2911934565 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e400eb90 con 0x7f08d0003be0 2023-11-08T08:22:11.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.767+0000 7f08edd24700 1 -- 172.21.15.40:0/2911934565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d0003be0 msgr2=0x7f08d0003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:11.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.767+0000 7f08edd24700 1 --2- 172.21.15.40:0/2911934565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d0003be0 0x7f08d0003fd0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009b70 tx=0x7f08d00058a0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.767+0000 7f08ed523700 1 -- 172.21.15.40:0/2911934565 reap_dead start 2023-11-08T08:22:11.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.767+0000 7f08edd24700 1 -- 172.21.15.40:0/2911934565 shutdown_connections 2023-11-08T08:22:11.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.767+0000 7f08edd24700 1 -- 172.21.15.40:0/2911934565 >> 172.21.15.40:0/2911934565 conn(0x7f08e80b0f80 msgr2=0x7f08e80ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:11.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.767+0000 7f08edd24700 1 -- 172.21.15.40:0/2911934565 shutdown_connections 2023-11-08T08:22:11.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.767+0000 7f08edd24700 1 -- 172.21.15.40:0/2911934565 wait complete. 2023-11-08T08:22:11.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.771+0000 7f08edd24700 1 Processor -- start 2023-11-08T08:22:11.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.771+0000 7f08edd24700 1 -- start start 2023-11-08T08:22:11.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.771+0000 7f08edd24700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e8155aa0 0x7f08e81570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:11.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.771+0000 7f08edd24700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e8155e70 0x7f08e81f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:11.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.771+0000 7f08edd24700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08e81575f0 0x7f08e81f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:11.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.771+0000 7f08edd24700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e81f65e0 con 0x7f08e81575f0 2023-11-08T08:22:11.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.771+0000 7f08edd24700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e81586d0 con 0x7f08e8155e70 2023-11-08T08:22:11.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.771+0000 7f08edd24700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e8157b50 con 0x7f08e8155aa0 2023-11-08T08:22:11.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.771+0000 7f08ed523700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08e81575f0 0x7f08e81f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:11.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.771+0000 7f08ecd22700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e8155aa0 0x7f08e81570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:11.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.771+0000 7f08ed523700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08e81575f0 0x7f08e81f6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34628/0 (socket says 172.21.15.40:34628) 2023-11-08T08:22:11.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.771+0000 7f08ed523700 1 -- 172.21.15.40:0/3525364365 learned_addr learned my addr 172.21.15.40:0/3525364365 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:11.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.771+0000 7f08dffff700 1 --2- 172.21.15.40:0/3525364365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e8155e70 0x7f08e81f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:11.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.775+0000 7f08ed523700 1 -- 172.21.15.40:0/3525364365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e8155aa0 msgr2=0x7f08e81570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:11.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.775+0000 7f08ed523700 1 --2- 172.21.15.40:0/3525364365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e8155aa0 0x7f08e81570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.775+0000 7f08ed523700 1 -- 172.21.15.40:0/3525364365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e8155e70 msgr2=0x7f08e81f6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.775+0000 7f08ed523700 1 --2- 172.21.15.40:0/3525364365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e8155e70 0x7f08e81f6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.775+0000 7f08ed523700 1 -- 172.21.15.40:0/3525364365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e80ad8f0 con 0x7f08e81575f0 2023-11-08T08:22:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.775+0000 7f08ed523700 1 --2- 172.21.15.40:0/3525364365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08e81575f0 0x7f08e81f6fb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f08e80cdbd0 tx=0x7f08e000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:11.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.775+0000 7f08dd7fa700 1 -- 172.21.15.40:0/3525364365 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0003c30 con 0x7f08e81575f0 2023-11-08T08:22:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.775+0000 7f08dd7fa700 1 -- 172.21.15.40:0/3525364365 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e0003db0 con 0x7f08e81575f0 2023-11-08T08:22:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.775+0000 7f08dd7fa700 1 -- 172.21.15.40:0/3525364365 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0005310 con 0x7f08e81575f0 2023-11-08T08:22:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.775+0000 7f08edd24700 1 -- 172.21.15.40:0/3525364365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e806b6b0 con 0x7f08e81575f0 2023-11-08T08:22:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.775+0000 7f08edd24700 1 -- 172.21.15.40:0/3525364365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e8160ae0 con 0x7f08e81575f0 2023-11-08T08:22:11.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.775+0000 7f08edd24700 1 -- 172.21.15.40:0/3525364365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e81f6750 con 0x7f08e81575f0 2023-11-08T08:22:11.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.775+0000 7f08dd7fa700 1 -- 172.21.15.40:0/3525364365 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f08e0005470 con 0x7f08e81575f0 2023-11-08T08:22:11.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.779+0000 7f08dd7fa700 1 --2- 172.21.15.40:0/3525364365 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f08d4064ee0 0x7f08d4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:11.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.779+0000 7f08dd7fa700 1 -- 172.21.15.40:0/3525364365 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f08e0081df0 con 0x7f08e81575f0 2023-11-08T08:22:11.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.779+0000 7f08ecd22700 1 --2- 172.21.15.40:0/3525364365 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f08d4064ee0 0x7f08d4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:11.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.779+0000 7f08ecd22700 1 --2- 172.21.15.40:0/3525364365 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f08d4064ee0 0x7f08d4067390 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f08e80ae280 tx=0x7f08e4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:11.783+0000 7f08dd7fa700 1 -- 172.21.15.40:0/3525364365 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f08e0050150 con 0x7f08e81575f0 2023-11-08T08:22:12.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.031+0000 7f08edd24700 1 -- 172.21.15.40:0/3525364365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f08e8073cb0 con 0x7f08e81575f0 2023-11-08T08:22:12.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.111+0000 7f08dd7fa700 1 -- 172.21.15.40:0/3525364365 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v184) v1 ==== 149+0+0 (secure 0 0 0) 0x7f08e0053770 con 0x7f08e81575f0 2023-11-08T08:22:12.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.115+0000 7f08edd24700 1 -- 172.21.15.40:0/3525364365 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f08d4064ee0 msgr2=0x7f08d4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:12.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.115+0000 7f08edd24700 1 --2- 172.21.15.40:0/3525364365 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f08d4064ee0 0x7f08d4067390 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f08e80ae280 tx=0x7f08e4008030 comp rx=0 tx=0).stop 2023-11-08T08:22:12.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.115+0000 7f08edd24700 1 -- 172.21.15.40:0/3525364365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08e81575f0 msgr2=0x7f08e81f6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:12.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.115+0000 7f08edd24700 1 --2- 172.21.15.40:0/3525364365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08e81575f0 0x7f08e81f6fb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f08e80cdbd0 tx=0x7f08e000ee90 comp rx=0 tx=0).stop 2023-11-08T08:22:12.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.115+0000 7f08edd24700 1 -- 172.21.15.40:0/3525364365 shutdown_connections 2023-11-08T08:22:12.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.115+0000 7f08edd24700 1 --2- 172.21.15.40:0/3525364365 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f08d4064ee0 0x7f08d4067390 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:12.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.115+0000 7f08edd24700 1 --2- 172.21.15.40:0/3525364365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08e81575f0 0x7f08e81f6fb0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:12.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.115+0000 7f08edd24700 1 --2- 172.21.15.40:0/3525364365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08e8155e70 0x7f08e81f6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:12.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.115+0000 7f08edd24700 1 --2- 172.21.15.40:0/3525364365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08e8155aa0 0x7f08e81570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:12.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.115+0000 7f08edd24700 1 -- 172.21.15.40:0/3525364365 >> 172.21.15.40:0/3525364365 conn(0x7f08e80b0f80 msgr2=0x7f08e806bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:12.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.115+0000 7f08edd24700 1 -- 172.21.15.40:0/3525364365 shutdown_connections 2023-11-08T08:22:12.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.115+0000 7f08edd24700 1 -- 172.21.15.40:0/3525364365 wait complete. 2023-11-08T08:22:12.146 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2023-11-08T08:22:12.146 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2023-11-08T08:22:12.147 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-08T08:22:12.147 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-08T08:22:12.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.323+0000 7efcb084c700 1 Processor -- start 2023-11-08T08:22:12.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.323+0000 7efcb084c700 1 -- start start 2023-11-08T08:22:12.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.323+0000 7efcb084c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcac071aa0 0x7efcac06d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:12.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.323+0000 7efcb084c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efcac069e00 0x7efcac06a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:12.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.323+0000 7efcb084c700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcac06c600 con 0x7efcac15c640 2023-11-08T08:22:12.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.323+0000 7efcb084c700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcac06cef0 con 0x7efcac069e00 2023-11-08T08:22:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.323+0000 7efcb084c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0d38f0 con 0x7efcac071aa0 2023-11-08T08:22:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.327+0000 7efcabfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efcac069e00 0x7efcac06a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.327+0000 7efcabfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efcac069e00 0x7efcac06a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34636/0 (socket says 172.21.15.40:34636) 2023-11-08T08:22:12.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.327+0000 7efcabfff700 1 -- 172.21.15.40:0/4199015742 learned_addr learned my addr 172.21.15.40:0/4199015742 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:12.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.327+0000 7efcaaffd700 1 --2- 172.21.15.40:0/4199015742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcac071aa0 0x7efcac06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:12.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.327+0000 7efcabfff700 1 -- 172.21.15.40:0/4199015742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcac071aa0 msgr2=0x7efcac06d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:12.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.327+0000 7efcabfff700 1 --2- 172.21.15.40:0/4199015742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcac071aa0 0x7efcac06d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:12.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.327+0000 7efcabfff700 1 -- 172.21.15.40:0/4199015742 >> v1:172.21.15.169:6789/0 conn(0x7efcac15c640 legacy=0x7efcac0d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:12.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.327+0000 7efcabfff700 1 -- 172.21.15.40:0/4199015742 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac0722f0 con 0x7efcac069e00 2023-11-08T08:22:12.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.327+0000 7efcabfff700 1 --2- 172.21.15.40:0/4199015742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcac069e00 0x7efcac06a280 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7efca4009e50 tx=0x7efca400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f0479442187537 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:12.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.327+0000 7efcaa7fc700 1 -- 172.21.15.40:0/4199015742 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca400e070 con 0x7efcac069e00 2023-11-08T08:22:12.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.327+0000 7efcaa7fc700 1 -- 172.21.15.40:0/4199015742 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca4003a40 con 0x7efcac069e00 2023-11-08T08:22:12.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.327+0000 7efcaa7fc700 1 -- 172.21.15.40:0/4199015742 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca4003df0 con 0x7efcac069e00 2023-11-08T08:22:12.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.327+0000 7efcb084c700 1 -- 172.21.15.40:0/4199015742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcac069e00 msgr2=0x7efcac06a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:12.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.327+0000 7efcb084c700 1 --2- 172.21.15.40:0/4199015742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcac069e00 0x7efcac06a280 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7efca4009e50 tx=0x7efca400b330 comp rx=0 tx=0).stop 2023-11-08T08:22:12.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.327+0000 7efcb084c700 1 -- 172.21.15.40:0/4199015742 shutdown_connections 2023-11-08T08:22:12.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.327+0000 7efcb084c700 1 --2- 172.21.15.40:0/4199015742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcac071aa0 0x7efcac06d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:12.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.327+0000 7efcb084c700 1 --2- 172.21.15.40:0/4199015742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcac069e00 0x7efcac06a280 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:12.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.327+0000 7efcb084c700 1 -- 172.21.15.40:0/4199015742 >> 172.21.15.40:0/4199015742 conn(0x7efcac0b6520 msgr2=0x7efcac071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:12.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.327+0000 7efcb084c700 1 -- 172.21.15.40:0/4199015742 shutdown_connections 2023-11-08T08:22:12.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.327+0000 7efcb084c700 1 -- 172.21.15.40:0/4199015742 wait complete. 2023-11-08T08:22:12.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.331+0000 7efcb084c700 1 Processor -- start 2023-11-08T08:22:12.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.331+0000 7efcb084c700 1 -- start start 2023-11-08T08:22:12.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efcb084c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcac069e00 0x7efcac1f15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:12.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efcb084c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcac071aa0 0x7efcac0cea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:12.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efcb084c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcac15c640 0x7efcac0cef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:12.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efcb084c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0cba80 con 0x7efcac069e00 2023-11-08T08:22:12.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efcb084c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0cca20 con 0x7efcac071aa0 2023-11-08T08:22:12.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efcb084c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0cc590 con 0x7efcac15c640 2023-11-08T08:22:12.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efcab7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcac069e00 0x7efcac1f15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:12.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efcab7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcac069e00 0x7efcac1f15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34642/0 (socket says 172.21.15.40:34642) 2023-11-08T08:22:12.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efcab7fe700 1 -- 172.21.15.40:0/2809470894 learned_addr learned my addr 172.21.15.40:0/2809470894 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:12.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efcaaffd700 1 --2- 172.21.15.40:0/2809470894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcac071aa0 0x7efcac0cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:12.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efcabfff700 1 --2- 172.21.15.40:0/2809470894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcac15c640 0x7efcac0cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:12.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efcab7fe700 1 -- 172.21.15.40:0/2809470894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcac15c640 msgr2=0x7efcac0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:12.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efcab7fe700 1 --2- 172.21.15.40:0/2809470894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcac15c640 0x7efcac0cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:12.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efcab7fe700 1 -- 172.21.15.40:0/2809470894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcac071aa0 msgr2=0x7efcac0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:12.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efcab7fe700 1 --2- 172.21.15.40:0/2809470894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcac071aa0 0x7efcac0cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:12.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efcab7fe700 1 -- 172.21.15.40:0/2809470894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca4009b30 con 0x7efcac069e00 2023-11-08T08:22:12.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efcaaffd700 1 --2- 172.21.15.40:0/2809470894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcac071aa0 0x7efcac0cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:22:12.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efcab7fe700 1 --2- 172.21.15.40:0/2809470894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcac069e00 0x7efcac1f15a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00c550 tx=0x7efc9c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:12.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efc93fff700 1 -- 172.21.15.40:0/2809470894 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c011790 con 0x7efcac069e00 2023-11-08T08:22:12.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efc93fff700 1 -- 172.21.15.40:0/2809470894 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9c0118f0 con 0x7efcac069e00 2023-11-08T08:22:12.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efcb084c700 1 -- 172.21.15.40:0/2809470894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcac072010 con 0x7efcac069e00 2023-11-08T08:22:12.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efc93fff700 1 -- 172.21.15.40:0/2809470894 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c011ca0 con 0x7efcac069e00 2023-11-08T08:22:12.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.335+0000 7efcb084c700 1 -- 172.21.15.40:0/2809470894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcac15ca10 con 0x7efcac069e00 2023-11-08T08:22:12.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.339+0000 7efcb084c700 1 -- 172.21.15.40:0/2809470894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcac06c830 con 0x7efcac069e00 2023-11-08T08:22:12.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.339+0000 7efc93fff700 1 -- 172.21.15.40:0/2809470894 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7efc9c010050 con 0x7efcac069e00 2023-11-08T08:22:12.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.339+0000 7efc93fff700 1 --2- 172.21.15.40:0/2809470894 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efc98064f10 0x7efc980673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:12.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.339+0000 7efc93fff700 1 -- 172.21.15.40:0/2809470894 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5616+0+0 (secure 0 0 0) 0x7efc9c081ac0 con 0x7efcac069e00 2023-11-08T08:22:12.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.339+0000 7efcaaffd700 1 --2- 172.21.15.40:0/2809470894 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efc98064f10 0x7efc980673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:12.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.343+0000 7efcaaffd700 1 --2- 172.21.15.40:0/2809470894 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efc98064f10 0x7efc980673c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efcac0609b0 tx=0x7efca0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:12.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.347+0000 7efc93fff700 1 -- 172.21.15.40:0/2809470894 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7efc9c04fcf0 con 0x7efcac069e00 2023-11-08T08:22:12.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:12.583+0000 7efcb084c700 1 -- 172.21.15.40:0/2809470894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7efcac073cb0 con 0x7efcac069e00 2023-11-08T08:22:13.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.115+0000 7efc93fff700 1 -- 172.21.15.40:0/2809470894 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v185) v1 ==== 172+0+0 (secure 0 0 0) 0x7efc9c053310 con 0x7efcac069e00 2023-11-08T08:22:13.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.115+0000 7efcb084c700 1 -- 172.21.15.40:0/2809470894 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efc98064f10 msgr2=0x7efc980673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:13.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.115+0000 7efcb084c700 1 --2- 172.21.15.40:0/2809470894 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efc98064f10 0x7efc980673c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efcac0609b0 tx=0x7efca0008040 comp rx=0 tx=0).stop 2023-11-08T08:22:13.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.115+0000 7efcb084c700 1 -- 172.21.15.40:0/2809470894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcac069e00 msgr2=0x7efcac1f15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:13.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.115+0000 7efcb084c700 1 --2- 172.21.15.40:0/2809470894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcac069e00 0x7efcac1f15a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00c550 tx=0x7efc9c00c910 comp rx=0 tx=0).stop 2023-11-08T08:22:13.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.119+0000 7efcb084c700 1 -- 172.21.15.40:0/2809470894 shutdown_connections 2023-11-08T08:22:13.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.119+0000 7efcb084c700 1 --2- 172.21.15.40:0/2809470894 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efc98064f10 0x7efc980673c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:13.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.119+0000 7efcb084c700 1 --2- 172.21.15.40:0/2809470894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efcac15c640 0x7efcac0cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:13.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.119+0000 7efcb084c700 1 --2- 172.21.15.40:0/2809470894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcac071aa0 0x7efcac0cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:13.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.119+0000 7efcb084c700 1 --2- 172.21.15.40:0/2809470894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcac069e00 0x7efcac1f15a0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:13.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.119+0000 7efcb084c700 1 -- 172.21.15.40:0/2809470894 >> 172.21.15.40:0/2809470894 conn(0x7efcac0b6520 msgr2=0x7efcac0d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:13.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.119+0000 7efcb084c700 1 -- 172.21.15.40:0/2809470894 shutdown_connections 2023-11-08T08:22:13.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.119+0000 7efcb084c700 1 -- 172.21.15.40:0/2809470894 wait complete. 2023-11-08T08:22:13.148 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-08T08:22:13.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.331+0000 7f20beab1700 1 Processor -- start 2023-11-08T08:22:13.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.331+0000 7f20beab1700 1 -- start start 2023-11-08T08:22:13.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.335+0000 7f20beab1700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b815c470 0x7f20b80c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:13.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.335+0000 7f20beab1700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f20b8070080 0x7f20b8070500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:13.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.335+0000 7f20beab1700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b815d610 con 0x7f20b80c7f00 2023-11-08T08:22:13.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.335+0000 7f20beab1700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b806aae0 con 0x7f20b8070080 2023-11-08T08:22:13.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.335+0000 7f20beab1700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b8069bf0 con 0x7f20b815c470 2023-11-08T08:22:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.335+0000 7f20bd2ae700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f20b80c7f00 0x7f20b815c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50942/0 (socket says 172.21.15.40:50942) 2023-11-08T08:22:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.335+0000 7f20bd2ae700 1 -- 172.21.15.40:0/3009491211 learned_addr learned my addr 172.21.15.40:0/3009491211 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.335+0000 7f20bdaaf700 1 --2- 172.21.15.40:0/3009491211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b815c470 0x7f20b80c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.335+0000 7f20be2b0700 1 --2- 172.21.15.40:0/3009491211 >> v2:172.21.15.169:3300/0 conn(0x7f20b8070080 0x7f20b8070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:13.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.335+0000 7f20bcaad700 1 -- 172.21.15.40:0/3009491211 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 803436877 0 0) 0x7f20b815d610 con 0x7f20b80c7f00 2023-11-08T08:22:13.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.335+0000 7f20bcaad700 1 -- 172.21.15.40:0/3009491211 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20a40035d0 con 0x7f20b80c7f00 2023-11-08T08:22:13.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.335+0000 7f20bdaaf700 1 -- 172.21.15.40:0/3009491211 >> v2:172.21.15.169:3300/0 conn(0x7f20b8070080 msgr2=0x7f20b8070500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:13.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.335+0000 7f20bdaaf700 1 --2- 172.21.15.40:0/3009491211 >> v2:172.21.15.169:3300/0 conn(0x7f20b8070080 0x7f20b8070500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:13.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.335+0000 7f20bdaaf700 1 -- 172.21.15.40:0/3009491211 >> v1:172.21.15.40:6789/0 conn(0x7f20b80c7f00 legacy=0x7f20b815c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:13.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.335+0000 7f20bdaaf700 1 -- 172.21.15.40:0/3009491211 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b8069ed0 con 0x7f20b815c470 2023-11-08T08:22:13.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.335+0000 7f20bdaaf700 1 --2- 172.21.15.40:0/3009491211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b815c470 0x7f20b80c79c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f20b4009850 tx=0x7f20b4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eed561143a5e4e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:13.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.339+0000 7f20bcaad700 1 -- 172.21.15.40:0/3009491211 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b4011830 con 0x7f20b815c470 2023-11-08T08:22:13.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.339+0000 7f20bcaad700 1 -- 172.21.15.40:0/3009491211 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b4011990 con 0x7f20b815c470 2023-11-08T08:22:13.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.339+0000 7f20bcaad700 1 -- 172.21.15.40:0/3009491211 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b4011d40 con 0x7f20b815c470 2023-11-08T08:22:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.339+0000 7f20beab1700 1 -- 172.21.15.40:0/3009491211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b815c470 msgr2=0x7f20b80c79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.339+0000 7f20beab1700 1 --2- 172.21.15.40:0/3009491211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b815c470 0x7f20b80c79c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f20b4009850 tx=0x7f20b4009b60 comp rx=0 tx=0).stop 2023-11-08T08:22:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.339+0000 7f20beab1700 1 -- 172.21.15.40:0/3009491211 shutdown_connections 2023-11-08T08:22:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.339+0000 7f20beab1700 1 --2- 172.21.15.40:0/3009491211 >> v2:172.21.15.169:3300/0 conn(0x7f20b8070080 0x7f20b8070500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:13.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.339+0000 7f20beab1700 1 --2- 172.21.15.40:0/3009491211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b815c470 0x7f20b80c79c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.339+0000 7f20beab1700 1 -- 172.21.15.40:0/3009491211 >> 172.21.15.40:0/3009491211 conn(0x7f20b80b53a0 msgr2=0x7f20b806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.339+0000 7f20beab1700 1 -- 172.21.15.40:0/3009491211 shutdown_connections 2023-11-08T08:22:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.339+0000 7f20beab1700 1 -- 172.21.15.40:0/3009491211 wait complete. 2023-11-08T08:22:13.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20beab1700 1 Processor -- start 2023-11-08T08:22:13.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20beab1700 1 -- start start 2023-11-08T08:22:13.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20beab1700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b8070080 0x7f20b808ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:13.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20beab1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b80c7f00 0x7f20b81f5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:13.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20beab1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b815c470 0x7f20b81f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:13.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20beab1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b808bf60 con 0x7f20b815c470 2023-11-08T08:22:13.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20beab1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b808c240 con 0x7f20b80c7f00 2023-11-08T08:22:13.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20beab1700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b808c520 con 0x7f20b8070080 2023-11-08T08:22:13.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20be2b0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b815c470 0x7f20b81f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:13.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20be2b0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b815c470 0x7f20b81f5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34654/0 (socket says 172.21.15.40:34654) 2023-11-08T08:22:13.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20be2b0700 1 -- 172.21.15.40:0/3333541481 learned_addr learned my addr 172.21.15.40:0/3333541481 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:13.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20bd2ae700 1 --2- 172.21.15.40:0/3333541481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b80c7f00 0x7f20b81f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:13.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20bdaaf700 1 --2- 172.21.15.40:0/3333541481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b8070080 0x7f20b808ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20be2b0700 1 -- 172.21.15.40:0/3333541481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b8070080 msgr2=0x7f20b808ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20be2b0700 1 --2- 172.21.15.40:0/3333541481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b8070080 0x7f20b808ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20be2b0700 1 -- 172.21.15.40:0/3333541481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b80c7f00 msgr2=0x7f20b81f5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20be2b0700 1 --2- 172.21.15.40:0/3333541481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b80c7f00 0x7f20b81f5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20be2b0700 1 -- 172.21.15.40:0/3333541481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b40094b0 con 0x7f20b815c470 2023-11-08T08:22:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20be2b0700 1 --2- 172.21.15.40:0/3333541481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b815c470 0x7f20b81f5e60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f20b000efc0 tx=0x7f20b000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20ae7fc700 1 -- 172.21.15.40:0/3333541481 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0009ea0 con 0x7f20b815c470 2023-11-08T08:22:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20ae7fc700 1 -- 172.21.15.40:0/3333541481 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b000f040 con 0x7f20b815c470 2023-11-08T08:22:13.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.343+0000 7f20beab1700 1 -- 172.21.15.40:0/3333541481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b806a490 con 0x7f20b815c470 2023-11-08T08:22:13.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.347+0000 7f20ae7fc700 1 -- 172.21.15.40:0/3333541481 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b000ced0 con 0x7f20b815c470 2023-11-08T08:22:13.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.347+0000 7f20beab1700 1 -- 172.21.15.40:0/3333541481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b806a1b0 con 0x7f20b815c470 2023-11-08T08:22:13.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.347+0000 7f20beab1700 1 -- 172.21.15.40:0/3333541481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b806a770 con 0x7f20b815c470 2023-11-08T08:22:13.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.347+0000 7f20ae7fc700 1 -- 172.21.15.40:0/3333541481 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f20b0014480 con 0x7f20b815c470 2023-11-08T08:22:13.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.351+0000 7f20ae7fc700 1 --2- 172.21.15.40:0/3333541481 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f209c064f10 0x7f209c0673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:13.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.351+0000 7f20ae7fc700 1 -- 172.21.15.40:0/3333541481 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f20b0082870 con 0x7f20b815c470 2023-11-08T08:22:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.351+0000 7f20bdaaf700 1 --2- 172.21.15.40:0/3333541481 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f209c064f10 0x7f209c0673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:13.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.351+0000 7f20bdaaf700 1 --2- 172.21.15.40:0/3333541481 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f209c064f10 0x7f209c0673c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f20b40090b0 tx=0x7f20b401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:13.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.355+0000 7f20ae7fc700 1 -- 172.21.15.40:0/3333541481 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f20b0050b70 con 0x7f20b815c470 2023-11-08T08:22:13.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.535+0000 7f20beab1700 1 -- 172.21.15.40:0/3333541481 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f20b8073d20 con 0x7f20b815c470 2023-11-08T08:22:13.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.543+0000 7f20ae7fc700 1 -- 172.21.15.40:0/3333541481 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f20b00180c0 con 0x7f20b815c470 2023-11-08T08:22:13.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.547+0000 7f20beab1700 1 -- 172.21.15.40:0/3333541481 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f209c064f10 msgr2=0x7f209c0673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:13.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.547+0000 7f20beab1700 1 --2- 172.21.15.40:0/3333541481 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f209c064f10 0x7f209c0673c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f20b40090b0 tx=0x7f20b401c040 comp rx=0 tx=0).stop 2023-11-08T08:22:13.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.547+0000 7f20beab1700 1 -- 172.21.15.40:0/3333541481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b815c470 msgr2=0x7f20b81f5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:13.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.547+0000 7f20beab1700 1 --2- 172.21.15.40:0/3333541481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b815c470 0x7f20b81f5e60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f20b000efc0 tx=0x7f20b000c3a0 comp rx=0 tx=0).stop 2023-11-08T08:22:13.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.547+0000 7f20beab1700 1 -- 172.21.15.40:0/3333541481 shutdown_connections 2023-11-08T08:22:13.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.547+0000 7f20beab1700 1 --2- 172.21.15.40:0/3333541481 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f209c064f10 0x7f209c0673c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:13.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.547+0000 7f20beab1700 1 --2- 172.21.15.40:0/3333541481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20b815c470 0x7f20b81f5e60 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:13.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.547+0000 7f20beab1700 1 --2- 172.21.15.40:0/3333541481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20b80c7f00 0x7f20b81f5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:13.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.547+0000 7f20beab1700 1 --2- 172.21.15.40:0/3333541481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20b8070080 0x7f20b808ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:13.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.547+0000 7f20beab1700 1 -- 172.21.15.40:0/3333541481 >> 172.21.15.40:0/3333541481 conn(0x7f20b80b53a0 msgr2=0x7f20b80d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:13.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.547+0000 7f20beab1700 1 -- 172.21.15.40:0/3333541481 shutdown_connections 2023-11-08T08:22:13.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.547+0000 7f20beab1700 1 -- 172.21.15.40:0/3333541481 wait complete. 2023-11-08T08:22:13.577 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T08:22:13.578 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T08:22:13.627 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T08:22:13.674 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T08:22:13.675 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:22:13.675 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:22:13.696 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16355412k,nr_inodes=4088853,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6645 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T08:22:13.696 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:22:13.697 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:22:13.736 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352316k,nr_inodes=4088079,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1804 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T08:22:13.736 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:22:13.737 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:22:13.755 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16355412k,nr_inodes=4088853,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6645 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T08:22:13.756 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:22:13.756 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:22:13.795 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352316k,nr_inodes=4088079,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1804 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T08:22:13.796 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-08T08:22:13.796 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:22:13.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.979+0000 7f15ef4dc700 1 Processor -- start 2023-11-08T08:22:13.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.979+0000 7f15ef4dc700 1 -- start start 2023-11-08T08:22:13.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.979+0000 7f15ef4dc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f15e8160710 0x7f15e80cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:13.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.979+0000 7f15ef4dc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15e80cc860 0x7f15e8070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:13.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.979+0000 7f15ef4dc700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f15e80705c0 0x7f15e8069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:13.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.979+0000 7f15ef4dc700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e8070a60 con 0x7f15e8160710 2023-11-08T08:22:13.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.979+0000 7f15ef4dc700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e806b1c0 con 0x7f15e80705c0 2023-11-08T08:22:13.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.979+0000 7f15ef4dc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e80ad6c0 con 0x7f15e80cc860 2023-11-08T08:22:13.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15ee4da700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f15e8160710 0x7f15e80cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:13.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15eecdb700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f15e80705c0 0x7f15e8069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:13.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15ee4da700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f15e8160710 0x7f15e80cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34666/0 (socket says 172.21.15.40:34666) 2023-11-08T08:22:13.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15ee4da700 1 -- 172.21.15.40:0/3360065997 learned_addr learned my addr 172.21.15.40:0/3360065997 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:13.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15edcd9700 1 --2- 172.21.15.40:0/3360065997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15e80cc860 0x7f15e8070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:13.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15eecdb700 1 -- 172.21.15.40:0/3360065997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15e80cc860 msgr2=0x7f15e8070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:13.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15eecdb700 1 --2- 172.21.15.40:0/3360065997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15e80cc860 0x7f15e8070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:13.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15eecdb700 1 -- 172.21.15.40:0/3360065997 >> v2:172.21.15.40:3300/0 conn(0x7f15e8160710 msgr2=0x7f15e80cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:13.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15eecdb700 1 --2- 172.21.15.40:0/3360065997 >> v2:172.21.15.40:3300/0 conn(0x7f15e8160710 0x7f15e80cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:13.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15eecdb700 1 -- 172.21.15.40:0/3360065997 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e80adc60 con 0x7f15e80705c0 2023-11-08T08:22:13.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15edcd9700 1 --2- 172.21.15.40:0/3360065997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15e80cc860 0x7f15e8070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:22:13.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15ee4da700 1 --2- 172.21.15.40:0/3360065997 >> v2:172.21.15.40:3300/0 conn(0x7f15e8160710 0x7f15e80cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:22:13.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15eecdb700 1 --2- 172.21.15.40:0/3360065997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e80705c0 0x7f15e8069bf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f15e80b4900 tx=0x7f15e000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41a08b299f9fc069 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:13.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15ed4d8700 1 -- 172.21.15.40:0/3360065997 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e000e070 con 0x7f15e80705c0 2023-11-08T08:22:13.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15ed4d8700 1 -- 172.21.15.40:0/3360065997 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e000bb20 con 0x7f15e80705c0 2023-11-08T08:22:13.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15ed4d8700 1 -- 172.21.15.40:0/3360065997 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e0012430 con 0x7f15e80705c0 2023-11-08T08:22:13.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15ef4dc700 1 -- 172.21.15.40:0/3360065997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e80705c0 msgr2=0x7f15e8069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:13.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15ef4dc700 1 --2- 172.21.15.40:0/3360065997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e80705c0 0x7f15e8069bf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f15e80b4900 tx=0x7f15e000b330 comp rx=0 tx=0).stop 2023-11-08T08:22:13.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15ef4dc700 1 -- 172.21.15.40:0/3360065997 shutdown_connections 2023-11-08T08:22:13.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15ef4dc700 1 --2- 172.21.15.40:0/3360065997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e80705c0 0x7f15e8069bf0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:13.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15ef4dc700 1 --2- 172.21.15.40:0/3360065997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15e80cc860 0x7f15e8070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:13.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15ef4dc700 1 --2- 172.21.15.40:0/3360065997 >> v2:172.21.15.40:3300/0 conn(0x7f15e8160710 0x7f15e80cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:13.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15ef4dc700 1 -- 172.21.15.40:0/3360065997 >> 172.21.15.40:0/3360065997 conn(0x7f15e80b0f80 msgr2=0x7f15e80ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:13.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.983+0000 7f15ef4dc700 1 -- 172.21.15.40:0/3360065997 shutdown_connections 2023-11-08T08:22:13.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.987+0000 7f15ef4dc700 1 -- 172.21.15.40:0/3360065997 wait complete. 2023-11-08T08:22:13.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.987+0000 7f15ef4dc700 1 Processor -- start 2023-11-08T08:22:13.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.987+0000 7f15ef4dc700 1 -- start start 2023-11-08T08:22:13.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15ef4dc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15e80705c0 0x7f15e80aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:13.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15ef4dc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15e80cc860 0x7f15e80af200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:13.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15ef4dc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e8160710 0x7f15e81f9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:13.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15ef4dc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e814f950 con 0x7f15e80cc860 2023-11-08T08:22:13.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15ef4dc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e814ed40 con 0x7f15e8160710 2023-11-08T08:22:13.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15ef4dc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e814f020 con 0x7f15e80705c0 2023-11-08T08:22:14.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15edcd9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15e80cc860 0x7f15e80af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:14.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15edcd9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15e80cc860 0x7f15e80af200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34678/0 (socket says 172.21.15.40:34678) 2023-11-08T08:22:14.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15edcd9700 1 -- 172.21.15.40:0/4135367853 learned_addr learned my addr 172.21.15.40:0/4135367853 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:14.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15ee4da700 1 --2- 172.21.15.40:0/4135367853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15e80705c0 0x7f15e80aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:14.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15eecdb700 1 --2- 172.21.15.40:0/4135367853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e8160710 0x7f15e81f9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:14.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15edcd9700 1 -- 172.21.15.40:0/4135367853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15e80705c0 msgr2=0x7f15e80aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15edcd9700 1 --2- 172.21.15.40:0/4135367853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15e80705c0 0x7f15e80aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15edcd9700 1 -- 172.21.15.40:0/4135367853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e8160710 msgr2=0x7f15e81f9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15edcd9700 1 --2- 172.21.15.40:0/4135367853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e8160710 0x7f15e81f9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15edcd9700 1 -- 172.21.15.40:0/4135367853 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d80094b0 con 0x7f15e80cc860 2023-11-08T08:22:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15edcd9700 1 --2- 172.21.15.40:0/4135367853 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15e80cc860 0x7f15e80af200 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f15d800ebe0 tx=0x7f15d800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15deffd700 1 -- 172.21.15.40:0/4135367853 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d800c850 con 0x7f15e80cc860 2023-11-08T08:22:14.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15deffd700 1 -- 172.21.15.40:0/4135367853 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d800c9b0 con 0x7f15e80cc860 2023-11-08T08:22:14.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15ef4dc700 1 -- 172.21.15.40:0/4135367853 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e4009570 con 0x7f15e80cc860 2023-11-08T08:22:14.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15deffd700 1 -- 172.21.15.40:0/4135367853 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d800cd40 con 0x7f15e80cc860 2023-11-08T08:22:14.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.991+0000 7f15ef4dc700 1 -- 172.21.15.40:0/4135367853 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e0009a70 con 0x7f15e80cc860 2023-11-08T08:22:14.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.995+0000 7f15ef4dc700 1 -- 172.21.15.40:0/4135367853 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e806b6b0 con 0x7f15e80cc860 2023-11-08T08:22:14.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.995+0000 7f15deffd700 1 -- 172.21.15.40:0/4135367853 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f15d8019030 con 0x7f15e80cc860 2023-11-08T08:22:14.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.999+0000 7f15deffd700 1 --2- 172.21.15.40:0/4135367853 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f15cc064f30 0x7f15cc0673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:14.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.999+0000 7f15deffd700 1 -- 172.21.15.40:0/4135367853 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f15d8081c00 con 0x7f15e80cc860 2023-11-08T08:22:14.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.999+0000 7f15ee4da700 1 --2- 172.21.15.40:0/4135367853 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f15cc064f30 0x7f15cc0673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:14.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:13.999+0000 7f15ee4da700 1 --2- 172.21.15.40:0/4135367853 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f15cc064f30 0x7f15cc0673e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f15e4006d50 tx=0x7f15e40092e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:14.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.007+0000 7f15deffd700 1 -- 172.21.15.40:0/4135367853 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f15d804ff10 con 0x7f15e80cc860 2023-11-08T08:22:14.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.219+0000 7f15ef4dc700 1 -- 172.21.15.40:0/4135367853 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15e80ad8f0 con 0x7f15e80cc860 2023-11-08T08:22:14.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.223+0000 7f15deffd700 1 -- 172.21.15.40:0/4135367853 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f15d8017de0 con 0x7f15e80cc860 2023-11-08T08:22:14.229 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:22:14.230 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1549516710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1549516710},{"type":"v1","addr":"172.21.15.169:6837","nonce":1549516710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5728,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/912870125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":912870125},{"type":"v1","addr":"172.21.15.169:6835","nonce":912870125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5772,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1045606427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1045606427},{"type":"v1","addr":"172.21.15.40:6834","nonce":1045606427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2256363304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2256363304},{"type":"v1","addr":"172.21.15.40:6835","nonce":2256363304}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[]} 2023-11-08T08:22:14.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.227+0000 7f15ef4dc700 1 -- 172.21.15.40:0/4135367853 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f15cc064f30 msgr2=0x7f15cc0673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:14.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.227+0000 7f15ef4dc700 1 --2- 172.21.15.40:0/4135367853 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f15cc064f30 0x7f15cc0673e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f15e4006d50 tx=0x7f15e40092e0 comp rx=0 tx=0).stop 2023-11-08T08:22:14.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.227+0000 7f15ef4dc700 1 -- 172.21.15.40:0/4135367853 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15e80cc860 msgr2=0x7f15e80af200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:14.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.227+0000 7f15ef4dc700 1 --2- 172.21.15.40:0/4135367853 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15e80cc860 0x7f15e80af200 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f15d800ebe0 tx=0x7f15d800eef0 comp rx=0 tx=0).stop 2023-11-08T08:22:14.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.227+0000 7f15ef4dc700 1 -- 172.21.15.40:0/4135367853 shutdown_connections 2023-11-08T08:22:14.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.227+0000 7f15ef4dc700 1 --2- 172.21.15.40:0/4135367853 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f15cc064f30 0x7f15cc0673e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:14.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.227+0000 7f15ef4dc700 1 --2- 172.21.15.40:0/4135367853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e8160710 0x7f15e81f9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:14.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.227+0000 7f15ef4dc700 1 --2- 172.21.15.40:0/4135367853 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15e80cc860 0x7f15e80af200 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:14.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.227+0000 7f15ef4dc700 1 --2- 172.21.15.40:0/4135367853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15e80705c0 0x7f15e80aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:14.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.227+0000 7f15ef4dc700 1 -- 172.21.15.40:0/4135367853 >> 172.21.15.40:0/4135367853 conn(0x7f15e80b0f80 msgr2=0x7f15e806a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:14.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.227+0000 7f15ef4dc700 1 -- 172.21.15.40:0/4135367853 shutdown_connections 2023-11-08T08:22:14.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.227+0000 7f15ef4dc700 1 -- 172.21.15.40:0/4135367853 wait complete. 2023-11-08T08:22:14.237 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 27 2023-11-08T08:22:14.256 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T08:22:14.256 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T08:22:14.257 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T08:22:14.257 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:22:14.257 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T08:22:14.258 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T08:22:14.258 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T08:22:14.258 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:22:14.259 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T08:22:14.259 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T08:22:14.259 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T08:22:14.259 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:22:14.260 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T08:22:14.260 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T08:22:14.260 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T08:22:14.260 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:22:14.273 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T08:22:14.274 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T08:22:14.281 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T08:22:14.282 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T08:22:14.283 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T08:22:14.283 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T08:22:14.284 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T08:22:14.285 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T08:22:14.331 INFO:tasks.ceph.mds.a:Started 2023-11-08T08:22:14.332 DEBUG:teuthology.parallel:result is None 2023-11-08T08:22:14.335 INFO:tasks.ceph.mds.c:Started 2023-11-08T08:22:14.336 DEBUG:teuthology.parallel:result is None 2023-11-08T08:22:14.337 INFO:tasks.ceph.mds.d:Started 2023-11-08T08:22:14.337 DEBUG:teuthology.parallel:result is None 2023-11-08T08:22:14.342 INFO:tasks.ceph.mds.b:Started 2023-11-08T08:22:14.343 DEBUG:teuthology.parallel:result is None 2023-11-08T08:22:14.343 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-08T08:22:14.445 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2023-11-08T08:22:14.448 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-11-08T08:22:14.461 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2023-11-08T08:22:14.471 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2023-11-08T08:22:14.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.527+0000 7f6909416700 1 Processor -- start 2023-11-08T08:22:14.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.527+0000 7f6909416700 1 -- start start 2023-11-08T08:22:14.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.531+0000 7f6909416700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f69040b43a0 0x7f690415e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:14.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.531+0000 7f6909416700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6904070790 0x7f690406bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:14.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.531+0000 7f6909416700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f690415a550 con 0x7f69040b1640 2023-11-08T08:22:14.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.531+0000 7f6909416700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f690406b0a0 con 0x7f69040b43a0 2023-11-08T08:22:14.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.531+0000 7f6909416700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69040caac0 con 0x7f6904070790 2023-11-08T08:22:14.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.531+0000 7f6902ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f69040b43a0 0x7f690415e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:14.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.531+0000 7f68fbfff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f69040b1640 0x7f6904070080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50968/0 (socket says 172.21.15.40:50968) 2023-11-08T08:22:14.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.531+0000 7f68fbfff700 1 -- 172.21.15.40:0/2152287389 learned_addr learned my addr 172.21.15.40:0/2152287389 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.531+0000 7f69037fe700 1 --2- 172.21.15.40:0/2152287389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6904070790 0x7f690406bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.531+0000 7f6902ffd700 1 -- 172.21.15.40:0/2152287389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6904070790 msgr2=0x7f690406bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.531+0000 7f6902ffd700 1 --2- 172.21.15.40:0/2152287389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6904070790 0x7f690406bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.531+0000 7f6902ffd700 1 -- 172.21.15.40:0/2152287389 >> v1:172.21.15.40:6789/0 conn(0x7f69040b1640 legacy=0x7f6904070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.531+0000 7f6902ffd700 1 -- 172.21.15.40:0/2152287389 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69040cada0 con 0x7f69040b43a0 2023-11-08T08:22:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.531+0000 7f69027fc700 1 -- 172.21.15.40:0/2152287389 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2573992026 0 0) 0x7f68ec001110 con 0x7f69040b1640 2023-11-08T08:22:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.531+0000 7f6902ffd700 1 --2- 172.21.15.40:0/2152287389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69040b43a0 0x7f690415e5d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009850 tx=0x7f68f4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3dd0de25c0454c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.535+0000 7f69027fc700 1 -- 172.21.15.40:0/2152287389 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4011830 con 0x7f69040b43a0 2023-11-08T08:22:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.535+0000 7f69027fc700 1 -- 172.21.15.40:0/2152287389 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f4011990 con 0x7f69040b43a0 2023-11-08T08:22:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.535+0000 7f69027fc700 1 -- 172.21.15.40:0/2152287389 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4011d20 con 0x7f69040b43a0 2023-11-08T08:22:14.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.535+0000 7f6909416700 1 -- 172.21.15.40:0/2152287389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69040b43a0 msgr2=0x7f690415e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:14.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.535+0000 7f6909416700 1 --2- 172.21.15.40:0/2152287389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69040b43a0 0x7f690415e5d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009850 tx=0x7f68f4009b60 comp rx=0 tx=0).stop 2023-11-08T08:22:14.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.535+0000 7f6909416700 1 -- 172.21.15.40:0/2152287389 shutdown_connections 2023-11-08T08:22:14.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.535+0000 7f6909416700 1 --2- 172.21.15.40:0/2152287389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6904070790 0x7f690406bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:14.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.535+0000 7f6909416700 1 --2- 172.21.15.40:0/2152287389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69040b43a0 0x7f690415e5d0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:14.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.535+0000 7f6909416700 1 -- 172.21.15.40:0/2152287389 >> 172.21.15.40:0/2152287389 conn(0x7f69040b0f80 msgr2=0x7f69040ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:14.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.535+0000 7f6909416700 1 -- 172.21.15.40:0/2152287389 shutdown_connections 2023-11-08T08:22:14.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.535+0000 7f6909416700 1 -- 172.21.15.40:0/2152287389 wait complete. 2023-11-08T08:22:14.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.539+0000 7f6909416700 1 Processor -- start 2023-11-08T08:22:14.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.539+0000 7f6909416700 1 -- start start 2023-11-08T08:22:14.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.539+0000 7f6909416700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6904070790 0x7f6904152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:14.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.539+0000 7f6909416700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69040b1640 0x7f69040c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:14.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.539+0000 7f6909416700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69040b43a0 0x7f6904157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:14.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.539+0000 7f6909416700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690415a6f0 con 0x7f69040b43a0 2023-11-08T08:22:14.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.539+0000 7f6909416700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69041539c0 con 0x7f6904070790 2023-11-08T08:22:14.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.539+0000 7f6909416700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690415b200 con 0x7f69040b1640 2023-11-08T08:22:14.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.539+0000 7f69037fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69040b43a0 0x7f6904157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:14.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.539+0000 7f68fbfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69040b1640 0x7f69040c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:14.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.543+0000 7f6902ffd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6904070790 0x7f6904152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:14.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.543+0000 7f69037fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69040b43a0 0x7f6904157fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34720/0 (socket says 172.21.15.40:34720) 2023-11-08T08:22:14.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.543+0000 7f69037fe700 1 -- 172.21.15.40:0/951716020 learned_addr learned my addr 172.21.15.40:0/951716020 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:14.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.543+0000 7f69037fe700 1 -- 172.21.15.40:0/951716020 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69040b1640 msgr2=0x7f69040c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:14.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.543+0000 7f69037fe700 1 --2- 172.21.15.40:0/951716020 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69040b1640 0x7f69040c7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:14.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.543+0000 7f69037fe700 1 -- 172.21.15.40:0/951716020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6904070790 msgr2=0x7f6904152ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:14.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.543+0000 7f69037fe700 1 --2- 172.21.15.40:0/951716020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6904070790 0x7f6904152ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:14.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.543+0000 7f69037fe700 1 -- 172.21.15.40:0/951716020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f40094b0 con 0x7f69040b43a0 2023-11-08T08:22:14.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.543+0000 7f69037fe700 1 --2- 172.21.15.40:0/951716020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69040b43a0 0x7f6904157fe0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f68fc007e60 tx=0x7f68fc00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:14.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.547+0000 7f68fb7fe700 1 -- 172.21.15.40:0/951716020 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc00ed30 con 0x7f69040b43a0 2023-11-08T08:22:14.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.547+0000 7f68fb7fe700 1 -- 172.21.15.40:0/951716020 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68fc014420 con 0x7f69040b43a0 2023-11-08T08:22:14.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.547+0000 7f68fb7fe700 1 -- 172.21.15.40:0/951716020 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc0147d0 con 0x7f69040b43a0 2023-11-08T08:22:14.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.547+0000 7f6909416700 1 -- 172.21.15.40:0/951716020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690406b5d0 con 0x7f69040b43a0 2023-11-08T08:22:14.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.547+0000 7f6909416700 1 -- 172.21.15.40:0/951716020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690406b2f0 con 0x7f69040b43a0 2023-11-08T08:22:14.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.547+0000 7f6909416700 1 -- 172.21.15.40:0/951716020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690415eb10 con 0x7f69040b43a0 2023-11-08T08:22:14.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.551+0000 7f68fb7fe700 1 -- 172.21.15.40:0/951716020 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f68fc012050 con 0x7f69040b43a0 2023-11-08T08:22:14.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.551+0000 7f68fb7fe700 1 --2- 172.21.15.40:0/951716020 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f68e4064f10 0x7f68e40673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:14.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.551+0000 7f68fb7fe700 1 -- 172.21.15.40:0/951716020 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f68fc082200 con 0x7f69040b43a0 2023-11-08T08:22:14.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.551+0000 7f6902ffd700 1 --2- 172.21.15.40:0/951716020 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f68e4064f10 0x7f68e40673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:14.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.555+0000 7f6902ffd700 1 --2- 172.21.15.40:0/951716020 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f68e4064f10 0x7f68e40673c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f68f4000c00 tx=0x7f68f4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:14.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.559+0000 7f68fb7fe700 1 -- 172.21.15.40:0/951716020 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f68fc050490 con 0x7f69040b43a0 2023-11-08T08:22:14.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:14.747+0000 7f6909416700 1 -- 172.21.15.40:0/951716020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6904073cb0 con 0x7f69040b43a0 2023-11-08T08:22:15.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.551+0000 7f68fb7fe700 1 -- 172.21.15.40:0/951716020 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v65) v1 ==== 97+0+0 (secure 0 0 0) 0x7f68fc053ab0 con 0x7f69040b43a0 2023-11-08T08:22:15.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.555+0000 7f6909416700 1 -- 172.21.15.40:0/951716020 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f68e4064f10 msgr2=0x7f68e40673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:15.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.555+0000 7f6909416700 1 --2- 172.21.15.40:0/951716020 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f68e4064f10 0x7f68e40673c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f68f4000c00 tx=0x7f68f4008030 comp rx=0 tx=0).stop 2023-11-08T08:22:15.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.555+0000 7f6909416700 1 -- 172.21.15.40:0/951716020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69040b43a0 msgr2=0x7f6904157fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:15.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.555+0000 7f6909416700 1 --2- 172.21.15.40:0/951716020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69040b43a0 0x7f6904157fe0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f68fc007e60 tx=0x7f68fc00e3c0 comp rx=0 tx=0).stop 2023-11-08T08:22:15.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.555+0000 7f6909416700 1 -- 172.21.15.40:0/951716020 shutdown_connections 2023-11-08T08:22:15.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.555+0000 7f6909416700 1 --2- 172.21.15.40:0/951716020 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f68e4064f10 0x7f68e40673c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:15.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.555+0000 7f6909416700 1 --2- 172.21.15.40:0/951716020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69040b43a0 0x7f6904157fe0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:15.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.555+0000 7f6909416700 1 --2- 172.21.15.40:0/951716020 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69040b1640 0x7f69040c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:15.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.555+0000 7f6909416700 1 --2- 172.21.15.40:0/951716020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6904070790 0x7f6904152ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:15.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.555+0000 7f6909416700 1 -- 172.21.15.40:0/951716020 >> 172.21.15.40:0/951716020 conn(0x7f69040b0f80 msgr2=0x7f6904069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:15.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.559+0000 7f6909416700 1 -- 172.21.15.40:0/951716020 shutdown_connections 2023-11-08T08:22:15.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.559+0000 7f6909416700 1 -- 172.21.15.40:0/951716020 wait complete. 2023-11-08T08:22:15.567 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2023-11-08T08:22:15.589 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-08T08:22:15.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.771+0000 7f662d09f700 1 Processor -- start 2023-11-08T08:22:15.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.771+0000 7f662d09f700 1 -- start start 2023-11-08T08:22:15.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.771+0000 7f662d09f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6628071640 0x7f66280d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:15.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.771+0000 7f662d09f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66280d0670 con 0x7f6628156820 2023-11-08T08:22:15.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.771+0000 7f662d09f700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66280d0c30 con 0x7f662815fe30 2023-11-08T08:22:15.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.771+0000 7f662d09f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6628070b50 con 0x7f6628071640 2023-11-08T08:22:15.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.771+0000 7f6627fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6628156820 0x7f66280b1ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50974/0 (socket says 172.21.15.40:50974) 2023-11-08T08:22:15.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.771+0000 7f6627fff700 1 -- 172.21.15.40:0/32882261 learned_addr learned my addr 172.21.15.40:0/32882261 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:15.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.771+0000 7f66277fe700 1 --2- 172.21.15.40:0/32882261 >> v2:172.21.15.40:3300/0 conn(0x7f6628071640 0x7f66280d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:15.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.771+0000 7f6626ffd700 1 -- 172.21.15.40:0/32882261 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884194574 0 0) 0x7f66280d0670 con 0x7f6628156820 2023-11-08T08:22:15.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.771+0000 7f6626ffd700 1 -- 172.21.15.40:0/32882261 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66100035d0 con 0x7f6628156820 2023-11-08T08:22:15.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.775+0000 7f66277fe700 1 -- 172.21.15.40:0/32882261 >> v1:172.21.15.169:6789/0 conn(0x7f662815fe30 legacy=0x7f6628160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:15.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.775+0000 7f66277fe700 1 -- 172.21.15.40:0/32882261 >> v1:172.21.15.40:6789/0 conn(0x7f6628156820 legacy=0x7f66280b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:15.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.775+0000 7f66277fe700 1 -- 172.21.15.40:0/32882261 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6628070e30 con 0x7f6628071640 2023-11-08T08:22:15.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.775+0000 7f66277fe700 1 --2- 172.21.15.40:0/32882261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6628071640 0x7f66280d2270 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f661c009850 tx=0x7f661c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa24233cccb86f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:15.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.775+0000 7f6626ffd700 1 -- 172.21.15.40:0/32882261 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c011830 con 0x7f6628071640 2023-11-08T08:22:15.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.775+0000 7f6626ffd700 1 -- 172.21.15.40:0/32882261 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661c011990 con 0x7f6628071640 2023-11-08T08:22:15.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.775+0000 7f6626ffd700 1 -- 172.21.15.40:0/32882261 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c011d40 con 0x7f6628071640 2023-11-08T08:22:15.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.775+0000 7f662d09f700 1 -- 172.21.15.40:0/32882261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6628071640 msgr2=0x7f66280d2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:15.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.775+0000 7f662d09f700 1 --2- 172.21.15.40:0/32882261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6628071640 0x7f66280d2270 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f661c009850 tx=0x7f661c009b60 comp rx=0 tx=0).stop 2023-11-08T08:22:15.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.775+0000 7f662d09f700 1 -- 172.21.15.40:0/32882261 shutdown_connections 2023-11-08T08:22:15.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.775+0000 7f662d09f700 1 --2- 172.21.15.40:0/32882261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6628071640 0x7f66280d2270 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:15.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.775+0000 7f662d09f700 1 -- 172.21.15.40:0/32882261 >> 172.21.15.40:0/32882261 conn(0x7f662806cb80 msgr2=0x7f66280b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:15.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.775+0000 7f662d09f700 1 -- 172.21.15.40:0/32882261 shutdown_connections 2023-11-08T08:22:15.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.775+0000 7f662d09f700 1 -- 172.21.15.40:0/32882261 wait complete. 2023-11-08T08:22:15.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.779+0000 7f662d09f700 1 Processor -- start 2023-11-08T08:22:15.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.779+0000 7f662d09f700 1 -- start start 2023-11-08T08:22:15.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.779+0000 7f662d09f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6628071640 0x7f662814eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:15.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.779+0000 7f662d09f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6628156820 0x7f66281452c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:15.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.779+0000 7f662d09f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f662815fe30 0x7f66280b78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:15.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.779+0000 7f662d09f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6628145800 con 0x7f6628071640 2023-11-08T08:22:15.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.779+0000 7f662d09f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66280b7e00 con 0x7f662815fe30 2023-11-08T08:22:15.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.779+0000 7f662d09f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66280b8050 con 0x7f6628156820 2023-11-08T08:22:15.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.779+0000 7f6627fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6628071640 0x7f662814eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:15.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.779+0000 7f6627fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6628071640 0x7f662814eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34734/0 (socket says 172.21.15.40:34734) 2023-11-08T08:22:15.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.779+0000 7f6627fff700 1 -- 172.21.15.40:0/1801592042 learned_addr learned my addr 172.21.15.40:0/1801592042 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:15.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.783+0000 7f662c89e700 1 --2- 172.21.15.40:0/1801592042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f662815fe30 0x7f66280b78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:15.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.783+0000 7f6627fff700 1 -- 172.21.15.40:0/1801592042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6628156820 msgr2=0x7f66281452c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:15.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.783+0000 7f66277fe700 1 --2- 172.21.15.40:0/1801592042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6628156820 0x7f66281452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:15.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.783+0000 7f6627fff700 1 --2- 172.21.15.40:0/1801592042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6628156820 0x7f66281452c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:15.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.783+0000 7f6627fff700 1 -- 172.21.15.40:0/1801592042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f662815fe30 msgr2=0x7f66280b78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:15.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.783+0000 7f6627fff700 1 --2- 172.21.15.40:0/1801592042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f662815fe30 0x7f66280b78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:15.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.783+0000 7f6627fff700 1 -- 172.21.15.40:0/1801592042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661c0094b0 con 0x7f6628071640 2023-11-08T08:22:15.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.783+0000 7f662c89e700 1 --2- 172.21.15.40:0/1801592042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f662815fe30 0x7f66280b78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:22:15.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.783+0000 7f66277fe700 1 --2- 172.21.15.40:0/1801592042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6628156820 0x7f66281452c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:22:15.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.783+0000 7f6627fff700 1 --2- 172.21.15.40:0/1801592042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6628071640 0x7f662814eb40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f662000cbc0 tx=0x7f662000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:15.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.783+0000 7f6624ff9700 1 -- 172.21.15.40:0/1801592042 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662000a9e0 con 0x7f6628071640 2023-11-08T08:22:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.783+0000 7f6624ff9700 1 -- 172.21.15.40:0/1801592042 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662000ab40 con 0x7f6628071640 2023-11-08T08:22:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.783+0000 7f662d09f700 1 -- 172.21.15.40:0/1801592042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66280d27b0 con 0x7f6628071640 2023-11-08T08:22:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.783+0000 7f6624ff9700 1 -- 172.21.15.40:0/1801592042 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6620012400 con 0x7f6628071640 2023-11-08T08:22:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.783+0000 7f662d09f700 1 -- 172.21.15.40:0/1801592042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66280d08c0 con 0x7f6628071640 2023-11-08T08:22:15.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.783+0000 7f662d09f700 1 -- 172.21.15.40:0/1801592042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6628156bf0 con 0x7f6628071640 2023-11-08T08:22:15.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.787+0000 7f6624ff9700 1 -- 172.21.15.40:0/1801592042 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f6620019030 con 0x7f6628071640 2023-11-08T08:22:15.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.787+0000 7f6624ff9700 1 --2- 172.21.15.40:0/1801592042 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6614064ee0 0x7f6614067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:15.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.787+0000 7f6624ff9700 1 -- 172.21.15.40:0/1801592042 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f6620081770 con 0x7f6628071640 2023-11-08T08:22:15.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.787+0000 7f66277fe700 1 --2- 172.21.15.40:0/1801592042 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6614064ee0 0x7f6614067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:15.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.791+0000 7f66277fe700 1 --2- 172.21.15.40:0/1801592042 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6614064ee0 0x7f6614067390 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f66280cfd00 tx=0x7f661c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:15.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:15.795+0000 7f6624ff9700 1 -- 172.21.15.40:0/1801592042 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f662004fa90 con 0x7f6628071640 2023-11-08T08:22:16.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.035+0000 7f662d09f700 1 -- 172.21.15.40:0/1801592042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6628073cb0 con 0x7f6628071640 2023-11-08T08:22:16.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.039+0000 7f6624ff9700 1 -- 172.21.15.40:0/1801592042 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f6620017020 con 0x7f6628071640 2023-11-08T08:22:16.046 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:22:16.046 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:22:16.046 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2023-11-08T08:22:16.047 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:16.047 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2023-11-08T08:22:16.047 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA3REtlj6YZGRAABMVq4pKyrp6U3tbqShtmuw==", 2023-11-08T08:22:16.047 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:22:16.047 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:22:16.048 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-11-08T08:22:16.048 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-11-08T08:22:16.048 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:22:16.048 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:16.048 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:22:16.049 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:16.049 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2023-11-08T08:22:16.049 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA3REtlAfqKLRAAmNg22Rw+FWmzeRDBKQtjfw==", 2023-11-08T08:22:16.049 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:22:16.049 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:22:16.050 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-11-08T08:22:16.050 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-11-08T08:22:16.050 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:22:16.050 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:16.050 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:22:16.051 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:16.051 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2023-11-08T08:22:16.051 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA3REtlRseiJBAAZFNQO+PK+kioVBn+S/54Hw==", 2023-11-08T08:22:16.051 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:22:16.051 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:22:16.052 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-11-08T08:22:16.052 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-11-08T08:22:16.052 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:22:16.052 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:16.052 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:22:16.053 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:16.053 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2023-11-08T08:22:16.053 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA3REtlZHCiOBAAW8bJst1bqTZ2jPOr+wXcpg==", 2023-11-08T08:22:16.053 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:22:16.053 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:22:16.054 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-11-08T08:22:16.054 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-11-08T08:22:16.054 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:22:16.054 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:16.054 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:22:16.055 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:16.055 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2023-11-08T08:22:16.055 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA7REtlvWsaLBAAws4XZXHoYl9KnhBp6LEh0w==", 2023-11-08T08:22:16.055 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:22:16.055 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:22:16.056 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:22:16.056 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:22:16.056 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:16.056 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:22:16.056 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:16.057 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2023-11-08T08:22:16.057 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA+REtl8FYFOhAAOY5q9LJAl6PS+essHcF8cw==", 2023-11-08T08:22:16.057 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:22:16.057 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:22:16.057 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:22:16.058 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:22:16.058 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:16.058 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:22:16.058 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:16.058 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2023-11-08T08:22:16.059 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBCREtljU1ODBAA1gVE4sX2QNdGHIjdRue/rA==", 2023-11-08T08:22:16.059 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:22:16.059 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:22:16.059 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:22:16.059 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:22:16.060 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:16.060 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:22:16.060 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:16.060 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2023-11-08T08:22:16.060 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBFREtlRZsSGxAAh7v5f7TVZNQozESV6k641g==", 2023-11-08T08:22:16.061 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:22:16.061 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:22:16.061 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:22:16.061 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:22:16.061 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:16.061 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:22:16.062 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:16.062 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2023-11-08T08:22:16.062 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBLREtla+qaNRAA0jaCFarD/RgWGp1ltYXQdA==", 2023-11-08T08:22:16.062 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:22:16.063 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:22:16.063 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:22:16.063 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:22:16.063 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:16.063 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:22:16.064 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:16.064 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2023-11-08T08:22:16.064 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPREtltHKNCRAA/TKgyxOGRkN5y/RHWr8vSQ==", 2023-11-08T08:22:16.064 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:22:16.064 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:22:16.065 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:22:16.065 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:22:16.065 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:16.065 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:22:16.065 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:16.066 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2023-11-08T08:22:16.066 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBSREtlpXC4GRAAZl6YWPVnbX3o1tyfEkR7CA==", 2023-11-08T08:22:16.066 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:22:16.066 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:22:16.067 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:22:16.067 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:22:16.067 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:16.067 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:22:16.067 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:16.068 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2023-11-08T08:22:16.068 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBVREtlvL/4JxAAZtGdthYoFzdMRs3DMcRXNw==", 2023-11-08T08:22:16.068 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:22:16.068 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-11-08T08:22:16.068 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-11-08T08:22:16.068 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:22:16.069 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:16.069 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:22:16.069 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:16.069 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2023-11-08T08:22:16.070 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA4REtl0YCpBBAAtap/irtK+GkSmR8g37HRVQ==", 2023-11-08T08:22:16.070 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:22:16.070 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:22:16.070 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2023-11-08T08:22:16.070 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-08T08:22:16.071 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:16.071 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:22:16.071 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:16.071 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2023-11-08T08:22:16.071 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA4REtlgapFERAAxg8GS7iiH06t2mCF5ImFXA==", 2023-11-08T08:22:16.071 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:22:16.072 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:22:16.072 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-11-08T08:22:16.072 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-11-08T08:22:16.072 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-11-08T08:22:16.072 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:16.073 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:22:16.073 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:16.073 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2023-11-08T08:22:16.073 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA4REtldlrMChAAC9+syjsFWWFwJGXPw82zwQ==", 2023-11-08T08:22:16.073 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:22:16.073 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:22:16.074 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-11-08T08:22:16.074 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-11-08T08:22:16.074 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-11-08T08:22:16.074 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:16.075 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:22:16.075 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:16.075 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2023-11-08T08:22:16.075 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA4REtllHEDFxAACJSjTZQaOTnyA0K3X/igXQ==", 2023-11-08T08:22:16.075 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:22:16.075 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-11-08T08:22:16.076 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-11-08T08:22:16.076 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-11-08T08:22:16.076 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-11-08T08:22:16.076 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:16.076 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:22:16.077 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:16.077 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2023-11-08T08:22:16.077 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA2REtlqqakKRAAzyr9EbqU+Pj0m2weeru80g==", 2023-11-08T08:22:16.077 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:22:16.077 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-11-08T08:22:16.078 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-11-08T08:22:16.078 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-11-08T08:22:16.078 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:22:16.078 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:16.079 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:22:16.079 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:16.079 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2023-11-08T08:22:16.079 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA3REtl6YeRERAAKW6fA2vM7HOpZGCulFFLzQ==", 2023-11-08T08:22:16.079 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:22:16.080 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-11-08T08:22:16.080 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-11-08T08:22:16.080 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:22:16.080 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:16.080 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:22:16.081 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:16.081 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2023-11-08T08:22:16.081 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQA3REtlmuaiChAAsI2681k5qbe8I0hdIXHJGg==", 2023-11-08T08:22:16.081 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-11-08T08:22:16.081 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-11-08T08:22:16.082 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-11-08T08:22:16.082 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-11-08T08:22:16.082 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:16.082 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:16.082 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-11-08T08:22:16.083 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:22:16.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.043+0000 7f662d09f700 1 -- 172.21.15.40:0/1801592042 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6614064ee0 msgr2=0x7f6614067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:16.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.043+0000 7f662d09f700 1 --2- 172.21.15.40:0/1801592042 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6614064ee0 0x7f6614067390 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f66280cfd00 tx=0x7f661c008030 comp rx=0 tx=0).stop 2023-11-08T08:22:16.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.043+0000 7f662d09f700 1 -- 172.21.15.40:0/1801592042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6628071640 msgr2=0x7f662814eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:16.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.043+0000 7f662d09f700 1 --2- 172.21.15.40:0/1801592042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6628071640 0x7f662814eb40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f662000cbc0 tx=0x7f662000cca0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.043+0000 7f662d09f700 1 -- 172.21.15.40:0/1801592042 shutdown_connections 2023-11-08T08:22:16.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.043+0000 7f662d09f700 1 --2- 172.21.15.40:0/1801592042 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6614064ee0 0x7f6614067390 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.043+0000 7f662d09f700 1 --2- 172.21.15.40:0/1801592042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f662815fe30 0x7f66280b78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.043+0000 7f662d09f700 1 --2- 172.21.15.40:0/1801592042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6628156820 0x7f66281452c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.043+0000 7f662d09f700 1 --2- 172.21.15.40:0/1801592042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6628071640 0x7f662814eb40 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.043+0000 7f662d09f700 1 -- 172.21.15.40:0/1801592042 >> 172.21.15.40:0/1801592042 conn(0x7f662806cb80 msgr2=0x7f662815e740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:16.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.043+0000 7f662d09f700 1 -- 172.21.15.40:0/1801592042 shutdown_connections 2023-11-08T08:22:16.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.043+0000 7f662d09f700 1 -- 172.21.15.40:0/1801592042 wait complete. 2023-11-08T08:22:16.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:22:16.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.255+0000 7f32fe370700 1 Processor -- start 2023-11-08T08:22:16.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.259+0000 7f32fe370700 1 -- start start 2023-11-08T08:22:16.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.259+0000 7f32fe370700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f32f8160710 0x7f32f80cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:16.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.259+0000 7f32fe370700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32f80cc860 0x7f32f8070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:16.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.259+0000 7f32fe370700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f32f80705c0 0x7f32f8069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:16.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.259+0000 7f32fe370700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f8070a60 con 0x7f32f80705c0 2023-11-08T08:22:16.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.259+0000 7f32fe370700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f806b1c0 con 0x7f32f8160710 2023-11-08T08:22:16.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.259+0000 7f32fe370700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80ad6c0 con 0x7f32f80cc860 2023-11-08T08:22:16.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.259+0000 7f32fdb6f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f32f80705c0 0x7f32f8069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:16.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.259+0000 7f32fcb6d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32f80cc860 0x7f32f8070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:16.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.259+0000 7f32fdb6f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f32f80705c0 0x7f32f8069bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34740/0 (socket says 172.21.15.40:34740) 2023-11-08T08:22:16.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.259+0000 7f32fcb6d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32f80cc860 0x7f32f8070080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:55776/0 (socket says 172.21.15.40:55776) 2023-11-08T08:22:16.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.259+0000 7f32fdb6f700 1 -- 172.21.15.40:0/2961581344 learned_addr learned my addr 172.21.15.40:0/2961581344 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:16.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.259+0000 7f32fd36e700 1 --2- 172.21.15.40:0/2961581344 >> v2:172.21.15.169:3300/0 conn(0x7f32f8160710 0x7f32f80cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:16.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.263+0000 7f32fdb6f700 1 -- 172.21.15.40:0/2961581344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32f80cc860 msgr2=0x7f32f8070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:16.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.263+0000 7f32fdb6f700 1 --2- 172.21.15.40:0/2961581344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32f80cc860 0x7f32f8070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.263+0000 7f32fdb6f700 1 -- 172.21.15.40:0/2961581344 >> v2:172.21.15.169:3300/0 conn(0x7f32f8160710 msgr2=0x7f32f80cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:16.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.263+0000 7f32fdb6f700 1 --2- 172.21.15.40:0/2961581344 >> v2:172.21.15.169:3300/0 conn(0x7f32f8160710 0x7f32f80cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.263+0000 7f32fdb6f700 1 -- 172.21.15.40:0/2961581344 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f80adc60 con 0x7f32f80705c0 2023-11-08T08:22:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.263+0000 7f32fdb6f700 1 --2- 172.21.15.40:0/2961581344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f80705c0 0x7f32f8069bf0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f32f8072ed0 tx=0x7f32f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0fab4c6d2fc8731 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.263+0000 7f32effff700 1 -- 172.21.15.40:0/2961581344 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f000e070 con 0x7f32f80705c0 2023-11-08T08:22:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.263+0000 7f32effff700 1 -- 172.21.15.40:0/2961581344 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f000b910 con 0x7f32f80705c0 2023-11-08T08:22:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.263+0000 7f32effff700 1 -- 172.21.15.40:0/2961581344 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f000bcc0 con 0x7f32f80705c0 2023-11-08T08:22:16.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.263+0000 7f32fe370700 1 -- 172.21.15.40:0/2961581344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f80705c0 msgr2=0x7f32f8069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.263+0000 7f32fe370700 1 --2- 172.21.15.40:0/2961581344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f80705c0 0x7f32f8069bf0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f32f8072ed0 tx=0x7f32f000b330 comp rx=0 tx=0).stop 2023-11-08T08:22:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.263+0000 7f32fe370700 1 -- 172.21.15.40:0/2961581344 shutdown_connections 2023-11-08T08:22:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.263+0000 7f32fe370700 1 --2- 172.21.15.40:0/2961581344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f80705c0 0x7f32f8069bf0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.263+0000 7f32fe370700 1 --2- 172.21.15.40:0/2961581344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32f80cc860 0x7f32f8070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.263+0000 7f32fe370700 1 --2- 172.21.15.40:0/2961581344 >> v2:172.21.15.169:3300/0 conn(0x7f32f8160710 0x7f32f80cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.263+0000 7f32fe370700 1 -- 172.21.15.40:0/2961581344 >> 172.21.15.40:0/2961581344 conn(0x7f32f80b0f80 msgr2=0x7f32f80ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:16.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.263+0000 7f32fe370700 1 -- 172.21.15.40:0/2961581344 shutdown_connections 2023-11-08T08:22:16.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.263+0000 7f32fe370700 1 -- 172.21.15.40:0/2961581344 wait complete. 2023-11-08T08:22:16.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.267+0000 7f32fe370700 1 Processor -- start 2023-11-08T08:22:16.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.267+0000 7f32fe370700 1 -- start start 2023-11-08T08:22:16.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32fe370700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f80705c0 0x7f32f80c3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:16.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32fe370700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f80cc860 0x7f32f815f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:16.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32fe370700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32f8160710 0x7f32f815f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:16.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32fe370700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f815e0f0 con 0x7f32f80cc860 2023-11-08T08:22:16.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32fe370700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f815fee0 con 0x7f32f80705c0 2023-11-08T08:22:16.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32fe370700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80c3630 con 0x7f32f8160710 2023-11-08T08:22:16.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32fcb6d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f80cc860 0x7f32f815f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:16.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32fcb6d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f80cc860 0x7f32f815f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34742/0 (socket says 172.21.15.40:34742) 2023-11-08T08:22:16.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32fcb6d700 1 -- 172.21.15.40:0/753084475 learned_addr learned my addr 172.21.15.40:0/753084475 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32fd36e700 1 --2- 172.21.15.40:0/753084475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f80705c0 0x7f32f80c3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32fdb6f700 1 --2- 172.21.15.40:0/753084475 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32f8160710 0x7f32f815f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32fcb6d700 1 -- 172.21.15.40:0/753084475 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32f8160710 msgr2=0x7f32f815f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32fcb6d700 1 --2- 172.21.15.40:0/753084475 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32f8160710 0x7f32f815f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32fcb6d700 1 -- 172.21.15.40:0/753084475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f80705c0 msgr2=0x7f32f80c3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32fcb6d700 1 --2- 172.21.15.40:0/753084475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f80705c0 0x7f32f80c3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32fcb6d700 1 -- 172.21.15.40:0/753084475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f0009b30 con 0x7f32f80cc860 2023-11-08T08:22:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32fdb6f700 1 --2- 172.21.15.40:0/753084475 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32f8160710 0x7f32f815f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:22:16.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32fcb6d700 1 --2- 172.21.15.40:0/753084475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f80cc860 0x7f32f815f460 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f32e800e9d0 tx=0x7f32e800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:16.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32edffb700 1 -- 172.21.15.40:0/753084475 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8009720 con 0x7f32f80cc860 2023-11-08T08:22:16.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32edffb700 1 -- 172.21.15.40:0/753084475 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e800c850 con 0x7f32f80cc860 2023-11-08T08:22:16.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32fe370700 1 -- 172.21.15.40:0/753084475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f806b6b0 con 0x7f32f80cc860 2023-11-08T08:22:16.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32edffb700 1 -- 172.21.15.40:0/753084475 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e800cbc0 con 0x7f32f80cc860 2023-11-08T08:22:16.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.271+0000 7f32fe370700 1 -- 172.21.15.40:0/753084475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f806b3d0 con 0x7f32f80cc860 2023-11-08T08:22:16.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.275+0000 7f32fe370700 1 -- 172.21.15.40:0/753084475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f80ad8f0 con 0x7f32f80cc860 2023-11-08T08:22:16.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.275+0000 7f32edffb700 1 -- 172.21.15.40:0/753084475 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f32e8019030 con 0x7f32f80cc860 2023-11-08T08:22:16.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.275+0000 7f32edffb700 1 --2- 172.21.15.40:0/753084475 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f32dc064ee0 0x7f32dc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:16.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.279+0000 7f32edffb700 1 -- 172.21.15.40:0/753084475 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f32e8081d80 con 0x7f32f80cc860 2023-11-08T08:22:16.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.279+0000 7f32fd36e700 1 --2- 172.21.15.40:0/753084475 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f32dc064ee0 0x7f32dc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:16.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.279+0000 7f32fd36e700 1 --2- 172.21.15.40:0/753084475 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f32dc064ee0 0x7f32dc067390 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009a60 tx=0x7f32f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:16.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.279+0000 7f32edffb700 1 -- 172.21.15.40:0/753084475 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f32e8050040 con 0x7f32f80cc860 2023-11-08T08:22:16.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.451+0000 7f32fe370700 1 -- 172.21.15.40:0/753084475 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32f8073cb0 con 0x7f32f80cc860 2023-11-08T08:22:16.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.455+0000 7f32edffb700 1 -- 172.21.15.40:0/753084475 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+14376 (secure 0 0 0) 0x7f32e8053660 con 0x7f32f80cc860 2023-11-08T08:22:16.463 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:22:16.463 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":65,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:22:15.548364+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:22:16.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.459+0000 7f32fe370700 1 -- 172.21.15.40:0/753084475 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f32dc064ee0 msgr2=0x7f32dc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:16.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.459+0000 7f32fe370700 1 --2- 172.21.15.40:0/753084475 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f32dc064ee0 0x7f32dc067390 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009a60 tx=0x7f32f4008040 comp rx=0 tx=0).stop 2023-11-08T08:22:16.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.459+0000 7f32fe370700 1 -- 172.21.15.40:0/753084475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f80cc860 msgr2=0x7f32f815f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:16.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.459+0000 7f32fe370700 1 --2- 172.21.15.40:0/753084475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f80cc860 0x7f32f815f460 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f32e800e9d0 tx=0x7f32e800ed90 comp rx=0 tx=0).stop 2023-11-08T08:22:16.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.459+0000 7f32fe370700 1 -- 172.21.15.40:0/753084475 shutdown_connections 2023-11-08T08:22:16.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.459+0000 7f32fe370700 1 --2- 172.21.15.40:0/753084475 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f32dc064ee0 0x7f32dc067390 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.459+0000 7f32fe370700 1 --2- 172.21.15.40:0/753084475 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32f8160710 0x7f32f815f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.459+0000 7f32fe370700 1 --2- 172.21.15.40:0/753084475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f80cc860 0x7f32f815f460 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.459+0000 7f32fe370700 1 --2- 172.21.15.40:0/753084475 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32f80705c0 0x7f32f80c3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.459+0000 7f32fe370700 1 -- 172.21.15.40:0/753084475 >> 172.21.15.40:0/753084475 conn(0x7f32f80b0f80 msgr2=0x7f32f81634e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:16.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.459+0000 7f32fe370700 1 -- 172.21.15.40:0/753084475 shutdown_connections 2023-11-08T08:22:16.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.459+0000 7f32fe370700 1 -- 172.21.15.40:0/753084475 wait complete. 2023-11-08T08:22:16.488 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T08:22:16.489 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:22:16.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.671+0000 7f4b7e220700 1 Processor -- start 2023-11-08T08:22:16.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.671+0000 7f4b7e220700 1 -- start start 2023-11-08T08:22:16.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.675+0000 7f4b7e220700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4b7815c150 0x7f4b780b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:16.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.675+0000 7f4b7e220700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b78070360 0x7f4b78160340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:16.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.675+0000 7f4b7e220700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b7815c520 con 0x7f4b7806ff10 2023-11-08T08:22:16.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.675+0000 7f4b7e220700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b780b4240 con 0x7f4b7815c150 2023-11-08T08:22:16.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.675+0000 7f4b7e220700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7806ac00 con 0x7f4b78070360 2023-11-08T08:22:16.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.675+0000 7f4b7d21e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4b7815c150 0x7f4b780b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:16.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.675+0000 7f4b7d21e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4b7815c150 0x7f4b780b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34748/0 (socket says 172.21.15.40:34748) 2023-11-08T08:22:16.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.675+0000 7f4b7da1f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b78070360 0x7f4b78160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:16.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.675+0000 7f4b7d21e700 1 -- 172.21.15.40:0/1670666698 learned_addr learned my addr 172.21.15.40:0/1670666698 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:16.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.675+0000 7f4b7ca1d700 1 --1- 172.21.15.40:0/1670666698 >> v1:172.21.15.169:6789/0 conn(0x7f4b7806ff10 0x7f4b7815fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:42974/0 (socket says 172.21.15.40:42974) 2023-11-08T08:22:16.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.675+0000 7f4b7d21e700 1 -- 172.21.15.40:0/1670666698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b78070360 msgr2=0x7f4b78160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:16.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.675+0000 7f4b7d21e700 1 --2- 172.21.15.40:0/1670666698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b78070360 0x7f4b78160340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.675+0000 7f4b7d21e700 1 -- 172.21.15.40:0/1670666698 >> v1:172.21.15.169:6789/0 conn(0x7f4b7806ff10 legacy=0x7f4b7815fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:16.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.675+0000 7f4b7d21e700 1 -- 172.21.15.40:0/1670666698 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7806cc50 con 0x7f4b7815c150 2023-11-08T08:22:16.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.675+0000 7f4b6ffff700 1 -- 172.21.15.40:0/1670666698 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2949127660 0 0) 0x7f4b7815c520 con 0x7f4b7806ff10 2023-11-08T08:22:16.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.679+0000 7f4b7d21e700 1 --2- 172.21.15.40:0/1670666698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b7815c150 0x7f4b780b3d00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009910 tx=0x7f4b74009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f16865b4382ff3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:16.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.679+0000 7f4b6ffff700 1 -- 172.21.15.40:0/1670666698 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b74011830 con 0x7f4b7815c150 2023-11-08T08:22:16.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.679+0000 7f4b6ffff700 1 -- 172.21.15.40:0/1670666698 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b74011990 con 0x7f4b7815c150 2023-11-08T08:22:16.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.679+0000 7f4b6ffff700 1 -- 172.21.15.40:0/1670666698 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b74011d40 con 0x7f4b7815c150 2023-11-08T08:22:16.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.679+0000 7f4b7e220700 1 -- 172.21.15.40:0/1670666698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b7815c150 msgr2=0x7f4b780b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:16.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.679+0000 7f4b7e220700 1 --2- 172.21.15.40:0/1670666698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b7815c150 0x7f4b780b3d00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009910 tx=0x7f4b74009c20 comp rx=0 tx=0).stop 2023-11-08T08:22:16.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.679+0000 7f4b7e220700 1 -- 172.21.15.40:0/1670666698 shutdown_connections 2023-11-08T08:22:16.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.679+0000 7f4b7e220700 1 --2- 172.21.15.40:0/1670666698 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b78070360 0x7f4b78160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.679+0000 7f4b7e220700 1 --2- 172.21.15.40:0/1670666698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b7815c150 0x7f4b780b3d00 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.679+0000 7f4b7e220700 1 -- 172.21.15.40:0/1670666698 >> 172.21.15.40:0/1670666698 conn(0x7f4b780d3680 msgr2=0x7f4b780cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:16.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.679+0000 7f4b7e220700 1 -- 172.21.15.40:0/1670666698 shutdown_connections 2023-11-08T08:22:16.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.679+0000 7f4b7e220700 1 -- 172.21.15.40:0/1670666698 wait complete. 2023-11-08T08:22:16.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.683+0000 7f4b7e220700 1 Processor -- start 2023-11-08T08:22:16.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.683+0000 7f4b7e220700 1 -- start start 2023-11-08T08:22:16.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.683+0000 7f4b7e220700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b7806ff10 0x7f4b7814fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:16.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.683+0000 7f4b7e220700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b78070360 0x7f4b780cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:16.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.683+0000 7f4b7e220700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b7815c150 0x7f4b780cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:16.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.683+0000 7f4b7e220700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b78157e40 con 0x7f4b78070360 2023-11-08T08:22:16.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.683+0000 7f4b7e220700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b78158ca0 con 0x7f4b7815c150 2023-11-08T08:22:16.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.683+0000 7f4b7e220700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b78158090 con 0x7f4b7806ff10 2023-11-08T08:22:16.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.683+0000 7f4b7ca1d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b78070360 0x7f4b780cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:16.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.683+0000 7f4b7ca1d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b78070360 0x7f4b780cbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34756/0 (socket says 172.21.15.40:34756) 2023-11-08T08:22:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.683+0000 7f4b7ca1d700 1 -- 172.21.15.40:0/239641212 learned_addr learned my addr 172.21.15.40:0/239641212 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.687+0000 7f4b7d21e700 1 --2- 172.21.15.40:0/239641212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b7806ff10 0x7f4b7814fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.687+0000 7f4b7da1f700 1 --2- 172.21.15.40:0/239641212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b7815c150 0x7f4b780cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:16.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.687+0000 7f4b7ca1d700 1 -- 172.21.15.40:0/239641212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b7806ff10 msgr2=0x7f4b7814fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:16.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.687+0000 7f4b7ca1d700 1 --2- 172.21.15.40:0/239641212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b7806ff10 0x7f4b7814fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.687+0000 7f4b7ca1d700 1 -- 172.21.15.40:0/239641212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b7815c150 msgr2=0x7f4b780cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:16.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.687+0000 7f4b7ca1d700 1 --2- 172.21.15.40:0/239641212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b7815c150 0x7f4b780cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.687+0000 7f4b7ca1d700 1 -- 172.21.15.40:0/239641212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b74009570 con 0x7f4b78070360 2023-11-08T08:22:16.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.687+0000 7f4b7ca1d700 1 --2- 172.21.15.40:0/239641212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b78070360 0x7f4b780cbad0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800c7b0 tx=0x7f4b6800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:16.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.687+0000 7f4b6dffb700 1 -- 172.21.15.40:0/239641212 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68004cf0 con 0x7f4b78070360 2023-11-08T08:22:16.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.687+0000 7f4b6dffb700 1 -- 172.21.15.40:0/239641212 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b68004e50 con 0x7f4b78070360 2023-11-08T08:22:16.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.687+0000 7f4b6dffb700 1 -- 172.21.15.40:0/239641212 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68007310 con 0x7f4b78070360 2023-11-08T08:22:16.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.687+0000 7f4b7e220700 1 -- 172.21.15.40:0/239641212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b780d2730 con 0x7f4b78070360 2023-11-08T08:22:16.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.687+0000 7f4b7e220700 1 -- 172.21.15.40:0/239641212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b780d2450 con 0x7f4b78070360 2023-11-08T08:22:16.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.687+0000 7f4b7e220700 1 -- 172.21.15.40:0/239641212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b780d05e0 con 0x7f4b78070360 2023-11-08T08:22:16.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.691+0000 7f4b6dffb700 1 -- 172.21.15.40:0/239641212 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4b6801a020 con 0x7f4b78070360 2023-11-08T08:22:16.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.691+0000 7f4b6dffb700 1 --2- 172.21.15.40:0/239641212 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4b5c064ee0 0x7f4b5c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:16.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.691+0000 7f4b6dffb700 1 -- 172.21.15.40:0/239641212 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f4b680818c0 con 0x7f4b78070360 2023-11-08T08:22:16.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.691+0000 7f4b7d21e700 1 --2- 172.21.15.40:0/239641212 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4b5c064ee0 0x7f4b5c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:16.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.695+0000 7f4b7d21e700 1 --2- 172.21.15.40:0/239641212 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4b5c064ee0 0x7f4b5c067390 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4b74014040 tx=0x7f4b74008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:16.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.699+0000 7f4b6dffb700 1 -- 172.21.15.40:0/239641212 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4b6804fb20 con 0x7f4b78070360 2023-11-08T08:22:16.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.891+0000 7f4b7e220700 1 -- 172.21.15.40:0/239641212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4b78073d20 con 0x7f4b78070360 2023-11-08T08:22:16.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.891+0000 7f4b6dffb700 1 -- 172.21.15.40:0/239641212 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4b68053140 con 0x7f4b78070360 2023-11-08T08:22:16.900 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:22:16.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.895+0000 7f4b7e220700 1 -- 172.21.15.40:0/239641212 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4b5c064ee0 msgr2=0x7f4b5c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:16.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.895+0000 7f4b7e220700 1 --2- 172.21.15.40:0/239641212 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4b5c064ee0 0x7f4b5c067390 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4b74014040 tx=0x7f4b74008030 comp rx=0 tx=0).stop 2023-11-08T08:22:16.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.895+0000 7f4b7e220700 1 -- 172.21.15.40:0/239641212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b78070360 msgr2=0x7f4b780cbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:16.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.895+0000 7f4b7e220700 1 --2- 172.21.15.40:0/239641212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b78070360 0x7f4b780cbad0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800c7b0 tx=0x7f4b6800cac0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.895+0000 7f4b7e220700 1 -- 172.21.15.40:0/239641212 shutdown_connections 2023-11-08T08:22:16.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.895+0000 7f4b7e220700 1 --2- 172.21.15.40:0/239641212 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4b5c064ee0 0x7f4b5c067390 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.895+0000 7f4b7e220700 1 --2- 172.21.15.40:0/239641212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b7815c150 0x7f4b780cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.895+0000 7f4b7e220700 1 --2- 172.21.15.40:0/239641212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b78070360 0x7f4b780cbad0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.895+0000 7f4b7e220700 1 --2- 172.21.15.40:0/239641212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b7806ff10 0x7f4b7814fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:16.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.895+0000 7f4b7e220700 1 -- 172.21.15.40:0/239641212 >> 172.21.15.40:0/239641212 conn(0x7f4b780d3680 msgr2=0x7f4b780cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:16.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.895+0000 7f4b7e220700 1 -- 172.21.15.40:0/239641212 shutdown_connections 2023-11-08T08:22:16.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:16.895+0000 7f4b7e220700 1 -- 172.21.15.40:0/239641212 wait complete. 2023-11-08T08:22:16.924 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:22:17.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.111+0000 7ffb1288c700 1 Processor -- start 2023-11-08T08:22:17.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.111+0000 7ffb1288c700 1 -- start start 2023-11-08T08:22:17.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb1288c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb0c070620 0x7ffb0c15d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb1288c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0c15d650 con 0x7ffb0c159cd0 2023-11-08T08:22:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb1288c700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0c15a0a0 con 0x7ffb0c0cc1a0 2023-11-08T08:22:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb1288c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c06be80 con 0x7ffb0c070620 2023-11-08T08:22:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb1188a700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ffb0c159cd0 0x7ffb0c06c890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50980/0 (socket says 172.21.15.40:50980) 2023-11-08T08:22:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb1188a700 1 -- 172.21.15.40:0/2514079343 learned_addr learned my addr 172.21.15.40:0/2514079343 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:17.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb1208b700 1 --2- 172.21.15.40:0/2514079343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb0c070620 0x7ffb0c15d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:17.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb10888700 1 -- 172.21.15.40:0/2514079343 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888025305 0 0) 0x7ffb0c15d650 con 0x7ffb0c159cd0 2023-11-08T08:22:17.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb10888700 1 -- 172.21.15.40:0/2514079343 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaf80035d0 con 0x7ffb0c159cd0 2023-11-08T08:22:17.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb10888700 1 -- 172.21.15.40:0/2514079343 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 588785114 0 0) 0x7ffb0c15a0a0 con 0x7ffb0c0cc1a0 2023-11-08T08:22:17.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb10888700 1 -- 172.21.15.40:0/2514079343 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb0c15d650 con 0x7ffb0c0cc1a0 2023-11-08T08:22:17.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb10888700 1 -- 172.21.15.40:0/2514079343 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 497187757 0 0) 0x7ffaf80035d0 con 0x7ffb0c159cd0 2023-11-08T08:22:17.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb10888700 1 -- 172.21.15.40:0/2514079343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb0c070620 msgr2=0x7ffb0c15d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:17.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb10888700 1 --2- 172.21.15.40:0/2514079343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb0c070620 0x7ffb0c15d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb10888700 1 -- 172.21.15.40:0/2514079343 >> v1:172.21.15.169:6789/0 conn(0x7ffb0c0cc1a0 legacy=0x7ffb0c06ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:17.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb10888700 1 -- 172.21.15.40:0/2514079343 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c0cdf80 con 0x7ffb0c159cd0 2023-11-08T08:22:17.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb10888700 1 -- 172.21.15.40:0/2514079343 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7ffb080015f0 con 0x7ffb0c159cd0 2023-11-08T08:22:17.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb10888700 1 -- 172.21.15.40:0/2514079343 >> v1:172.21.15.40:6789/0 conn(0x7ffb0c159cd0 legacy=0x7ffb0c06c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:17.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb10888700 1 --2- 172.21.15.40:0/2514079343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb0c070620 0x7ffaf8003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:17.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb10888700 1 --2- 172.21.15.40:0/2514079343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaf8004190 0x7ffaf8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:17.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb10888700 1 --2- 172.21.15.40:0/2514079343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaf8004db0 0x7ffaf8005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:17.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb10888700 1 -- 172.21.15.40:0/2514079343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c06be80 con 0x7ffaf8004190 2023-11-08T08:22:17.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb10888700 1 -- 172.21.15.40:0/2514079343 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf800bb40 con 0x7ffaf8004db0 2023-11-08T08:22:17.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb10888700 1 -- 172.21.15.40:0/2514079343 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf800bd00 con 0x7ffb0c070620 2023-11-08T08:22:17.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb11089700 1 --2- 172.21.15.40:0/2514079343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaf8004190 0x7ffaf8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:17.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.115+0000 7ffb1188a700 1 --2- 172.21.15.40:0/2514079343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb0c070620 0x7ffaf8003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:17.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.119+0000 7ffb1208b700 1 --2- 172.21.15.40:0/2514079343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaf8004db0 0x7ffaf8005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:17.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.119+0000 7ffb11089700 1 -- 172.21.15.40:0/2514079343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb0c070620 msgr2=0x7ffaf8003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:17.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.119+0000 7ffb11089700 1 --2- 172.21.15.40:0/2514079343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb0c070620 0x7ffaf8003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.119+0000 7ffb11089700 1 -- 172.21.15.40:0/2514079343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaf8004db0 msgr2=0x7ffaf8005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:17.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.119+0000 7ffb11089700 1 --2- 172.21.15.40:0/2514079343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaf8004db0 0x7ffaf8005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.119+0000 7ffb11089700 1 -- 172.21.15.40:0/2514079343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf80035d0 con 0x7ffaf8004190 2023-11-08T08:22:17.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.119+0000 7ffb11089700 1 --2- 172.21.15.40:0/2514079343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaf8004190 0x7ffaf8004870 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ffafc001960 tx=0x7ffaf80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1bb2573e4469727 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:17.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.119+0000 7ffb10888700 1 -- 172.21.15.40:0/2514079343 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc001450 con 0x7ffaf8004190 2023-11-08T08:22:17.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.119+0000 7ffb10888700 1 -- 172.21.15.40:0/2514079343 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffafc0052f0 con 0x7ffaf8004190 2023-11-08T08:22:17.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.119+0000 7ffb10888700 1 -- 172.21.15.40:0/2514079343 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc0036a0 con 0x7ffaf8004190 2023-11-08T08:22:17.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.119+0000 7ffb1288c700 1 -- 172.21.15.40:0/2514079343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaf8004190 msgr2=0x7ffaf8004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:17.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.119+0000 7ffb1288c700 1 --2- 172.21.15.40:0/2514079343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffaf8004190 0x7ffaf8004870 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ffafc001960 tx=0x7ffaf80058a0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.119+0000 7ffb1208b700 1 -- 172.21.15.40:0/2514079343 reap_dead start 2023-11-08T08:22:17.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.119+0000 7ffb1288c700 1 -- 172.21.15.40:0/2514079343 shutdown_connections 2023-11-08T08:22:17.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.119+0000 7ffb1288c700 1 -- 172.21.15.40:0/2514079343 >> 172.21.15.40:0/2514079343 conn(0x7ffb0c0cf280 msgr2=0x7ffb0c0cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:17.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.119+0000 7ffb1288c700 1 -- 172.21.15.40:0/2514079343 shutdown_connections 2023-11-08T08:22:17.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.119+0000 7ffb1288c700 1 -- 172.21.15.40:0/2514079343 wait complete. 2023-11-08T08:22:17.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.123+0000 7ffb1288c700 1 Processor -- start 2023-11-08T08:22:17.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.123+0000 7ffb1288c700 1 -- start start 2023-11-08T08:22:17.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.123+0000 7ffb1288c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaf8004190 0x7ffb0c141f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:17.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.123+0000 7ffb1288c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb0c154bf0 0x7ffb0c0c5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:17.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.123+0000 7ffb1288c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb0c155060 0x7ffb0c0c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:17.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.123+0000 7ffb1288c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c152e60 con 0x7ffb0c155060 2023-11-08T08:22:17.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.123+0000 7ffb1288c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c153900 con 0x7ffaf8004190 2023-11-08T08:22:17.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.123+0000 7ffb1288c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c153b50 con 0x7ffb0c154bf0 2023-11-08T08:22:17.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.123+0000 7ffb1208b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb0c155060 0x7ffb0c0c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:17.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.123+0000 7ffb1208b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb0c155060 0x7ffb0c0c6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34778/0 (socket says 172.21.15.40:34778) 2023-11-08T08:22:17.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.123+0000 7ffb1208b700 1 -- 172.21.15.40:0/3655350456 learned_addr learned my addr 172.21.15.40:0/3655350456 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:17.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.127+0000 7ffb11089700 1 --2- 172.21.15.40:0/3655350456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb0c154bf0 0x7ffb0c0c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:17.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.127+0000 7ffb1188a700 1 --2- 172.21.15.40:0/3655350456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaf8004190 0x7ffb0c141f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:17.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.127+0000 7ffb1208b700 1 -- 172.21.15.40:0/3655350456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb0c154bf0 msgr2=0x7ffb0c0c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:17.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.127+0000 7ffb1208b700 1 --2- 172.21.15.40:0/3655350456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb0c154bf0 0x7ffb0c0c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.127+0000 7ffb1208b700 1 -- 172.21.15.40:0/3655350456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaf8004190 msgr2=0x7ffb0c141f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:17.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.127+0000 7ffb1208b700 1 --2- 172.21.15.40:0/3655350456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaf8004190 0x7ffb0c141f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.127+0000 7ffb1208b700 1 -- 172.21.15.40:0/3655350456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c06c090 con 0x7ffb0c155060 2023-11-08T08:22:17.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.127+0000 7ffb1208b700 1 --2- 172.21.15.40:0/3655350456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb0c155060 0x7ffb0c0c6380 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c070a10 tx=0x7ffb04011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:17.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.127+0000 7ffb027fc700 1 -- 172.21.15.40:0/3655350456 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0400e040 con 0x7ffb0c155060 2023-11-08T08:22:17.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.127+0000 7ffb027fc700 1 -- 172.21.15.40:0/3655350456 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0400f040 con 0x7ffb0c155060 2023-11-08T08:22:17.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.127+0000 7ffb027fc700 1 -- 172.21.15.40:0/3655350456 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0400f8f0 con 0x7ffb0c155060 2023-11-08T08:22:17.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.127+0000 7ffb1288c700 1 -- 172.21.15.40:0/3655350456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0c0ce260 con 0x7ffb0c155060 2023-11-08T08:22:17.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.127+0000 7ffb1288c700 1 -- 172.21.15.40:0/3655350456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0c06c370 con 0x7ffb0c155060 2023-11-08T08:22:17.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.127+0000 7ffb1288c700 1 -- 172.21.15.40:0/3655350456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0c0cca80 con 0x7ffb0c155060 2023-11-08T08:22:17.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.131+0000 7ffb027fc700 1 -- 172.21.15.40:0/3655350456 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ffb0400fa50 con 0x7ffb0c155060 2023-11-08T08:22:17.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.131+0000 7ffb027fc700 1 --2- 172.21.15.40:0/3655350456 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ffaf0064f10 0x7ffaf00673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:17.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.131+0000 7ffb027fc700 1 -- 172.21.15.40:0/3655350456 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5530+0+0 (secure 0 0 0) 0x7ffb040821e0 con 0x7ffb0c155060 2023-11-08T08:22:17.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.131+0000 7ffb1188a700 1 --2- 172.21.15.40:0/3655350456 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ffaf0064f10 0x7ffaf00673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:17.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.135+0000 7ffb1188a700 1 --2- 172.21.15.40:0/3655350456 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ffaf0064f10 0x7ffaf00673c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ffb08000fd0 tx=0x7ffb08001a60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:17.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.139+0000 7ffb027fc700 1 -- 172.21.15.40:0/3655350456 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ffb040504a0 con 0x7ffb0c155060 2023-11-08T08:22:17.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.315+0000 7ffb1288c700 1 -- 172.21.15.40:0/3655350456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ffb0c073d20 con 0x7ffb0c155060 2023-11-08T08:22:17.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.315+0000 7ffb027fc700 1 -- 172.21.15.40:0/3655350456 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v65) v1 ==== 92+0+10 (secure 0 0 0) 0x7ffb04053ac0 con 0x7ffb0c155060 2023-11-08T08:22:17.325 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:22:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.319+0000 7ffb1288c700 1 -- 172.21.15.40:0/3655350456 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ffaf0064f10 msgr2=0x7ffaf00673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.319+0000 7ffb1288c700 1 --2- 172.21.15.40:0/3655350456 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ffaf0064f10 0x7ffaf00673c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ffb08000fd0 tx=0x7ffb08001a60 comp rx=0 tx=0).stop 2023-11-08T08:22:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.319+0000 7ffb1288c700 1 -- 172.21.15.40:0/3655350456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb0c155060 msgr2=0x7ffb0c0c6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.319+0000 7ffb1288c700 1 --2- 172.21.15.40:0/3655350456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb0c155060 0x7ffb0c0c6380 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c070a10 tx=0x7ffb04011ed0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.319+0000 7ffb1288c700 1 -- 172.21.15.40:0/3655350456 shutdown_connections 2023-11-08T08:22:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.319+0000 7ffb1288c700 1 --2- 172.21.15.40:0/3655350456 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ffaf0064f10 0x7ffaf00673c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.319+0000 7ffb1288c700 1 --2- 172.21.15.40:0/3655350456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb0c155060 0x7ffb0c0c6380 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.319+0000 7ffb1288c700 1 --2- 172.21.15.40:0/3655350456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb0c154bf0 0x7ffb0c0c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.319+0000 7ffb1288c700 1 --2- 172.21.15.40:0/3655350456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffaf8004190 0x7ffb0c141f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.319+0000 7ffb1288c700 1 -- 172.21.15.40:0/3655350456 >> 172.21.15.40:0/3655350456 conn(0x7ffb0c0cf280 msgr2=0x7ffb0c0c8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.319+0000 7ffb1288c700 1 -- 172.21.15.40:0/3655350456 shutdown_connections 2023-11-08T08:22:17.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.319+0000 7ffb1288c700 1 -- 172.21.15.40:0/3655350456 wait complete. 2023-11-08T08:22:17.351 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T08:22:17.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.523+0000 7f75e759e700 1 Processor -- start 2023-11-08T08:22:17.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.523+0000 7f75e759e700 1 -- start start 2023-11-08T08:22:17.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.527+0000 7f75e759e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f75e815c650 0x7f75e806b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:17.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.527+0000 7f75e759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75e80cd0d0 0x7f75e80b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:17.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.527+0000 7f75e759e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f75e80b1b80 0x7f75e806bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:17.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.527+0000 7f75e759e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e806b140 con 0x7f75e80b1b80 2023-11-08T08:22:17.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.527+0000 7f75e759e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e80c9d40 con 0x7f75e815c650 2023-11-08T08:22:17.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.527+0000 7f75e759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e806b820 con 0x7f75e80cd0d0 2023-11-08T08:22:17.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.527+0000 7f75e6d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f75e80b1b80 0x7f75e806bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:17.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.527+0000 7f75e6d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f75e80b1b80 0x7f75e806bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35756/0 (socket says 172.21.15.40:35756) 2023-11-08T08:22:17.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.527+0000 7f75e6d9d700 1 -- 172.21.15.40:0/3703008101 learned_addr learned my addr 172.21.15.40:0/3703008101 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:17.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.527+0000 7f75ddd9b700 1 --2- 172.21.15.40:0/3703008101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75e80cd0d0 0x7f75e80b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:17.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.527+0000 7f75e659c700 1 --2- 172.21.15.40:0/3703008101 >> v2:172.21.15.169:3300/0 conn(0x7f75e815c650 0x7f75e806b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:17.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.527+0000 7f75e6d9d700 1 -- 172.21.15.40:0/3703008101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75e80cd0d0 msgr2=0x7f75e80b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:17.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.527+0000 7f75e6d9d700 1 --2- 172.21.15.40:0/3703008101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75e80cd0d0 0x7f75e80b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.527+0000 7f75e6d9d700 1 -- 172.21.15.40:0/3703008101 >> v2:172.21.15.169:3300/0 conn(0x7f75e815c650 msgr2=0x7f75e806b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:17.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.527+0000 7f75e6d9d700 1 --2- 172.21.15.40:0/3703008101 >> v2:172.21.15.169:3300/0 conn(0x7f75e815c650 0x7f75e806b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.527+0000 7f75e6d9d700 1 -- 172.21.15.40:0/3703008101 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e80cd5d0 con 0x7f75e80b1b80 2023-11-08T08:22:17.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.527+0000 7f75e6d9d700 1 --2- 172.21.15.40:0/3703008101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e80b1b80 0x7f75e806bed0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f75e80cc1e0 tx=0x7f75e0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54af5e0437ced844 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:17.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.527+0000 7f75e5d9b700 1 -- 172.21.15.40:0/3703008101 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0011830 con 0x7f75e80b1b80 2023-11-08T08:22:17.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.527+0000 7f75e5d9b700 1 -- 172.21.15.40:0/3703008101 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e0011990 con 0x7f75e80b1b80 2023-11-08T08:22:17.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.527+0000 7f75e5d9b700 1 -- 172.21.15.40:0/3703008101 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0011d20 con 0x7f75e80b1b80 2023-11-08T08:22:17.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.531+0000 7f75e759e700 1 -- 172.21.15.40:0/3703008101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e80b1b80 msgr2=0x7f75e806bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:17.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.531+0000 7f75e759e700 1 --2- 172.21.15.40:0/3703008101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e80b1b80 0x7f75e806bed0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f75e80cc1e0 tx=0x7f75e0009bf0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.531+0000 7f75e759e700 1 -- 172.21.15.40:0/3703008101 shutdown_connections 2023-11-08T08:22:17.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.531+0000 7f75e759e700 1 --2- 172.21.15.40:0/3703008101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e80b1b80 0x7f75e806bed0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.531+0000 7f75e759e700 1 --2- 172.21.15.40:0/3703008101 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75e80cd0d0 0x7f75e80b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.531+0000 7f75e759e700 1 --2- 172.21.15.40:0/3703008101 >> v2:172.21.15.169:3300/0 conn(0x7f75e815c650 0x7f75e806b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.531+0000 7f75e759e700 1 -- 172.21.15.40:0/3703008101 >> 172.21.15.40:0/3703008101 conn(0x7f75e80b0f80 msgr2=0x7f75e80ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:17.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.531+0000 7f75e759e700 1 -- 172.21.15.40:0/3703008101 shutdown_connections 2023-11-08T08:22:17.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.531+0000 7f75e759e700 1 -- 172.21.15.40:0/3703008101 wait complete. 2023-11-08T08:22:17.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.531+0000 7f75e759e700 1 Processor -- start 2023-11-08T08:22:17.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.531+0000 7f75e759e700 1 -- start start 2023-11-08T08:22:17.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.535+0000 7f75e759e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75e80b1b80 0x7f75e814fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:17.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.535+0000 7f75e759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e80cd0d0 0x7f75e814bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:17.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.535+0000 7f75e759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75e815c650 0x7f75e80aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:17.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.535+0000 7f75e759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e8151640 con 0x7f75e80cd0d0 2023-11-08T08:22:17.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.535+0000 7f75e759e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e81502d0 con 0x7f75e80b1b80 2023-11-08T08:22:17.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.535+0000 7f75e759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e814c290 con 0x7f75e815c650 2023-11-08T08:22:17.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.535+0000 7f75ddd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e80cd0d0 0x7f75e814bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:17.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.535+0000 7f75ddd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e80cd0d0 0x7f75e814bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35760/0 (socket says 172.21.15.40:35760) 2023-11-08T08:22:17.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.535+0000 7f75ddd9b700 1 -- 172.21.15.40:0/4107799742 learned_addr learned my addr 172.21.15.40:0/4107799742 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:17.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.535+0000 7f75e659c700 1 --2- 172.21.15.40:0/4107799742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75e80b1b80 0x7f75e814fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:17.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.535+0000 7f75e6d9d700 1 --2- 172.21.15.40:0/4107799742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75e815c650 0x7f75e80aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:17.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.535+0000 7f75ddd9b700 1 -- 172.21.15.40:0/4107799742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75e815c650 msgr2=0x7f75e80aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:17.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.535+0000 7f75ddd9b700 1 --2- 172.21.15.40:0/4107799742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75e815c650 0x7f75e80aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.535+0000 7f75ddd9b700 1 -- 172.21.15.40:0/4107799742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75e80b1b80 msgr2=0x7f75e814fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:17.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.535+0000 7f75ddd9b700 1 --2- 172.21.15.40:0/4107799742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75e80b1b80 0x7f75e814fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.535+0000 7f75ddd9b700 1 -- 172.21.15.40:0/4107799742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e0009570 con 0x7f75e80cd0d0 2023-11-08T08:22:17.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.535+0000 7f75ddd9b700 1 --2- 172.21.15.40:0/4107799742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e80cd0d0 0x7f75e814bd50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f75d000e900 tx=0x7f75d000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:17.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.539+0000 7f75df7fe700 1 -- 172.21.15.40:0/4107799742 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0013680 con 0x7f75e80cd0d0 2023-11-08T08:22:17.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.539+0000 7f75df7fe700 1 -- 172.21.15.40:0/4107799742 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d00137e0 con 0x7f75e80cd0d0 2023-11-08T08:22:17.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.539+0000 7f75e759e700 1 -- 172.21.15.40:0/4107799742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e80c99d0 con 0x7f75e80cd0d0 2023-11-08T08:22:17.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.539+0000 7f75df7fe700 1 -- 172.21.15.40:0/4107799742 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0013b70 con 0x7f75e80cd0d0 2023-11-08T08:22:17.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.539+0000 7f75e759e700 1 -- 172.21.15.40:0/4107799742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e80cd8b0 con 0x7f75e80cd0d0 2023-11-08T08:22:17.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.539+0000 7f75e759e700 1 -- 172.21.15.40:0/4107799742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e80c9680 con 0x7f75e80cd0d0 2023-11-08T08:22:17.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.543+0000 7f75df7fe700 1 -- 172.21.15.40:0/4107799742 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f75d0019030 con 0x7f75e80cd0d0 2023-11-08T08:22:17.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.543+0000 7f75df7fe700 1 --2- 172.21.15.40:0/4107799742 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f75c8064ee0 0x7f75c8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:17.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.543+0000 7f75df7fe700 1 -- 172.21.15.40:0/4107799742 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f75d0081c30 con 0x7f75e80cd0d0 2023-11-08T08:22:17.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.543+0000 7f75e659c700 1 --2- 172.21.15.40:0/4107799742 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f75c8064ee0 0x7f75c8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:17.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.543+0000 7f75e659c700 1 --2- 172.21.15.40:0/4107799742 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f75c8064ee0 0x7f75c8067390 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f75d8003b50 tx=0x7f75d8009830 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:17.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.551+0000 7f75df7fe700 1 -- 172.21.15.40:0/4107799742 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f75d004fef0 con 0x7f75e80cd0d0 2023-11-08T08:22:17.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.731+0000 7f75e759e700 1 -- 172.21.15.40:0/4107799742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f75e8073cb0 con 0x7f75e80cd0d0 2023-11-08T08:22:17.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.731+0000 7f75df7fe700 1 -- 172.21.15.40:0/4107799742 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v65) v1 ==== 84+0+126 (secure 0 0 0) 0x7f75d0017090 con 0x7f75e80cd0d0 2023-11-08T08:22:17.743 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:22:17.744 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2023-11-08T08:22:17.744 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:17.744 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2023-11-08T08:22:17.744 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2023-11-08T08:22:17.744 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:22:17.745 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:17.745 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 2, 2023-11-08T08:22:17.745 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": "rbd" 2023-11-08T08:22:17.745 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:17.745 INFO:teuthology.orchestra.run.smithi040.stdout:] 2023-11-08T08:22:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.735+0000 7f75e759e700 1 -- 172.21.15.40:0/4107799742 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f75c8064ee0 msgr2=0x7f75c8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.735+0000 7f75e759e700 1 --2- 172.21.15.40:0/4107799742 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f75c8064ee0 0x7f75c8067390 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f75d8003b50 tx=0x7f75d8009830 comp rx=0 tx=0).stop 2023-11-08T08:22:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.735+0000 7f75e759e700 1 -- 172.21.15.40:0/4107799742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e80cd0d0 msgr2=0x7f75e814bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.735+0000 7f75e759e700 1 --2- 172.21.15.40:0/4107799742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e80cd0d0 0x7f75e814bd50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f75d000e900 tx=0x7f75d000ec10 comp rx=0 tx=0).stop 2023-11-08T08:22:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.735+0000 7f75e759e700 1 -- 172.21.15.40:0/4107799742 shutdown_connections 2023-11-08T08:22:17.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.735+0000 7f75e759e700 1 --2- 172.21.15.40:0/4107799742 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f75c8064ee0 0x7f75c8067390 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.735+0000 7f75e759e700 1 --2- 172.21.15.40:0/4107799742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75e815c650 0x7f75e80aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.735+0000 7f75e759e700 1 --2- 172.21.15.40:0/4107799742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e80cd0d0 0x7f75e814bd50 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.735+0000 7f75e759e700 1 --2- 172.21.15.40:0/4107799742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75e80b1b80 0x7f75e814fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.735+0000 7f75e759e700 1 -- 172.21.15.40:0/4107799742 >> 172.21.15.40:0/4107799742 conn(0x7f75e80b0f80 msgr2=0x7f75e80ad540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:17.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.735+0000 7f75e759e700 1 -- 172.21.15.40:0/4107799742 shutdown_connections 2023-11-08T08:22:17.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.735+0000 7f75e759e700 1 -- 172.21.15.40:0/4107799742 wait complete. 2023-11-08T08:22:17.762 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T08:22:17.763 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T08:22:17.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.951+0000 7f04d1612700 1 Processor -- start 2023-11-08T08:22:17.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.951+0000 7f04d1612700 1 -- start start 2023-11-08T08:22:17.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.955+0000 7f04d1612700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f04cc15c6e0 0x7f04cc071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:17.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.955+0000 7f04d1612700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04cc15cab0 con 0x7f04cc06a510 2023-11-08T08:22:17.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.955+0000 7f04d1612700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04cc0702d0 con 0x7f04cc071fe0 2023-11-08T08:22:17.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.955+0000 7f04d1612700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04cc0d2800 con 0x7f04cc15c6e0 2023-11-08T08:22:17.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.955+0000 7f04d0e11700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f04cc06a510 0x7f04cc06c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48932/0 (socket says 172.21.15.40:48932) 2023-11-08T08:22:17.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.955+0000 7f04d0e11700 1 -- 172.21.15.40:0/784671874 learned_addr learned my addr 172.21.15.40:0/784671874 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:17.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.955+0000 7f04cbfff700 1 --2- 172.21.15.40:0/784671874 >> v2:172.21.15.169:3300/0 conn(0x7f04cc15c6e0 0x7f04cc071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:17.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.955+0000 7f04caffd700 1 -- 172.21.15.40:0/784671874 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2827451147 0 0) 0x7f04cc15cab0 con 0x7f04cc06a510 2023-11-08T08:22:17.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.955+0000 7f04caffd700 1 -- 172.21.15.40:0/784671874 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04b40035d0 con 0x7f04cc06a510 2023-11-08T08:22:17.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.955+0000 7f04cbfff700 1 -- 172.21.15.40:0/784671874 >> v1:172.21.15.169:6789/0 conn(0x7f04cc071fe0 legacy=0x7f04cc069e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:17.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.955+0000 7f04cbfff700 1 -- 172.21.15.40:0/784671874 >> v1:172.21.15.40:6789/0 conn(0x7f04cc06a510 legacy=0x7f04cc06c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:17.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.955+0000 7f04cbfff700 1 -- 172.21.15.40:0/784671874 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc0d2a10 con 0x7f04cc15c6e0 2023-11-08T08:22:17.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.959+0000 7f04cbfff700 1 --2- 172.21.15.40:0/784671874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04cc15c6e0 0x7f04cc071aa0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009850 tx=0x7f04bc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb352788709ba7e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:17.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.959+0000 7f04caffd700 1 -- 172.21.15.40:0/784671874 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc011830 con 0x7f04cc15c6e0 2023-11-08T08:22:17.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.959+0000 7f04caffd700 1 -- 172.21.15.40:0/784671874 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc011990 con 0x7f04cc15c6e0 2023-11-08T08:22:17.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.959+0000 7f04caffd700 1 -- 172.21.15.40:0/784671874 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc011d40 con 0x7f04cc15c6e0 2023-11-08T08:22:17.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.959+0000 7f04d1612700 1 -- 172.21.15.40:0/784671874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04cc15c6e0 msgr2=0x7f04cc071aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:17.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.959+0000 7f04d1612700 1 --2- 172.21.15.40:0/784671874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04cc15c6e0 0x7f04cc071aa0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009850 tx=0x7f04bc009b60 comp rx=0 tx=0).stop 2023-11-08T08:22:17.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.959+0000 7f04d1612700 1 -- 172.21.15.40:0/784671874 shutdown_connections 2023-11-08T08:22:17.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.959+0000 7f04d1612700 1 --2- 172.21.15.40:0/784671874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04cc15c6e0 0x7f04cc071aa0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.959+0000 7f04d1612700 1 -- 172.21.15.40:0/784671874 >> 172.21.15.40:0/784671874 conn(0x7f04cc0b65b0 msgr2=0x7f04cc071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:17.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.959+0000 7f04d1612700 1 -- 172.21.15.40:0/784671874 shutdown_connections 2023-11-08T08:22:17.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.959+0000 7f04d1612700 1 -- 172.21.15.40:0/784671874 wait complete. 2023-11-08T08:22:17.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.963+0000 7f04d1612700 1 Processor -- start 2023-11-08T08:22:17.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.963+0000 7f04d1612700 1 -- start start 2023-11-08T08:22:17.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04d1612700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04cc06a510 0x7f04cc09af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:17.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04d1612700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04cc071fe0 0x7f04cc0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:17.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04d1612700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04cc15c6e0 0x7f04cc0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:17.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04d1612700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc1f5f00 con 0x7f04cc06a510 2023-11-08T08:22:17.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04d1612700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc09b890 con 0x7f04cc071fe0 2023-11-08T08:22:17.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04d1612700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc09bb70 con 0x7f04cc15c6e0 2023-11-08T08:22:17.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04cbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04cc06a510 0x7f04cc09af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:17.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04cbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04cc06a510 0x7f04cc09af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35766/0 (socket says 172.21.15.40:35766) 2023-11-08T08:22:17.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04cbfff700 1 -- 172.21.15.40:0/1320911383 learned_addr learned my addr 172.21.15.40:0/1320911383 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04d0e11700 1 --2- 172.21.15.40:0/1320911383 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04cc15c6e0 0x7f04cc0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04cb7fe700 1 --2- 172.21.15.40:0/1320911383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04cc071fe0 0x7f04cc0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04cbfff700 1 -- 172.21.15.40:0/1320911383 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04cc15c6e0 msgr2=0x7f04cc0cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04cbfff700 1 --2- 172.21.15.40:0/1320911383 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04cc15c6e0 0x7f04cc0cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04cbfff700 1 -- 172.21.15.40:0/1320911383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04cc071fe0 msgr2=0x7f04cc0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:17.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04cbfff700 1 --2- 172.21.15.40:0/1320911383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04cc071fe0 0x7f04cc0cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:17.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04cbfff700 1 -- 172.21.15.40:0/1320911383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04bc0094b0 con 0x7f04cc06a510 2023-11-08T08:22:17.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04cbfff700 1 --2- 172.21.15.40:0/1320911383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04cc06a510 0x7f04cc09af90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f04bc014040 tx=0x7f04bc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:17.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04c8ff9700 1 -- 172.21.15.40:0/1320911383 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc019980 con 0x7f04cc06a510 2023-11-08T08:22:17.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04c8ff9700 1 -- 172.21.15.40:0/1320911383 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc01e070 con 0x7f04cc06a510 2023-11-08T08:22:17.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04c8ff9700 1 -- 172.21.15.40:0/1320911383 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc019d20 con 0x7f04cc06a510 2023-11-08T08:22:17.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04d1612700 1 -- 172.21.15.40:0/1320911383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04cc0d2fd0 con 0x7f04cc06a510 2023-11-08T08:22:17.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.967+0000 7f04d1612700 1 -- 172.21.15.40:0/1320911383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04cc0d2cf0 con 0x7f04cc06a510 2023-11-08T08:22:17.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.971+0000 7f04d1612700 1 -- 172.21.15.40:0/1320911383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04cc0d32b0 con 0x7f04cc06a510 2023-11-08T08:22:17.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.971+0000 7f04c8ff9700 1 -- 172.21.15.40:0/1320911383 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f04bc00f050 con 0x7f04cc06a510 2023-11-08T08:22:17.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.971+0000 7f04c8ff9700 1 --2- 172.21.15.40:0/1320911383 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f04b8064ee0 0x7f04b8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:17.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.971+0000 7f04c8ff9700 1 -- 172.21.15.40:0/1320911383 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f04bc082db0 con 0x7f04cc06a510 2023-11-08T08:22:17.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.971+0000 7f04cb7fe700 1 --2- 172.21.15.40:0/1320911383 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f04b8064ee0 0x7f04b8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:17.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.975+0000 7f04cb7fe700 1 --2- 172.21.15.40:0/1320911383 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f04b8064ee0 0x7f04b8067390 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f04cc1f9550 tx=0x7f04c0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:17.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:17.979+0000 7f04c8ff9700 1 -- 172.21.15.40:0/1320911383 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f04bc087070 con 0x7f04cc06a510 2023-11-08T08:22:18.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:18.155+0000 7f04d1612700 1 -- 172.21.15.40:0/1320911383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f04cc073cb0 con 0x7f04cc06a510 2023-11-08T08:22:19.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.167+0000 7f04c8ff9700 1 -- 172.21.15.40:0/1320911383 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v66) v1 ==== 138+0+0 (secure 0 0 0) 0x7f04bc0200b0 con 0x7f04cc06a510 2023-11-08T08:22:19.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.171+0000 7f04d1612700 1 -- 172.21.15.40:0/1320911383 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f04b8064ee0 msgr2=0x7f04b8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:19.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.171+0000 7f04d1612700 1 --2- 172.21.15.40:0/1320911383 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f04b8064ee0 0x7f04b8067390 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f04cc1f9550 tx=0x7f04c0009040 comp rx=0 tx=0).stop 2023-11-08T08:22:19.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.171+0000 7f04d1612700 1 -- 172.21.15.40:0/1320911383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04cc06a510 msgr2=0x7f04cc09af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:19.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.171+0000 7f04d1612700 1 --2- 172.21.15.40:0/1320911383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04cc06a510 0x7f04cc09af90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f04bc014040 tx=0x7f04bc004770 comp rx=0 tx=0).stop 2023-11-08T08:22:19.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.171+0000 7f04d1612700 1 -- 172.21.15.40:0/1320911383 shutdown_connections 2023-11-08T08:22:19.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.171+0000 7f04d1612700 1 --2- 172.21.15.40:0/1320911383 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f04b8064ee0 0x7f04b8067390 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:19.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.171+0000 7f04d1612700 1 --2- 172.21.15.40:0/1320911383 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04cc15c6e0 0x7f04cc0cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:19.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.171+0000 7f04d1612700 1 --2- 172.21.15.40:0/1320911383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04cc071fe0 0x7f04cc0cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:19.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.171+0000 7f04d1612700 1 --2- 172.21.15.40:0/1320911383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04cc06a510 0x7f04cc09af90 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:19.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.171+0000 7f04d1612700 1 -- 172.21.15.40:0/1320911383 >> 172.21.15.40:0/1320911383 conn(0x7f04cc0b65b0 msgr2=0x7f04cc071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:19.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.171+0000 7f04d1612700 1 -- 172.21.15.40:0/1320911383 shutdown_connections 2023-11-08T08:22:19.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.175+0000 7f04d1612700 1 -- 172.21.15.40:0/1320911383 wait complete. 2023-11-08T08:22:19.183 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2023-11-08T08:22:19.202 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T08:22:19.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.387+0000 7fec99b00700 1 Processor -- start 2023-11-08T08:22:19.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.387+0000 7fec99b00700 1 -- start start 2023-11-08T08:22:19.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.391+0000 7fec99b00700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fec9415c7c0 0x7fec9406cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.391+0000 7fec99b00700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec940cdba0 con 0x7fec940cddf0 2023-11-08T08:22:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.391+0000 7fec99b00700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec940cd8d0 con 0x7fec94161030 2023-11-08T08:22:19.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.391+0000 7fec99b00700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec9406c010 con 0x7fec9415c7c0 2023-11-08T08:22:19.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.391+0000 7fec992ff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fec940cddf0 0x7fec940ce1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48942/0 (socket says 172.21.15.40:48942) 2023-11-08T08:22:19.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.391+0000 7fec992ff700 1 -- 172.21.15.40:0/95672001 learned_addr learned my addr 172.21.15.40:0/95672001 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.391+0000 7fec98afe700 1 --2- 172.21.15.40:0/95672001 >> v2:172.21.15.169:3300/0 conn(0x7fec9415c7c0 0x7fec9406cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.391+0000 7fec937fe700 1 -- 172.21.15.40:0/95672001 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800112505 0 0) 0x7fec940cdba0 con 0x7fec940cddf0 2023-11-08T08:22:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.391+0000 7fec937fe700 1 -- 172.21.15.40:0/95672001 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec7c0035d0 con 0x7fec940cddf0 2023-11-08T08:22:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.391+0000 7fec937fe700 1 -- 172.21.15.40:0/95672001 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306265938 0 0) 0x7fec940cd8d0 con 0x7fec94161030 2023-11-08T08:22:19.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.391+0000 7fec937fe700 1 -- 172.21.15.40:0/95672001 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec940cdba0 con 0x7fec94161030 2023-11-08T08:22:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.391+0000 7fec98afe700 1 -- 172.21.15.40:0/95672001 >> v1:172.21.15.169:6789/0 conn(0x7fec94161030 legacy=0x7fec940cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.391+0000 7fec98afe700 1 -- 172.21.15.40:0/95672001 >> v1:172.21.15.40:6789/0 conn(0x7fec940cddf0 legacy=0x7fec940ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.391+0000 7fec98afe700 1 -- 172.21.15.40:0/95672001 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9406c2f0 con 0x7fec9415c7c0 2023-11-08T08:22:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.395+0000 7fec98afe700 1 --2- 172.21.15.40:0/95672001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec9415c7c0 0x7fec9406cdd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fec88009850 tx=0x7fec88009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a41dda4eb7735ef server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:19.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.395+0000 7fec937fe700 1 -- 172.21.15.40:0/95672001 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c0019f0 con 0x7fec9415c7c0 2023-11-08T08:22:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.395+0000 7fec937fe700 1 -- 172.21.15.40:0/95672001 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec88010590 con 0x7fec9415c7c0 2023-11-08T08:22:19.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.395+0000 7fec937fe700 1 -- 172.21.15.40:0/95672001 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec88010940 con 0x7fec9415c7c0 2023-11-08T08:22:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.395+0000 7fec99b00700 1 -- 172.21.15.40:0/95672001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec9415c7c0 msgr2=0x7fec9406cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.395+0000 7fec99b00700 1 --2- 172.21.15.40:0/95672001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec9415c7c0 0x7fec9406cdd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fec88009850 tx=0x7fec88009b60 comp rx=0 tx=0).stop 2023-11-08T08:22:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.395+0000 7fec99b00700 1 -- 172.21.15.40:0/95672001 shutdown_connections 2023-11-08T08:22:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.395+0000 7fec99b00700 1 --2- 172.21.15.40:0/95672001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec9415c7c0 0x7fec9406cdd0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:19.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.395+0000 7fec99b00700 1 -- 172.21.15.40:0/95672001 >> 172.21.15.40:0/95672001 conn(0x7fec940d3800 msgr2=0x7fec940cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.395+0000 7fec99b00700 1 -- 172.21.15.40:0/95672001 shutdown_connections 2023-11-08T08:22:19.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.395+0000 7fec99b00700 1 -- 172.21.15.40:0/95672001 wait complete. 2023-11-08T08:22:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.399+0000 7fec99b00700 1 Processor -- start 2023-11-08T08:22:19.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.399+0000 7fec99b00700 1 -- start start 2023-11-08T08:22:19.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.399+0000 7fec99b00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec940cddf0 0x7fec941f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:19.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.399+0000 7fec99b00700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec9415c7c0 0x7fec940ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:19.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.399+0000 7fec99b00700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec940c7120 0x7fec940cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:19.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.399+0000 7fec99b00700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec940cb110 con 0x7fec9415c7c0 2023-11-08T08:22:19.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.399+0000 7fec99b00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec940cab90 con 0x7fec940cddf0 2023-11-08T08:22:19.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.399+0000 7fec99b00700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec940cae30 con 0x7fec940c7120 2023-11-08T08:22:19.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.399+0000 7fec93fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec9415c7c0 0x7fec940ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:19.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.399+0000 7fec93fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec9415c7c0 0x7fec940ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35780/0 (socket says 172.21.15.40:35780) 2023-11-08T08:22:19.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.399+0000 7fec93fff700 1 -- 172.21.15.40:0/1303558249 learned_addr learned my addr 172.21.15.40:0/1303558249 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:19.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.399+0000 7fec992ff700 1 --2- 172.21.15.40:0/1303558249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec940c7120 0x7fec940cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:19.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.399+0000 7fec98afe700 1 --2- 172.21.15.40:0/1303558249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec940cddf0 0x7fec941f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:19.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.403+0000 7fec93fff700 1 -- 172.21.15.40:0/1303558249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec940c7120 msgr2=0x7fec940cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:19.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.403+0000 7fec93fff700 1 --2- 172.21.15.40:0/1303558249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec940c7120 0x7fec940cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:19.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.403+0000 7fec93fff700 1 -- 172.21.15.40:0/1303558249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec940cddf0 msgr2=0x7fec941f5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:19.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.403+0000 7fec93fff700 1 --2- 172.21.15.40:0/1303558249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec940cddf0 0x7fec941f5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:19.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.403+0000 7fec93fff700 1 -- 172.21.15.40:0/1303558249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec880094b0 con 0x7fec9415c7c0 2023-11-08T08:22:19.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.403+0000 7fec992ff700 1 --2- 172.21.15.40:0/1303558249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec940c7120 0x7fec940cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:22:19.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.403+0000 7fec93fff700 1 --2- 172.21.15.40:0/1303558249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec9415c7c0 0x7fec940ca650 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fec8400c520 tx=0x7fec8400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:19.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.403+0000 7fec917fa700 1 -- 172.21.15.40:0/1303558249 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec840052d0 con 0x7fec9415c7c0 2023-11-08T08:22:19.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.403+0000 7fec917fa700 1 -- 172.21.15.40:0/1303558249 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec84005450 con 0x7fec9415c7c0 2023-11-08T08:22:19.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.403+0000 7fec99b00700 1 -- 172.21.15.40:0/1303558249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9406c8b0 con 0x7fec9415c7c0 2023-11-08T08:22:19.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.403+0000 7fec917fa700 1 -- 172.21.15.40:0/1303558249 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec840038e0 con 0x7fec9415c7c0 2023-11-08T08:22:19.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.403+0000 7fec99b00700 1 -- 172.21.15.40:0/1303558249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9406c5d0 con 0x7fec9415c7c0 2023-11-08T08:22:19.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.403+0000 7fec99b00700 1 -- 172.21.15.40:0/1303558249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9415cb90 con 0x7fec9415c7c0 2023-11-08T08:22:19.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.407+0000 7fec917fa700 1 -- 172.21.15.40:0/1303558249 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fec84019030 con 0x7fec9415c7c0 2023-11-08T08:22:19.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.407+0000 7fec917fa700 1 --2- 172.21.15.40:0/1303558249 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fec80064ee0 0x7fec80067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:19.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.407+0000 7fec917fa700 1 -- 172.21.15.40:0/1303558249 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5900+0+0 (secure 0 0 0) 0x7fec84081170 con 0x7fec9415c7c0 2023-11-08T08:22:19.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.411+0000 7fec98afe700 1 --2- 172.21.15.40:0/1303558249 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fec80064ee0 0x7fec80067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:19.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.411+0000 7fec98afe700 1 --2- 172.21.15.40:0/1303558249 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fec80064ee0 0x7fec80067390 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fec88013040 tx=0x7fec88008000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:19.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.415+0000 7fec917fa700 1 -- 172.21.15.40:0/1303558249 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fec8404f260 con 0x7fec9415c7c0 2023-11-08T08:22:19.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:19.591+0000 7fec99b00700 1 -- 172.21.15.40:0/1303558249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fec94073cb0 con 0x7fec9415c7c0 2023-11-08T08:22:20.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.179+0000 7fec917fa700 1 -- 172.21.15.40:0/1303558249 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v67) v1 ==== 170+0+0 (secure 0 0 0) 0x7fec84052880 con 0x7fec9415c7c0 2023-11-08T08:22:20.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.187+0000 7fec99b00700 1 -- 172.21.15.40:0/1303558249 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fec80064ee0 msgr2=0x7fec80067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:20.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.187+0000 7fec99b00700 1 --2- 172.21.15.40:0/1303558249 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fec80064ee0 0x7fec80067390 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fec88013040 tx=0x7fec88008000 comp rx=0 tx=0).stop 2023-11-08T08:22:20.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.187+0000 7fec99b00700 1 -- 172.21.15.40:0/1303558249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec9415c7c0 msgr2=0x7fec940ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:20.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.187+0000 7fec99b00700 1 --2- 172.21.15.40:0/1303558249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec9415c7c0 0x7fec940ca650 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fec8400c520 tx=0x7fec8400c830 comp rx=0 tx=0).stop 2023-11-08T08:22:20.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.191+0000 7fec99b00700 1 -- 172.21.15.40:0/1303558249 shutdown_connections 2023-11-08T08:22:20.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.191+0000 7fec99b00700 1 --2- 172.21.15.40:0/1303558249 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fec80064ee0 0x7fec80067390 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.191+0000 7fec99b00700 1 --2- 172.21.15.40:0/1303558249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec940c7120 0x7fec940cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.191+0000 7fec99b00700 1 --2- 172.21.15.40:0/1303558249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec9415c7c0 0x7fec940ca650 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.191+0000 7fec99b00700 1 --2- 172.21.15.40:0/1303558249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec940cddf0 0x7fec941f5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.191+0000 7fec99b00700 1 -- 172.21.15.40:0/1303558249 >> 172.21.15.40:0/1303558249 conn(0x7fec940d3800 msgr2=0x7fec9415bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.195+0000 7fec99b00700 1 -- 172.21.15.40:0/1303558249 shutdown_connections 2023-11-08T08:22:20.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.195+0000 7fec99b00700 1 -- 172.21.15.40:0/1303558249 wait complete. 2023-11-08T08:22:20.201 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2023-11-08T08:22:20.222 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T08:22:20.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.407+0000 7f28b490c700 1 Processor -- start 2023-11-08T08:22:20.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.407+0000 7f28b490c700 1 -- start start 2023-11-08T08:22:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.411+0000 7f28b490c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28b0159cd0 0x7f28b015d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.411+0000 7f28b490c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f28b00cc1a0 0x7f28b006c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.411+0000 7f28b490c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28b015d650 con 0x7f28b006ff10 2023-11-08T08:22:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.411+0000 7f28b490c700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28b006be80 con 0x7f28b00cc1a0 2023-11-08T08:22:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.411+0000 7f28b490c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b015a0a0 con 0x7f28b0159cd0 2023-11-08T08:22:20.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.411+0000 7f28a7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f28b00cc1a0 0x7f28b006c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:20.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.411+0000 7f28af59e700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f28b006ff10 0x7f28b0069bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48956/0 (socket says 172.21.15.40:48956) 2023-11-08T08:22:20.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.411+0000 7f28aed9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28b0159cd0 0x7f28b015d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:20.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.411+0000 7f28a7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f28b00cc1a0 0x7f28b006c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35790/0 (socket says 172.21.15.40:35790) 2023-11-08T08:22:20.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.411+0000 7f28af59e700 1 -- 172.21.15.40:0/2336983247 learned_addr learned my addr 172.21.15.40:0/2336983247 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:20.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.415+0000 7f28a7fff700 1 -- 172.21.15.40:0/2336983247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28b0159cd0 msgr2=0x7f28b015d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:20.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.415+0000 7f28a7fff700 1 --2- 172.21.15.40:0/2336983247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28b0159cd0 0x7f28b015d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:20.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.415+0000 7f28a7fff700 1 -- 172.21.15.40:0/2336983247 >> v1:172.21.15.40:6789/0 conn(0x7f28b006ff10 legacy=0x7f28b0069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:20.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.415+0000 7f28a7fff700 1 -- 172.21.15.40:0/2336983247 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b00cdf80 con 0x7f28b00cc1a0 2023-11-08T08:22:20.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.415+0000 7f28ae59c700 1 -- 172.21.15.40:0/2336983247 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4041624903 0 0) 0x7f28b015d650 con 0x7f28b006ff10 2023-11-08T08:22:20.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.415+0000 7f28aed9d700 1 --2- 172.21.15.40:0/2336983247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28b0159cd0 0x7f28b015d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:22:20.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.415+0000 7f28a7fff700 1 --2- 172.21.15.40:0/2336983247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b00cc1a0 0x7f28b006c890 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2898009850 tx=0x7f2898009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5234e332d38e20aa server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:20.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.415+0000 7f28ae59c700 1 -- 172.21.15.40:0/2336983247 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2898011830 con 0x7f28b00cc1a0 2023-11-08T08:22:20.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.415+0000 7f28ae59c700 1 -- 172.21.15.40:0/2336983247 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2898011990 con 0x7f28b00cc1a0 2023-11-08T08:22:20.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.415+0000 7f28ae59c700 1 -- 172.21.15.40:0/2336983247 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2898011d40 con 0x7f28b00cc1a0 2023-11-08T08:22:20.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.415+0000 7f28b490c700 1 -- 172.21.15.40:0/2336983247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b00cc1a0 msgr2=0x7f28b006c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:20.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.415+0000 7f28b490c700 1 --2- 172.21.15.40:0/2336983247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b00cc1a0 0x7f28b006c890 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2898009850 tx=0x7f2898009b60 comp rx=0 tx=0).stop 2023-11-08T08:22:20.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.415+0000 7f28b490c700 1 -- 172.21.15.40:0/2336983247 shutdown_connections 2023-11-08T08:22:20.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.415+0000 7f28b490c700 1 --2- 172.21.15.40:0/2336983247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b00cc1a0 0x7f28b006c890 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:20.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.415+0000 7f28b490c700 1 --2- 172.21.15.40:0/2336983247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28b0159cd0 0x7f28b015d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:20.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.415+0000 7f28b490c700 1 -- 172.21.15.40:0/2336983247 >> 172.21.15.40:0/2336983247 conn(0x7f28b00cf280 msgr2=0x7f28b00cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:20.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.415+0000 7f28b490c700 1 -- 172.21.15.40:0/2336983247 shutdown_connections 2023-11-08T08:22:20.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.415+0000 7f28b490c700 1 -- 172.21.15.40:0/2336983247 wait complete. 2023-11-08T08:22:20.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.419+0000 7f28b490c700 1 Processor -- start 2023-11-08T08:22:20.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.419+0000 7f28b490c700 1 -- start start 2023-11-08T08:22:20.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.419+0000 7f28b490c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28b006ff10 0x7f28b00b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:20.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.419+0000 7f28b490c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b00cc1a0 0x7f28b00c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:20.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.419+0000 7f28b490c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28b0159cd0 0x7f28b00b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:20.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.419+0000 7f28b490c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b01f5290 con 0x7f28b00cc1a0 2023-11-08T08:22:20.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.419+0000 7f28b490c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b01f5570 con 0x7f28b006ff10 2023-11-08T08:22:20.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.419+0000 7f28b490c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b01f5850 con 0x7f28b0159cd0 2023-11-08T08:22:20.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.419+0000 7f28a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b00cc1a0 0x7f28b00c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:20.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.419+0000 7f28a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b00cc1a0 0x7f28b00c6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35792/0 (socket says 172.21.15.40:35792) 2023-11-08T08:22:20.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.419+0000 7f28a7fff700 1 -- 172.21.15.40:0/52812344 learned_addr learned my addr 172.21.15.40:0/52812344 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:20.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.419+0000 7f28af59e700 1 --2- 172.21.15.40:0/52812344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28b0159cd0 0x7f28b00b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:20.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.419+0000 7f28aed9d700 1 --2- 172.21.15.40:0/52812344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28b006ff10 0x7f28b00b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:20.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.419+0000 7f28a7fff700 1 -- 172.21.15.40:0/52812344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28b0159cd0 msgr2=0x7f28b00b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:20.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.419+0000 7f28a7fff700 1 --2- 172.21.15.40:0/52812344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28b0159cd0 0x7f28b00b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:20.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.419+0000 7f28a7fff700 1 -- 172.21.15.40:0/52812344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28b006ff10 msgr2=0x7f28b00b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:20.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.419+0000 7f28a7fff700 1 --2- 172.21.15.40:0/52812344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28b006ff10 0x7f28b00b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:20.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.419+0000 7f28a7fff700 1 -- 172.21.15.40:0/52812344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a00094b0 con 0x7f28b00cc1a0 2023-11-08T08:22:20.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.423+0000 7f28a7fff700 1 --2- 172.21.15.40:0/52812344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b00cc1a0 0x7f28b00c6480 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2898000c00 tx=0x7f2898004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:20.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.423+0000 7f28a77fe700 1 -- 172.21.15.40:0/52812344 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289801a990 con 0x7f28b00cc1a0 2023-11-08T08:22:20.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.423+0000 7f28a77fe700 1 -- 172.21.15.40:0/52812344 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289801f070 con 0x7f28b00cc1a0 2023-11-08T08:22:20.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.423+0000 7f28a77fe700 1 -- 172.21.15.40:0/52812344 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289801ad30 con 0x7f28b00cc1a0 2023-11-08T08:22:20.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.423+0000 7f28b490c700 1 -- 172.21.15.40:0/52812344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28980094b0 con 0x7f28b00cc1a0 2023-11-08T08:22:20.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.423+0000 7f28b490c700 1 -- 172.21.15.40:0/52812344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b006c090 con 0x7f28b00cc1a0 2023-11-08T08:22:20.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.423+0000 7f28b490c700 1 -- 172.21.15.40:0/52812344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b00ce260 con 0x7f28b00cc1a0 2023-11-08T08:22:20.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.427+0000 7f28a77fe700 1 -- 172.21.15.40:0/52812344 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f289800f050 con 0x7f28b00cc1a0 2023-11-08T08:22:20.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.427+0000 7f28a77fe700 1 --2- 172.21.15.40:0/52812344 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2890064ee0 0x7f2890067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:20.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.427+0000 7f28a77fe700 1 -- 172.21.15.40:0/52812344 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f28980830a0 con 0x7f28b00cc1a0 2023-11-08T08:22:20.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.427+0000 7f28aed9d700 1 --2- 172.21.15.40:0/52812344 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2890064ee0 0x7f2890067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:20.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.427+0000 7f28aed9d700 1 --2- 172.21.15.40:0/52812344 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2890064ee0 0x7f2890067390 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f28a0009ce0 tx=0x7f28a0009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:20.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.435+0000 7f28a77fe700 1 -- 172.21.15.40:0/52812344 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f2898051070 con 0x7f28b00cc1a0 2023-11-08T08:22:20.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.647+0000 7f28b490c700 1 -- 172.21.15.40:0/52812344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f28b00cca80 con 0x7f28b00cc1a0 2023-11-08T08:22:20.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.663+0000 7f28a77fe700 1 -- 172.21.15.40:0/52812344 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v29) v1 ==== 174+0+0 (secure 0 0 0) 0x7f28980150c0 con 0x7f28b00cc1a0 2023-11-08T08:22:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.667+0000 7f28b490c700 1 -- 172.21.15.40:0/52812344 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2890064ee0 msgr2=0x7f2890067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.667+0000 7f28b490c700 1 --2- 172.21.15.40:0/52812344 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2890064ee0 0x7f2890067390 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f28a0009ce0 tx=0x7f28a0009240 comp rx=0 tx=0).stop 2023-11-08T08:22:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.667+0000 7f28b490c700 1 -- 172.21.15.40:0/52812344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b00cc1a0 msgr2=0x7f28b00c6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.667+0000 7f28b490c700 1 --2- 172.21.15.40:0/52812344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b00cc1a0 0x7f28b00c6480 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2898000c00 tx=0x7f2898004770 comp rx=0 tx=0).stop 2023-11-08T08:22:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.667+0000 7f28b490c700 1 -- 172.21.15.40:0/52812344 shutdown_connections 2023-11-08T08:22:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.667+0000 7f28b490c700 1 --2- 172.21.15.40:0/52812344 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2890064ee0 0x7f2890067390 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.667+0000 7f28b490c700 1 --2- 172.21.15.40:0/52812344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28b0159cd0 0x7f28b00b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.667+0000 7f28b490c700 1 --2- 172.21.15.40:0/52812344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28b00cc1a0 0x7f28b00c6480 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.667+0000 7f28b490c700 1 --2- 172.21.15.40:0/52812344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28b006ff10 0x7f28b00b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.667+0000 7f28b490c700 1 -- 172.21.15.40:0/52812344 >> 172.21.15.40:0/52812344 conn(0x7f28b00cf280 msgr2=0x7f28b006b140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:20.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.667+0000 7f28b490c700 1 -- 172.21.15.40:0/52812344 shutdown_connections 2023-11-08T08:22:20.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.667+0000 7f28b490c700 1 -- 172.21.15.40:0/52812344 wait complete. 2023-11-08T08:22:20.679 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 10 and data pool 11 2023-11-08T08:22:20.709 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:22:20.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.879+0000 7fa459ff2700 1 Processor -- start 2023-11-08T08:22:20.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.879+0000 7fa459ff2700 1 -- start start 2023-11-08T08:22:20.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.883+0000 7fa459ff2700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa4541591c0 0x7fa4540ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:20.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.883+0000 7fa459ff2700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa454159590 0x7fa45406bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:20.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.883+0000 7fa459ff2700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa45406b510 con 0x7fa4540cace0 2023-11-08T08:22:20.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.883+0000 7fa459ff2700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4540b0c00 con 0x7fa4541591c0 2023-11-08T08:22:20.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.883+0000 7fa459ff2700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4540b0ee0 con 0x7fa454159590 2023-11-08T08:22:20.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.883+0000 7fa4537fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa4541591c0 0x7fa4540ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:20.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.883+0000 7fa4537fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa4541591c0 0x7fa4540ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35796/0 (socket says 172.21.15.40:35796) 2023-11-08T08:22:20.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.883+0000 7fa4537fe700 1 -- 172.21.15.40:0/1796419299 learned_addr learned my addr 172.21.15.40:0/1796419299 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:20.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.883+0000 7fa452ffd700 1 --2- 172.21.15.40:0/1796419299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa454159590 0x7fa45406bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:20.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.883+0000 7fa4537fe700 1 -- 172.21.15.40:0/1796419299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa454159590 msgr2=0x7fa45406bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:20.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.883+0000 7fa4537fe700 1 --2- 172.21.15.40:0/1796419299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa454159590 0x7fa45406bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:20.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.883+0000 7fa4537fe700 1 -- 172.21.15.40:0/1796419299 >> v1:172.21.15.169:6789/0 conn(0x7fa4540cace0 legacy=0x7fa4540ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:20.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.883+0000 7fa4537fe700 1 -- 172.21.15.40:0/1796419299 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4540ca280 con 0x7fa4541591c0 2023-11-08T08:22:20.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.883+0000 7fa452ffd700 1 --2- 172.21.15.40:0/1796419299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa454159590 0x7fa45406bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:22:20.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.883+0000 7fa4537fe700 1 --2- 172.21.15.40:0/1796419299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4541591c0 0x7fa4540ca7a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa44400bbb0 tx=0x7fa44400bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4b41a2b057c0cb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:20.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.883+0000 7fa4527fc700 1 -- 172.21.15.40:0/1796419299 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44400dd90 con 0x7fa4541591c0 2023-11-08T08:22:20.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.883+0000 7fa4527fc700 1 -- 172.21.15.40:0/1796419299 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa444010470 con 0x7fa4541591c0 2023-11-08T08:22:20.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.883+0000 7fa4527fc700 1 -- 172.21.15.40:0/1796419299 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444010820 con 0x7fa4541591c0 2023-11-08T08:22:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.887+0000 7fa459ff2700 1 -- 172.21.15.40:0/1796419299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4541591c0 msgr2=0x7fa4540ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.887+0000 7fa459ff2700 1 --2- 172.21.15.40:0/1796419299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4541591c0 0x7fa4540ca7a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa44400bbb0 tx=0x7fa44400bec0 comp rx=0 tx=0).stop 2023-11-08T08:22:20.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.887+0000 7fa459ff2700 1 -- 172.21.15.40:0/1796419299 shutdown_connections 2023-11-08T08:22:20.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.887+0000 7fa459ff2700 1 --2- 172.21.15.40:0/1796419299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa454159590 0x7fa45406bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:20.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.887+0000 7fa459ff2700 1 --2- 172.21.15.40:0/1796419299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4541591c0 0x7fa4540ca7a0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:20.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.887+0000 7fa459ff2700 1 -- 172.21.15.40:0/1796419299 >> 172.21.15.40:0/1796419299 conn(0x7fa4540cf350 msgr2=0x7fa4540b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:20.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.887+0000 7fa459ff2700 1 -- 172.21.15.40:0/1796419299 shutdown_connections 2023-11-08T08:22:20.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.887+0000 7fa459ff2700 1 -- 172.21.15.40:0/1796419299 wait complete. 2023-11-08T08:22:20.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.891+0000 7fa459ff2700 1 Processor -- start 2023-11-08T08:22:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.891+0000 7fa459ff2700 1 -- start start 2023-11-08T08:22:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.891+0000 7fa459ff2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4540cace0 0x7fa4541556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.891+0000 7fa459ff2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4541591c0 0x7fa4540b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.891+0000 7fa459ff2700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa454159590 0x7fa45406c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.891+0000 7fa459ff2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa454155c30 con 0x7fa4540cace0 2023-11-08T08:22:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.891+0000 7fa459ff2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4540b3080 con 0x7fa4541591c0 2023-11-08T08:22:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.891+0000 7fa459ff2700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4540b3db0 con 0x7fa454159590 2023-11-08T08:22:20.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.891+0000 7fa4537fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4540cace0 0x7fa4541556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:20.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.891+0000 7fa4537fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4540cace0 0x7fa4541556f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35812/0 (socket says 172.21.15.40:35812) 2023-11-08T08:22:20.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.895+0000 7fa4537fe700 1 -- 172.21.15.40:0/2043233559 learned_addr learned my addr 172.21.15.40:0/2043233559 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:20.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.895+0000 7fa453fff700 1 --2- 172.21.15.40:0/2043233559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa454159590 0x7fa45406c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:20.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.895+0000 7fa452ffd700 1 --2- 172.21.15.40:0/2043233559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4541591c0 0x7fa4540b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:20.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.895+0000 7fa4537fe700 1 -- 172.21.15.40:0/2043233559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa454159590 msgr2=0x7fa45406c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:20.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.895+0000 7fa4537fe700 1 --2- 172.21.15.40:0/2043233559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa454159590 0x7fa45406c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:20.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.895+0000 7fa4537fe700 1 -- 172.21.15.40:0/2043233559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4541591c0 msgr2=0x7fa4540b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:20.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.895+0000 7fa4537fe700 1 --2- 172.21.15.40:0/2043233559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4541591c0 0x7fa4540b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:20.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.895+0000 7fa4537fe700 1 -- 172.21.15.40:0/2043233559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44400b810 con 0x7fa4540cace0 2023-11-08T08:22:20.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.895+0000 7fa453fff700 1 --2- 172.21.15.40:0/2043233559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa454159590 0x7fa45406c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:22:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.895+0000 7fa452ffd700 1 --2- 172.21.15.40:0/2043233559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4541591c0 0x7fa4540b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:22:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.895+0000 7fa4537fe700 1 --2- 172.21.15.40:0/2043233559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4540cace0 0x7fa4541556f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa45406f6f0 tx=0x7fa444004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.895+0000 7fa43bfff700 1 -- 172.21.15.40:0/2043233559 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444003d10 con 0x7fa4540cace0 2023-11-08T08:22:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.895+0000 7fa43bfff700 1 -- 172.21.15.40:0/2043233559 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa444003e90 con 0x7fa4540cace0 2023-11-08T08:22:20.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.895+0000 7fa459ff2700 1 -- 172.21.15.40:0/2043233559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4540cd340 con 0x7fa4540cace0 2023-11-08T08:22:20.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.895+0000 7fa43bfff700 1 -- 172.21.15.40:0/2043233559 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444010820 con 0x7fa4540cace0 2023-11-08T08:22:20.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.895+0000 7fa459ff2700 1 -- 172.21.15.40:0/2043233559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4540cd920 con 0x7fa4540cace0 2023-11-08T08:22:20.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.895+0000 7fa459ff2700 1 -- 172.21.15.40:0/2043233559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4540cd4d0 con 0x7fa4540cace0 2023-11-08T08:22:20.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.899+0000 7fa43bfff700 1 -- 172.21.15.40:0/2043233559 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa444019400 con 0x7fa4540cace0 2023-11-08T08:22:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.899+0000 7fa43bfff700 1 --2- 172.21.15.40:0/2043233559 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa440064ee0 0x7fa440067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.899+0000 7fa43bfff700 1 -- 172.21.15.40:0/2043233559 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa4440837a0 con 0x7fa4540cace0 2023-11-08T08:22:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.899+0000 7fa452ffd700 1 --2- 172.21.15.40:0/2043233559 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa440064ee0 0x7fa440067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:20.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.903+0000 7fa452ffd700 1 --2- 172.21.15.40:0/2043233559 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa440064ee0 0x7fa440067390 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa448006fd0 tx=0x7fa448008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:20.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:20.907+0000 7fa43bfff700 1 -- 172.21.15.40:0/2043233559 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa444051730 con 0x7fa4540cace0 2023-11-08T08:22:21.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.079+0000 7fa459ff2700 1 -- 172.21.15.40:0/2043233559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4540726c0 con 0x7fa4540cace0 2023-11-08T08:22:21.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.083+0000 7fa43bfff700 1 -- 172.21.15.40:0/2043233559 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+18100 (secure 0 0 0) 0x7fa444006da0 con 0x7fa4540cace0 2023-11-08T08:22:21.088 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:22:21.088 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":68,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:22:20.649933+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:22:18.224463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T08:22:19.658954+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:22:21.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.087+0000 7fa459ff2700 1 -- 172.21.15.40:0/2043233559 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa440064ee0 msgr2=0x7fa440067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.087+0000 7fa459ff2700 1 --2- 172.21.15.40:0/2043233559 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa440064ee0 0x7fa440067390 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa448006fd0 tx=0x7fa448008040 comp rx=0 tx=0).stop 2023-11-08T08:22:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.087+0000 7fa459ff2700 1 -- 172.21.15.40:0/2043233559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4540cace0 msgr2=0x7fa4541556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.087+0000 7fa459ff2700 1 --2- 172.21.15.40:0/2043233559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4540cace0 0x7fa4541556f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa45406f6f0 tx=0x7fa444004770 comp rx=0 tx=0).stop 2023-11-08T08:22:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.087+0000 7fa459ff2700 1 -- 172.21.15.40:0/2043233559 shutdown_connections 2023-11-08T08:22:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.087+0000 7fa459ff2700 1 --2- 172.21.15.40:0/2043233559 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa440064ee0 0x7fa440067390 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.087+0000 7fa459ff2700 1 --2- 172.21.15.40:0/2043233559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa454159590 0x7fa45406c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:21.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.087+0000 7fa459ff2700 1 --2- 172.21.15.40:0/2043233559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4541591c0 0x7fa4540b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:21.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.087+0000 7fa459ff2700 1 --2- 172.21.15.40:0/2043233559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4540cace0 0x7fa4541556f0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:21.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.087+0000 7fa459ff2700 1 -- 172.21.15.40:0/2043233559 >> 172.21.15.40:0/2043233559 conn(0x7fa4540cf350 msgr2=0x7fa45415b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:21.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.087+0000 7fa459ff2700 1 -- 172.21.15.40:0/2043233559 shutdown_connections 2023-11-08T08:22:21.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.087+0000 7fa459ff2700 1 -- 172.21.15.40:0/2043233559 wait complete. 2023-11-08T08:22:21.115 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:22:21.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.291+0000 7f2fbb158700 1 Processor -- start 2023-11-08T08:22:21.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.291+0000 7f2fbb158700 1 -- start start 2023-11-08T08:22:21.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fbb158700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2fb4160710 0x7f2fb40cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:21.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fbb158700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2fb40cc860 0x7f2fb4070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:21.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fbb158700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fb40705c0 0x7f2fb4069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:21.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fbb158700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb4070a60 con 0x7f2fb4160710 2023-11-08T08:22:21.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fbb158700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb406b1c0 con 0x7f2fb40cc860 2023-11-08T08:22:21.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fbb158700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb40ad6c0 con 0x7f2fb40705c0 2023-11-08T08:22:21.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fba156700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2fb4160710 0x7f2fb40cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:21.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fba156700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2fb4160710 0x7f2fb40cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35818/0 (socket says 172.21.15.40:35818) 2023-11-08T08:22:21.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fba156700 1 -- 172.21.15.40:0/3156689501 learned_addr learned my addr 172.21.15.40:0/3156689501 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:21.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fb9955700 1 --2- 172.21.15.40:0/3156689501 >> v2:172.21.15.169:3300/0 conn(0x7f2fb40cc860 0x7f2fb4070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:21.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fba957700 1 --2- 172.21.15.40:0/3156689501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fb40705c0 0x7f2fb4069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:21.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fba156700 1 -- 172.21.15.40:0/3156689501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fb40705c0 msgr2=0x7f2fb4069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:21.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fba156700 1 --2- 172.21.15.40:0/3156689501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fb40705c0 0x7f2fb4069bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:21.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fba156700 1 -- 172.21.15.40:0/3156689501 >> v2:172.21.15.169:3300/0 conn(0x7f2fb40cc860 msgr2=0x7f2fb4070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:21.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fba156700 1 --2- 172.21.15.40:0/3156689501 >> v2:172.21.15.169:3300/0 conn(0x7f2fb40cc860 0x7f2fb4070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:21.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fba156700 1 -- 172.21.15.40:0/3156689501 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb40adc60 con 0x7f2fb4160710 2023-11-08T08:22:21.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fba156700 1 --2- 172.21.15.40:0/3156689501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fb4160710 0x7f2fb40cc320 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2fb41641b0 tx=0x7f2fb0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c13446479cac0a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:21.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fb9154700 1 -- 172.21.15.40:0/3156689501 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb000b8a0 con 0x7f2fb4160710 2023-11-08T08:22:21.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fb9154700 1 -- 172.21.15.40:0/3156689501 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb000ba00 con 0x7f2fb4160710 2023-11-08T08:22:21.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fb9154700 1 -- 172.21.15.40:0/3156689501 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb000bd90 con 0x7f2fb4160710 2023-11-08T08:22:21.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fbb158700 1 -- 172.21.15.40:0/3156689501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fb4160710 msgr2=0x7f2fb40cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:21.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.295+0000 7f2fbb158700 1 --2- 172.21.15.40:0/3156689501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fb4160710 0x7f2fb40cc320 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2fb41641b0 tx=0x7f2fb0009bf0 comp rx=0 tx=0).stop 2023-11-08T08:22:21.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.299+0000 7f2fbb158700 1 -- 172.21.15.40:0/3156689501 shutdown_connections 2023-11-08T08:22:21.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.299+0000 7f2fbb158700 1 --2- 172.21.15.40:0/3156689501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fb40705c0 0x7f2fb4069bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:21.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.299+0000 7f2fbb158700 1 --2- 172.21.15.40:0/3156689501 >> v2:172.21.15.169:3300/0 conn(0x7f2fb40cc860 0x7f2fb4070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:21.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.299+0000 7f2fbb158700 1 --2- 172.21.15.40:0/3156689501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fb4160710 0x7f2fb40cc320 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:21.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.299+0000 7f2fbb158700 1 -- 172.21.15.40:0/3156689501 >> 172.21.15.40:0/3156689501 conn(0x7f2fb40b0f80 msgr2=0x7f2fb40ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:21.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.299+0000 7f2fbb158700 1 -- 172.21.15.40:0/3156689501 shutdown_connections 2023-11-08T08:22:21.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.299+0000 7f2fbb158700 1 -- 172.21.15.40:0/3156689501 wait complete. 2023-11-08T08:22:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.299+0000 7f2fbb158700 1 Processor -- start 2023-11-08T08:22:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.299+0000 7f2fbb158700 1 -- start start 2023-11-08T08:22:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2fbb158700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fb40705c0 0x7f2fb41f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2fbb158700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fb40cc860 0x7f2fb415eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2fbb158700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fb4160710 0x7f2fb415f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2fbb158700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb415ff20 con 0x7f2fb40705c0 2023-11-08T08:22:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2fbb158700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb41f5de0 con 0x7f2fb4160710 2023-11-08T08:22:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2fbb158700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb415f680 con 0x7f2fb40cc860 2023-11-08T08:22:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2fba156700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fb40705c0 0x7f2fb41f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:21.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2fba156700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fb40705c0 0x7f2fb41f58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35832/0 (socket says 172.21.15.40:35832) 2023-11-08T08:22:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2fba156700 1 -- 172.21.15.40:0/3066480136 learned_addr learned my addr 172.21.15.40:0/3066480136 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2fb9955700 1 --2- 172.21.15.40:0/3066480136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fb40cc860 0x7f2fb415eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2fba957700 1 --2- 172.21.15.40:0/3066480136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fb4160710 0x7f2fb415f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2fba156700 1 -- 172.21.15.40:0/3066480136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fb40cc860 msgr2=0x7f2fb415eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:21.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2fba156700 1 --2- 172.21.15.40:0/3066480136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fb40cc860 0x7f2fb415eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:21.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2fba156700 1 -- 172.21.15.40:0/3066480136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fb4160710 msgr2=0x7f2fb415f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:21.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2fba156700 1 --2- 172.21.15.40:0/3066480136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fb4160710 0x7f2fb415f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:21.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2fba156700 1 -- 172.21.15.40:0/3066480136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb0009570 con 0x7f2fb40705c0 2023-11-08T08:22:21.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2fba156700 1 --2- 172.21.15.40:0/3066480136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fb40705c0 0x7f2fb41f58a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0000c00 tx=0x7f2fb0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:21.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2faaffd700 1 -- 172.21.15.40:0/3066480136 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb00038c0 con 0x7f2fb40705c0 2023-11-08T08:22:21.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2faaffd700 1 -- 172.21.15.40:0/3066480136 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb001e070 con 0x7f2fb40705c0 2023-11-08T08:22:21.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2faaffd700 1 -- 172.21.15.40:0/3066480136 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0003c80 con 0x7f2fb40705c0 2023-11-08T08:22:21.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2fbb158700 1 -- 172.21.15.40:0/3066480136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb406b6b0 con 0x7f2fb40705c0 2023-11-08T08:22:21.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.303+0000 7f2fbb158700 1 -- 172.21.15.40:0/3066480136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb406b3d0 con 0x7f2fb40705c0 2023-11-08T08:22:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.307+0000 7f2fbb158700 1 -- 172.21.15.40:0/3066480136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb40ad8f0 con 0x7f2fb40705c0 2023-11-08T08:22:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.307+0000 7f2faaffd700 1 -- 172.21.15.40:0/3066480136 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f2fb000f050 con 0x7f2fb40705c0 2023-11-08T08:22:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.311+0000 7f2faaffd700 1 --2- 172.21.15.40:0/3066480136 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2f98064ee0 0x7f2f98067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:21.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.311+0000 7f2faaffd700 1 -- 172.21.15.40:0/3066480136 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2fb0020070 con 0x7f2fb40705c0 2023-11-08T08:22:21.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.311+0000 7f2fb9955700 1 --2- 172.21.15.40:0/3066480136 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2f98064ee0 0x7f2f98067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:21.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.311+0000 7f2fb9955700 1 --2- 172.21.15.40:0/3066480136 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2f98064ee0 0x7f2f98067390 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2fb40710f0 tx=0x7f2fa4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.315+0000 7f2faaffd700 1 -- 172.21.15.40:0/3066480136 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f2fb0051690 con 0x7f2fb40705c0 2023-11-08T08:22:21.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.491+0000 7f2fbb158700 1 -- 172.21.15.40:0/3066480136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fb40cf190 con 0x7f2fb40705c0 2023-11-08T08:22:21.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.495+0000 7f2faaffd700 1 -- 172.21.15.40:0/3066480136 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+18100 (secure 0 0 0) 0x7f2fb0054cb0 con 0x7f2fb40705c0 2023-11-08T08:22:21.499 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:22:21.500 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":68,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:22:20.649933+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:22:18.224463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T08:22:19.658954+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:22:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.499+0000 7f2fbb158700 1 -- 172.21.15.40:0/3066480136 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2f98064ee0 msgr2=0x7f2f98067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.499+0000 7f2fbb158700 1 --2- 172.21.15.40:0/3066480136 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2f98064ee0 0x7f2f98067390 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2fb40710f0 tx=0x7f2fa4008040 comp rx=0 tx=0).stop 2023-11-08T08:22:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.499+0000 7f2fbb158700 1 -- 172.21.15.40:0/3066480136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fb40705c0 msgr2=0x7f2fb41f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.499+0000 7f2fbb158700 1 --2- 172.21.15.40:0/3066480136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fb40705c0 0x7f2fb41f58a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0000c00 tx=0x7f2fb0004770 comp rx=0 tx=0).stop 2023-11-08T08:22:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.499+0000 7f2fbb158700 1 -- 172.21.15.40:0/3066480136 shutdown_connections 2023-11-08T08:22:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.499+0000 7f2fbb158700 1 --2- 172.21.15.40:0/3066480136 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2f98064ee0 0x7f2f98067390 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.499+0000 7f2fbb158700 1 --2- 172.21.15.40:0/3066480136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fb4160710 0x7f2fb415f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.499+0000 7f2fbb158700 1 --2- 172.21.15.40:0/3066480136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fb40cc860 0x7f2fb415eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.499+0000 7f2fbb158700 1 --2- 172.21.15.40:0/3066480136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fb40705c0 0x7f2fb41f58a0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.499+0000 7f2fbb158700 1 -- 172.21.15.40:0/3066480136 >> 172.21.15.40:0/3066480136 conn(0x7f2fb40b0f80 msgr2=0x7f2fb40aeb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:21.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.499+0000 7f2fbb158700 1 -- 172.21.15.40:0/3066480136 shutdown_connections 2023-11-08T08:22:21.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.499+0000 7f2fbb158700 1 -- 172.21.15.40:0/3066480136 wait complete. 2023-11-08T08:22:21.525 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T08:22:21.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.699+0000 7fcb2a64c700 1 Processor -- start 2023-11-08T08:22:21.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.699+0000 7fcb2a64c700 1 -- start start 2023-11-08T08:22:21.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.703+0000 7fcb2a64c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcb2406c590 0x7fcb240ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:21.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.703+0000 7fcb2a64c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb240cd110 con 0x7fcb2406cbc0 2023-11-08T08:22:21.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.703+0000 7fcb2a64c700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb240cd340 con 0x7fcb2415c360 2023-11-08T08:22:21.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.703+0000 7fcb2a64c700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb240c9a40 con 0x7fcb2406c590 2023-11-08T08:22:21.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.703+0000 7fcb28e49700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcb2406c590 0x7fcb240ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:21.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.703+0000 7fcb29e4b700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcb2406cbc0 0x7fcb240c9e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48964/0 (socket says 172.21.15.40:48964) 2023-11-08T08:22:21.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.703+0000 7fcb29e4b700 1 -- 172.21.15.40:0/2623879235 learned_addr learned my addr 172.21.15.40:0/2623879235 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:21.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.703+0000 7fcb28e49700 1 -- 172.21.15.40:0/2623879235 >> v1:172.21.15.169:6789/0 conn(0x7fcb2415c360 legacy=0x7fcb2406be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:21.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.703+0000 7fcb28e49700 1 -- 172.21.15.40:0/2623879235 >> v1:172.21.15.40:6789/0 conn(0x7fcb2406cbc0 legacy=0x7fcb240c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:21.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.703+0000 7fcb28e49700 1 -- 172.21.15.40:0/2623879235 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb240cd620 con 0x7fcb2406c590 2023-11-08T08:22:21.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.703+0000 7fcb1bfff700 1 -- 172.21.15.40:0/2623879235 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636993250 0 0) 0x7fcb240cd110 con 0x7fcb2406cbc0 2023-11-08T08:22:21.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.707+0000 7fcb28e49700 1 --2- 172.21.15.40:0/2623879235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb2406c590 0x7fcb240ccbd0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009850 tx=0x7fcb14009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e9b14fb94bb223 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:21.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.707+0000 7fcb1bfff700 1 -- 172.21.15.40:0/2623879235 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb14011830 con 0x7fcb2406c590 2023-11-08T08:22:21.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.707+0000 7fcb1bfff700 1 -- 172.21.15.40:0/2623879235 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb14011990 con 0x7fcb2406c590 2023-11-08T08:22:21.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.707+0000 7fcb1bfff700 1 -- 172.21.15.40:0/2623879235 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb14011d20 con 0x7fcb2406c590 2023-11-08T08:22:21.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.707+0000 7fcb2a64c700 1 -- 172.21.15.40:0/2623879235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb2406c590 msgr2=0x7fcb240ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:21.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.707+0000 7fcb2a64c700 1 --2- 172.21.15.40:0/2623879235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb2406c590 0x7fcb240ccbd0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009850 tx=0x7fcb14009b60 comp rx=0 tx=0).stop 2023-11-08T08:22:21.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.707+0000 7fcb2a64c700 1 -- 172.21.15.40:0/2623879235 shutdown_connections 2023-11-08T08:22:21.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.707+0000 7fcb2a64c700 1 --2- 172.21.15.40:0/2623879235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb2406c590 0x7fcb240ccbd0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:21.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.707+0000 7fcb2a64c700 1 -- 172.21.15.40:0/2623879235 >> 172.21.15.40:0/2623879235 conn(0x7fcb240cf280 msgr2=0x7fcb240cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:21.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.707+0000 7fcb2a64c700 1 -- 172.21.15.40:0/2623879235 shutdown_connections 2023-11-08T08:22:21.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.707+0000 7fcb2a64c700 1 -- 172.21.15.40:0/2623879235 wait complete. 2023-11-08T08:22:21.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb2a64c700 1 Processor -- start 2023-11-08T08:22:21.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb2a64c700 1 -- start start 2023-11-08T08:22:21.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb2a64c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb2406c590 0x7fcb2414e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:21.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb2a64c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb2406cbc0 0x7fcb241521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:21.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb2a64c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb2415c360 0x7fcb24159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:21.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb2a64c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb24152020 con 0x7fcb2406cbc0 2023-11-08T08:22:21.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb2a64c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb241509a0 con 0x7fcb2415c360 2023-11-08T08:22:21.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb2a64c700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb241506c0 con 0x7fcb2406c590 2023-11-08T08:22:21.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb28e49700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb2406cbc0 0x7fcb241521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:21.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb28e49700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb2406cbc0 0x7fcb241521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35838/0 (socket says 172.21.15.40:35838) 2023-11-08T08:22:21.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb28e49700 1 -- 172.21.15.40:0/2543818507 learned_addr learned my addr 172.21.15.40:0/2543818507 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:21.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb29e4b700 1 --2- 172.21.15.40:0/2543818507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb2415c360 0x7fcb24159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:21.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb2964a700 1 --2- 172.21.15.40:0/2543818507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb2406c590 0x7fcb2414e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:21.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb28e49700 1 -- 172.21.15.40:0/2543818507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb2406c590 msgr2=0x7fcb2414e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:21.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb28e49700 1 --2- 172.21.15.40:0/2543818507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb2406c590 0x7fcb2414e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:21.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb28e49700 1 -- 172.21.15.40:0/2543818507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb2415c360 msgr2=0x7fcb24159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:21.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb28e49700 1 --2- 172.21.15.40:0/2543818507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb2415c360 0x7fcb24159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:21.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb28e49700 1 -- 172.21.15.40:0/2543818507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb140094b0 con 0x7fcb2406cbc0 2023-11-08T08:22:21.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb29e4b700 1 --2- 172.21.15.40:0/2543818507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb2415c360 0x7fcb24159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:22:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb28e49700 1 --2- 172.21.15.40:0/2543818507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb2406cbc0 0x7fcb241521a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009fd0 tx=0x7fcb14004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb19ffb700 1 -- 172.21.15.40:0/2543818507 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb140036c0 con 0x7fcb2406cbc0 2023-11-08T08:22:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb19ffb700 1 -- 172.21.15.40:0/2543818507 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1401e070 con 0x7fcb2406cbc0 2023-11-08T08:22:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb19ffb700 1 -- 172.21.15.40:0/2543818507 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb14003a60 con 0x7fcb2406cbc0 2023-11-08T08:22:21.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.711+0000 7fcb2a64c700 1 -- 172.21.15.40:0/2543818507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2415c730 con 0x7fcb2406cbc0 2023-11-08T08:22:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.715+0000 7fcb2a64c700 1 -- 172.21.15.40:0/2543818507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb240c9660 con 0x7fcb2406cbc0 2023-11-08T08:22:21.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.715+0000 7fcb2a64c700 1 -- 172.21.15.40:0/2543818507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb240c9350 con 0x7fcb2406cbc0 2023-11-08T08:22:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.715+0000 7fcb19ffb700 1 -- 172.21.15.40:0/2543818507 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fcb1402a020 con 0x7fcb2406cbc0 2023-11-08T08:22:21.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.719+0000 7fcb19ffb700 1 --2- 172.21.15.40:0/2543818507 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcb08064ee0 0x7fcb08067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:21.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.719+0000 7fcb19ffb700 1 -- 172.21.15.40:0/2543818507 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fcb14083240 con 0x7fcb2406cbc0 2023-11-08T08:22:21.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.719+0000 7fcb2964a700 1 --2- 172.21.15.40:0/2543818507 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcb08064ee0 0x7fcb08067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.719+0000 7fcb2964a700 1 --2- 172.21.15.40:0/2543818507 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcb08064ee0 0x7fcb08067390 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcb240b01e0 tx=0x7fcb20009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.727+0000 7fcb19ffb700 1 -- 172.21.15.40:0/2543818507 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fcb140511b0 con 0x7fcb2406cbc0 2023-11-08T08:22:21.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:21.935+0000 7fcb2a64c700 1 -- 172.21.15.40:0/2543818507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcb24073d20 con 0x7fcb2406cbc0 2023-11-08T08:22:22.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:22.847+0000 7fcb19ffb700 1 -- 172.21.15.40:0/2543818507 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcb140547d0 con 0x7fcb2406cbc0 2023-11-08T08:22:22.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:22.855+0000 7fcb2a64c700 1 -- 172.21.15.40:0/2543818507 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcb08064ee0 msgr2=0x7fcb08067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:22.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:22.855+0000 7fcb2a64c700 1 --2- 172.21.15.40:0/2543818507 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcb08064ee0 0x7fcb08067390 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcb240b01e0 tx=0x7fcb20009040 comp rx=0 tx=0).stop 2023-11-08T08:22:22.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:22.855+0000 7fcb2a64c700 1 -- 172.21.15.40:0/2543818507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb2406cbc0 msgr2=0x7fcb241521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:22.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:22.855+0000 7fcb2a64c700 1 --2- 172.21.15.40:0/2543818507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb2406cbc0 0x7fcb241521a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009fd0 tx=0x7fcb14004770 comp rx=0 tx=0).stop 2023-11-08T08:22:22.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:22.855+0000 7fcb2a64c700 1 -- 172.21.15.40:0/2543818507 shutdown_connections 2023-11-08T08:22:22.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:22.855+0000 7fcb2a64c700 1 --2- 172.21.15.40:0/2543818507 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcb08064ee0 0x7fcb08067390 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:22.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:22.855+0000 7fcb2a64c700 1 --2- 172.21.15.40:0/2543818507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb2415c360 0x7fcb24159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:22.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:22.855+0000 7fcb2a64c700 1 --2- 172.21.15.40:0/2543818507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb2406cbc0 0x7fcb241521a0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:22.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:22.855+0000 7fcb2a64c700 1 --2- 172.21.15.40:0/2543818507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb2406c590 0x7fcb2414e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:22.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:22.855+0000 7fcb2a64c700 1 -- 172.21.15.40:0/2543818507 >> 172.21.15.40:0/2543818507 conn(0x7fcb240cf280 msgr2=0x7fcb240c47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:22.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:22.855+0000 7fcb2a64c700 1 -- 172.21.15.40:0/2543818507 shutdown_connections 2023-11-08T08:22:22.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:22.855+0000 7fcb2a64c700 1 -- 172.21.15.40:0/2543818507 wait complete. 2023-11-08T08:22:22.883 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-08T08:22:22.883 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T08:22:23.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.063+0000 7f5f92908700 1 Processor -- start 2023-11-08T08:22:23.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.063+0000 7f5f92908700 1 -- start start 2023-11-08T08:22:23.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.063+0000 7f5f92908700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5f8c070620 0x7f5f8c0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:23.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.063+0000 7f5f92908700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8c15c520 con 0x7f5f8c15c150 2023-11-08T08:22:23.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.063+0000 7f5f92908700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8c06cc50 con 0x7f5f8c069bf0 2023-11-08T08:22:23.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.063+0000 7f5f92908700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f8c0b4240 con 0x7f5f8c070620 2023-11-08T08:22:23.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.067+0000 7f5f91906700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5f8c15c150 0x7f5f8c06ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48976/0 (socket says 172.21.15.40:48976) 2023-11-08T08:22:23.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.067+0000 7f5f91906700 1 -- 172.21.15.40:0/2105612136 learned_addr learned my addr 172.21.15.40:0/2105612136 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:23.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.067+0000 7f5f91105700 1 --2- 172.21.15.40:0/2105612136 >> v2:172.21.15.169:3300/0 conn(0x7f5f8c070620 0x7f5f8c0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:23.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.067+0000 7f5f90904700 1 -- 172.21.15.40:0/2105612136 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461422016 0 0) 0x7f5f8c15c520 con 0x7f5f8c15c150 2023-11-08T08:22:23.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.067+0000 7f5f90904700 1 -- 172.21.15.40:0/2105612136 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f780035d0 con 0x7f5f8c15c150 2023-11-08T08:22:23.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.067+0000 7f5f90904700 1 -- 172.21.15.40:0/2105612136 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1556344944 0 0) 0x7f5f8c06cc50 con 0x7f5f8c069bf0 2023-11-08T08:22:23.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.067+0000 7f5f90904700 1 -- 172.21.15.40:0/2105612136 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f8c15c520 con 0x7f5f8c069bf0 2023-11-08T08:22:23.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.067+0000 7f5f91105700 1 -- 172.21.15.40:0/2105612136 >> v1:172.21.15.169:6789/0 conn(0x7f5f8c069bf0 legacy=0x7f5f8c069ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:23.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.067+0000 7f5f91105700 1 -- 172.21.15.40:0/2105612136 >> v1:172.21.15.40:6789/0 conn(0x7f5f8c15c150 legacy=0x7f5f8c06ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:23.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.067+0000 7f5f91105700 1 -- 172.21.15.40:0/2105612136 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8c0d2450 con 0x7f5f8c070620 2023-11-08T08:22:23.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.067+0000 7f5f91105700 1 --2- 172.21.15.40:0/2105612136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f8c070620 0x7f5f8c0b3d00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009840 tx=0x7f5f7c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf1a9c45dbabcf73 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:23.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.067+0000 7f5f90904700 1 -- 172.21.15.40:0/2105612136 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c005e50 con 0x7f5f8c070620 2023-11-08T08:22:23.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.067+0000 7f5f90904700 1 -- 172.21.15.40:0/2105612136 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7c006030 con 0x7f5f8c070620 2023-11-08T08:22:23.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.067+0000 7f5f90904700 1 -- 172.21.15.40:0/2105612136 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c0063a0 con 0x7f5f8c070620 2023-11-08T08:22:23.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.071+0000 7f5f92908700 1 -- 172.21.15.40:0/2105612136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f8c070620 msgr2=0x7f5f8c0b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:23.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.071+0000 7f5f92908700 1 --2- 172.21.15.40:0/2105612136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f8c070620 0x7f5f8c0b3d00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009840 tx=0x7f5f7c009b50 comp rx=0 tx=0).stop 2023-11-08T08:22:23.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.071+0000 7f5f92908700 1 -- 172.21.15.40:0/2105612136 shutdown_connections 2023-11-08T08:22:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.071+0000 7f5f92908700 1 --2- 172.21.15.40:0/2105612136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f8c070620 0x7f5f8c0b3d00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.071+0000 7f5f92908700 1 -- 172.21.15.40:0/2105612136 >> 172.21.15.40:0/2105612136 conn(0x7f5f8c0d3680 msgr2=0x7f5f8c0cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.071+0000 7f5f92908700 1 -- 172.21.15.40:0/2105612136 shutdown_connections 2023-11-08T08:22:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.071+0000 7f5f92908700 1 -- 172.21.15.40:0/2105612136 wait complete. 2023-11-08T08:22:23.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.075+0000 7f5f92908700 1 Processor -- start 2023-11-08T08:22:23.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.075+0000 7f5f92908700 1 -- start start 2023-11-08T08:22:23.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.075+0000 7f5f92908700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f8c069bf0 0x7f5f8c1546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:23.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.075+0000 7f5f92908700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f8c070620 0x7f5f8c0cf620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:23.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.075+0000 7f5f92908700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f8c15c150 0x7f5f8c150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:23.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.075+0000 7f5f92908700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c0cd1b0 con 0x7f5f8c070620 2023-11-08T08:22:23.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.075+0000 7f5f92908700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c0cd490 con 0x7f5f8c069bf0 2023-11-08T08:22:23.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.075+0000 7f5f92908700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c0cd770 con 0x7f5f8c15c150 2023-11-08T08:22:23.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.075+0000 7f5f91105700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f8c070620 0x7f5f8c0cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:23.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.075+0000 7f5f91105700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f8c070620 0x7f5f8c0cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35846/0 (socket says 172.21.15.40:35846) 2023-11-08T08:22:23.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.075+0000 7f5f91105700 1 -- 172.21.15.40:0/1206141335 learned_addr learned my addr 172.21.15.40:0/1206141335 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:23.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.075+0000 7f5f91906700 1 --2- 172.21.15.40:0/1206141335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f8c069bf0 0x7f5f8c1546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:23.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.075+0000 7f5f92107700 1 --2- 172.21.15.40:0/1206141335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f8c15c150 0x7f5f8c150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:23.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.075+0000 7f5f91105700 1 -- 172.21.15.40:0/1206141335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f8c15c150 msgr2=0x7f5f8c150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:23.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.075+0000 7f5f91105700 1 --2- 172.21.15.40:0/1206141335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f8c15c150 0x7f5f8c150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:23.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.075+0000 7f5f91105700 1 -- 172.21.15.40:0/1206141335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f8c069bf0 msgr2=0x7f5f8c1546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:23.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.075+0000 7f5f91105700 1 --2- 172.21.15.40:0/1206141335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f8c069bf0 0x7f5f8c1546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:23.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.075+0000 7f5f91105700 1 -- 172.21.15.40:0/1206141335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c009450 con 0x7f5f8c070620 2023-11-08T08:22:23.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.079+0000 7f5f91105700 1 --2- 172.21.15.40:0/1206141335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f8c070620 0x7f5f8c0cf620 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c012000 tx=0x7f5f7c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:23.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.079+0000 7f5f827fc700 1 -- 172.21.15.40:0/1206141335 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c0038c0 con 0x7f5f8c070620 2023-11-08T08:22:23.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.079+0000 7f5f827fc700 1 -- 172.21.15.40:0/1206141335 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7c020070 con 0x7f5f8c070620 2023-11-08T08:22:23.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.079+0000 7f5f827fc700 1 -- 172.21.15.40:0/1206141335 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c003c60 con 0x7f5f8c070620 2023-11-08T08:22:23.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.079+0000 7f5f92908700 1 -- 172.21.15.40:0/1206141335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f8c0d05e0 con 0x7f5f8c070620 2023-11-08T08:22:23.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.079+0000 7f5f92908700 1 -- 172.21.15.40:0/1206141335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f8c0d2730 con 0x7f5f8c070620 2023-11-08T08:22:23.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.079+0000 7f5f92908700 1 -- 172.21.15.40:0/1206141335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f8c0d2140 con 0x7f5f8c070620 2023-11-08T08:22:23.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.083+0000 7f5f827fc700 1 -- 172.21.15.40:0/1206141335 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5f7c02b020 con 0x7f5f8c070620 2023-11-08T08:22:23.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.083+0000 7f5f827fc700 1 --2- 172.21.15.40:0/1206141335 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5f70064ee0 0x7f5f70067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:23.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.083+0000 7f5f827fc700 1 -- 172.21.15.40:0/1206141335 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5f7c083690 con 0x7f5f8c070620 2023-11-08T08:22:23.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.083+0000 7f5f91906700 1 --2- 172.21.15.40:0/1206141335 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5f70064ee0 0x7f5f70067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:23.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.083+0000 7f5f91906700 1 --2- 172.21.15.40:0/1206141335 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5f70064ee0 0x7f5f70067390 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5f88000f60 tx=0x7f5f88007e20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:23.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.091+0000 7f5f827fc700 1 -- 172.21.15.40:0/1206141335 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f5f7c051600 con 0x7f5f8c070620 2023-11-08T08:22:23.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.303+0000 7f5f92908700 1 -- 172.21.15.40:0/1206141335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5f8c073d20 con 0x7f5f8c070620 2023-11-08T08:22:23.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.859+0000 7f5f827fc700 1 -- 172.21.15.40:0/1206141335 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5f7c054c20 con 0x7f5f8c070620 2023-11-08T08:22:23.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.863+0000 7f5f92908700 1 -- 172.21.15.40:0/1206141335 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5f70064ee0 msgr2=0x7f5f70067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:23.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.863+0000 7f5f92908700 1 --2- 172.21.15.40:0/1206141335 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5f70064ee0 0x7f5f70067390 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5f88000f60 tx=0x7f5f88007e20 comp rx=0 tx=0).stop 2023-11-08T08:22:23.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.863+0000 7f5f92908700 1 -- 172.21.15.40:0/1206141335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f8c070620 msgr2=0x7f5f8c0cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:23.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.863+0000 7f5f92908700 1 --2- 172.21.15.40:0/1206141335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f8c070620 0x7f5f8c0cf620 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c012000 tx=0x7f5f7c004770 comp rx=0 tx=0).stop 2023-11-08T08:22:23.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.863+0000 7f5f92908700 1 -- 172.21.15.40:0/1206141335 shutdown_connections 2023-11-08T08:22:23.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.863+0000 7f5f92908700 1 --2- 172.21.15.40:0/1206141335 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5f70064ee0 0x7f5f70067390 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:23.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.863+0000 7f5f92908700 1 --2- 172.21.15.40:0/1206141335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f8c15c150 0x7f5f8c150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:23.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.863+0000 7f5f92908700 1 --2- 172.21.15.40:0/1206141335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f8c070620 0x7f5f8c0cf620 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:23.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.863+0000 7f5f92908700 1 --2- 172.21.15.40:0/1206141335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f8c069bf0 0x7f5f8c1546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:23.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.863+0000 7f5f92908700 1 -- 172.21.15.40:0/1206141335 >> 172.21.15.40:0/1206141335 conn(0x7f5f8c0d3680 msgr2=0x7f5f8c06ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:23.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.863+0000 7f5f92908700 1 -- 172.21.15.40:0/1206141335 shutdown_connections 2023-11-08T08:22:23.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:23.863+0000 7f5f92908700 1 -- 172.21.15.40:0/1206141335 wait complete. 2023-11-08T08:22:23.892 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T08:22:23.893 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:22:24.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.067+0000 7fe1a962f700 1 Processor -- start 2023-11-08T08:22:24.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.067+0000 7fe1a962f700 1 -- start start 2023-11-08T08:22:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.067+0000 7fe1a962f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe1a415c650 0x7fe1a406b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.067+0000 7fe1a962f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1a40c9d40 con 0x7fe1a40cd0d0 2023-11-08T08:22:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a962f700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1a406b820 con 0x7fe1a406bed0 2023-11-08T08:22:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a962f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a406b140 con 0x7fe1a415c650 2023-11-08T08:22:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe19b7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe1a40cd0d0 0x7fe1a40b1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48988/0 (socket says 172.21.15.40:48988) 2023-11-08T08:22:24.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe19b7fe700 1 -- 172.21.15.40:0/2515633389 learned_addr learned my addr 172.21.15.40:0/2515633389 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:24.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a3fff700 1 --2- 172.21.15.40:0/2515633389 >> v2:172.21.15.40:3300/0 conn(0x7fe1a415c650 0x7fe1a406b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:24.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a37fe700 1 -- 172.21.15.40:0/2515633389 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4021002556 0 0) 0x7fe1a40c9d40 con 0x7fe1a40cd0d0 2023-11-08T08:22:24.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a37fe700 1 -- 172.21.15.40:0/2515633389 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1900035d0 con 0x7fe1a40cd0d0 2023-11-08T08:22:24.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a37fe700 1 -- 172.21.15.40:0/2515633389 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3265491748 0 0) 0x7fe1a406b820 con 0x7fe1a406bed0 2023-11-08T08:22:24.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a37fe700 1 -- 172.21.15.40:0/2515633389 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1a40c9d40 con 0x7fe1a406bed0 2023-11-08T08:22:24.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a3fff700 1 -- 172.21.15.40:0/2515633389 >> v1:172.21.15.169:6789/0 conn(0x7fe1a406bed0 legacy=0x7fe1a406c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:24.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a3fff700 1 -- 172.21.15.40:0/2515633389 >> v1:172.21.15.40:6789/0 conn(0x7fe1a40cd0d0 legacy=0x7fe1a40b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:24.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a3fff700 1 -- 172.21.15.40:0/2515633389 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a40b1d50 con 0x7fe1a415c650 2023-11-08T08:22:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a3fff700 1 --2- 172.21.15.40:0/2515633389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1a415c650 0x7fe1a406b2e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe194009d90 tx=0x7fe19400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4905d12b1758bb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a37fe700 1 -- 172.21.15.40:0/2515633389 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19400e070 con 0x7fe1a415c650 2023-11-08T08:22:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a37fe700 1 -- 172.21.15.40:0/2515633389 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19400bb20 con 0x7fe1a415c650 2023-11-08T08:22:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a37fe700 1 -- 172.21.15.40:0/2515633389 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194012430 con 0x7fe1a415c650 2023-11-08T08:22:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a962f700 1 -- 172.21.15.40:0/2515633389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1a415c650 msgr2=0x7fe1a406b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a962f700 1 --2- 172.21.15.40:0/2515633389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1a415c650 0x7fe1a406b2e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe194009d90 tx=0x7fe19400b330 comp rx=0 tx=0).stop 2023-11-08T08:22:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a962f700 1 -- 172.21.15.40:0/2515633389 shutdown_connections 2023-11-08T08:22:24.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a962f700 1 --2- 172.21.15.40:0/2515633389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1a415c650 0x7fe1a406b2e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a962f700 1 -- 172.21.15.40:0/2515633389 >> 172.21.15.40:0/2515633389 conn(0x7fe1a40b0f80 msgr2=0x7fe1a40ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:24.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a962f700 1 -- 172.21.15.40:0/2515633389 shutdown_connections 2023-11-08T08:22:24.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.071+0000 7fe1a962f700 1 -- 172.21.15.40:0/2515633389 wait complete. 2023-11-08T08:22:24.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.075+0000 7fe1a962f700 1 Processor -- start 2023-11-08T08:22:24.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.075+0000 7fe1a962f700 1 -- start start 2023-11-08T08:22:24.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.075+0000 7fe1a962f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a406bed0 0x7fe1a415a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:24.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.075+0000 7fe1a962f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1a40cd0d0 0x7fe1a40c3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:24.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.075+0000 7fe1a962f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a415c650 0x7fe1a41ecdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:24.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.075+0000 7fe1a962f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a40c6c10 con 0x7fe1a40cd0d0 2023-11-08T08:22:24.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.075+0000 7fe1a962f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a40c6ef0 con 0x7fe1a406bed0 2023-11-08T08:22:24.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.075+0000 7fe1a962f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a40c71d0 con 0x7fe1a415c650 2023-11-08T08:22:24.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.079+0000 7fe19b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1a40cd0d0 0x7fe1a40c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:24.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.079+0000 7fe19b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1a40cd0d0 0x7fe1a40c3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35856/0 (socket says 172.21.15.40:35856) 2023-11-08T08:22:24.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.079+0000 7fe19b7fe700 1 -- 172.21.15.40:0/805490586 learned_addr learned my addr 172.21.15.40:0/805490586 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:24.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.079+0000 7fe1a3fff700 1 --2- 172.21.15.40:0/805490586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a406bed0 0x7fe1a415a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:24.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.079+0000 7fe1a8e2e700 1 --2- 172.21.15.40:0/805490586 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a415c650 0x7fe1a41ecdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:24.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.079+0000 7fe19b7fe700 1 -- 172.21.15.40:0/805490586 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a415c650 msgr2=0x7fe1a41ecdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:24.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.079+0000 7fe19b7fe700 1 --2- 172.21.15.40:0/805490586 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a415c650 0x7fe1a41ecdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.079+0000 7fe19b7fe700 1 -- 172.21.15.40:0/805490586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a406bed0 msgr2=0x7fe1a415a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:24.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.079+0000 7fe19b7fe700 1 --2- 172.21.15.40:0/805490586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a406bed0 0x7fe1a415a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.079+0000 7fe19b7fe700 1 -- 172.21.15.40:0/805490586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe194009a70 con 0x7fe1a40cd0d0 2023-11-08T08:22:24.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.079+0000 7fe1a3fff700 1 --2- 172.21.15.40:0/805490586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a406bed0 0x7fe1a415a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:22:24.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.079+0000 7fe1a8e2e700 1 --2- 172.21.15.40:0/805490586 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a415c650 0x7fe1a41ecdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:22:24.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.079+0000 7fe19b7fe700 1 --2- 172.21.15.40:0/805490586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1a40cd0d0 0x7fe1a40c3540 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00c580 tx=0x7fe18c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:24.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.079+0000 7fe1a17fa700 1 -- 172.21.15.40:0/805490586 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c005d90 con 0x7fe1a40cd0d0 2023-11-08T08:22:24.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.079+0000 7fe1a17fa700 1 -- 172.21.15.40:0/805490586 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18c0070d0 con 0x7fe1a40cd0d0 2023-11-08T08:22:24.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.079+0000 7fe1a962f700 1 -- 172.21.15.40:0/805490586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a40cd8b0 con 0x7fe1a40cd0d0 2023-11-08T08:22:24.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.079+0000 7fe1a17fa700 1 -- 172.21.15.40:0/805490586 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c007440 con 0x7fe1a40cd0d0 2023-11-08T08:22:24.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.079+0000 7fe1a962f700 1 -- 172.21.15.40:0/805490586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a40cd5d0 con 0x7fe1a40cd0d0 2023-11-08T08:22:24.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.083+0000 7fe1a962f700 1 -- 172.21.15.40:0/805490586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1a40c99d0 con 0x7fe1a40cd0d0 2023-11-08T08:22:24.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.083+0000 7fe1a17fa700 1 -- 172.21.15.40:0/805490586 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fe18c019030 con 0x7fe1a40cd0d0 2023-11-08T08:22:24.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.083+0000 7fe1a17fa700 1 --2- 172.21.15.40:0/805490586 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe184064ee0 0x7fe184067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:24.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.083+0000 7fe1a17fa700 1 -- 172.21.15.40:0/805490586 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe18c081440 con 0x7fe1a40cd0d0 2023-11-08T08:22:24.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.087+0000 7fe1a3fff700 1 --2- 172.21.15.40:0/805490586 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe184064ee0 0x7fe184067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:24.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.087+0000 7fe1a3fff700 1 --2- 172.21.15.40:0/805490586 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe184064ee0 0x7fe184067390 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe194014040 tx=0x7fe19401d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:24.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.091+0000 7fe1a17fa700 1 -- 172.21.15.40:0/805490586 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fe18c04a400 con 0x7fe1a40cd0d0 2023-11-08T08:22:24.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.295+0000 7fe1a962f700 1 -- 172.21.15.40:0/805490586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1a4073cb0 con 0x7fe1a40cd0d0 2023-11-08T08:22:24.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.295+0000 7fe1a17fa700 1 -- 172.21.15.40:0/805490586 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4650 (secure 0 0 0) 0x7fe18c04f460 con 0x7fe1a40cd0d0 2023-11-08T08:22:24.300 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:22:24.300 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/307473250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":307473250},{"type":"v1","addr":"172.21.15.169:6835","nonce":307473250}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6148,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/924518366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":924518366},{"type":"v1","addr":"172.21.15.169:6837","nonce":924518366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4294005934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4294005934},{"type":"v1","addr":"172.21.15.40:6833","nonce":4294005934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:22:20.650687+0000","modified":"2023-11-08T08:22:23.857369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6237":{"gid":6237,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3395630689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3395630689},{"type":"v1","addr":"172.21.15.40:6835","nonce":3395630689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T08:22:24.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.299+0000 7fe1a962f700 1 -- 172.21.15.40:0/805490586 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe184064ee0 msgr2=0x7fe184067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:24.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.299+0000 7fe1a962f700 1 --2- 172.21.15.40:0/805490586 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe184064ee0 0x7fe184067390 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe194014040 tx=0x7fe19401d040 comp rx=0 tx=0).stop 2023-11-08T08:22:24.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.299+0000 7fe1a962f700 1 -- 172.21.15.40:0/805490586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1a40cd0d0 msgr2=0x7fe1a40c3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:24.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.299+0000 7fe1a962f700 1 --2- 172.21.15.40:0/805490586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1a40cd0d0 0x7fe1a40c3540 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00c580 tx=0x7fe18c00c940 comp rx=0 tx=0).stop 2023-11-08T08:22:24.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.299+0000 7fe1a962f700 1 -- 172.21.15.40:0/805490586 shutdown_connections 2023-11-08T08:22:24.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.299+0000 7fe1a962f700 1 --2- 172.21.15.40:0/805490586 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe184064ee0 0x7fe184067390 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.299+0000 7fe1a962f700 1 --2- 172.21.15.40:0/805490586 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a415c650 0x7fe1a41ecdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.299+0000 7fe1a962f700 1 --2- 172.21.15.40:0/805490586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe1a40cd0d0 0x7fe1a40c3540 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.299+0000 7fe1a962f700 1 --2- 172.21.15.40:0/805490586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a406bed0 0x7fe1a415a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.299+0000 7fe1a962f700 1 -- 172.21.15.40:0/805490586 >> 172.21.15.40:0/805490586 conn(0x7fe1a40b0f80 msgr2=0x7fe1a40b3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:24.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.299+0000 7fe1a962f700 1 -- 172.21.15.40:0/805490586 shutdown_connections 2023-11-08T08:22:24.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.299+0000 7fe1a962f700 1 -- 172.21.15.40:0/805490586 wait complete. 2023-11-08T08:22:24.307 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 33 2023-11-08T08:22:24.326 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:22:24.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.507+0000 7f98e891d700 1 Processor -- start 2023-11-08T08:22:24.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.507+0000 7f98e891d700 1 -- start start 2023-11-08T08:22:24.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.507+0000 7f98e891d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98e4160710 0x7f98e40cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:24.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.507+0000 7f98e891d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98e4070790 0x7f98e4164250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:24.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.507+0000 7f98e891d700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e40ad6c0 con 0x7f98e40cc860 2023-11-08T08:22:24.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.507+0000 7f98e891d700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e406b1c0 con 0x7f98e4160710 2023-11-08T08:22:24.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.507+0000 7f98e891d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e40adc60 con 0x7f98e4070790 2023-11-08T08:22:24.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.511+0000 7f98dbfff700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f98e40cc860 0x7f98e4070080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:38870/0 (socket says 172.21.15.40:38870) 2023-11-08T08:22:24.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.511+0000 7f98dbfff700 1 -- 172.21.15.40:0/1141174449 learned_addr learned my addr 172.21.15.40:0/1141174449 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:24.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.511+0000 7f98e37fe700 1 --2- 172.21.15.40:0/1141174449 >> v2:172.21.15.40:3300/0 conn(0x7f98e4160710 0x7f98e40cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:24.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.511+0000 7f98e3fff700 1 --2- 172.21.15.40:0/1141174449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98e4070790 0x7f98e4164250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:24.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.511+0000 7f98e2ffd700 1 -- 172.21.15.40:0/1141174449 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546743353 0 0) 0x7f98e40ad6c0 con 0x7f98e40cc860 2023-11-08T08:22:24.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.511+0000 7f98e2ffd700 1 -- 172.21.15.40:0/1141174449 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98d00035d0 con 0x7f98e40cc860 2023-11-08T08:22:24.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.511+0000 7f98e37fe700 1 -- 172.21.15.40:0/1141174449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98e4070790 msgr2=0x7f98e4164250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:24.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.511+0000 7f98e37fe700 1 --2- 172.21.15.40:0/1141174449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98e4070790 0x7f98e4164250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.511+0000 7f98e37fe700 1 -- 172.21.15.40:0/1141174449 >> v1:172.21.15.169:6789/0 conn(0x7f98e40cc860 legacy=0x7f98e4070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:24.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.511+0000 7f98e37fe700 1 -- 172.21.15.40:0/1141174449 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e406b3d0 con 0x7f98e4160710 2023-11-08T08:22:24.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.511+0000 7f98e37fe700 1 --2- 172.21.15.40:0/1141174449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98e4160710 0x7f98e40cc320 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009850 tx=0x7f98d4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facaa158af27021d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:24.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.511+0000 7f98e2ffd700 1 -- 172.21.15.40:0/1141174449 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4011830 con 0x7f98e4160710 2023-11-08T08:22:24.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.511+0000 7f98e2ffd700 1 -- 172.21.15.40:0/1141174449 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d4011990 con 0x7f98e4160710 2023-11-08T08:22:24.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.511+0000 7f98e2ffd700 1 -- 172.21.15.40:0/1141174449 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4011d00 con 0x7f98e4160710 2023-11-08T08:22:24.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.511+0000 7f98e891d700 1 -- 172.21.15.40:0/1141174449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98e4160710 msgr2=0x7f98e40cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:24.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.511+0000 7f98e891d700 1 --2- 172.21.15.40:0/1141174449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98e4160710 0x7f98e40cc320 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009850 tx=0x7f98d4009b60 comp rx=0 tx=0).stop 2023-11-08T08:22:24.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.511+0000 7f98e891d700 1 -- 172.21.15.40:0/1141174449 shutdown_connections 2023-11-08T08:22:24.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.511+0000 7f98e891d700 1 --2- 172.21.15.40:0/1141174449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98e4070790 0x7f98e4164250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.511+0000 7f98e891d700 1 --2- 172.21.15.40:0/1141174449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98e4160710 0x7f98e40cc320 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.511+0000 7f98e891d700 1 -- 172.21.15.40:0/1141174449 >> 172.21.15.40:0/1141174449 conn(0x7f98e40b0f80 msgr2=0x7f98e40ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:24.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.515+0000 7f98e891d700 1 -- 172.21.15.40:0/1141174449 shutdown_connections 2023-11-08T08:22:24.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.515+0000 7f98e891d700 1 -- 172.21.15.40:0/1141174449 wait complete. 2023-11-08T08:22:24.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.515+0000 7f98e891d700 1 Processor -- start 2023-11-08T08:22:24.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e891d700 1 -- start start 2023-11-08T08:22:24.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e891d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98e4070790 0x7f98e4159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:24.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e891d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e40cc860 0x7f98e415f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:24.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e891d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98e4160710 0x7f98e415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:24.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e891d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e40b2ea0 con 0x7f98e4070790 2023-11-08T08:22:24.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e891d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e415a250 con 0x7f98e40cc860 2023-11-08T08:22:24.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e891d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e415da90 con 0x7f98e4160710 2023-11-08T08:22:24.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98e4070790 0x7f98e4159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:24.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98e4070790 0x7f98e4159d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35866/0 (socket says 172.21.15.40:35866) 2023-11-08T08:22:24.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e37fe700 1 -- 172.21.15.40:0/2730336533 learned_addr learned my addr 172.21.15.40:0/2730336533 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:24.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98dbfff700 1 --2- 172.21.15.40:0/2730336533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e40cc860 0x7f98e415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:24.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e3fff700 1 --2- 172.21.15.40:0/2730336533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98e4160710 0x7f98e415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:24.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e37fe700 1 -- 172.21.15.40:0/2730336533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98e4160710 msgr2=0x7f98e415a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:24.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e37fe700 1 --2- 172.21.15.40:0/2730336533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98e4160710 0x7f98e415a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e37fe700 1 -- 172.21.15.40:0/2730336533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e40cc860 msgr2=0x7f98e415f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:24.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e37fe700 1 --2- 172.21.15.40:0/2730336533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e40cc860 0x7f98e415f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e37fe700 1 -- 172.21.15.40:0/2730336533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d40094b0 con 0x7f98e4070790 2023-11-08T08:22:24.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e37fe700 1 --2- 172.21.15.40:0/2730336533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98e4070790 0x7f98e4159d10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f98d40090b0 tx=0x7f98d4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:24.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e0ff9700 1 -- 172.21.15.40:0/2730336533 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4004550 con 0x7f98e4070790 2023-11-08T08:22:24.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e891d700 1 -- 172.21.15.40:0/2730336533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e40ad8f0 con 0x7f98e4070790 2023-11-08T08:22:24.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e0ff9700 1 -- 172.21.15.40:0/2730336533 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d401e070 con 0x7f98e4070790 2023-11-08T08:22:24.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.519+0000 7f98e0ff9700 1 -- 172.21.15.40:0/2730336533 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4003900 con 0x7f98e4070790 2023-11-08T08:22:24.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.523+0000 7f98e891d700 1 -- 172.21.15.40:0/2730336533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e406b6b0 con 0x7f98e4070790 2023-11-08T08:22:24.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.523+0000 7f98e891d700 1 -- 172.21.15.40:0/2730336533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e4160ae0 con 0x7f98e4070790 2023-11-08T08:22:24.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.523+0000 7f98e0ff9700 1 -- 172.21.15.40:0/2730336533 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f98d400f050 con 0x7f98e4070790 2023-11-08T08:22:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.527+0000 7f98e0ff9700 1 --2- 172.21.15.40:0/2730336533 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98c4064ee0 0x7f98c4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:24.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.527+0000 7f98e0ff9700 1 -- 172.21.15.40:0/2730336533 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f98d4082ff0 con 0x7f98e4070790 2023-11-08T08:22:24.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.527+0000 7f98dbfff700 1 --2- 172.21.15.40:0/2730336533 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98c4064ee0 0x7f98c4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:24.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.527+0000 7f98dbfff700 1 --2- 172.21.15.40:0/2730336533 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98c4064ee0 0x7f98c4067390 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f98e40b2a90 tx=0x7f98cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:24.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.531+0000 7f98e0ff9700 1 -- 172.21.15.40:0/2730336533 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f98d40b3400 con 0x7f98e4070790 2023-11-08T08:22:24.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.707+0000 7f98e891d700 1 -- 172.21.15.40:0/2730336533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98e40b2d00 con 0x7f98e4070790 2023-11-08T08:22:24.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.711+0000 7f98e0ff9700 1 -- 172.21.15.40:0/2730336533 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+18087 (secure 0 0 0) 0x7f98e40b2d00 con 0x7f98e4070790 2023-11-08T08:22:24.716 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:22:24.716 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":69,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:22:21.656911+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:22:18.224463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T08:22:19.658954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:22:24.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.715+0000 7f98e891d700 1 -- 172.21.15.40:0/2730336533 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98c4064ee0 msgr2=0x7f98c4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:24.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.715+0000 7f98e891d700 1 --2- 172.21.15.40:0/2730336533 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98c4064ee0 0x7f98c4067390 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f98e40b2a90 tx=0x7f98cc009040 comp rx=0 tx=0).stop 2023-11-08T08:22:24.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.715+0000 7f98e891d700 1 -- 172.21.15.40:0/2730336533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98e4070790 msgr2=0x7f98e4159d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:24.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.715+0000 7f98e891d700 1 --2- 172.21.15.40:0/2730336533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98e4070790 0x7f98e4159d10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f98d40090b0 tx=0x7f98d4004770 comp rx=0 tx=0).stop 2023-11-08T08:22:24.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.715+0000 7f98e891d700 1 -- 172.21.15.40:0/2730336533 shutdown_connections 2023-11-08T08:22:24.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.715+0000 7f98e891d700 1 --2- 172.21.15.40:0/2730336533 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f98c4064ee0 0x7f98c4067390 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.715+0000 7f98e891d700 1 --2- 172.21.15.40:0/2730336533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98e4160710 0x7f98e415a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.715+0000 7f98e891d700 1 --2- 172.21.15.40:0/2730336533 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98e40cc860 0x7f98e415f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.715+0000 7f98e891d700 1 --2- 172.21.15.40:0/2730336533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98e4070790 0x7f98e4159d10 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.715+0000 7f98e891d700 1 -- 172.21.15.40:0/2730336533 >> 172.21.15.40:0/2730336533 conn(0x7f98e40b0f80 msgr2=0x7f98e406cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:24.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.715+0000 7f98e891d700 1 -- 172.21.15.40:0/2730336533 shutdown_connections 2023-11-08T08:22:24.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.715+0000 7f98e891d700 1 -- 172.21.15.40:0/2730336533 wait complete. 2023-11-08T08:22:24.741 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T08:22:24.742 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T08:22:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.915+0000 7fae98337700 1 Processor -- start 2023-11-08T08:22:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.915+0000 7fae98337700 1 -- start start 2023-11-08T08:22:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.915+0000 7fae98337700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae90163cb0 0x7fae90164cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.915+0000 7fae98337700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fae900cdfb0 0x7fae9006cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.915+0000 7fae98337700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fae9006d2f0 0x7fae9006bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.915+0000 7fae98337700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae900cc290 con 0x7fae9006d2f0 2023-11-08T08:22:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.915+0000 7fae98337700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae9006d770 con 0x7fae900cdfb0 2023-11-08T08:22:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.915+0000 7fae98337700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae90164080 con 0x7fae90163cb0 2023-11-08T08:22:24.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.915+0000 7fae968d4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fae9006d2f0 0x7fae9006bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:24.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.915+0000 7fae968d4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fae9006d2f0 0x7fae9006bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35882/0 (socket says 172.21.15.40:35882) 2023-11-08T08:22:24.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.915+0000 7fae968d4700 1 -- 172.21.15.40:0/2725810821 learned_addr learned my addr 172.21.15.40:0/2725810821 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:24.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.919+0000 7fae958d2700 1 --2- 172.21.15.40:0/2725810821 >> v2:172.21.15.169:3300/0 conn(0x7fae900cdfb0 0x7fae9006cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:24.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.919+0000 7fae960d3700 1 --2- 172.21.15.40:0/2725810821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae90163cb0 0x7fae90164cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:24.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.919+0000 7fae968d4700 1 -- 172.21.15.40:0/2725810821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae90163cb0 msgr2=0x7fae90164cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:24.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.919+0000 7fae968d4700 1 --2- 172.21.15.40:0/2725810821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae90163cb0 0x7fae90164cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.919+0000 7fae968d4700 1 -- 172.21.15.40:0/2725810821 >> v2:172.21.15.169:3300/0 conn(0x7fae900cdfb0 msgr2=0x7fae9006cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:24.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.919+0000 7fae968d4700 1 --2- 172.21.15.40:0/2725810821 >> v2:172.21.15.169:3300/0 conn(0x7fae900cdfb0 0x7fae9006cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.919+0000 7fae968d4700 1 -- 172.21.15.40:0/2725810821 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae90069bf0 con 0x7fae9006d2f0 2023-11-08T08:22:24.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.919+0000 7fae968d4700 1 --2- 172.21.15.40:0/2725810821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae9006d2f0 0x7fae9006bb00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fae9006b970 tx=0x7fae8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fcd57bf2568e36 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:24.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.919+0000 7fae950d1700 1 -- 172.21.15.40:0/2725810821 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8800e070 con 0x7fae9006d2f0 2023-11-08T08:22:24.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.919+0000 7fae950d1700 1 -- 172.21.15.40:0/2725810821 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8800b910 con 0x7fae9006d2f0 2023-11-08T08:22:24.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.919+0000 7fae950d1700 1 -- 172.21.15.40:0/2725810821 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8800bcc0 con 0x7fae9006d2f0 2023-11-08T08:22:24.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.919+0000 7fae98337700 1 -- 172.21.15.40:0/2725810821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae9006d2f0 msgr2=0x7fae9006bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:24.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.919+0000 7fae98337700 1 --2- 172.21.15.40:0/2725810821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae9006d2f0 0x7fae9006bb00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fae9006b970 tx=0x7fae8800b330 comp rx=0 tx=0).stop 2023-11-08T08:22:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.919+0000 7fae98337700 1 -- 172.21.15.40:0/2725810821 shutdown_connections 2023-11-08T08:22:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.919+0000 7fae98337700 1 --2- 172.21.15.40:0/2725810821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae9006d2f0 0x7fae9006bb00 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.919+0000 7fae98337700 1 --2- 172.21.15.40:0/2725810821 >> v2:172.21.15.169:3300/0 conn(0x7fae900cdfb0 0x7fae9006cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.919+0000 7fae98337700 1 --2- 172.21.15.40:0/2725810821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae90163cb0 0x7fae90164cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.919+0000 7fae98337700 1 -- 172.21.15.40:0/2725810821 >> 172.21.15.40:0/2725810821 conn(0x7fae900cf350 msgr2=0x7fae900b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:24.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.919+0000 7fae98337700 1 -- 172.21.15.40:0/2725810821 shutdown_connections 2023-11-08T08:22:24.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.919+0000 7fae98337700 1 -- 172.21.15.40:0/2725810821 wait complete. 2023-11-08T08:22:24.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.923+0000 7fae98337700 1 Processor -- start 2023-11-08T08:22:24.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.923+0000 7fae98337700 1 -- start start 2023-11-08T08:22:24.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae98337700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae9006d2f0 0x7fae900c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:24.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae98337700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae900cdfb0 0x7fae900c1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:24.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae98337700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae90163cb0 0x7fae900c3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:24.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae98337700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae900c2470 con 0x7fae90163cb0 2023-11-08T08:22:24.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae98337700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae900c3e60 con 0x7fae9006d2f0 2023-11-08T08:22:24.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae98337700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae900c4090 con 0x7fae900cdfb0 2023-11-08T08:22:24.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae968d4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae90163cb0 0x7fae900c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:24.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae968d4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae90163cb0 0x7fae900c3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35888/0 (socket says 172.21.15.40:35888) 2023-11-08T08:22:24.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae968d4700 1 -- 172.21.15.40:0/2906888801 learned_addr learned my addr 172.21.15.40:0/2906888801 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:24.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae960d3700 1 --2- 172.21.15.40:0/2906888801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae9006d2f0 0x7fae900c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:24.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae958d2700 1 --2- 172.21.15.40:0/2906888801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae900cdfb0 0x7fae900c1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:24.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae968d4700 1 -- 172.21.15.40:0/2906888801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae900cdfb0 msgr2=0x7fae900c1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:24.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae968d4700 1 --2- 172.21.15.40:0/2906888801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae900cdfb0 0x7fae900c1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae968d4700 1 -- 172.21.15.40:0/2906888801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae9006d2f0 msgr2=0x7fae900c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:24.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae968d4700 1 --2- 172.21.15.40:0/2906888801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae9006d2f0 0x7fae900c5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:24.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae968d4700 1 -- 172.21.15.40:0/2906888801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae88009b30 con 0x7fae90163cb0 2023-11-08T08:22:24.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae968d4700 1 --2- 172.21.15.40:0/2906888801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae90163cb0 0x7fae900c3920 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fae9006f6f0 tx=0x7fae88004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:24.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae86ffd700 1 -- 172.21.15.40:0/2906888801 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8800ce80 con 0x7fae90163cb0 2023-11-08T08:22:24.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae86ffd700 1 -- 172.21.15.40:0/2906888801 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae88004250 con 0x7fae90163cb0 2023-11-08T08:22:24.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae98337700 1 -- 172.21.15.40:0/2906888801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae900cc4a0 con 0x7fae90163cb0 2023-11-08T08:22:24.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae86ffd700 1 -- 172.21.15.40:0/2906888801 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88004600 con 0x7fae90163cb0 2023-11-08T08:22:24.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.927+0000 7fae98337700 1 -- 172.21.15.40:0/2906888801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae901651f0 con 0x7fae90163cb0 2023-11-08T08:22:24.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.931+0000 7fae98337700 1 -- 172.21.15.40:0/2906888801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae900cc780 con 0x7fae90163cb0 2023-11-08T08:22:24.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.931+0000 7fae86ffd700 1 -- 172.21.15.40:0/2906888801 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fae8802b070 con 0x7fae90163cb0 2023-11-08T08:22:24.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.931+0000 7fae86ffd700 1 --2- 172.21.15.40:0/2906888801 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fae74064ee0 0x7fae74067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:24.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.931+0000 7fae86ffd700 1 -- 172.21.15.40:0/2906888801 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fae880833f0 con 0x7fae90163cb0 2023-11-08T08:22:24.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.935+0000 7fae960d3700 1 --2- 172.21.15.40:0/2906888801 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fae74064ee0 0x7fae74067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:24.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.935+0000 7fae960d3700 1 --2- 172.21.15.40:0/2906888801 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fae74064ee0 0x7fae74067390 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009b00 tx=0x7fae8c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:24.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:24.939+0000 7fae86ffd700 1 -- 172.21.15.40:0/2906888801 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fae880512e0 con 0x7fae90163cb0 2023-11-08T08:22:25.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.107+0000 7fae98337700 1 -- 172.21.15.40:0/2906888801 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fae900726c0 con 0x7fae74064ee0 2023-11-08T08:22:25.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.127+0000 7fae86ffd700 1 -- 172.21.15.40:0/2906888801 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309194 (secure 0 0 0) 0x7fae900726c0 con 0x7fae74064ee0 2023-11-08T08:22:25.137 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:22:25.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.135+0000 7fae98337700 1 -- 172.21.15.40:0/2906888801 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fae74064ee0 msgr2=0x7fae74067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:25.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.135+0000 7fae98337700 1 --2- 172.21.15.40:0/2906888801 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fae74064ee0 0x7fae74067390 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009b00 tx=0x7fae8c008040 comp rx=0 tx=0).stop 2023-11-08T08:22:25.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.135+0000 7fae98337700 1 -- 172.21.15.40:0/2906888801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae90163cb0 msgr2=0x7fae900c3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:25.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.135+0000 7fae98337700 1 --2- 172.21.15.40:0/2906888801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae90163cb0 0x7fae900c3920 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fae9006f6f0 tx=0x7fae88004af0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.135+0000 7fae98337700 1 -- 172.21.15.40:0/2906888801 shutdown_connections 2023-11-08T08:22:25.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.135+0000 7fae98337700 1 --2- 172.21.15.40:0/2906888801 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fae74064ee0 0x7fae74067390 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.135+0000 7fae98337700 1 --2- 172.21.15.40:0/2906888801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae90163cb0 0x7fae900c3920 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.135+0000 7fae98337700 1 --2- 172.21.15.40:0/2906888801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae900cdfb0 0x7fae900c1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.135+0000 7fae98337700 1 --2- 172.21.15.40:0/2906888801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae9006d2f0 0x7fae900c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.135+0000 7fae98337700 1 -- 172.21.15.40:0/2906888801 >> 172.21.15.40:0/2906888801 conn(0x7fae900cf350 msgr2=0x7fae90162ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:25.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.135+0000 7fae98337700 1 -- 172.21.15.40:0/2906888801 shutdown_connections 2023-11-08T08:22:25.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.135+0000 7fae98337700 1 -- 172.21.15.40:0/2906888801 wait complete. 2023-11-08T08:22:25.143 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-11-08T08:22:25.143 INFO:tasks.ceph.mds.c.smithi040.stderr:2023-11-08T08:22:25.139+0000 7f32d2f42700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T08:22:25.159 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2023-11-08T08:22:23.955960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":88104,"kb_used_data":7528,"kb_used_omap":0,"kb_used_meta":80512,"kb_avail":754886616,"statfs":{"total":773094113280,"available":773003894784,"internally_reserved":0,"allocated":7708672,"data_stored":4197368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":82444288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":52,"apply_latency_ms":52,"commit_latency_ns":52000000,"apply_latency_ns":52000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.788665"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.699681+0000","last_change":"2023-11-08T08:22:20.693079+0000","last_active":"2023-11-08T08:22:20.699681+0000","last_peered":"2023-11-08T08:22:20.699681+0000","last_clean":"2023-11-08T08:22:20.699681+0000","last_became_active":"2023-11-08T08:22:20.692841+0000","last_became_peered":"2023-11-08T08:22:20.692841+0000","last_unstale":"2023-11-08T08:22:20.699681+0000","last_undegraded":"2023-11-08T08:22:20.699681+0000","last_fullsized":"2023-11-08T08:22:20.699681+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:35:24.924456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703941+0000","last_change":"2023-11-08T08:22:20.695194+0000","last_active":"2023-11-08T08:22:20.703941+0000","last_peered":"2023-11-08T08:22:20.703941+0000","last_clean":"2023-11-08T08:22:20.703941+0000","last_became_active":"2023-11-08T08:22:20.694528+0000","last_became_peered":"2023-11-08T08:22:20.694528+0000","last_unstale":"2023-11-08T08:22:20.703941+0000","last_undegraded":"2023-11-08T08:22:20.703941+0000","last_fullsized":"2023-11-08T08:22:20.703941+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:16:07.614218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.701955+0000","last_change":"2023-11-08T08:22:20.688375+0000","last_active":"2023-11-08T08:22:20.701955+0000","last_peered":"2023-11-08T08:22:20.701955+0000","last_clean":"2023-11-08T08:22:20.701955+0000","last_became_active":"2023-11-08T08:22:20.687746+0000","last_became_peered":"2023-11-08T08:22:20.687746+0000","last_unstale":"2023-11-08T08:22:20.701955+0000","last_undegraded":"2023-11-08T08:22:20.701955+0000","last_fullsized":"2023-11-08T08:22:20.701955+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:29:48.111099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.699690+0000","last_change":"2023-11-08T08:22:20.695073+0000","last_active":"2023-11-08T08:22:20.699690+0000","last_peered":"2023-11-08T08:22:20.699690+0000","last_clean":"2023-11-08T08:22:20.699690+0000","last_became_active":"2023-11-08T08:22:20.694827+0000","last_became_peered":"2023-11-08T08:22:20.694827+0000","last_unstale":"2023-11-08T08:22:20.699690+0000","last_undegraded":"2023-11-08T08:22:20.699690+0000","last_fullsized":"2023-11-08T08:22:20.699690+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:00:18.591720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.695879+0000","last_change":"2023-11-08T08:22:20.684638+0000","last_active":"2023-11-08T08:22:20.695879+0000","last_peered":"2023-11-08T08:22:20.695879+0000","last_clean":"2023-11-08T08:22:20.695879+0000","last_became_active":"2023-11-08T08:22:20.684115+0000","last_became_peered":"2023-11-08T08:22:20.684115+0000","last_unstale":"2023-11-08T08:22:20.695879+0000","last_undegraded":"2023-11-08T08:22:20.695879+0000","last_fullsized":"2023-11-08T08:22:20.695879+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:20:55.507408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.702788+0000","last_change":"2023-11-08T08:22:20.688220+0000","last_active":"2023-11-08T08:22:20.702788+0000","last_peered":"2023-11-08T08:22:20.702788+0000","last_clean":"2023-11-08T08:22:20.702788+0000","last_became_active":"2023-11-08T08:22:20.687910+0000","last_became_peered":"2023-11-08T08:22:20.687910+0000","last_unstale":"2023-11-08T08:22:20.702788+0000","last_undegraded":"2023-11-08T08:22:20.702788+0000","last_fullsized":"2023-11-08T08:22:20.702788+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:40:54.193871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.700373+0000","last_change":"2023-11-08T08:22:20.694001+0000","last_active":"2023-11-08T08:22:20.700373+0000","last_peered":"2023-11-08T08:22:20.700373+0000","last_clean":"2023-11-08T08:22:20.700373+0000","last_became_active":"2023-11-08T08:22:20.693582+0000","last_became_peered":"2023-11-08T08:22:20.693582+0000","last_unstale":"2023-11-08T08:22:20.700373+0000","last_undegraded":"2023-11-08T08:22:20.700373+0000","last_fullsized":"2023-11-08T08:22:20.700373+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:54:34.009215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.702918+0000","last_change":"2023-11-08T08:22:20.688974+0000","last_active":"2023-11-08T08:22:20.702918+0000","last_peered":"2023-11-08T08:22:20.702918+0000","last_clean":"2023-11-08T08:22:20.702918+0000","last_became_active":"2023-11-08T08:22:20.688735+0000","last_became_peered":"2023-11-08T08:22:20.688735+0000","last_unstale":"2023-11-08T08:22:20.702918+0000","last_undegraded":"2023-11-08T08:22:20.702918+0000","last_fullsized":"2023-11-08T08:22:20.702918+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:06:31.570228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703678+0000","last_change":"2023-11-08T08:22:20.698657+0000","last_active":"2023-11-08T08:22:20.703678+0000","last_peered":"2023-11-08T08:22:20.703678+0000","last_clean":"2023-11-08T08:22:20.703678+0000","last_became_active":"2023-11-08T08:22:20.697916+0000","last_became_peered":"2023-11-08T08:22:20.697916+0000","last_unstale":"2023-11-08T08:22:20.703678+0000","last_undegraded":"2023-11-08T08:22:20.703678+0000","last_fullsized":"2023-11-08T08:22:20.703678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:51.802894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705433+0000","last_change":"2023-11-08T08:22:20.696337+0000","last_active":"2023-11-08T08:22:20.705433+0000","last_peered":"2023-11-08T08:22:20.705433+0000","last_clean":"2023-11-08T08:22:20.705433+0000","last_became_active":"2023-11-08T08:22:20.694555+0000","last_became_peered":"2023-11-08T08:22:20.694555+0000","last_unstale":"2023-11-08T08:22:20.705433+0000","last_undegraded":"2023-11-08T08:22:20.705433+0000","last_fullsized":"2023-11-08T08:22:20.705433+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:55:16.752604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.702650+0000","last_change":"2023-11-08T08:22:20.694924+0000","last_active":"2023-11-08T08:22:20.702650+0000","last_peered":"2023-11-08T08:22:20.702650+0000","last_clean":"2023-11-08T08:22:20.702650+0000","last_became_active":"2023-11-08T08:22:20.692623+0000","last_became_peered":"2023-11-08T08:22:20.692623+0000","last_unstale":"2023-11-08T08:22:20.702650+0000","last_undegraded":"2023-11-08T08:22:20.702650+0000","last_fullsized":"2023-11-08T08:22:20.702650+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:02.368358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.699674+0000","last_change":"2023-11-08T08:22:20.692162+0000","last_active":"2023-11-08T08:22:20.699674+0000","last_peered":"2023-11-08T08:22:20.699674+0000","last_clean":"2023-11-08T08:22:20.699674+0000","last_became_active":"2023-11-08T08:22:20.691560+0000","last_became_peered":"2023-11-08T08:22:20.691560+0000","last_unstale":"2023-11-08T08:22:20.699674+0000","last_undegraded":"2023-11-08T08:22:20.699674+0000","last_fullsized":"2023-11-08T08:22:20.699674+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:30:07.090301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.692248+0000","last_change":"2023-11-08T08:22:20.682225+0000","last_active":"2023-11-08T08:22:20.692248+0000","last_peered":"2023-11-08T08:22:20.692248+0000","last_clean":"2023-11-08T08:22:20.692248+0000","last_became_active":"2023-11-08T08:22:20.681934+0000","last_became_peered":"2023-11-08T08:22:20.681934+0000","last_unstale":"2023-11-08T08:22:20.692248+0000","last_undegraded":"2023-11-08T08:22:20.692248+0000","last_fullsized":"2023-11-08T08:22:20.692248+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:33:21.532456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703810+0000","last_change":"2023-11-08T08:22:20.701356+0000","last_active":"2023-11-08T08:22:20.703810+0000","last_peered":"2023-11-08T08:22:20.703810+0000","last_clean":"2023-11-08T08:22:20.703810+0000","last_became_active":"2023-11-08T08:22:20.701126+0000","last_became_peered":"2023-11-08T08:22:20.701126+0000","last_unstale":"2023-11-08T08:22:20.703810+0000","last_undegraded":"2023-11-08T08:22:20.703810+0000","last_fullsized":"2023-11-08T08:22:20.703810+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:38:00.466999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.692508+0000","last_change":"2023-11-08T08:22:20.685750+0000","last_active":"2023-11-08T08:22:20.692508+0000","last_peered":"2023-11-08T08:22:20.692508+0000","last_clean":"2023-11-08T08:22:20.692508+0000","last_became_active":"2023-11-08T08:22:20.685489+0000","last_became_peered":"2023-11-08T08:22:20.685489+0000","last_unstale":"2023-11-08T08:22:20.692508+0000","last_undegraded":"2023-11-08T08:22:20.692508+0000","last_fullsized":"2023-11-08T08:22:20.692508+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:29:48.111099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.695824+0000","last_change":"2023-11-08T08:22:20.682298+0000","last_active":"2023-11-08T08:22:20.695824+0000","last_peered":"2023-11-08T08:22:20.695824+0000","last_clean":"2023-11-08T08:22:20.695824+0000","last_became_active":"2023-11-08T08:22:20.682033+0000","last_became_peered":"2023-11-08T08:22:20.682033+0000","last_unstale":"2023-11-08T08:22:20.695824+0000","last_undegraded":"2023-11-08T08:22:20.695824+0000","last_fullsized":"2023-11-08T08:22:20.695824+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:42:59.040872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703643+0000","last_change":"2023-11-08T08:22:20.698493+0000","last_active":"2023-11-08T08:22:20.703643+0000","last_peered":"2023-11-08T08:22:20.703643+0000","last_clean":"2023-11-08T08:22:20.703643+0000","last_became_active":"2023-11-08T08:22:20.697582+0000","last_became_peered":"2023-11-08T08:22:20.697582+0000","last_unstale":"2023-11-08T08:22:20.703643+0000","last_undegraded":"2023-11-08T08:22:20.703643+0000","last_fullsized":"2023-11-08T08:22:20.703643+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:05:14.039538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.699885+0000","last_change":"2023-11-08T08:22:20.695572+0000","last_active":"2023-11-08T08:22:20.699885+0000","last_peered":"2023-11-08T08:22:20.699885+0000","last_clean":"2023-11-08T08:22:20.699885+0000","last_became_active":"2023-11-08T08:22:20.695336+0000","last_became_peered":"2023-11-08T08:22:20.695336+0000","last_unstale":"2023-11-08T08:22:20.699885+0000","last_undegraded":"2023-11-08T08:22:20.699885+0000","last_fullsized":"2023-11-08T08:22:20.699885+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:50:59.392279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703243+0000","last_change":"2023-11-08T08:22:20.691681+0000","last_active":"2023-11-08T08:22:20.703243+0000","last_peered":"2023-11-08T08:22:20.703243+0000","last_clean":"2023-11-08T08:22:20.703243+0000","last_became_active":"2023-11-08T08:22:20.691442+0000","last_became_peered":"2023-11-08T08:22:20.691442+0000","last_unstale":"2023-11-08T08:22:20.703243+0000","last_undegraded":"2023-11-08T08:22:20.703243+0000","last_fullsized":"2023-11-08T08:22:20.703243+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:31:36.439012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.701444+0000","last_change":"2023-11-08T08:22:20.697764+0000","last_active":"2023-11-08T08:22:20.701444+0000","last_peered":"2023-11-08T08:22:20.701444+0000","last_clean":"2023-11-08T08:22:20.701444+0000","last_became_active":"2023-11-08T08:22:20.697433+0000","last_became_peered":"2023-11-08T08:22:20.697433+0000","last_unstale":"2023-11-08T08:22:20.701444+0000","last_undegraded":"2023-11-08T08:22:20.701444+0000","last_fullsized":"2023-11-08T08:22:20.701444+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:52:08.554540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705173+0000","last_change":"2023-11-08T08:22:20.692276+0000","last_active":"2023-11-08T08:22:20.705173+0000","last_peered":"2023-11-08T08:22:20.705173+0000","last_clean":"2023-11-08T08:22:20.705173+0000","last_became_active":"2023-11-08T08:22:20.691566+0000","last_became_peered":"2023-11-08T08:22:20.691566+0000","last_unstale":"2023-11-08T08:22:20.705173+0000","last_undegraded":"2023-11-08T08:22:20.705173+0000","last_fullsized":"2023-11-08T08:22:20.705173+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:05.120014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705489+0000","last_change":"2023-11-08T08:22:20.697498+0000","last_active":"2023-11-08T08:22:20.705489+0000","last_peered":"2023-11-08T08:22:20.705489+0000","last_clean":"2023-11-08T08:22:20.705489+0000","last_became_active":"2023-11-08T08:22:20.693981+0000","last_became_peered":"2023-11-08T08:22:20.693981+0000","last_unstale":"2023-11-08T08:22:20.705489+0000","last_undegraded":"2023-11-08T08:22:20.705489+0000","last_fullsized":"2023-11-08T08:22:20.705489+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:06:39.151828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.699439+0000","last_change":"2023-11-08T08:22:20.690864+0000","last_active":"2023-11-08T08:22:20.699439+0000","last_peered":"2023-11-08T08:22:20.699439+0000","last_clean":"2023-11-08T08:22:20.699439+0000","last_became_active":"2023-11-08T08:22:20.690508+0000","last_became_peered":"2023-11-08T08:22:20.690508+0000","last_unstale":"2023-11-08T08:22:20.699439+0000","last_undegraded":"2023-11-08T08:22:20.699439+0000","last_fullsized":"2023-11-08T08:22:20.699439+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:04:06.670822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.689230+0000","last_change":"2023-11-08T08:22:20.687094+0000","last_active":"2023-11-08T08:22:20.689230+0000","last_peered":"2023-11-08T08:22:20.689230+0000","last_clean":"2023-11-08T08:22:20.689230+0000","last_became_active":"2023-11-08T08:22:20.686819+0000","last_became_peered":"2023-11-08T08:22:20.686819+0000","last_unstale":"2023-11-08T08:22:20.689230+0000","last_undegraded":"2023-11-08T08:22:20.689230+0000","last_fullsized":"2023-11-08T08:22:20.689230+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:32:32.438878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.695591+0000","last_change":"2023-11-08T08:22:20.679585+0000","last_active":"2023-11-08T08:22:20.695591+0000","last_peered":"2023-11-08T08:22:20.695591+0000","last_clean":"2023-11-08T08:22:20.695591+0000","last_became_active":"2023-11-08T08:22:20.679332+0000","last_became_peered":"2023-11-08T08:22:20.679332+0000","last_unstale":"2023-11-08T08:22:20.695591+0000","last_undegraded":"2023-11-08T08:22:20.695591+0000","last_fullsized":"2023-11-08T08:22:20.695591+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:36:46.493034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.706663+0000","last_change":"2023-11-08T08:22:20.703721+0000","last_active":"2023-11-08T08:22:20.706663+0000","last_peered":"2023-11-08T08:22:20.706663+0000","last_clean":"2023-11-08T08:22:20.706663+0000","last_became_active":"2023-11-08T08:22:20.703468+0000","last_became_peered":"2023-11-08T08:22:20.703468+0000","last_unstale":"2023-11-08T08:22:20.706663+0000","last_undegraded":"2023-11-08T08:22:20.706663+0000","last_fullsized":"2023-11-08T08:22:20.706663+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:47:00.548303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.701974+0000","last_change":"2023-11-08T08:22:20.688538+0000","last_active":"2023-11-08T08:22:20.701974+0000","last_peered":"2023-11-08T08:22:20.701974+0000","last_clean":"2023-11-08T08:22:20.701974+0000","last_became_active":"2023-11-08T08:22:20.688131+0000","last_became_peered":"2023-11-08T08:22:20.688131+0000","last_unstale":"2023-11-08T08:22:20.701974+0000","last_undegraded":"2023-11-08T08:22:20.701974+0000","last_fullsized":"2023-11-08T08:22:20.701974+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:33:21.532456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705407+0000","last_change":"2023-11-08T08:22:20.694494+0000","last_active":"2023-11-08T08:22:20.705407+0000","last_peered":"2023-11-08T08:22:20.705407+0000","last_clean":"2023-11-08T08:22:20.705407+0000","last_became_active":"2023-11-08T08:22:20.693971+0000","last_became_peered":"2023-11-08T08:22:20.693971+0000","last_unstale":"2023-11-08T08:22:20.705407+0000","last_undegraded":"2023-11-08T08:22:20.705407+0000","last_fullsized":"2023-11-08T08:22:20.705407+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:28.160795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.704273+0000","last_change":"2023-11-08T08:22:20.701051+0000","last_active":"2023-11-08T08:22:20.704273+0000","last_peered":"2023-11-08T08:22:20.704273+0000","last_clean":"2023-11-08T08:22:20.704273+0000","last_became_active":"2023-11-08T08:22:20.700810+0000","last_became_peered":"2023-11-08T08:22:20.700810+0000","last_unstale":"2023-11-08T08:22:20.704273+0000","last_undegraded":"2023-11-08T08:22:20.704273+0000","last_fullsized":"2023-11-08T08:22:20.704273+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:07:58.205197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705757+0000","last_change":"2023-11-08T08:22:20.699818+0000","last_active":"2023-11-08T08:22:20.705757+0000","last_peered":"2023-11-08T08:22:20.705757+0000","last_clean":"2023-11-08T08:22:20.705757+0000","last_became_active":"2023-11-08T08:22:20.698491+0000","last_became_peered":"2023-11-08T08:22:20.698491+0000","last_unstale":"2023-11-08T08:22:20.705757+0000","last_undegraded":"2023-11-08T08:22:20.705757+0000","last_fullsized":"2023-11-08T08:22:20.705757+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:24:54.219012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703400+0000","last_change":"2023-11-08T08:22:20.697645+0000","last_active":"2023-11-08T08:22:20.703400+0000","last_peered":"2023-11-08T08:22:20.703400+0000","last_clean":"2023-11-08T08:22:20.703400+0000","last_became_active":"2023-11-08T08:22:20.697370+0000","last_became_peered":"2023-11-08T08:22:20.697370+0000","last_unstale":"2023-11-08T08:22:20.703400+0000","last_undegraded":"2023-11-08T08:22:20.703400+0000","last_fullsized":"2023-11-08T08:22:20.703400+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:02.368358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.695382+0000","last_change":"2023-11-08T08:22:20.680839+0000","last_active":"2023-11-08T08:22:20.695382+0000","last_peered":"2023-11-08T08:22:20.695382+0000","last_clean":"2023-11-08T08:22:20.695382+0000","last_became_active":"2023-11-08T08:22:20.680508+0000","last_became_peered":"2023-11-08T08:22:20.680508+0000","last_unstale":"2023-11-08T08:22:20.695382+0000","last_undegraded":"2023-11-08T08:22:20.695382+0000","last_fullsized":"2023-11-08T08:22:20.695382+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:50:50.206439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.699429+0000","last_change":"2023-11-08T08:22:20.690797+0000","last_active":"2023-11-08T08:22:20.699429+0000","last_peered":"2023-11-08T08:22:20.699429+0000","last_clean":"2023-11-08T08:22:20.699429+0000","last_became_active":"2023-11-08T08:22:20.689924+0000","last_became_peered":"2023-11-08T08:22:20.689924+0000","last_unstale":"2023-11-08T08:22:20.699429+0000","last_undegraded":"2023-11-08T08:22:20.699429+0000","last_fullsized":"2023-11-08T08:22:20.699429+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:15:39.778545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.696732+0000","last_change":"2023-11-08T08:22:20.213448+0000","last_active":"2023-11-08T08:22:20.696732+0000","last_peered":"2023-11-08T08:22:20.696732+0000","last_clean":"2023-11-08T08:22:20.696732+0000","last_became_active":"2023-11-08T08:22:20.213166+0000","last_became_peered":"2023-11-08T08:22:20.213166+0000","last_unstale":"2023-11-08T08:22:20.696732+0000","last_undegraded":"2023-11-08T08:22:20.696732+0000","last_fullsized":"2023-11-08T08:22:20.696732+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:30:48.040211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703660+0000","last_change":"2023-11-08T08:22:20.699394+0000","last_active":"2023-11-08T08:22:20.703660+0000","last_peered":"2023-11-08T08:22:20.703660+0000","last_clean":"2023-11-08T08:22:20.703660+0000","last_became_active":"2023-11-08T08:22:20.698235+0000","last_became_peered":"2023-11-08T08:22:20.698235+0000","last_unstale":"2023-11-08T08:22:20.703660+0000","last_undegraded":"2023-11-08T08:22:20.703660+0000","last_fullsized":"2023-11-08T08:22:20.703660+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:11:49.456593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.688674+0000","last_change":"2023-11-08T08:22:20.195444+0000","last_active":"2023-11-08T08:22:20.688674+0000","last_peered":"2023-11-08T08:22:20.688674+0000","last_clean":"2023-11-08T08:22:20.688674+0000","last_became_active":"2023-11-08T08:22:20.195150+0000","last_became_peered":"2023-11-08T08:22:20.195150+0000","last_unstale":"2023-11-08T08:22:20.688674+0000","last_undegraded":"2023-11-08T08:22:20.688674+0000","last_fullsized":"2023-11-08T08:22:20.688674+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:46:44.381504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.702965+0000","last_change":"2023-11-08T08:22:20.689488+0000","last_active":"2023-11-08T08:22:20.702965+0000","last_peered":"2023-11-08T08:22:20.702965+0000","last_clean":"2023-11-08T08:22:20.702965+0000","last_became_active":"2023-11-08T08:22:20.689246+0000","last_became_peered":"2023-11-08T08:22:20.689246+0000","last_unstale":"2023-11-08T08:22:20.702965+0000","last_undegraded":"2023-11-08T08:22:20.702965+0000","last_fullsized":"2023-11-08T08:22:20.702965+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:36:04.953710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.704178+0000","last_change":"2023-11-08T08:22:20.197944+0000","last_active":"2023-11-08T08:22:20.704178+0000","last_peered":"2023-11-08T08:22:20.704178+0000","last_clean":"2023-11-08T08:22:20.704178+0000","last_became_active":"2023-11-08T08:22:20.197436+0000","last_became_peered":"2023-11-08T08:22:20.197436+0000","last_unstale":"2023-11-08T08:22:20.704178+0000","last_undegraded":"2023-11-08T08:22:20.704178+0000","last_fullsized":"2023-11-08T08:22:20.704178+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:15:38.772418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.696783+0000","last_change":"2023-11-08T08:22:20.688734+0000","last_active":"2023-11-08T08:22:20.696783+0000","last_peered":"2023-11-08T08:22:20.696783+0000","last_clean":"2023-11-08T08:22:20.696783+0000","last_became_active":"2023-11-08T08:22:20.688431+0000","last_became_peered":"2023-11-08T08:22:20.688431+0000","last_unstale":"2023-11-08T08:22:20.696783+0000","last_undegraded":"2023-11-08T08:22:20.696783+0000","last_fullsized":"2023-11-08T08:22:20.696783+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:24:54.219012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.701888+0000","last_change":"2023-11-08T08:22:20.205863+0000","last_active":"2023-11-08T08:22:20.701888+0000","last_peered":"2023-11-08T08:22:20.701888+0000","last_clean":"2023-11-08T08:22:20.701888+0000","last_became_active":"2023-11-08T08:22:20.205092+0000","last_became_peered":"2023-11-08T08:22:20.205092+0000","last_unstale":"2023-11-08T08:22:20.701888+0000","last_undegraded":"2023-11-08T08:22:20.701888+0000","last_fullsized":"2023-11-08T08:22:20.701888+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:37:59.460872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705849+0000","last_change":"2023-11-08T08:22:20.701106+0000","last_active":"2023-11-08T08:22:20.705849+0000","last_peered":"2023-11-08T08:22:20.705849+0000","last_clean":"2023-11-08T08:22:20.705849+0000","last_became_active":"2023-11-08T08:22:20.700839+0000","last_became_peered":"2023-11-08T08:22:20.700839+0000","last_unstale":"2023-11-08T08:22:20.705849+0000","last_undegraded":"2023-11-08T08:22:20.705849+0000","last_fullsized":"2023-11-08T08:22:20.705849+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:15:39.778545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"68'1","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.689146+0000","last_change":"2023-11-08T08:22:20.205809+0000","last_active":"2023-11-08T08:22:20.689146+0000","last_peered":"2023-11-08T08:22:20.689146+0000","last_clean":"2023-11-08T08:22:20.689146+0000","last_became_active":"2023-11-08T08:22:20.205342+0000","last_became_peered":"2023-11-08T08:22:20.205342+0000","last_unstale":"2023-11-08T08:22:20.689146+0000","last_undegraded":"2023-11-08T08:22:20.689146+0000","last_fullsized":"2023-11-08T08:22:20.689146+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:38:15.497203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.695199+0000","last_change":"2023-11-08T08:22:20.678975+0000","last_active":"2023-11-08T08:22:20.695199+0000","last_peered":"2023-11-08T08:22:20.695199+0000","last_clean":"2023-11-08T08:22:20.695199+0000","last_became_active":"2023-11-08T08:22:20.678710+0000","last_became_peered":"2023-11-08T08:22:20.678710+0000","last_unstale":"2023-11-08T08:22:20.695199+0000","last_undegraded":"2023-11-08T08:22:20.695199+0000","last_fullsized":"2023-11-08T08:22:20.695199+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:29:48.111099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.702638+0000","last_change":"2023-11-08T08:22:20.196451+0000","last_active":"2023-11-08T08:22:20.702638+0000","last_peered":"2023-11-08T08:22:20.702638+0000","last_clean":"2023-11-08T08:22:20.702638+0000","last_became_active":"2023-11-08T08:22:20.196177+0000","last_became_peered":"2023-11-08T08:22:20.196177+0000","last_unstale":"2023-11-08T08:22:20.702638+0000","last_undegraded":"2023-11-08T08:22:20.702638+0000","last_fullsized":"2023-11-08T08:22:20.702638+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:50:58.386151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.695560+0000","last_change":"2023-11-08T08:22:20.684815+0000","last_active":"2023-11-08T08:22:20.695560+0000","last_peered":"2023-11-08T08:22:20.695560+0000","last_clean":"2023-11-08T08:22:20.695560+0000","last_became_active":"2023-11-08T08:22:20.684564+0000","last_became_peered":"2023-11-08T08:22:20.684564+0000","last_unstale":"2023-11-08T08:22:20.695560+0000","last_undegraded":"2023-11-08T08:22:20.695560+0000","last_fullsized":"2023-11-08T08:22:20.695560+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:53.594420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.681503+0000","last_change":"2023-11-08T08:22:20.203943+0000","last_active":"2023-11-08T08:22:20.681503+0000","last_peered":"2023-11-08T08:22:20.681503+0000","last_clean":"2023-11-08T08:22:20.681503+0000","last_became_active":"2023-11-08T08:22:20.203563+0000","last_became_peered":"2023-11-08T08:22:20.203563+0000","last_unstale":"2023-11-08T08:22:20.681503+0000","last_undegraded":"2023-11-08T08:22:20.681503+0000","last_fullsized":"2023-11-08T08:22:20.681503+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:21:31.953212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.689176+0000","last_change":"2023-11-08T08:22:20.673450+0000","last_active":"2023-11-08T08:22:20.689176+0000","last_peered":"2023-11-08T08:22:20.689176+0000","last_clean":"2023-11-08T08:22:20.689176+0000","last_became_active":"2023-11-08T08:22:20.672934+0000","last_became_peered":"2023-11-08T08:22:20.672934+0000","last_unstale":"2023-11-08T08:22:20.689176+0000","last_undegraded":"2023-11-08T08:22:20.689176+0000","last_fullsized":"2023-11-08T08:22:20.689176+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:44:54.466782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.676558+0000","last_change":"2023-11-08T08:22:20.201122+0000","last_active":"2023-11-08T08:22:20.676558+0000","last_peered":"2023-11-08T08:22:20.676558+0000","last_clean":"2023-11-08T08:22:20.676558+0000","last_became_active":"2023-11-08T08:22:20.200111+0000","last_became_peered":"2023-11-08T08:22:20.200111+0000","last_unstale":"2023-11-08T08:22:20.676558+0000","last_undegraded":"2023-11-08T08:22:20.676558+0000","last_fullsized":"2023-11-08T08:22:20.676558+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:35:01.443129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705890+0000","last_change":"2023-11-08T08:22:20.701546+0000","last_active":"2023-11-08T08:22:20.705890+0000","last_peered":"2023-11-08T08:22:20.705890+0000","last_clean":"2023-11-08T08:22:20.705890+0000","last_became_active":"2023-11-08T08:22:20.700917+0000","last_became_peered":"2023-11-08T08:22:20.700917+0000","last_unstale":"2023-11-08T08:22:20.705890+0000","last_undegraded":"2023-11-08T08:22:20.705890+0000","last_fullsized":"2023-11-08T08:22:20.705890+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:32:32.438878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.671635+0000","last_change":"2023-11-08T08:22:20.200725+0000","last_active":"2023-11-08T08:22:20.671635+0000","last_peered":"2023-11-08T08:22:20.671635+0000","last_clean":"2023-11-08T08:22:20.671635+0000","last_became_active":"2023-11-08T08:22:20.200248+0000","last_became_peered":"2023-11-08T08:22:20.200248+0000","last_unstale":"2023-11-08T08:22:20.671635+0000","last_undegraded":"2023-11-08T08:22:20.671635+0000","last_fullsized":"2023-11-08T08:22:20.671635+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:32:40.141129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.689381+0000","last_change":"2023-11-08T08:22:20.680838+0000","last_active":"2023-11-08T08:22:20.689381+0000","last_peered":"2023-11-08T08:22:20.689381+0000","last_clean":"2023-11-08T08:22:20.689381+0000","last_became_active":"2023-11-08T08:22:20.680574+0000","last_became_peered":"2023-11-08T08:22:20.680574+0000","last_unstale":"2023-11-08T08:22:20.689381+0000","last_undegraded":"2023-11-08T08:22:20.689381+0000","last_fullsized":"2023-11-08T08:22:20.689381+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:54:34.009215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.682593+0000","last_change":"2023-11-08T08:22:20.201396+0000","last_active":"2023-11-08T08:22:20.682593+0000","last_peered":"2023-11-08T08:22:20.682593+0000","last_clean":"2023-11-08T08:22:20.682593+0000","last_became_active":"2023-11-08T08:22:20.200974+0000","last_became_peered":"2023-11-08T08:22:20.200974+0000","last_unstale":"2023-11-08T08:22:20.682593+0000","last_undegraded":"2023-11-08T08:22:20.682593+0000","last_fullsized":"2023-11-08T08:22:20.682593+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:00:17.585593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.702985+0000","last_change":"2023-11-08T08:22:20.690026+0000","last_active":"2023-11-08T08:22:20.702985+0000","last_peered":"2023-11-08T08:22:20.702985+0000","last_clean":"2023-11-08T08:22:20.702985+0000","last_became_active":"2023-11-08T08:22:20.689789+0000","last_became_peered":"2023-11-08T08:22:20.689789+0000","last_unstale":"2023-11-08T08:22:20.702985+0000","last_undegraded":"2023-11-08T08:22:20.702985+0000","last_fullsized":"2023-11-08T08:22:20.702985+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:18:33.815298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"68'3","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703932+0000","last_change":"2023-11-08T08:22:20.206871+0000","last_active":"2023-11-08T08:22:20.703932+0000","last_peered":"2023-11-08T08:22:20.703932+0000","last_clean":"2023-11-08T08:22:20.703932+0000","last_became_active":"2023-11-08T08:22:20.206479+0000","last_became_peered":"2023-11-08T08:22:20.206479+0000","last_unstale":"2023-11-08T08:22:20.703932+0000","last_undegraded":"2023-11-08T08:22:20.703932+0000","last_fullsized":"2023-11-08T08:22:20.703932+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:12:19.948868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.702226+0000","last_change":"2023-11-08T08:22:20.688710+0000","last_active":"2023-11-08T08:22:20.702226+0000","last_peered":"2023-11-08T08:22:20.702226+0000","last_clean":"2023-11-08T08:22:20.702226+0000","last_became_active":"2023-11-08T08:22:20.688412+0000","last_became_peered":"2023-11-08T08:22:20.688412+0000","last_unstale":"2023-11-08T08:22:20.702226+0000","last_undegraded":"2023-11-08T08:22:20.702226+0000","last_fullsized":"2023-11-08T08:22:20.702226+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:52:33.736496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.707794+0000","last_change":"2023-11-08T08:22:20.199781+0000","last_active":"2023-11-08T08:22:20.707794+0000","last_peered":"2023-11-08T08:22:20.707794+0000","last_clean":"2023-11-08T08:22:20.707794+0000","last_became_active":"2023-11-08T08:22:20.199438+0000","last_became_peered":"2023-11-08T08:22:20.199438+0000","last_unstale":"2023-11-08T08:22:20.707794+0000","last_undegraded":"2023-11-08T08:22:20.707794+0000","last_fullsized":"2023-11-08T08:22:20.707794+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:38:33.814839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.699794+0000","last_change":"2023-11-08T08:22:20.693515+0000","last_active":"2023-11-08T08:22:20.699794+0000","last_peered":"2023-11-08T08:22:20.699794+0000","last_clean":"2023-11-08T08:22:20.699794+0000","last_became_active":"2023-11-08T08:22:20.693061+0000","last_became_peered":"2023-11-08T08:22:20.693061+0000","last_unstale":"2023-11-08T08:22:20.699794+0000","last_undegraded":"2023-11-08T08:22:20.699794+0000","last_fullsized":"2023-11-08T08:22:20.699794+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:31:09.871375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.696079+0000","last_change":"2023-11-08T08:22:20.204028+0000","last_active":"2023-11-08T08:22:20.696079+0000","last_peered":"2023-11-08T08:22:20.696079+0000","last_clean":"2023-11-08T08:22:20.696079+0000","last_became_active":"2023-11-08T08:22:20.203521+0000","last_became_peered":"2023-11-08T08:22:20.203521+0000","last_unstale":"2023-11-08T08:22:20.696079+0000","last_undegraded":"2023-11-08T08:22:20.696079+0000","last_fullsized":"2023-11-08T08:22:20.696079+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:21:33.321814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705973+0000","last_change":"2023-11-08T08:22:20.702065+0000","last_active":"2023-11-08T08:22:20.705973+0000","last_peered":"2023-11-08T08:22:20.705973+0000","last_clean":"2023-11-08T08:22:20.705973+0000","last_became_active":"2023-11-08T08:22:20.701780+0000","last_became_peered":"2023-11-08T08:22:20.701780+0000","last_unstale":"2023-11-08T08:22:20.705973+0000","last_undegraded":"2023-11-08T08:22:20.705973+0000","last_fullsized":"2023-11-08T08:22:20.705973+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:35:24.924456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.683889+0000","last_change":"2023-11-08T08:22:20.203768+0000","last_active":"2023-11-08T08:22:20.683889+0000","last_peered":"2023-11-08T08:22:20.683889+0000","last_clean":"2023-11-08T08:22:20.683889+0000","last_became_active":"2023-11-08T08:22:20.202505+0000","last_became_peered":"2023-11-08T08:22:20.202505+0000","last_unstale":"2023-11-08T08:22:20.683889+0000","last_undegraded":"2023-11-08T08:22:20.683889+0000","last_fullsized":"2023-11-08T08:22:20.683889+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:30:06.084174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.698298+0000","last_change":"2023-11-08T08:22:20.690875+0000","last_active":"2023-11-08T08:22:20.698298+0000","last_peered":"2023-11-08T08:22:20.698298+0000","last_clean":"2023-11-08T08:22:20.698298+0000","last_became_active":"2023-11-08T08:22:20.690458+0000","last_became_peered":"2023-11-08T08:22:20.690458+0000","last_unstale":"2023-11-08T08:22:20.698298+0000","last_undegraded":"2023-11-08T08:22:20.698298+0000","last_fullsized":"2023-11-08T08:22:20.698298+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:39:04.388075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.683408+0000","last_change":"2023-11-08T08:22:20.198290+0000","last_active":"2023-11-08T08:22:20.683408+0000","last_peered":"2023-11-08T08:22:20.683408+0000","last_clean":"2023-11-08T08:22:20.683408+0000","last_became_active":"2023-11-08T08:22:20.197828+0000","last_became_peered":"2023-11-08T08:22:20.197828+0000","last_unstale":"2023-11-08T08:22:20.683408+0000","last_undegraded":"2023-11-08T08:22:20.683408+0000","last_fullsized":"2023-11-08T08:22:20.683408+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:16:06.608091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.696215+0000","last_change":"2023-11-08T08:22:20.684025+0000","last_active":"2023-11-08T08:22:20.696215+0000","last_peered":"2023-11-08T08:22:20.696215+0000","last_clean":"2023-11-08T08:22:20.696215+0000","last_became_active":"2023-11-08T08:22:20.683726+0000","last_became_peered":"2023-11-08T08:22:20.683726+0000","last_unstale":"2023-11-08T08:22:20.696215+0000","last_undegraded":"2023-11-08T08:22:20.696215+0000","last_fullsized":"2023-11-08T08:22:20.696215+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:12.659020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.704640+0000","last_change":"2023-11-08T08:22:20.199622+0000","last_active":"2023-11-08T08:22:20.704640+0000","last_peered":"2023-11-08T08:22:20.704640+0000","last_clean":"2023-11-08T08:22:20.704640+0000","last_became_active":"2023-11-08T08:22:20.199363+0000","last_became_peered":"2023-11-08T08:22:20.199363+0000","last_unstale":"2023-11-08T08:22:20.704640+0000","last_undegraded":"2023-11-08T08:22:20.704640+0000","last_fullsized":"2023-11-08T08:22:20.704640+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:24:53.212884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.700114+0000","last_change":"2023-11-08T08:22:20.694688+0000","last_active":"2023-11-08T08:22:20.700114+0000","last_peered":"2023-11-08T08:22:20.700114+0000","last_clean":"2023-11-08T08:22:20.700114+0000","last_became_active":"2023-11-08T08:22:20.694040+0000","last_became_peered":"2023-11-08T08:22:20.694040+0000","last_unstale":"2023-11-08T08:22:20.700114+0000","last_undegraded":"2023-11-08T08:22:20.700114+0000","last_fullsized":"2023-11-08T08:22:20.700114+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:20:23.943501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.706057+0000","last_change":"2023-11-08T08:22:20.207231+0000","last_active":"2023-11-08T08:22:20.706057+0000","last_peered":"2023-11-08T08:22:20.706057+0000","last_clean":"2023-11-08T08:22:20.706057+0000","last_became_active":"2023-11-08T08:22:20.206725+0000","last_became_peered":"2023-11-08T08:22:20.206725+0000","last_unstale":"2023-11-08T08:22:20.706057+0000","last_undegraded":"2023-11-08T08:22:20.706057+0000","last_fullsized":"2023-11-08T08:22:20.706057+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:11:48.450466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705787+0000","last_change":"2023-11-08T08:22:20.699810+0000","last_active":"2023-11-08T08:22:20.705787+0000","last_peered":"2023-11-08T08:22:20.705787+0000","last_clean":"2023-11-08T08:22:20.705787+0000","last_became_active":"2023-11-08T08:22:20.699512+0000","last_became_peered":"2023-11-08T08:22:20.699512+0000","last_unstale":"2023-11-08T08:22:20.705787+0000","last_undegraded":"2023-11-08T08:22:20.705787+0000","last_fullsized":"2023-11-08T08:22:20.705787+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:32:41.147256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.684842+0000","last_change":"2023-11-08T08:22:20.199547+0000","last_active":"2023-11-08T08:22:20.684842+0000","last_peered":"2023-11-08T08:22:20.684842+0000","last_clean":"2023-11-08T08:22:20.684842+0000","last_became_active":"2023-11-08T08:22:20.199032+0000","last_became_peered":"2023-11-08T08:22:20.199032+0000","last_unstale":"2023-11-08T08:22:20.684842+0000","last_undegraded":"2023-11-08T08:22:20.684842+0000","last_fullsized":"2023-11-08T08:22:20.684842+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:40:53.187744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.698055+0000","last_change":"2023-11-08T08:22:20.689245+0000","last_active":"2023-11-08T08:22:20.698055+0000","last_peered":"2023-11-08T08:22:20.698055+0000","last_clean":"2023-11-08T08:22:20.698055+0000","last_became_active":"2023-11-08T08:22:20.688509+0000","last_became_peered":"2023-11-08T08:22:20.688509+0000","last_unstale":"2023-11-08T08:22:20.698055+0000","last_undegraded":"2023-11-08T08:22:20.698055+0000","last_fullsized":"2023-11-08T08:22:20.698055+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:32:41.147256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.682621+0000","last_change":"2023-11-08T08:22:20.190145+0000","last_active":"2023-11-08T08:22:20.682621+0000","last_peered":"2023-11-08T08:22:20.682621+0000","last_clean":"2023-11-08T08:22:20.682621+0000","last_became_active":"2023-11-08T08:22:20.189793+0000","last_became_peered":"2023-11-08T08:22:20.189793+0000","last_unstale":"2023-11-08T08:22:20.682621+0000","last_undegraded":"2023-11-08T08:22:20.682621+0000","last_fullsized":"2023-11-08T08:22:20.682621+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:41:16.549439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705501+0000","last_change":"2023-11-08T08:22:20.701070+0000","last_active":"2023-11-08T08:22:20.705501+0000","last_peered":"2023-11-08T08:22:20.705501+0000","last_clean":"2023-11-08T08:22:20.705501+0000","last_became_active":"2023-11-08T08:22:20.700804+0000","last_became_peered":"2023-11-08T08:22:20.700804+0000","last_unstale":"2023-11-08T08:22:20.705501+0000","last_undegraded":"2023-11-08T08:22:20.705501+0000","last_fullsized":"2023-11-08T08:22:20.705501+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:47:00.548303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.662046+0000","last_change":"2023-11-08T08:22:20.189567+0000","last_active":"2023-11-08T08:22:20.662046+0000","last_peered":"2023-11-08T08:22:20.662046+0000","last_clean":"2023-11-08T08:22:20.662046+0000","last_became_active":"2023-11-08T08:22:20.189048+0000","last_became_peered":"2023-11-08T08:22:20.189048+0000","last_unstale":"2023-11-08T08:22:20.662046+0000","last_undegraded":"2023-11-08T08:22:20.662046+0000","last_fullsized":"2023-11-08T08:22:20.662046+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:22:21.856135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703791+0000","last_change":"2023-11-08T08:22:20.701353+0000","last_active":"2023-11-08T08:22:20.703791+0000","last_peered":"2023-11-08T08:22:20.703791+0000","last_clean":"2023-11-08T08:22:20.703791+0000","last_became_active":"2023-11-08T08:22:20.700940+0000","last_became_peered":"2023-11-08T08:22:20.700940+0000","last_unstale":"2023-11-08T08:22:20.703791+0000","last_undegraded":"2023-11-08T08:22:20.703791+0000","last_fullsized":"2023-11-08T08:22:20.703791+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:30:07.090301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.680241+0000","last_change":"2023-11-08T08:22:20.192146+0000","last_active":"2023-11-08T08:22:20.680241+0000","last_peered":"2023-11-08T08:22:20.680241+0000","last_clean":"2023-11-08T08:22:20.680241+0000","last_became_active":"2023-11-08T08:22:20.191886+0000","last_became_peered":"2023-11-08T08:22:20.191886+0000","last_unstale":"2023-11-08T08:22:20.680241+0000","last_undegraded":"2023-11-08T08:22:20.680241+0000","last_fullsized":"2023-11-08T08:22:20.680241+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:23:39.012885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.701836+0000","last_change":"2023-11-08T08:22:20.685906+0000","last_active":"2023-11-08T08:22:20.701836+0000","last_peered":"2023-11-08T08:22:20.701836+0000","last_clean":"2023-11-08T08:22:20.701836+0000","last_became_active":"2023-11-08T08:22:20.685586+0000","last_became_peered":"2023-11-08T08:22:20.685586+0000","last_unstale":"2023-11-08T08:22:20.701836+0000","last_undegraded":"2023-11-08T08:22:20.701836+0000","last_fullsized":"2023-11-08T08:22:20.701836+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:15:21.780103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.684368+0000","last_change":"2023-11-08T08:22:20.204475+0000","last_active":"2023-11-08T08:22:20.684368+0000","last_peered":"2023-11-08T08:22:20.684368+0000","last_clean":"2023-11-08T08:22:20.684368+0000","last_became_active":"2023-11-08T08:22:20.203660+0000","last_became_peered":"2023-11-08T08:22:20.203660+0000","last_unstale":"2023-11-08T08:22:20.684368+0000","last_undegraded":"2023-11-08T08:22:20.684368+0000","last_fullsized":"2023-11-08T08:22:20.684368+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:06:37.792121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.695918+0000","last_change":"2023-11-08T08:22:20.690205+0000","last_active":"2023-11-08T08:22:20.695918+0000","last_peered":"2023-11-08T08:22:20.695918+0000","last_clean":"2023-11-08T08:22:20.695918+0000","last_became_active":"2023-11-08T08:22:20.689931+0000","last_became_peered":"2023-11-08T08:22:20.689931+0000","last_unstale":"2023-11-08T08:22:20.695918+0000","last_undegraded":"2023-11-08T08:22:20.695918+0000","last_fullsized":"2023-11-08T08:22:20.695918+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:05:14.039538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.670590+0000","last_change":"2023-11-08T08:22:20.204551+0000","last_active":"2023-11-08T08:22:20.670590+0000","last_peered":"2023-11-08T08:22:20.670590+0000","last_clean":"2023-11-08T08:22:20.670590+0000","last_became_active":"2023-11-08T08:22:20.204257+0000","last_became_peered":"2023-11-08T08:22:20.204257+0000","last_unstale":"2023-11-08T08:22:20.670590+0000","last_undegraded":"2023-11-08T08:22:20.670590+0000","last_fullsized":"2023-11-08T08:22:20.670590+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:46:59.542176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703173+0000","last_change":"2023-11-08T08:22:20.693657+0000","last_active":"2023-11-08T08:22:20.703173+0000","last_peered":"2023-11-08T08:22:20.703173+0000","last_clean":"2023-11-08T08:22:20.703173+0000","last_became_active":"2023-11-08T08:22:20.693007+0000","last_became_peered":"2023-11-08T08:22:20.693007+0000","last_unstale":"2023-11-08T08:22:20.703173+0000","last_undegraded":"2023-11-08T08:22:20.703173+0000","last_fullsized":"2023-11-08T08:22:20.703173+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:35:17.751012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.682992+0000","last_change":"2023-11-08T08:22:20.203458+0000","last_active":"2023-11-08T08:22:20.682992+0000","last_peered":"2023-11-08T08:22:20.682992+0000","last_clean":"2023-11-08T08:22:20.682992+0000","last_became_active":"2023-11-08T08:22:20.203089+0000","last_became_peered":"2023-11-08T08:22:20.203089+0000","last_unstale":"2023-11-08T08:22:20.682992+0000","last_undegraded":"2023-11-08T08:22:20.682992+0000","last_fullsized":"2023-11-08T08:22:20.682992+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:52.588292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.700478+0000","last_change":"2023-11-08T08:22:20.689029+0000","last_active":"2023-11-08T08:22:20.700478+0000","last_peered":"2023-11-08T08:22:20.700478+0000","last_clean":"2023-11-08T08:22:20.700478+0000","last_became_active":"2023-11-08T08:22:20.688756+0000","last_became_peered":"2023-11-08T08:22:20.688756+0000","last_unstale":"2023-11-08T08:22:20.700478+0000","last_undegraded":"2023-11-08T08:22:20.700478+0000","last_fullsized":"2023-11-08T08:22:20.700478+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:20:23.943501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.706306+0000","last_change":"2023-11-08T08:22:20.206983+0000","last_active":"2023-11-08T08:22:20.706306+0000","last_peered":"2023-11-08T08:22:20.706306+0000","last_clean":"2023-11-08T08:22:20.706306+0000","last_became_active":"2023-11-08T08:22:20.206349+0000","last_became_peered":"2023-11-08T08:22:20.206349+0000","last_unstale":"2023-11-08T08:22:20.706306+0000","last_undegraded":"2023-11-08T08:22:20.706306+0000","last_fullsized":"2023-11-08T08:22:20.706306+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:11.652893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.200062+0000","last_change":"2023-11-08T08:18:58.771310+0000","last_active":"2023-11-08T08:22:20.200062+0000","last_peered":"2023-11-08T08:22:20.200062+0000","last_clean":"2023-11-08T08:22:20.200062+0000","last_became_active":"2023-11-08T08:18:58.770891+0000","last_became_peered":"2023-11-08T08:18:58.770891+0000","last_unstale":"2023-11-08T08:22:20.200062+0000","last_undegraded":"2023-11-08T08:22:20.200062+0000","last_fullsized":"2023-11-08T08:22:20.200062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.699660+0000","last_change":"2023-11-08T08:22:20.692436+0000","last_active":"2023-11-08T08:22:20.699660+0000","last_peered":"2023-11-08T08:22:20.699660+0000","last_clean":"2023-11-08T08:22:20.699660+0000","last_became_active":"2023-11-08T08:22:20.692139+0000","last_became_peered":"2023-11-08T08:22:20.692139+0000","last_unstale":"2023-11-08T08:22:20.699660+0000","last_undegraded":"2023-11-08T08:22:20.699660+0000","last_fullsized":"2023-11-08T08:22:20.699660+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:50:50.206439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705086+0000","last_change":"2023-11-08T08:22:20.200670+0000","last_active":"2023-11-08T08:22:20.705086+0000","last_peered":"2023-11-08T08:22:20.705086+0000","last_clean":"2023-11-08T08:22:20.705086+0000","last_became_active":"2023-11-08T08:22:20.200366+0000","last_became_peered":"2023-11-08T08:22:20.200366+0000","last_unstale":"2023-11-08T08:22:20.705086+0000","last_undegraded":"2023-11-08T08:22:20.705086+0000","last_fullsized":"2023-11-08T08:22:20.705086+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:54:33.003087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":50,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.193571+0000","last_change":"2023-11-08T08:18:58.770483+0000","last_active":"2023-11-08T08:22:20.193571+0000","last_peered":"2023-11-08T08:22:20.193571+0000","last_clean":"2023-11-08T08:22:20.193571+0000","last_became_active":"2023-11-08T08:18:58.770040+0000","last_became_peered":"2023-11-08T08:18:58.770040+0000","last_unstale":"2023-11-08T08:22:20.193571+0000","last_undegraded":"2023-11-08T08:22:20.193571+0000","last_fullsized":"2023-11-08T08:22:20.193571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.699657+0000","last_change":"2023-11-08T08:22:20.691973+0000","last_active":"2023-11-08T08:22:20.699657+0000","last_peered":"2023-11-08T08:22:20.699657+0000","last_clean":"2023-11-08T08:22:20.699657+0000","last_became_active":"2023-11-08T08:22:20.691242+0000","last_became_peered":"2023-11-08T08:22:20.691242+0000","last_unstale":"2023-11-08T08:22:20.699657+0000","last_undegraded":"2023-11-08T08:22:20.699657+0000","last_fullsized":"2023-11-08T08:22:20.699657+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:51.802894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.702724+0000","last_change":"2023-11-08T08:22:20.203707+0000","last_active":"2023-11-08T08:22:20.702724+0000","last_peered":"2023-11-08T08:22:20.702724+0000","last_clean":"2023-11-08T08:22:20.702724+0000","last_became_active":"2023-11-08T08:22:20.203159+0000","last_became_peered":"2023-11-08T08:22:20.203159+0000","last_unstale":"2023-11-08T08:22:20.702724+0000","last_undegraded":"2023-11-08T08:22:20.702724+0000","last_fullsized":"2023-11-08T08:22:20.702724+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:20:54.501281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":52,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.204636+0000","last_change":"2023-11-08T08:18:58.766848+0000","last_active":"2023-11-08T08:22:20.204636+0000","last_peered":"2023-11-08T08:22:20.204636+0000","last_clean":"2023-11-08T08:22:20.204636+0000","last_became_active":"2023-11-08T08:18:58.766378+0000","last_became_peered":"2023-11-08T08:18:58.766378+0000","last_unstale":"2023-11-08T08:22:20.204636+0000","last_undegraded":"2023-11-08T08:22:20.204636+0000","last_fullsized":"2023-11-08T08:22:20.204636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703394+0000","last_change":"2023-11-08T08:22:20.696868+0000","last_active":"2023-11-08T08:22:20.703394+0000","last_peered":"2023-11-08T08:22:20.703394+0000","last_clean":"2023-11-08T08:22:20.703394+0000","last_became_active":"2023-11-08T08:22:20.696572+0000","last_became_peered":"2023-11-08T08:22:20.696572+0000","last_unstale":"2023-11-08T08:22:20.703394+0000","last_undegraded":"2023-11-08T08:22:20.703394+0000","last_fullsized":"2023-11-08T08:22:20.703394+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:14.575132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.670678+0000","last_change":"2023-11-08T08:22:20.208213+0000","last_active":"2023-11-08T08:22:20.670678+0000","last_peered":"2023-11-08T08:22:20.670678+0000","last_clean":"2023-11-08T08:22:20.670678+0000","last_became_active":"2023-11-08T08:22:20.207888+0000","last_became_peered":"2023-11-08T08:22:20.207888+0000","last_unstale":"2023-11-08T08:22:20.670678+0000","last_undegraded":"2023-11-08T08:22:20.670678+0000","last_fullsized":"2023-11-08T08:22:20.670678+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:18:45.247325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":50,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.194310+0000","last_change":"2023-11-08T08:18:58.769764+0000","last_active":"2023-11-08T08:22:20.194310+0000","last_peered":"2023-11-08T08:22:20.194310+0000","last_clean":"2023-11-08T08:22:20.194310+0000","last_became_active":"2023-11-08T08:18:58.769357+0000","last_became_peered":"2023-11-08T08:18:58.769357+0000","last_unstale":"2023-11-08T08:22:20.194310+0000","last_undegraded":"2023-11-08T08:22:20.194310+0000","last_fullsized":"2023-11-08T08:22:20.194310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:13.748094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703721+0000","last_change":"2023-11-08T08:22:20.689657+0000","last_active":"2023-11-08T08:22:20.703721+0000","last_peered":"2023-11-08T08:22:20.703721+0000","last_clean":"2023-11-08T08:22:20.703721+0000","last_became_active":"2023-11-08T08:22:20.689384+0000","last_became_peered":"2023-11-08T08:22:20.689384+0000","last_unstale":"2023-11-08T08:22:20.703721+0000","last_undegraded":"2023-11-08T08:22:20.703721+0000","last_fullsized":"2023-11-08T08:22:20.703721+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:20:23.943501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.682574+0000","last_change":"2023-11-08T08:22:20.193829+0000","last_active":"2023-11-08T08:22:20.682574+0000","last_peered":"2023-11-08T08:22:20.682574+0000","last_clean":"2023-11-08T08:22:20.682574+0000","last_became_active":"2023-11-08T08:22:20.193503+0000","last_became_peered":"2023-11-08T08:22:20.193503+0000","last_unstale":"2023-11-08T08:22:20.682574+0000","last_undegraded":"2023-11-08T08:22:20.682574+0000","last_fullsized":"2023-11-08T08:22:20.682574+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:50.796767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.189418+0000","last_change":"2023-11-08T08:18:58.771460+0000","last_active":"2023-11-08T08:22:20.189418+0000","last_peered":"2023-11-08T08:22:20.189418+0000","last_clean":"2023-11-08T08:22:20.189418+0000","last_became_active":"2023-11-08T08:18:58.770648+0000","last_became_peered":"2023-11-08T08:18:58.770648+0000","last_unstale":"2023-11-08T08:22:20.189418+0000","last_undegraded":"2023-11-08T08:22:20.189418+0000","last_fullsized":"2023-11-08T08:22:20.189418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.696664+0000","last_change":"2023-11-08T08:22:20.685820+0000","last_active":"2023-11-08T08:22:20.696664+0000","last_peered":"2023-11-08T08:22:20.696664+0000","last_clean":"2023-11-08T08:22:20.696664+0000","last_became_active":"2023-11-08T08:22:20.685570+0000","last_became_peered":"2023-11-08T08:22:20.685570+0000","last_unstale":"2023-11-08T08:22:20.696664+0000","last_undegraded":"2023-11-08T08:22:20.696664+0000","last_fullsized":"2023-11-08T08:22:20.696664+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:52:08.554540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.683551+0000","last_change":"2023-11-08T08:22:20.193136+0000","last_active":"2023-11-08T08:22:20.683551+0000","last_peered":"2023-11-08T08:22:20.683551+0000","last_clean":"2023-11-08T08:22:20.683551+0000","last_became_active":"2023-11-08T08:22:20.192807+0000","last_became_peered":"2023-11-08T08:22:20.192807+0000","last_unstale":"2023-11-08T08:22:20.683551+0000","last_undegraded":"2023-11-08T08:22:20.683551+0000","last_fullsized":"2023-11-08T08:22:20.683551+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:06:30.564100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":53,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.186292+0000","last_change":"2023-11-08T08:18:58.973732+0000","last_active":"2023-11-08T08:22:20.186292+0000","last_peered":"2023-11-08T08:22:20.186292+0000","last_clean":"2023-11-08T08:22:20.186292+0000","last_became_active":"2023-11-08T08:18:58.973007+0000","last_became_peered":"2023-11-08T08:18:58.973007+0000","last_unstale":"2023-11-08T08:22:20.186292+0000","last_undegraded":"2023-11-08T08:22:20.186292+0000","last_fullsized":"2023-11-08T08:22:20.186292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.696983+0000","last_change":"2023-11-08T08:22:20.678891+0000","last_active":"2023-11-08T08:22:20.696983+0000","last_peered":"2023-11-08T08:22:20.696983+0000","last_clean":"2023-11-08T08:22:20.696983+0000","last_became_active":"2023-11-08T08:22:20.678596+0000","last_became_peered":"2023-11-08T08:22:20.678596+0000","last_unstale":"2023-11-08T08:22:20.696983+0000","last_undegraded":"2023-11-08T08:22:20.696983+0000","last_fullsized":"2023-11-08T08:22:20.696983+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:44:54.466782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.660754+0000","last_change":"2023-11-08T08:22:20.186891+0000","last_active":"2023-11-08T08:22:20.660754+0000","last_peered":"2023-11-08T08:22:20.660754+0000","last_clean":"2023-11-08T08:22:20.660754+0000","last_became_active":"2023-11-08T08:22:20.186535+0000","last_became_peered":"2023-11-08T08:22:20.186535+0000","last_unstale":"2023-11-08T08:22:20.660754+0000","last_undegraded":"2023-11-08T08:22:20.660754+0000","last_fullsized":"2023-11-08T08:22:20.660754+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:08:46.040304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.202004+0000","last_change":"2023-11-08T08:18:58.770136+0000","last_active":"2023-11-08T08:22:20.202004+0000","last_peered":"2023-11-08T08:22:20.202004+0000","last_clean":"2023-11-08T08:22:20.202004+0000","last_became_active":"2023-11-08T08:18:58.769797+0000","last_became_peered":"2023-11-08T08:18:58.769797+0000","last_unstale":"2023-11-08T08:22:20.202004+0000","last_undegraded":"2023-11-08T08:22:20.202004+0000","last_fullsized":"2023-11-08T08:22:20.202004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:29:12.107128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.696577+0000","last_change":"2023-11-08T08:22:20.681462+0000","last_active":"2023-11-08T08:22:20.696577+0000","last_peered":"2023-11-08T08:22:20.696577+0000","last_clean":"2023-11-08T08:22:20.696577+0000","last_became_active":"2023-11-08T08:22:20.680690+0000","last_became_peered":"2023-11-08T08:22:20.680690+0000","last_unstale":"2023-11-08T08:22:20.696577+0000","last_undegraded":"2023-11-08T08:22:20.696577+0000","last_fullsized":"2023-11-08T08:22:20.696577+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:39:04.388075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.701747+0000","last_change":"2023-11-08T08:22:20.204262+0000","last_active":"2023-11-08T08:22:20.701747+0000","last_peered":"2023-11-08T08:22:20.701747+0000","last_clean":"2023-11-08T08:22:20.701747+0000","last_became_active":"2023-11-08T08:22:20.204012+0000","last_became_peered":"2023-11-08T08:22:20.204012+0000","last_unstale":"2023-11-08T08:22:20.701747+0000","last_undegraded":"2023-11-08T08:22:20.701747+0000","last_fullsized":"2023-11-08T08:22:20.701747+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:38:33.814839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.189420+0000","last_change":"2023-11-08T08:18:58.777140+0000","last_active":"2023-11-08T08:22:20.189420+0000","last_peered":"2023-11-08T08:22:20.189420+0000","last_clean":"2023-11-08T08:22:20.189420+0000","last_became_active":"2023-11-08T08:18:58.776834+0000","last_became_peered":"2023-11-08T08:18:58.776834+0000","last_unstale":"2023-11-08T08:22:20.189420+0000","last_undegraded":"2023-11-08T08:22:20.189420+0000","last_fullsized":"2023-11-08T08:22:20.189420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":205,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.201913+0000","last_change":"2023-11-08T08:18:53.780089+0000","last_active":"2023-11-08T08:22:20.201913+0000","last_peered":"2023-11-08T08:22:20.201913+0000","last_clean":"2023-11-08T08:22:20.201913+0000","last_became_active":"2023-11-08T08:18:53.779533+0000","last_became_peered":"2023-11-08T08:18:53.779533+0000","last_unstale":"2023-11-08T08:22:20.201913+0000","last_undegraded":"2023-11-08T08:22:20.201913+0000","last_fullsized":"2023-11-08T08:22:20.201913+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_clean_scrub_stamp":"2023-11-08T08:18:52.744693+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:58:24.877070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14884,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":13504,"kb_avail":94356956,"statfs":{"total":96636764160,"available":96621522944,"internally_reserved":0,"allocated":1404928,"data_stored":966246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13828096},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.083,"15min":1.028},"min":{"1min":0.74399999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5669999999999999,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.194,"15min":1.1699999999999999},"min":{"1min":1.028,"5min":0.746,"15min":0.746},"max":{"1min":1.5660000000000001,"5min":1.667,"15min":1.667},"last":1.2709999999999999}]},{"osd":1,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1859999999999999,"15min":1.1930000000000001},"min":{"1min":0.77900000000000003,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.5589999999999999,"5min":1.972,"15min":1.972},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.294,"15min":1.3160000000000001},"min":{"1min":0.99099999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.696,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.93200000000000005}]},{"osd":2,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.329,"15min":1.4039999999999999},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.597,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":1.302},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.319,"15min":1.401},"min":{"1min":0.874,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.522,"5min":4.9089999999999998,"15min":4.9089999999999998},"last":0.99199999999999999}]},{"osd":3,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.2250000000000001,"15min":1.252},"min":{"1min":0.84999999999999998,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.3460000000000001,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.292,"15min":1.3080000000000001},"min":{"1min":1.0649999999999999,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.8049999999999999,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":1.2130000000000001}]},{"osd":4,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.2370000000000001,"15min":1.264},"min":{"1min":0.79800000000000004,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.476,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.3939999999999999,"15min":1.425},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.643,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":1.24}]},{"osd":5,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.3180000000000001,"15min":1.3700000000000001},"min":{"1min":0.89500000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.329,"5min":2.73,"15min":2.73},"last":1.024},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3160000000000001,"15min":1.3360000000000001},"min":{"1min":0.85599999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.6060000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.80900000000000005}]},{"osd":6,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.423,"15min":1.4490000000000001},"min":{"1min":0.88900000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.762,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.3799999999999999,"15min":1.4339999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.5109999999999999,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.83799999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9568,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362272,"statfs":{"total":96636764160,"available":96626966528,"internally_reserved":0,"allocated":811008,"data_stored":375878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.121,"15min":1.173},"min":{"1min":0.67800000000000005,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5229999999999999,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.048,"5min":1.0840000000000001,"15min":1.0980000000000001},"min":{"1min":0.60799999999999998,"5min":0.501,"15min":0.501},"max":{"1min":1.4359999999999999,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.94699999999999995}]},{"osd":1,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2110000000000001,"15min":1.1950000000000001},"min":{"1min":0.58499999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.7509999999999999,"5min":1.782,"15min":1.782},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2110000000000001,"15min":1.2070000000000001},"min":{"1min":0.626,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.429}]},{"osd":2,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.169,"15min":1.1819999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7989999999999999,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.244,"15min":1.286},"min":{"1min":0.81599999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.6399999999999999,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.288}]},{"osd":3,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2729999999999999,"15min":1.2869999999999999},"min":{"1min":0.80800000000000005,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.3899999999999999,"15min":1.407},"min":{"1min":1.052,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.6349999999999998,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":1.5600000000000001}]},{"osd":4,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.23,"15min":1.258},"min":{"1min":0.76200000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.5649999999999999,"5min":2.02,"15min":2.02},"last":1.52},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.202,"15min":1.194},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.599,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.0549999999999999}]},{"osd":5,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.262,"15min":1.24},"min":{"1min":0.94899999999999995,"5min":0.88,"15min":0.88},"max":{"1min":1.655,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3600000000000001,"15min":1.3700000000000001},"min":{"1min":0.96799999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":1.6299999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3720000000000001,"15min":1.3819999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.8899999999999999,"5min":3.133,"15min":3.133},"last":1.379},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4490000000000001,"15min":1.4410000000000001},"min":{"1min":1.173,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.8400000000000001,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.7669999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9632,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362208,"statfs":{"total":96636764160,"available":96626900992,"internally_reserved":0,"allocated":811008,"data_stored":375878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1359999999999999,"15min":1.133},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.494,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":1.234},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2290000000000001,"15min":1.238},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.5189999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.6479999999999999}]},{"osd":1,"last update":"Wed Nov 8 08:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.1140000000000001,"15min":1.135},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.333,"5min":2.012,"15min":2.012},"last":1.149},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.151,"15min":1.1870000000000001},"min":{"1min":0.76000000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.395,"5min":1.667,"15min":1.667},"last":0.98699999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1850000000000001,"15min":1.196},"min":{"1min":0.86299999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.583,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.103},{"interface":"front","average":{"1min":1.05,"5min":1.177,"15min":1.2170000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.4630000000000001,"5min":1.796,"15min":1.796},"last":1.5860000000000001}]},{"osd":3,"last update":"Wed Nov 8 08:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.3120000000000001,"15min":1.3380000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5049999999999999,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.2849999999999999,"15min":1.325},"min":{"1min":0.81000000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.4490000000000001,"5min":3.7999999999999998,"15min":3.7999999999999998},"last":1.3029999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.157,"15min":1.2010000000000001},"min":{"1min":0.78300000000000003,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.609,"5min":1.629,"15min":1.629},"last":1.343},{"interface":"front","average":{"1min":1.089,"5min":1.27,"15min":1.333},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.4710000000000001,"5min":3.073,"15min":3.073},"last":1.7310000000000001}]},{"osd":6,"last update":"Wed Nov 8 08:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.276,"15min":1.341},"min":{"1min":0.85699999999999998,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.4279999999999999,"5min":3.8650000000000002,"15min":3.8650000000000002},"last":1.393},{"interface":"front","average":{"1min":1.173,"5min":1.327,"15min":1.3759999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.49,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.4259999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.4319999999999999,"15min":1.4830000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.5780000000000001,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.4279999999999999,"15min":1.4890000000000001},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.446,"5min":3.9369999999999998,"15min":3.9369999999999998},"last":1.5469999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9812,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362028,"statfs":{"total":96636764160,"available":96626716672,"internally_reserved":0,"allocated":798720,"data_stored":368930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.151,"15min":1.1459999999999999},"min":{"1min":0.73799999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.55,"5min":1.655,"15min":1.655},"last":1.28},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2250000000000001,"15min":1.234},"min":{"1min":0.82799999999999996,"5min":0.747,"15min":0.747},"max":{"1min":1.5700000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":1.3680000000000001}]},{"osd":1,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.071,"15min":1.0860000000000001},"min":{"1min":0.80300000000000005,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.397,"5min":1.99,"15min":1.99},"last":1.022},{"interface":"front","average":{"1min":1.109,"5min":1.1859999999999999,"15min":1.21},"min":{"1min":0.82899999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.4159999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.86599999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.137,"15min":1.129},"min":{"1min":0.70399999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.6240000000000001,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.171,"5min":1.2350000000000001,"15min":1.262},"min":{"1min":0.84199999999999997,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.4970000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.4039999999999999}]},{"osd":3,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.212,"15min":1.206},"min":{"1min":0.96199999999999997,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.6040000000000001,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":1.052},{"interface":"front","average":{"1min":1.181,"5min":1.2250000000000001,"15min":1.244},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.5429999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.3169999999999999}]},{"osd":5,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.262,"15min":1.2889999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.502,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2529999999999999,"15min":1.2569999999999999},"min":{"1min":0.77000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.6850000000000001,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":0.89800000000000002}]},{"osd":6,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.262,"15min":1.2809999999999999},"min":{"1min":0.92000000000000004,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.615,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.361,"15min":1.3700000000000001},"min":{"1min":0.83799999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.569,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.98299999999999998}]},{"osd":7,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4410000000000001},"min":{"1min":1.093,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.736,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.4530000000000001,"15min":1.4830000000000001},"min":{"1min":1.0880000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.5429999999999999,"5min":2.399,"15min":2.399},"last":1.2170000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9844,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361996,"statfs":{"total":96636764160,"available":96626683904,"internally_reserved":0,"allocated":831488,"data_stored":383438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.89300000000000002,"15min":0.90900000000000003},"min":{"1min":0.57199999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.151,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":1.298},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.91900000000000004,"15min":0.93799999999999994},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.151,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.1990000000000001}]},{"osd":1,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.91900000000000004,"15min":0.93700000000000006},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4550000000000001,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.016,"5min":1.101,"15min":1.1240000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.3129999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.4219999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.1080000000000001,"15min":1.1100000000000001},"min":{"1min":0.67200000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1910000000000001,"15min":1.2170000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.4390000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.736}]},{"osd":4,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.23,"15min":1.2929999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.3300000000000001,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.149},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.355,"15min":1.395},"min":{"1min":0.86199999999999999,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.8300000000000001,"5min":2.27,"15min":2.27},"last":1.2749999999999999}]},{"osd":5,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.4019999999999999,"15min":1.4379999999999999},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.5880000000000001,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.224},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.391,"15min":1.4259999999999999},"min":{"1min":0.92800000000000005,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.915,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":2.1160000000000001}]},{"osd":6,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.417,"15min":1.474},"min":{"1min":0.90200000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":3.0950000000000002,"5min":3.0950000000000002,"15min":3.0950000000000002},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.498,"15min":1.538},"min":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"max":{"1min":3.02,"5min":3.02,"15min":3.02},"last":1.4490000000000001}]},{"osd":7,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.5469999999999999,"15min":1.597},"min":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"max":{"1min":2.0110000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":2.0230000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.502,"15min":1.5820000000000001},"min":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":1.6890000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.339}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9656,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362184,"statfs":{"total":96636764160,"available":96626876416,"internally_reserved":0,"allocated":835584,"data_stored":384874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.91600000000000004,"15min":0.94199999999999995},"min":{"1min":0.623,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.153},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.98099999999999998,"15min":1.0009999999999999},"min":{"1min":0.64600000000000002,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5369999999999999,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.81100000000000005}]},{"osd":1,"last update":"Wed Nov 8 08:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.0600000000000001,"15min":1.105},"min":{"1min":0.75800000000000001,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.29,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.026,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.63500000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.5760000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.874}]},{"osd":3,"last update":"Wed Nov 8 08:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.99199999999999999,"15min":1.012},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.2569999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.2689999999999999,"15min":1.3420000000000001},"min":{"1min":0.71399999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.546,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":1.024}]},{"osd":4,"last update":"Wed Nov 8 08:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.3959999999999999,"15min":1.4690000000000001},"min":{"1min":0.82299999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.651,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.3300000000000001,"15min":1.365},"min":{"1min":0.91700000000000004,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.6859999999999999,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.069}]},{"osd":5,"last update":"Wed Nov 8 08:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3959999999999999,"15min":1.4359999999999999},"min":{"1min":1.081,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.708,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.331},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.337,"15min":1.349},"min":{"1min":0.93300000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.5209999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.202}]},{"osd":6,"last update":"Wed Nov 8 08:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.4099999999999999,"15min":1.429},"min":{"1min":1.0169999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.597,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.502,"15min":1.5780000000000001},"min":{"1min":0.86399999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.7869999999999999,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":1.5089999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.444,"15min":1.476},"min":{"1min":1.1619999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.7529999999999999,"5min":2.867,"15min":2.867},"last":1.444},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4770000000000001,"15min":1.4870000000000001},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.821,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.482}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9632,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362208,"statfs":{"total":96636764160,"available":96626900992,"internally_reserved":0,"allocated":811008,"data_stored":375878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.877,"15min":0.89500000000000002},"min":{"1min":0.56100000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.03,"15min":1.044},"min":{"1min":0.64500000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.1830000000000001}]},{"osd":2,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.96399999999999997,"15min":0.98899999999999999},"min":{"1min":0.73199999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.238,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.3},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.1399999999999999,"15min":1.1850000000000001},"min":{"1min":0.76000000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6439999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.256}]},{"osd":3,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0920000000000001,"15min":1.1060000000000001},"min":{"1min":0.72599999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7230000000000001,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.345},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.071,"15min":1.113},"min":{"1min":0.50600000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.7010000000000001,"5min":1.972,"15min":1.972},"last":0.92200000000000004}]},{"osd":4,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.2230000000000001,"15min":1.2330000000000001},"min":{"1min":0.93200000000000005,"5min":0.746,"15min":0.746},"max":{"1min":1.8160000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.3080000000000001,"15min":1.3440000000000001},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":1.9359999999999999,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":1.4390000000000001}]},{"osd":5,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3149999999999999,"15min":1.3360000000000001},"min":{"1min":1,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.7769999999999999,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.385},{"interface":"front","average":{"1min":1.222,"5min":1.3120000000000001,"15min":1.3440000000000001},"min":{"1min":0.83499999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.599,"5min":2.48,"15min":2.48},"last":1.4730000000000001}]},{"osd":6,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.425,"15min":1.4450000000000001},"min":{"1min":1.196,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.875,"5min":3.149,"15min":3.149},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.45,"15min":1.474},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":1.903,"5min":2.169,"15min":2.169},"last":1.502}]},{"osd":7,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.528,"15min":1.5640000000000001},"min":{"1min":1.2210000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"max":{"1min":2.0009999999999999,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.552},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.456,"15min":1.474},"min":{"1min":1.0469999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.7170000000000001,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.54}]}]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15076,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":13696,"kb_avail":94356764,"statfs":{"total":96636764160,"available":96621326336,"internally_reserved":0,"allocated":1404928,"data_stored":966246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14024704},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.93400000000000005,"15min":0.96999999999999997},"min":{"1min":0.625,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.5820000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.93200000000000005,"15min":0.96199999999999997},"min":{"1min":0.70899999999999996,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1950000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.93799999999999994}]},{"osd":2,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.97599999999999998,"15min":1.002},"min":{"1min":0.69299999999999995,"5min":0.497,"15min":0.497},"max":{"1min":1.323,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.024,"5min":1.081,"15min":1.111},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.0609999999999999}]},{"osd":3,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.111,"15min":1.1599999999999999},"min":{"1min":0.61499999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.103},{"interface":"front","average":{"1min":1.135,"5min":1.1659999999999999,"15min":1.2},"min":{"1min":0.69699999999999995,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.77,"5min":1.786,"15min":1.786},"last":1.1439999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.452,"15min":1.516},"min":{"1min":0.94799999999999995,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.909,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.4379999999999999,"15min":1.48},"min":{"1min":1.1619999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.6990000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.6739999999999999}]},{"osd":5,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.458,"15min":1.51},"min":{"1min":0.93100000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.012,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.536},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.5740000000000001,"15min":1.6479999999999999},"min":{"1min":1.105,"5min":1.0660000000000001,"15min":1.0660000000000001},"max":{"1min":2.1080000000000001,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.202}]},{"osd":6,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.468,"15min":1.53},"min":{"1min":0.94799999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.706,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.8879999999999999},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.53,"15min":1.581},"min":{"1min":1.117,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":1.857,"5min":2.048,"15min":2.048},"last":1.8089999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.585,"15min":1.651},"min":{"1min":1.069,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":1.9510000000000001,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.9139999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.613,"15min":1.6870000000000001},"min":{"1min":1.123,"5min":1.0169999999999999,"15min":1.0169999999999999},"max":{"1min":2.2069999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":1.29}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T08:22:25.167 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T08:22:25.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.383+0000 7f963965e700 1 Processor -- start 2023-11-08T08:22:25.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.383+0000 7f963965e700 1 -- start start 2023-11-08T08:22:25.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.383+0000 7f963965e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f963415a480 0x7f96340cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:25.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.383+0000 7f963965e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f96340c8d10 0x7f96340cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:25.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.383+0000 7f963965e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f96340ccd10 0x7f963415dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:25.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.383+0000 7f963965e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f963415a850 con 0x7f96340ccd10 2023-11-08T08:22:25.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.383+0000 7f963965e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96340cada0 con 0x7f96340c8d10 2023-11-08T08:22:25.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.383+0000 7f963965e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96340ce060 con 0x7f963415a480 2023-11-08T08:22:25.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.383+0000 7f96337fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f96340ccd10 0x7f963415dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:25.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.383+0000 7f96337fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f96340ccd10 0x7f963415dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35902/0 (socket says 172.21.15.40:35902) 2023-11-08T08:22:25.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.383+0000 7f96337fe700 1 -- 172.21.15.40:0/4217196872 learned_addr learned my addr 172.21.15.40:0/4217196872 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:25.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.383+0000 7f9632ffd700 1 --2- 172.21.15.40:0/4217196872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f963415a480 0x7f96340cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:25.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.383+0000 7f962bfff700 1 --2- 172.21.15.40:0/4217196872 >> v2:172.21.15.169:3300/0 conn(0x7f96340c8d10 0x7f96340cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:25.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.387+0000 7f96337fe700 1 -- 172.21.15.40:0/4217196872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f963415a480 msgr2=0x7f96340cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:25.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.387+0000 7f96337fe700 1 --2- 172.21.15.40:0/4217196872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f963415a480 0x7f96340cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.387+0000 7f96337fe700 1 -- 172.21.15.40:0/4217196872 >> v2:172.21.15.169:3300/0 conn(0x7f96340c8d10 msgr2=0x7f96340cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:25.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.387+0000 7f96337fe700 1 --2- 172.21.15.40:0/4217196872 >> v2:172.21.15.169:3300/0 conn(0x7f96340c8d10 0x7f96340cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.387+0000 7f96337fe700 1 -- 172.21.15.40:0/4217196872 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96340ce300 con 0x7f96340ccd10 2023-11-08T08:22:25.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.387+0000 7f96337fe700 1 --2- 172.21.15.40:0/4217196872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96340ccd10 0x7f963415dee0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f963406b8b0 tx=0x7f962c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a444b3b223d2bd24 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:25.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.387+0000 7f96327fc700 1 -- 172.21.15.40:0/4217196872 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c00e070 con 0x7f96340ccd10 2023-11-08T08:22:25.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.387+0000 7f96327fc700 1 -- 172.21.15.40:0/4217196872 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962c00b910 con 0x7f96340ccd10 2023-11-08T08:22:25.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.387+0000 7f96327fc700 1 -- 172.21.15.40:0/4217196872 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c00bcc0 con 0x7f96340ccd10 2023-11-08T08:22:25.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.387+0000 7f963965e700 1 -- 172.21.15.40:0/4217196872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96340ccd10 msgr2=0x7f963415dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:25.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.387+0000 7f963965e700 1 --2- 172.21.15.40:0/4217196872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96340ccd10 0x7f963415dee0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f963406b8b0 tx=0x7f962c00b330 comp rx=0 tx=0).stop 2023-11-08T08:22:25.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.387+0000 7f963965e700 1 -- 172.21.15.40:0/4217196872 shutdown_connections 2023-11-08T08:22:25.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.387+0000 7f963965e700 1 --2- 172.21.15.40:0/4217196872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96340ccd10 0x7f963415dee0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.387+0000 7f963965e700 1 --2- 172.21.15.40:0/4217196872 >> v2:172.21.15.169:3300/0 conn(0x7f96340c8d10 0x7f96340cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.387+0000 7f963965e700 1 --2- 172.21.15.40:0/4217196872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f963415a480 0x7f96340cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.387+0000 7f963965e700 1 -- 172.21.15.40:0/4217196872 >> 172.21.15.40:0/4217196872 conn(0x7f96340cf350 msgr2=0x7f96340b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:25.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.387+0000 7f963965e700 1 -- 172.21.15.40:0/4217196872 shutdown_connections 2023-11-08T08:22:25.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.387+0000 7f963965e700 1 -- 172.21.15.40:0/4217196872 wait complete. 2023-11-08T08:22:25.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.391+0000 7f963965e700 1 Processor -- start 2023-11-08T08:22:25.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.391+0000 7f963965e700 1 -- start start 2023-11-08T08:22:25.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.391+0000 7f963965e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96340c8d10 0x7f9634153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:25.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.391+0000 7f963965e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96340ccd10 0x7f96340b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:25.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.391+0000 7f963965e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f963415a480 0x7f96340b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:25.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.391+0000 7f963965e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96340c85a0 con 0x7f963415a480 2023-11-08T08:22:25.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.391+0000 7f963965e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96340c6910 con 0x7f96340c8d10 2023-11-08T08:22:25.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.391+0000 7f963965e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96340c6bb0 con 0x7f96340ccd10 2023-11-08T08:22:25.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.391+0000 7f96337fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f963415a480 0x7f96340b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:25.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.391+0000 7f96337fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f963415a480 0x7f96340b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35910/0 (socket says 172.21.15.40:35910) 2023-11-08T08:22:25.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.391+0000 7f96337fe700 1 -- 172.21.15.40:0/2539622170 learned_addr learned my addr 172.21.15.40:0/2539622170 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:25.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.391+0000 7f962bfff700 1 --2- 172.21.15.40:0/2539622170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96340ccd10 0x7f96340b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:25.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.391+0000 7f9632ffd700 1 --2- 172.21.15.40:0/2539622170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96340c8d10 0x7f9634153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:25.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.395+0000 7f96337fe700 1 -- 172.21.15.40:0/2539622170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96340ccd10 msgr2=0x7f96340b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:25.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.395+0000 7f96337fe700 1 --2- 172.21.15.40:0/2539622170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96340ccd10 0x7f96340b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.395+0000 7f96337fe700 1 -- 172.21.15.40:0/2539622170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96340c8d10 msgr2=0x7f9634153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:25.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.395+0000 7f96337fe700 1 --2- 172.21.15.40:0/2539622170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96340c8d10 0x7f9634153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.395+0000 7f96337fe700 1 -- 172.21.15.40:0/2539622170 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c009b30 con 0x7f963415a480 2023-11-08T08:22:25.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.395+0000 7f96337fe700 1 --2- 172.21.15.40:0/2539622170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f963415a480 0x7f96340b3180 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f962c00b370 tx=0x7f962c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:25.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.395+0000 7f962b7fe700 1 -- 172.21.15.40:0/2539622170 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c0099b0 con 0x7f963415a480 2023-11-08T08:22:25.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.395+0000 7f962b7fe700 1 -- 172.21.15.40:0/2539622170 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962c0092e0 con 0x7f963415a480 2023-11-08T08:22:25.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.395+0000 7f962b7fe700 1 -- 172.21.15.40:0/2539622170 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c004450 con 0x7f963415a480 2023-11-08T08:22:25.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.395+0000 7f963965e700 1 -- 172.21.15.40:0/2539622170 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96340add50 con 0x7f963415a480 2023-11-08T08:22:25.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.395+0000 7f963965e700 1 -- 172.21.15.40:0/2539622170 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96340caa30 con 0x7f963415a480 2023-11-08T08:22:25.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.395+0000 7f963965e700 1 -- 172.21.15.40:0/2539622170 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96340ca6e0 con 0x7f963415a480 2023-11-08T08:22:25.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.399+0000 7f962b7fe700 1 -- 172.21.15.40:0/2539622170 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f962c0045b0 con 0x7f963415a480 2023-11-08T08:22:25.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.399+0000 7f962b7fe700 1 --2- 172.21.15.40:0/2539622170 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9614064ee0 0x7f9614067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:25.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.399+0000 7f962b7fe700 1 -- 172.21.15.40:0/2539622170 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f962c083530 con 0x7f963415a480 2023-11-08T08:22:25.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.399+0000 7f9632ffd700 1 --2- 172.21.15.40:0/2539622170 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9614064ee0 0x7f9614067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:25.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.403+0000 7f9632ffd700 1 --2- 172.21.15.40:0/2539622170 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9614064ee0 0x7f9614067390 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9624009b50 tx=0x7f9624009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:25.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.403+0000 7f962b7fe700 1 -- 172.21.15.40:0/2539622170 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f962c0514d0 con 0x7f963415a480 2023-11-08T08:22:25.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.575+0000 7f963965e700 1 -- 172.21.15.40:0/2539622170 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f96340726c0 con 0x7f9614064ee0 2023-11-08T08:22:25.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.595+0000 7f962b7fe700 1 -- 172.21.15.40:0/2539622170 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309194 (secure 0 0 0) 0x7f96340726c0 con 0x7f9614064ee0 2023-11-08T08:22:25.606 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:22:25.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.603+0000 7f963965e700 1 -- 172.21.15.40:0/2539622170 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9614064ee0 msgr2=0x7f9614067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:25.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.603+0000 7f963965e700 1 --2- 172.21.15.40:0/2539622170 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9614064ee0 0x7f9614067390 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9624009b50 tx=0x7f9624009240 comp rx=0 tx=0).stop 2023-11-08T08:22:25.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.603+0000 7f963965e700 1 -- 172.21.15.40:0/2539622170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f963415a480 msgr2=0x7f96340b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.603+0000 7f963965e700 1 --2- 172.21.15.40:0/2539622170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f963415a480 0x7f96340b3180 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f962c00b370 tx=0x7f962c004af0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.603+0000 7f963965e700 1 -- 172.21.15.40:0/2539622170 shutdown_connections 2023-11-08T08:22:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.603+0000 7f963965e700 1 --2- 172.21.15.40:0/2539622170 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9614064ee0 0x7f9614067390 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.603+0000 7f963965e700 1 --2- 172.21.15.40:0/2539622170 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f963415a480 0x7f96340b3180 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.603+0000 7f963965e700 1 --2- 172.21.15.40:0/2539622170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96340ccd10 0x7f96340b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.603+0000 7f963965e700 1 --2- 172.21.15.40:0/2539622170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96340c8d10 0x7f9634153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.603+0000 7f963965e700 1 -- 172.21.15.40:0/2539622170 >> 172.21.15.40:0/2539622170 conn(0x7f96340cf350 msgr2=0x7f96340aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:25.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.603+0000 7f963965e700 1 -- 172.21.15.40:0/2539622170 shutdown_connections 2023-11-08T08:22:25.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.603+0000 7f963965e700 1 -- 172.21.15.40:0/2539622170 wait complete. 2023-11-08T08:22:25.612 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-11-08T08:22:25.630 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2023-11-08T08:22:23.955960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":88104,"kb_used_data":7528,"kb_used_omap":0,"kb_used_meta":80512,"kb_avail":754886616,"statfs":{"total":773094113280,"available":773003894784,"internally_reserved":0,"allocated":7708672,"data_stored":4197368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":82444288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":52,"apply_latency_ms":52,"commit_latency_ns":52000000,"apply_latency_ns":52000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.788665"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.699681+0000","last_change":"2023-11-08T08:22:20.693079+0000","last_active":"2023-11-08T08:22:20.699681+0000","last_peered":"2023-11-08T08:22:20.699681+0000","last_clean":"2023-11-08T08:22:20.699681+0000","last_became_active":"2023-11-08T08:22:20.692841+0000","last_became_peered":"2023-11-08T08:22:20.692841+0000","last_unstale":"2023-11-08T08:22:20.699681+0000","last_undegraded":"2023-11-08T08:22:20.699681+0000","last_fullsized":"2023-11-08T08:22:20.699681+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:35:24.924456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703941+0000","last_change":"2023-11-08T08:22:20.695194+0000","last_active":"2023-11-08T08:22:20.703941+0000","last_peered":"2023-11-08T08:22:20.703941+0000","last_clean":"2023-11-08T08:22:20.703941+0000","last_became_active":"2023-11-08T08:22:20.694528+0000","last_became_peered":"2023-11-08T08:22:20.694528+0000","last_unstale":"2023-11-08T08:22:20.703941+0000","last_undegraded":"2023-11-08T08:22:20.703941+0000","last_fullsized":"2023-11-08T08:22:20.703941+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:16:07.614218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.701955+0000","last_change":"2023-11-08T08:22:20.688375+0000","last_active":"2023-11-08T08:22:20.701955+0000","last_peered":"2023-11-08T08:22:20.701955+0000","last_clean":"2023-11-08T08:22:20.701955+0000","last_became_active":"2023-11-08T08:22:20.687746+0000","last_became_peered":"2023-11-08T08:22:20.687746+0000","last_unstale":"2023-11-08T08:22:20.701955+0000","last_undegraded":"2023-11-08T08:22:20.701955+0000","last_fullsized":"2023-11-08T08:22:20.701955+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:29:48.111099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.699690+0000","last_change":"2023-11-08T08:22:20.695073+0000","last_active":"2023-11-08T08:22:20.699690+0000","last_peered":"2023-11-08T08:22:20.699690+0000","last_clean":"2023-11-08T08:22:20.699690+0000","last_became_active":"2023-11-08T08:22:20.694827+0000","last_became_peered":"2023-11-08T08:22:20.694827+0000","last_unstale":"2023-11-08T08:22:20.699690+0000","last_undegraded":"2023-11-08T08:22:20.699690+0000","last_fullsized":"2023-11-08T08:22:20.699690+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:00:18.591720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.695879+0000","last_change":"2023-11-08T08:22:20.684638+0000","last_active":"2023-11-08T08:22:20.695879+0000","last_peered":"2023-11-08T08:22:20.695879+0000","last_clean":"2023-11-08T08:22:20.695879+0000","last_became_active":"2023-11-08T08:22:20.684115+0000","last_became_peered":"2023-11-08T08:22:20.684115+0000","last_unstale":"2023-11-08T08:22:20.695879+0000","last_undegraded":"2023-11-08T08:22:20.695879+0000","last_fullsized":"2023-11-08T08:22:20.695879+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:20:55.507408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.702788+0000","last_change":"2023-11-08T08:22:20.688220+0000","last_active":"2023-11-08T08:22:20.702788+0000","last_peered":"2023-11-08T08:22:20.702788+0000","last_clean":"2023-11-08T08:22:20.702788+0000","last_became_active":"2023-11-08T08:22:20.687910+0000","last_became_peered":"2023-11-08T08:22:20.687910+0000","last_unstale":"2023-11-08T08:22:20.702788+0000","last_undegraded":"2023-11-08T08:22:20.702788+0000","last_fullsized":"2023-11-08T08:22:20.702788+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:40:54.193871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.700373+0000","last_change":"2023-11-08T08:22:20.694001+0000","last_active":"2023-11-08T08:22:20.700373+0000","last_peered":"2023-11-08T08:22:20.700373+0000","last_clean":"2023-11-08T08:22:20.700373+0000","last_became_active":"2023-11-08T08:22:20.693582+0000","last_became_peered":"2023-11-08T08:22:20.693582+0000","last_unstale":"2023-11-08T08:22:20.700373+0000","last_undegraded":"2023-11-08T08:22:20.700373+0000","last_fullsized":"2023-11-08T08:22:20.700373+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:54:34.009215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.702918+0000","last_change":"2023-11-08T08:22:20.688974+0000","last_active":"2023-11-08T08:22:20.702918+0000","last_peered":"2023-11-08T08:22:20.702918+0000","last_clean":"2023-11-08T08:22:20.702918+0000","last_became_active":"2023-11-08T08:22:20.688735+0000","last_became_peered":"2023-11-08T08:22:20.688735+0000","last_unstale":"2023-11-08T08:22:20.702918+0000","last_undegraded":"2023-11-08T08:22:20.702918+0000","last_fullsized":"2023-11-08T08:22:20.702918+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:06:31.570228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703678+0000","last_change":"2023-11-08T08:22:20.698657+0000","last_active":"2023-11-08T08:22:20.703678+0000","last_peered":"2023-11-08T08:22:20.703678+0000","last_clean":"2023-11-08T08:22:20.703678+0000","last_became_active":"2023-11-08T08:22:20.697916+0000","last_became_peered":"2023-11-08T08:22:20.697916+0000","last_unstale":"2023-11-08T08:22:20.703678+0000","last_undegraded":"2023-11-08T08:22:20.703678+0000","last_fullsized":"2023-11-08T08:22:20.703678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:51.802894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705433+0000","last_change":"2023-11-08T08:22:20.696337+0000","last_active":"2023-11-08T08:22:20.705433+0000","last_peered":"2023-11-08T08:22:20.705433+0000","last_clean":"2023-11-08T08:22:20.705433+0000","last_became_active":"2023-11-08T08:22:20.694555+0000","last_became_peered":"2023-11-08T08:22:20.694555+0000","last_unstale":"2023-11-08T08:22:20.705433+0000","last_undegraded":"2023-11-08T08:22:20.705433+0000","last_fullsized":"2023-11-08T08:22:20.705433+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:55:16.752604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.702650+0000","last_change":"2023-11-08T08:22:20.694924+0000","last_active":"2023-11-08T08:22:20.702650+0000","last_peered":"2023-11-08T08:22:20.702650+0000","last_clean":"2023-11-08T08:22:20.702650+0000","last_became_active":"2023-11-08T08:22:20.692623+0000","last_became_peered":"2023-11-08T08:22:20.692623+0000","last_unstale":"2023-11-08T08:22:20.702650+0000","last_undegraded":"2023-11-08T08:22:20.702650+0000","last_fullsized":"2023-11-08T08:22:20.702650+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:02.368358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.699674+0000","last_change":"2023-11-08T08:22:20.692162+0000","last_active":"2023-11-08T08:22:20.699674+0000","last_peered":"2023-11-08T08:22:20.699674+0000","last_clean":"2023-11-08T08:22:20.699674+0000","last_became_active":"2023-11-08T08:22:20.691560+0000","last_became_peered":"2023-11-08T08:22:20.691560+0000","last_unstale":"2023-11-08T08:22:20.699674+0000","last_undegraded":"2023-11-08T08:22:20.699674+0000","last_fullsized":"2023-11-08T08:22:20.699674+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:30:07.090301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.692248+0000","last_change":"2023-11-08T08:22:20.682225+0000","last_active":"2023-11-08T08:22:20.692248+0000","last_peered":"2023-11-08T08:22:20.692248+0000","last_clean":"2023-11-08T08:22:20.692248+0000","last_became_active":"2023-11-08T08:22:20.681934+0000","last_became_peered":"2023-11-08T08:22:20.681934+0000","last_unstale":"2023-11-08T08:22:20.692248+0000","last_undegraded":"2023-11-08T08:22:20.692248+0000","last_fullsized":"2023-11-08T08:22:20.692248+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:33:21.532456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703810+0000","last_change":"2023-11-08T08:22:20.701356+0000","last_active":"2023-11-08T08:22:20.703810+0000","last_peered":"2023-11-08T08:22:20.703810+0000","last_clean":"2023-11-08T08:22:20.703810+0000","last_became_active":"2023-11-08T08:22:20.701126+0000","last_became_peered":"2023-11-08T08:22:20.701126+0000","last_unstale":"2023-11-08T08:22:20.703810+0000","last_undegraded":"2023-11-08T08:22:20.703810+0000","last_fullsized":"2023-11-08T08:22:20.703810+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:38:00.466999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.692508+0000","last_change":"2023-11-08T08:22:20.685750+0000","last_active":"2023-11-08T08:22:20.692508+0000","last_peered":"2023-11-08T08:22:20.692508+0000","last_clean":"2023-11-08T08:22:20.692508+0000","last_became_active":"2023-11-08T08:22:20.685489+0000","last_became_peered":"2023-11-08T08:22:20.685489+0000","last_unstale":"2023-11-08T08:22:20.692508+0000","last_undegraded":"2023-11-08T08:22:20.692508+0000","last_fullsized":"2023-11-08T08:22:20.692508+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:29:48.111099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.695824+0000","last_change":"2023-11-08T08:22:20.682298+0000","last_active":"2023-11-08T08:22:20.695824+0000","last_peered":"2023-11-08T08:22:20.695824+0000","last_clean":"2023-11-08T08:22:20.695824+0000","last_became_active":"2023-11-08T08:22:20.682033+0000","last_became_peered":"2023-11-08T08:22:20.682033+0000","last_unstale":"2023-11-08T08:22:20.695824+0000","last_undegraded":"2023-11-08T08:22:20.695824+0000","last_fullsized":"2023-11-08T08:22:20.695824+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:42:59.040872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703643+0000","last_change":"2023-11-08T08:22:20.698493+0000","last_active":"2023-11-08T08:22:20.703643+0000","last_peered":"2023-11-08T08:22:20.703643+0000","last_clean":"2023-11-08T08:22:20.703643+0000","last_became_active":"2023-11-08T08:22:20.697582+0000","last_became_peered":"2023-11-08T08:22:20.697582+0000","last_unstale":"2023-11-08T08:22:20.703643+0000","last_undegraded":"2023-11-08T08:22:20.703643+0000","last_fullsized":"2023-11-08T08:22:20.703643+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:05:14.039538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.699885+0000","last_change":"2023-11-08T08:22:20.695572+0000","last_active":"2023-11-08T08:22:20.699885+0000","last_peered":"2023-11-08T08:22:20.699885+0000","last_clean":"2023-11-08T08:22:20.699885+0000","last_became_active":"2023-11-08T08:22:20.695336+0000","last_became_peered":"2023-11-08T08:22:20.695336+0000","last_unstale":"2023-11-08T08:22:20.699885+0000","last_undegraded":"2023-11-08T08:22:20.699885+0000","last_fullsized":"2023-11-08T08:22:20.699885+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:50:59.392279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703243+0000","last_change":"2023-11-08T08:22:20.691681+0000","last_active":"2023-11-08T08:22:20.703243+0000","last_peered":"2023-11-08T08:22:20.703243+0000","last_clean":"2023-11-08T08:22:20.703243+0000","last_became_active":"2023-11-08T08:22:20.691442+0000","last_became_peered":"2023-11-08T08:22:20.691442+0000","last_unstale":"2023-11-08T08:22:20.703243+0000","last_undegraded":"2023-11-08T08:22:20.703243+0000","last_fullsized":"2023-11-08T08:22:20.703243+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:31:36.439012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.701444+0000","last_change":"2023-11-08T08:22:20.697764+0000","last_active":"2023-11-08T08:22:20.701444+0000","last_peered":"2023-11-08T08:22:20.701444+0000","last_clean":"2023-11-08T08:22:20.701444+0000","last_became_active":"2023-11-08T08:22:20.697433+0000","last_became_peered":"2023-11-08T08:22:20.697433+0000","last_unstale":"2023-11-08T08:22:20.701444+0000","last_undegraded":"2023-11-08T08:22:20.701444+0000","last_fullsized":"2023-11-08T08:22:20.701444+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:52:08.554540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705173+0000","last_change":"2023-11-08T08:22:20.692276+0000","last_active":"2023-11-08T08:22:20.705173+0000","last_peered":"2023-11-08T08:22:20.705173+0000","last_clean":"2023-11-08T08:22:20.705173+0000","last_became_active":"2023-11-08T08:22:20.691566+0000","last_became_peered":"2023-11-08T08:22:20.691566+0000","last_unstale":"2023-11-08T08:22:20.705173+0000","last_undegraded":"2023-11-08T08:22:20.705173+0000","last_fullsized":"2023-11-08T08:22:20.705173+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:05.120014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705489+0000","last_change":"2023-11-08T08:22:20.697498+0000","last_active":"2023-11-08T08:22:20.705489+0000","last_peered":"2023-11-08T08:22:20.705489+0000","last_clean":"2023-11-08T08:22:20.705489+0000","last_became_active":"2023-11-08T08:22:20.693981+0000","last_became_peered":"2023-11-08T08:22:20.693981+0000","last_unstale":"2023-11-08T08:22:20.705489+0000","last_undegraded":"2023-11-08T08:22:20.705489+0000","last_fullsized":"2023-11-08T08:22:20.705489+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:06:39.151828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.699439+0000","last_change":"2023-11-08T08:22:20.690864+0000","last_active":"2023-11-08T08:22:20.699439+0000","last_peered":"2023-11-08T08:22:20.699439+0000","last_clean":"2023-11-08T08:22:20.699439+0000","last_became_active":"2023-11-08T08:22:20.690508+0000","last_became_peered":"2023-11-08T08:22:20.690508+0000","last_unstale":"2023-11-08T08:22:20.699439+0000","last_undegraded":"2023-11-08T08:22:20.699439+0000","last_fullsized":"2023-11-08T08:22:20.699439+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:04:06.670822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.689230+0000","last_change":"2023-11-08T08:22:20.687094+0000","last_active":"2023-11-08T08:22:20.689230+0000","last_peered":"2023-11-08T08:22:20.689230+0000","last_clean":"2023-11-08T08:22:20.689230+0000","last_became_active":"2023-11-08T08:22:20.686819+0000","last_became_peered":"2023-11-08T08:22:20.686819+0000","last_unstale":"2023-11-08T08:22:20.689230+0000","last_undegraded":"2023-11-08T08:22:20.689230+0000","last_fullsized":"2023-11-08T08:22:20.689230+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:32:32.438878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.695591+0000","last_change":"2023-11-08T08:22:20.679585+0000","last_active":"2023-11-08T08:22:20.695591+0000","last_peered":"2023-11-08T08:22:20.695591+0000","last_clean":"2023-11-08T08:22:20.695591+0000","last_became_active":"2023-11-08T08:22:20.679332+0000","last_became_peered":"2023-11-08T08:22:20.679332+0000","last_unstale":"2023-11-08T08:22:20.695591+0000","last_undegraded":"2023-11-08T08:22:20.695591+0000","last_fullsized":"2023-11-08T08:22:20.695591+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:36:46.493034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.706663+0000","last_change":"2023-11-08T08:22:20.703721+0000","last_active":"2023-11-08T08:22:20.706663+0000","last_peered":"2023-11-08T08:22:20.706663+0000","last_clean":"2023-11-08T08:22:20.706663+0000","last_became_active":"2023-11-08T08:22:20.703468+0000","last_became_peered":"2023-11-08T08:22:20.703468+0000","last_unstale":"2023-11-08T08:22:20.706663+0000","last_undegraded":"2023-11-08T08:22:20.706663+0000","last_fullsized":"2023-11-08T08:22:20.706663+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:47:00.548303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.701974+0000","last_change":"2023-11-08T08:22:20.688538+0000","last_active":"2023-11-08T08:22:20.701974+0000","last_peered":"2023-11-08T08:22:20.701974+0000","last_clean":"2023-11-08T08:22:20.701974+0000","last_became_active":"2023-11-08T08:22:20.688131+0000","last_became_peered":"2023-11-08T08:22:20.688131+0000","last_unstale":"2023-11-08T08:22:20.701974+0000","last_undegraded":"2023-11-08T08:22:20.701974+0000","last_fullsized":"2023-11-08T08:22:20.701974+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:33:21.532456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705407+0000","last_change":"2023-11-08T08:22:20.694494+0000","last_active":"2023-11-08T08:22:20.705407+0000","last_peered":"2023-11-08T08:22:20.705407+0000","last_clean":"2023-11-08T08:22:20.705407+0000","last_became_active":"2023-11-08T08:22:20.693971+0000","last_became_peered":"2023-11-08T08:22:20.693971+0000","last_unstale":"2023-11-08T08:22:20.705407+0000","last_undegraded":"2023-11-08T08:22:20.705407+0000","last_fullsized":"2023-11-08T08:22:20.705407+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:39:28.160795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.704273+0000","last_change":"2023-11-08T08:22:20.701051+0000","last_active":"2023-11-08T08:22:20.704273+0000","last_peered":"2023-11-08T08:22:20.704273+0000","last_clean":"2023-11-08T08:22:20.704273+0000","last_became_active":"2023-11-08T08:22:20.700810+0000","last_became_peered":"2023-11-08T08:22:20.700810+0000","last_unstale":"2023-11-08T08:22:20.704273+0000","last_undegraded":"2023-11-08T08:22:20.704273+0000","last_fullsized":"2023-11-08T08:22:20.704273+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:07:58.205197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705757+0000","last_change":"2023-11-08T08:22:20.699818+0000","last_active":"2023-11-08T08:22:20.705757+0000","last_peered":"2023-11-08T08:22:20.705757+0000","last_clean":"2023-11-08T08:22:20.705757+0000","last_became_active":"2023-11-08T08:22:20.698491+0000","last_became_peered":"2023-11-08T08:22:20.698491+0000","last_unstale":"2023-11-08T08:22:20.705757+0000","last_undegraded":"2023-11-08T08:22:20.705757+0000","last_fullsized":"2023-11-08T08:22:20.705757+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:24:54.219012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703400+0000","last_change":"2023-11-08T08:22:20.697645+0000","last_active":"2023-11-08T08:22:20.703400+0000","last_peered":"2023-11-08T08:22:20.703400+0000","last_clean":"2023-11-08T08:22:20.703400+0000","last_became_active":"2023-11-08T08:22:20.697370+0000","last_became_peered":"2023-11-08T08:22:20.697370+0000","last_unstale":"2023-11-08T08:22:20.703400+0000","last_undegraded":"2023-11-08T08:22:20.703400+0000","last_fullsized":"2023-11-08T08:22:20.703400+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:02.368358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.695382+0000","last_change":"2023-11-08T08:22:20.680839+0000","last_active":"2023-11-08T08:22:20.695382+0000","last_peered":"2023-11-08T08:22:20.695382+0000","last_clean":"2023-11-08T08:22:20.695382+0000","last_became_active":"2023-11-08T08:22:20.680508+0000","last_became_peered":"2023-11-08T08:22:20.680508+0000","last_unstale":"2023-11-08T08:22:20.695382+0000","last_undegraded":"2023-11-08T08:22:20.695382+0000","last_fullsized":"2023-11-08T08:22:20.695382+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:50:50.206439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.699429+0000","last_change":"2023-11-08T08:22:20.690797+0000","last_active":"2023-11-08T08:22:20.699429+0000","last_peered":"2023-11-08T08:22:20.699429+0000","last_clean":"2023-11-08T08:22:20.699429+0000","last_became_active":"2023-11-08T08:22:20.689924+0000","last_became_peered":"2023-11-08T08:22:20.689924+0000","last_unstale":"2023-11-08T08:22:20.699429+0000","last_undegraded":"2023-11-08T08:22:20.699429+0000","last_fullsized":"2023-11-08T08:22:20.699429+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:15:39.778545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.696732+0000","last_change":"2023-11-08T08:22:20.213448+0000","last_active":"2023-11-08T08:22:20.696732+0000","last_peered":"2023-11-08T08:22:20.696732+0000","last_clean":"2023-11-08T08:22:20.696732+0000","last_became_active":"2023-11-08T08:22:20.213166+0000","last_became_peered":"2023-11-08T08:22:20.213166+0000","last_unstale":"2023-11-08T08:22:20.696732+0000","last_undegraded":"2023-11-08T08:22:20.696732+0000","last_fullsized":"2023-11-08T08:22:20.696732+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:30:48.040211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703660+0000","last_change":"2023-11-08T08:22:20.699394+0000","last_active":"2023-11-08T08:22:20.703660+0000","last_peered":"2023-11-08T08:22:20.703660+0000","last_clean":"2023-11-08T08:22:20.703660+0000","last_became_active":"2023-11-08T08:22:20.698235+0000","last_became_peered":"2023-11-08T08:22:20.698235+0000","last_unstale":"2023-11-08T08:22:20.703660+0000","last_undegraded":"2023-11-08T08:22:20.703660+0000","last_fullsized":"2023-11-08T08:22:20.703660+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:11:49.456593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.688674+0000","last_change":"2023-11-08T08:22:20.195444+0000","last_active":"2023-11-08T08:22:20.688674+0000","last_peered":"2023-11-08T08:22:20.688674+0000","last_clean":"2023-11-08T08:22:20.688674+0000","last_became_active":"2023-11-08T08:22:20.195150+0000","last_became_peered":"2023-11-08T08:22:20.195150+0000","last_unstale":"2023-11-08T08:22:20.688674+0000","last_undegraded":"2023-11-08T08:22:20.688674+0000","last_fullsized":"2023-11-08T08:22:20.688674+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:46:44.381504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.702965+0000","last_change":"2023-11-08T08:22:20.689488+0000","last_active":"2023-11-08T08:22:20.702965+0000","last_peered":"2023-11-08T08:22:20.702965+0000","last_clean":"2023-11-08T08:22:20.702965+0000","last_became_active":"2023-11-08T08:22:20.689246+0000","last_became_peered":"2023-11-08T08:22:20.689246+0000","last_unstale":"2023-11-08T08:22:20.702965+0000","last_undegraded":"2023-11-08T08:22:20.702965+0000","last_fullsized":"2023-11-08T08:22:20.702965+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:36:04.953710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.704178+0000","last_change":"2023-11-08T08:22:20.197944+0000","last_active":"2023-11-08T08:22:20.704178+0000","last_peered":"2023-11-08T08:22:20.704178+0000","last_clean":"2023-11-08T08:22:20.704178+0000","last_became_active":"2023-11-08T08:22:20.197436+0000","last_became_peered":"2023-11-08T08:22:20.197436+0000","last_unstale":"2023-11-08T08:22:20.704178+0000","last_undegraded":"2023-11-08T08:22:20.704178+0000","last_fullsized":"2023-11-08T08:22:20.704178+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:15:38.772418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.696783+0000","last_change":"2023-11-08T08:22:20.688734+0000","last_active":"2023-11-08T08:22:20.696783+0000","last_peered":"2023-11-08T08:22:20.696783+0000","last_clean":"2023-11-08T08:22:20.696783+0000","last_became_active":"2023-11-08T08:22:20.688431+0000","last_became_peered":"2023-11-08T08:22:20.688431+0000","last_unstale":"2023-11-08T08:22:20.696783+0000","last_undegraded":"2023-11-08T08:22:20.696783+0000","last_fullsized":"2023-11-08T08:22:20.696783+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:24:54.219012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.701888+0000","last_change":"2023-11-08T08:22:20.205863+0000","last_active":"2023-11-08T08:22:20.701888+0000","last_peered":"2023-11-08T08:22:20.701888+0000","last_clean":"2023-11-08T08:22:20.701888+0000","last_became_active":"2023-11-08T08:22:20.205092+0000","last_became_peered":"2023-11-08T08:22:20.205092+0000","last_unstale":"2023-11-08T08:22:20.701888+0000","last_undegraded":"2023-11-08T08:22:20.701888+0000","last_fullsized":"2023-11-08T08:22:20.701888+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:37:59.460872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705849+0000","last_change":"2023-11-08T08:22:20.701106+0000","last_active":"2023-11-08T08:22:20.705849+0000","last_peered":"2023-11-08T08:22:20.705849+0000","last_clean":"2023-11-08T08:22:20.705849+0000","last_became_active":"2023-11-08T08:22:20.700839+0000","last_became_peered":"2023-11-08T08:22:20.700839+0000","last_unstale":"2023-11-08T08:22:20.705849+0000","last_undegraded":"2023-11-08T08:22:20.705849+0000","last_fullsized":"2023-11-08T08:22:20.705849+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:15:39.778545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"68'1","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.689146+0000","last_change":"2023-11-08T08:22:20.205809+0000","last_active":"2023-11-08T08:22:20.689146+0000","last_peered":"2023-11-08T08:22:20.689146+0000","last_clean":"2023-11-08T08:22:20.689146+0000","last_became_active":"2023-11-08T08:22:20.205342+0000","last_became_peered":"2023-11-08T08:22:20.205342+0000","last_unstale":"2023-11-08T08:22:20.689146+0000","last_undegraded":"2023-11-08T08:22:20.689146+0000","last_fullsized":"2023-11-08T08:22:20.689146+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:38:15.497203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.695199+0000","last_change":"2023-11-08T08:22:20.678975+0000","last_active":"2023-11-08T08:22:20.695199+0000","last_peered":"2023-11-08T08:22:20.695199+0000","last_clean":"2023-11-08T08:22:20.695199+0000","last_became_active":"2023-11-08T08:22:20.678710+0000","last_became_peered":"2023-11-08T08:22:20.678710+0000","last_unstale":"2023-11-08T08:22:20.695199+0000","last_undegraded":"2023-11-08T08:22:20.695199+0000","last_fullsized":"2023-11-08T08:22:20.695199+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:29:48.111099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.702638+0000","last_change":"2023-11-08T08:22:20.196451+0000","last_active":"2023-11-08T08:22:20.702638+0000","last_peered":"2023-11-08T08:22:20.702638+0000","last_clean":"2023-11-08T08:22:20.702638+0000","last_became_active":"2023-11-08T08:22:20.196177+0000","last_became_peered":"2023-11-08T08:22:20.196177+0000","last_unstale":"2023-11-08T08:22:20.702638+0000","last_undegraded":"2023-11-08T08:22:20.702638+0000","last_fullsized":"2023-11-08T08:22:20.702638+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:50:58.386151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.695560+0000","last_change":"2023-11-08T08:22:20.684815+0000","last_active":"2023-11-08T08:22:20.695560+0000","last_peered":"2023-11-08T08:22:20.695560+0000","last_clean":"2023-11-08T08:22:20.695560+0000","last_became_active":"2023-11-08T08:22:20.684564+0000","last_became_peered":"2023-11-08T08:22:20.684564+0000","last_unstale":"2023-11-08T08:22:20.695560+0000","last_undegraded":"2023-11-08T08:22:20.695560+0000","last_fullsized":"2023-11-08T08:22:20.695560+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:53.594420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.681503+0000","last_change":"2023-11-08T08:22:20.203943+0000","last_active":"2023-11-08T08:22:20.681503+0000","last_peered":"2023-11-08T08:22:20.681503+0000","last_clean":"2023-11-08T08:22:20.681503+0000","last_became_active":"2023-11-08T08:22:20.203563+0000","last_became_peered":"2023-11-08T08:22:20.203563+0000","last_unstale":"2023-11-08T08:22:20.681503+0000","last_undegraded":"2023-11-08T08:22:20.681503+0000","last_fullsized":"2023-11-08T08:22:20.681503+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:21:31.953212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.689176+0000","last_change":"2023-11-08T08:22:20.673450+0000","last_active":"2023-11-08T08:22:20.689176+0000","last_peered":"2023-11-08T08:22:20.689176+0000","last_clean":"2023-11-08T08:22:20.689176+0000","last_became_active":"2023-11-08T08:22:20.672934+0000","last_became_peered":"2023-11-08T08:22:20.672934+0000","last_unstale":"2023-11-08T08:22:20.689176+0000","last_undegraded":"2023-11-08T08:22:20.689176+0000","last_fullsized":"2023-11-08T08:22:20.689176+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:44:54.466782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.676558+0000","last_change":"2023-11-08T08:22:20.201122+0000","last_active":"2023-11-08T08:22:20.676558+0000","last_peered":"2023-11-08T08:22:20.676558+0000","last_clean":"2023-11-08T08:22:20.676558+0000","last_became_active":"2023-11-08T08:22:20.200111+0000","last_became_peered":"2023-11-08T08:22:20.200111+0000","last_unstale":"2023-11-08T08:22:20.676558+0000","last_undegraded":"2023-11-08T08:22:20.676558+0000","last_fullsized":"2023-11-08T08:22:20.676558+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:35:01.443129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705890+0000","last_change":"2023-11-08T08:22:20.701546+0000","last_active":"2023-11-08T08:22:20.705890+0000","last_peered":"2023-11-08T08:22:20.705890+0000","last_clean":"2023-11-08T08:22:20.705890+0000","last_became_active":"2023-11-08T08:22:20.700917+0000","last_became_peered":"2023-11-08T08:22:20.700917+0000","last_unstale":"2023-11-08T08:22:20.705890+0000","last_undegraded":"2023-11-08T08:22:20.705890+0000","last_fullsized":"2023-11-08T08:22:20.705890+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:32:32.438878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.671635+0000","last_change":"2023-11-08T08:22:20.200725+0000","last_active":"2023-11-08T08:22:20.671635+0000","last_peered":"2023-11-08T08:22:20.671635+0000","last_clean":"2023-11-08T08:22:20.671635+0000","last_became_active":"2023-11-08T08:22:20.200248+0000","last_became_peered":"2023-11-08T08:22:20.200248+0000","last_unstale":"2023-11-08T08:22:20.671635+0000","last_undegraded":"2023-11-08T08:22:20.671635+0000","last_fullsized":"2023-11-08T08:22:20.671635+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:32:40.141129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.689381+0000","last_change":"2023-11-08T08:22:20.680838+0000","last_active":"2023-11-08T08:22:20.689381+0000","last_peered":"2023-11-08T08:22:20.689381+0000","last_clean":"2023-11-08T08:22:20.689381+0000","last_became_active":"2023-11-08T08:22:20.680574+0000","last_became_peered":"2023-11-08T08:22:20.680574+0000","last_unstale":"2023-11-08T08:22:20.689381+0000","last_undegraded":"2023-11-08T08:22:20.689381+0000","last_fullsized":"2023-11-08T08:22:20.689381+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:54:34.009215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.682593+0000","last_change":"2023-11-08T08:22:20.201396+0000","last_active":"2023-11-08T08:22:20.682593+0000","last_peered":"2023-11-08T08:22:20.682593+0000","last_clean":"2023-11-08T08:22:20.682593+0000","last_became_active":"2023-11-08T08:22:20.200974+0000","last_became_peered":"2023-11-08T08:22:20.200974+0000","last_unstale":"2023-11-08T08:22:20.682593+0000","last_undegraded":"2023-11-08T08:22:20.682593+0000","last_fullsized":"2023-11-08T08:22:20.682593+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:00:17.585593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.702985+0000","last_change":"2023-11-08T08:22:20.690026+0000","last_active":"2023-11-08T08:22:20.702985+0000","last_peered":"2023-11-08T08:22:20.702985+0000","last_clean":"2023-11-08T08:22:20.702985+0000","last_became_active":"2023-11-08T08:22:20.689789+0000","last_became_peered":"2023-11-08T08:22:20.689789+0000","last_unstale":"2023-11-08T08:22:20.702985+0000","last_undegraded":"2023-11-08T08:22:20.702985+0000","last_fullsized":"2023-11-08T08:22:20.702985+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:18:33.815298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"68'3","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703932+0000","last_change":"2023-11-08T08:22:20.206871+0000","last_active":"2023-11-08T08:22:20.703932+0000","last_peered":"2023-11-08T08:22:20.703932+0000","last_clean":"2023-11-08T08:22:20.703932+0000","last_became_active":"2023-11-08T08:22:20.206479+0000","last_became_peered":"2023-11-08T08:22:20.206479+0000","last_unstale":"2023-11-08T08:22:20.703932+0000","last_undegraded":"2023-11-08T08:22:20.703932+0000","last_fullsized":"2023-11-08T08:22:20.703932+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:12:19.948868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.702226+0000","last_change":"2023-11-08T08:22:20.688710+0000","last_active":"2023-11-08T08:22:20.702226+0000","last_peered":"2023-11-08T08:22:20.702226+0000","last_clean":"2023-11-08T08:22:20.702226+0000","last_became_active":"2023-11-08T08:22:20.688412+0000","last_became_peered":"2023-11-08T08:22:20.688412+0000","last_unstale":"2023-11-08T08:22:20.702226+0000","last_undegraded":"2023-11-08T08:22:20.702226+0000","last_fullsized":"2023-11-08T08:22:20.702226+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:52:33.736496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.707794+0000","last_change":"2023-11-08T08:22:20.199781+0000","last_active":"2023-11-08T08:22:20.707794+0000","last_peered":"2023-11-08T08:22:20.707794+0000","last_clean":"2023-11-08T08:22:20.707794+0000","last_became_active":"2023-11-08T08:22:20.199438+0000","last_became_peered":"2023-11-08T08:22:20.199438+0000","last_unstale":"2023-11-08T08:22:20.707794+0000","last_undegraded":"2023-11-08T08:22:20.707794+0000","last_fullsized":"2023-11-08T08:22:20.707794+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:38:33.814839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.699794+0000","last_change":"2023-11-08T08:22:20.693515+0000","last_active":"2023-11-08T08:22:20.699794+0000","last_peered":"2023-11-08T08:22:20.699794+0000","last_clean":"2023-11-08T08:22:20.699794+0000","last_became_active":"2023-11-08T08:22:20.693061+0000","last_became_peered":"2023-11-08T08:22:20.693061+0000","last_unstale":"2023-11-08T08:22:20.699794+0000","last_undegraded":"2023-11-08T08:22:20.699794+0000","last_fullsized":"2023-11-08T08:22:20.699794+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:31:09.871375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.696079+0000","last_change":"2023-11-08T08:22:20.204028+0000","last_active":"2023-11-08T08:22:20.696079+0000","last_peered":"2023-11-08T08:22:20.696079+0000","last_clean":"2023-11-08T08:22:20.696079+0000","last_became_active":"2023-11-08T08:22:20.203521+0000","last_became_peered":"2023-11-08T08:22:20.203521+0000","last_unstale":"2023-11-08T08:22:20.696079+0000","last_undegraded":"2023-11-08T08:22:20.696079+0000","last_fullsized":"2023-11-08T08:22:20.696079+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:21:33.321814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705973+0000","last_change":"2023-11-08T08:22:20.702065+0000","last_active":"2023-11-08T08:22:20.705973+0000","last_peered":"2023-11-08T08:22:20.705973+0000","last_clean":"2023-11-08T08:22:20.705973+0000","last_became_active":"2023-11-08T08:22:20.701780+0000","last_became_peered":"2023-11-08T08:22:20.701780+0000","last_unstale":"2023-11-08T08:22:20.705973+0000","last_undegraded":"2023-11-08T08:22:20.705973+0000","last_fullsized":"2023-11-08T08:22:20.705973+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:35:24.924456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.683889+0000","last_change":"2023-11-08T08:22:20.203768+0000","last_active":"2023-11-08T08:22:20.683889+0000","last_peered":"2023-11-08T08:22:20.683889+0000","last_clean":"2023-11-08T08:22:20.683889+0000","last_became_active":"2023-11-08T08:22:20.202505+0000","last_became_peered":"2023-11-08T08:22:20.202505+0000","last_unstale":"2023-11-08T08:22:20.683889+0000","last_undegraded":"2023-11-08T08:22:20.683889+0000","last_fullsized":"2023-11-08T08:22:20.683889+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:30:06.084174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.698298+0000","last_change":"2023-11-08T08:22:20.690875+0000","last_active":"2023-11-08T08:22:20.698298+0000","last_peered":"2023-11-08T08:22:20.698298+0000","last_clean":"2023-11-08T08:22:20.698298+0000","last_became_active":"2023-11-08T08:22:20.690458+0000","last_became_peered":"2023-11-08T08:22:20.690458+0000","last_unstale":"2023-11-08T08:22:20.698298+0000","last_undegraded":"2023-11-08T08:22:20.698298+0000","last_fullsized":"2023-11-08T08:22:20.698298+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:39:04.388075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.683408+0000","last_change":"2023-11-08T08:22:20.198290+0000","last_active":"2023-11-08T08:22:20.683408+0000","last_peered":"2023-11-08T08:22:20.683408+0000","last_clean":"2023-11-08T08:22:20.683408+0000","last_became_active":"2023-11-08T08:22:20.197828+0000","last_became_peered":"2023-11-08T08:22:20.197828+0000","last_unstale":"2023-11-08T08:22:20.683408+0000","last_undegraded":"2023-11-08T08:22:20.683408+0000","last_fullsized":"2023-11-08T08:22:20.683408+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:16:06.608091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.696215+0000","last_change":"2023-11-08T08:22:20.684025+0000","last_active":"2023-11-08T08:22:20.696215+0000","last_peered":"2023-11-08T08:22:20.696215+0000","last_clean":"2023-11-08T08:22:20.696215+0000","last_became_active":"2023-11-08T08:22:20.683726+0000","last_became_peered":"2023-11-08T08:22:20.683726+0000","last_unstale":"2023-11-08T08:22:20.696215+0000","last_undegraded":"2023-11-08T08:22:20.696215+0000","last_fullsized":"2023-11-08T08:22:20.696215+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:12.659020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.704640+0000","last_change":"2023-11-08T08:22:20.199622+0000","last_active":"2023-11-08T08:22:20.704640+0000","last_peered":"2023-11-08T08:22:20.704640+0000","last_clean":"2023-11-08T08:22:20.704640+0000","last_became_active":"2023-11-08T08:22:20.199363+0000","last_became_peered":"2023-11-08T08:22:20.199363+0000","last_unstale":"2023-11-08T08:22:20.704640+0000","last_undegraded":"2023-11-08T08:22:20.704640+0000","last_fullsized":"2023-11-08T08:22:20.704640+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:24:53.212884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.700114+0000","last_change":"2023-11-08T08:22:20.694688+0000","last_active":"2023-11-08T08:22:20.700114+0000","last_peered":"2023-11-08T08:22:20.700114+0000","last_clean":"2023-11-08T08:22:20.700114+0000","last_became_active":"2023-11-08T08:22:20.694040+0000","last_became_peered":"2023-11-08T08:22:20.694040+0000","last_unstale":"2023-11-08T08:22:20.700114+0000","last_undegraded":"2023-11-08T08:22:20.700114+0000","last_fullsized":"2023-11-08T08:22:20.700114+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:20:23.943501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.706057+0000","last_change":"2023-11-08T08:22:20.207231+0000","last_active":"2023-11-08T08:22:20.706057+0000","last_peered":"2023-11-08T08:22:20.706057+0000","last_clean":"2023-11-08T08:22:20.706057+0000","last_became_active":"2023-11-08T08:22:20.206725+0000","last_became_peered":"2023-11-08T08:22:20.206725+0000","last_unstale":"2023-11-08T08:22:20.706057+0000","last_undegraded":"2023-11-08T08:22:20.706057+0000","last_fullsized":"2023-11-08T08:22:20.706057+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:11:48.450466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705787+0000","last_change":"2023-11-08T08:22:20.699810+0000","last_active":"2023-11-08T08:22:20.705787+0000","last_peered":"2023-11-08T08:22:20.705787+0000","last_clean":"2023-11-08T08:22:20.705787+0000","last_became_active":"2023-11-08T08:22:20.699512+0000","last_became_peered":"2023-11-08T08:22:20.699512+0000","last_unstale":"2023-11-08T08:22:20.705787+0000","last_undegraded":"2023-11-08T08:22:20.705787+0000","last_fullsized":"2023-11-08T08:22:20.705787+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:32:41.147256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.684842+0000","last_change":"2023-11-08T08:22:20.199547+0000","last_active":"2023-11-08T08:22:20.684842+0000","last_peered":"2023-11-08T08:22:20.684842+0000","last_clean":"2023-11-08T08:22:20.684842+0000","last_became_active":"2023-11-08T08:22:20.199032+0000","last_became_peered":"2023-11-08T08:22:20.199032+0000","last_unstale":"2023-11-08T08:22:20.684842+0000","last_undegraded":"2023-11-08T08:22:20.684842+0000","last_fullsized":"2023-11-08T08:22:20.684842+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T10:40:53.187744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.698055+0000","last_change":"2023-11-08T08:22:20.689245+0000","last_active":"2023-11-08T08:22:20.698055+0000","last_peered":"2023-11-08T08:22:20.698055+0000","last_clean":"2023-11-08T08:22:20.698055+0000","last_became_active":"2023-11-08T08:22:20.688509+0000","last_became_peered":"2023-11-08T08:22:20.688509+0000","last_unstale":"2023-11-08T08:22:20.698055+0000","last_undegraded":"2023-11-08T08:22:20.698055+0000","last_fullsized":"2023-11-08T08:22:20.698055+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T12:32:41.147256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.682621+0000","last_change":"2023-11-08T08:22:20.190145+0000","last_active":"2023-11-08T08:22:20.682621+0000","last_peered":"2023-11-08T08:22:20.682621+0000","last_clean":"2023-11-08T08:22:20.682621+0000","last_became_active":"2023-11-08T08:22:20.189793+0000","last_became_peered":"2023-11-08T08:22:20.189793+0000","last_unstale":"2023-11-08T08:22:20.682621+0000","last_undegraded":"2023-11-08T08:22:20.682621+0000","last_fullsized":"2023-11-08T08:22:20.682621+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:41:16.549439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705501+0000","last_change":"2023-11-08T08:22:20.701070+0000","last_active":"2023-11-08T08:22:20.705501+0000","last_peered":"2023-11-08T08:22:20.705501+0000","last_clean":"2023-11-08T08:22:20.705501+0000","last_became_active":"2023-11-08T08:22:20.700804+0000","last_became_peered":"2023-11-08T08:22:20.700804+0000","last_unstale":"2023-11-08T08:22:20.705501+0000","last_undegraded":"2023-11-08T08:22:20.705501+0000","last_fullsized":"2023-11-08T08:22:20.705501+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:47:00.548303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.662046+0000","last_change":"2023-11-08T08:22:20.189567+0000","last_active":"2023-11-08T08:22:20.662046+0000","last_peered":"2023-11-08T08:22:20.662046+0000","last_clean":"2023-11-08T08:22:20.662046+0000","last_became_active":"2023-11-08T08:22:20.189048+0000","last_became_peered":"2023-11-08T08:22:20.189048+0000","last_unstale":"2023-11-08T08:22:20.662046+0000","last_undegraded":"2023-11-08T08:22:20.662046+0000","last_fullsized":"2023-11-08T08:22:20.662046+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:22:21.856135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703791+0000","last_change":"2023-11-08T08:22:20.701353+0000","last_active":"2023-11-08T08:22:20.703791+0000","last_peered":"2023-11-08T08:22:20.703791+0000","last_clean":"2023-11-08T08:22:20.703791+0000","last_became_active":"2023-11-08T08:22:20.700940+0000","last_became_peered":"2023-11-08T08:22:20.700940+0000","last_unstale":"2023-11-08T08:22:20.703791+0000","last_undegraded":"2023-11-08T08:22:20.703791+0000","last_fullsized":"2023-11-08T08:22:20.703791+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:30:07.090301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.680241+0000","last_change":"2023-11-08T08:22:20.192146+0000","last_active":"2023-11-08T08:22:20.680241+0000","last_peered":"2023-11-08T08:22:20.680241+0000","last_clean":"2023-11-08T08:22:20.680241+0000","last_became_active":"2023-11-08T08:22:20.191886+0000","last_became_peered":"2023-11-08T08:22:20.191886+0000","last_unstale":"2023-11-08T08:22:20.680241+0000","last_undegraded":"2023-11-08T08:22:20.680241+0000","last_fullsized":"2023-11-08T08:22:20.680241+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:23:39.012885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.701836+0000","last_change":"2023-11-08T08:22:20.685906+0000","last_active":"2023-11-08T08:22:20.701836+0000","last_peered":"2023-11-08T08:22:20.701836+0000","last_clean":"2023-11-08T08:22:20.701836+0000","last_became_active":"2023-11-08T08:22:20.685586+0000","last_became_peered":"2023-11-08T08:22:20.685586+0000","last_unstale":"2023-11-08T08:22:20.701836+0000","last_undegraded":"2023-11-08T08:22:20.701836+0000","last_fullsized":"2023-11-08T08:22:20.701836+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:15:21.780103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.684368+0000","last_change":"2023-11-08T08:22:20.204475+0000","last_active":"2023-11-08T08:22:20.684368+0000","last_peered":"2023-11-08T08:22:20.684368+0000","last_clean":"2023-11-08T08:22:20.684368+0000","last_became_active":"2023-11-08T08:22:20.203660+0000","last_became_peered":"2023-11-08T08:22:20.203660+0000","last_unstale":"2023-11-08T08:22:20.684368+0000","last_undegraded":"2023-11-08T08:22:20.684368+0000","last_fullsized":"2023-11-08T08:22:20.684368+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:06:37.792121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.695918+0000","last_change":"2023-11-08T08:22:20.690205+0000","last_active":"2023-11-08T08:22:20.695918+0000","last_peered":"2023-11-08T08:22:20.695918+0000","last_clean":"2023-11-08T08:22:20.695918+0000","last_became_active":"2023-11-08T08:22:20.689931+0000","last_became_peered":"2023-11-08T08:22:20.689931+0000","last_unstale":"2023-11-08T08:22:20.695918+0000","last_undegraded":"2023-11-08T08:22:20.695918+0000","last_fullsized":"2023-11-08T08:22:20.695918+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:05:14.039538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.670590+0000","last_change":"2023-11-08T08:22:20.204551+0000","last_active":"2023-11-08T08:22:20.670590+0000","last_peered":"2023-11-08T08:22:20.670590+0000","last_clean":"2023-11-08T08:22:20.670590+0000","last_became_active":"2023-11-08T08:22:20.204257+0000","last_became_peered":"2023-11-08T08:22:20.204257+0000","last_unstale":"2023-11-08T08:22:20.670590+0000","last_undegraded":"2023-11-08T08:22:20.670590+0000","last_fullsized":"2023-11-08T08:22:20.670590+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:46:59.542176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703173+0000","last_change":"2023-11-08T08:22:20.693657+0000","last_active":"2023-11-08T08:22:20.703173+0000","last_peered":"2023-11-08T08:22:20.703173+0000","last_clean":"2023-11-08T08:22:20.703173+0000","last_became_active":"2023-11-08T08:22:20.693007+0000","last_became_peered":"2023-11-08T08:22:20.693007+0000","last_unstale":"2023-11-08T08:22:20.703173+0000","last_undegraded":"2023-11-08T08:22:20.703173+0000","last_fullsized":"2023-11-08T08:22:20.703173+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:35:17.751012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.682992+0000","last_change":"2023-11-08T08:22:20.203458+0000","last_active":"2023-11-08T08:22:20.682992+0000","last_peered":"2023-11-08T08:22:20.682992+0000","last_clean":"2023-11-08T08:22:20.682992+0000","last_became_active":"2023-11-08T08:22:20.203089+0000","last_became_peered":"2023-11-08T08:22:20.203089+0000","last_unstale":"2023-11-08T08:22:20.682992+0000","last_undegraded":"2023-11-08T08:22:20.682992+0000","last_fullsized":"2023-11-08T08:22:20.682992+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:21:52.588292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.700478+0000","last_change":"2023-11-08T08:22:20.689029+0000","last_active":"2023-11-08T08:22:20.700478+0000","last_peered":"2023-11-08T08:22:20.700478+0000","last_clean":"2023-11-08T08:22:20.700478+0000","last_became_active":"2023-11-08T08:22:20.688756+0000","last_became_peered":"2023-11-08T08:22:20.688756+0000","last_unstale":"2023-11-08T08:22:20.700478+0000","last_undegraded":"2023-11-08T08:22:20.700478+0000","last_fullsized":"2023-11-08T08:22:20.700478+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:20:23.943501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.706306+0000","last_change":"2023-11-08T08:22:20.206983+0000","last_active":"2023-11-08T08:22:20.706306+0000","last_peered":"2023-11-08T08:22:20.706306+0000","last_clean":"2023-11-08T08:22:20.706306+0000","last_became_active":"2023-11-08T08:22:20.206349+0000","last_became_peered":"2023-11-08T08:22:20.206349+0000","last_unstale":"2023-11-08T08:22:20.706306+0000","last_undegraded":"2023-11-08T08:22:20.706306+0000","last_fullsized":"2023-11-08T08:22:20.706306+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:11.652893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.200062+0000","last_change":"2023-11-08T08:18:58.771310+0000","last_active":"2023-11-08T08:22:20.200062+0000","last_peered":"2023-11-08T08:22:20.200062+0000","last_clean":"2023-11-08T08:22:20.200062+0000","last_became_active":"2023-11-08T08:18:58.770891+0000","last_became_peered":"2023-11-08T08:18:58.770891+0000","last_unstale":"2023-11-08T08:22:20.200062+0000","last_undegraded":"2023-11-08T08:22:20.200062+0000","last_fullsized":"2023-11-08T08:22:20.200062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.699660+0000","last_change":"2023-11-08T08:22:20.692436+0000","last_active":"2023-11-08T08:22:20.699660+0000","last_peered":"2023-11-08T08:22:20.699660+0000","last_clean":"2023-11-08T08:22:20.699660+0000","last_became_active":"2023-11-08T08:22:20.692139+0000","last_became_peered":"2023-11-08T08:22:20.692139+0000","last_unstale":"2023-11-08T08:22:20.699660+0000","last_undegraded":"2023-11-08T08:22:20.699660+0000","last_fullsized":"2023-11-08T08:22:20.699660+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:50:50.206439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.705086+0000","last_change":"2023-11-08T08:22:20.200670+0000","last_active":"2023-11-08T08:22:20.705086+0000","last_peered":"2023-11-08T08:22:20.705086+0000","last_clean":"2023-11-08T08:22:20.705086+0000","last_became_active":"2023-11-08T08:22:20.200366+0000","last_became_peered":"2023-11-08T08:22:20.200366+0000","last_unstale":"2023-11-08T08:22:20.705086+0000","last_undegraded":"2023-11-08T08:22:20.705086+0000","last_fullsized":"2023-11-08T08:22:20.705086+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:54:33.003087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":50,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.193571+0000","last_change":"2023-11-08T08:18:58.770483+0000","last_active":"2023-11-08T08:22:20.193571+0000","last_peered":"2023-11-08T08:22:20.193571+0000","last_clean":"2023-11-08T08:22:20.193571+0000","last_became_active":"2023-11-08T08:18:58.770040+0000","last_became_peered":"2023-11-08T08:18:58.770040+0000","last_unstale":"2023-11-08T08:22:20.193571+0000","last_undegraded":"2023-11-08T08:22:20.193571+0000","last_fullsized":"2023-11-08T08:22:20.193571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.699657+0000","last_change":"2023-11-08T08:22:20.691973+0000","last_active":"2023-11-08T08:22:20.699657+0000","last_peered":"2023-11-08T08:22:20.699657+0000","last_clean":"2023-11-08T08:22:20.699657+0000","last_became_active":"2023-11-08T08:22:20.691242+0000","last_became_peered":"2023-11-08T08:22:20.691242+0000","last_unstale":"2023-11-08T08:22:20.699657+0000","last_undegraded":"2023-11-08T08:22:20.699657+0000","last_fullsized":"2023-11-08T08:22:20.699657+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:51.802894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.702724+0000","last_change":"2023-11-08T08:22:20.203707+0000","last_active":"2023-11-08T08:22:20.702724+0000","last_peered":"2023-11-08T08:22:20.702724+0000","last_clean":"2023-11-08T08:22:20.702724+0000","last_became_active":"2023-11-08T08:22:20.203159+0000","last_became_peered":"2023-11-08T08:22:20.203159+0000","last_unstale":"2023-11-08T08:22:20.702724+0000","last_undegraded":"2023-11-08T08:22:20.702724+0000","last_fullsized":"2023-11-08T08:22:20.702724+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:20:54.501281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":52,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.204636+0000","last_change":"2023-11-08T08:18:58.766848+0000","last_active":"2023-11-08T08:22:20.204636+0000","last_peered":"2023-11-08T08:22:20.204636+0000","last_clean":"2023-11-08T08:22:20.204636+0000","last_became_active":"2023-11-08T08:18:58.766378+0000","last_became_peered":"2023-11-08T08:18:58.766378+0000","last_unstale":"2023-11-08T08:22:20.204636+0000","last_undegraded":"2023-11-08T08:22:20.204636+0000","last_fullsized":"2023-11-08T08:22:20.204636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703394+0000","last_change":"2023-11-08T08:22:20.696868+0000","last_active":"2023-11-08T08:22:20.703394+0000","last_peered":"2023-11-08T08:22:20.703394+0000","last_clean":"2023-11-08T08:22:20.703394+0000","last_became_active":"2023-11-08T08:22:20.696572+0000","last_became_peered":"2023-11-08T08:22:20.696572+0000","last_unstale":"2023-11-08T08:22:20.703394+0000","last_undegraded":"2023-11-08T08:22:20.703394+0000","last_fullsized":"2023-11-08T08:22:20.703394+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:14.575132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.670678+0000","last_change":"2023-11-08T08:22:20.208213+0000","last_active":"2023-11-08T08:22:20.670678+0000","last_peered":"2023-11-08T08:22:20.670678+0000","last_clean":"2023-11-08T08:22:20.670678+0000","last_became_active":"2023-11-08T08:22:20.207888+0000","last_became_peered":"2023-11-08T08:22:20.207888+0000","last_unstale":"2023-11-08T08:22:20.670678+0000","last_undegraded":"2023-11-08T08:22:20.670678+0000","last_fullsized":"2023-11-08T08:22:20.670678+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:18:45.247325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":50,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.194310+0000","last_change":"2023-11-08T08:18:58.769764+0000","last_active":"2023-11-08T08:22:20.194310+0000","last_peered":"2023-11-08T08:22:20.194310+0000","last_clean":"2023-11-08T08:22:20.194310+0000","last_became_active":"2023-11-08T08:18:58.769357+0000","last_became_peered":"2023-11-08T08:18:58.769357+0000","last_unstale":"2023-11-08T08:22:20.194310+0000","last_undegraded":"2023-11-08T08:22:20.194310+0000","last_fullsized":"2023-11-08T08:22:20.194310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:13.748094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.703721+0000","last_change":"2023-11-08T08:22:20.689657+0000","last_active":"2023-11-08T08:22:20.703721+0000","last_peered":"2023-11-08T08:22:20.703721+0000","last_clean":"2023-11-08T08:22:20.703721+0000","last_became_active":"2023-11-08T08:22:20.689384+0000","last_became_peered":"2023-11-08T08:22:20.689384+0000","last_unstale":"2023-11-08T08:22:20.703721+0000","last_undegraded":"2023-11-08T08:22:20.703721+0000","last_fullsized":"2023-11-08T08:22:20.703721+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T11:20:23.943501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.682574+0000","last_change":"2023-11-08T08:22:20.193829+0000","last_active":"2023-11-08T08:22:20.682574+0000","last_peered":"2023-11-08T08:22:20.682574+0000","last_clean":"2023-11-08T08:22:20.682574+0000","last_became_active":"2023-11-08T08:22:20.193503+0000","last_became_peered":"2023-11-08T08:22:20.193503+0000","last_unstale":"2023-11-08T08:22:20.682574+0000","last_undegraded":"2023-11-08T08:22:20.682574+0000","last_fullsized":"2023-11-08T08:22:20.682574+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:50.796767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.189418+0000","last_change":"2023-11-08T08:18:58.771460+0000","last_active":"2023-11-08T08:22:20.189418+0000","last_peered":"2023-11-08T08:22:20.189418+0000","last_clean":"2023-11-08T08:22:20.189418+0000","last_became_active":"2023-11-08T08:18:58.770648+0000","last_became_peered":"2023-11-08T08:18:58.770648+0000","last_unstale":"2023-11-08T08:22:20.189418+0000","last_undegraded":"2023-11-08T08:22:20.189418+0000","last_fullsized":"2023-11-08T08:22:20.189418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.696664+0000","last_change":"2023-11-08T08:22:20.685820+0000","last_active":"2023-11-08T08:22:20.696664+0000","last_peered":"2023-11-08T08:22:20.696664+0000","last_clean":"2023-11-08T08:22:20.696664+0000","last_became_active":"2023-11-08T08:22:20.685570+0000","last_became_peered":"2023-11-08T08:22:20.685570+0000","last_unstale":"2023-11-08T08:22:20.696664+0000","last_undegraded":"2023-11-08T08:22:20.696664+0000","last_fullsized":"2023-11-08T08:22:20.696664+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:52:08.554540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.683551+0000","last_change":"2023-11-08T08:22:20.193136+0000","last_active":"2023-11-08T08:22:20.683551+0000","last_peered":"2023-11-08T08:22:20.683551+0000","last_clean":"2023-11-08T08:22:20.683551+0000","last_became_active":"2023-11-08T08:22:20.192807+0000","last_became_peered":"2023-11-08T08:22:20.192807+0000","last_unstale":"2023-11-08T08:22:20.683551+0000","last_undegraded":"2023-11-08T08:22:20.683551+0000","last_fullsized":"2023-11-08T08:22:20.683551+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:06:30.564100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":53,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.186292+0000","last_change":"2023-11-08T08:18:58.973732+0000","last_active":"2023-11-08T08:22:20.186292+0000","last_peered":"2023-11-08T08:22:20.186292+0000","last_clean":"2023-11-08T08:22:20.186292+0000","last_became_active":"2023-11-08T08:18:58.973007+0000","last_became_peered":"2023-11-08T08:18:58.973007+0000","last_unstale":"2023-11-08T08:22:20.186292+0000","last_undegraded":"2023-11-08T08:22:20.186292+0000","last_fullsized":"2023-11-08T08:22:20.186292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.696983+0000","last_change":"2023-11-08T08:22:20.678891+0000","last_active":"2023-11-08T08:22:20.696983+0000","last_peered":"2023-11-08T08:22:20.696983+0000","last_clean":"2023-11-08T08:22:20.696983+0000","last_became_active":"2023-11-08T08:22:20.678596+0000","last_became_peered":"2023-11-08T08:22:20.678596+0000","last_unstale":"2023-11-08T08:22:20.696983+0000","last_undegraded":"2023-11-08T08:22:20.696983+0000","last_fullsized":"2023-11-08T08:22:20.696983+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:44:54.466782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.660754+0000","last_change":"2023-11-08T08:22:20.186891+0000","last_active":"2023-11-08T08:22:20.660754+0000","last_peered":"2023-11-08T08:22:20.660754+0000","last_clean":"2023-11-08T08:22:20.660754+0000","last_became_active":"2023-11-08T08:22:20.186535+0000","last_became_peered":"2023-11-08T08:22:20.186535+0000","last_unstale":"2023-11-08T08:22:20.660754+0000","last_undegraded":"2023-11-08T08:22:20.660754+0000","last_fullsized":"2023-11-08T08:22:20.660754+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:08:46.040304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.202004+0000","last_change":"2023-11-08T08:18:58.770136+0000","last_active":"2023-11-08T08:22:20.202004+0000","last_peered":"2023-11-08T08:22:20.202004+0000","last_clean":"2023-11-08T08:22:20.202004+0000","last_became_active":"2023-11-08T08:18:58.769797+0000","last_became_peered":"2023-11-08T08:18:58.769797+0000","last_unstale":"2023-11-08T08:22:20.202004+0000","last_undegraded":"2023-11-08T08:22:20.202004+0000","last_fullsized":"2023-11-08T08:22:20.202004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:29:12.107128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.696577+0000","last_change":"2023-11-08T08:22:20.681462+0000","last_active":"2023-11-08T08:22:20.696577+0000","last_peered":"2023-11-08T08:22:20.696577+0000","last_clean":"2023-11-08T08:22:20.696577+0000","last_became_active":"2023-11-08T08:22:20.680690+0000","last_became_peered":"2023-11-08T08:22:20.680690+0000","last_unstale":"2023-11-08T08:22:20.696577+0000","last_undegraded":"2023-11-08T08:22:20.696577+0000","last_fullsized":"2023-11-08T08:22:20.696577+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:20.162037+0000","last_clean_scrub_stamp":"2023-11-08T08:22:20.162037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:39:04.388075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.701747+0000","last_change":"2023-11-08T08:22:20.204262+0000","last_active":"2023-11-08T08:22:20.701747+0000","last_peered":"2023-11-08T08:22:20.701747+0000","last_clean":"2023-11-08T08:22:20.701747+0000","last_became_active":"2023-11-08T08:22:20.204012+0000","last_became_peered":"2023-11-08T08:22:20.204012+0000","last_unstale":"2023-11-08T08:22:20.701747+0000","last_undegraded":"2023-11-08T08:22:20.701747+0000","last_fullsized":"2023-11-08T08:22:20.701747+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:22:19.155910+0000","last_clean_scrub_stamp":"2023-11-08T08:22:19.155910+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:38:33.814839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.189420+0000","last_change":"2023-11-08T08:18:58.777140+0000","last_active":"2023-11-08T08:22:20.189420+0000","last_peered":"2023-11-08T08:22:20.189420+0000","last_clean":"2023-11-08T08:22:20.189420+0000","last_became_active":"2023-11-08T08:18:58.776834+0000","last_became_peered":"2023-11-08T08:18:58.776834+0000","last_unstale":"2023-11-08T08:22:20.189420+0000","last_undegraded":"2023-11-08T08:22:20.189420+0000","last_fullsized":"2023-11-08T08:22:20.189420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":205,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-08T08:22:20.201913+0000","last_change":"2023-11-08T08:18:53.780089+0000","last_active":"2023-11-08T08:22:20.201913+0000","last_peered":"2023-11-08T08:22:20.201913+0000","last_clean":"2023-11-08T08:22:20.201913+0000","last_became_active":"2023-11-08T08:18:53.779533+0000","last_became_peered":"2023-11-08T08:18:53.779533+0000","last_unstale":"2023-11-08T08:22:20.201913+0000","last_undegraded":"2023-11-08T08:22:20.201913+0000","last_fullsized":"2023-11-08T08:22:20.201913+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_clean_scrub_stamp":"2023-11-08T08:18:52.744693+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:58:24.877070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":14884,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":13504,"kb_avail":94356956,"statfs":{"total":96636764160,"available":96621522944,"internally_reserved":0,"allocated":1404928,"data_stored":966246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13828096},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.083,"15min":1.028},"min":{"1min":0.74399999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5669999999999999,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.194,"15min":1.1699999999999999},"min":{"1min":1.028,"5min":0.746,"15min":0.746},"max":{"1min":1.5660000000000001,"5min":1.667,"15min":1.667},"last":1.2709999999999999}]},{"osd":1,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1859999999999999,"15min":1.1930000000000001},"min":{"1min":0.77900000000000003,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.5589999999999999,"5min":1.972,"15min":1.972},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.294,"15min":1.3160000000000001},"min":{"1min":0.99099999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.696,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.93200000000000005}]},{"osd":2,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.329,"15min":1.4039999999999999},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.597,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":1.302},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.319,"15min":1.401},"min":{"1min":0.874,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.522,"5min":4.9089999999999998,"15min":4.9089999999999998},"last":0.99199999999999999}]},{"osd":3,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.2250000000000001,"15min":1.252},"min":{"1min":0.84999999999999998,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.3460000000000001,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.292,"15min":1.3080000000000001},"min":{"1min":1.0649999999999999,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.8049999999999999,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":1.2130000000000001}]},{"osd":4,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.2370000000000001,"15min":1.264},"min":{"1min":0.79800000000000004,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.476,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.3939999999999999,"15min":1.425},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.643,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":1.24}]},{"osd":5,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.3180000000000001,"15min":1.3700000000000001},"min":{"1min":0.89500000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.329,"5min":2.73,"15min":2.73},"last":1.024},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3160000000000001,"15min":1.3360000000000001},"min":{"1min":0.85599999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.6060000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.80900000000000005}]},{"osd":6,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.423,"15min":1.4490000000000001},"min":{"1min":0.88900000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.762,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.3799999999999999,"15min":1.4339999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.5109999999999999,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.83799999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9568,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362272,"statfs":{"total":96636764160,"available":96626966528,"internally_reserved":0,"allocated":811008,"data_stored":375878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.121,"15min":1.173},"min":{"1min":0.67800000000000005,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5229999999999999,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.048,"5min":1.0840000000000001,"15min":1.0980000000000001},"min":{"1min":0.60799999999999998,"5min":0.501,"15min":0.501},"max":{"1min":1.4359999999999999,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.94699999999999995}]},{"osd":1,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2110000000000001,"15min":1.1950000000000001},"min":{"1min":0.58499999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.7509999999999999,"5min":1.782,"15min":1.782},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.2110000000000001,"15min":1.2070000000000001},"min":{"1min":0.626,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.429}]},{"osd":2,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.169,"15min":1.1819999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7989999999999999,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.244,"15min":1.286},"min":{"1min":0.81599999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.6399999999999999,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.288}]},{"osd":3,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2729999999999999,"15min":1.2869999999999999},"min":{"1min":0.80800000000000005,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.3899999999999999,"15min":1.407},"min":{"1min":1.052,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.6349999999999998,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":1.5600000000000001}]},{"osd":4,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.23,"15min":1.258},"min":{"1min":0.76200000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.5649999999999999,"5min":2.02,"15min":2.02},"last":1.52},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.202,"15min":1.194},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.599,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.0549999999999999}]},{"osd":5,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.262,"15min":1.24},"min":{"1min":0.94899999999999995,"5min":0.88,"15min":0.88},"max":{"1min":1.655,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3600000000000001,"15min":1.3700000000000001},"min":{"1min":0.96799999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":1.6299999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3720000000000001,"15min":1.3819999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.8899999999999999,"5min":3.133,"15min":3.133},"last":1.379},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4490000000000001,"15min":1.4410000000000001},"min":{"1min":1.173,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.8400000000000001,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.7669999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9632,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362208,"statfs":{"total":96636764160,"available":96626900992,"internally_reserved":0,"allocated":811008,"data_stored":375878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1359999999999999,"15min":1.133},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.494,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":1.234},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2290000000000001,"15min":1.238},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.5189999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.6479999999999999}]},{"osd":1,"last update":"Wed Nov 8 08:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.1140000000000001,"15min":1.135},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.333,"5min":2.012,"15min":2.012},"last":1.149},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.151,"15min":1.1870000000000001},"min":{"1min":0.76000000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.395,"5min":1.667,"15min":1.667},"last":0.98699999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1850000000000001,"15min":1.196},"min":{"1min":0.86299999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.583,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.103},{"interface":"front","average":{"1min":1.05,"5min":1.177,"15min":1.2170000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.4630000000000001,"5min":1.796,"15min":1.796},"last":1.5860000000000001}]},{"osd":3,"last update":"Wed Nov 8 08:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.3120000000000001,"15min":1.3380000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5049999999999999,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.2849999999999999,"15min":1.325},"min":{"1min":0.81000000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.4490000000000001,"5min":3.7999999999999998,"15min":3.7999999999999998},"last":1.3029999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.157,"15min":1.2010000000000001},"min":{"1min":0.78300000000000003,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.609,"5min":1.629,"15min":1.629},"last":1.343},{"interface":"front","average":{"1min":1.089,"5min":1.27,"15min":1.333},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.4710000000000001,"5min":3.073,"15min":3.073},"last":1.7310000000000001}]},{"osd":6,"last update":"Wed Nov 8 08:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.276,"15min":1.341},"min":{"1min":0.85699999999999998,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.4279999999999999,"5min":3.8650000000000002,"15min":3.8650000000000002},"last":1.393},{"interface":"front","average":{"1min":1.173,"5min":1.327,"15min":1.3759999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.49,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.4259999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.4319999999999999,"15min":1.4830000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.5780000000000001,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.4279999999999999,"15min":1.4890000000000001},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.446,"5min":3.9369999999999998,"15min":3.9369999999999998},"last":1.5469999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9812,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362028,"statfs":{"total":96636764160,"available":96626716672,"internally_reserved":0,"allocated":798720,"data_stored":368930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.151,"15min":1.1459999999999999},"min":{"1min":0.73799999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.55,"5min":1.655,"15min":1.655},"last":1.28},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2250000000000001,"15min":1.234},"min":{"1min":0.82799999999999996,"5min":0.747,"15min":0.747},"max":{"1min":1.5700000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":1.3680000000000001}]},{"osd":1,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.071,"15min":1.0860000000000001},"min":{"1min":0.80300000000000005,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.397,"5min":1.99,"15min":1.99},"last":1.022},{"interface":"front","average":{"1min":1.109,"5min":1.1859999999999999,"15min":1.21},"min":{"1min":0.82899999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.4159999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.86599999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.137,"15min":1.129},"min":{"1min":0.70399999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.6240000000000001,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.171,"5min":1.2350000000000001,"15min":1.262},"min":{"1min":0.84199999999999997,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.4970000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.4039999999999999}]},{"osd":3,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.212,"15min":1.206},"min":{"1min":0.96199999999999997,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.6040000000000001,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":1.052},{"interface":"front","average":{"1min":1.181,"5min":1.2250000000000001,"15min":1.244},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.5429999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.3169999999999999}]},{"osd":5,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.262,"15min":1.2889999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.502,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2529999999999999,"15min":1.2569999999999999},"min":{"1min":0.77000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.6850000000000001,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":0.89800000000000002}]},{"osd":6,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.262,"15min":1.2809999999999999},"min":{"1min":0.92000000000000004,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.615,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.361,"15min":1.3700000000000001},"min":{"1min":0.83799999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.569,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.98299999999999998}]},{"osd":7,"last update":"Wed Nov 8 08:21:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4410000000000001},"min":{"1min":1.093,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.736,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.4530000000000001,"15min":1.4830000000000001},"min":{"1min":1.0880000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.5429999999999999,"5min":2.399,"15min":2.399},"last":1.2170000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9844,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361996,"statfs":{"total":96636764160,"available":96626683904,"internally_reserved":0,"allocated":831488,"data_stored":383438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.89300000000000002,"15min":0.90900000000000003},"min":{"1min":0.57199999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.151,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":1.298},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.91900000000000004,"15min":0.93799999999999994},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.151,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.1990000000000001}]},{"osd":1,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.91900000000000004,"15min":0.93700000000000006},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4550000000000001,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.016,"5min":1.101,"15min":1.1240000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.3129999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.4219999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.1080000000000001,"15min":1.1100000000000001},"min":{"1min":0.67200000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1910000000000001,"15min":1.2170000000000001},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.4390000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.736}]},{"osd":4,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.23,"15min":1.2929999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.3300000000000001,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.149},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.355,"15min":1.395},"min":{"1min":0.86199999999999999,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.8300000000000001,"5min":2.27,"15min":2.27},"last":1.2749999999999999}]},{"osd":5,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.4019999999999999,"15min":1.4379999999999999},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.5880000000000001,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.224},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.391,"15min":1.4259999999999999},"min":{"1min":0.92800000000000005,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.915,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":2.1160000000000001}]},{"osd":6,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.417,"15min":1.474},"min":{"1min":0.90200000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":3.0950000000000002,"5min":3.0950000000000002,"15min":3.0950000000000002},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.498,"15min":1.538},"min":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"max":{"1min":3.02,"5min":3.02,"15min":3.02},"last":1.4490000000000001}]},{"osd":7,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.5469999999999999,"15min":1.597},"min":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"max":{"1min":2.0110000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":2.0230000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.502,"15min":1.5820000000000001},"min":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":1.6890000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.339}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9656,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362184,"statfs":{"total":96636764160,"available":96626876416,"internally_reserved":0,"allocated":835584,"data_stored":384874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.91600000000000004,"15min":0.94199999999999995},"min":{"1min":0.623,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.153},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.98099999999999998,"15min":1.0009999999999999},"min":{"1min":0.64600000000000002,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5369999999999999,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.81100000000000005}]},{"osd":1,"last update":"Wed Nov 8 08:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.0600000000000001,"15min":1.105},"min":{"1min":0.75800000000000001,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.29,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.026,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.63500000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.5760000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.874}]},{"osd":3,"last update":"Wed Nov 8 08:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.99199999999999999,"15min":1.012},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.2569999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.2689999999999999,"15min":1.3420000000000001},"min":{"1min":0.71399999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.546,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":1.024}]},{"osd":4,"last update":"Wed Nov 8 08:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.3959999999999999,"15min":1.4690000000000001},"min":{"1min":0.82299999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.651,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.3300000000000001,"15min":1.365},"min":{"1min":0.91700000000000004,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.6859999999999999,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.069}]},{"osd":5,"last update":"Wed Nov 8 08:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3959999999999999,"15min":1.4359999999999999},"min":{"1min":1.081,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.708,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.331},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.337,"15min":1.349},"min":{"1min":0.93300000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.5209999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.202}]},{"osd":6,"last update":"Wed Nov 8 08:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.4099999999999999,"15min":1.429},"min":{"1min":1.0169999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.597,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.502,"15min":1.5780000000000001},"min":{"1min":0.86399999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.7869999999999999,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":1.5089999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.444,"15min":1.476},"min":{"1min":1.1619999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.7529999999999999,"5min":2.867,"15min":2.867},"last":1.444},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4770000000000001,"15min":1.4870000000000001},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.821,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.482}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9632,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362208,"statfs":{"total":96636764160,"available":96626900992,"internally_reserved":0,"allocated":811008,"data_stored":375878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.877,"15min":0.89500000000000002},"min":{"1min":0.56100000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.03,"15min":1.044},"min":{"1min":0.64500000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.1830000000000001}]},{"osd":2,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.96399999999999997,"15min":0.98899999999999999},"min":{"1min":0.73199999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.238,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.3},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.1399999999999999,"15min":1.1850000000000001},"min":{"1min":0.76000000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6439999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.256}]},{"osd":3,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0920000000000001,"15min":1.1060000000000001},"min":{"1min":0.72599999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7230000000000001,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.345},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.071,"15min":1.113},"min":{"1min":0.50600000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.7010000000000001,"5min":1.972,"15min":1.972},"last":0.92200000000000004}]},{"osd":4,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.2230000000000001,"15min":1.2330000000000001},"min":{"1min":0.93200000000000005,"5min":0.746,"15min":0.746},"max":{"1min":1.8160000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.3080000000000001,"15min":1.3440000000000001},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":1.9359999999999999,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":1.4390000000000001}]},{"osd":5,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3149999999999999,"15min":1.3360000000000001},"min":{"1min":1,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.7769999999999999,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.385},{"interface":"front","average":{"1min":1.222,"5min":1.3120000000000001,"15min":1.3440000000000001},"min":{"1min":0.83499999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.599,"5min":2.48,"15min":2.48},"last":1.4730000000000001}]},{"osd":6,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.425,"15min":1.4450000000000001},"min":{"1min":1.196,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.875,"5min":3.149,"15min":3.149},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.45,"15min":1.474},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":1.903,"5min":2.169,"15min":2.169},"last":1.502}]},{"osd":7,"last update":"Wed Nov 8 08:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.528,"15min":1.5640000000000001},"min":{"1min":1.2210000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"max":{"1min":2.0009999999999999,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.552},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.456,"15min":1.474},"min":{"1min":1.0469999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.7170000000000001,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.54}]}]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15076,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":13696,"kb_avail":94356764,"statfs":{"total":96636764160,"available":96621326336,"internally_reserved":0,"allocated":1404928,"data_stored":966246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14024704},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.93400000000000005,"15min":0.96999999999999997},"min":{"1min":0.625,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.5820000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.93200000000000005,"15min":0.96199999999999997},"min":{"1min":0.70899999999999996,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1950000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.93799999999999994}]},{"osd":2,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.97599999999999998,"15min":1.002},"min":{"1min":0.69299999999999995,"5min":0.497,"15min":0.497},"max":{"1min":1.323,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.024,"5min":1.081,"15min":1.111},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.0609999999999999}]},{"osd":3,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.111,"15min":1.1599999999999999},"min":{"1min":0.61499999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.103},{"interface":"front","average":{"1min":1.135,"5min":1.1659999999999999,"15min":1.2},"min":{"1min":0.69699999999999995,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.77,"5min":1.786,"15min":1.786},"last":1.1439999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.452,"15min":1.516},"min":{"1min":0.94799999999999995,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.909,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.4379999999999999,"15min":1.48},"min":{"1min":1.1619999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.6990000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.6739999999999999}]},{"osd":5,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.458,"15min":1.51},"min":{"1min":0.93100000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.012,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.536},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.5740000000000001,"15min":1.6479999999999999},"min":{"1min":1.105,"5min":1.0660000000000001,"15min":1.0660000000000001},"max":{"1min":2.1080000000000001,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.202}]},{"osd":6,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.468,"15min":1.53},"min":{"1min":0.94799999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":1.706,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.8879999999999999},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.53,"15min":1.581},"min":{"1min":1.117,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":1.857,"5min":2.048,"15min":2.048},"last":1.8089999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.585,"15min":1.651},"min":{"1min":1.069,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":1.9510000000000001,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.9139999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.613,"15min":1.6870000000000001},"min":{"1min":1.123,"5min":1.0169999999999999,"15min":1.0169999999999999},"max":{"1min":2.2069999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":1.29}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T08:22:25.638 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T08:22:25.638 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-08T08:22:25.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.859+0000 7f9123253700 1 Processor -- start 2023-11-08T08:22:25.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.863+0000 7f9123253700 1 -- start start 2023-11-08T08:22:25.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.863+0000 7f9123253700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f911c15c7c0 0x7f911c06cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:25.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.863+0000 7f9123253700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f911c161030 0x7f911c0cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:25.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.863+0000 7f9123253700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0cd700 0x7f911c0cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:25.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.863+0000 7f9123253700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f911c0cdba0 con 0x7f911c15c7c0 2023-11-08T08:22:25.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.863+0000 7f9123253700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f911c06c010 con 0x7f911c161030 2023-11-08T08:22:25.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.863+0000 7f9123253700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911c06c2f0 con 0x7f911c0cd700 2023-11-08T08:22:25.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.867+0000 7f9122251700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f911c15c7c0 0x7f911c06cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:25.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.867+0000 7f9122251700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f911c15c7c0 0x7f911c06cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35918/0 (socket says 172.21.15.40:35918) 2023-11-08T08:22:25.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.867+0000 7f9122251700 1 -- 172.21.15.40:0/1356113992 learned_addr learned my addr 172.21.15.40:0/1356113992 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:25.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.867+0000 7f9122a52700 1 --2- 172.21.15.40:0/1356113992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0cd700 0x7f911c0cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:25.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.867+0000 7f9121a50700 1 --2- 172.21.15.40:0/1356113992 >> v2:172.21.15.169:3300/0 conn(0x7f911c161030 0x7f911c0cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:25.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.867+0000 7f9122251700 1 -- 172.21.15.40:0/1356113992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0cd700 msgr2=0x7f911c0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:25.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.867+0000 7f9122251700 1 --2- 172.21.15.40:0/1356113992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0cd700 0x7f911c0cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.867+0000 7f9122251700 1 -- 172.21.15.40:0/1356113992 >> v2:172.21.15.169:3300/0 conn(0x7f911c161030 msgr2=0x7f911c0cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:25.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.867+0000 7f9122251700 1 --2- 172.21.15.40:0/1356113992 >> v2:172.21.15.169:3300/0 conn(0x7f911c161030 0x7f911c0cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.867+0000 7f9122251700 1 -- 172.21.15.40:0/1356113992 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c06c5d0 con 0x7f911c15c7c0 2023-11-08T08:22:25.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.867+0000 7f9121a50700 1 --2- 172.21.15.40:0/1356113992 >> v2:172.21.15.169:3300/0 conn(0x7f911c161030 0x7f911c0cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:22:25.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.867+0000 7f9122251700 1 --2- 172.21.15.40:0/1356113992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f911c15c7c0 0x7f911c06cdd0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f911c071590 tx=0x7f9118009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836a2d7fd1fbb7b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:25.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.867+0000 7f912124f700 1 -- 172.21.15.40:0/1356113992 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118011830 con 0x7f911c15c7c0 2023-11-08T08:22:25.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.867+0000 7f912124f700 1 -- 172.21.15.40:0/1356113992 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9118011990 con 0x7f911c15c7c0 2023-11-08T08:22:25.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.867+0000 7f912124f700 1 -- 172.21.15.40:0/1356113992 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118011d20 con 0x7f911c15c7c0 2023-11-08T08:22:25.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.867+0000 7f9123253700 1 -- 172.21.15.40:0/1356113992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f911c15c7c0 msgr2=0x7f911c06cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:25.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.867+0000 7f9123253700 1 --2- 172.21.15.40:0/1356113992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f911c15c7c0 0x7f911c06cdd0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f911c071590 tx=0x7f9118009bf0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.871+0000 7f9123253700 1 -- 172.21.15.40:0/1356113992 shutdown_connections 2023-11-08T08:22:25.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.871+0000 7f9123253700 1 --2- 172.21.15.40:0/1356113992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0cd700 0x7f911c0cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.871+0000 7f9123253700 1 --2- 172.21.15.40:0/1356113992 >> v2:172.21.15.169:3300/0 conn(0x7f911c161030 0x7f911c0cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.871+0000 7f9123253700 1 --2- 172.21.15.40:0/1356113992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f911c15c7c0 0x7f911c06cdd0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.871+0000 7f9123253700 1 -- 172.21.15.40:0/1356113992 >> 172.21.15.40:0/1356113992 conn(0x7f911c0d3800 msgr2=0x7f911c0cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:25.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.871+0000 7f9123253700 1 -- 172.21.15.40:0/1356113992 shutdown_connections 2023-11-08T08:22:25.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.871+0000 7f9123253700 1 -- 172.21.15.40:0/1356113992 wait complete. 2023-11-08T08:22:25.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.871+0000 7f9123253700 1 Processor -- start 2023-11-08T08:22:25.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9123253700 1 -- start start 2023-11-08T08:22:25.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9123253700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f911c0cd700 0x7f911c15ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:25.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9123253700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f911c15c7c0 0x7f911c0c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:25.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9123253700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0caff0 0x7f911c0cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:25.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9123253700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0c9490 con 0x7f911c0cd700 2023-11-08T08:22:25.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9123253700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c15cec0 con 0x7f911c15c7c0 2023-11-08T08:22:25.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9123253700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911c15d1a0 con 0x7f911c0caff0 2023-11-08T08:22:25.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9122251700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f911c0cd700 0x7f911c15ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:25.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9122251700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f911c0cd700 0x7f911c15ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35924/0 (socket says 172.21.15.40:35924) 2023-11-08T08:22:25.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9122251700 1 -- 172.21.15.40:0/697332572 learned_addr learned my addr 172.21.15.40:0/697332572 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:25.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9122251700 1 -- 172.21.15.40:0/697332572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0caff0 msgr2=0x7f911c0cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:25.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9122a52700 1 --2- 172.21.15.40:0/697332572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0caff0 0x7f911c0cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:25.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9121a50700 1 --2- 172.21.15.40:0/697332572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f911c15c7c0 0x7f911c0c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:25.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9122251700 1 --2- 172.21.15.40:0/697332572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0caff0 0x7f911c0cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9122251700 1 -- 172.21.15.40:0/697332572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f911c15c7c0 msgr2=0x7f911c0c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:25.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9122251700 1 --2- 172.21.15.40:0/697332572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f911c15c7c0 0x7f911c0c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:25.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9122251700 1 -- 172.21.15.40:0/697332572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9118009570 con 0x7f911c0cd700 2023-11-08T08:22:25.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9122a52700 1 --2- 172.21.15.40:0/697332572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0caff0 0x7f911c0cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:22:25.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9121a50700 1 --2- 172.21.15.40:0/697332572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f911c15c7c0 0x7f911c0c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:22:25.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9122251700 1 --2- 172.21.15.40:0/697332572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f911c0cd700 0x7f911c15ad40 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f911c06fdb0 tx=0x7f9118004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:25.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9112ffd700 1 -- 172.21.15.40:0/697332572 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118004380 con 0x7f911c0cd700 2023-11-08T08:22:25.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9112ffd700 1 -- 172.21.15.40:0/697332572 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911801f070 con 0x7f911c0cd700 2023-11-08T08:22:25.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9112ffd700 1 -- 172.21.15.40:0/697332572 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91180036c0 con 0x7f911c0cd700 2023-11-08T08:22:25.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.875+0000 7f9123253700 1 -- 172.21.15.40:0/697332572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911c15cb90 con 0x7f911c0cd700 2023-11-08T08:22:25.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.879+0000 7f9123253700 1 -- 172.21.15.40:0/697332572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911c06c8b0 con 0x7f911c0cd700 2023-11-08T08:22:25.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.879+0000 7f9123253700 1 -- 172.21.15.40:0/697332572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911c0ce330 con 0x7f911c0cd700 2023-11-08T08:22:25.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.879+0000 7f9112ffd700 1 -- 172.21.15.40:0/697332572 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f911802a020 con 0x7f911c0cd700 2023-11-08T08:22:25.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.883+0000 7f9112ffd700 1 --2- 172.21.15.40:0/697332572 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9100064f30 0x7f91000673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:25.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.883+0000 7f9112ffd700 1 -- 172.21.15.40:0/697332572 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9118083490 con 0x7f911c0cd700 2023-11-08T08:22:25.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.883+0000 7f9121a50700 1 --2- 172.21.15.40:0/697332572 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9100064f30 0x7f91000673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:25.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.883+0000 7f9121a50700 1 --2- 172.21.15.40:0/697332572 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9100064f30 0x7f91000673e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f911c0d1f40 tx=0x7f910c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:25.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:25.887+0000 7f9112ffd700 1 -- 172.21.15.40:0/697332572 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f9118051480 con 0x7f911c0cd700 2023-11-08T08:22:26.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.131+0000 7f9123253700 1 -- 172.21.15.40:0/697332572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f911c073cb0 con 0x7f911c0cd700 2023-11-08T08:22:26.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.139+0000 7f9112ffd700 1 -- 172.21.15.40:0/697332572 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7f9118054aa0 con 0x7f911c0cd700 2023-11-08T08:22:26.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.143+0000 7f9123253700 1 -- 172.21.15.40:0/697332572 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9100064f30 msgr2=0x7f91000673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:26.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.143+0000 7f9123253700 1 --2- 172.21.15.40:0/697332572 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9100064f30 0x7f91000673e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f911c0d1f40 tx=0x7f910c008040 comp rx=0 tx=0).stop 2023-11-08T08:22:26.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.143+0000 7f9123253700 1 -- 172.21.15.40:0/697332572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f911c0cd700 msgr2=0x7f911c15ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:26.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.143+0000 7f9123253700 1 --2- 172.21.15.40:0/697332572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f911c0cd700 0x7f911c15ad40 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f911c06fdb0 tx=0x7f9118004770 comp rx=0 tx=0).stop 2023-11-08T08:22:26.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.143+0000 7f9123253700 1 -- 172.21.15.40:0/697332572 shutdown_connections 2023-11-08T08:22:26.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.143+0000 7f9123253700 1 --2- 172.21.15.40:0/697332572 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9100064f30 0x7f91000673e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:26.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.143+0000 7f9123253700 1 --2- 172.21.15.40:0/697332572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f911c0caff0 0x7f911c0cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:26.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.143+0000 7f9123253700 1 --2- 172.21.15.40:0/697332572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f911c15c7c0 0x7f911c0c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:26.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.143+0000 7f9123253700 1 --2- 172.21.15.40:0/697332572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f911c0cd700 0x7f911c15ad40 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:26.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.143+0000 7f9123253700 1 -- 172.21.15.40:0/697332572 >> 172.21.15.40:0/697332572 conn(0x7f911c0d3800 msgr2=0x7f911c0b79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:26.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.143+0000 7f9123253700 1 -- 172.21.15.40:0/697332572 shutdown_connections 2023-11-08T08:22:26.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.143+0000 7f9123253700 1 -- 172.21.15.40:0/697332572 wait complete. 2023-11-08T08:22:26.154 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2023-11-08T08:22:26.173 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:22:26.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.347+0000 7f8f7cc66700 1 Processor -- start 2023-11-08T08:22:26.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.347+0000 7f8f7cc66700 1 -- start start 2023-11-08T08:22:26.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.347+0000 7f8f7cc66700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8f780b1640 0x7f8f78070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:26.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.347+0000 7f8f7cc66700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f7815a550 con 0x7f8f780b43a0 2023-11-08T08:22:26.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.347+0000 7f8f7cc66700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f7806b0a0 con 0x7f8f780c9040 2023-11-08T08:22:26.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.347+0000 7f8f7cc66700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f780caac0 con 0x7f8f780b1640 2023-11-08T08:22:26.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.347+0000 7f8f777fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8f780b43a0 0x7f8f7815e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49004/0 (socket says 172.21.15.40:49004) 2023-11-08T08:22:26.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.347+0000 7f8f777fe700 1 -- 172.21.15.40:0/1514505647 learned_addr learned my addr 172.21.15.40:0/1514505647 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:26.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.347+0000 7f8f6effd700 1 --2- 172.21.15.40:0/1514505647 >> v2:172.21.15.169:3300/0 conn(0x7f8f780b1640 0x7f8f78070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:26.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.347+0000 7f8f76ffd700 1 -- 172.21.15.40:0/1514505647 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1681319076 0 0) 0x7f8f68001110 con 0x7f8f780b43a0 2023-11-08T08:22:26.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f76ffd700 1 -- 172.21.15.40:0/1514505647 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f640035d0 con 0x7f8f780b43a0 2023-11-08T08:22:26.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f76ffd700 1 -- 172.21.15.40:0/1514505647 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3081101976 0 0) 0x7f8f640035d0 con 0x7f8f780b43a0 2023-11-08T08:22:26.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f76ffd700 1 -- 172.21.15.40:0/1514505647 >> v2:172.21.15.169:3300/0 conn(0x7f8f780b1640 msgr2=0x7f8f78070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:26.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f76ffd700 1 --2- 172.21.15.40:0/1514505647 >> v2:172.21.15.169:3300/0 conn(0x7f8f780b1640 0x7f8f78070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:26.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f76ffd700 1 -- 172.21.15.40:0/1514505647 >> v1:172.21.15.169:6789/0 conn(0x7f8f780c9040 legacy=0x7f8f7806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:26.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f76ffd700 1 -- 172.21.15.40:0/1514505647 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f780cada0 con 0x7f8f780b43a0 2023-11-08T08:22:26.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f76ffd700 1 -- 172.21.15.40:0/1514505647 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f8f68001800 con 0x7f8f780b43a0 2023-11-08T08:22:26.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f76ffd700 1 -- 172.21.15.40:0/1514505647 >> v1:172.21.15.40:6789/0 conn(0x7f8f780b43a0 legacy=0x7f8f7815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:26.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f76ffd700 1 --2- 172.21.15.40:0/1514505647 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f64003a20 0x7f8f64003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:26.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f76ffd700 1 --2- 172.21.15.40:0/1514505647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f64004870 0x7f8f64004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:26.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f76ffd700 1 --2- 172.21.15.40:0/1514505647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f64005220 0x7f8f6400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:26.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f76ffd700 1 -- 172.21.15.40:0/1514505647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f780caac0 con 0x7f8f64005220 2023-11-08T08:22:26.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f76ffd700 1 -- 172.21.15.40:0/1514505647 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6400c130 con 0x7f8f64004870 2023-11-08T08:22:26.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f76ffd700 1 -- 172.21.15.40:0/1514505647 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6400c2f0 con 0x7f8f64003a20 2023-11-08T08:22:26.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f77fff700 1 --2- 172.21.15.40:0/1514505647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f64005220 0x7f8f6400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:26.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f6effd700 1 --2- 172.21.15.40:0/1514505647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f64004870 0x7f8f64004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:26.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f777fe700 1 --2- 172.21.15.40:0/1514505647 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f64003a20 0x7f8f64003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:26.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f77fff700 1 -- 172.21.15.40:0/1514505647 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f64003a20 msgr2=0x7f8f64003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f77fff700 1 --2- 172.21.15.40:0/1514505647 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f64003a20 0x7f8f64003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f77fff700 1 -- 172.21.15.40:0/1514505647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f64004870 msgr2=0x7f8f64004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f77fff700 1 --2- 172.21.15.40:0/1514505647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f64004870 0x7f8f64004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f77fff700 1 -- 172.21.15.40:0/1514505647 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f640035d0 con 0x7f8f64005220 2023-11-08T08:22:26.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f777fe700 1 --2- 172.21.15.40:0/1514505647 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f64003a20 0x7f8f64003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:22:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f77fff700 1 -- 172.21.15.40:0/1514505647 reap_dead start 2023-11-08T08:22:26.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f77fff700 1 --2- 172.21.15.40:0/1514505647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f64005220 0x7f8f6400ba30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000c190 tx=0x7f8f640058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91587bfd4b08f069 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:26.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f76ffd700 1 -- 172.21.15.40:0/1514505647 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f70001db0 con 0x7f8f64005220 2023-11-08T08:22:26.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f76ffd700 1 -- 172.21.15.40:0/1514505647 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7000ccd0 con 0x7f8f64005220 2023-11-08T08:22:26.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.351+0000 7f8f76ffd700 1 -- 172.21.15.40:0/1514505647 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f70003c60 con 0x7f8f64005220 2023-11-08T08:22:26.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.355+0000 7f8f7cc66700 1 -- 172.21.15.40:0/1514505647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f64005220 msgr2=0x7f8f6400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:26.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.355+0000 7f8f7cc66700 1 --2- 172.21.15.40:0/1514505647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f64005220 0x7f8f6400ba30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000c190 tx=0x7f8f640058a0 comp rx=0 tx=0).stop 2023-11-08T08:22:26.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.355+0000 7f8f7cc66700 1 -- 172.21.15.40:0/1514505647 shutdown_connections 2023-11-08T08:22:26.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.355+0000 7f8f7cc66700 1 --2- 172.21.15.40:0/1514505647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f64005220 0x7f8f6400ba30 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:26.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.355+0000 7f8f7cc66700 1 -- 172.21.15.40:0/1514505647 >> 172.21.15.40:0/1514505647 conn(0x7f8f780b0f80 msgr2=0x7f8f780ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:26.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.355+0000 7f8f7cc66700 1 -- 172.21.15.40:0/1514505647 shutdown_connections 2023-11-08T08:22:26.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.355+0000 7f8f7cc66700 1 -- 172.21.15.40:0/1514505647 wait complete. 2023-11-08T08:22:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.355+0000 7f8f7cc66700 1 Processor -- start 2023-11-08T08:22:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.355+0000 7f8f7cc66700 1 -- start start 2023-11-08T08:22:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f7cc66700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f64005220 0x7f8f780c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f7cc66700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f7814ebe0 0x7f8f78157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f7cc66700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f7814df30 0x7f8f7814e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f7cc66700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f780c2cd0 con 0x7f8f7814ebe0 2023-11-08T08:22:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f7cc66700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f780c3530 con 0x7f8f64005220 2023-11-08T08:22:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f7cc66700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f780c2f70 con 0x7f8f7814df30 2023-11-08T08:22:26.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f6effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f7814ebe0 0x7f8f78157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f6effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f7814ebe0 0x7f8f78157b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35946/0 (socket says 172.21.15.40:35946) 2023-11-08T08:22:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f6effd700 1 -- 172.21.15.40:0/860532201 learned_addr learned my addr 172.21.15.40:0/860532201 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f77fff700 1 --2- 172.21.15.40:0/860532201 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f7814df30 0x7f8f7814e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f777fe700 1 --2- 172.21.15.40:0/860532201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f64005220 0x7f8f780c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f6effd700 1 -- 172.21.15.40:0/860532201 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f7814df30 msgr2=0x7f8f7814e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f6effd700 1 --2- 172.21.15.40:0/860532201 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f7814df30 0x7f8f7814e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:26.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f6effd700 1 -- 172.21.15.40:0/860532201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f64005220 msgr2=0x7f8f780c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:26.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f6effd700 1 --2- 172.21.15.40:0/860532201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f64005220 0x7f8f780c4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:26.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f6effd700 1 -- 172.21.15.40:0/860532201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7806b5d0 con 0x7f8f7814ebe0 2023-11-08T08:22:26.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f6effd700 1 --2- 172.21.15.40:0/860532201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f7814ebe0 0x7f8f78157b30 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8f600108f0 tx=0x7f8f60010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:26.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f74ff9700 1 -- 172.21.15.40:0/860532201 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60016670 con 0x7f8f7814ebe0 2023-11-08T08:22:26.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f74ff9700 1 -- 172.21.15.40:0/860532201 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f600167d0 con 0x7f8f7814ebe0 2023-11-08T08:22:26.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f74ff9700 1 -- 172.21.15.40:0/860532201 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60016b60 con 0x7f8f7814ebe0 2023-11-08T08:22:26.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f7cc66700 1 -- 172.21.15.40:0/860532201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f7806b2f0 con 0x7f8f7814ebe0 2023-11-08T08:22:26.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.359+0000 7f8f7cc66700 1 -- 172.21.15.40:0/860532201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f780ca7b0 con 0x7f8f7814ebe0 2023-11-08T08:22:26.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.363+0000 7f8f7cc66700 1 -- 172.21.15.40:0/860532201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f78073cb0 con 0x7f8f7814ebe0 2023-11-08T08:22:26.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.363+0000 7f8f74ff9700 1 -- 172.21.15.40:0/860532201 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f8f6001c030 con 0x7f8f7814ebe0 2023-11-08T08:22:26.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.363+0000 7f8f74ff9700 1 --2- 172.21.15.40:0/860532201 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8f58064f10 0x7f8f580673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:26.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.367+0000 7f8f74ff9700 1 -- 172.21.15.40:0/860532201 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8f60082160 con 0x7f8f7814ebe0 2023-11-08T08:22:26.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.367+0000 7f8f777fe700 1 --2- 172.21.15.40:0/860532201 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8f58064f10 0x7f8f580673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:26.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.367+0000 7f8f777fe700 1 --2- 172.21.15.40:0/860532201 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8f58064f10 0x7f8f580673c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8f68001930 tx=0x7f8f68009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:26.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.371+0000 7f8f74ff9700 1 -- 172.21.15.40:0/860532201 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f8f600500d0 con 0x7f8f7814ebe0 2023-11-08T08:22:26.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.579+0000 7f8f7cc66700 1 -- 172.21.15.40:0/860532201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f78060910 con 0x7f8f7814ebe0 2023-11-08T08:22:26.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.583+0000 7f8f74ff9700 1 -- 172.21.15.40:0/860532201 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f8f600536f0 con 0x7f8f7814ebe0 2023-11-08T08:22:26.586 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:22:26.586 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/307473250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":307473250},{"type":"v1","addr":"172.21.15.169:6835","nonce":307473250}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6148,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/924518366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":924518366},{"type":"v1","addr":"172.21.15.169:6837","nonce":924518366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4294005934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4294005934},{"type":"v1","addr":"172.21.15.40:6833","nonce":4294005934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:22:20.650687+0000","modified":"2023-11-08T08:22:23.857369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6237":{"gid":6237,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3395630689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3395630689},{"type":"v1","addr":"172.21.15.40:6835","nonce":3395630689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T08:22:26.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.583+0000 7f8f7cc66700 1 -- 172.21.15.40:0/860532201 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8f58064f10 msgr2=0x7f8f580673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.583+0000 7f8f7cc66700 1 --2- 172.21.15.40:0/860532201 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8f58064f10 0x7f8f580673c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8f68001930 tx=0x7f8f68009040 comp rx=0 tx=0).stop 2023-11-08T08:22:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.583+0000 7f8f7cc66700 1 -- 172.21.15.40:0/860532201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f7814ebe0 msgr2=0x7f8f78157b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.583+0000 7f8f7cc66700 1 --2- 172.21.15.40:0/860532201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f7814ebe0 0x7f8f78157b30 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8f600108f0 tx=0x7f8f60010c00 comp rx=0 tx=0).stop 2023-11-08T08:22:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.583+0000 7f8f7cc66700 1 -- 172.21.15.40:0/860532201 shutdown_connections 2023-11-08T08:22:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.583+0000 7f8f7cc66700 1 --2- 172.21.15.40:0/860532201 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8f58064f10 0x7f8f580673c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:26.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.583+0000 7f8f7cc66700 1 --2- 172.21.15.40:0/860532201 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f7814df30 0x7f8f7814e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:26.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.583+0000 7f8f7cc66700 1 --2- 172.21.15.40:0/860532201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f7814ebe0 0x7f8f78157b30 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:26.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.583+0000 7f8f7cc66700 1 --2- 172.21.15.40:0/860532201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f64005220 0x7f8f780c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:26.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.583+0000 7f8f7cc66700 1 -- 172.21.15.40:0/860532201 >> 172.21.15.40:0/860532201 conn(0x7f8f780b0f80 msgr2=0x7f8f780aeee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:26.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.583+0000 7f8f7cc66700 1 -- 172.21.15.40:0/860532201 shutdown_connections 2023-11-08T08:22:26.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:26.583+0000 7f8f7cc66700 1 -- 172.21.15.40:0/860532201 wait complete. 2023-11-08T08:22:26.591 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 33 2023-11-08T08:22:26.857 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:22:26.861 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:22:27.013 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T08:22:20.650687+0000', 'modified': '2023-11-08T08:22:23.857369+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6237}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6237': {'gid': 6237, 'name': 'c', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6835/3395630689', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 3395630689}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 3395630689}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T08:22:27.014 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T08:22:27.014 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-08T08:22:27.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.179+0000 7f3fdd6c4700 1 Processor -- start 2023-11-08T08:22:27.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.179+0000 7f3fdd6c4700 1 -- start start 2023-11-08T08:22:27.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.183+0000 7f3fdd6c4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3fd8158f70 0x7f3fd80ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.183+0000 7f3fdd6c4700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd8159340 con 0x7f3fd8071100 2023-11-08T08:22:27.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.183+0000 7f3fdd6c4700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd80cea00 con 0x7f3fd80cd410 2023-11-08T08:22:27.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.183+0000 7f3fdd6c4700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd806b3c0 con 0x7f3fd8158f70 2023-11-08T08:22:27.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.183+0000 7f3fd7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3fd8158f70 0x7f3fd80ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.183+0000 7f3fd7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3fd8158f70 0x7f3fd80ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35950/0 (socket says 172.21.15.40:35950) 2023-11-08T08:22:27.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.183+0000 7f3fd7fff700 1 -- 172.21.15.40:0/4250892889 learned_addr learned my addr 172.21.15.40:0/4250892889 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:27.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.183+0000 7f3fdcec3700 1 --1- 172.21.15.40:0/4250892889 >> v1:172.21.15.40:6789/0 conn(0x7f3fd8071100 0x7f3fd80ca5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49018/0 (socket says 172.21.15.40:49018) 2023-11-08T08:22:27.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.183+0000 7f3fd7fff700 1 -- 172.21.15.40:0/4250892889 >> v1:172.21.15.169:6789/0 conn(0x7f3fd80cd410 legacy=0x7f3fd806c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:27.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.183+0000 7f3fd7fff700 1 -- 172.21.15.40:0/4250892889 >> v1:172.21.15.40:6789/0 conn(0x7f3fd8071100 legacy=0x7f3fd80ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:27.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.183+0000 7f3fd6ffd700 1 -- 172.21.15.40:0/4250892889 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3727214586 0 0) 0x7f3fd80cea00 con 0x7f3fd80cd410 2023-11-08T08:22:27.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.183+0000 7f3fd7fff700 1 -- 172.21.15.40:0/4250892889 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd806b6a0 con 0x7f3fd8158f70 2023-11-08T08:22:27.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.183+0000 7f3fd7fff700 1 --2- 172.21.15.40:0/4250892889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd8158f70 0x7f3fd80ce320 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009910 tx=0x7f3fc8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6d6a08bee61119 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:27.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.183+0000 7f3fd6ffd700 1 -- 172.21.15.40:0/4250892889 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc800ba60 con 0x7f3fd8158f70 2023-11-08T08:22:27.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.183+0000 7f3fd6ffd700 1 -- 172.21.15.40:0/4250892889 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc800bbc0 con 0x7f3fd8158f70 2023-11-08T08:22:27.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.183+0000 7f3fd6ffd700 1 -- 172.21.15.40:0/4250892889 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8009e60 con 0x7f3fd8158f70 2023-11-08T08:22:27.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.187+0000 7f3fdd6c4700 1 -- 172.21.15.40:0/4250892889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd8158f70 msgr2=0x7f3fd80ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:27.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.187+0000 7f3fdd6c4700 1 --2- 172.21.15.40:0/4250892889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd8158f70 0x7f3fd80ce320 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009910 tx=0x7f3fc8009c20 comp rx=0 tx=0).stop 2023-11-08T08:22:27.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.187+0000 7f3fdd6c4700 1 -- 172.21.15.40:0/4250892889 shutdown_connections 2023-11-08T08:22:27.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.187+0000 7f3fdd6c4700 1 --2- 172.21.15.40:0/4250892889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd8158f70 0x7f3fd80ce320 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.187+0000 7f3fdd6c4700 1 -- 172.21.15.40:0/4250892889 >> 172.21.15.40:0/4250892889 conn(0x7f3fd80b1200 msgr2=0x7f3fd80b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:27.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.187+0000 7f3fdd6c4700 1 -- 172.21.15.40:0/4250892889 shutdown_connections 2023-11-08T08:22:27.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.187+0000 7f3fdd6c4700 1 -- 172.21.15.40:0/4250892889 wait complete. 2023-11-08T08:22:27.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.187+0000 7f3fdd6c4700 1 Processor -- start 2023-11-08T08:22:27.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.187+0000 7f3fdd6c4700 1 -- start start 2023-11-08T08:22:27.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fdd6c4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd8071100 0x7f3fd8151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fdd6c4700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fd80cd410 0x7f3fd80c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fdd6c4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fd8158f70 0x7f3fd80b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fdd6c4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd80c8af0 con 0x7f3fd8071100 2023-11-08T08:22:27.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fdd6c4700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd80c7f20 con 0x7f3fd8158f70 2023-11-08T08:22:27.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fdd6c4700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd80c81c0 con 0x7f3fd80cd410 2023-11-08T08:22:27.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fd7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd8071100 0x7f3fd8151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fd7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd8071100 0x7f3fd8151be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35956/0 (socket says 172.21.15.40:35956) 2023-11-08T08:22:27.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fd7fff700 1 -- 172.21.15.40:0/3597766947 learned_addr learned my addr 172.21.15.40:0/3597766947 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:27.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fdcec3700 1 --2- 172.21.15.40:0/3597766947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fd8158f70 0x7f3fd80b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fd77fe700 1 --2- 172.21.15.40:0/3597766947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fd80cd410 0x7f3fd80c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fd7fff700 1 -- 172.21.15.40:0/3597766947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fd80cd410 msgr2=0x7f3fd80c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:27.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fd7fff700 1 --2- 172.21.15.40:0/3597766947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fd80cd410 0x7f3fd80c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fd7fff700 1 -- 172.21.15.40:0/3597766947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fd8158f70 msgr2=0x7f3fd80b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:27.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fd7fff700 1 --2- 172.21.15.40:0/3597766947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fd8158f70 0x7f3fd80b4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fd7fff700 1 -- 172.21.15.40:0/3597766947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc8009570 con 0x7f3fd8071100 2023-11-08T08:22:27.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fd7fff700 1 --2- 172.21.15.40:0/3597766947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd8071100 0x7f3fd8151be0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8000c00 tx=0x7f3fc8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:27.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fd4ff9700 1 -- 172.21.15.40:0/3597766947 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8011ab0 con 0x7f3fd8071100 2023-11-08T08:22:27.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fd4ff9700 1 -- 172.21.15.40:0/3597766947 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc801f070 con 0x7f3fd8071100 2023-11-08T08:22:27.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fdd6c4700 1 -- 172.21.15.40:0/3597766947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd806bc60 con 0x7f3fd8071100 2023-11-08T08:22:27.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fd4ff9700 1 -- 172.21.15.40:0/3597766947 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8011e50 con 0x7f3fd8071100 2023-11-08T08:22:27.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.191+0000 7f3fdd6c4700 1 -- 172.21.15.40:0/3597766947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd806b980 con 0x7f3fd8071100 2023-11-08T08:22:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.195+0000 7f3fd4ff9700 1 -- 172.21.15.40:0/3597766947 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f3fc800f050 con 0x7f3fd8071100 2023-11-08T08:22:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.199+0000 7f3fd4ff9700 1 --2- 172.21.15.40:0/3597766947 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3fc4064ee0 0x7f3fc4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.199+0000 7f3fd4ff9700 1 -- 172.21.15.40:0/3597766947 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3fc8021070 con 0x7f3fd8071100 2023-11-08T08:22:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.199+0000 7f3fd77fe700 1 --2- 172.21.15.40:0/3597766947 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3fc4064ee0 0x7f3fc4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.199+0000 7f3fe1bac740 1 Processor -- start 2023-11-08T08:22:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.199+0000 7f3fd77fe700 1 --2- 172.21.15.40:0/3597766947 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3fc4064ee0 0x7f3fc4067390 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80b43a0 tx=0x7f3fcc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.199+0000 7f3fe1bac740 1 -- start start 2023-11-08T08:22:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fe1bac740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4ae0 0x1094e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fe1bac740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xde4710 0x1095340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fe1bac740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xe2f790 0x1095880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fe1bac740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0xf53ae0 con 0xde4710 2023-11-08T08:22:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fe1bac740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0xfc3bd0 con 0xe2f790 2023-11-08T08:22:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fe1bac740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0xfbced0 con 0xde4ae0 2023-11-08T08:22:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fd77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xde4710 0x1095340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fd77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xde4710 0x1095340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35972/0 (socket says 172.21.15.40:35972) 2023-11-08T08:22:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fd77fe700 1 -- 172.21.15.40:0/4073926017 learned_addr learned my addr 172.21.15.40:0/4073926017 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fd7fff700 1 --2- 172.21.15.40:0/4073926017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4ae0 0x1094e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fdcec3700 1 --2- 172.21.15.40:0/4073926017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xe2f790 0x1095880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fd77fe700 1 -- 172.21.15.40:0/4073926017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4ae0 msgr2=0x1094e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fd77fe700 1 --2- 172.21.15.40:0/4073926017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4ae0 0x1094e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fd77fe700 1 -- 172.21.15.40:0/4073926017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xe2f790 msgr2=0x1095880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fd77fe700 1 --2- 172.21.15.40:0/4073926017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xe2f790 0x1095880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fd77fe700 1 -- 172.21.15.40:0/4073926017 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xfe2cc0 con 0xde4710 2023-11-08T08:22:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fd7fff700 1 --2- 172.21.15.40:0/4073926017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4ae0 0x1094e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:22:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fd77fe700 1 --2- 172.21.15.40:0/4073926017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xde4710 0x1095340 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc005320 tx=0x7f3fcc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c3273ee97efa85 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fbdffb700 1 -- 172.21.15.40:0/4073926017 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc003d40 con 0xde4710 2023-11-08T08:22:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fbdffb700 1 -- 172.21.15.40:0/4073926017 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fcc022420 con 0xde4710 2023-11-08T08:22:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fbdffb700 1 -- 172.21.15.40:0/4073926017 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc003ea0 con 0xde4710 2023-11-08T08:22:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fe1bac740 1 -- 172.21.15.40:0/4073926017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xde4710 msgr2=0x1095340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:27.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.203+0000 7f3fe1bac740 1 --2- 172.21.15.40:0/4073926017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xde4710 0x1095340 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc005320 tx=0x7f3fcc011000 comp rx=0 tx=0).stop 2023-11-08T08:22:27.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fe1bac740 1 -- 172.21.15.40:0/4073926017 shutdown_connections 2023-11-08T08:22:27.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fe1bac740 1 --2- 172.21.15.40:0/4073926017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xe2f790 0x1095880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fe1bac740 1 --2- 172.21.15.40:0/4073926017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xde4710 0x1095340 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fe1bac740 1 --2- 172.21.15.40:0/4073926017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4ae0 0x1094e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fe1bac740 1 -- 172.21.15.40:0/4073926017 >> 172.21.15.40:0/4073926017 conn(0xde4eb0 msgr2=0x107d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:27.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fe1bac740 1 -- 172.21.15.40:0/4073926017 shutdown_connections 2023-11-08T08:22:27.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fe1bac740 1 -- 172.21.15.40:0/4073926017 wait complete. 2023-11-08T08:22:27.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fe1bac740 1 -- start start 2023-11-08T08:22:27.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fe1bac740 1 Processor -- start 2023-11-08T08:22:27.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fe1bac740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4710 0x1056d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fe1bac740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xde4ae0 0x106d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fe1bac740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2f790 0x107ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fe1bac740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0xe69b20 con 0xe2f790 2023-11-08T08:22:27.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fe1bac740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1057260 con 0xde4ae0 2023-11-08T08:22:27.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fe1bac740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x106db00 con 0xde4710 2023-11-08T08:22:27.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fe1bac740 10 client.0 _ll_register_callbacks cb 0x10190d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:22:27.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fdcec3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2f790 0x107ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fdcec3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2f790 0x107ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35982/0 (socket says 172.21.15.40:35982) 2023-11-08T08:22:27.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fdcec3700 1 -- 172.21.15.40:0/1351134086 learned_addr learned my addr 172.21.15.40:0/1351134086 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:27.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fd7fff700 1 --2- 172.21.15.40:0/1351134086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4710 0x1056d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.207+0000 7f3fd77fe700 1 --2- 172.21.15.40:0/1351134086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xde4ae0 0x106d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fdcec3700 1 -- 172.21.15.40:0/1351134086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4710 msgr2=0x1056d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:27.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fdcec3700 1 --2- 172.21.15.40:0/1351134086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4710 0x1056d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fdcec3700 1 -- 172.21.15.40:0/1351134086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xde4ae0 msgr2=0x106d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:27.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fdcec3700 1 --2- 172.21.15.40:0/1351134086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xde4ae0 0x106d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fdcec3700 1 -- 172.21.15.40:0/1351134086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fcc00f040 con 0xe2f790 2023-11-08T08:22:27.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fdcec3700 1 --2- 172.21.15.40:0/1351134086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2f790 0x107ca70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0007b10 tx=0x7f3fd0011530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ae6e3e26d641df server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:27.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:22:27.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fb6ffd700 1 -- 172.21.15.40:0/1351134086 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0011a30 con 0xe2f790 2023-11-08T08:22:27.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fb6ffd700 1 -- 172.21.15.40:0/1351134086 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3fd0011b90 con 0xe2f790 2023-11-08T08:22:27.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fe1bac740 1 -- 172.21.15.40:0/1351134086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xfb5f50 con 0xe2f790 2023-11-08T08:22:27.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fb6ffd700 1 -- 172.21.15.40:0/1351134086 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fd0011d50 con 0xe2f790 2023-11-08T08:22:27.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fe1bac740 10 client.6393 fetch_fsmap learned FSMap version 33 2023-11-08T08:22:27.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fe1bac740 1 -- 172.21.15.40:0/1351134086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0xe309e0 con 0xe2f790 2023-11-08T08:22:27.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fb6ffd700 1 -- 172.21.15.40:0/1351134086 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xfb5f50 con 0xe2f790 2023-11-08T08:22:27.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fb6ffd700 1 -- 172.21.15.40:0/1351134086 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f3fd001e420 con 0xe2f790 2023-11-08T08:22:27.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fe1bac740 10 client.6393 fetch_fsmap finished waiting for FSMap version 33 2023-11-08T08:22:27.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fe1bac740 10 client.6393 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T08:22:27.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fe1bac740 20 client.6393 populate_metadata read hostname 'smithi040' 2023-11-08T08:22:27.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fe1bac740 1 --2- 172.21.15.40:0/1351134086 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x106bb80 0x10aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fe1bac740 4 client.6393 mds_command: new command op to 6237 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-08T08:22:27.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fe1bac740 1 -- 172.21.15.40:0/1351134086 --> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xfb5c70 con 0x106bb80 2023-11-08T08:22:27.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.211+0000 7f3fd7fff700 1 --2- 172.21.15.40:0/1351134086 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x106bb80 0x10aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.215+0000 7f3fd7fff700 1 --2- 172.21.15.40:0/1351134086 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x106bb80 0x10aefc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64df88450b5a62a9 server_cookie=2c970c17aeda3654 in_seq=0 out_seq=0 2023-11-08T08:22:27.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.215+0000 7f3fb6ffd700 10 client.6393 ms_handle_connect on v2:172.21.15.40:6834/3395630689 2023-11-08T08:22:27.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.215+0000 7f3fb6ffd700 1 -- 172.21.15.40:0/1351134086 <== mds.0 v2:172.21.15.40:6834/3395630689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0xfb5c70 con 0x106bb80 2023-11-08T08:22:27.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.215+0000 7f3fb6ffd700 10 client.6393 handle_command_reply: tid=0 2023-11-08T08:22:27.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.215+0000 7f3fe1bac740 1 client.6393 shutdown 2023-11-08T08:22:27.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.215+0000 7f3fe1bac740 1 -- 172.21.15.40:0/1351134086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2f790 msgr2=0x107ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:27.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.215+0000 7f3fe1bac740 1 --2- 172.21.15.40:0/1351134086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2f790 0x107ca70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0007b10 tx=0x7f3fd0011530 comp rx=0 tx=0).stop 2023-11-08T08:22:27.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.215+0000 7f3fe1bac740 1 -- 172.21.15.40:0/1351134086 shutdown_connections 2023-11-08T08:22:27.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.215+0000 7f3fe1bac740 1 --2- 172.21.15.40:0/1351134086 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x106bb80 0x10aefc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.215+0000 7f3fe1bac740 1 --2- 172.21.15.40:0/1351134086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2f790 0x107ca70 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.215+0000 7f3fe1bac740 1 --2- 172.21.15.40:0/1351134086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xde4ae0 0x106d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.215+0000 7f3fb6ffd700 0 client.6393 ms_handle_reset on v2:172.21.15.40:6834/3395630689 2023-11-08T08:22:27.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.215+0000 7f3fe1bac740 1 --2- 172.21.15.40:0/1351134086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4710 0x1056d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.215+0000 7f3fe1bac740 1 -- 172.21.15.40:0/1351134086 >> 172.21.15.40:0/1351134086 conn(0xde4eb0 msgr2=0x107d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:27.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.215+0000 7f3fe1bac740 1 -- 172.21.15.40:0/1351134086 shutdown_connections 2023-11-08T08:22:27.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.215+0000 7f3fe1bac740 1 -- 172.21.15.40:0/1351134086 wait complete. 2023-11-08T08:22:27.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.215+0000 7f3fe1bac740 20 client.6393 trim_cache size 0 max 16384 2023-11-08T08:22:27.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.239+0000 7f3fe1bac740 1 Processor -- start 2023-11-08T08:22:27.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.239+0000 7f3fe1bac740 1 -- start start 2023-11-08T08:22:27.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.239+0000 7f3fe1bac740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4710 0x1092bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.239+0000 7f3fe1bac740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xde4ae0 0x107d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.239+0000 7f3fe1bac740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2f790 0x107daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.239+0000 7f3fe1bac740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0xf92de0 con 0xe2f790 2023-11-08T08:22:27.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.239+0000 7f3fe1bac740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0xf8dec0 con 0xde4ae0 2023-11-08T08:22:27.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.239+0000 7f3fe1bac740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0xfba2d0 con 0xde4710 2023-11-08T08:22:27.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.239+0000 7f3fdcec3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2f790 0x107daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.239+0000 7f3fdcec3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2f790 0x107daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35994/0 (socket says 172.21.15.40:35994) 2023-11-08T08:22:27.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.239+0000 7f3fdcec3700 1 -- 172.21.15.40:0/2343945074 learned_addr learned my addr 172.21.15.40:0/2343945074 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:27.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.239+0000 7f3fd77fe700 1 --2- 172.21.15.40:0/2343945074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xde4ae0 0x107d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.239+0000 7f3fd7fff700 1 --2- 172.21.15.40:0/2343945074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4710 0x1092bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.239+0000 7f3fdcec3700 1 -- 172.21.15.40:0/2343945074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4710 msgr2=0x1092bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:27.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.239+0000 7f3fdcec3700 1 --2- 172.21.15.40:0/2343945074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4710 0x1092bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.239+0000 7f3fdcec3700 1 -- 172.21.15.40:0/2343945074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xde4ae0 msgr2=0x107d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:27.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.239+0000 7f3fdcec3700 1 --2- 172.21.15.40:0/2343945074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xde4ae0 0x107d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.239+0000 7f3fdcec3700 1 -- 172.21.15.40:0/2343945074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd8069f50 con 0xe2f790 2023-11-08T08:22:27.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.239+0000 7f3fdcec3700 1 --2- 172.21.15.40:0/2343945074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2f790 0x107daa0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0021040 tx=0x7f3fd0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e1388f7d0f86f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:27.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.243+0000 7f3fbe7fc700 1 -- 172.21.15.40:0/2343945074 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0007720 con 0xe2f790 2023-11-08T08:22:27.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.243+0000 7f3fbe7fc700 1 -- 172.21.15.40:0/2343945074 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd000b050 con 0xe2f790 2023-11-08T08:22:27.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.243+0000 7f3fbe7fc700 1 -- 172.21.15.40:0/2343945074 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0001d30 con 0xe2f790 2023-11-08T08:22:27.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.243+0000 7f3fe1bac740 1 -- 172.21.15.40:0/2343945074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2f790 msgr2=0x107daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:27.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.243+0000 7f3fe1bac740 1 --2- 172.21.15.40:0/2343945074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2f790 0x107daa0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0021040 tx=0x7f3fd0015000 comp rx=0 tx=0).stop 2023-11-08T08:22:27.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.243+0000 7f3fe1bac740 1 -- 172.21.15.40:0/2343945074 shutdown_connections 2023-11-08T08:22:27.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.243+0000 7f3fe1bac740 1 --2- 172.21.15.40:0/2343945074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2f790 0x107daa0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.243+0000 7f3fe1bac740 1 --2- 172.21.15.40:0/2343945074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xde4ae0 0x107d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.243+0000 7f3fe1bac740 1 --2- 172.21.15.40:0/2343945074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4710 0x1092bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.243+0000 7f3fe1bac740 1 -- 172.21.15.40:0/2343945074 >> 172.21.15.40:0/2343945074 conn(0xde4eb0 msgr2=0x1070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:27.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.243+0000 7f3fe1bac740 1 -- 172.21.15.40:0/2343945074 shutdown_connections 2023-11-08T08:22:27.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.243+0000 7f3fe1bac740 1 -- 172.21.15.40:0/2343945074 wait complete. 2023-11-08T08:22:27.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.243+0000 7f3fe1bac740 1 -- start start 2023-11-08T08:22:27.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.243+0000 7f3fe1bac740 1 Processor -- start 2023-11-08T08:22:27.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.247+0000 7f3fe1bac740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xde4710 0x106c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.247+0000 7f3fe1bac740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4ae0 0x1068a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.247+0000 7f3fe1bac740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xe2f790 0x1103860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.247+0000 7f3fe1bac740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0xfba6e0 con 0xde4710 2023-11-08T08:22:27.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.247+0000 7f3fe1bac740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1066b60 con 0xe2f790 2023-11-08T08:22:27.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.247+0000 7f3fe1bac740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x106cda0 con 0xde4ae0 2023-11-08T08:22:27.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.247+0000 7f3fe1bac740 10 client.0 _ll_register_callbacks cb 0xf692d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:22:27.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.247+0000 7f3fd7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xde4710 0x106c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.247+0000 7f3fd7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xde4710 0x106c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35996/0 (socket says 172.21.15.40:35996) 2023-11-08T08:22:27.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.247+0000 7f3fd7fff700 1 -- 172.21.15.40:0/4276653326 learned_addr learned my addr 172.21.15.40:0/4276653326 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:27.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.247+0000 7f3fdcec3700 1 --2- 172.21.15.40:0/4276653326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xe2f790 0x1103860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.247+0000 7f3fd77fe700 1 --2- 172.21.15.40:0/4276653326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4ae0 0x1068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.247+0000 7f3fd7fff700 1 -- 172.21.15.40:0/4276653326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4ae0 msgr2=0x1068a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:27.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.247+0000 7f3fd7fff700 1 --2- 172.21.15.40:0/4276653326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4ae0 0x1068a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.247+0000 7f3fd7fff700 1 -- 172.21.15.40:0/4276653326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xe2f790 msgr2=0x1103860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:27.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.247+0000 7f3fd7fff700 1 --2- 172.21.15.40:0/4276653326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xe2f790 0x1103860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.247+0000 7f3fd7fff700 1 -- 172.21.15.40:0/4276653326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xe309e0 con 0xde4710 2023-11-08T08:22:27.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fd7fff700 1 --2- 172.21.15.40:0/4276653326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xde4710 0x106c860 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80556f0 tx=0x7f3fc805b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ea205e243fda7d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:27.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:22:27.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fb6ffd700 1 -- 172.21.15.40:0/4276653326 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc805d3c0 con 0xde4710 2023-11-08T08:22:27.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fb6ffd700 1 -- 172.21.15.40:0/4276653326 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3fc805d520 con 0xde4710 2023-11-08T08:22:27.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fe1bac740 1 -- 172.21.15.40:0/4276653326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fcc022420 con 0xde4710 2023-11-08T08:22:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fb6ffd700 1 -- 172.21.15.40:0/4276653326 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fc805b0b0 con 0xde4710 2023-11-08T08:22:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fe1bac740 10 client.6399 fetch_fsmap learned FSMap version 33 2023-11-08T08:22:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fe1bac740 1 -- 172.21.15.40:0/4276653326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0xf69a20 con 0xde4710 2023-11-08T08:22:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fb6ffd700 1 -- 172.21.15.40:0/4276653326 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fcc022420 con 0xde4710 2023-11-08T08:22:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fb6ffd700 1 -- 172.21.15.40:0/4276653326 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f3fc805b210 con 0xde4710 2023-11-08T08:22:27.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fe1bac740 10 client.6399 fetch_fsmap finished waiting for FSMap version 33 2023-11-08T08:22:27.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fe1bac740 10 client.6399 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T08:22:27.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fe1bac740 20 client.6399 populate_metadata read hostname 'smithi040' 2023-11-08T08:22:27.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fe1bac740 1 --2- 172.21.15.40:0/4276653326 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x1069b80 0x1069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fe1bac740 4 client.6399 mds_command: new command op to 6237 tid=0[{"prefix": "status"}] 2023-11-08T08:22:27.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fe1bac740 1 -- 172.21.15.40:0/4276653326 --> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] -- command(tid 0: {"prefix": "status"}) v1 -- 0xe61260 con 0x1069b80 2023-11-08T08:22:27.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fd77fe700 1 --2- 172.21.15.40:0/4276653326 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x1069b80 0x1069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fd77fe700 1 --2- 172.21.15.40:0/4276653326 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x1069b80 0x1069f50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0d9a223fce40d00 server_cookie=bf5e7bfb2ac45ab0 in_seq=0 out_seq=0 2023-11-08T08:22:27.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fb6ffd700 10 client.6399 ms_handle_connect on v2:172.21.15.40:6834/3395630689 2023-11-08T08:22:27.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fb6ffd700 1 -- 172.21.15.40:0/4276653326 <== mds.0 v2:172.21.15.40:6834/3395630689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0xe61260 con 0x1069b80 2023-11-08T08:22:27.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fb6ffd700 10 client.6399 handle_command_reply: tid=0 2023-11-08T08:22:27.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fe1bac740 1 client.6399 shutdown 2023-11-08T08:22:27.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fe1bac740 1 -- 172.21.15.40:0/4276653326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xde4710 msgr2=0x106c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:27.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.251+0000 7f3fe1bac740 1 --2- 172.21.15.40:0/4276653326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xde4710 0x106c860 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80556f0 tx=0x7f3fc805b040 comp rx=0 tx=0).stop 2023-11-08T08:22:27.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fe1bac740 1 -- 172.21.15.40:0/4276653326 shutdown_connections 2023-11-08T08:22:27.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fe1bac740 1 --2- 172.21.15.40:0/4276653326 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x1069b80 0x1069f50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fe1bac740 1 --2- 172.21.15.40:0/4276653326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xe2f790 0x1103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fe1bac740 1 --2- 172.21.15.40:0/4276653326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xde4ae0 0x1068a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fe1bac740 1 --2- 172.21.15.40:0/4276653326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xde4710 0x106c860 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fb6ffd700 0 client.6399 ms_handle_reset on v2:172.21.15.40:6834/3395630689 2023-11-08T08:22:27.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fe1bac740 1 -- 172.21.15.40:0/4276653326 >> 172.21.15.40:0/4276653326 conn(0xde4eb0 msgr2=0x1070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:27.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fe1bac740 1 -- 172.21.15.40:0/4276653326 shutdown_connections 2023-11-08T08:22:27.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fe1bac740 1 -- 172.21.15.40:0/4276653326 wait complete. 2023-11-08T08:22:27.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fe1bac740 20 client.6399 trim_cache size 0 max 16384 2023-11-08T08:22:27.265 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:22:27.266 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "54ed45f4-d5b0-41c3-8c3b-b5a18b844f25", 2023-11-08T08:22:27.266 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2023-11-08T08:22:27.266 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 6237, 2023-11-08T08:22:27.266 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-11-08T08:22:27.266 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-11-08T08:22:27.267 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-11-08T08:22:27.267 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 6.5975734069999996, 2023-11-08T08:22:27.267 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 33, 2023-11-08T08:22:27.267 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 69, 2023-11-08T08:22:27.267 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 69, 2023-11-08T08:22:27.268 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 12.802519171 2023-11-08T08:22:27.268 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:22:27.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fdd6c4700 1 -- 172.21.15.40:0/3597766947 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3fc4064ee0 msgr2=0x7f3fc4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:27.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fdd6c4700 1 --2- 172.21.15.40:0/3597766947 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3fc4064ee0 0x7f3fc4067390 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80b43a0 tx=0x7f3fcc009040 comp rx=0 tx=0).stop 2023-11-08T08:22:27.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fdd6c4700 1 -- 172.21.15.40:0/3597766947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd8071100 msgr2=0x7f3fd8151be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:27.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fdd6c4700 1 --2- 172.21.15.40:0/3597766947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd8071100 0x7f3fd8151be0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8000c00 tx=0x7f3fc8004770 comp rx=0 tx=0).stop 2023-11-08T08:22:27.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fdd6c4700 1 -- 172.21.15.40:0/3597766947 shutdown_connections 2023-11-08T08:22:27.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fdd6c4700 1 --2- 172.21.15.40:0/3597766947 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f3fc4064ee0 0x7f3fc4067390 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fdd6c4700 1 --2- 172.21.15.40:0/3597766947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fd8158f70 0x7f3fd80b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fdd6c4700 1 --2- 172.21.15.40:0/3597766947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fd80cd410 0x7f3fd80c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fdd6c4700 1 --2- 172.21.15.40:0/3597766947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd8071100 0x7f3fd8151be0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fdd6c4700 1 -- 172.21.15.40:0/3597766947 >> 172.21.15.40:0/3597766947 conn(0x7f3fd80b1200 msgr2=0x7f3fd8060660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:27.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fdd6c4700 1 -- 172.21.15.40:0/3597766947 shutdown_connections 2023-11-08T08:22:27.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.255+0000 7f3fdd6c4700 1 -- 172.21.15.40:0/3597766947 wait complete. 2023-11-08T08:22:27.283 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T08:22:27.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.471+0000 7faa90fa7700 1 Processor -- start 2023-11-08T08:22:27.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.471+0000 7faa90fa7700 1 -- start start 2023-11-08T08:22:27.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.471+0000 7faa90fa7700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faa8c15a480 0x7faa8c0cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.471+0000 7faa90fa7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa8c069d60 0x7faa8c06a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.471+0000 7faa90fa7700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa8c15a850 con 0x7faa8c0c8d10 2023-11-08T08:22:27.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.471+0000 7faa90fa7700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa8c0ce060 con 0x7faa8c15a480 2023-11-08T08:22:27.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.471+0000 7faa90fa7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c0ce300 con 0x7faa8c069d60 2023-11-08T08:22:27.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.475+0000 7faa8a59c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faa8c15a480 0x7faa8c0cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.475+0000 7faa8a59c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faa8c15a480 0x7faa8c0cc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33052/0 (socket says 172.21.15.40:33052) 2023-11-08T08:22:27.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.475+0000 7faa8a59c700 1 -- 172.21.15.40:0/388242020 learned_addr learned my addr 172.21.15.40:0/388242020 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:27.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.475+0000 7faa8ad9d700 1 --2- 172.21.15.40:0/388242020 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa8c069d60 0x7faa8c06a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.475+0000 7faa8a59c700 1 -- 172.21.15.40:0/388242020 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa8c069d60 msgr2=0x7faa8c06a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:27.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.475+0000 7faa8a59c700 1 --2- 172.21.15.40:0/388242020 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa8c069d60 0x7faa8c06a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.475+0000 7faa8a59c700 1 -- 172.21.15.40:0/388242020 >> v1:172.21.15.40:6789/0 conn(0x7faa8c0c8d10 legacy=0x7faa8c0cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:27.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.475+0000 7faa8a59c700 1 -- 172.21.15.40:0/388242020 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8c0cada0 con 0x7faa8c15a480 2023-11-08T08:22:27.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.475+0000 7faa8a59c700 1 --2- 172.21.15.40:0/388242020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa8c15a480 0x7faa8c0cc290 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009910 tx=0x7faa7c009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afabcac1b06c61d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:27.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.475+0000 7faa8959a700 1 -- 172.21.15.40:0/388242020 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c011830 con 0x7faa8c15a480 2023-11-08T08:22:27.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.475+0000 7faa8959a700 1 -- 172.21.15.40:0/388242020 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7c011990 con 0x7faa8c15a480 2023-11-08T08:22:27.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.475+0000 7faa8959a700 1 -- 172.21.15.40:0/388242020 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c011d40 con 0x7faa8c15a480 2023-11-08T08:22:27.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.475+0000 7faa90fa7700 1 -- 172.21.15.40:0/388242020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa8c15a480 msgr2=0x7faa8c0cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:27.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.475+0000 7faa90fa7700 1 --2- 172.21.15.40:0/388242020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa8c15a480 0x7faa8c0cc290 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009910 tx=0x7faa7c009c20 comp rx=0 tx=0).stop 2023-11-08T08:22:27.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.479+0000 7faa90fa7700 1 -- 172.21.15.40:0/388242020 shutdown_connections 2023-11-08T08:22:27.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.479+0000 7faa90fa7700 1 --2- 172.21.15.40:0/388242020 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa8c069d60 0x7faa8c06a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.479+0000 7faa90fa7700 1 --2- 172.21.15.40:0/388242020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa8c15a480 0x7faa8c0cc290 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.479+0000 7faa90fa7700 1 -- 172.21.15.40:0/388242020 >> 172.21.15.40:0/388242020 conn(0x7faa8c0cf350 msgr2=0x7faa8c0b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:27.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.479+0000 7faa90fa7700 1 -- 172.21.15.40:0/388242020 shutdown_connections 2023-11-08T08:22:27.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.479+0000 7faa90fa7700 1 -- 172.21.15.40:0/388242020 wait complete. 2023-11-08T08:22:27.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.479+0000 7faa90fa7700 1 Processor -- start 2023-11-08T08:22:27.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.483+0000 7faa90fa7700 1 -- start start 2023-11-08T08:22:27.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.483+0000 7faa90fa7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa8c069d60 0x7faa8c1f09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.483+0000 7faa90fa7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa8c0c8d10 0x7faa8c14eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.483+0000 7faa90fa7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa8c15a480 0x7faa8c0c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.483+0000 7faa90fa7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c15d550 con 0x7faa8c0c8d10 2023-11-08T08:22:27.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.483+0000 7faa90fa7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c0c6840 con 0x7faa8c069d60 2023-11-08T08:22:27.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.483+0000 7faa90fa7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c1f0f10 con 0x7faa8c15a480 2023-11-08T08:22:27.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.483+0000 7faa89d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa8c0c8d10 0x7faa8c14eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.483+0000 7faa89d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa8c0c8d10 0x7faa8c14eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53660/0 (socket says 172.21.15.40:53660) 2023-11-08T08:22:27.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.483+0000 7faa8a59c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa8c069d60 0x7faa8c1f09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.483+0000 7faa89d9b700 1 -- 172.21.15.40:0/3820124225 learned_addr learned my addr 172.21.15.40:0/3820124225 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:27.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.483+0000 7faa8ad9d700 1 --2- 172.21.15.40:0/3820124225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa8c15a480 0x7faa8c0c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.483+0000 7faa89d9b700 1 -- 172.21.15.40:0/3820124225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa8c15a480 msgr2=0x7faa8c0c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:27.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.483+0000 7faa89d9b700 1 --2- 172.21.15.40:0/3820124225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa8c15a480 0x7faa8c0c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.483+0000 7faa89d9b700 1 -- 172.21.15.40:0/3820124225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa8c069d60 msgr2=0x7faa8c1f09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:27.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.483+0000 7faa89d9b700 1 --2- 172.21.15.40:0/3820124225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa8c069d60 0x7faa8c1f09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.483+0000 7faa89d9b700 1 -- 172.21.15.40:0/3820124225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c009570 con 0x7faa8c0c8d10 2023-11-08T08:22:27.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.483+0000 7faa8a59c700 1 --2- 172.21.15.40:0/3820124225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa8c069d60 0x7faa8c1f09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:22:27.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.487+0000 7faa89d9b700 1 --2- 172.21.15.40:0/3820124225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa8c0c8d10 0x7faa8c14eef0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7faa8000c530 tx=0x7faa8000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:27.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.487+0000 7faa7affd700 1 -- 172.21.15.40:0/3820124225 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8000cea0 con 0x7faa8c0c8d10 2023-11-08T08:22:27.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.487+0000 7faa7affd700 1 -- 172.21.15.40:0/3820124225 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa800052d0 con 0x7faa8c0c8d10 2023-11-08T08:22:27.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.487+0000 7faa7affd700 1 -- 172.21.15.40:0/3820124225 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa800036a0 con 0x7faa8c0c8d10 2023-11-08T08:22:27.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.487+0000 7faa90fa7700 1 -- 172.21.15.40:0/3820124225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8c0add50 con 0x7faa8c0c8d10 2023-11-08T08:22:27.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.487+0000 7faa90fa7700 1 -- 172.21.15.40:0/3820124225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa8c0caa30 con 0x7faa8c0c8d10 2023-11-08T08:22:27.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.487+0000 7faa90fa7700 1 -- 172.21.15.40:0/3820124225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa8c0ca6e0 con 0x7faa8c0c8d10 2023-11-08T08:22:27.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.491+0000 7faa7affd700 1 -- 172.21.15.40:0/3820124225 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7faa80019030 con 0x7faa8c0c8d10 2023-11-08T08:22:27.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.491+0000 7faa7affd700 1 --2- 172.21.15.40:0/3820124225 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7faa6c064ee0 0x7faa6c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:27.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.491+0000 7faa7affd700 1 -- 172.21.15.40:0/3820124225 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7faa80081280 con 0x7faa8c0c8d10 2023-11-08T08:22:27.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.491+0000 7faa8a59c700 1 --2- 172.21.15.40:0/3820124225 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7faa6c064ee0 0x7faa6c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:27.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.495+0000 7faa8a59c700 1 --2- 172.21.15.40:0/3820124225 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7faa6c064ee0 0x7faa6c067390 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faa7c000c00 tx=0x7faa7c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:27.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.499+0000 7faa7affd700 1 -- 172.21.15.40:0/3820124225 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7faa8004f1f0 con 0x7faa8c0c8d10 2023-11-08T08:22:27.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.715+0000 7faa90fa7700 1 -- 172.21.15.40:0/3820124225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7faa8c0c69f0 con 0x7faa8c0c8d10 2023-11-08T08:22:27.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.715+0000 7faa7affd700 1 -- 172.21.15.40:0/3820124225 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) v1 ==== 53+0+73 (secure 0 0 0) 0x7faa8c0c69f0 con 0x7faa8c0c8d10 2023-11-08T08:22:27.721 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T08:22:27.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.723+0000 7faa90fa7700 1 -- 172.21.15.40:0/3820124225 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7faa6c064ee0 msgr2=0x7faa6c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:27.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.723+0000 7faa90fa7700 1 --2- 172.21.15.40:0/3820124225 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7faa6c064ee0 0x7faa6c067390 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faa7c000c00 tx=0x7faa7c008030 comp rx=0 tx=0).stop 2023-11-08T08:22:27.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.723+0000 7faa90fa7700 1 -- 172.21.15.40:0/3820124225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa8c0c8d10 msgr2=0x7faa8c14eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:27.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.723+0000 7faa90fa7700 1 --2- 172.21.15.40:0/3820124225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa8c0c8d10 0x7faa8c14eef0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7faa8000c530 tx=0x7faa8000c840 comp rx=0 tx=0).stop 2023-11-08T08:22:27.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.723+0000 7faa90fa7700 1 -- 172.21.15.40:0/3820124225 shutdown_connections 2023-11-08T08:22:27.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.723+0000 7faa90fa7700 1 --2- 172.21.15.40:0/3820124225 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7faa6c064ee0 0x7faa6c067390 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.723+0000 7faa90fa7700 1 --2- 172.21.15.40:0/3820124225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa8c15a480 0x7faa8c0c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.723+0000 7faa90fa7700 1 --2- 172.21.15.40:0/3820124225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa8c0c8d10 0x7faa8c14eef0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.723+0000 7faa90fa7700 1 --2- 172.21.15.40:0/3820124225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa8c069d60 0x7faa8c1f09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:27.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.723+0000 7faa90fa7700 1 -- 172.21.15.40:0/3820124225 >> 172.21.15.40:0/3820124225 conn(0x7faa8c0cf350 msgr2=0x7faa8c0afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:27.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.723+0000 7faa90fa7700 1 -- 172.21.15.40:0/3820124225 shutdown_connections 2023-11-08T08:22:27.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:27.723+0000 7faa90fa7700 1 -- 172.21.15.40:0/3820124225 wait complete. 2023-11-08T08:22:27.748 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-08T08:22:27.749 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T08:22:27.749 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-11-08T08:22:27.749 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T08:22:27.750 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T08:22:27.750 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-08T08:22:27.750 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-08T08:22:27.751 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T08:22:27.751 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T08:22:27.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:22:27.752 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-11-08T08:22:27.775 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T08:22:27.775 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T08:22:27.775 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T08:22:27.776 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:22:27.776 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-11-08T08:22:27.776 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:22:27.776 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:22:27.776 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:22:27.777 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2023-11-08T08:22:27.777 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:22:27.777 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:22:27.777 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2023-11-08T08:22:27.777 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T08:22:27.778 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-11-08T08:22:27.778 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2023-11-08T08:22:27.778 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-08T08:22:27.778 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42173sec preferred_lft 42173sec 2023-11-08T08:22:27.779 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link 2023-11-08T08:22:27.779 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:22:27.780 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-08T08:22:27.780 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-11-08T08:22:27.780 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2023-11-08T08:22:27.780 INFO:teuthology.orchestra.run.smithi040.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-08T08:22:27.781 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 02:42:94:e4:3d:2e brd ff:ff:ff:ff:ff:ff 2023-11-08T08:22:27.781 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-08T08:22:27.781 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:22:27.781 INFO:teuthology.orchestra.run.smithi040.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T08:22:27.782 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0e:a3:2a:89:6f:47 brd ff:ff:ff:ff:ff:ff 2023-11-08T08:22:27.782 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T08:22:27.782 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:22:27.783 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::a4cc:6bff:fe07:287b/64 scope link 2023-11-08T08:22:27.783 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:22:27.783 INFO:teuthology.orchestra.run.smithi040.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T08:22:27.784 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0e:a3:2a:89:6f:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T08:22:27.784 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ca3:2aff:fe89:6f47/64 scope link 2023-11-08T08:22:27.784 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:22:27.784 INFO:teuthology.orchestra.run.smithi040.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T08:22:27.785 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 72:b3:2e:65:5a:02 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T08:22:27.785 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::70b3:2eff:fe65:5a02/64 scope link 2023-11-08T08:22:27.785 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-11-08T08:22:27.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:22:27.787 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T08:22:27.854 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-11-08T08:22:27.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:22:27.857 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-11-08T08:22:27.924 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-11-08T08:22:27.924 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T08:22:27.924 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-08T08:22:27.924 INFO:teuthology.orchestra.run.smithi040.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-08T08:22:27.925 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-08T08:22:27.925 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T08:22:27.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:22:27.926 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-11-08T08:22:27.927 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-11-08T08:22:27.927 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-08T08:22:27.927 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-08T08:22:27.928 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-08T08:22:27.928 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-11-08T08:22:28.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:22:28.066 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-11-08T08:22:28.085 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T08:22:28.086 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T08:22:28.087 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T08:22:28.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:22:28.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2023-11-08T08:22:28.165 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T08:22:28.166 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T08:22:28.225 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T08:22:28.226 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T08:22:28.227 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T08:22:28.286 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-08T08:22:28.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:22:28.287 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:22:28.422 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:22:28.422 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T08:22:28.449 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T08:22:28.449 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T08:22:28.449 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-08T08:22:28.449 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@54ed45f4-d5b0-41c3-8c3b-b5a18b844f25.cephfs=/ 2023-11-08T08:22:28.450 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2023-11-08T08:22:28.455 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T08:22:28.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:22:28.456 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T08:22:28.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:22:28.491 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T08:22:28.569 INFO:teuthology.orchestra.run.smithi040.stdout:client6286 2023-11-08T08:22:28.570 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:22:28.570 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:22:28.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:22:28.633 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2023-11-08T08:22:28.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:22:28.708 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2023-11-08T08:22:28.775 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-08T08:22:28.775 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-11-08T08:22:28.776 DEBUG:teuthology.orchestra.run.smithi040:> import stat 2023-11-08T08:22:28.776 DEBUG:teuthology.orchestra.run.smithi040:> 2023-11-08T08:22:28.776 DEBUG:teuthology.orchestra.run.smithi040:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2023-11-08T08:22:28.777 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-11-08T08:22:28.919 INFO:teuthology.orchestra.run.smithi040.stdout:1099511627777 2023-11-08T08:22:29.111 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:22:29.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.331+0000 7f9800818700 1 Processor -- start 2023-11-08T08:22:29.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.331+0000 7f9800818700 1 -- start start 2023-11-08T08:22:29.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.335+0000 7f9800818700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f97fc071aa0 0x7f97fc06d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:29.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.335+0000 7f9800818700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f97fc069e00 0x7f97fc06a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:29.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.335+0000 7f9800818700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97fc06c600 con 0x7f97fc15c640 2023-11-08T08:22:29.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.335+0000 7f9800818700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97fc06cef0 con 0x7f97fc069e00 2023-11-08T08:22:29.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.335+0000 7f9800818700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97fc0d38f0 con 0x7f97fc071aa0 2023-11-08T08:22:29.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.335+0000 7f97fad9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f97fc15c640 0x7f97fc0d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37464/0 (socket says 172.21.15.40:37464) 2023-11-08T08:22:29.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.335+0000 7f97fad9d700 1 -- 172.21.15.40:0/443221292 learned_addr learned my addr 172.21.15.40:0/443221292 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:29.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.335+0000 7f97fb59e700 1 --2- 172.21.15.40:0/443221292 >> v2:172.21.15.40:3300/0 conn(0x7f97fc069e00 0x7f97fc06a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:29.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.335+0000 7f97f259c700 1 --2- 172.21.15.40:0/443221292 >> v2:172.21.15.169:3300/0 conn(0x7f97fc071aa0 0x7f97fc06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:29.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.335+0000 7f97fa59c700 1 -- 172.21.15.40:0/443221292 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438166626 0 0) 0x7f97fc06c600 con 0x7f97fc15c640 2023-11-08T08:22:29.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.335+0000 7f97fa59c700 1 -- 172.21.15.40:0/443221292 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97e80035d0 con 0x7f97fc15c640 2023-11-08T08:22:29.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.335+0000 7f97fb59e700 1 -- 172.21.15.40:0/443221292 >> v2:172.21.15.169:3300/0 conn(0x7f97fc071aa0 msgr2=0x7f97fc06d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:29.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.335+0000 7f97fb59e700 1 --2- 172.21.15.40:0/443221292 >> v2:172.21.15.169:3300/0 conn(0x7f97fc071aa0 0x7f97fc06d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:29.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.335+0000 7f97fb59e700 1 -- 172.21.15.40:0/443221292 >> v1:172.21.15.40:6789/0 conn(0x7f97fc15c640 legacy=0x7f97fc0d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:29.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.335+0000 7f97fa59c700 1 -- 172.21.15.40:0/443221292 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4212442278 0 0) 0x7f97e80035d0 con 0x7f97fc15c640 2023-11-08T08:22:29.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.335+0000 7f97fb59e700 1 -- 172.21.15.40:0/443221292 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97fc0722f0 con 0x7f97fc069e00 2023-11-08T08:22:29.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.335+0000 7f97fb59e700 1 --2- 172.21.15.40:0/443221292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97fc069e00 0x7f97fc06a280 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f97fc070e70 tx=0x7f97f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd230ceb9bc25603 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:29.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.335+0000 7f97fa59c700 1 -- 172.21.15.40:0/443221292 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f400e070 con 0x7f97fc069e00 2023-11-08T08:22:29.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.335+0000 7f97fa59c700 1 -- 172.21.15.40:0/443221292 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f4003a40 con 0x7f97fc069e00 2023-11-08T08:22:29.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.335+0000 7f97fa59c700 1 -- 172.21.15.40:0/443221292 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec001670 con 0x7f97fc069e00 2023-11-08T08:22:29.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.339+0000 7f9800818700 1 -- 172.21.15.40:0/443221292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97fc069e00 msgr2=0x7f97fc06a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:29.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.339+0000 7f9800818700 1 --2- 172.21.15.40:0/443221292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97fc069e00 0x7f97fc06a280 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f97fc070e70 tx=0x7f97f400b330 comp rx=0 tx=0).stop 2023-11-08T08:22:29.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.339+0000 7f9800818700 1 -- 172.21.15.40:0/443221292 shutdown_connections 2023-11-08T08:22:29.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.339+0000 7f9800818700 1 --2- 172.21.15.40:0/443221292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97fc069e00 0x7f97fc06a280 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:29.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.339+0000 7f9800818700 1 --2- 172.21.15.40:0/443221292 >> v2:172.21.15.169:3300/0 conn(0x7f97fc071aa0 0x7f97fc06d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:29.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.339+0000 7f9800818700 1 -- 172.21.15.40:0/443221292 >> 172.21.15.40:0/443221292 conn(0x7f97fc0b6520 msgr2=0x7f97fc071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:29.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.339+0000 7f9800818700 1 -- 172.21.15.40:0/443221292 shutdown_connections 2023-11-08T08:22:29.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.339+0000 7f9800818700 1 -- 172.21.15.40:0/443221292 wait complete. 2023-11-08T08:22:29.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.339+0000 7f9800818700 1 Processor -- start 2023-11-08T08:22:29.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f9800818700 1 -- start start 2023-11-08T08:22:29.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f9800818700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97fc069e00 0x7f97fc1e3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:29.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f9800818700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97fc071aa0 0x7f97fc1e3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:29.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f9800818700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97fc15c640 0x7f97fc0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:29.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f9800818700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc0c7230 con 0x7f97fc071aa0 2023-11-08T08:22:29.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f9800818700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc0c65f0 con 0x7f97fc069e00 2023-11-08T08:22:29.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f9800818700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc0c6890 con 0x7f97fc15c640 2023-11-08T08:22:29.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f97f259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97fc071aa0 0x7f97fc1e3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:29.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f97f259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97fc071aa0 0x7f97fc1e3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53676/0 (socket says 172.21.15.40:53676) 2023-11-08T08:22:29.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f97f259c700 1 -- 172.21.15.40:0/2888174394 learned_addr learned my addr 172.21.15.40:0/2888174394 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:29.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f97fad9d700 1 --2- 172.21.15.40:0/2888174394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97fc069e00 0x7f97fc1e3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:29.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f97fb59e700 1 --2- 172.21.15.40:0/2888174394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97fc15c640 0x7f97fc0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:29.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f97f259c700 1 -- 172.21.15.40:0/2888174394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97fc15c640 msgr2=0x7f97fc0c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:29.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f97f259c700 1 --2- 172.21.15.40:0/2888174394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97fc15c640 0x7f97fc0c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:29.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f97f259c700 1 -- 172.21.15.40:0/2888174394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97fc069e00 msgr2=0x7f97fc1e3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:29.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f97f259c700 1 --2- 172.21.15.40:0/2888174394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97fc069e00 0x7f97fc1e3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:29.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f97f259c700 1 -- 172.21.15.40:0/2888174394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f4009a70 con 0x7f97fc071aa0 2023-11-08T08:22:29.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f97fb59e700 1 --2- 172.21.15.40:0/2888174394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97fc15c640 0x7f97fc0c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:22:29.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f97f259c700 1 --2- 172.21.15.40:0/2888174394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97fc071aa0 0x7f97fc1e3e30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f97e400e840 tx=0x7f97e400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:29.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f97f3fff700 1 -- 172.21.15.40:0/2888174394 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e40095a0 con 0x7f97fc071aa0 2023-11-08T08:22:29.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f97f3fff700 1 -- 172.21.15.40:0/2888174394 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e400c690 con 0x7f97fc071aa0 2023-11-08T08:22:29.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f9800818700 1 -- 172.21.15.40:0/2888174394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97fc072010 con 0x7f97fc071aa0 2023-11-08T08:22:29.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f97f3fff700 1 -- 172.21.15.40:0/2888174394 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e400ca00 con 0x7f97fc071aa0 2023-11-08T08:22:29.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.343+0000 7f9800818700 1 -- 172.21.15.40:0/2888174394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97fc15ca10 con 0x7f97fc071aa0 2023-11-08T08:22:29.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.347+0000 7f9800818700 1 -- 172.21.15.40:0/2888174394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97fc06c830 con 0x7f97fc071aa0 2023-11-08T08:22:29.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.347+0000 7f97f3fff700 1 -- 172.21.15.40:0/2888174394 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f97e4019030 con 0x7f97fc071aa0 2023-11-08T08:22:29.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.351+0000 7f97f3fff700 1 --2- 172.21.15.40:0/2888174394 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f97dc064ee0 0x7f97dc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:29.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.351+0000 7f97f3fff700 1 -- 172.21.15.40:0/2888174394 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f97e4081ea0 con 0x7f97fc071aa0 2023-11-08T08:22:29.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.351+0000 7f97fad9d700 1 --2- 172.21.15.40:0/2888174394 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f97dc064ee0 0x7f97dc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:29.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.351+0000 7f97fad9d700 1 --2- 172.21.15.40:0/2888174394 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f97dc064ee0 0x7f97dc067390 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f97ec001eb0 tx=0x7f97ec001ce0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:29.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.355+0000 7f97f3fff700 1 -- 172.21.15.40:0/2888174394 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f97e404fe10 con 0x7f97fc071aa0 2023-11-08T08:22:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.567+0000 7f9800818700 1 -- 172.21.15.40:0/2888174394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97fc0c7370 con 0x7f97fc071aa0 2023-11-08T08:22:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.571+0000 7f97f3fff700 1 -- 172.21.15.40:0/2888174394 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f97e4054e60 con 0x7f97fc071aa0 2023-11-08T08:22:29.574 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:22:29.574 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/307473250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":307473250},{"type":"v1","addr":"172.21.15.169:6835","nonce":307473250}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6148,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/924518366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":924518366},{"type":"v1","addr":"172.21.15.169:6837","nonce":924518366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4294005934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4294005934},{"type":"v1","addr":"172.21.15.40:6833","nonce":4294005934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:22:20.650687+0000","modified":"2023-11-08T08:22:23.857369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6237":{"gid":6237,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3395630689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3395630689},{"type":"v1","addr":"172.21.15.40:6835","nonce":3395630689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T08:22:29.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.575+0000 7f9800818700 1 -- 172.21.15.40:0/2888174394 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f97dc064ee0 msgr2=0x7f97dc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:29.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.575+0000 7f9800818700 1 --2- 172.21.15.40:0/2888174394 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f97dc064ee0 0x7f97dc067390 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f97ec001eb0 tx=0x7f97ec001ce0 comp rx=0 tx=0).stop 2023-11-08T08:22:29.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.575+0000 7f9800818700 1 -- 172.21.15.40:0/2888174394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97fc071aa0 msgr2=0x7f97fc1e3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:29.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.575+0000 7f9800818700 1 --2- 172.21.15.40:0/2888174394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97fc071aa0 0x7f97fc1e3e30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f97e400e840 tx=0x7f97e400ec00 comp rx=0 tx=0).stop 2023-11-08T08:22:29.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.575+0000 7f9800818700 1 -- 172.21.15.40:0/2888174394 shutdown_connections 2023-11-08T08:22:29.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.575+0000 7f9800818700 1 --2- 172.21.15.40:0/2888174394 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f97dc064ee0 0x7f97dc067390 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:29.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.575+0000 7f9800818700 1 --2- 172.21.15.40:0/2888174394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97fc15c640 0x7f97fc0c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:29.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.575+0000 7f9800818700 1 --2- 172.21.15.40:0/2888174394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97fc071aa0 0x7f97fc1e3e30 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:29.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.575+0000 7f9800818700 1 --2- 172.21.15.40:0/2888174394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97fc069e00 0x7f97fc1e3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:29.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.575+0000 7f9800818700 1 -- 172.21.15.40:0/2888174394 >> 172.21.15.40:0/2888174394 conn(0x7f97fc0b6520 msgr2=0x7f97fc0d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:29.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.575+0000 7f9800818700 1 -- 172.21.15.40:0/2888174394 shutdown_connections 2023-11-08T08:22:29.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.575+0000 7f9800818700 1 -- 172.21.15.40:0/2888174394 wait complete. 2023-11-08T08:22:29.582 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 33 2023-11-08T08:22:29.602 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-11-08T08:22:29.822 INFO:teuthology.orchestra.run.smithi040.stdout:{"message":"","return_code":0} 2023-11-08T08:22:29.822 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-08T08:22:29.823 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-08T08:22:29.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.891+0000 7fc381a24140 1 Processor -- start 2023-11-08T08:22:29.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.891+0000 7fc381a24140 1 -- start start 2023-11-08T08:22:29.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.891+0000 7fc381a24140 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563774460700 0x5637745ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:29.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.891+0000 7fc381a24140 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563774493980 0x5637745cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:29.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.891+0000 7fc381a24140 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56377451ef40 0x5637744ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:29.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.891+0000 7fc381a24140 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x563774388240 con 0x563774460700 2023-11-08T08:22:29.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.891+0000 7fc381a24140 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637744acfb0 con 0x56377451ef40 2023-11-08T08:22:29.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.891+0000 7fc381a24140 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x56377451fb20 con 0x563774493980 2023-11-08T08:22:29.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.891+0000 7fc37f559700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563774460700 0x5637745ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:29.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.891+0000 7fc37f559700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563774460700 0x5637745ca6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53690/0 (socket says 172.21.15.40:53690) 2023-11-08T08:22:29.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.891+0000 7fc37f559700 1 -- 172.21.15.40:0/3080415213 learned_addr learned my addr 172.21.15.40:0/3080415213 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:29.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.891+0000 7fc37ed58700 1 --2- 172.21.15.40:0/3080415213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563774493980 0x5637745cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:29.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.891+0000 7fc37fd5a700 1 --2- 172.21.15.40:0/3080415213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56377451ef40 0x5637744ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:29.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.891+0000 7fc37f559700 1 -- 172.21.15.40:0/3080415213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563774493980 msgr2=0x5637745cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:29.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.891+0000 7fc37f559700 1 --2- 172.21.15.40:0/3080415213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563774493980 0x5637745cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:29.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.891+0000 7fc37f559700 1 -- 172.21.15.40:0/3080415213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56377451ef40 msgr2=0x5637744ae5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:29.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.891+0000 7fc37f559700 1 --2- 172.21.15.40:0/3080415213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56377451ef40 0x5637744ae5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:29.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.891+0000 7fc37f559700 1 -- 172.21.15.40:0/3080415213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37800a100 con 0x563774460700 2023-11-08T08:22:29.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.895+0000 7fc37f559700 1 --2- 172.21.15.40:0/3080415213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563774460700 0x5637745ca6b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc370001ce0 tx=0x7fc37000b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e58d3879a23302 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:29.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.895+0000 7fc37dd56700 1 -- 172.21.15.40:0/3080415213 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37000be20 con 0x563774460700 2023-11-08T08:22:29.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.895+0000 7fc37dd56700 1 -- 172.21.15.40:0/3080415213 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37000e470 con 0x563774460700 2023-11-08T08:22:29.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.895+0000 7fc37dd56700 1 -- 172.21.15.40:0/3080415213 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37000e820 con 0x563774460700 2023-11-08T08:22:29.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.895+0000 7fc381a24140 1 -- 172.21.15.40:0/3080415213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563774460700 msgr2=0x5637745ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:29.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.895+0000 7fc381a24140 1 --2- 172.21.15.40:0/3080415213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563774460700 0x5637745ca6b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc370001ce0 tx=0x7fc37000b5d0 comp rx=0 tx=0).stop 2023-11-08T08:22:29.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.895+0000 7fc381a24140 1 -- 172.21.15.40:0/3080415213 shutdown_connections 2023-11-08T08:22:29.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.895+0000 7fc381a24140 1 --2- 172.21.15.40:0/3080415213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56377451ef40 0x5637744ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:29.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.895+0000 7fc381a24140 1 --2- 172.21.15.40:0/3080415213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563774493980 0x5637745cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:29.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.895+0000 7fc381a24140 1 --2- 172.21.15.40:0/3080415213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563774460700 0x5637745ca6b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:29.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.895+0000 7fc381a24140 1 -- 172.21.15.40:0/3080415213 >> 172.21.15.40:0/3080415213 conn(0x563774466c30 msgr2=0x563774488ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:29.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.895+0000 7fc381a24140 1 -- 172.21.15.40:0/3080415213 shutdown_connections 2023-11-08T08:22:29.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.895+0000 7fc381a24140 1 -- 172.21.15.40:0/3080415213 wait complete. 2023-11-08T08:22:29.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.895+0000 7fc381a24140 1 Processor -- start 2023-11-08T08:22:29.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.895+0000 7fc381a24140 1 -- start start 2023-11-08T08:22:29.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc381a24140 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563774460700 0x563774525060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:29.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc381a24140 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563774493980 0x5637745223f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:29.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc381a24140 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56377451ef40 0x56377451d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:29.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc381a24140 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637745c9f30 con 0x563774493980 2023-11-08T08:22:29.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc381a24140 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637745ca1d0 con 0x56377451ef40 2023-11-08T08:22:29.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc381a24140 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5637745ca470 con 0x563774460700 2023-11-08T08:22:29.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc37fd5a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56377451ef40 0x56377451d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:29.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc37f559700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563774460700 0x563774525060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:29.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc37fd5a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56377451ef40 0x56377451d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33098/0 (socket says 172.21.15.40:33098) 2023-11-08T08:22:29.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc37fd5a700 1 -- 172.21.15.40:0/2055772542 learned_addr learned my addr 172.21.15.40:0/2055772542 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc37ed58700 1 --2- 172.21.15.40:0/2055772542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563774493980 0x5637745223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc37fd5a700 1 -- 172.21.15.40:0/2055772542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563774460700 msgr2=0x563774525060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc37fd5a700 1 --2- 172.21.15.40:0/2055772542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563774460700 0x563774525060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc37fd5a700 1 -- 172.21.15.40:0/2055772542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563774493980 msgr2=0x5637745223f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc37fd5a700 1 --2- 172.21.15.40:0/2055772542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563774493980 0x5637745223f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:29.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc37fd5a700 1 -- 172.21.15.40:0/2055772542 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc368001e10 con 0x56377451ef40 2023-11-08T08:22:29.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc37fd5a700 1 --2- 172.21.15.40:0/2055772542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56377451ef40 0x56377451d7b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc378003000 tx=0x7fc378003330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:29.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc37cd54700 1 -- 172.21.15.40:0/2055772542 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37801b680 con 0x56377451ef40 2023-11-08T08:22:29.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc37cd54700 1 -- 172.21.15.40:0/2055772542 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37801b7e0 con 0x56377451ef40 2023-11-08T08:22:29.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc37cd54700 1 -- 172.21.15.40:0/2055772542 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37801bb70 con 0x56377451ef40 2023-11-08T08:22:29.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc381a24140 1 -- 172.21.15.40:0/2055772542 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5637743a0390 con 0x56377451ef40 2023-11-08T08:22:29.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.899+0000 7fc381a24140 1 -- 172.21.15.40:0/2055772542 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637743938b0 con 0x56377451ef40 2023-11-08T08:22:29.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.903+0000 7fc37cd54700 1 -- 172.21.15.40:0/2055772542 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc37802d070 con 0x56377451ef40 2023-11-08T08:22:29.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.903+0000 7fc37cd54700 1 --2- 172.21.15.40:0/2055772542 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc36c063840 0x7fc36c065cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:29.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.907+0000 7fc37cd54700 1 -- 172.21.15.40:0/2055772542 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc378081e50 con 0x56377451ef40 2023-11-08T08:22:29.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.907+0000 7fc381a24140 1 --2- 172.21.15.40:0/2055772542 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x563774522930 0x563774464ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:29.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.907+0000 7fc37f559700 1 --2- 172.21.15.40:0/2055772542 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc36c063840 0x7fc36c065cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:29.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.907+0000 7fc381a24140 1 -- 172.21.15.40:0/2055772542 --> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] -- osd_op(unknown.0.0:1 11.7 11:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69) v8 -- 0x563774462d40 con 0x563774522930 2023-11-08T08:22:29.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.907+0000 7fc37ed58700 1 --2- 172.21.15.40:0/2055772542 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x563774522930 0x563774464ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:29.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.907+0000 7fc37f559700 1 --2- 172.21.15.40:0/2055772542 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc36c063840 0x7fc36c065cf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc37000b610 tx=0x7fc37001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:29.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.907+0000 7fc37ed58700 1 --2- 172.21.15.40:0/2055772542 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x563774522930 0x563774464ad0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:29.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.907+0000 7fc37ed58700 1 -- 172.21.15.40:0/2055772542 <== osd.5 v2:172.21.15.169:6809/35793 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7fc374009900 con 0x563774522930 2023-11-08T08:22:29.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.907+0000 7fc381a24140 1 -- 172.21.15.40:0/2055772542 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x563774522930 msgr2=0x563774464ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:29.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.907+0000 7fc381a24140 1 --2- 172.21.15.40:0/2055772542 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x563774522930 0x563774464ad0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:29.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.907+0000 7fc381a24140 1 -- 172.21.15.40:0/2055772542 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc36c063840 msgr2=0x7fc36c065cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:29.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.907+0000 7fc381a24140 1 --2- 172.21.15.40:0/2055772542 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc36c063840 0x7fc36c065cf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc37000b610 tx=0x7fc37001c040 comp rx=0 tx=0).stop 2023-11-08T08:22:29.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.907+0000 7fc381a24140 1 -- 172.21.15.40:0/2055772542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56377451ef40 msgr2=0x56377451d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:29.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.907+0000 7fc381a24140 1 --2- 172.21.15.40:0/2055772542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56377451ef40 0x56377451d7b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc378003000 tx=0x7fc378003330 comp rx=0 tx=0).stop 2023-11-08T08:22:29.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.907+0000 7fc37fd5a700 1 -- 172.21.15.40:0/2055772542 reap_dead start 2023-11-08T08:22:29.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.907+0000 7fc381a24140 1 -- 172.21.15.40:0/2055772542 shutdown_connections 2023-11-08T08:22:29.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.907+0000 7fc381a24140 1 -- 172.21.15.40:0/2055772542 >> 172.21.15.40:0/2055772542 conn(0x563774466c30 msgr2=0x5637744ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:29.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.907+0000 7fc381a24140 1 -- 172.21.15.40:0/2055772542 shutdown_connections 2023-11-08T08:22:29.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:29.907+0000 7fc381a24140 1 -- 172.21.15.40:0/2055772542 wait complete. 2023-11-08T08:22:29.921 INFO:teuthology.orchestra.run.smithi040.stdout:Xfiledir_test 2023-11-08T08:22:29.921 DEBUG:teuthology.orchestra.run.smithi040:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-08T08:22:30.025 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:22:30.025 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511627777, 2023-11-08T08:22:30.025 INFO:teuthology.orchestra.run.smithi040.stdout: "ancestors": [ 2023-11-08T08:22:30.026 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:30.026 INFO:teuthology.orchestra.run.smithi040.stdout: "dirino": 1099511627776, 2023-11-08T08:22:30.026 INFO:teuthology.orchestra.run.smithi040.stdout: "dname": "file", 2023-11-08T08:22:30.027 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 2 2023-11-08T08:22:30.027 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-11-08T08:22:30.027 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:22:30.027 INFO:teuthology.orchestra.run.smithi040.stdout: "dirino": 1, 2023-11-08T08:22:30.027 INFO:teuthology.orchestra.run.smithi040.stdout: "dname": "dir_test", 2023-11-08T08:22:30.028 INFO:teuthology.orchestra.run.smithi040.stdout: "version": 4 2023-11-08T08:22:30.028 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:30.028 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-11-08T08:22:30.028 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 11, 2023-11-08T08:22:30.029 INFO:teuthology.orchestra.run.smithi040.stdout: "old_pools": [] 2023-11-08T08:22:30.029 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:22:30.029 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:22:30.032 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2023-11-08T08:22:30.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.099+0000 7fb6c550f140 1 Processor -- start 2023-11-08T08:22:30.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.099+0000 7fb6c550f140 1 -- start start 2023-11-08T08:22:30.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.099+0000 7fb6c550f140 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560066f0a710 0x560066f348b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.099+0000 7fb6c550f140 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560066f3d990 0x56006706c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.099+0000 7fb6c550f140 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560066fc8f50 0x560066f585d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.099+0000 7fb6c550f140 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560066e9f180 con 0x560066f3d990 2023-11-08T08:22:30.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.099+0000 7fb6c550f140 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560066e32240 con 0x560066fc8f50 2023-11-08T08:22:30.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.099+0000 7fb6c550f140 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x560066fe4fb0 con 0x560066f0a710 2023-11-08T08:22:30.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.099+0000 7fb6ba843700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560066f3d990 0x56006706c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.099+0000 7fb6ba843700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560066f3d990 0x56006706c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53726/0 (socket says 172.21.15.40:53726) 2023-11-08T08:22:30.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.099+0000 7fb6ba843700 1 -- 172.21.15.40:0/1419239690 learned_addr learned my addr 172.21.15.40:0/1419239690 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:30.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.099+0000 7fb6c3845700 1 --2- 172.21.15.40:0/1419239690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560066fc8f50 0x560066f585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.099+0000 7fb6c3044700 1 --2- 172.21.15.40:0/1419239690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560066f0a710 0x560066f348b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.103+0000 7fb6ba843700 1 -- 172.21.15.40:0/1419239690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560066f0a710 msgr2=0x560066f348b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:30.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.103+0000 7fb6ba843700 1 --2- 172.21.15.40:0/1419239690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560066f0a710 0x560066f348b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.103+0000 7fb6ba843700 1 -- 172.21.15.40:0/1419239690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560066fc8f50 msgr2=0x560066f585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:30.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.103+0000 7fb6ba843700 1 --2- 172.21.15.40:0/1419239690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560066fc8f50 0x560066f585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.103+0000 7fb6ba843700 1 -- 172.21.15.40:0/1419239690 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac00a100 con 0x560066f3d990 2023-11-08T08:22:30.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.103+0000 7fb6ba843700 1 --2- 172.21.15.40:0/1419239690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560066f3d990 0x56006706c7e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac00a3f0 tx=0x7fb6ac00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21d2c42ea18deda server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:30.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.103+0000 7fb6c2042700 1 -- 172.21.15.40:0/1419239690 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac00f790 con 0x560066f3d990 2023-11-08T08:22:30.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.103+0000 7fb6c2042700 1 -- 172.21.15.40:0/1419239690 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ac00fa60 con 0x560066f3d990 2023-11-08T08:22:30.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.103+0000 7fb6c2042700 1 -- 172.21.15.40:0/1419239690 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac00fdf0 con 0x560066f3d990 2023-11-08T08:22:30.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.103+0000 7fb6c550f140 1 -- 172.21.15.40:0/1419239690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560066f3d990 msgr2=0x56006706c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.103+0000 7fb6c550f140 1 --2- 172.21.15.40:0/1419239690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560066f3d990 0x56006706c7e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac00a3f0 tx=0x7fb6ac00f000 comp rx=0 tx=0).stop 2023-11-08T08:22:30.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.103+0000 7fb6c550f140 1 -- 172.21.15.40:0/1419239690 shutdown_connections 2023-11-08T08:22:30.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.103+0000 7fb6c550f140 1 --2- 172.21.15.40:0/1419239690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560066fc8f50 0x560066f585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.103+0000 7fb6c550f140 1 --2- 172.21.15.40:0/1419239690 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560066f3d990 0x56006706c7e0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.103+0000 7fb6c550f140 1 --2- 172.21.15.40:0/1419239690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560066f0a710 0x560066f348b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.103+0000 7fb6c550f140 1 -- 172.21.15.40:0/1419239690 >> 172.21.15.40:0/1419239690 conn(0x560066f10c40 msgr2=0x560066f56c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:30.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.103+0000 7fb6c550f140 1 -- 172.21.15.40:0/1419239690 shutdown_connections 2023-11-08T08:22:30.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.103+0000 7fb6c550f140 1 -- 172.21.15.40:0/1419239690 wait complete. 2023-11-08T08:22:30.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.103+0000 7fb6c550f140 1 Processor -- start 2023-11-08T08:22:30.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.103+0000 7fb6c550f140 1 -- start start 2023-11-08T08:22:30.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c550f140 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560066f0a710 0x56006707abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c550f140 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560066f3d990 0x560066fd6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c550f140 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560066fc8f50 0x560066f32b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c550f140 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560066fd76c0 con 0x560066f0a710 2023-11-08T08:22:30.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c550f140 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560066fd6c40 con 0x560066fc8f50 2023-11-08T08:22:30.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c550f140 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x560066fd6ee0 con 0x560066f3d990 2023-11-08T08:22:30.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c3044700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560066f0a710 0x56006707abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c3044700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560066f0a710 0x56006707abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53736/0 (socket says 172.21.15.40:53736) 2023-11-08T08:22:30.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c3044700 1 -- 172.21.15.40:0/1504950417 learned_addr learned my addr 172.21.15.40:0/1504950417 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:30.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c3845700 1 --2- 172.21.15.40:0/1504950417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560066fc8f50 0x560066f32b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6ba843700 1 --2- 172.21.15.40:0/1504950417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560066f3d990 0x560066fd6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c3044700 1 -- 172.21.15.40:0/1504950417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560066f3d990 msgr2=0x560066fd6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c3044700 1 --2- 172.21.15.40:0/1504950417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560066f3d990 0x560066fd6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c3044700 1 -- 172.21.15.40:0/1504950417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560066fc8f50 msgr2=0x560066f32b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c3044700 1 --2- 172.21.15.40:0/1504950417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560066fc8f50 0x560066f32b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c3044700 1 -- 172.21.15.40:0/1504950417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560066e4a390 con 0x560066f0a710 2023-11-08T08:22:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c3044700 1 --2- 172.21.15.40:0/1504950417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560066f0a710 0x56006707abf0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4002d10 tx=0x7fb6b400c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:30.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c1040700 1 -- 172.21.15.40:0/1504950417 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4002bb0 con 0x560066f0a710 2023-11-08T08:22:30.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c1040700 1 -- 172.21.15.40:0/1504950417 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b4008030 con 0x560066f0a710 2023-11-08T08:22:30.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c550f140 1 -- 172.21.15.40:0/1504950417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560066e3d8b0 con 0x560066f0a710 2023-11-08T08:22:30.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c1040700 1 -- 172.21.15.40:0/1504950417 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4002e60 con 0x560066f0a710 2023-11-08T08:22:30.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.107+0000 7fb6c550f140 1 -- 172.21.15.40:0/1504950417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560066e4a5d0 con 0x560066f0a710 2023-11-08T08:22:30.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.111+0000 7fb6c1040700 1 -- 172.21.15.40:0/1504950417 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fb6b4021070 con 0x560066f0a710 2023-11-08T08:22:30.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.111+0000 7fb6c1040700 1 --2- 172.21.15.40:0/1504950417 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb6a4063840 0x7fb6a4065cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.115+0000 7fb6c1040700 1 -- 172.21.15.40:0/1504950417 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb6b4080c20 con 0x560066f0a710 2023-11-08T08:22:30.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.115+0000 7fb6c550f140 1 --2- 172.21.15.40:0/1504950417 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x560067079360 0x560067084590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.115+0000 7fb6c550f140 1 -- 172.21.15.40:0/1504950417 --> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] -- osd_op(unknown.0.0:1 11.7 11:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69) v8 -- 0x560067084ad0 con 0x560067079360 2023-11-08T08:22:30.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.115+0000 7fb6ba843700 1 --2- 172.21.15.40:0/1504950417 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb6a4063840 0x7fb6a4065cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.115+0000 7fb6c3845700 1 --2- 172.21.15.40:0/1504950417 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x560067079360 0x560067084590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.115+0000 7fb6ba843700 1 --2- 172.21.15.40:0/1504950417 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb6a4063840 0x7fb6a4065cf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac022000 tx=0x7fb6ac008000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:30.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.115+0000 7fb6c3845700 1 --2- 172.21.15.40:0/1504950417 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x560067079360 0x560067084590 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:30.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.119+0000 7fb6c3845700 1 -- 172.21.15.40:0/1504950417 <== osd.5 v2:172.21.15.169:6809/35793 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v69'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fb6bc00f780 con 0x560067079360 2023-11-08T08:22:30.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.119+0000 7fb6c550f140 1 -- 172.21.15.40:0/1504950417 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x560067079360 msgr2=0x560067084590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.119+0000 7fb6c550f140 1 --2- 172.21.15.40:0/1504950417 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x560067079360 0x560067084590 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.119+0000 7fb6c550f140 1 -- 172.21.15.40:0/1504950417 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb6a4063840 msgr2=0x7fb6a4065cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.119+0000 7fb6c550f140 1 --2- 172.21.15.40:0/1504950417 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb6a4063840 0x7fb6a4065cf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac022000 tx=0x7fb6ac008000 comp rx=0 tx=0).stop 2023-11-08T08:22:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.119+0000 7fb6c550f140 1 -- 172.21.15.40:0/1504950417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560066f0a710 msgr2=0x56006707abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.119+0000 7fb6c550f140 1 --2- 172.21.15.40:0/1504950417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560066f0a710 0x56006707abf0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4002d10 tx=0x7fb6b400c530 comp rx=0 tx=0).stop 2023-11-08T08:22:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.119+0000 7fb6c3845700 1 -- 172.21.15.40:0/1504950417 reap_dead start 2023-11-08T08:22:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.119+0000 7fb6c550f140 1 -- 172.21.15.40:0/1504950417 shutdown_connections 2023-11-08T08:22:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.119+0000 7fb6c550f140 1 -- 172.21.15.40:0/1504950417 >> 172.21.15.40:0/1504950417 conn(0x560066f10c40 msgr2=0x560066f585d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.119+0000 7fb6c550f140 1 -- 172.21.15.40:0/1504950417 shutdown_connections 2023-11-08T08:22:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.119+0000 7fb6c550f140 1 -- 172.21.15.40:0/1504950417 wait complete. 2023-11-08T08:22:30.129 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2023-11-08T08:22:30.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.315+0000 7ff0ce6d1700 1 Processor -- start 2023-11-08T08:22:30.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.315+0000 7ff0ce6d1700 1 -- start start 2023-11-08T08:22:30.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.319+0000 7ff0ce6d1700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0c815c6e0 0x7ff0c8071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.319+0000 7ff0ce6d1700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c815cab0 con 0x7ff0c8071fe0 2023-11-08T08:22:30.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.319+0000 7ff0ce6d1700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c80702d0 con 0x7ff0c806a510 2023-11-08T08:22:30.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.319+0000 7ff0ce6d1700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80d2800 con 0x7ff0c815c6e0 2023-11-08T08:22:30.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.319+0000 7ff0ccece700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff0c8071fe0 0x7ff0c8069e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37492/0 (socket says 172.21.15.40:37492) 2023-11-08T08:22:30.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.319+0000 7ff0ccece700 1 -- 172.21.15.40:0/3166835953 learned_addr learned my addr 172.21.15.40:0/3166835953 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:30.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.319+0000 7ff0cd6cf700 1 --2- 172.21.15.40:0/3166835953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0c815c6e0 0x7ff0c8071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.319+0000 7ff0bffff700 1 -- 172.21.15.40:0/3166835953 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3444141375 0 0) 0x7ff0c815cab0 con 0x7ff0c8071fe0 2023-11-08T08:22:30.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.319+0000 7ff0bffff700 1 -- 172.21.15.40:0/3166835953 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0b40035d0 con 0x7ff0c8071fe0 2023-11-08T08:22:30.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.319+0000 7ff0cd6cf700 1 -- 172.21.15.40:0/3166835953 >> v1:172.21.15.169:6789/0 conn(0x7ff0c806a510 legacy=0x7ff0c806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.319+0000 7ff0cd6cf700 1 -- 172.21.15.40:0/3166835953 >> v1:172.21.15.40:6789/0 conn(0x7ff0c8071fe0 legacy=0x7ff0c8069e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.319+0000 7ff0cd6cf700 1 -- 172.21.15.40:0/3166835953 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c80d2a10 con 0x7ff0c815c6e0 2023-11-08T08:22:30.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.319+0000 7ff0cd6cf700 1 --2- 172.21.15.40:0/3166835953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0c815c6e0 0x7ff0c8071aa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009880 tx=0x7ff0c4009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=514260cb74b97746 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:30.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.319+0000 7ff0bffff700 1 -- 172.21.15.40:0/3166835953 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4011830 con 0x7ff0c815c6e0 2023-11-08T08:22:30.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.319+0000 7ff0bffff700 1 -- 172.21.15.40:0/3166835953 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c4011990 con 0x7ff0c815c6e0 2023-11-08T08:22:30.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.319+0000 7ff0bffff700 1 -- 172.21.15.40:0/3166835953 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4011d20 con 0x7ff0c815c6e0 2023-11-08T08:22:30.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.323+0000 7ff0ce6d1700 1 -- 172.21.15.40:0/3166835953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0c815c6e0 msgr2=0x7ff0c8071aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.323+0000 7ff0ce6d1700 1 --2- 172.21.15.40:0/3166835953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0c815c6e0 0x7ff0c8071aa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009880 tx=0x7ff0c4009b90 comp rx=0 tx=0).stop 2023-11-08T08:22:30.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.323+0000 7ff0ce6d1700 1 -- 172.21.15.40:0/3166835953 shutdown_connections 2023-11-08T08:22:30.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.323+0000 7ff0ce6d1700 1 --2- 172.21.15.40:0/3166835953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0c815c6e0 0x7ff0c8071aa0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.323+0000 7ff0ce6d1700 1 -- 172.21.15.40:0/3166835953 >> 172.21.15.40:0/3166835953 conn(0x7ff0c80b65b0 msgr2=0x7ff0c8071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:30.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.323+0000 7ff0ce6d1700 1 -- 172.21.15.40:0/3166835953 shutdown_connections 2023-11-08T08:22:30.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.323+0000 7ff0ce6d1700 1 -- 172.21.15.40:0/3166835953 wait complete. 2023-11-08T08:22:30.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.323+0000 7ff0ce6d1700 1 Processor -- start 2023-11-08T08:22:30.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.323+0000 7ff0ce6d1700 1 -- start start 2023-11-08T08:22:30.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.323+0000 7ff0ce6d1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0c806a510 0x7ff0c809b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.323+0000 7ff0ce6d1700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0c8071fe0 0x7ff0c81e5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.323+0000 7ff0ce6d1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0c815c6e0 0x7ff0c81e6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.323+0000 7ff0ce6d1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80ca840 con 0x7ff0c815c6e0 2023-11-08T08:22:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.323+0000 7ff0ce6d1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80c9c00 con 0x7ff0c806a510 2023-11-08T08:22:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.323+0000 7ff0ce6d1700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80cac60 con 0x7ff0c8071fe0 2023-11-08T08:22:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.327+0000 7ff0cded0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0c815c6e0 0x7ff0c81e6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.327+0000 7ff0cd6cf700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0c806a510 0x7ff0c809b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.327+0000 7ff0cded0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0c815c6e0 0x7ff0c81e6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53744/0 (socket says 172.21.15.40:53744) 2023-11-08T08:22:30.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.327+0000 7ff0cded0700 1 -- 172.21.15.40:0/1058856268 learned_addr learned my addr 172.21.15.40:0/1058856268 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:30.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.327+0000 7ff0ccece700 1 --2- 172.21.15.40:0/1058856268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0c8071fe0 0x7ff0c81e5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.327+0000 7ff0cded0700 1 -- 172.21.15.40:0/1058856268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0c8071fe0 msgr2=0x7ff0c81e5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.327+0000 7ff0cded0700 1 --2- 172.21.15.40:0/1058856268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0c8071fe0 0x7ff0c81e5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.327+0000 7ff0cded0700 1 -- 172.21.15.40:0/1058856268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0c806a510 msgr2=0x7ff0c809b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.327+0000 7ff0cded0700 1 --2- 172.21.15.40:0/1058856268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0c806a510 0x7ff0c809b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.327+0000 7ff0cded0700 1 -- 172.21.15.40:0/1058856268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c40094b0 con 0x7ff0c815c6e0 2023-11-08T08:22:30.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.327+0000 7ff0cded0700 1 --2- 172.21.15.40:0/1058856268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0c815c6e0 0x7ff0c81e6390 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000cbc0 tx=0x7ff0c000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:30.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.327+0000 7ff0bdffb700 1 -- 172.21.15.40:0/1058856268 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0003c20 con 0x7ff0c815c6e0 2023-11-08T08:22:30.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.327+0000 7ff0bdffb700 1 -- 172.21.15.40:0/1058856268 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c0003d80 con 0x7ff0c815c6e0 2023-11-08T08:22:30.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.327+0000 7ff0ce6d1700 1 -- 172.21.15.40:0/1058856268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c80d2fd0 con 0x7ff0c815c6e0 2023-11-08T08:22:30.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.327+0000 7ff0bdffb700 1 -- 172.21.15.40:0/1058856268 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c000aae0 con 0x7ff0c815c6e0 2023-11-08T08:22:30.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.327+0000 7ff0ce6d1700 1 -- 172.21.15.40:0/1058856268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c80d2cf0 con 0x7ff0c815c6e0 2023-11-08T08:22:30.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.331+0000 7ff0bdffb700 1 -- 172.21.15.40:0/1058856268 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff0c0010050 con 0x7ff0c815c6e0 2023-11-08T08:22:30.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.335+0000 7ff0bdffb700 1 --2- 172.21.15.40:0/1058856268 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff0ac064ee0 0x7ff0ac067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.335+0000 7ff0bdffb700 1 -- 172.21.15.40:0/1058856268 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff0c0081da0 con 0x7ff0c815c6e0 2023-11-08T08:22:30.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.335+0000 7ff0cd6cf700 1 --2- 172.21.15.40:0/1058856268 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff0ac064ee0 0x7ff0ac067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.335+0000 7ff0cd6cf700 1 --2- 172.21.15.40:0/1058856268 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff0ac064ee0 0x7ff0ac067390 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40090b0 tx=0x7ff0c4010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:30.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.335+0000 7ff0d2bb7740 1 Processor -- start 2023-11-08T08:22:30.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.335+0000 7ff0d2bb7740 1 -- start start 2023-11-08T08:22:30.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0d2bb7740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222fd10 0x24e70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0d2bb7740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x222f940 0x24e7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0d2bb7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x227a9c0 0x249bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0d2bb7740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x239ec00 con 0x222fd10 2023-11-08T08:22:30.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0d2bb7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x240ecf0 con 0x227a9c0 2023-11-08T08:22:30.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0d2bb7740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x2407ff0 con 0x222f940 2023-11-08T08:22:30.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0cd6cf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222fd10 0x24e70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0cd6cf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222fd10 0x24e70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53748/0 (socket says 172.21.15.40:53748) 2023-11-08T08:22:30.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0cd6cf700 1 -- 172.21.15.40:0/2559722642 learned_addr learned my addr 172.21.15.40:0/2559722642 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:30.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0ccece700 1 --2- 172.21.15.40:0/2559722642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x222f940 0x24e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0cded0700 1 --2- 172.21.15.40:0/2559722642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x227a9c0 0x249bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0cd6cf700 1 -- 172.21.15.40:0/2559722642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x222f940 msgr2=0x24e7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:30.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0cd6cf700 1 --2- 172.21.15.40:0/2559722642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x222f940 0x24e7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0cd6cf700 1 -- 172.21.15.40:0/2559722642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x227a9c0 msgr2=0x249bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:30.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0cd6cf700 1 --2- 172.21.15.40:0/2559722642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x227a9c0 0x249bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0cd6cf700 1 -- 172.21.15.40:0/2559722642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x242dec0 con 0x222fd10 2023-11-08T08:22:30.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0cded0700 1 --2- 172.21.15.40:0/2559722642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x227a9c0 0x249bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:22:30.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0cd6cf700 1 --2- 172.21.15.40:0/2559722642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222fd10 0x24e70c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff0c806ecf0 tx=0x7ff0c4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f363f576819d01d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:30.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0b2ffd700 1 -- 172.21.15.40:0/2559722642 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c40038c0 con 0x222fd10 2023-11-08T08:22:30.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0b2ffd700 1 -- 172.21.15.40:0/2559722642 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c402c070 con 0x222fd10 2023-11-08T08:22:30.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0b2ffd700 1 -- 172.21.15.40:0/2559722642 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4003c20 con 0x222fd10 2023-11-08T08:22:30.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/2559722642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222fd10 msgr2=0x24e70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0d2bb7740 1 --2- 172.21.15.40:0/2559722642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222fd10 0x24e70c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff0c806ecf0 tx=0x7ff0c4009f90 comp rx=0 tx=0).stop 2023-11-08T08:22:30.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/2559722642 shutdown_connections 2023-11-08T08:22:30.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0d2bb7740 1 --2- 172.21.15.40:0/2559722642 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x227a9c0 0x249bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0d2bb7740 1 --2- 172.21.15.40:0/2559722642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x222f940 0x24e7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0d2bb7740 1 --2- 172.21.15.40:0/2559722642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222fd10 0x24e70c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/2559722642 >> 172.21.15.40:0/2559722642 conn(0x22300e0 msgr2=0x24c8790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:30.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.339+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/2559722642 shutdown_connections 2023-11-08T08:22:30.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.343+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/2559722642 wait complete. 2023-11-08T08:22:30.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.343+0000 7ff0d2bb7740 1 -- start start 2023-11-08T08:22:30.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.343+0000 7ff0d2bb7740 1 Processor -- start 2023-11-08T08:22:30.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0d2bb7740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222f940 0x249eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0d2bb7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x222fd10 0x24c1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0d2bb7740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x227a9c0 0x24c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0d2bb7740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x22b4d50 con 0x222f940 2023-11-08T08:22:30.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0d2bb7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x24c2040 con 0x222fd10 2023-11-08T08:22:30.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0d2bb7740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x249dfd0 con 0x227a9c0 2023-11-08T08:22:30.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0d2bb7740 10 client.0 _ll_register_callbacks cb 0x24648f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:22:30.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0cd6cf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222f940 0x249eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0cd6cf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222f940 0x249eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53764/0 (socket says 172.21.15.40:53764) 2023-11-08T08:22:30.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0cd6cf700 1 -- 172.21.15.40:0/797823951 learned_addr learned my addr 172.21.15.40:0/797823951 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:30.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0cded0700 1 --2- 172.21.15.40:0/797823951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x227a9c0 0x24c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0ccece700 1 --2- 172.21.15.40:0/797823951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x222fd10 0x24c1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0cd6cf700 1 -- 172.21.15.40:0/797823951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x227a9c0 msgr2=0x24c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:30.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0cd6cf700 1 --2- 172.21.15.40:0/797823951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x227a9c0 0x24c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0cd6cf700 1 -- 172.21.15.40:0/797823951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x222fd10 msgr2=0x24c1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:30.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0cd6cf700 1 --2- 172.21.15.40:0/797823951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x222fd10 0x24c1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0cd6cf700 1 -- 172.21.15.40:0/797823951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c4022070 con 0x222f940 2023-11-08T08:22:30.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0cd6cf700 1 --2- 172.21.15.40:0/797823951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222f940 0x249eef0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x2430160 tx=0x7ff0c40047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f4cc92a5ab13 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:30.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:22:30.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0a7fff700 1 -- 172.21.15.40:0/797823951 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4003c40 con 0x222f940 2023-11-08T08:22:30.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0a7fff700 1 -- 172.21.15.40:0/797823951 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff0c4003da0 con 0x222f940 2023-11-08T08:22:30.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/797823951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2435e30 con 0x222f940 2023-11-08T08:22:30.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0a7fff700 1 -- 172.21.15.40:0/797823951 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0c4031430 con 0x222f940 2023-11-08T08:22:30.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0d2bb7740 10 client.6450 fetch_fsmap learned FSMap version 33 2023-11-08T08:22:30.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/797823951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x227bc10 con 0x222f940 2023-11-08T08:22:30.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0a7fff700 1 -- 172.21.15.40:0/797823951 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2435e30 con 0x222f940 2023-11-08T08:22:30.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0a7fff700 1 -- 172.21.15.40:0/797823951 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff0c4031590 con 0x222f940 2023-11-08T08:22:30.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0d2bb7740 10 client.6450 fetch_fsmap finished waiting for FSMap version 33 2023-11-08T08:22:30.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0d2bb7740 10 client.6450 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-08T08:22:30.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0d2bb7740 20 client.6450 populate_metadata read hostname 'smithi040' 2023-11-08T08:22:30.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0d2bb7740 1 --2- 172.21.15.40:0/797823951 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x24a0b20 0x24ecc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0d2bb7740 4 client.6450 mds_command: new command op to 6237 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-08T08:22:30.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/797823951 --> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x2401070 con 0x24a0b20 2023-11-08T08:22:30.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.347+0000 7ff0ccece700 1 --2- 172.21.15.40:0/797823951 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x24a0b20 0x24ecc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.351+0000 7ff0ccece700 1 --2- 172.21.15.40:0/797823951 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x24a0b20 0x24ecc90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68c9b32900fd7895 server_cookie=22d4569ab9b2bf2f in_seq=0 out_seq=0 2023-11-08T08:22:30.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.351+0000 7ff0a7fff700 10 client.6450 ms_handle_connect on v2:172.21.15.40:6834/3395630689 2023-11-08T08:22:30.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.351+0000 7ff0a7fff700 1 -- 172.21.15.40:0/797823951 <== mds.0 v2:172.21.15.40:6834/3395630689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x2401070 con 0x24a0b20 2023-11-08T08:22:30.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.351+0000 7ff0a7fff700 10 client.6450 handle_command_reply: tid=0 2023-11-08T08:22:30.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.351+0000 7ff0d2bb7740 1 client.6450 shutdown 2023-11-08T08:22:30.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.351+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/797823951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222f940 msgr2=0x249eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.351+0000 7ff0d2bb7740 1 --2- 172.21.15.40:0/797823951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222f940 0x249eef0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x2430160 tx=0x7ff0c40047e0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.351+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/797823951 shutdown_connections 2023-11-08T08:22:30.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.351+0000 7ff0d2bb7740 1 --2- 172.21.15.40:0/797823951 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x24a0b20 0x24ecc90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.351+0000 7ff0d2bb7740 1 --2- 172.21.15.40:0/797823951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x227a9c0 0x24c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.351+0000 7ff0d2bb7740 1 --2- 172.21.15.40:0/797823951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x222fd10 0x24c1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.351+0000 7ff0a7fff700 0 client.6450 ms_handle_reset on v2:172.21.15.40:6834/3395630689 2023-11-08T08:22:30.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.351+0000 7ff0d2bb7740 1 --2- 172.21.15.40:0/797823951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222f940 0x249eef0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.351+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/797823951 >> 172.21.15.40:0/797823951 conn(0x22300e0 msgr2=0x24c4190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:30.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.351+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/797823951 shutdown_connections 2023-11-08T08:22:30.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.351+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/797823951 wait complete. 2023-11-08T08:22:30.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.351+0000 7ff0d2bb7740 20 client.6450 trim_cache size 0 max 16384 2023-11-08T08:22:30.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.371+0000 7ff0d2bb7740 1 Processor -- start 2023-11-08T08:22:30.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.371+0000 7ff0d2bb7740 1 -- start start 2023-11-08T08:22:30.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.371+0000 7ff0d2bb7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x22300e0 0x24ea600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.371+0000 7ff0d2bb7740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222f940 0x24eab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.371+0000 7ff0d2bb7740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x222fd10 0x24eb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.371+0000 7ff0d2bb7740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x23ddf00 con 0x222f940 2023-11-08T08:22:30.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.371+0000 7ff0d2bb7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x240bff0 con 0x22300e0 2023-11-08T08:22:30.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.371+0000 7ff0d2bb7740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x2405800 con 0x222fd10 2023-11-08T08:22:30.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.371+0000 7ff0ccece700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222f940 0x24eab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.371+0000 7ff0cd6cf700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x22300e0 0x24ea600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.371+0000 7ff0cded0700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x222fd10 0x24eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.371+0000 7ff0cd6cf700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x22300e0 0x24ea600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:33156/0 (socket says 172.21.15.40:33156) 2023-11-08T08:22:30.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.371+0000 7ff0cd6cf700 1 -- 172.21.15.40:0/2708045927 learned_addr learned my addr 172.21.15.40:0/2708045927 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:30.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.375+0000 7ff0cd6cf700 1 -- 172.21.15.40:0/2708045927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x222fd10 msgr2=0x24eb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:30.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.375+0000 7ff0cd6cf700 1 --2- 172.21.15.40:0/2708045927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x222fd10 0x24eb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.375+0000 7ff0cd6cf700 1 -- 172.21.15.40:0/2708045927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222f940 msgr2=0x24eab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:30.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.375+0000 7ff0cd6cf700 1 --2- 172.21.15.40:0/2708045927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222f940 0x24eab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.375+0000 7ff0cd6cf700 1 -- 172.21.15.40:0/2708045927 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c80ca440 con 0x22300e0 2023-11-08T08:22:30.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.375+0000 7ff0cd6cf700 1 --2- 172.21.15.40:0/2708045927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x22300e0 0x24ea600 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4006c60 tx=0x7ff0c4037000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41380a8307955c5d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:30.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.375+0000 7ff0b37fe700 1 -- 172.21.15.40:0/2708045927 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4031a50 con 0x22300e0 2023-11-08T08:22:30.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.375+0000 7ff0b37fe700 1 -- 172.21.15.40:0/2708045927 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c4011c00 con 0x22300e0 2023-11-08T08:22:30.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.375+0000 7ff0b37fe700 1 -- 172.21.15.40:0/2708045927 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4003e40 con 0x22300e0 2023-11-08T08:22:30.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.375+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/2708045927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x22300e0 msgr2=0x24ea600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.375+0000 7ff0d2bb7740 1 --2- 172.21.15.40:0/2708045927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x22300e0 0x24ea600 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4006c60 tx=0x7ff0c4037000 comp rx=0 tx=0).stop 2023-11-08T08:22:30.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.375+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/2708045927 shutdown_connections 2023-11-08T08:22:30.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.375+0000 7ff0d2bb7740 1 --2- 172.21.15.40:0/2708045927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x222fd10 0x24eb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.375+0000 7ff0d2bb7740 1 --2- 172.21.15.40:0/2708045927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222f940 0x24eab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.375+0000 7ff0d2bb7740 1 --2- 172.21.15.40:0/2708045927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x22300e0 0x24ea600 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.375+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/2708045927 >> 172.21.15.40:0/2708045927 conn(0x24c1c70 msgr2=0x24e8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.375+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/2708045927 shutdown_connections 2023-11-08T08:22:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.375+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/2708045927 wait complete. 2023-11-08T08:22:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.375+0000 7ff0d2bb7740 1 -- start start 2023-11-08T08:22:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.375+0000 7ff0d2bb7740 1 Processor -- start 2023-11-08T08:22:30.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0d2bb7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x222f940 0x2539eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0d2bb7740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222fd10 0x24bbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0d2bb7740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x22300e0 0x24bc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0d2bb7740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x23b8310 con 0x222fd10 2023-11-08T08:22:30.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0d2bb7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x2403960 con 0x222f940 2023-11-08T08:22:30.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0d2bb7740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x249e110 con 0x22300e0 2023-11-08T08:22:30.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0d2bb7740 10 client.0 _ll_register_callbacks cb 0x23d2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:22:30.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0ccece700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222fd10 0x24bbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0ccece700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222fd10 0x24bbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53790/0 (socket says 172.21.15.40:53790) 2023-11-08T08:22:30.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0ccece700 1 -- 172.21.15.40:0/1834650171 learned_addr learned my addr 172.21.15.40:0/1834650171 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:30.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0cd6cf700 1 --2- 172.21.15.40:0/1834650171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x222f940 0x2539eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0cded0700 1 --2- 172.21.15.40:0/1834650171 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x22300e0 0x24bc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0cd6cf700 1 -- 172.21.15.40:0/1834650171 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x22300e0 msgr2=0x24bc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0cd6cf700 1 --2- 172.21.15.40:0/1834650171 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x22300e0 0x24bc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0cd6cf700 1 -- 172.21.15.40:0/1834650171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222fd10 msgr2=0x24bbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0cd6cf700 1 --2- 172.21.15.40:0/1834650171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222fd10 0x24bbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0cd6cf700 1 -- 172.21.15.40:0/1834650171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x227bc10 con 0x222f940 2023-11-08T08:22:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0ccece700 1 --2- 172.21.15.40:0/1834650171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222fd10 0x24bbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:22:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0cd6cf700 1 --2- 172.21.15.40:0/1834650171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x222f940 0x2539eb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4006c30 tx=0x7ff0c4008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f138775a9f69993a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-11-08T08:22:30.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0a7fff700 1 -- 172.21.15.40:0/1834650171 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4031a50 con 0x222f940 2023-11-08T08:22:30.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/1834650171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0c402c070 con 0x222f940 2023-11-08T08:22:30.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0a7fff700 1 -- 172.21.15.40:0/1834650171 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff0c4031640 con 0x222f940 2023-11-08T08:22:30.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0a7fff700 1 -- 172.21.15.40:0/1834650171 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0c4039420 con 0x222f940 2023-11-08T08:22:30.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0d2bb7740 10 client.6325 fetch_fsmap learned FSMap version 33 2023-11-08T08:22:30.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/1834650171 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x23b4b70 con 0x222f940 2023-11-08T08:22:30.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0a7fff700 1 -- 172.21.15.40:0/1834650171 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c402c070 con 0x222f940 2023-11-08T08:22:30.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0a7fff700 1 -- 172.21.15.40:0/1834650171 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff0c4039580 con 0x222f940 2023-11-08T08:22:30.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0d2bb7740 10 client.6325 fetch_fsmap finished waiting for FSMap version 33 2023-11-08T08:22:30.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0d2bb7740 10 client.6325 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-08T08:22:30.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0d2bb7740 20 client.6325 populate_metadata read hostname 'smithi040' 2023-11-08T08:22:30.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.379+0000 7ff0d2bb7740 1 --2- 172.21.15.40:0/1834650171 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x227a9c0 0x24bc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.383+0000 7ff0d2bb7740 4 client.6325 mds_command: new command op to 6237 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2023-11-08T08:22:30.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.383+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/1834650171 --> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x2400d90 con 0x227a9c0 2023-11-08T08:22:30.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.383+0000 7ff0ccece700 1 --2- 172.21.15.40:0/1834650171 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x227a9c0 0x24bc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.383+0000 7ff0ccece700 1 --2- 172.21.15.40:0/1834650171 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x227a9c0 0x24bc700 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b90ebfea0a9193c server_cookie=3a79f6055027726 in_seq=0 out_seq=0 2023-11-08T08:22:30.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.383+0000 7ff0a7fff700 10 client.6325 ms_handle_connect on v2:172.21.15.40:6834/3395630689 2023-11-08T08:22:30.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.383+0000 7ff0a7fff700 1 -- 172.21.15.40:0/1834650171 <== mds.0 v2:172.21.15.40:6834/3395630689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x2400d90 con 0x227a9c0 2023-11-08T08:22:30.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.383+0000 7ff0a7fff700 10 client.6325 handle_command_reply: tid=0 2023-11-08T08:22:30.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.383+0000 7ff0d2bb7740 1 client.6325 shutdown 2023-11-08T08:22:30.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.383+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/1834650171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x222f940 msgr2=0x2539eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.383+0000 7ff0d2bb7740 1 --2- 172.21.15.40:0/1834650171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x222f940 0x2539eb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4006c30 tx=0x7ff0c4008030 comp rx=0 tx=0).stop 2023-11-08T08:22:30.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.383+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/1834650171 shutdown_connections 2023-11-08T08:22:30.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.383+0000 7ff0d2bb7740 1 --2- 172.21.15.40:0/1834650171 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x227a9c0 0x24bc700 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.383+0000 7ff0d2bb7740 1 --2- 172.21.15.40:0/1834650171 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x22300e0 0x24bc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.383+0000 7ff0d2bb7740 1 --2- 172.21.15.40:0/1834650171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x222fd10 0x24bbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.383+0000 7ff0d2bb7740 1 --2- 172.21.15.40:0/1834650171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x222f940 0x2539eb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.383+0000 7ff0a7fff700 0 client.6325 ms_handle_reset on v2:172.21.15.40:6834/3395630689 2023-11-08T08:22:30.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.383+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/1834650171 >> 172.21.15.40:0/1834650171 conn(0x24c1c70 msgr2=0x24e9580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:30.396 INFO:tasks.ceph.mds.c.smithi040.stderr:2023-11-08T08:22:30.383+0000 7f32cf73b700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6286=0-4194304@1} caps={6286=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6286 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55cb2535db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(11)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T08:22:30.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.383+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/1834650171 shutdown_connections 2023-11-08T08:22:30.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.383+0000 7ff0d2bb7740 1 -- 172.21.15.40:0/1834650171 wait complete. 2023-11-08T08:22:30.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.387+0000 7ff0d2bb7740 20 client.6325 trim_cache size 0 max 16384 2023-11-08T08:22:30.398 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:22:30.398 INFO:teuthology.orchestra.run.smithi040.stdout: "return_code": 0, 2023-11-08T08:22:30.398 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_tag": "3750476d-8094-4366-ab54-48c84ebb62cd", 2023-11-08T08:22:30.398 INFO:teuthology.orchestra.run.smithi040.stdout: "mode": "asynchronous" 2023-11-08T08:22:30.398 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:22:30.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.387+0000 7ff0ce6d1700 1 -- 172.21.15.40:0/1058856268 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff0ac064ee0 msgr2=0x7ff0ac067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.387+0000 7ff0ce6d1700 1 --2- 172.21.15.40:0/1058856268 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff0ac064ee0 0x7ff0ac067390 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40090b0 tx=0x7ff0c4010000 comp rx=0 tx=0).stop 2023-11-08T08:22:30.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.387+0000 7ff0ce6d1700 1 -- 172.21.15.40:0/1058856268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0c815c6e0 msgr2=0x7ff0c81e6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.387+0000 7ff0ce6d1700 1 --2- 172.21.15.40:0/1058856268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0c815c6e0 0x7ff0c81e6390 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000cbc0 tx=0x7ff0c000ced0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.387+0000 7ff0ce6d1700 1 -- 172.21.15.40:0/1058856268 shutdown_connections 2023-11-08T08:22:30.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.387+0000 7ff0ce6d1700 1 --2- 172.21.15.40:0/1058856268 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff0ac064ee0 0x7ff0ac067390 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.387+0000 7ff0ce6d1700 1 --2- 172.21.15.40:0/1058856268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0c815c6e0 0x7ff0c81e6390 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.387+0000 7ff0ce6d1700 1 --2- 172.21.15.40:0/1058856268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0c8071fe0 0x7ff0c81e5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.387+0000 7ff0ce6d1700 1 --2- 172.21.15.40:0/1058856268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0c806a510 0x7ff0c809b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.387+0000 7ff0ce6d1700 1 -- 172.21.15.40:0/1058856268 >> 172.21.15.40:0/1058856268 conn(0x7ff0c80b65b0 msgr2=0x7ff0c8071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:30.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.387+0000 7ff0ce6d1700 1 -- 172.21.15.40:0/1058856268 shutdown_connections 2023-11-08T08:22:30.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.387+0000 7ff0ce6d1700 1 -- 172.21.15.40:0/1058856268 wait complete. 2023-11-08T08:22:30.412 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-08T08:22:30.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.591+0000 7f9c33fff700 1 Processor -- start 2023-11-08T08:22:30.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.591+0000 7f9c33fff700 1 -- start start 2023-11-08T08:22:30.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.595+0000 7f9c33fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3406bd50 0x7f9c3406c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.595+0000 7f9c33fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c340ca980 con 0x7f9c340cb9e0 2023-11-08T08:22:30.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.595+0000 7f9c33fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c340cac20 con 0x7f9c340cd240 2023-11-08T08:22:30.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.595+0000 7f9c33fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340ce990 con 0x7f9c3406bd50 2023-11-08T08:22:30.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.595+0000 7f9c327fc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9c340cb9e0 0x7f9c340ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37508/0 (socket says 172.21.15.40:37508) 2023-11-08T08:22:30.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.595+0000 7f9c327fc700 1 -- 172.21.15.40:0/895002883 learned_addr learned my addr 172.21.15.40:0/895002883 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:22:30.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.595+0000 7f9c32ffd700 1 --2- 172.21.15.40:0/895002883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3406bd50 0x7f9c3406c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.595+0000 7f9c31ffb700 1 -- 172.21.15.40:0/895002883 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2606149873 0 0) 0x7f9c340ca980 con 0x7f9c340cb9e0 2023-11-08T08:22:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.595+0000 7f9c31ffb700 1 -- 172.21.15.40:0/895002883 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c1c0035d0 con 0x7f9c340cb9e0 2023-11-08T08:22:30.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.595+0000 7f9c32ffd700 1 -- 172.21.15.40:0/895002883 >> v1:172.21.15.169:6789/0 conn(0x7f9c340cd240 legacy=0x7f9c3406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.595+0000 7f9c32ffd700 1 -- 172.21.15.40:0/895002883 >> v1:172.21.15.40:6789/0 conn(0x7f9c340cb9e0 legacy=0x7f9c340ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.599+0000 7f9c32ffd700 1 -- 172.21.15.40:0/895002883 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c340caf00 con 0x7f9c3406bd50 2023-11-08T08:22:30.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.599+0000 7f9c32ffd700 1 --2- 172.21.15.40:0/895002883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3406bd50 0x7f9c3406c120 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009880 tx=0x7f9c24009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23110ec15d7bae0e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:30.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.599+0000 7f9c31ffb700 1 -- 172.21.15.40:0/895002883 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24011830 con 0x7f9c3406bd50 2023-11-08T08:22:30.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.599+0000 7f9c31ffb700 1 -- 172.21.15.40:0/895002883 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c24011990 con 0x7f9c3406bd50 2023-11-08T08:22:30.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.599+0000 7f9c31ffb700 1 -- 172.21.15.40:0/895002883 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24011d20 con 0x7f9c3406bd50 2023-11-08T08:22:30.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.599+0000 7f9c33fff700 1 -- 172.21.15.40:0/895002883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3406bd50 msgr2=0x7f9c3406c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.599+0000 7f9c33fff700 1 --2- 172.21.15.40:0/895002883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3406bd50 0x7f9c3406c120 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009880 tx=0x7f9c24009b90 comp rx=0 tx=0).stop 2023-11-08T08:22:30.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.599+0000 7f9c33fff700 1 -- 172.21.15.40:0/895002883 shutdown_connections 2023-11-08T08:22:30.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.599+0000 7f9c33fff700 1 --2- 172.21.15.40:0/895002883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3406bd50 0x7f9c3406c120 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.599+0000 7f9c33fff700 1 -- 172.21.15.40:0/895002883 >> 172.21.15.40:0/895002883 conn(0x7f9c340d0300 msgr2=0x7f9c340cc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:30.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.599+0000 7f9c33fff700 1 -- 172.21.15.40:0/895002883 shutdown_connections 2023-11-08T08:22:30.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.599+0000 7f9c33fff700 1 -- 172.21.15.40:0/895002883 wait complete. 2023-11-08T08:22:30.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.603+0000 7f9c33fff700 1 Processor -- start 2023-11-08T08:22:30.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.603+0000 7f9c33fff700 1 -- start start 2023-11-08T08:22:30.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.603+0000 7f9c33fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c3406bd50 0x7f9c340c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.603+0000 7f9c33fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c340cb9e0 0x7f9c34151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.603+0000 7f9c33fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c340cd240 0x7f9c340c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.603+0000 7f9c33fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340c5e00 con 0x7f9c340cd240 2023-11-08T08:22:30.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.603+0000 7f9c33fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340c69d0 con 0x7f9c3406bd50 2023-11-08T08:22:30.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.603+0000 7f9c33fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340b39c0 con 0x7f9c340cb9e0 2023-11-08T08:22:30.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.603+0000 7f9c337fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c340cd240 0x7f9c340c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.603+0000 7f9c337fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c340cd240 0x7f9c340c58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53802/0 (socket says 172.21.15.40:53802) 2023-11-08T08:22:30.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.603+0000 7f9c337fe700 1 -- 172.21.15.40:0/2132656538 learned_addr learned my addr 172.21.15.40:0/2132656538 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:30.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.603+0000 7f9c327fc700 1 --2- 172.21.15.40:0/2132656538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c340cb9e0 0x7f9c34151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.603+0000 7f9c32ffd700 1 --2- 172.21.15.40:0/2132656538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c3406bd50 0x7f9c340c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.603+0000 7f9c337fe700 1 -- 172.21.15.40:0/2132656538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c340cb9e0 msgr2=0x7f9c34151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.603+0000 7f9c337fe700 1 --2- 172.21.15.40:0/2132656538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c340cb9e0 0x7f9c34151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.603+0000 7f9c337fe700 1 -- 172.21.15.40:0/2132656538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c3406bd50 msgr2=0x7f9c340c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.603+0000 7f9c337fe700 1 --2- 172.21.15.40:0/2132656538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c3406bd50 0x7f9c340c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.603+0000 7f9c337fe700 1 -- 172.21.15.40:0/2132656538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c240094b0 con 0x7f9c340cd240 2023-11-08T08:22:30.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.607+0000 7f9c337fe700 1 --2- 172.21.15.40:0/2132656538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c340cd240 0x7f9c340c58c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c00cc50 tx=0x7f9c2c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:30.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.607+0000 7f9c1b7fe700 1 -- 172.21.15.40:0/2132656538 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c00ad80 con 0x7f9c340cd240 2023-11-08T08:22:30.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.607+0000 7f9c1b7fe700 1 -- 172.21.15.40:0/2132656538 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c00ce50 con 0x7f9c340cd240 2023-11-08T08:22:30.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.607+0000 7f9c33fff700 1 -- 172.21.15.40:0/2132656538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c340cb4c0 con 0x7f9c340cd240 2023-11-08T08:22:30.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.607+0000 7f9c1b7fe700 1 -- 172.21.15.40:0/2132656538 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c003c60 con 0x7f9c340cd240 2023-11-08T08:22:30.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.607+0000 7f9c33fff700 1 -- 172.21.15.40:0/2132656538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c340cb1e0 con 0x7f9c340cd240 2023-11-08T08:22:30.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.611+0000 7f9c1b7fe700 1 -- 172.21.15.40:0/2132656538 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f9c2c010050 con 0x7f9c340cd240 2023-11-08T08:22:30.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.611+0000 7f9c3c524740 1 Processor -- start 2023-11-08T08:22:30.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.611+0000 7f9c1b7fe700 1 --2- 172.21.15.40:0/2132656538 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9c20064ee0 0x7f9c20067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.611+0000 7f9c1b7fe700 1 -- 172.21.15.40:0/2132656538 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9c2c082030 con 0x7f9c340cd240 2023-11-08T08:22:30.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.611+0000 7f9c327fc700 1 --2- 172.21.15.40:0/2132656538 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9c20064ee0 0x7f9c20067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.611+0000 7f9c3c524740 1 -- start start 2023-11-08T08:22:30.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.611+0000 7f9c327fc700 1 --2- 172.21.15.40:0/2132656538 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9c20064ee0 0x7f9c20067390 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9c340b0600 tx=0x7f9c28009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:30.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c3c524740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1e95c10 0x21157d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c3c524740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1e95840 0x2115d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c3c524740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1ee08c0 0x2116250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c3c524740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x2004d20 con 0x1e95840 2023-11-08T08:22:30.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c3c524740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x2074d60 con 0x1e95c10 2023-11-08T08:22:30.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c3c524740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x206e060 con 0x1ee08c0 2023-11-08T08:22:30.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c327fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1e95840 0x2115d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c327fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1e95840 0x2115d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53812/0 (socket says 172.21.15.40:53812) 2023-11-08T08:22:30.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c327fc700 1 -- 172.21.15.40:0/2529489128 learned_addr learned my addr 172.21.15.40:0/2529489128 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:30.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c32ffd700 1 --2- 172.21.15.40:0/2529489128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1e95c10 0x21157d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c337fe700 1 --2- 172.21.15.40:0/2529489128 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1ee08c0 0x2116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c327fc700 1 -- 172.21.15.40:0/2529489128 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1ee08c0 msgr2=0x2116250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:30.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c327fc700 1 --2- 172.21.15.40:0/2529489128 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1ee08c0 0x2116250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c327fc700 1 -- 172.21.15.40:0/2529489128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1e95c10 msgr2=0x21157d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:30.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c327fc700 1 --2- 172.21.15.40:0/2529489128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1e95c10 0x21157d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c327fc700 1 -- 172.21.15.40:0/2529489128 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x20940a0 con 0x1e95840 2023-11-08T08:22:30.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c327fc700 1 --2- 172.21.15.40:0/2529489128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1e95840 0x2115d10 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f9c280059b0 tx=0x7f9c28011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a98d5c6859e981 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:30.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c18ff9700 1 -- 172.21.15.40:0/2529489128 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c28005e20 con 0x1e95840 2023-11-08T08:22:30.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c18ff9700 1 -- 172.21.15.40:0/2529489128 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c28007770 con 0x1e95840 2023-11-08T08:22:30.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c18ff9700 1 -- 172.21.15.40:0/2529489128 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2800e7c0 con 0x1e95840 2023-11-08T08:22:30.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c3c524740 1 -- 172.21.15.40:0/2529489128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1e95840 msgr2=0x2115d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c3c524740 1 --2- 172.21.15.40:0/2529489128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1e95840 0x2115d10 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f9c280059b0 tx=0x7f9c28011000 comp rx=0 tx=0).stop 2023-11-08T08:22:30.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c3c524740 1 -- 172.21.15.40:0/2529489128 shutdown_connections 2023-11-08T08:22:30.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c3c524740 1 --2- 172.21.15.40:0/2529489128 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1ee08c0 0x2116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c3c524740 1 --2- 172.21.15.40:0/2529489128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1e95840 0x2115d10 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c3c524740 1 --2- 172.21.15.40:0/2529489128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1e95c10 0x21157d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c3c524740 1 -- 172.21.15.40:0/2529489128 >> 172.21.15.40:0/2529489128 conn(0x1e95fe0 msgr2=0x212eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:30.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.615+0000 7f9c3c524740 1 -- 172.21.15.40:0/2529489128 shutdown_connections 2023-11-08T08:22:30.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.619+0000 7f9c3c524740 1 -- 172.21.15.40:0/2529489128 wait complete. 2023-11-08T08:22:30.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.619+0000 7f9c3c524740 1 -- start start 2023-11-08T08:22:30.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.619+0000 7f9c3c524740 1 Processor -- start 2023-11-08T08:22:30.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.619+0000 7f9c3c524740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1e95840 0x2154620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.619+0000 7f9c3c524740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1e95c10 0x2114820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.619+0000 7f9c3c524740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1ee08c0 0x2114d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.619+0000 7f9c3c524740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f1ac50 con 0x1ee08c0 2023-11-08T08:22:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.619+0000 7f9c3c524740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x2154e10 con 0x1e95c10 2023-11-08T08:22:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.619+0000 7f9c3c524740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x21550b0 con 0x1e95840 2023-11-08T08:22:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.619+0000 7f9c3c524740 10 client.0 _ll_register_callbacks cb 0x20679a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:22:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.619+0000 7f9c337fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1ee08c0 0x2114d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c32ffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1e95840 0x2154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c337fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1ee08c0 0x2114d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53822/0 (socket says 172.21.15.40:53822) 2023-11-08T08:22:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c337fe700 1 -- 172.21.15.40:0/2089070240 learned_addr learned my addr 172.21.15.40:0/2089070240 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c327fc700 1 --2- 172.21.15.40:0/2089070240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1e95c10 0x2114820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c337fe700 1 -- 172.21.15.40:0/2089070240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1e95840 msgr2=0x2154620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c337fe700 1 --2- 172.21.15.40:0/2089070240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1e95840 0x2154620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c337fe700 1 -- 172.21.15.40:0/2089070240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1e95c10 msgr2=0x2114820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c337fe700 1 --2- 172.21.15.40:0/2089070240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1e95c10 0x2114820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c337fe700 1 -- 172.21.15.40:0/2089070240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2800f040 con 0x1ee08c0 2023-11-08T08:22:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c337fe700 1 --2- 172.21.15.40:0/2089070240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1ee08c0 0x2114d60 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c053960 tx=0x7f9c2c05bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705c4b55353c0111 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:22:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c11ffb700 1 -- 172.21.15.40:0/2089070240 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c05ba60 con 0x1ee08c0 2023-11-08T08:22:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c11ffb700 1 -- 172.21.15.40:0/2089070240 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9c2c057bc0 con 0x1ee08c0 2023-11-08T08:22:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c3c524740 1 -- 172.21.15.40:0/2089070240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x20670e0 con 0x1ee08c0 2023-11-08T08:22:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c11ffb700 1 -- 172.21.15.40:0/2089070240 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c2c057d80 con 0x1ee08c0 2023-11-08T08:22:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c3c524740 10 client.6468 fetch_fsmap learned FSMap version 33 2023-11-08T08:22:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c3c524740 1 -- 172.21.15.40:0/2089070240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x1ee1b10 con 0x1ee08c0 2023-11-08T08:22:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c11ffb700 1 -- 172.21.15.40:0/2089070240 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x20670e0 con 0x1ee08c0 2023-11-08T08:22:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c11ffb700 1 -- 172.21.15.40:0/2089070240 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9c2c04fa50 con 0x1ee08c0 2023-11-08T08:22:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c3c524740 10 client.6468 fetch_fsmap finished waiting for FSMap version 33 2023-11-08T08:22:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c3c524740 10 client.6468 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-08T08:22:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c3c524740 20 client.6468 populate_metadata read hostname 'smithi040' 2023-11-08T08:22:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c3c524740 1 --2- 172.21.15.40:0/2089070240 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x2153610 0x2102e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c3c524740 4 client.6468 mds_command: new command op to 6237 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-08T08:22:30.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c3c524740 1 -- 172.21.15.40:0/2089070240 --> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x2066e00 con 0x2153610 2023-11-08T08:22:30.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.623+0000 7f9c32ffd700 1 --2- 172.21.15.40:0/2089070240 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x2153610 0x2102e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.627+0000 7f9c11ffb700 10 client.6468 ms_handle_connect on v2:172.21.15.40:6834/3395630689 2023-11-08T08:22:30.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.627+0000 7f9c32ffd700 1 --2- 172.21.15.40:0/2089070240 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x2153610 0x2102e60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c41e5780b78f664d server_cookie=d8e964e3221b0340 in_seq=0 out_seq=0 2023-11-08T08:22:30.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.627+0000 7f9c11ffb700 1 -- 172.21.15.40:0/2089070240 <== mds.0 v2:172.21.15.40:6834/3395630689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x2066e00 con 0x2153610 2023-11-08T08:22:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.627+0000 7f9c11ffb700 10 client.6468 handle_command_reply: tid=0 2023-11-08T08:22:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.627+0000 7f9c3c524740 1 client.6468 shutdown 2023-11-08T08:22:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.627+0000 7f9c3c524740 1 -- 172.21.15.40:0/2089070240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1ee08c0 msgr2=0x2114d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.627+0000 7f9c3c524740 1 --2- 172.21.15.40:0/2089070240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1ee08c0 0x2114d60 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c053960 tx=0x7f9c2c05bd70 comp rx=0 tx=0).stop 2023-11-08T08:22:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.627+0000 7f9c3c524740 1 -- 172.21.15.40:0/2089070240 shutdown_connections 2023-11-08T08:22:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.627+0000 7f9c3c524740 1 --2- 172.21.15.40:0/2089070240 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x2153610 0x2102e60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.627+0000 7f9c3c524740 1 --2- 172.21.15.40:0/2089070240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1ee08c0 0x2114d60 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.627+0000 7f9c3c524740 1 --2- 172.21.15.40:0/2089070240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1e95c10 0x2114820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.627+0000 7f9c11ffb700 0 client.6468 ms_handle_reset on v2:172.21.15.40:6834/3395630689 2023-11-08T08:22:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.627+0000 7f9c3c524740 1 --2- 172.21.15.40:0/2089070240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1e95840 0x2154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.627+0000 7f9c3c524740 1 -- 172.21.15.40:0/2089070240 >> 172.21.15.40:0/2089070240 conn(0x1e95fe0 msgr2=0x2107860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.627+0000 7f9c3c524740 1 -- 172.21.15.40:0/2089070240 shutdown_connections 2023-11-08T08:22:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.627+0000 7f9c3c524740 1 -- 172.21.15.40:0/2089070240 wait complete. 2023-11-08T08:22:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.627+0000 7f9c3c524740 20 client.6468 trim_cache size 0 max 16384 2023-11-08T08:22:30.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.647+0000 7f9c3c524740 1 Processor -- start 2023-11-08T08:22:30.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.647+0000 7f9c3c524740 1 -- start start 2023-11-08T08:22:30.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.647+0000 7f9c3c524740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1e95840 0x2101510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.647+0000 7f9c3c524740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1e95c10 0x2101a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.647+0000 7f9c3c524740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1ee08c0 0x2150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.647+0000 7f9c3c524740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x206cef0 con 0x1e95840 2023-11-08T08:22:30.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.647+0000 7f9c3c524740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x200d510 con 0x1ee08c0 2023-11-08T08:22:30.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.647+0000 7f9c3c524740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x2033440 con 0x1e95c10 2023-11-08T08:22:30.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c32ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1e95840 0x2101510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c32ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1e95840 0x2101510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53828/0 (socket says 172.21.15.40:53828) 2023-11-08T08:22:30.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c32ffd700 1 -- 172.21.15.40:0/423279680 learned_addr learned my addr 172.21.15.40:0/423279680 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:30.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c337fe700 1 --2- 172.21.15.40:0/423279680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1ee08c0 0x2150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c327fc700 1 --2- 172.21.15.40:0/423279680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1e95c10 0x2101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c32ffd700 1 -- 172.21.15.40:0/423279680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1e95c10 msgr2=0x2101a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:30.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c32ffd700 1 --2- 172.21.15.40:0/423279680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1e95c10 0x2101a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c32ffd700 1 -- 172.21.15.40:0/423279680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1ee08c0 msgr2=0x2150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c32ffd700 1 --2- 172.21.15.40:0/423279680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1ee08c0 0x2150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c32ffd700 1 -- 172.21.15.40:0/423279680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c340b41e0 con 0x1e95840 2023-11-08T08:22:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c337fe700 1 --2- 172.21.15.40:0/423279680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1ee08c0 0x2150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:22:30.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c32ffd700 1 --2- 172.21.15.40:0/423279680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1e95840 0x2101510 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f9c24013040 tx=0x7f9c24009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04e330cdabddf54 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:30.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c197fa700 1 -- 172.21.15.40:0/423279680 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2401f040 con 0x1e95840 2023-11-08T08:22:30.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c197fa700 1 -- 172.21.15.40:0/423279680 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c240290f0 con 0x1e95840 2023-11-08T08:22:30.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c197fa700 1 -- 172.21.15.40:0/423279680 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24006cc0 con 0x1e95840 2023-11-08T08:22:30.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c3c524740 1 -- 172.21.15.40:0/423279680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1e95840 msgr2=0x2101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c3c524740 1 --2- 172.21.15.40:0/423279680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1e95840 0x2101510 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f9c24013040 tx=0x7f9c24009f90 comp rx=0 tx=0).stop 2023-11-08T08:22:30.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c3c524740 1 -- 172.21.15.40:0/423279680 shutdown_connections 2023-11-08T08:22:30.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c3c524740 1 --2- 172.21.15.40:0/423279680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1ee08c0 0x2150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c3c524740 1 --2- 172.21.15.40:0/423279680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1e95c10 0x2101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c3c524740 1 --2- 172.21.15.40:0/423279680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1e95840 0x2101510 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c3c524740 1 -- 172.21.15.40:0/423279680 >> 172.21.15.40:0/423279680 conn(0x1e95fe0 msgr2=0x212a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:30.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c3c524740 1 -- 172.21.15.40:0/423279680 shutdown_connections 2023-11-08T08:22:30.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.651+0000 7f9c3c524740 1 -- 172.21.15.40:0/423279680 wait complete. 2023-11-08T08:22:30.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.655+0000 7f9c3c524740 1 -- start start 2023-11-08T08:22:30.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.655+0000 7f9c3c524740 1 Processor -- start 2023-11-08T08:22:30.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.655+0000 7f9c3c524740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1e95840 0x21ac940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.655+0000 7f9c3c524740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1e95c10 0x21689c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.655+0000 7f9c3c524740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1ee08c0 0x2151120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.655+0000 7f9c3c524740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x2043f70 con 0x1ee08c0 2023-11-08T08:22:30.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.655+0000 7f9c3c524740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x206b460 con 0x1e95c10 2023-11-08T08:22:30.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.655+0000 7f9c3c524740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x21ace80 con 0x1e95840 2023-11-08T08:22:30.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.655+0000 7f9c3c524740 10 client.0 _ll_register_callbacks cb 0x7f9c28001c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:22:30.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c337fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1ee08c0 0x2151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c337fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1ee08c0 0x2151120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53830/0 (socket says 172.21.15.40:53830) 2023-11-08T08:22:30.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c32ffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1e95840 0x21ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c337fe700 1 -- 172.21.15.40:0/2591978307 learned_addr learned my addr 172.21.15.40:0/2591978307 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:22:30.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c327fc700 1 --2- 172.21.15.40:0/2591978307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1e95c10 0x21689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c337fe700 1 -- 172.21.15.40:0/2591978307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1e95840 msgr2=0x21ac940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:30.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c337fe700 1 --2- 172.21.15.40:0/2591978307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1e95840 0x21ac940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c337fe700 1 -- 172.21.15.40:0/2591978307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1e95c10 msgr2=0x21689c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:22:30.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c337fe700 1 --2- 172.21.15.40:0/2591978307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1e95c10 0x21689c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c337fe700 1 -- 172.21.15.40:0/2591978307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c340caf00 con 0x1ee08c0 2023-11-08T08:22:30.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c337fe700 1 --2- 172.21.15.40:0/2591978307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1ee08c0 0x2151120 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c050160 tx=0x7f9c2c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5ec3b6ef8c8fe9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:22:30.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:22:30.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c11ffb700 1 -- 172.21.15.40:0/2591978307 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c05bd70 con 0x1ee08c0 2023-11-08T08:22:30.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c3c524740 1 -- 172.21.15.40:0/2591978307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c28007770 con 0x1ee08c0 2023-11-08T08:22:30.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c11ffb700 1 -- 172.21.15.40:0/2591978307 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9c2c052bc0 con 0x1ee08c0 2023-11-08T08:22:30.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c11ffb700 1 -- 172.21.15.40:0/2591978307 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c2c052dc0 con 0x1ee08c0 2023-11-08T08:22:30.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c3c524740 10 client.6474 fetch_fsmap learned FSMap version 33 2023-11-08T08:22:30.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c3c524740 1 -- 172.21.15.40:0/2591978307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x2034970 con 0x1ee08c0 2023-11-08T08:22:30.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c11ffb700 1 -- 172.21.15.40:0/2591978307 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c28007770 con 0x1ee08c0 2023-11-08T08:22:30.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c11ffb700 1 -- 172.21.15.40:0/2591978307 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9c2c07a340 con 0x1ee08c0 2023-11-08T08:22:30.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c3c524740 10 client.6474 fetch_fsmap finished waiting for FSMap version 33 2023-11-08T08:22:30.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c3c524740 10 client.6474 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-08T08:22:30.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c3c524740 20 client.6474 populate_metadata read hostname 'smithi040' 2023-11-08T08:22:30.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c3c524740 1 --2- 172.21.15.40:0/2591978307 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x2162f40 0x2151660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:22:30.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c3c524740 4 client.6474 mds_command: new command op to 6237 tid=0[{"prefix": "scrub status"}] 2023-11-08T08:22:30.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c3c524740 1 -- 172.21.15.40:0/2591978307 --> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x1f12390 con 0x2162f40 2023-11-08T08:22:30.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.659+0000 7f9c32ffd700 1 --2- 172.21.15.40:0/2591978307 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x2162f40 0x2151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:22:30.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c32ffd700 1 --2- 172.21.15.40:0/2591978307 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x2162f40 0x2151660 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=701e5846eff394c6 server_cookie=63bc8d58d0d6d3a1 in_seq=0 out_seq=0 2023-11-08T08:22:30.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c11ffb700 10 client.6474 ms_handle_connect on v2:172.21.15.40:6834/3395630689 2023-11-08T08:22:30.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c11ffb700 1 -- 172.21.15.40:0/2591978307 <== mds.0 v2:172.21.15.40:6834/3395630689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x1f12390 con 0x2162f40 2023-11-08T08:22:30.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c11ffb700 10 client.6474 handle_command_reply: tid=0 2023-11-08T08:22:30.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c3c524740 1 client.6474 shutdown 2023-11-08T08:22:30.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c3c524740 1 -- 172.21.15.40:0/2591978307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1ee08c0 msgr2=0x2151120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c3c524740 1 --2- 172.21.15.40:0/2591978307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1ee08c0 0x2151120 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c050160 tx=0x7f9c2c01c000 comp rx=0 tx=0).stop 2023-11-08T08:22:30.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c3c524740 1 -- 172.21.15.40:0/2591978307 shutdown_connections 2023-11-08T08:22:30.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c3c524740 1 --2- 172.21.15.40:0/2591978307 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x2162f40 0x2151660 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c3c524740 1 --2- 172.21.15.40:0/2591978307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1ee08c0 0x2151120 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c3c524740 1 --2- 172.21.15.40:0/2591978307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1e95c10 0x21689c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c3c524740 1 --2- 172.21.15.40:0/2591978307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1e95840 0x21ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c11ffb700 0 client.6474 ms_handle_reset on v2:172.21.15.40:6834/3395630689 2023-11-08T08:22:30.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c3c524740 1 -- 172.21.15.40:0/2591978307 >> 172.21.15.40:0/2591978307 conn(0x1e95fe0 msgr2=0x2105440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:30.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c3c524740 1 -- 172.21.15.40:0/2591978307 shutdown_connections 2023-11-08T08:22:30.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c3c524740 1 -- 172.21.15.40:0/2591978307 wait complete. 2023-11-08T08:22:30.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c3c524740 20 client.6474 trim_cache size 0 max 16384 2023-11-08T08:22:30.675 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:22:30.676 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-08T08:22:30.676 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-11-08T08:22:30.676 INFO:teuthology.orchestra.run.smithi040.stdout: "3750476d-8094-4366-ab54-48c84ebb62cd": { 2023-11-08T08:22:30.676 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-11-08T08:22:30.677 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "3750476d-8094-4366-ab54-48c84ebb62cd", 2023-11-08T08:22:30.677 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive" 2023-11-08T08:22:30.677 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:30.677 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:22:30.677 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:22:30.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c33fff700 1 -- 172.21.15.40:0/2132656538 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9c20064ee0 msgr2=0x7f9c20067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c33fff700 1 --2- 172.21.15.40:0/2132656538 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9c20064ee0 0x7f9c20067390 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9c340b0600 tx=0x7f9c28009040 comp rx=0 tx=0).stop 2023-11-08T08:22:30.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c33fff700 1 -- 172.21.15.40:0/2132656538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c340cd240 msgr2=0x7f9c340c58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:22:30.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c33fff700 1 --2- 172.21.15.40:0/2132656538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c340cd240 0x7f9c340c58c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c00cc50 tx=0x7f9c2c00a340 comp rx=0 tx=0).stop 2023-11-08T08:22:30.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c33fff700 1 -- 172.21.15.40:0/2132656538 shutdown_connections 2023-11-08T08:22:30.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c33fff700 1 --2- 172.21.15.40:0/2132656538 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9c20064ee0 0x7f9c20067390 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c33fff700 1 --2- 172.21.15.40:0/2132656538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c340cd240 0x7f9c340c58c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c33fff700 1 --2- 172.21.15.40:0/2132656538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c340cb9e0 0x7f9c34151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c33fff700 1 --2- 172.21.15.40:0/2132656538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c3406bd50 0x7f9c340c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:22:30.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c33fff700 1 -- 172.21.15.40:0/2132656538 >> 172.21.15.40:0/2132656538 conn(0x7f9c340d0300 msgr2=0x7f9c3406a480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:22:30.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c33fff700 1 -- 172.21.15.40:0/2132656538 shutdown_connections 2023-11-08T08:22:30.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:22:30.663+0000 7f9c33fff700 1 -- 172.21.15.40:0/2132656538 wait complete. 2023-11-08T08:22:30.690 INFO:tasks.cephfs.filesystem:scrub status for tag:3750476d-8094-4366-ab54-48c84ebb62cd - {'path': '/', 'tag': '3750476d-8094-4366-ab54-48c84ebb62cd', 'options': 'recursive'} 2023-11-08T08:22:37.037 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-11-08T08:22:37.030+0000 7f513abf9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-11-08T08:22:56.959 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:22:56.967 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:23:00.691 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-08T08:23:00.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.874+0000 7f440a11d700 1 Processor -- start 2023-11-08T08:23:00.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.874+0000 7f440a11d700 1 -- start start 2023-11-08T08:23:00.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.874+0000 7f440a11d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f44040ced50 0x7f440406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.878+0000 7f440a11d700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44040d24d0 con 0x7f4404070790 2023-11-08T08:23:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.878+0000 7f440a11d700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44040d27b0 con 0x7f4404160770 2023-11-08T08:23:00.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.878+0000 7f440a11d700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44040d08d0 con 0x7f44040ced50 2023-11-08T08:23:00.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.878+0000 7f440991c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4404070790 0x7f4404069cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33922/0 (socket says 172.21.15.40:33922) 2023-11-08T08:23:00.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.878+0000 7f440991c700 1 -- 172.21.15.40:0/3920118824 learned_addr learned my addr 172.21.15.40:0/3920118824 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:00.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.878+0000 7f440911b700 1 --2- 172.21.15.40:0/3920118824 >> v2:172.21.15.169:3300/0 conn(0x7f44040ced50 0x7f440406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:00.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.878+0000 7f43fbfff700 1 -- 172.21.15.40:0/3920118824 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1635376088 0 0) 0x7f44040d24d0 con 0x7f4404070790 2023-11-08T08:23:00.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.878+0000 7f43fbfff700 1 -- 172.21.15.40:0/3920118824 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43f00035d0 con 0x7f4404070790 2023-11-08T08:23:00.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.878+0000 7f440911b700 1 -- 172.21.15.40:0/3920118824 >> v1:172.21.15.169:6789/0 conn(0x7f4404160770 legacy=0x7f4404070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:00.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.878+0000 7f440911b700 1 -- 172.21.15.40:0/3920118824 >> v1:172.21.15.40:6789/0 conn(0x7f4404070790 legacy=0x7f4404069cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:00.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.878+0000 7f440911b700 1 -- 172.21.15.40:0/3920118824 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44040d0bb0 con 0x7f44040ced50 2023-11-08T08:23:00.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.878+0000 7f440911b700 1 --2- 172.21.15.40:0/3920118824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44040ced50 0x7f440406afc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4400009850 tx=0x7f4400009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea80d8ecb8437c8a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:00.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.878+0000 7f43fbfff700 1 -- 172.21.15.40:0/3920118824 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400011830 con 0x7f44040ced50 2023-11-08T08:23:00.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.878+0000 7f43fbfff700 1 -- 172.21.15.40:0/3920118824 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4400011990 con 0x7f44040ced50 2023-11-08T08:23:00.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.878+0000 7f43fbfff700 1 -- 172.21.15.40:0/3920118824 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400011d40 con 0x7f44040ced50 2023-11-08T08:23:00.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.878+0000 7f440a11d700 1 -- 172.21.15.40:0/3920118824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44040ced50 msgr2=0x7f440406afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:00.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.882+0000 7f440a11d700 1 --2- 172.21.15.40:0/3920118824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44040ced50 0x7f440406afc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4400009850 tx=0x7f4400009b60 comp rx=0 tx=0).stop 2023-11-08T08:23:00.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.882+0000 7f440a11d700 1 -- 172.21.15.40:0/3920118824 shutdown_connections 2023-11-08T08:23:00.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.882+0000 7f440a11d700 1 --2- 172.21.15.40:0/3920118824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44040ced50 0x7f440406afc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.882+0000 7f440a11d700 1 -- 172.21.15.40:0/3920118824 >> 172.21.15.40:0/3920118824 conn(0x7f44040b53a0 msgr2=0x7f44040d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:00.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.882+0000 7f440a11d700 1 -- 172.21.15.40:0/3920118824 shutdown_connections 2023-11-08T08:23:00.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.882+0000 7f440a11d700 1 -- 172.21.15.40:0/3920118824 wait complete. 2023-11-08T08:23:00.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.882+0000 7f440a11d700 1 Processor -- start 2023-11-08T08:23:00.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.882+0000 7f440a11d700 1 -- start start 2023-11-08T08:23:00.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f440a11d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4404070790 0x7f440415a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:00.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f440a11d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44040ced50 0x7f440415cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:00.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f440a11d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4404160770 0x7f44040b3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:00.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f440a11d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440415df40 con 0x7f4404070790 2023-11-08T08:23:00.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f440a11d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440415a640 con 0x7f4404160770 2023-11-08T08:23:00.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f440a11d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44040cac30 con 0x7f44040ced50 2023-11-08T08:23:00.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f440911b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4404070790 0x7f440415a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:00.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f440911b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4404070790 0x7f440415a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60090/0 (socket says 172.21.15.40:60090) 2023-11-08T08:23:00.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f440911b700 1 -- 172.21.15.40:0/2113276143 learned_addr learned my addr 172.21.15.40:0/2113276143 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:00.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f440891a700 1 --2- 172.21.15.40:0/2113276143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44040ced50 0x7f440415cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:00.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f440991c700 1 --2- 172.21.15.40:0/2113276143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4404160770 0x7f44040b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:00.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f440911b700 1 -- 172.21.15.40:0/2113276143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44040ced50 msgr2=0x7f440415cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:00.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f440911b700 1 --2- 172.21.15.40:0/2113276143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44040ced50 0x7f440415cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f440911b700 1 -- 172.21.15.40:0/2113276143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4404160770 msgr2=0x7f44040b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:00.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f440911b700 1 --2- 172.21.15.40:0/2113276143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4404160770 0x7f44040b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f440911b700 1 -- 172.21.15.40:0/2113276143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44000094b0 con 0x7f4404070790 2023-11-08T08:23:00.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f440911b700 1 --2- 172.21.15.40:0/2113276143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4404070790 0x7f440415a100 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4400000c00 tx=0x7f4400004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:00.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f43f9ffb700 1 -- 172.21.15.40:0/2113276143 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440001aac0 con 0x7f4404070790 2023-11-08T08:23:00.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f43f9ffb700 1 -- 172.21.15.40:0/2113276143 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440001f070 con 0x7f4404070790 2023-11-08T08:23:00.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f440a11d700 1 -- 172.21.15.40:0/2113276143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4404160bf0 con 0x7f4404070790 2023-11-08T08:23:00.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f43f9ffb700 1 -- 172.21.15.40:0/2113276143 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440001ae60 con 0x7f4404070790 2023-11-08T08:23:00.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.886+0000 7f440a11d700 1 -- 172.21.15.40:0/2113276143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44040cf120 con 0x7f4404070790 2023-11-08T08:23:00.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.890+0000 7f43f9ffb700 1 -- 172.21.15.40:0/2113276143 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f440000f050 con 0x7f4404070790 2023-11-08T08:23:00.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.894+0000 7f43f9ffb700 1 --2- 172.21.15.40:0/2113276143 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f43e8064f10 0x7f43e80673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:00.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.894+0000 7f43f9ffb700 1 -- 172.21.15.40:0/2113276143 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f4400083220 con 0x7f4404070790 2023-11-08T08:23:00.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.894+0000 7f440891a700 1 --2- 172.21.15.40:0/2113276143 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f43e8064f10 0x7f43e80673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:00.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.894+0000 7f440e606740 1 Processor -- start 2023-11-08T08:23:00.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.894+0000 7f440e606740 1 -- start start 2023-11-08T08:23:00.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.894+0000 7f440891a700 1 --2- 172.21.15.40:0/2113276143 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f43e8064f10 0x7f43e80673c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f44040b3f30 tx=0x7f43f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:00.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.894+0000 7f440e606740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297c10 0x154ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:00.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440e606740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1297840 0x154f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:00.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440e606740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x12e28c0 0x154b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:00.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440e606740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x1406d20 con 0x1297c10 2023-11-08T08:23:00.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440e606740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1476d60 con 0x1297840 2023-11-08T08:23:00.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440e606740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1470060 con 0x12e28c0 2023-11-08T08:23:00.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440911b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297c10 0x154ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:00.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440911b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297c10 0x154ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60100/0 (socket says 172.21.15.40:60100) 2023-11-08T08:23:00.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440911b700 1 -- 172.21.15.40:0/2980726591 learned_addr learned my addr 172.21.15.40:0/2980726591 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:00.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440991c700 1 --2- 172.21.15.40:0/2980726591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x12e28c0 0x154b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:00.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440911b700 1 -- 172.21.15.40:0/2980726591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x12e28c0 msgr2=0x154b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:00.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440891a700 1 --2- 172.21.15.40:0/2980726591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1297840 0x154f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:00.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440911b700 1 --2- 172.21.15.40:0/2980726591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x12e28c0 0x154b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440911b700 1 -- 172.21.15.40:0/2980726591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1297840 msgr2=0x154f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:00.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440911b700 1 --2- 172.21.15.40:0/2980726591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1297840 0x154f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440911b700 1 -- 172.21.15.40:0/2980726591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x14960a0 con 0x1297c10 2023-11-08T08:23:00.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440991c700 1 --2- 172.21.15.40:0/2980726591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x12e28c0 0x154b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:23:00.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440891a700 1 --2- 172.21.15.40:0/2980726591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1297840 0x154f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:23:00.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440911b700 1 --2- 172.21.15.40:0/2980726591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297c10 0x154ee00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4400081670 tx=0x7f4400054840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0b386d4d20b566 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:00.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f43eeffd700 1 -- 172.21.15.40:0/2980726591 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400054300 con 0x1297c10 2023-11-08T08:23:00.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f43eeffd700 1 -- 172.21.15.40:0/2980726591 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4400054460 con 0x1297c10 2023-11-08T08:23:00.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f43eeffd700 1 -- 172.21.15.40:0/2980726591 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44000545d0 con 0x1297c10 2023-11-08T08:23:00.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440e606740 1 -- 172.21.15.40:0/2980726591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297c10 msgr2=0x154ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:00.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440e606740 1 --2- 172.21.15.40:0/2980726591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297c10 0x154ee00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4400081670 tx=0x7f4400054840 comp rx=0 tx=0).stop 2023-11-08T08:23:00.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440e606740 1 -- 172.21.15.40:0/2980726591 shutdown_connections 2023-11-08T08:23:00.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440e606740 1 --2- 172.21.15.40:0/2980726591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x12e28c0 0x154b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440e606740 1 --2- 172.21.15.40:0/2980726591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1297840 0x154f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440e606740 1 --2- 172.21.15.40:0/2980726591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297c10 0x154ee00 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440e606740 1 -- 172.21.15.40:0/2980726591 >> 172.21.15.40:0/2980726591 conn(0x1297fe0 msgr2=0x1530d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:00.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440e606740 1 -- 172.21.15.40:0/2980726591 shutdown_connections 2023-11-08T08:23:00.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.898+0000 7f440e606740 1 -- 172.21.15.40:0/2980726591 wait complete. 2023-11-08T08:23:00.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440e606740 1 -- start start 2023-11-08T08:23:00.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440e606740 1 Processor -- start 2023-11-08T08:23:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440e606740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297840 0x152c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440e606740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1297c10 0x1511f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440e606740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x12e28c0 0x1512a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440e606740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x131cc50 con 0x1297840 2023-11-08T08:23:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440e606740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1512f50 con 0x1297c10 2023-11-08T08:23:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440e606740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x152c8b0 con 0x12e28c0 2023-11-08T08:23:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440e606740 10 client.0 _ll_register_callbacks cb 0x14699a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:23:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440911b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297840 0x152c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440911b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297840 0x152c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60114/0 (socket says 172.21.15.40:60114) 2023-11-08T08:23:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440911b700 1 -- 172.21.15.40:0/3986158054 learned_addr learned my addr 172.21.15.40:0/3986158054 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:00.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440891a700 1 --2- 172.21.15.40:0/3986158054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1297c10 0x1511f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:00.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440991c700 1 --2- 172.21.15.40:0/3986158054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x12e28c0 0x1512a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:00.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440911b700 1 -- 172.21.15.40:0/3986158054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x12e28c0 msgr2=0x1512a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:00.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440911b700 1 --2- 172.21.15.40:0/3986158054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x12e28c0 0x1512a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440911b700 1 -- 172.21.15.40:0/3986158054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1297c10 msgr2=0x1511f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:00.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440911b700 1 --2- 172.21.15.40:0/3986158054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1297c10 0x1511f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440911b700 1 -- 172.21.15.40:0/3986158054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44040cf120 con 0x1297840 2023-11-08T08:23:00.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440991c700 1 --2- 172.21.15.40:0/3986158054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x12e28c0 0x1512a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:23:00.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.902+0000 7f440891a700 1 --2- 172.21.15.40:0/3986158054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1297c10 0x1511f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:23:00.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f440911b700 1 --2- 172.21.15.40:0/3986158054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297840 0x152c1e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f440005a010 tx=0x7f440005a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149461efe1027623 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:00.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f43e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:23:00.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f43e3fff700 1 -- 172.21.15.40:0/3986158054 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400054300 con 0x1297840 2023-11-08T08:23:00.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f43e3fff700 1 -- 172.21.15.40:0/3986158054 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f440005f070 con 0x1297840 2023-11-08T08:23:00.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f440e606740 1 -- 172.21.15.40:0/3986158054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x14690e0 con 0x1297840 2023-11-08T08:23:00.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f43e3fff700 1 -- 172.21.15.40:0/3986158054 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f440005cc10 con 0x1297840 2023-11-08T08:23:00.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f440e606740 10 client.6486 fetch_fsmap learned FSMap version 34 2023-11-08T08:23:00.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f440e606740 1 -- 172.21.15.40:0/3986158054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x12e3b10 con 0x1297840 2023-11-08T08:23:00.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f43e3fff700 1 -- 172.21.15.40:0/3986158054 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x14690e0 con 0x1297840 2023-11-08T08:23:00.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f43e3fff700 1 -- 172.21.15.40:0/3986158054 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 34) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4400068080 con 0x1297840 2023-11-08T08:23:00.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f440e606740 10 client.6486 fetch_fsmap finished waiting for FSMap version 34 2023-11-08T08:23:00.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f440e606740 10 client.6486 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-08T08:23:00.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f440e606740 20 client.6486 populate_metadata read hostname 'smithi040' 2023-11-08T08:23:00.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f440e606740 1 --2- 172.21.15.40:0/3986158054 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x152d7d0 0x152dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:00.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f440e606740 4 client.6486 mds_command: new command op to 6237 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-08T08:23:00.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f440e606740 1 -- 172.21.15.40:0/3986158054 --> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1468e00 con 0x152d7d0 2023-11-08T08:23:00.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f440891a700 1 --2- 172.21.15.40:0/3986158054 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x152d7d0 0x152dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:00.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f440891a700 1 --2- 172.21.15.40:0/3986158054 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x152d7d0 0x152dbd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5a0f5de5e5bbbe8 server_cookie=31367e671c6ff48d in_seq=0 out_seq=0 2023-11-08T08:23:00.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f43e3fff700 10 client.6486 ms_handle_connect on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:00.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f43e3fff700 1 -- 172.21.15.40:0/3986158054 <== mds.0 v2:172.21.15.40:6834/3395630689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x1468e00 con 0x152d7d0 2023-11-08T08:23:00.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.906+0000 7f43e3fff700 10 client.6486 handle_command_reply: tid=0 2023-11-08T08:23:00.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.910+0000 7f440e606740 1 client.6486 shutdown 2023-11-08T08:23:00.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.910+0000 7f440e606740 1 -- 172.21.15.40:0/3986158054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297840 msgr2=0x152c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:00.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.910+0000 7f440e606740 1 --2- 172.21.15.40:0/3986158054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297840 0x152c1e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f440005a010 tx=0x7f440005a040 comp rx=0 tx=0).stop 2023-11-08T08:23:00.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.910+0000 7f440e606740 1 -- 172.21.15.40:0/3986158054 shutdown_connections 2023-11-08T08:23:00.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.910+0000 7f440e606740 1 --2- 172.21.15.40:0/3986158054 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x152d7d0 0x152dbd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.910+0000 7f440e606740 1 --2- 172.21.15.40:0/3986158054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x12e28c0 0x1512a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.910+0000 7f43e3fff700 0 client.6486 ms_handle_reset on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:00.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.910+0000 7f440e606740 1 --2- 172.21.15.40:0/3986158054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1297c10 0x1511f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.910+0000 7f440e606740 1 --2- 172.21.15.40:0/3986158054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297840 0x152c1e0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.910+0000 7f440e606740 1 -- 172.21.15.40:0/3986158054 >> 172.21.15.40:0/3986158054 conn(0x1297fe0 msgr2=0x15276f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:00.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.910+0000 7f440e606740 1 -- 172.21.15.40:0/3986158054 shutdown_connections 2023-11-08T08:23:00.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.910+0000 7f440e606740 1 -- 172.21.15.40:0/3986158054 wait complete. 2023-11-08T08:23:00.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.910+0000 7f440e606740 20 client.6486 trim_cache size 0 max 16384 2023-11-08T08:23:00.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.930+0000 7f440e606740 1 Processor -- start 2023-11-08T08:23:00.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.930+0000 7f440e606740 1 -- start start 2023-11-08T08:23:00.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.930+0000 7f440e606740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1297840 0x1511630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:00.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.930+0000 7f440e606740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297c10 0x1511b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:00.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.930+0000 7f440e606740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x12e28c0 0x15544a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:00.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.930+0000 7f440e606740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x1432140 con 0x1297c10 2023-11-08T08:23:00.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.930+0000 7f440e606740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1435440 con 0x1297840 2023-11-08T08:23:00.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.930+0000 7f440e606740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1406200 con 0x12e28c0 2023-11-08T08:23:00.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.930+0000 7f440891a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297c10 0x1511b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:00.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.930+0000 7f440991c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x12e28c0 0x15544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:00.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.930+0000 7f440891a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297c10 0x1511b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60122/0 (socket says 172.21.15.40:60122) 2023-11-08T08:23:00.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.930+0000 7f440891a700 1 -- 172.21.15.40:0/1338551264 learned_addr learned my addr 172.21.15.40:0/1338551264 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:00.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.930+0000 7f440911b700 1 --2- 172.21.15.40:0/1338551264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1297840 0x1511630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:00.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.930+0000 7f440991c700 1 -- 172.21.15.40:0/1338551264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1297840 msgr2=0x1511630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:00.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.930+0000 7f440991c700 1 --2- 172.21.15.40:0/1338551264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1297840 0x1511630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.930+0000 7f440991c700 1 -- 172.21.15.40:0/1338551264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297c10 msgr2=0x1511b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:00.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.930+0000 7f440991c700 1 --2- 172.21.15.40:0/1338551264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297c10 0x1511b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.930+0000 7f440991c700 1 -- 172.21.15.40:0/1338551264 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44040cb0d0 con 0x12e28c0 2023-11-08T08:23:00.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.934+0000 7f440991c700 1 --2- 172.21.15.40:0/1338551264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x12e28c0 0x15544a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f43fc001560 tx=0x7f43fc001f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1494b91a0df892 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:00.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.934+0000 7f43ef7fe700 1 -- 172.21.15.40:0/1338551264 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc001fb0 con 0x12e28c0 2023-11-08T08:23:00.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.934+0000 7f43ef7fe700 1 -- 172.21.15.40:0/1338551264 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc00d030 con 0x12e28c0 2023-11-08T08:23:00.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.934+0000 7f43ef7fe700 1 -- 172.21.15.40:0/1338551264 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc007740 con 0x12e28c0 2023-11-08T08:23:00.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.934+0000 7f440e606740 1 -- 172.21.15.40:0/1338551264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x12e28c0 msgr2=0x15544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:00.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.934+0000 7f440e606740 1 --2- 172.21.15.40:0/1338551264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x12e28c0 0x15544a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f43fc001560 tx=0x7f43fc001f40 comp rx=0 tx=0).stop 2023-11-08T08:23:00.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.934+0000 7f440e606740 1 -- 172.21.15.40:0/1338551264 shutdown_connections 2023-11-08T08:23:00.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.934+0000 7f440e606740 1 --2- 172.21.15.40:0/1338551264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x12e28c0 0x15544a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.934+0000 7f440e606740 1 --2- 172.21.15.40:0/1338551264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297c10 0x1511b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.934+0000 7f440e606740 1 --2- 172.21.15.40:0/1338551264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1297840 0x1511630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.934+0000 7f440e606740 1 -- 172.21.15.40:0/1338551264 >> 172.21.15.40:0/1338551264 conn(0x1297fe0 msgr2=0x1510f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:00.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.934+0000 7f440e606740 1 -- 172.21.15.40:0/1338551264 shutdown_connections 2023-11-08T08:23:00.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.934+0000 7f440e606740 1 -- 172.21.15.40:0/1338551264 wait complete. 2023-11-08T08:23:00.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.934+0000 7f440e606740 1 -- start start 2023-11-08T08:23:00.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.934+0000 7f440e606740 1 Processor -- start 2023-11-08T08:23:00.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440e606740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297840 0x1547db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:00.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440e606740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1297c10 0x1546880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:00.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440e606740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x12e28c0 0x15adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:00.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440e606740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x140f510 con 0x1297840 2023-11-08T08:23:00.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440e606740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x152bfa0 con 0x12e28c0 2023-11-08T08:23:00.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440e606740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x15473d0 con 0x1297c10 2023-11-08T08:23:00.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440e606740 10 client.0 _ll_register_callbacks cb 0x7f4400051150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:23:00.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440911b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297840 0x1547db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:00.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440911b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297840 0x1547db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60128/0 (socket says 172.21.15.40:60128) 2023-11-08T08:23:00.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440911b700 1 -- 172.21.15.40:0/1236214106 learned_addr learned my addr 172.21.15.40:0/1236214106 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:00.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440991c700 1 --2- 172.21.15.40:0/1236214106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x12e28c0 0x15adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:00.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440891a700 1 --2- 172.21.15.40:0/1236214106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1297c10 0x1546880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:00.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440991c700 1 -- 172.21.15.40:0/1236214106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1297c10 msgr2=0x1546880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:00.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440991c700 1 --2- 172.21.15.40:0/1236214106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1297c10 0x1546880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440991c700 1 -- 172.21.15.40:0/1236214106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297840 msgr2=0x1547db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:00.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440991c700 1 --2- 172.21.15.40:0/1236214106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297840 0x1547db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440991c700 1 -- 172.21.15.40:0/1236214106 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43fc00f040 con 0x12e28c0 2023-11-08T08:23:00.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440911b700 1 --2- 172.21.15.40:0/1236214106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297840 0x1547db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:23:00.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440991c700 1 --2- 172.21.15.40:0/1236214106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x12e28c0 0x15adb20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f43fc018000 tx=0x7f43fc001250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90286977d9a9441a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:00.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f43e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.169:3300/0 2023-11-08T08:23:00.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f43e3fff700 1 -- 172.21.15.40:0/1236214106 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc001790 con 0x12e28c0 2023-11-08T08:23:00.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440e606740 1 -- 172.21.15.40:0/1236214106 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4400054460 con 0x12e28c0 2023-11-08T08:23:00.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f43e3fff700 1 -- 172.21.15.40:0/1236214106 <== mon.1 v2:172.21.15.169:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f43fc010b60 con 0x12e28c0 2023-11-08T08:23:00.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f43e3fff700 1 -- 172.21.15.40:0/1236214106 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43fc010d80 con 0x12e28c0 2023-11-08T08:23:00.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440e606740 10 client.6358 fetch_fsmap learned FSMap version 34 2023-11-08T08:23:00.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.938+0000 7f440e606740 1 -- 172.21.15.40:0/1236214106 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x141caf0 con 0x12e28c0 2023-11-08T08:23:00.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f43e3fff700 1 -- 172.21.15.40:0/1236214106 <== mon.1 v2:172.21.15.169:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4400054460 con 0x12e28c0 2023-11-08T08:23:00.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f43e3fff700 1 -- 172.21.15.40:0/1236214106 <== mon.1 v2:172.21.15.169:3300/0 5 ==== fsmap(e 34) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f43fc010220 con 0x12e28c0 2023-11-08T08:23:00.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f440e606740 10 client.6358 fetch_fsmap finished waiting for FSMap version 34 2023-11-08T08:23:00.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f440e606740 10 client.6358 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-08T08:23:00.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f440e606740 20 client.6358 populate_metadata read hostname 'smithi040' 2023-11-08T08:23:00.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f440e606740 1 --2- 172.21.15.40:0/1236214106 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x15ae060 0x152a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:00.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f440e606740 4 client.6358 mds_command: new command op to 6237 tid=0[{"prefix": "scrub status"}] 2023-11-08T08:23:00.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f440e606740 1 -- 172.21.15.40:0/1236214106 --> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x1314390 con 0x15ae060 2023-11-08T08:23:00.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f440911b700 1 --2- 172.21.15.40:0/1236214106 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x15ae060 0x152a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:00.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f43e3fff700 10 client.6358 ms_handle_connect on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:00.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f440911b700 1 --2- 172.21.15.40:0/1236214106 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x15ae060 0x152a5e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71151486f99ac043 server_cookie=95a6834ef755c6e2 in_seq=0 out_seq=0 2023-11-08T08:23:00.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f43e3fff700 1 -- 172.21.15.40:0/1236214106 <== mds.0 v2:172.21.15.40:6834/3395630689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x1314390 con 0x15ae060 2023-11-08T08:23:00.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f43e3fff700 10 client.6358 handle_command_reply: tid=0 2023-11-08T08:23:00.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f440e606740 1 client.6358 shutdown 2023-11-08T08:23:00.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f440e606740 1 -- 172.21.15.40:0/1236214106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x12e28c0 msgr2=0x15adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:00.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f440e606740 1 --2- 172.21.15.40:0/1236214106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x12e28c0 0x15adb20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f43fc018000 tx=0x7f43fc001250 comp rx=0 tx=0).stop 2023-11-08T08:23:00.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f440e606740 1 -- 172.21.15.40:0/1236214106 shutdown_connections 2023-11-08T08:23:00.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f440e606740 1 --2- 172.21.15.40:0/1236214106 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x15ae060 0x152a5e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f440e606740 1 --2- 172.21.15.40:0/1236214106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x12e28c0 0x15adb20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f43e3fff700 0 client.6358 ms_handle_reset on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f440e606740 1 --2- 172.21.15.40:0/1236214106 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1297c10 0x1546880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f440e606740 1 --2- 172.21.15.40:0/1236214106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1297840 0x1547db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f440e606740 1 -- 172.21.15.40:0/1236214106 >> 172.21.15.40:0/1236214106 conn(0x1297fe0 msgr2=0x1554d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:00.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f440e606740 1 -- 172.21.15.40:0/1236214106 shutdown_connections 2023-11-08T08:23:00.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f440e606740 1 -- 172.21.15.40:0/1236214106 wait complete. 2023-11-08T08:23:00.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.942+0000 7f440e606740 20 client.6358 trim_cache size 0 max 16384 2023-11-08T08:23:00.966 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:23:00.967 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "no active scrubs running", 2023-11-08T08:23:00.967 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": {} 2023-11-08T08:23:00.967 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:23:00.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.946+0000 7f440a11d700 1 -- 172.21.15.40:0/2113276143 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f43e8064f10 msgr2=0x7f43e80673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:00.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.946+0000 7f440a11d700 1 --2- 172.21.15.40:0/2113276143 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f43e8064f10 0x7f43e80673c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f44040b3f30 tx=0x7f43f4009040 comp rx=0 tx=0).stop 2023-11-08T08:23:00.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.946+0000 7f440a11d700 1 -- 172.21.15.40:0/2113276143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4404070790 msgr2=0x7f440415a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:00.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.946+0000 7f440a11d700 1 --2- 172.21.15.40:0/2113276143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4404070790 0x7f440415a100 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4400000c00 tx=0x7f4400004770 comp rx=0 tx=0).stop 2023-11-08T08:23:00.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.946+0000 7f440a11d700 1 -- 172.21.15.40:0/2113276143 shutdown_connections 2023-11-08T08:23:00.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.946+0000 7f440a11d700 1 --2- 172.21.15.40:0/2113276143 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f43e8064f10 0x7f43e80673c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.946+0000 7f440a11d700 1 --2- 172.21.15.40:0/2113276143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4404160770 0x7f44040b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.946+0000 7f440a11d700 1 --2- 172.21.15.40:0/2113276143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44040ced50 0x7f440415cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.946+0000 7f440a11d700 1 --2- 172.21.15.40:0/2113276143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4404070790 0x7f440415a100 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.946+0000 7f440a11d700 1 -- 172.21.15.40:0/2113276143 >> 172.21.15.40:0/2113276143 conn(0x7f44040b53a0 msgr2=0x7f440406bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.946+0000 7f440a11d700 1 -- 172.21.15.40:0/2113276143 shutdown_connections 2023-11-08T08:23:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:00.946+0000 7f440a11d700 1 -- 172.21.15.40:0/2113276143 wait complete. 2023-11-08T08:23:00.974 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-08T08:23:00.974 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:23:01.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.154+0000 7f201b569700 1 Processor -- start 2023-11-08T08:23:01.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.154+0000 7f201b569700 1 -- start start 2023-11-08T08:23:01.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.154+0000 7f201b569700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2014071aa0 0x7f201406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.154+0000 7f201b569700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2014069e00 0x7f201406a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.154+0000 7f201b569700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f201406c600 con 0x7f201415c640 2023-11-08T08:23:01.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.154+0000 7f201b569700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f201406cef0 con 0x7f2014069e00 2023-11-08T08:23:01.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.158+0000 7f201b569700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20140d38f0 con 0x7f2014071aa0 2023-11-08T08:23:01.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.158+0000 7f2019d66700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2014071aa0 0x7f201406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.158+0000 7f2019d66700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2014071aa0 0x7f201406d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:47054/0 (socket says 172.21.15.40:47054) 2023-11-08T08:23:01.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.158+0000 7f2019d66700 1 -- 172.21.15.40:0/1673128410 learned_addr learned my addr 172.21.15.40:0/1673128410 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:01.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.158+0000 7f201ad68700 1 --2- 172.21.15.40:0/1673128410 >> v2:172.21.15.40:3300/0 conn(0x7f2014069e00 0x7f201406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.158+0000 7f2019d66700 1 -- 172.21.15.40:0/1673128410 >> v2:172.21.15.40:3300/0 conn(0x7f2014069e00 msgr2=0x7f201406a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:01.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.158+0000 7f2019d66700 1 --2- 172.21.15.40:0/1673128410 >> v2:172.21.15.40:3300/0 conn(0x7f2014069e00 0x7f201406a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.158+0000 7f2019d66700 1 -- 172.21.15.40:0/1673128410 >> v1:172.21.15.169:6789/0 conn(0x7f201415c640 legacy=0x7f20140d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:01.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.158+0000 7f2019d66700 1 -- 172.21.15.40:0/1673128410 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20140722f0 con 0x7f2014071aa0 2023-11-08T08:23:01.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.158+0000 7f2019565700 1 -- 172.21.15.40:0/1673128410 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1322293128 0 0) 0x7f201406c600 con 0x7f201415c640 2023-11-08T08:23:01.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.158+0000 7f2019d66700 1 --2- 172.21.15.40:0/1673128410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2014071aa0 0x7f201406d310 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2014070e70 tx=0x7f2004009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9daba247e7aa2e16 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:01.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.158+0000 7f2019565700 1 -- 172.21.15.40:0/1673128410 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2004011830 con 0x7f2014071aa0 2023-11-08T08:23:01.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.158+0000 7f2019565700 1 -- 172.21.15.40:0/1673128410 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2004011990 con 0x7f2014071aa0 2023-11-08T08:23:01.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.158+0000 7f2019565700 1 -- 172.21.15.40:0/1673128410 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2004011d20 con 0x7f2014071aa0 2023-11-08T08:23:01.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.162+0000 7f201b569700 1 -- 172.21.15.40:0/1673128410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2014071aa0 msgr2=0x7f201406d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:01.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.162+0000 7f201b569700 1 --2- 172.21.15.40:0/1673128410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2014071aa0 0x7f201406d310 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2014070e70 tx=0x7f2004009bf0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.162+0000 7f201b569700 1 -- 172.21.15.40:0/1673128410 shutdown_connections 2023-11-08T08:23:01.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.162+0000 7f201b569700 1 --2- 172.21.15.40:0/1673128410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2014071aa0 0x7f201406d310 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.162+0000 7f201b569700 1 --2- 172.21.15.40:0/1673128410 >> v2:172.21.15.40:3300/0 conn(0x7f2014069e00 0x7f201406a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.162+0000 7f201b569700 1 -- 172.21.15.40:0/1673128410 >> 172.21.15.40:0/1673128410 conn(0x7f20140b6520 msgr2=0x7f2014071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:01.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.162+0000 7f201b569700 1 -- 172.21.15.40:0/1673128410 shutdown_connections 2023-11-08T08:23:01.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.162+0000 7f201b569700 1 -- 172.21.15.40:0/1673128410 wait complete. 2023-11-08T08:23:01.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.166+0000 7f201b569700 1 Processor -- start 2023-11-08T08:23:01.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.166+0000 7f201b569700 1 -- start start 2023-11-08T08:23:01.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.166+0000 7f201b569700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2014069e00 0x7f20141f1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.166+0000 7f201b569700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2014071aa0 0x7f201415a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.166+0000 7f201b569700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f201415c640 0x7f201415a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.166+0000 7f201b569700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201409c110 con 0x7f2014069e00 2023-11-08T08:23:01.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.166+0000 7f201b569700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20140cb860 con 0x7f2014071aa0 2023-11-08T08:23:01.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.166+0000 7f201b569700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20140ca7c0 con 0x7f201415c640 2023-11-08T08:23:01.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.166+0000 7f201a567700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2014069e00 0x7f20141f1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.166+0000 7f201a567700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2014069e00 0x7f20141f1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60144/0 (socket says 172.21.15.40:60144) 2023-11-08T08:23:01.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.166+0000 7f201a567700 1 -- 172.21.15.40:0/1719067123 learned_addr learned my addr 172.21.15.40:0/1719067123 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:01.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.166+0000 7f2019d66700 1 --2- 172.21.15.40:0/1719067123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2014071aa0 0x7f201415a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.166+0000 7f201ad68700 1 --2- 172.21.15.40:0/1719067123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f201415c640 0x7f201415a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.166+0000 7f201a567700 1 -- 172.21.15.40:0/1719067123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f201415c640 msgr2=0x7f201415a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:01.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.166+0000 7f201a567700 1 --2- 172.21.15.40:0/1719067123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f201415c640 0x7f201415a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.166+0000 7f201a567700 1 -- 172.21.15.40:0/1719067123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2014071aa0 msgr2=0x7f201415a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:01.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.166+0000 7f201a567700 1 --2- 172.21.15.40:0/1719067123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2014071aa0 0x7f201415a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.170+0000 7f201a567700 1 -- 172.21.15.40:0/1719067123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2004009570 con 0x7f2014069e00 2023-11-08T08:23:01.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.170+0000 7f201a567700 1 --2- 172.21.15.40:0/1719067123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2014069e00 0x7f20141f1e50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2010001f90 tx=0x7f201000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:01.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.170+0000 7f200affd700 1 -- 172.21.15.40:0/1719067123 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010003e50 con 0x7f2014069e00 2023-11-08T08:23:01.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.170+0000 7f200affd700 1 -- 172.21.15.40:0/1719067123 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20100020e0 con 0x7f2014069e00 2023-11-08T08:23:01.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.170+0000 7f201b569700 1 -- 172.21.15.40:0/1719067123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2014072010 con 0x7f2014069e00 2023-11-08T08:23:01.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.170+0000 7f200affd700 1 -- 172.21.15.40:0/1719067123 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201000ab20 con 0x7f2014069e00 2023-11-08T08:23:01.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.170+0000 7f201b569700 1 -- 172.21.15.40:0/1719067123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f201415ca10 con 0x7f2014069e00 2023-11-08T08:23:01.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.170+0000 7f201b569700 1 -- 172.21.15.40:0/1719067123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f201406c830 con 0x7f2014069e00 2023-11-08T08:23:01.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.174+0000 7f200affd700 1 -- 172.21.15.40:0/1719067123 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f2010019030 con 0x7f2014069e00 2023-11-08T08:23:01.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.174+0000 7f200affd700 1 --2- 172.21.15.40:0/1719067123 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1ff8064ee0 0x7f1ff8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.174+0000 7f200affd700 1 -- 172.21.15.40:0/1719067123 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f2010081ac0 con 0x7f2014069e00 2023-11-08T08:23:01.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.174+0000 7f2019d66700 1 --2- 172.21.15.40:0/1719067123 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1ff8064ee0 0x7f1ff8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.178+0000 7f2019d66700 1 --2- 172.21.15.40:0/1719067123 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1ff8064ee0 0x7f1ff8067390 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2004014040 tx=0x7f2004008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:01.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.182+0000 7f200affd700 1 -- 172.21.15.40:0/1719067123 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f201004f9f0 con 0x7f2014069e00 2023-11-08T08:23:01.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.406+0000 7f201b569700 1 -- 172.21.15.40:0/1719067123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2014073cb0 con 0x7f2014069e00 2023-11-08T08:23:01.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.406+0000 7f200affd700 1 -- 172.21.15.40:0/1719067123 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f2010053010 con 0x7f2014069e00 2023-11-08T08:23:01.411 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:23:01.411 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/307473250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":307473250},{"type":"v1","addr":"172.21.15.169:6835","nonce":307473250}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6148,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/924518366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":924518366},{"type":"v1","addr":"172.21.15.169:6837","nonce":924518366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4294005934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4294005934},{"type":"v1","addr":"172.21.15.40:6833","nonce":4294005934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:22:20.650687+0000","modified":"2023-11-08T08:22:37.032476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6237":{"gid":6237,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":7,"addr":"172.21.15.40:6835/3395630689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3395630689},{"type":"v1","addr":"172.21.15.40:6835","nonce":3395630689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T08:23:01.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.406+0000 7f201b569700 1 -- 172.21.15.40:0/1719067123 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1ff8064ee0 msgr2=0x7f1ff8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:01.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.406+0000 7f201b569700 1 --2- 172.21.15.40:0/1719067123 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1ff8064ee0 0x7f1ff8067390 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2004014040 tx=0x7f2004008030 comp rx=0 tx=0).stop 2023-11-08T08:23:01.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.406+0000 7f201b569700 1 -- 172.21.15.40:0/1719067123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2014069e00 msgr2=0x7f20141f1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:01.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.406+0000 7f201b569700 1 --2- 172.21.15.40:0/1719067123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2014069e00 0x7f20141f1e50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2010001f90 tx=0x7f201000a340 comp rx=0 tx=0).stop 2023-11-08T08:23:01.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.406+0000 7f201b569700 1 -- 172.21.15.40:0/1719067123 shutdown_connections 2023-11-08T08:23:01.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.406+0000 7f201b569700 1 --2- 172.21.15.40:0/1719067123 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1ff8064ee0 0x7f1ff8067390 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.406+0000 7f201b569700 1 --2- 172.21.15.40:0/1719067123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f201415c640 0x7f201415a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.406+0000 7f201b569700 1 --2- 172.21.15.40:0/1719067123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2014071aa0 0x7f201415a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.406+0000 7f201b569700 1 --2- 172.21.15.40:0/1719067123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2014069e00 0x7f20141f1e50 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.410+0000 7f201b569700 1 -- 172.21.15.40:0/1719067123 >> 172.21.15.40:0/1719067123 conn(0x7f20140b6520 msgr2=0x7f20140d1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:01.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.410+0000 7f201b569700 1 -- 172.21.15.40:0/1719067123 shutdown_connections 2023-11-08T08:23:01.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.410+0000 7f201b569700 1 -- 172.21.15.40:0/1719067123 wait complete. 2023-11-08T08:23:01.417 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 34 2023-11-08T08:23:01.434 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2023-11-08T08:23:01.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.610+0000 7fda15298700 1 Processor -- start 2023-11-08T08:23:01.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.610+0000 7fda15298700 1 -- start start 2023-11-08T08:23:01.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.614+0000 7fda15298700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fda100b1640 0x7fda10070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.614+0000 7fda15298700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda1015a550 con 0x7fda100c9040 2023-11-08T08:23:01.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.614+0000 7fda15298700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda1006b0a0 con 0x7fda100b43a0 2023-11-08T08:23:01.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.614+0000 7fda15298700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda100caac0 con 0x7fda100b1640 2023-11-08T08:23:01.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.614+0000 7fda07fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fda100b1640 0x7fda10070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.614+0000 7fda07fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fda100b1640 0x7fda10070080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60158/0 (socket says 172.21.15.40:60158) 2023-11-08T08:23:01.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.614+0000 7fda07fff700 1 -- 172.21.15.40:0/128339424 learned_addr learned my addr 172.21.15.40:0/128339424 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:01.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.614+0000 7fda07fff700 1 -- 172.21.15.40:0/128339424 >> v1:172.21.15.169:6789/0 conn(0x7fda100b43a0 legacy=0x7fda1015e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:01.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.614+0000 7fda07fff700 1 -- 172.21.15.40:0/128339424 >> v1:172.21.15.40:6789/0 conn(0x7fda100c9040 legacy=0x7fda1006bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:01.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.614+0000 7fda07fff700 1 -- 172.21.15.40:0/128339424 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda100cada0 con 0x7fda100b1640 2023-11-08T08:23:01.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.614+0000 7fda07fff700 1 --2- 172.21.15.40:0/128339424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda100b1640 0x7fda10070080 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009910 tx=0x7fd9f8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8dcd32cafe880f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:01.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.614+0000 7fda0f7fe700 1 -- 172.21.15.40:0/128339424 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f800ba60 con 0x7fda100b1640 2023-11-08T08:23:01.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.614+0000 7fda0f7fe700 1 -- 172.21.15.40:0/128339424 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f800bbc0 con 0x7fda100b1640 2023-11-08T08:23:01.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.614+0000 7fda0f7fe700 1 -- 172.21.15.40:0/128339424 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8009e60 con 0x7fda100b1640 2023-11-08T08:23:01.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.614+0000 7fda15298700 1 -- 172.21.15.40:0/128339424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda100b1640 msgr2=0x7fda10070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:01.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.614+0000 7fda15298700 1 --2- 172.21.15.40:0/128339424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda100b1640 0x7fda10070080 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009910 tx=0x7fd9f8009c20 comp rx=0 tx=0).stop 2023-11-08T08:23:01.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.618+0000 7fda15298700 1 -- 172.21.15.40:0/128339424 shutdown_connections 2023-11-08T08:23:01.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.618+0000 7fda15298700 1 --2- 172.21.15.40:0/128339424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda100b1640 0x7fda10070080 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.618+0000 7fda15298700 1 -- 172.21.15.40:0/128339424 >> 172.21.15.40:0/128339424 conn(0x7fda100b0f80 msgr2=0x7fda100ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:01.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.618+0000 7fda15298700 1 -- 172.21.15.40:0/128339424 shutdown_connections 2023-11-08T08:23:01.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.618+0000 7fda15298700 1 -- 172.21.15.40:0/128339424 wait complete. 2023-11-08T08:23:01.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.618+0000 7fda15298700 1 Processor -- start 2023-11-08T08:23:01.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.618+0000 7fda15298700 1 -- start start 2023-11-08T08:23:01.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda15298700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda100b1640 0x7fda100c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda15298700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda100b43a0 0x7fda100c2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda15298700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda100c9040 0x7fda1006a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda15298700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda100c6b10 con 0x7fda100b1640 2023-11-08T08:23:01.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda15298700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda100c2b00 con 0x7fda100c9040 2023-11-08T08:23:01.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda15298700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda100c2de0 con 0x7fda100b43a0 2023-11-08T08:23:01.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda0ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda100b1640 0x7fda100c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda0ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda100b1640 0x7fda100c4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60174/0 (socket says 172.21.15.40:60174) 2023-11-08T08:23:01.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda0ffff700 1 -- 172.21.15.40:0/2145284183 learned_addr learned my addr 172.21.15.40:0/2145284183 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:01.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda07fff700 1 --2- 172.21.15.40:0/2145284183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda100b43a0 0x7fda100c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda14a97700 1 --2- 172.21.15.40:0/2145284183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda100c9040 0x7fda1006a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda0ffff700 1 -- 172.21.15.40:0/2145284183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda100b43a0 msgr2=0x7fda100c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:01.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda0ffff700 1 --2- 172.21.15.40:0/2145284183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda100b43a0 0x7fda100c2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda0ffff700 1 -- 172.21.15.40:0/2145284183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda100c9040 msgr2=0x7fda1006a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:01.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda0ffff700 1 --2- 172.21.15.40:0/2145284183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda100c9040 0x7fda1006a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda0ffff700 1 -- 172.21.15.40:0/2145284183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f8009570 con 0x7fda100b1640 2023-11-08T08:23:01.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda14a97700 1 --2- 172.21.15.40:0/2145284183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda100c9040 0x7fda1006a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:23:01.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda07fff700 1 --2- 172.21.15.40:0/2145284183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda100b43a0 0x7fda100c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:23:01.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda0ffff700 1 --2- 172.21.15.40:0/2145284183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda100b1640 0x7fda100c4960 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fda0000c500 tx=0x7fda0000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:01.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda0d7fa700 1 -- 172.21.15.40:0/2145284183 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda000052d0 con 0x7fda100b1640 2023-11-08T08:23:01.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda0d7fa700 1 -- 172.21.15.40:0/2145284183 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda00005430 con 0x7fda100b1640 2023-11-08T08:23:01.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda0d7fa700 1 -- 172.21.15.40:0/2145284183 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda000055a0 con 0x7fda100b1640 2023-11-08T08:23:01.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda15298700 1 -- 172.21.15.40:0/2145284183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda1006b5d0 con 0x7fda100b1640 2023-11-08T08:23:01.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.622+0000 7fda15298700 1 -- 172.21.15.40:0/2145284183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1006b2f0 con 0x7fda100b1640 2023-11-08T08:23:01.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.626+0000 7fda0d7fa700 1 -- 172.21.15.40:0/2145284183 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fda00019030 con 0x7fda100b1640 2023-11-08T08:23:01.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.626+0000 7fda0d7fa700 1 --2- 172.21.15.40:0/2145284183 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd9f4064ee0 0x7fd9f4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.630+0000 7fda0d7fa700 1 -- 172.21.15.40:0/2145284183 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fda00081440 con 0x7fda100b1640 2023-11-08T08:23:01.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.630+0000 7fda07fff700 1 --2- 172.21.15.40:0/2145284183 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd9f4064ee0 0x7fd9f4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.630+0000 7fda07fff700 1 --2- 172.21.15.40:0/2145284183 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd9f4064ee0 0x7fd9f4067390 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fda1015d8d0 tx=0x7fd9f8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:01.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.630+0000 7fda19782740 1 Processor -- start 2023-11-08T08:23:01.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.630+0000 7fda19782740 1 -- start start 2023-11-08T08:23:01.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.630+0000 7fda19782740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xc8fd20 0xf4eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.630+0000 7fda19782740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xf4f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.630+0000 7fda19782740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xcda9d0 0xefd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.630+0000 7fda19782740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0xdfedc0 con 0xc8f950 2023-11-08T08:23:01.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.630+0000 7fda19782740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0xe6ee20 con 0xcda9d0 2023-11-08T08:23:01.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.630+0000 7fda19782740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0xe68120 con 0xc8fd20 2023-11-08T08:23:01.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda07fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xf4f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda07fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xf4f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60188/0 (socket says 172.21.15.40:60188) 2023-11-08T08:23:01.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda07fff700 1 -- 172.21.15.40:0/3258165978 learned_addr learned my addr 172.21.15.40:0/3258165978 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:01.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda0ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xc8fd20 0xf4eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda14a97700 1 --2- 172.21.15.40:0/3258165978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xcda9d0 0xefd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda07fff700 1 -- 172.21.15.40:0/3258165978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xc8fd20 msgr2=0xf4eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:01.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda07fff700 1 --2- 172.21.15.40:0/3258165978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xc8fd20 0xf4eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda07fff700 1 -- 172.21.15.40:0/3258165978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xcda9d0 msgr2=0xefd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:01.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda07fff700 1 --2- 172.21.15.40:0/3258165978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xcda9d0 0xefd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda07fff700 1 -- 172.21.15.40:0/3258165978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xe8e9b0 con 0xc8f950 2023-11-08T08:23:01.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda0ffff700 1 --2- 172.21.15.40:0/3258165978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xc8fd20 0xf4eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:23:01.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda14a97700 1 --2- 172.21.15.40:0/3258165978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xcda9d0 0xefd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:23:01.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda07fff700 1 --2- 172.21.15.40:0/3258165978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xf4f3f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800b3c0 tx=0x7fd9f8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82737c97ce25dba5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:01.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda05ffb700 1 -- 172.21.15.40:0/3258165978 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80044d0 con 0xc8f950 2023-11-08T08:23:01.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda05ffb700 1 -- 172.21.15.40:0/3258165978 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f802a070 con 0xc8f950 2023-11-08T08:23:01.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda05ffb700 1 -- 172.21.15.40:0/3258165978 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80038c0 con 0xc8f950 2023-11-08T08:23:01.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda19782740 1 -- 172.21.15.40:0/3258165978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 msgr2=0xf4f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:01.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda19782740 1 --2- 172.21.15.40:0/3258165978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xf4f3f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800b3c0 tx=0x7fd9f8010000 comp rx=0 tx=0).stop 2023-11-08T08:23:01.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda19782740 1 -- 172.21.15.40:0/3258165978 shutdown_connections 2023-11-08T08:23:01.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda19782740 1 --2- 172.21.15.40:0/3258165978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xcda9d0 0xefd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda19782740 1 --2- 172.21.15.40:0/3258165978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xf4f3f0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda19782740 1 --2- 172.21.15.40:0/3258165978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xc8fd20 0xf4eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda19782740 1 -- 172.21.15.40:0/3258165978 >> 172.21.15.40:0/3258165978 conn(0xc900f0 msgr2=0xf28e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:01.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda19782740 1 -- 172.21.15.40:0/3258165978 shutdown_connections 2023-11-08T08:23:01.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.634+0000 7fda19782740 1 -- 172.21.15.40:0/3258165978 wait complete. 2023-11-08T08:23:01.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.638+0000 7fda19782740 1 -- start start 2023-11-08T08:23:01.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.638+0000 7fda19782740 1 Processor -- start 2023-11-08T08:23:01.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.638+0000 7fda19782740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xf4ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.638+0000 7fda19782740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xc8fd20 0xf00310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.638+0000 7fda19782740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xcda9d0 0xf00850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.638+0000 7fda19782740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0xd14d60 con 0xc8f950 2023-11-08T08:23:01.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.638+0000 7fda19782740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0xeff800 con 0xc8fd20 2023-11-08T08:23:01.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.638+0000 7fda19782740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0xefed80 con 0xcda9d0 2023-11-08T08:23:01.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda19782740 10 client.0 _ll_register_callbacks cb 0xe61a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:23:01.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda0ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xf4ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda0ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xf4ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60204/0 (socket says 172.21.15.40:60204) 2023-11-08T08:23:01.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda0ffff700 1 -- 172.21.15.40:0/1299448203 learned_addr learned my addr 172.21.15.40:0/1299448203 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:01.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda07fff700 1 --2- 172.21.15.40:0/1299448203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xc8fd20 0xf00310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda14a97700 1 --2- 172.21.15.40:0/1299448203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xcda9d0 0xf00850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda0ffff700 1 -- 172.21.15.40:0/1299448203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xcda9d0 msgr2=0xf00850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:01.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda0ffff700 1 --2- 172.21.15.40:0/1299448203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xcda9d0 0xf00850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda0ffff700 1 -- 172.21.15.40:0/1299448203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xc8fd20 msgr2=0xf00310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:01.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda0ffff700 1 --2- 172.21.15.40:0/1299448203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xc8fd20 0xf00310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda0ffff700 1 -- 172.21.15.40:0/1299448203 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda100cada0 con 0xc8f950 2023-11-08T08:23:01.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda0ffff700 1 --2- 172.21.15.40:0/1299448203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xf4ae50 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fda0005ab90 tx=0x7fda0004f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828330265b91ae66 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:01.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fd9feffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:23:01.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fd9feffd700 1 -- 172.21.15.40:0/1299448203 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0006a1c0 con 0xc8f950 2023-11-08T08:23:01.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fd9feffd700 1 -- 172.21.15.40:0/1299448203 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fda0006a320 con 0xc8f950 2023-11-08T08:23:01.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda19782740 1 -- 172.21.15.40:0/1299448203 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xe611a0 con 0xc8f950 2023-11-08T08:23:01.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fd9feffd700 1 -- 172.21.15.40:0/1299448203 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda0004ee20 con 0xc8f950 2023-11-08T08:23:01.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda19782740 10 client.6513 fetch_fsmap learned FSMap version 34 2023-11-08T08:23:01.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda19782740 1 -- 172.21.15.40:0/1299448203 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0xcdbc20 con 0xc8f950 2023-11-08T08:23:01.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fd9feffd700 1 -- 172.21.15.40:0/1299448203 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xe611a0 con 0xc8f950 2023-11-08T08:23:01.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fd9feffd700 1 -- 172.21.15.40:0/1299448203 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 34) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fda0004ef80 con 0xc8f950 2023-11-08T08:23:01.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda19782740 10 client.6513 fetch_fsmap finished waiting for FSMap version 34 2023-11-08T08:23:01.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda19782740 10 client.6513 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T08:23:01.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda19782740 20 client.6513 populate_metadata read hostname 'smithi040' 2023-11-08T08:23:01.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda19782740 1 --2- 172.21.15.40:0/1299448203 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0xf4b450 0xf58c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda19782740 4 client.6513 mds_command: new command op to 6237 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-08T08:23:01.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda19782740 1 -- 172.21.15.40:0/1299448203 --> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xe60ec0 con 0xf4b450 2023-11-08T08:23:01.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.642+0000 7fda07fff700 1 --2- 172.21.15.40:0/1299448203 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0xf4b450 0xf58c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.646+0000 7fda07fff700 1 --2- 172.21.15.40:0/1299448203 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0xf4b450 0xf58c10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e8796b8f5540682 server_cookie=bb6ddf8ce72a9334 in_seq=0 out_seq=0 2023-11-08T08:23:01.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.646+0000 7fd9feffd700 10 client.6513 ms_handle_connect on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:01.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.646+0000 7fd9feffd700 1 -- 172.21.15.40:0/1299448203 <== mds.0 v2:172.21.15.40:6834/3395630689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0xe60ec0 con 0xf4b450 2023-11-08T08:23:01.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.646+0000 7fd9feffd700 10 client.6513 handle_command_reply: tid=0 2023-11-08T08:23:01.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.646+0000 7fda19782740 1 client.6513 shutdown 2023-11-08T08:23:01.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.646+0000 7fda19782740 1 -- 172.21.15.40:0/1299448203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 msgr2=0xf4ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:01.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.646+0000 7fda19782740 1 --2- 172.21.15.40:0/1299448203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xf4ae50 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fda0005ab90 tx=0x7fda0004f420 comp rx=0 tx=0).stop 2023-11-08T08:23:01.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.646+0000 7fda19782740 1 -- 172.21.15.40:0/1299448203 shutdown_connections 2023-11-08T08:23:01.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.646+0000 7fda19782740 1 --2- 172.21.15.40:0/1299448203 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0xf4b450 0xf58c10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.646+0000 7fda19782740 1 --2- 172.21.15.40:0/1299448203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xcda9d0 0xf00850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.646+0000 7fda19782740 1 --2- 172.21.15.40:0/1299448203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xc8fd20 0xf00310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.646+0000 7fda19782740 1 --2- 172.21.15.40:0/1299448203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xf4ae50 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.646+0000 7fda19782740 1 -- 172.21.15.40:0/1299448203 >> 172.21.15.40:0/1299448203 conn(0xc900f0 msgr2=0xf02700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:01.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.646+0000 7fd9feffd700 0 client.6513 ms_handle_reset on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:01.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.646+0000 7fda19782740 1 -- 172.21.15.40:0/1299448203 shutdown_connections 2023-11-08T08:23:01.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.646+0000 7fda19782740 1 -- 172.21.15.40:0/1299448203 wait complete. 2023-11-08T08:23:01.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.646+0000 7fda19782740 20 client.6513 trim_cache size 0 max 16384 2023-11-08T08:23:01.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.662+0000 7fda19782740 1 Processor -- start 2023-11-08T08:23:01.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.662+0000 7fda19782740 1 -- start start 2023-11-08T08:23:01.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda19782740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xc900f0 0xf57d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda19782740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xefcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda19782740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xc8fd20 0xefd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda19782740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0xf58740 con 0xc8f950 2023-11-08T08:23:01.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda19782740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0xe65930 con 0xc8fd20 2023-11-08T08:23:01.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda19782740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0xe66e40 con 0xc900f0 2023-11-08T08:23:01.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda07fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xefcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda07fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xefcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60220/0 (socket says 172.21.15.40:60220) 2023-11-08T08:23:01.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda07fff700 1 -- 172.21.15.40:0/650953939 learned_addr learned my addr 172.21.15.40:0/650953939 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:01.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda0ffff700 1 --2- 172.21.15.40:0/650953939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xc900f0 0xf57d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda14a97700 1 --2- 172.21.15.40:0/650953939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xc8fd20 0xefd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda07fff700 1 -- 172.21.15.40:0/650953939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xc900f0 msgr2=0xf57d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:01.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda07fff700 1 --2- 172.21.15.40:0/650953939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xc900f0 0xf57d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda07fff700 1 -- 172.21.15.40:0/650953939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xc8fd20 msgr2=0xefd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:01.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda07fff700 1 --2- 172.21.15.40:0/650953939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xc8fd20 0xefd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda07fff700 1 -- 172.21.15.40:0/650953939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1006ace0 con 0xc8f950 2023-11-08T08:23:01.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda0ffff700 1 --2- 172.21.15.40:0/650953939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xc900f0 0xf57d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:23:01.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda07fff700 1 --2- 172.21.15.40:0/650953939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xefcc00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800b990 tx=0x7fd9f80092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7662dd114fb59d3d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:01.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda067fc700 1 -- 172.21.15.40:0/650953939 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80045d0 con 0xc8f950 2023-11-08T08:23:01.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda067fc700 1 -- 172.21.15.40:0/650953939 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f801f070 con 0xc8f950 2023-11-08T08:23:01.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda067fc700 1 -- 172.21.15.40:0/650953939 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f803ca50 con 0xc8f950 2023-11-08T08:23:01.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda19782740 1 -- 172.21.15.40:0/650953939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 msgr2=0xefcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:01.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.666+0000 7fda19782740 1 --2- 172.21.15.40:0/650953939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xefcc00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800b990 tx=0x7fd9f80092e0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda19782740 1 -- 172.21.15.40:0/650953939 shutdown_connections 2023-11-08T08:23:01.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda19782740 1 --2- 172.21.15.40:0/650953939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xc8fd20 0xefd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda19782740 1 --2- 172.21.15.40:0/650953939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xefcc00 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda19782740 1 --2- 172.21.15.40:0/650953939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xc900f0 0xf57d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda19782740 1 -- 172.21.15.40:0/650953939 >> 172.21.15.40:0/650953939 conn(0xf4afc0 msgr2=0xefff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:01.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda19782740 1 -- 172.21.15.40:0/650953939 shutdown_connections 2023-11-08T08:23:01.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda19782740 1 -- 172.21.15.40:0/650953939 wait complete. 2023-11-08T08:23:01.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda19782740 1 -- start start 2023-11-08T08:23:01.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda19782740 1 Processor -- start 2023-11-08T08:23:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda19782740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xf007a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda19782740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xc8fd20 0xfaaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda19782740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xc900f0 0xfab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda19782740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0xe700b0 con 0xc8f950 2023-11-08T08:23:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda19782740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0xf01320 con 0xc8fd20 2023-11-08T08:23:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda19782740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0xe6c120 con 0xc900f0 2023-11-08T08:23:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda19782740 10 client.0 _ll_register_callbacks cb 0x7fd9f8006d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:23:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda0ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xf007a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda0ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xf007a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60224/0 (socket says 172.21.15.40:60224) 2023-11-08T08:23:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda0ffff700 1 -- 172.21.15.40:0/501966784 learned_addr learned my addr 172.21.15.40:0/501966784 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda14a97700 1 --2- 172.21.15.40:0/501966784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xc900f0 0xfab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.670+0000 7fda07fff700 1 --2- 172.21.15.40:0/501966784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xc8fd20 0xfaaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fda0ffff700 1 -- 172.21.15.40:0/501966784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xc900f0 msgr2=0xfab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:01.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fda0ffff700 1 --2- 172.21.15.40:0/501966784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xc900f0 0xfab010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fda0ffff700 1 -- 172.21.15.40:0/501966784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xc8fd20 msgr2=0xfaaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:01.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fda0ffff700 1 --2- 172.21.15.40:0/501966784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xc8fd20 0xfaaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fda0ffff700 1 -- 172.21.15.40:0/501966784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xcdbc20 con 0xc8f950 2023-11-08T08:23:01.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fda0ffff700 1 --2- 172.21.15.40:0/501966784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xf007a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fda00051380 tx=0x7fda0000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99b7f30190c15a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:01.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fd9feffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:23:01.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fd9feffd700 1 -- 172.21.15.40:0/501966784 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda000011c0 con 0xc8f950 2023-11-08T08:23:01.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fd9feffd700 1 -- 172.21.15.40:0/501966784 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fda00066340 con 0xc8f950 2023-11-08T08:23:01.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fda19782740 1 -- 172.21.15.40:0/501966784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9f802a070 con 0xc8f950 2023-11-08T08:23:01.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fd9feffd700 1 -- 172.21.15.40:0/501966784 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda0004f370 con 0xc8f950 2023-11-08T08:23:01.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fda19782740 10 client.6519 fetch_fsmap learned FSMap version 34 2023-11-08T08:23:01.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fda19782740 1 -- 172.21.15.40:0/501966784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0xe8e9b0 con 0xc8f950 2023-11-08T08:23:01.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fd9feffd700 1 -- 172.21.15.40:0/501966784 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f802a070 con 0xc8f950 2023-11-08T08:23:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fd9feffd700 1 -- 172.21.15.40:0/501966784 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 34) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fda0007f3a0 con 0xc8f950 2023-11-08T08:23:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fda19782740 10 client.6519 fetch_fsmap finished waiting for FSMap version 34 2023-11-08T08:23:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fda19782740 10 client.6519 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T08:23:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fda19782740 20 client.6519 populate_metadata read hostname 'smithi040' 2023-11-08T08:23:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fda19782740 1 --2- 172.21.15.40:0/501966784 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0xcda9d0 0xf65450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fda19782740 4 client.6519 mds_command: new command op to 6237 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-11-08T08:23:01.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fda19782740 1 -- 172.21.15.40:0/501966784 --> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0xd0c4a0 con 0xcda9d0 2023-11-08T08:23:01.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.674+0000 7fda07fff700 1 --2- 172.21.15.40:0/501966784 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0xcda9d0 0xf65450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda07fff700 1 --2- 172.21.15.40:0/501966784 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0xcda9d0 0xf65450 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b1f507d9d3e7e5e server_cookie=b09e49c08d819580 in_seq=0 out_seq=0 2023-11-08T08:23:01.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fd9feffd700 10 client.6519 ms_handle_connect on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:01.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fd9feffd700 1 -- 172.21.15.40:0/501966784 <== mds.0 v2:172.21.15.40:6834/3395630689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+141 (crc 0 0 0) 0xd0c4a0 con 0xcda9d0 2023-11-08T08:23:01.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fd9feffd700 10 client.6519 handle_command_reply: tid=0 2023-11-08T08:23:01.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda19782740 1 client.6519 shutdown 2023-11-08T08:23:01.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda19782740 1 -- 172.21.15.40:0/501966784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 msgr2=0xf007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:01.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda19782740 1 --2- 172.21.15.40:0/501966784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xf007a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fda00051380 tx=0x7fda0000cf50 comp rx=0 tx=0).stop 2023-11-08T08:23:01.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda19782740 1 -- 172.21.15.40:0/501966784 shutdown_connections 2023-11-08T08:23:01.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda19782740 1 --2- 172.21.15.40:0/501966784 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0xcda9d0 0xf65450 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda19782740 1 --2- 172.21.15.40:0/501966784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0xc900f0 0xfab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda19782740 1 --2- 172.21.15.40:0/501966784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0xc8fd20 0xfaaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda19782740 1 --2- 172.21.15.40:0/501966784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xc8f950 0xf007a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda19782740 1 -- 172.21.15.40:0/501966784 >> 172.21.15.40:0/501966784 conn(0xf4afc0 msgr2=0xf4a170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:01.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fd9feffd700 0 client.6519 ms_handle_reset on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:01.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda19782740 1 -- 172.21.15.40:0/501966784 shutdown_connections 2023-11-08T08:23:01.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda19782740 1 -- 172.21.15.40:0/501966784 wait complete. 2023-11-08T08:23:01.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda19782740 20 client.6519 trim_cache size 0 max 16384 2023-11-08T08:23:01.692 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:23:01.692 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2023-11-08T08:23:01.692 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:23:01.693 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-11-08T08:23:01.693 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1858360389, 2023-11-08T08:23:01.693 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511627777, 2023-11-08T08:23:01.693 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/dir_test/file" 2023-11-08T08:23:01.693 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:23:01.694 INFO:teuthology.orchestra.run.smithi040.stdout:] 2023-11-08T08:23:01.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda15298700 1 -- 172.21.15.40:0/2145284183 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd9f4064ee0 msgr2=0x7fd9f4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:01.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda15298700 1 --2- 172.21.15.40:0/2145284183 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd9f4064ee0 0x7fd9f4067390 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fda1015d8d0 tx=0x7fd9f8008030 comp rx=0 tx=0).stop 2023-11-08T08:23:01.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda15298700 1 -- 172.21.15.40:0/2145284183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda100b1640 msgr2=0x7fda100c4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:01.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda15298700 1 --2- 172.21.15.40:0/2145284183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda100b1640 0x7fda100c4960 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fda0000c500 tx=0x7fda0000c8c0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda15298700 1 -- 172.21.15.40:0/2145284183 shutdown_connections 2023-11-08T08:23:01.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda15298700 1 --2- 172.21.15.40:0/2145284183 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd9f4064ee0 0x7fd9f4067390 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda15298700 1 --2- 172.21.15.40:0/2145284183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda100c9040 0x7fda1006a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda15298700 1 --2- 172.21.15.40:0/2145284183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda100b43a0 0x7fda100c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda15298700 1 --2- 172.21.15.40:0/2145284183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda100b1640 0x7fda100c4960 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda15298700 1 -- 172.21.15.40:0/2145284183 >> 172.21.15.40:0/2145284183 conn(0x7fda100b0f80 msgr2=0x7fda10159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:01.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda15298700 1 -- 172.21.15.40:0/2145284183 shutdown_connections 2023-11-08T08:23:01.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.678+0000 7fda15298700 1 -- 172.21.15.40:0/2145284183 wait complete. 2023-11-08T08:23:01.709 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-08T08:23:01.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.882+0000 7f7b5ab69700 1 Processor -- start 2023-11-08T08:23:01.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.882+0000 7f7b5ab69700 1 -- start start 2023-11-08T08:23:01.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.882+0000 7f7b5ab69700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7b540afe70 0x7f7b5415e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.882+0000 7f7b5ab69700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7b540cb0f0 0x7f7b5415a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.882+0000 7f7b5ab69700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b540ce600 0x7f7b540cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.882+0000 7f7b5ab69700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b540cefc0 con 0x7f7b540afe70 2023-11-08T08:23:01.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.882+0000 7f7b5ab69700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b5415b620 con 0x7f7b540cb0f0 2023-11-08T08:23:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.882+0000 7f7b5ab69700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5415e0b0 con 0x7f7b540ce600 2023-11-08T08:23:01.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.882+0000 7f7b58905700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7b540afe70 0x7f7b5415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.882+0000 7f7b58905700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7b540afe70 0x7f7b5415e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60236/0 (socket says 172.21.15.40:60236) 2023-11-08T08:23:01.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.882+0000 7f7b58905700 1 -- 172.21.15.40:0/3860377444 learned_addr learned my addr 172.21.15.40:0/3860377444 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:01.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.882+0000 7f7b4bfff700 1 --2- 172.21.15.40:0/3860377444 >> v2:172.21.15.169:3300/0 conn(0x7f7b540cb0f0 0x7f7b5415a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.882+0000 7f7b59106700 1 --2- 172.21.15.40:0/3860377444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b540ce600 0x7f7b540cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.882+0000 7f7b58905700 1 -- 172.21.15.40:0/3860377444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b540ce600 msgr2=0x7f7b540cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:01.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.882+0000 7f7b58905700 1 --2- 172.21.15.40:0/3860377444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b540ce600 0x7f7b540cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.882+0000 7f7b58905700 1 -- 172.21.15.40:0/3860377444 >> v2:172.21.15.169:3300/0 conn(0x7f7b540cb0f0 msgr2=0x7f7b5415a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:01.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.882+0000 7f7b58905700 1 --2- 172.21.15.40:0/3860377444 >> v2:172.21.15.169:3300/0 conn(0x7f7b540cb0f0 0x7f7b5415a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.882+0000 7f7b58905700 1 -- 172.21.15.40:0/3860377444 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5415b060 con 0x7f7b540afe70 2023-11-08T08:23:01.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.882+0000 7f7b4bfff700 1 --2- 172.21.15.40:0/3860377444 >> v2:172.21.15.169:3300/0 conn(0x7f7b540cb0f0 0x7f7b5415a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:23:01.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.886+0000 7f7b58905700 1 --2- 172.21.15.40:0/3860377444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b540afe70 0x7f7b5415e540 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7b540c4530 tx=0x7f7b50009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26dfee8653ca2fe8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:01.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.886+0000 7f7b4b7fe700 1 -- 172.21.15.40:0/3860377444 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50011830 con 0x7f7b540afe70 2023-11-08T08:23:01.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.886+0000 7f7b4b7fe700 1 -- 172.21.15.40:0/3860377444 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b50011990 con 0x7f7b540afe70 2023-11-08T08:23:01.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.886+0000 7f7b4b7fe700 1 -- 172.21.15.40:0/3860377444 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50011d20 con 0x7f7b540afe70 2023-11-08T08:23:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.886+0000 7f7b5ab69700 1 -- 172.21.15.40:0/3860377444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b540afe70 msgr2=0x7f7b5415e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.886+0000 7f7b5ab69700 1 --2- 172.21.15.40:0/3860377444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b540afe70 0x7f7b5415e540 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7b540c4530 tx=0x7f7b50009bf0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.886+0000 7f7b5ab69700 1 -- 172.21.15.40:0/3860377444 shutdown_connections 2023-11-08T08:23:01.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.886+0000 7f7b5ab69700 1 --2- 172.21.15.40:0/3860377444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b540ce600 0x7f7b540cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.886+0000 7f7b5ab69700 1 --2- 172.21.15.40:0/3860377444 >> v2:172.21.15.169:3300/0 conn(0x7f7b540cb0f0 0x7f7b5415a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.886+0000 7f7b5ab69700 1 --2- 172.21.15.40:0/3860377444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b540afe70 0x7f7b5415e540 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.886+0000 7f7b5ab69700 1 -- 172.21.15.40:0/3860377444 >> 172.21.15.40:0/3860377444 conn(0x7f7b5406e5d0 msgr2=0x7f7b5406e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:01.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.886+0000 7f7b5ab69700 1 -- 172.21.15.40:0/3860377444 shutdown_connections 2023-11-08T08:23:01.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.886+0000 7f7b5ab69700 1 -- 172.21.15.40:0/3860377444 wait complete. 2023-11-08T08:23:01.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.890+0000 7f7b5ab69700 1 Processor -- start 2023-11-08T08:23:01.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.890+0000 7f7b5ab69700 1 -- start start 2023-11-08T08:23:01.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.890+0000 7f7b5ab69700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b540afe70 0x7f7b540b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.890+0000 7f7b5ab69700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b540cb0f0 0x7f7b540b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.890+0000 7f7b5ab69700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b540ce600 0x7f7b540b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.890+0000 7f7b5ab69700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540b42e0 con 0x7f7b540cb0f0 2023-11-08T08:23:01.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.890+0000 7f7b5ab69700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540c4560 con 0x7f7b540afe70 2023-11-08T08:23:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.890+0000 7f7b5ab69700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540c4b20 con 0x7f7b540ce600 2023-11-08T08:23:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.890+0000 7f7b4bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b540cb0f0 0x7f7b540b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.890+0000 7f7b4bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b540cb0f0 0x7f7b540b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60238/0 (socket says 172.21.15.40:60238) 2023-11-08T08:23:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.890+0000 7f7b4bfff700 1 -- 172.21.15.40:0/2220453014 learned_addr learned my addr 172.21.15.40:0/2220453014 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.890+0000 7f7b59106700 1 --2- 172.21.15.40:0/2220453014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b540ce600 0x7f7b540b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.890+0000 7f7b58905700 1 --2- 172.21.15.40:0/2220453014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b540afe70 0x7f7b540b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.890+0000 7f7b4bfff700 1 -- 172.21.15.40:0/2220453014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b540ce600 msgr2=0x7f7b540b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:01.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.890+0000 7f7b4bfff700 1 --2- 172.21.15.40:0/2220453014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b540ce600 0x7f7b540b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.890+0000 7f7b4bfff700 1 -- 172.21.15.40:0/2220453014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b540afe70 msgr2=0x7f7b540b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:01.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.890+0000 7f7b4bfff700 1 --2- 172.21.15.40:0/2220453014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b540afe70 0x7f7b540b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:01.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.890+0000 7f7b4bfff700 1 -- 172.21.15.40:0/2220453014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b440094b0 con 0x7f7b540cb0f0 2023-11-08T08:23:01.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.894+0000 7f7b4bfff700 1 --2- 172.21.15.40:0/2220453014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b540cb0f0 0x7f7b540b3860 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400ec10 tx=0x7f7b4400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:01.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.894+0000 7f7b497fa700 1 -- 172.21.15.40:0/2220453014 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4400c850 con 0x7f7b540cb0f0 2023-11-08T08:23:01.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.894+0000 7f7b497fa700 1 -- 172.21.15.40:0/2220453014 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4400c9b0 con 0x7f7b540cb0f0 2023-11-08T08:23:01.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.894+0000 7f7b497fa700 1 -- 172.21.15.40:0/2220453014 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4400cd40 con 0x7f7b540cb0f0 2023-11-08T08:23:01.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.894+0000 7f7b5ab69700 1 -- 172.21.15.40:0/2220453014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b50009570 con 0x7f7b540cb0f0 2023-11-08T08:23:01.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.894+0000 7f7b5ab69700 1 -- 172.21.15.40:0/2220453014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b5415dd40 con 0x7f7b540cb0f0 2023-11-08T08:23:01.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.894+0000 7f7b5ab69700 1 -- 172.21.15.40:0/2220453014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b5415b340 con 0x7f7b540cb0f0 2023-11-08T08:23:01.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.898+0000 7f7b497fa700 1 -- 172.21.15.40:0/2220453014 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7b44019030 con 0x7f7b540cb0f0 2023-11-08T08:23:01.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.898+0000 7f7b497fa700 1 --2- 172.21.15.40:0/2220453014 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7b40064f60 0x7f7b40067410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:01.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.898+0000 7f7b497fa700 1 -- 172.21.15.40:0/2220453014 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f7b44081f80 con 0x7f7b540cb0f0 2023-11-08T08:23:01.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.898+0000 7f7b58905700 1 --2- 172.21.15.40:0/2220453014 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7b40064f60 0x7f7b40067410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:01.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.898+0000 7f7b58905700 1 --2- 172.21.15.40:0/2220453014 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7b40064f60 0x7f7b40067410 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7b50000c00 tx=0x7f7b50008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:01.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:01.902+0000 7f7b497fa700 1 -- 172.21.15.40:0/2220453014 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7b4404ff30 con 0x7f7b540cb0f0 2023-11-08T08:23:02.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.118+0000 7f7b5ab69700 1 -- 172.21.15.40:0/2220453014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7b540b0240 con 0x7f7b540cb0f0 2023-11-08T08:23:02.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.122+0000 7f7b497fa700 1 -- 172.21.15.40:0/2220453014 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f7b4400cea0 con 0x7f7b540cb0f0 2023-11-08T08:23:02.125 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:23:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-08T08:23:02.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.126+0000 7f7b5ab69700 1 -- 172.21.15.40:0/2220453014 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7b40064f60 msgr2=0x7f7b40067410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.126+0000 7f7b5ab69700 1 --2- 172.21.15.40:0/2220453014 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7b40064f60 0x7f7b40067410 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7b50000c00 tx=0x7f7b50008030 comp rx=0 tx=0).stop 2023-11-08T08:23:02.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.126+0000 7f7b5ab69700 1 -- 172.21.15.40:0/2220453014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b540cb0f0 msgr2=0x7f7b540b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.126+0000 7f7b5ab69700 1 --2- 172.21.15.40:0/2220453014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b540cb0f0 0x7f7b540b3860 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400ec10 tx=0x7f7b4400ef20 comp rx=0 tx=0).stop 2023-11-08T08:23:02.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.126+0000 7f7b5ab69700 1 -- 172.21.15.40:0/2220453014 shutdown_connections 2023-11-08T08:23:02.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.126+0000 7f7b5ab69700 1 --2- 172.21.15.40:0/2220453014 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7b40064f60 0x7f7b40067410 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.126+0000 7f7b5ab69700 1 --2- 172.21.15.40:0/2220453014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b540ce600 0x7f7b540b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.126+0000 7f7b5ab69700 1 --2- 172.21.15.40:0/2220453014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b540cb0f0 0x7f7b540b3860 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.126+0000 7f7b5ab69700 1 --2- 172.21.15.40:0/2220453014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b540afe70 0x7f7b540b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.126+0000 7f7b5ab69700 1 -- 172.21.15.40:0/2220453014 >> 172.21.15.40:0/2220453014 conn(0x7f7b5406e5d0 msgr2=0x7f7b54159f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:02.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.126+0000 7f7b5ab69700 1 -- 172.21.15.40:0/2220453014 shutdown_connections 2023-11-08T08:23:02.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.126+0000 7f7b5ab69700 1 -- 172.21.15.40:0/2220453014 wait complete. 2023-11-08T08:23:02.151 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive 2023-11-08T08:23:02.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.322+0000 7ff37645a700 1 Processor -- start 2023-11-08T08:23:02.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.326+0000 7ff37645a700 1 -- start start 2023-11-08T08:23:02.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.326+0000 7ff37645a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3700cc060 0x7ff3700cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.326+0000 7ff37645a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff37015c5f0 0x7ff3700cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.326+0000 7ff37645a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3700b20e0 con 0x7ff370160ba0 2023-11-08T08:23:02.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.326+0000 7ff37645a700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3700b55c0 con 0x7ff37015c5f0 2023-11-08T08:23:02.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.326+0000 7ff37645a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3700cde60 con 0x7ff3700cc060 2023-11-08T08:23:02.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.326+0000 7ff375c59700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff370160ba0 0x7ff3700cf5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33930/0 (socket says 172.21.15.40:33930) 2023-11-08T08:23:02.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.326+0000 7ff375c59700 1 -- 172.21.15.40:0/556123493 learned_addr learned my addr 172.21.15.40:0/556123493 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:02.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.326+0000 7ff375458700 1 --2- 172.21.15.40:0/556123493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3700cc060 0x7ff3700cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.326+0000 7ff374c57700 1 --2- 172.21.15.40:0/556123493 >> v2:172.21.15.169:3300/0 conn(0x7ff37015c5f0 0x7ff3700cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.330+0000 7ff367fff700 1 -- 172.21.15.40:0/556123493 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810292274 0 0) 0x7ff3700b20e0 con 0x7ff370160ba0 2023-11-08T08:23:02.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.330+0000 7ff367fff700 1 -- 172.21.15.40:0/556123493 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff35c0035d0 con 0x7ff370160ba0 2023-11-08T08:23:02.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.330+0000 7ff374c57700 1 -- 172.21.15.40:0/556123493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3700cc060 msgr2=0x7ff3700cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:02.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.330+0000 7ff374c57700 1 --2- 172.21.15.40:0/556123493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3700cc060 0x7ff3700cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.330+0000 7ff374c57700 1 -- 172.21.15.40:0/556123493 >> v1:172.21.15.40:6789/0 conn(0x7ff370160ba0 legacy=0x7ff3700cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.330+0000 7ff374c57700 1 -- 172.21.15.40:0/556123493 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3700c9c10 con 0x7ff37015c5f0 2023-11-08T08:23:02.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.330+0000 7ff374c57700 1 --2- 172.21.15.40:0/556123493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff37015c5f0 0x7ff3700cd550 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff360009850 tx=0x7ff360009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7804d331a97965cf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:02.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.330+0000 7ff367fff700 1 -- 172.21.15.40:0/556123493 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360011830 con 0x7ff37015c5f0 2023-11-08T08:23:02.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.330+0000 7ff367fff700 1 -- 172.21.15.40:0/556123493 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff360011990 con 0x7ff37015c5f0 2023-11-08T08:23:02.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.330+0000 7ff367fff700 1 -- 172.21.15.40:0/556123493 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360011d40 con 0x7ff37015c5f0 2023-11-08T08:23:02.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.330+0000 7ff37645a700 1 -- 172.21.15.40:0/556123493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff37015c5f0 msgr2=0x7ff3700cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.330+0000 7ff37645a700 1 --2- 172.21.15.40:0/556123493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff37015c5f0 0x7ff3700cd550 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff360009850 tx=0x7ff360009b60 comp rx=0 tx=0).stop 2023-11-08T08:23:02.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.330+0000 7ff37645a700 1 -- 172.21.15.40:0/556123493 shutdown_connections 2023-11-08T08:23:02.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.330+0000 7ff37645a700 1 --2- 172.21.15.40:0/556123493 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff37015c5f0 0x7ff3700cd550 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.330+0000 7ff37645a700 1 --2- 172.21.15.40:0/556123493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3700cc060 0x7ff3700cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.330+0000 7ff37645a700 1 -- 172.21.15.40:0/556123493 >> 172.21.15.40:0/556123493 conn(0x7ff37006e100 msgr2=0x7ff37006e500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:02.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.330+0000 7ff37645a700 1 -- 172.21.15.40:0/556123493 shutdown_connections 2023-11-08T08:23:02.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.330+0000 7ff37645a700 1 -- 172.21.15.40:0/556123493 wait complete. 2023-11-08T08:23:02.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.334+0000 7ff37645a700 1 Processor -- start 2023-11-08T08:23:02.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.334+0000 7ff37645a700 1 -- start start 2023-11-08T08:23:02.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff37645a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3700cc060 0x7ff3700c2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff37645a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff37015c5f0 0x7ff3701f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff37645a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff370160ba0 0x7ff3700ad320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff37645a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3700ae790 con 0x7ff3700cc060 2023-11-08T08:23:02.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff37645a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3700cab80 con 0x7ff370160ba0 2023-11-08T08:23:02.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff37645a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3700c3760 con 0x7ff37015c5f0 2023-11-08T08:23:02.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff375458700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3700cc060 0x7ff3700c2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff375458700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3700cc060 0x7ff3700c2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60240/0 (socket says 172.21.15.40:60240) 2023-11-08T08:23:02.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff375458700 1 -- 172.21.15.40:0/2760854635 learned_addr learned my addr 172.21.15.40:0/2760854635 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:02.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff374c57700 1 --2- 172.21.15.40:0/2760854635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff37015c5f0 0x7ff3701f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff375c59700 1 --2- 172.21.15.40:0/2760854635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff370160ba0 0x7ff3700ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff375458700 1 -- 172.21.15.40:0/2760854635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff37015c5f0 msgr2=0x7ff3701f5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff375458700 1 --2- 172.21.15.40:0/2760854635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff37015c5f0 0x7ff3701f5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff375458700 1 -- 172.21.15.40:0/2760854635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff370160ba0 msgr2=0x7ff3700ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff375458700 1 --2- 172.21.15.40:0/2760854635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff370160ba0 0x7ff3700ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff375458700 1 -- 172.21.15.40:0/2760854635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3600094b0 con 0x7ff3700cc060 2023-11-08T08:23:02.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff375458700 1 --2- 172.21.15.40:0/2760854635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3700cc060 0x7ff3700c2ad0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00ea00 tx=0x7ff36c00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:02.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff365ffb700 1 -- 172.21.15.40:0/2760854635 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c009720 con 0x7ff3700cc060 2023-11-08T08:23:02.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff365ffb700 1 -- 172.21.15.40:0/2760854635 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36c00eea0 con 0x7ff3700cc060 2023-11-08T08:23:02.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff365ffb700 1 -- 172.21.15.40:0/2760854635 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c00ca50 con 0x7ff3700cc060 2023-11-08T08:23:02.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.338+0000 7ff37645a700 1 -- 172.21.15.40:0/2760854635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3700b4800 con 0x7ff3700cc060 2023-11-08T08:23:02.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.342+0000 7ff37645a700 1 -- 172.21.15.40:0/2760854635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3700cc970 con 0x7ff3700cc060 2023-11-08T08:23:02.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.346+0000 7ff365ffb700 1 -- 172.21.15.40:0/2760854635 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff36c019030 con 0x7ff3700cc060 2023-11-08T08:23:02.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.346+0000 7ff365ffb700 1 --2- 172.21.15.40:0/2760854635 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff354064ee0 0x7ff354067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.346+0000 7ff365ffb700 1 -- 172.21.15.40:0/2760854635 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7ff36c081fb0 con 0x7ff3700cc060 2023-11-08T08:23:02.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.346+0000 7ff37a943740 1 Processor -- start 2023-11-08T08:23:02.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.346+0000 7ff374c57700 1 --2- 172.21.15.40:0/2760854635 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff354064ee0 0x7ff354067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.346+0000 7ff37a943740 1 -- start start 2023-11-08T08:23:02.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff374c57700 1 --2- 172.21.15.40:0/2760854635 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff354064ee0 0x7ff354067390 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff3600090b0 tx=0x7ff360008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:02.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff37a943740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1c6bd80 0x1f19d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff37a943740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6b9b0 0x1f22e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff37a943740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1cb6a30 0x1f23340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff37a943740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ddac80 con 0x1c6b9b0 2023-11-08T08:23:02.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff37a943740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e4ad60 con 0x1c6bd80 2023-11-08T08:23:02.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff37a943740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e44060 con 0x1cb6a30 2023-11-08T08:23:02.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff374c57700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6b9b0 0x1f22e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff374c57700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6b9b0 0x1f22e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60252/0 (socket says 172.21.15.40:60252) 2023-11-08T08:23:02.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff374c57700 1 -- 172.21.15.40:0/1620379764 learned_addr learned my addr 172.21.15.40:0/1620379764 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:02.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff375c59700 1 --2- 172.21.15.40:0/1620379764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1cb6a30 0x1f23340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff375458700 1 --2- 172.21.15.40:0/1620379764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1c6bd80 0x1f19d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff374c57700 1 -- 172.21.15.40:0/1620379764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1cb6a30 msgr2=0x1f23340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:02.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff374c57700 1 --2- 172.21.15.40:0/1620379764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1cb6a30 0x1f23340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff374c57700 1 -- 172.21.15.40:0/1620379764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1c6bd80 msgr2=0x1f19d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:02.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff374c57700 1 --2- 172.21.15.40:0/1620379764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1c6bd80 0x1f19d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff374c57700 1 -- 172.21.15.40:0/1620379764 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1e694e0 con 0x1c6b9b0 2023-11-08T08:23:02.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff374c57700 1 --2- 172.21.15.40:0/1620379764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6b9b0 0x1f22e00 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff360009eb0 tx=0x7ff360010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414e9bec9a846173 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:02.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff352ffd700 1 -- 172.21.15.40:0/1620379764 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36002c070 con 0x1c6b9b0 2023-11-08T08:23:02.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff352ffd700 1 -- 172.21.15.40:0/1620379764 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff360011c00 con 0x1c6b9b0 2023-11-08T08:23:02.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff352ffd700 1 -- 172.21.15.40:0/1620379764 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360020070 con 0x1c6b9b0 2023-11-08T08:23:02.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff37a943740 1 -- 172.21.15.40:0/1620379764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6b9b0 msgr2=0x1f22e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff37a943740 1 --2- 172.21.15.40:0/1620379764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6b9b0 0x1f22e00 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff360009eb0 tx=0x7ff360010000 comp rx=0 tx=0).stop 2023-11-08T08:23:02.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff37a943740 1 -- 172.21.15.40:0/1620379764 shutdown_connections 2023-11-08T08:23:02.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff37a943740 1 --2- 172.21.15.40:0/1620379764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1cb6a30 0x1f23340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff37a943740 1 --2- 172.21.15.40:0/1620379764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6b9b0 0x1f22e00 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff37a943740 1 --2- 172.21.15.40:0/1620379764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1c6bd80 0x1f19d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.350+0000 7ff37a943740 1 -- 172.21.15.40:0/1620379764 >> 172.21.15.40:0/1620379764 conn(0x1c6c150 msgr2=0x1f04350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:02.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.354+0000 7ff37a943740 1 -- 172.21.15.40:0/1620379764 shutdown_connections 2023-11-08T08:23:02.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.354+0000 7ff37a943740 1 -- 172.21.15.40:0/1620379764 wait complete. 2023-11-08T08:23:02.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.354+0000 7ff37a943740 1 -- start start 2023-11-08T08:23:02.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.354+0000 7ff37a943740 1 Processor -- start 2023-11-08T08:23:02.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.354+0000 7ff37a943740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1c6b9b0 0x1edcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.354+0000 7ff37a943740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6bd80 0x1f28340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.354+0000 7ff37a943740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1cb6a30 0x1edc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.354+0000 7ff37a943740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cf0dc0 con 0x1c6bd80 2023-11-08T08:23:02.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.354+0000 7ff37a943740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f19a90 con 0x1c6b9b0 2023-11-08T08:23:02.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.354+0000 7ff37a943740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f1b110 con 0x1cb6a30 2023-11-08T08:23:02.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.354+0000 7ff37a943740 10 client.0 _ll_register_callbacks cb 0x1e9fb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:23:02.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff374c57700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6bd80 0x1f28340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff374c57700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6bd80 0x1f28340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60262/0 (socket says 172.21.15.40:60262) 2023-11-08T08:23:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff374c57700 1 -- 172.21.15.40:0/738462424 learned_addr learned my addr 172.21.15.40:0/738462424 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff375c59700 1 --2- 172.21.15.40:0/738462424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1cb6a30 0x1edc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff375458700 1 --2- 172.21.15.40:0/738462424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1c6b9b0 0x1edcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff374c57700 1 -- 172.21.15.40:0/738462424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1cb6a30 msgr2=0x1edc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:02.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff374c57700 1 --2- 172.21.15.40:0/738462424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1cb6a30 0x1edc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff374c57700 1 -- 172.21.15.40:0/738462424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1c6b9b0 msgr2=0x1edcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff374c57700 1 --2- 172.21.15.40:0/738462424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1c6b9b0 0x1edcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff374c57700 1 -- 172.21.15.40:0/738462424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3700c9c10 con 0x1c6bd80 2023-11-08T08:23:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff375c59700 1 --2- 172.21.15.40:0/738462424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1cb6a30 0x1edc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:23:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff374c57700 1 --2- 172.21.15.40:0/738462424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6bd80 0x1f28340 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff360009210 tx=0x7ff3600047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2169cb92cbf500d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:02.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff34bfff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:23:02.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff34bfff700 1 -- 172.21.15.40:0/738462424 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360003c10 con 0x1c6bd80 2023-11-08T08:23:02.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff34bfff700 1 -- 172.21.15.40:0/738462424 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7ff360003d70 con 0x1c6bd80 2023-11-08T08:23:02.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff37a943740 1 -- 172.21.15.40:0/738462424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1e3d0e0 con 0x1c6bd80 2023-11-08T08:23:02.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff34bfff700 1 -- 172.21.15.40:0/738462424 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff36003a410 con 0x1c6bd80 2023-11-08T08:23:02.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff37a943740 10 client.6537 fetch_fsmap learned FSMap version 34 2023-11-08T08:23:02.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff37a943740 1 -- 172.21.15.40:0/738462424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x1cb7c80 con 0x1c6bd80 2023-11-08T08:23:02.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff34bfff700 1 -- 172.21.15.40:0/738462424 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1e3d0e0 con 0x1c6bd80 2023-11-08T08:23:02.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff34bfff700 1 -- 172.21.15.40:0/738462424 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 34) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff36003a570 con 0x1c6bd80 2023-11-08T08:23:02.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff37a943740 10 client.6537 fetch_fsmap finished waiting for FSMap version 34 2023-11-08T08:23:02.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff37a943740 10 client.6537 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-08T08:23:02.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff37a943740 20 client.6537 populate_metadata read hostname 'smithi040' 2023-11-08T08:23:02.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff37a943740 1 --2- 172.21.15.40:0/738462424 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x1f1aca0 0x1edc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff37a943740 4 client.6537 mds_command: new command op to 6237 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-08T08:23:02.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff37a943740 1 -- 172.21.15.40:0/738462424 --> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1e3ce00 con 0x1f1aca0 2023-11-08T08:23:02.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff375458700 1 --2- 172.21.15.40:0/738462424 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x1f1aca0 0x1edc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff375458700 1 --2- 172.21.15.40:0/738462424 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x1f1aca0 0x1edc5e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba395da22f8204af server_cookie=c0a835ac4419ddd3 in_seq=0 out_seq=0 2023-11-08T08:23:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.358+0000 7ff34bfff700 10 client.6537 ms_handle_connect on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.362+0000 7ff34bfff700 1 -- 172.21.15.40:0/738462424 <== mds.0 v2:172.21.15.40:6834/3395630689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x1e3ce00 con 0x1f1aca0 2023-11-08T08:23:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.362+0000 7ff34bfff700 10 client.6537 handle_command_reply: tid=0 2023-11-08T08:23:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.362+0000 7ff37a943740 1 client.6537 shutdown 2023-11-08T08:23:02.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.362+0000 7ff37a943740 1 -- 172.21.15.40:0/738462424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6bd80 msgr2=0x1f28340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.362+0000 7ff37a943740 1 --2- 172.21.15.40:0/738462424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6bd80 0x1f28340 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff360009210 tx=0x7ff3600047e0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.362+0000 7ff37a943740 1 -- 172.21.15.40:0/738462424 shutdown_connections 2023-11-08T08:23:02.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.362+0000 7ff37a943740 1 --2- 172.21.15.40:0/738462424 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x1f1aca0 0x1edc5e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.362+0000 7ff37a943740 1 --2- 172.21.15.40:0/738462424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1cb6a30 0x1edc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.362+0000 7ff37a943740 1 --2- 172.21.15.40:0/738462424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6bd80 0x1f28340 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.362+0000 7ff37a943740 1 --2- 172.21.15.40:0/738462424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1c6b9b0 0x1edcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.362+0000 7ff37a943740 1 -- 172.21.15.40:0/738462424 >> 172.21.15.40:0/738462424 conn(0x1c6c150 msgr2=0x1f03be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:02.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.362+0000 7ff34bfff700 0 client.6537 ms_handle_reset on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:02.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.362+0000 7ff37a943740 1 -- 172.21.15.40:0/738462424 shutdown_connections 2023-11-08T08:23:02.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.362+0000 7ff37a943740 1 -- 172.21.15.40:0/738462424 wait complete. 2023-11-08T08:23:02.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.362+0000 7ff37a943740 20 client.6537 trim_cache size 0 max 16384 2023-11-08T08:23:02.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.382+0000 7ff37a943740 1 Processor -- start 2023-11-08T08:23:02.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.382+0000 7ff37a943740 1 -- start start 2023-11-08T08:23:02.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.382+0000 7ff37a943740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1c6b9b0 0x1eec310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.386+0000 7ff37a943740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1c6bd80 0x1eec850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.386+0000 7ff37a943740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1cb6a30 0x1f27c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.386+0000 7ff37a943740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x1de79c0 con 0x1cb6a30 2023-11-08T08:23:02.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.386+0000 7ff37a943740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e481d0 con 0x1c6bd80 2023-11-08T08:23:02.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.386+0000 7ff37a943740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1df5260 con 0x1c6b9b0 2023-11-08T08:23:02.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.386+0000 7ff375c59700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1cb6a30 0x1f27c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.386+0000 7ff375c59700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1cb6a30 0x1f27c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60266/0 (socket says 172.21.15.40:60266) 2023-11-08T08:23:02.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.386+0000 7ff375c59700 1 -- 172.21.15.40:0/557040114 learned_addr learned my addr 172.21.15.40:0/557040114 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:02.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.386+0000 7ff374c57700 1 --2- 172.21.15.40:0/557040114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1c6bd80 0x1eec850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.386+0000 7ff375458700 1 --2- 172.21.15.40:0/557040114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1c6b9b0 0x1eec310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.386+0000 7ff375c59700 1 -- 172.21.15.40:0/557040114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1c6b9b0 msgr2=0x1eec310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:02.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.386+0000 7ff375c59700 1 --2- 172.21.15.40:0/557040114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1c6b9b0 0x1eec310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.386+0000 7ff375c59700 1 -- 172.21.15.40:0/557040114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1c6bd80 msgr2=0x1eec850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:02.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.386+0000 7ff375c59700 1 --2- 172.21.15.40:0/557040114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1c6bd80 0x1eec850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.386+0000 7ff375c59700 1 -- 172.21.15.40:0/557040114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3700adbc0 con 0x1cb6a30 2023-11-08T08:23:02.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.386+0000 7ff375c59700 1 --2- 172.21.15.40:0/557040114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1cb6a30 0x1f27c40 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff368002140 tx=0x7ff368001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a707652e90ecc376 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:02.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.386+0000 7ff3537fe700 1 -- 172.21.15.40:0/557040114 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff368014b00 con 0x1cb6a30 2023-11-08T08:23:02.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.386+0000 7ff3537fe700 1 -- 172.21.15.40:0/557040114 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff368014c80 con 0x1cb6a30 2023-11-08T08:23:02.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.386+0000 7ff3537fe700 1 -- 172.21.15.40:0/557040114 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff368016420 con 0x1cb6a30 2023-11-08T08:23:02.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff37a943740 1 -- 172.21.15.40:0/557040114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1cb6a30 msgr2=0x1f27c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff37a943740 1 --2- 172.21.15.40:0/557040114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1cb6a30 0x1f27c40 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff368002140 tx=0x7ff368001970 comp rx=0 tx=0).stop 2023-11-08T08:23:02.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff37a943740 1 -- 172.21.15.40:0/557040114 shutdown_connections 2023-11-08T08:23:02.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff37a943740 1 --2- 172.21.15.40:0/557040114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1cb6a30 0x1f27c40 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff37a943740 1 --2- 172.21.15.40:0/557040114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1c6bd80 0x1eec850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff37a943740 1 --2- 172.21.15.40:0/557040114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1c6b9b0 0x1eec310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff37a943740 1 -- 172.21.15.40:0/557040114 >> 172.21.15.40:0/557040114 conn(0x1c6c150 msgr2=0x1f1c200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:02.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff37a943740 1 -- 172.21.15.40:0/557040114 shutdown_connections 2023-11-08T08:23:02.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff37a943740 1 -- 172.21.15.40:0/557040114 wait complete. 2023-11-08T08:23:02.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff37a943740 1 -- start start 2023-11-08T08:23:02.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff37a943740 1 Processor -- start 2023-11-08T08:23:02.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff37a943740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1c6b9b0 0x1ed68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff37a943740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6bd80 0x1f03fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff37a943740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1cb6a30 0x1f817c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff37a943740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f1c080 con 0x1c6bd80 2023-11-08T08:23:02.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff37a943740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1edd1f0 con 0x1cb6a30 2023-11-08T08:23:02.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff37a943740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ed6e00 con 0x1c6b9b0 2023-11-08T08:23:02.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff37a943740 10 client.0 _ll_register_callbacks cb 0x1e6a910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:23:02.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff374c57700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6bd80 0x1f03fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff374c57700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6bd80 0x1f03fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60274/0 (socket says 172.21.15.40:60274) 2023-11-08T08:23:02.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.390+0000 7ff374c57700 1 -- 172.21.15.40:0/2798852794 learned_addr learned my addr 172.21.15.40:0/2798852794 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:02.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff375c59700 1 --2- 172.21.15.40:0/2798852794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1cb6a30 0x1f817c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff375458700 1 --2- 172.21.15.40:0/2798852794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1c6b9b0 0x1ed68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff374c57700 1 -- 172.21.15.40:0/2798852794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1c6b9b0 msgr2=0x1ed68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:02.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff374c57700 1 --2- 172.21.15.40:0/2798852794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1c6b9b0 0x1ed68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff374c57700 1 -- 172.21.15.40:0/2798852794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1cb6a30 msgr2=0x1f817c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:02.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff374c57700 1 --2- 172.21.15.40:0/2798852794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1cb6a30 0x1f817c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff374c57700 1 -- 172.21.15.40:0/2798852794 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff35c0035d0 con 0x1c6bd80 2023-11-08T08:23:02.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff375458700 1 --2- 172.21.15.40:0/2798852794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1c6b9b0 0x1ed68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:23:02.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff374c57700 1 --2- 172.21.15.40:0/2798852794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6bd80 0x1f03fe0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff360009850 tx=0x7ff36002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e3d79b8d344e42 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:02.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff34bfff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:23:02.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff34bfff700 1 -- 172.21.15.40:0/2798852794 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360004600 con 0x1c6bd80 2023-11-08T08:23:02.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff37a943740 1 -- 172.21.15.40:0/2798852794 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff360011c00 con 0x1c6bd80 2023-11-08T08:23:02.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff34bfff700 1 -- 172.21.15.40:0/2798852794 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7ff360045030 con 0x1c6bd80 2023-11-08T08:23:02.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff34bfff700 1 -- 172.21.15.40:0/2798852794 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff360003e20 con 0x1c6bd80 2023-11-08T08:23:02.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff37a943740 10 client.6543 fetch_fsmap learned FSMap version 34 2023-11-08T08:23:02.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff37a943740 1 -- 172.21.15.40:0/2798852794 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x1df0bb0 con 0x1c6bd80 2023-11-08T08:23:02.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff34bfff700 1 -- 172.21.15.40:0/2798852794 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff360011c00 con 0x1c6bd80 2023-11-08T08:23:02.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff34bfff700 1 -- 172.21.15.40:0/2798852794 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 34) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff360026410 con 0x1c6bd80 2023-11-08T08:23:02.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff37a943740 10 client.6543 fetch_fsmap finished waiting for FSMap version 34 2023-11-08T08:23:02.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff37a943740 10 client.6543 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-08T08:23:02.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff37a943740 20 client.6543 populate_metadata read hostname 'smithi040' 2023-11-08T08:23:02.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff37a943740 1 --2- 172.21.15.40:0/2798852794 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x1f04dd0 0x1f81d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff37a943740 4 client.6543 mds_command: new command op to 6237 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-11-08T08:23:02.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff37a943740 1 -- 172.21.15.40:0/2798852794 --> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x1ce8500 con 0x1f04dd0 2023-11-08T08:23:02.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.394+0000 7ff375458700 1 --2- 172.21.15.40:0/2798852794 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x1f04dd0 0x1f81d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff375458700 1 --2- 172.21.15.40:0/2798852794 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x1f04dd0 0x1f81d00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a73e7888cfbf69b2 server_cookie=25c20bf336357745 in_seq=0 out_seq=0 2023-11-08T08:23:02.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff34bfff700 10 client.6543 ms_handle_connect on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:02.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff34bfff700 1 -- 172.21.15.40:0/2798852794 <== mds.0 v2:172.21.15.40:6834/3395630689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x1ce8500 con 0x1f04dd0 2023-11-08T08:23:02.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff34bfff700 10 client.6543 handle_command_reply: tid=0 2023-11-08T08:23:02.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37a943740 1 client.6543 shutdown 2023-11-08T08:23:02.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37a943740 1 -- 172.21.15.40:0/2798852794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6bd80 msgr2=0x1f03fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37a943740 1 --2- 172.21.15.40:0/2798852794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6bd80 0x1f03fe0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff360009850 tx=0x7ff36002c000 comp rx=0 tx=0).stop 2023-11-08T08:23:02.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37a943740 1 -- 172.21.15.40:0/2798852794 shutdown_connections 2023-11-08T08:23:02.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37a943740 1 --2- 172.21.15.40:0/2798852794 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x1f04dd0 0x1f81d00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37a943740 1 --2- 172.21.15.40:0/2798852794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1cb6a30 0x1f817c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37a943740 1 --2- 172.21.15.40:0/2798852794 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1c6bd80 0x1f03fe0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37a943740 1 --2- 172.21.15.40:0/2798852794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1c6b9b0 0x1ed68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37a943740 1 -- 172.21.15.40:0/2798852794 >> 172.21.15.40:0/2798852794 conn(0x1c6c150 msgr2=0x1eed320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:02.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff34bfff700 0 client.6543 ms_handle_reset on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:02.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37a943740 1 -- 172.21.15.40:0/2798852794 shutdown_connections 2023-11-08T08:23:02.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37a943740 1 -- 172.21.15.40:0/2798852794 wait complete. 2023-11-08T08:23:02.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37a943740 20 client.6543 trim_cache size 0 max 16384 2023-11-08T08:23:02.411 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:23:02.412 INFO:teuthology.orchestra.run.smithi040.stdout: "return_code": 0, 2023-11-08T08:23:02.412 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_tag": "c0bf6c0f-0848-46f5-a616-87d7f2dcd9e6", 2023-11-08T08:23:02.412 INFO:teuthology.orchestra.run.smithi040.stdout: "mode": "asynchronous" 2023-11-08T08:23:02.412 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:23:02.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37645a700 1 -- 172.21.15.40:0/2760854635 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff354064ee0 msgr2=0x7ff354067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37645a700 1 --2- 172.21.15.40:0/2760854635 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff354064ee0 0x7ff354067390 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff3600090b0 tx=0x7ff360008030 comp rx=0 tx=0).stop 2023-11-08T08:23:02.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37645a700 1 -- 172.21.15.40:0/2760854635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3700cc060 msgr2=0x7ff3700c2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37645a700 1 --2- 172.21.15.40:0/2760854635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3700cc060 0x7ff3700c2ad0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00ea00 tx=0x7ff36c00ed10 comp rx=0 tx=0).stop 2023-11-08T08:23:02.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37645a700 1 -- 172.21.15.40:0/2760854635 shutdown_connections 2023-11-08T08:23:02.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37645a700 1 --2- 172.21.15.40:0/2760854635 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff354064ee0 0x7ff354067390 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37645a700 1 --2- 172.21.15.40:0/2760854635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff370160ba0 0x7ff3700ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37645a700 1 --2- 172.21.15.40:0/2760854635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff37015c5f0 0x7ff3701f5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37645a700 1 --2- 172.21.15.40:0/2760854635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3700cc060 0x7ff3700c2ad0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.398+0000 7ff37645a700 1 -- 172.21.15.40:0/2760854635 >> 172.21.15.40:0/2760854635 conn(0x7ff37006e100 msgr2=0x7ff37006e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:02.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.402+0000 7ff37645a700 1 -- 172.21.15.40:0/2760854635 shutdown_connections 2023-11-08T08:23:02.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.402+0000 7ff37645a700 1 -- 172.21.15.40:0/2760854635 wait complete. 2023-11-08T08:23:02.428 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-08T08:23:02.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.610+0000 7fdd65d84700 1 Processor -- start 2023-11-08T08:23:02.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.610+0000 7fdd65d84700 1 -- start start 2023-11-08T08:23:02.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.614+0000 7fdd65d84700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdd60071aa0 0x7fdd6006d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.614+0000 7fdd65d84700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd6006c600 con 0x7fdd6015c640 2023-11-08T08:23:02.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.614+0000 7fdd65d84700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd600722f0 con 0x7fdd60069e00 2023-11-08T08:23:02.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.614+0000 7fdd65d84700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6006cef0 con 0x7fdd60071aa0 2023-11-08T08:23:02.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.614+0000 7fdd64d82700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdd6015c640 0x7fdd600d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33936/0 (socket says 172.21.15.40:33936) 2023-11-08T08:23:02.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.614+0000 7fdd64d82700 1 -- 172.21.15.40:0/2764046603 learned_addr learned my addr 172.21.15.40:0/2764046603 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:02.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.614+0000 7fdd5ffff700 1 --2- 172.21.15.40:0/2764046603 >> v2:172.21.15.169:3300/0 conn(0x7fdd60071aa0 0x7fdd6006d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.614+0000 7fdd5f7fe700 1 -- 172.21.15.40:0/2764046603 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359985935 0 0) 0x7fdd6006c600 con 0x7fdd6015c640 2023-11-08T08:23:02.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.614+0000 7fdd5f7fe700 1 -- 172.21.15.40:0/2764046603 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd500035d0 con 0x7fdd6015c640 2023-11-08T08:23:02.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.614+0000 7fdd5f7fe700 1 -- 172.21.15.40:0/2764046603 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 275512324 0 0) 0x7fdd600722f0 con 0x7fdd60069e00 2023-11-08T08:23:02.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.614+0000 7fdd5f7fe700 1 -- 172.21.15.40:0/2764046603 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd6006c600 con 0x7fdd60069e00 2023-11-08T08:23:02.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.614+0000 7fdd5ffff700 1 -- 172.21.15.40:0/2764046603 >> v1:172.21.15.169:6789/0 conn(0x7fdd60069e00 legacy=0x7fdd600d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.614+0000 7fdd5f7fe700 1 -- 172.21.15.40:0/2764046603 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2945813540 0 0) 0x7fdd500035d0 con 0x7fdd6015c640 2023-11-08T08:23:02.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.614+0000 7fdd5ffff700 1 -- 172.21.15.40:0/2764046603 >> v1:172.21.15.40:6789/0 conn(0x7fdd6015c640 legacy=0x7fdd600d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.614+0000 7fdd5ffff700 1 -- 172.21.15.40:0/2764046603 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6015ca10 con 0x7fdd60071aa0 2023-11-08T08:23:02.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.614+0000 7fdd5ffff700 1 --2- 172.21.15.40:0/2764046603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd60071aa0 0x7fdd6006d310 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009850 tx=0x7fdd4c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8997f39287ca1ca server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:02.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.618+0000 7fdd5f7fe700 1 -- 172.21.15.40:0/2764046603 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54001670 con 0x7fdd60071aa0 2023-11-08T08:23:02.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.618+0000 7fdd5f7fe700 1 -- 172.21.15.40:0/2764046603 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c004030 con 0x7fdd60071aa0 2023-11-08T08:23:02.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.618+0000 7fdd5f7fe700 1 -- 172.21.15.40:0/2764046603 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c0043c0 con 0x7fdd60071aa0 2023-11-08T08:23:02.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.618+0000 7fdd65d84700 1 -- 172.21.15.40:0/2764046603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd60071aa0 msgr2=0x7fdd6006d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.618+0000 7fdd65d84700 1 --2- 172.21.15.40:0/2764046603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd60071aa0 0x7fdd6006d310 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009850 tx=0x7fdd4c009b60 comp rx=0 tx=0).stop 2023-11-08T08:23:02.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.618+0000 7fdd65d84700 1 -- 172.21.15.40:0/2764046603 shutdown_connections 2023-11-08T08:23:02.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.618+0000 7fdd65d84700 1 --2- 172.21.15.40:0/2764046603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd60071aa0 0x7fdd6006d310 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.618+0000 7fdd65d84700 1 -- 172.21.15.40:0/2764046603 >> 172.21.15.40:0/2764046603 conn(0x7fdd600b6520 msgr2=0x7fdd60071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:02.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.618+0000 7fdd65d84700 1 -- 172.21.15.40:0/2764046603 shutdown_connections 2023-11-08T08:23:02.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.618+0000 7fdd65d84700 1 -- 172.21.15.40:0/2764046603 wait complete. 2023-11-08T08:23:02.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.622+0000 7fdd65d84700 1 Processor -- start 2023-11-08T08:23:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.622+0000 7fdd65d84700 1 -- start start 2023-11-08T08:23:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.622+0000 7fdd65d84700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd60069e00 0x7fdd601f11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.622+0000 7fdd65d84700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd60071aa0 0x7fdd600cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.622+0000 7fdd65d84700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd6015c640 0x7fdd600c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.622+0000 7fdd65d84700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd600d38f0 con 0x7fdd6015c640 2023-11-08T08:23:02.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.622+0000 7fdd65d84700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60154040 con 0x7fdd60071aa0 2023-11-08T08:23:02.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.622+0000 7fdd65d84700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd601f1710 con 0x7fdd60069e00 2023-11-08T08:23:02.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.622+0000 7fdd65583700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd6015c640 0x7fdd600c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.626+0000 7fdd65583700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd6015c640 0x7fdd600c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60284/0 (socket says 172.21.15.40:60284) 2023-11-08T08:23:02.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.626+0000 7fdd65583700 1 -- 172.21.15.40:0/3769018771 learned_addr learned my addr 172.21.15.40:0/3769018771 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:02.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.626+0000 7fdd5ffff700 1 --2- 172.21.15.40:0/3769018771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd60071aa0 0x7fdd600cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.626+0000 7fdd64d82700 1 --2- 172.21.15.40:0/3769018771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd60069e00 0x7fdd601f11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.626+0000 7fdd65583700 1 -- 172.21.15.40:0/3769018771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd60069e00 msgr2=0x7fdd601f11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.626+0000 7fdd65583700 1 --2- 172.21.15.40:0/3769018771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd60069e00 0x7fdd601f11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.626+0000 7fdd65583700 1 -- 172.21.15.40:0/3769018771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd60071aa0 msgr2=0x7fdd600cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.626+0000 7fdd65583700 1 --2- 172.21.15.40:0/3769018771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd60071aa0 0x7fdd600cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.626+0000 7fdd65583700 1 -- 172.21.15.40:0/3769018771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c0094b0 con 0x7fdd6015c640 2023-11-08T08:23:02.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.626+0000 7fdd65583700 1 --2- 172.21.15.40:0/3769018771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd6015c640 0x7fdd600c86b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800cbc0 tx=0x7fdd5800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:02.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.626+0000 7fdd5d7fa700 1 -- 172.21.15.40:0/3769018771 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58003c20 con 0x7fdd6015c640 2023-11-08T08:23:02.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.626+0000 7fdd5d7fa700 1 -- 172.21.15.40:0/3769018771 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd58003d80 con 0x7fdd6015c640 2023-11-08T08:23:02.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.626+0000 7fdd5d7fa700 1 -- 172.21.15.40:0/3769018771 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5800aaa0 con 0x7fdd6015c640 2023-11-08T08:23:02.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.626+0000 7fdd65d84700 1 -- 172.21.15.40:0/3769018771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6006c830 con 0x7fdd6015c640 2023-11-08T08:23:02.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.626+0000 7fdd65d84700 1 -- 172.21.15.40:0/3769018771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd60072010 con 0x7fdd6015c640 2023-11-08T08:23:02.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.630+0000 7fdd5d7fa700 1 -- 172.21.15.40:0/3769018771 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fdd58010050 con 0x7fdd6015c640 2023-11-08T08:23:02.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.634+0000 7fdd5d7fa700 1 --2- 172.21.15.40:0/3769018771 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdd44064ee0 0x7fdd44067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.634+0000 7fdd5d7fa700 1 -- 172.21.15.40:0/3769018771 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fdd58081f70 con 0x7fdd6015c640 2023-11-08T08:23:02.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.634+0000 7fdd64d82700 1 --2- 172.21.15.40:0/3769018771 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdd44064ee0 0x7fdd44067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.634+0000 7fdd6a272740 1 Processor -- start 2023-11-08T08:23:02.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.634+0000 7fdd6a272740 1 -- start start 2023-11-08T08:23:02.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.634+0000 7fdd64d82700 1 --2- 172.21.15.40:0/3769018771 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdd44064ee0 0x7fdd44067390 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdd54001380 tx=0x7fdd54000f10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:02.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.634+0000 7fdd6a272740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1722c10 0x19a27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.634+0000 7fdd6a272740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722840 0x19a2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.634+0000 7fdd6a272740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x176d8c0 0x19a3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.634+0000 7fdd6a272740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x1891d20 con 0x1722840 2023-11-08T08:23:02.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.634+0000 7fdd6a272740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1901d60 con 0x176d8c0 2023-11-08T08:23:02.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.634+0000 7fdd6a272740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x18fb060 con 0x1722c10 2023-11-08T08:23:02.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.634+0000 7fdd5ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722840 0x19a2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.634+0000 7fdd5ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722840 0x19a2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60298/0 (socket says 172.21.15.40:60298) 2023-11-08T08:23:02.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.634+0000 7fdd5ffff700 1 -- 172.21.15.40:0/1002080468 learned_addr learned my addr 172.21.15.40:0/1002080468 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:02.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd64d82700 1 --2- 172.21.15.40:0/1002080468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1722c10 0x19a27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd65583700 1 --2- 172.21.15.40:0/1002080468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x176d8c0 0x19a3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd5ffff700 1 -- 172.21.15.40:0/1002080468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1722c10 msgr2=0x19a27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:02.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd5ffff700 1 --2- 172.21.15.40:0/1002080468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1722c10 0x19a27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd5ffff700 1 -- 172.21.15.40:0/1002080468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x176d8c0 msgr2=0x19a3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:02.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd5ffff700 1 --2- 172.21.15.40:0/1002080468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x176d8c0 0x19a3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd5ffff700 1 -- 172.21.15.40:0/1002080468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x19210a0 con 0x1722840 2023-11-08T08:23:02.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd5ffff700 1 --2- 172.21.15.40:0/1002080468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722840 0x19a2d10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009eb0 tx=0x7fdd4c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8feebb46c515138 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:02.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd4a7fc700 1 -- 172.21.15.40:0/1002080468 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c020070 con 0x1722840 2023-11-08T08:23:02.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd4a7fc700 1 -- 172.21.15.40:0/1002080468 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c0041a0 con 0x1722840 2023-11-08T08:23:02.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd4a7fc700 1 -- 172.21.15.40:0/1002080468 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c0036c0 con 0x1722840 2023-11-08T08:23:02.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd6a272740 1 -- 172.21.15.40:0/1002080468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722840 msgr2=0x19a2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd6a272740 1 --2- 172.21.15.40:0/1002080468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722840 0x19a2d10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009eb0 tx=0x7fdd4c010000 comp rx=0 tx=0).stop 2023-11-08T08:23:02.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd6a272740 1 -- 172.21.15.40:0/1002080468 shutdown_connections 2023-11-08T08:23:02.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd6a272740 1 --2- 172.21.15.40:0/1002080468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x176d8c0 0x19a3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd6a272740 1 --2- 172.21.15.40:0/1002080468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722840 0x19a2d10 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd6a272740 1 --2- 172.21.15.40:0/1002080468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1722c10 0x19a27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd6a272740 1 -- 172.21.15.40:0/1002080468 >> 172.21.15.40:0/1002080468 conn(0x1722fe0 msgr2=0x19bbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:02.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd6a272740 1 -- 172.21.15.40:0/1002080468 shutdown_connections 2023-11-08T08:23:02.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd6a272740 1 -- 172.21.15.40:0/1002080468 wait complete. 2023-11-08T08:23:02.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.638+0000 7fdd6a272740 1 -- start start 2023-11-08T08:23:02.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd6a272740 1 Processor -- start 2023-11-08T08:23:02.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd6a272740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1722840 0x19e2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd6a272740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722c10 0x1992d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd6a272740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x176d8c0 0x1996710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd6a272740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x17a7c50 con 0x1722c10 2023-11-08T08:23:02.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd6a272740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x19e2940 con 0x1722840 2023-11-08T08:23:02.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd6a272740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x19936a0 con 0x176d8c0 2023-11-08T08:23:02.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd6a272740 10 client.0 _ll_register_callbacks cb 0x18f49a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:23:02.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd5ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722c10 0x1992d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd5ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722c10 0x1992d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60300/0 (socket says 172.21.15.40:60300) 2023-11-08T08:23:02.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd5ffff700 1 -- 172.21.15.40:0/665752516 learned_addr learned my addr 172.21.15.40:0/665752516 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:02.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd64d82700 1 --2- 172.21.15.40:0/665752516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1722840 0x19e2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd65583700 1 --2- 172.21.15.40:0/665752516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x176d8c0 0x1996710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd5ffff700 1 -- 172.21.15.40:0/665752516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x176d8c0 msgr2=0x1996710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:02.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd5ffff700 1 --2- 172.21.15.40:0/665752516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x176d8c0 0x1996710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd5ffff700 1 -- 172.21.15.40:0/665752516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1722840 msgr2=0x19e2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:02.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd5ffff700 1 --2- 172.21.15.40:0/665752516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1722840 0x19e2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd5ffff700 1 -- 172.21.15.40:0/665752516 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6015ca10 con 0x1722c10 2023-11-08T08:23:02.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd5ffff700 1 --2- 172.21.15.40:0/665752516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722c10 0x1992d20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x189d250 tx=0x7fdd4c0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a23e6fa6d96229 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:02.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd3f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:23:02.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd3f7fe700 1 -- 172.21.15.40:0/665752516 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c003c50 con 0x1722c10 2023-11-08T08:23:02.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd3f7fe700 1 -- 172.21.15.40:0/665752516 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fdd4c003db0 con 0x1722c10 2023-11-08T08:23:02.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd6a272740 1 -- 172.21.15.40:0/665752516 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x18f40e0 con 0x1722c10 2023-11-08T08:23:02.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd3f7fe700 1 -- 172.21.15.40:0/665752516 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd4c011630 con 0x1722c10 2023-11-08T08:23:02.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd6a272740 10 client.6555 fetch_fsmap learned FSMap version 34 2023-11-08T08:23:02.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.642+0000 7fdd6a272740 1 -- 172.21.15.40:0/665752516 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x176eb10 con 0x1722c10 2023-11-08T08:23:02.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd3f7fe700 1 -- 172.21.15.40:0/665752516 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x18f40e0 con 0x1722c10 2023-11-08T08:23:02.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd3f7fe700 1 -- 172.21.15.40:0/665752516 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 34) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fdd4c018070 con 0x1722c10 2023-11-08T08:23:02.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd6a272740 10 client.6555 fetch_fsmap finished waiting for FSMap version 34 2023-11-08T08:23:02.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd6a272740 10 client.6555 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-08T08:23:02.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd6a272740 20 client.6555 populate_metadata read hostname 'smithi040' 2023-11-08T08:23:02.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd6a272740 1 --2- 172.21.15.40:0/665752516 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x19a4b40 0x1991950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd6a272740 4 client.6555 mds_command: new command op to 6237 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-08T08:23:02.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd6a272740 1 -- 172.21.15.40:0/665752516 --> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x18f3e00 con 0x19a4b40 2023-11-08T08:23:02.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd64d82700 1 --2- 172.21.15.40:0/665752516 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x19a4b40 0x1991950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd64d82700 1 --2- 172.21.15.40:0/665752516 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x19a4b40 0x1991950 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=285fd3a1b4ff7de5 server_cookie=977c9348896520a in_seq=0 out_seq=0 2023-11-08T08:23:02.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd3f7fe700 10 client.6555 ms_handle_connect on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:02.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd3f7fe700 1 -- 172.21.15.40:0/665752516 <== mds.0 v2:172.21.15.40:6834/3395630689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x18f3e00 con 0x19a4b40 2023-11-08T08:23:02.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd3f7fe700 10 client.6555 handle_command_reply: tid=0 2023-11-08T08:23:02.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd6a272740 1 client.6555 shutdown 2023-11-08T08:23:02.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd6a272740 1 -- 172.21.15.40:0/665752516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722c10 msgr2=0x1992d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd6a272740 1 --2- 172.21.15.40:0/665752516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722c10 0x1992d20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x189d250 tx=0x7fdd4c0047e0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd6a272740 1 -- 172.21.15.40:0/665752516 shutdown_connections 2023-11-08T08:23:02.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd6a272740 1 --2- 172.21.15.40:0/665752516 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x19a4b40 0x1991950 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd6a272740 1 --2- 172.21.15.40:0/665752516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x176d8c0 0x1996710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd6a272740 1 --2- 172.21.15.40:0/665752516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722c10 0x1992d20 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd6a272740 1 --2- 172.21.15.40:0/665752516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1722840 0x19e2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd3f7fe700 0 client.6555 ms_handle_reset on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:02.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd6a272740 1 -- 172.21.15.40:0/665752516 >> 172.21.15.40:0/665752516 conn(0x1722fe0 msgr2=0x19a3970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:02.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.646+0000 7fdd6a272740 1 -- 172.21.15.40:0/665752516 shutdown_connections 2023-11-08T08:23:02.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.650+0000 7fdd6a272740 1 -- 172.21.15.40:0/665752516 wait complete. 2023-11-08T08:23:02.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.650+0000 7fdd6a272740 20 client.6555 trim_cache size 0 max 16384 2023-11-08T08:23:02.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.666+0000 7fdd6a272740 1 Processor -- start 2023-11-08T08:23:02.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.670+0000 7fdd6a272740 1 -- start start 2023-11-08T08:23:02.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.670+0000 7fdd6a272740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722fe0 0x19a31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.670+0000 7fdd6a272740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1722840 0x19a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.670+0000 7fdd6a272740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1722c10 0x19bacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.670+0000 7fdd6a272740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x18caba0 con 0x1722fe0 2023-11-08T08:23:02.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.670+0000 7fdd6a272740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x189a510 con 0x1722840 2023-11-08T08:23:02.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.670+0000 7fdd6a272740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x18f8460 con 0x1722c10 2023-11-08T08:23:02.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.670+0000 7fdd64d82700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722fe0 0x19a31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.670+0000 7fdd64d82700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722fe0 0x19a31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60312/0 (socket says 172.21.15.40:60312) 2023-11-08T08:23:02.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.670+0000 7fdd64d82700 1 -- 172.21.15.40:0/4159437094 learned_addr learned my addr 172.21.15.40:0/4159437094 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:02.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.670+0000 7fdd65583700 1 --2- 172.21.15.40:0/4159437094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1722c10 0x19bacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.670+0000 7fdd5ffff700 1 --2- 172.21.15.40:0/4159437094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1722840 0x19a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.670+0000 7fdd64d82700 1 -- 172.21.15.40:0/4159437094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1722c10 msgr2=0x19bacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:02.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.670+0000 7fdd64d82700 1 --2- 172.21.15.40:0/4159437094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1722c10 0x19bacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.670+0000 7fdd64d82700 1 -- 172.21.15.40:0/4159437094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1722840 msgr2=0x19a36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:02.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.670+0000 7fdd64d82700 1 --2- 172.21.15.40:0/4159437094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1722840 0x19a36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.670+0000 7fdd64d82700 1 -- 172.21.15.40:0/4159437094 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd600cdb20 con 0x1722fe0 2023-11-08T08:23:02.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.670+0000 7fdd64d82700 1 --2- 172.21.15.40:0/4159437094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722fe0 0x19a31a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fdd54001e80 tx=0x7fdd54012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29bbd7beafb386ac server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:02.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.670+0000 7fdd4affd700 1 -- 172.21.15.40:0/4159437094 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54029030 con 0x1722fe0 2023-11-08T08:23:02.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.670+0000 7fdd4affd700 1 -- 172.21.15.40:0/4159437094 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd540250b0 con 0x1722fe0 2023-11-08T08:23:02.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.674+0000 7fdd4affd700 1 -- 172.21.15.40:0/4159437094 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54025440 con 0x1722fe0 2023-11-08T08:23:02.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.674+0000 7fdd6a272740 1 -- 172.21.15.40:0/4159437094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722fe0 msgr2=0x19a31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.674+0000 7fdd6a272740 1 --2- 172.21.15.40:0/4159437094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722fe0 0x19a31a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fdd54001e80 tx=0x7fdd54012000 comp rx=0 tx=0).stop 2023-11-08T08:23:02.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.674+0000 7fdd6a272740 1 -- 172.21.15.40:0/4159437094 shutdown_connections 2023-11-08T08:23:02.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.674+0000 7fdd6a272740 1 --2- 172.21.15.40:0/4159437094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1722c10 0x19bacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.674+0000 7fdd6a272740 1 --2- 172.21.15.40:0/4159437094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1722840 0x19a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.674+0000 7fdd6a272740 1 --2- 172.21.15.40:0/4159437094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722fe0 0x19a31a0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.674+0000 7fdd6a272740 1 -- 172.21.15.40:0/4159437094 >> 172.21.15.40:0/4159437094 conn(0x19e2570 msgr2=0x19b73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:02.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.674+0000 7fdd6a272740 1 -- 172.21.15.40:0/4159437094 shutdown_connections 2023-11-08T08:23:02.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.674+0000 7fdd6a272740 1 -- 172.21.15.40:0/4159437094 wait complete. 2023-11-08T08:23:02.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.674+0000 7fdd6a272740 1 -- start start 2023-11-08T08:23:02.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.674+0000 7fdd6a272740 1 Processor -- start 2023-11-08T08:23:02.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd6a272740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722840 0x19ed240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd6a272740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1722c10 0x19efb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd6a272740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1722fe0 0x19f0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd6a272740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x18fdbc0 con 0x1722840 2023-11-08T08:23:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd6a272740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x1891200 con 0x1722c10 2023-11-08T08:23:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd6a272740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x19ebe80 con 0x1722fe0 2023-11-08T08:23:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd6a272740 10 client.0 _ll_register_callbacks cb 0x7fdd50003760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:23:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd64d82700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722840 0x19ed240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd64d82700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722840 0x19ed240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60326/0 (socket says 172.21.15.40:60326) 2023-11-08T08:23:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd64d82700 1 -- 172.21.15.40:0/1110249334 learned_addr learned my addr 172.21.15.40:0/1110249334 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd64d82700 1 -- 172.21.15.40:0/1110249334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1722fe0 msgr2=0x19f0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd64d82700 1 --2- 172.21.15.40:0/1110249334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1722fe0 0x19f0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd64d82700 1 -- 172.21.15.40:0/1110249334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1722c10 msgr2=0x19efb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd64d82700 1 --2- 172.21.15.40:0/1110249334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1722c10 0x19efb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd64d82700 1 -- 172.21.15.40:0/1110249334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5400f930 con 0x1722840 2023-11-08T08:23:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd64d82700 1 --2- 172.21.15.40:0/1110249334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722840 0x19ed240 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fdd54001eb0 tx=0x7fdd54001820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2982dc27f6a861b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd3f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:23:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd3f7fe700 1 -- 172.21.15.40:0/1110249334 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54025ea0 con 0x1722840 2023-11-08T08:23:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd3f7fe700 1 -- 172.21.15.40:0/1110249334 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fdd5402a070 con 0x1722840 2023-11-08T08:23:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd6a272740 1 -- 172.21.15.40:0/1110249334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd4c0041a0 con 0x1722840 2023-11-08T08:23:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd3f7fe700 1 -- 172.21.15.40:0/1110249334 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd54038420 con 0x1722840 2023-11-08T08:23:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd6a272740 10 client.6561 fetch_fsmap learned FSMap version 34 2023-11-08T08:23:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.678+0000 7fdd6a272740 1 -- 172.21.15.40:0/1110249334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x18a7af0 con 0x1722840 2023-11-08T08:23:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd3f7fe700 1 -- 172.21.15.40:0/1110249334 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c0041a0 con 0x1722840 2023-11-08T08:23:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd3f7fe700 1 -- 172.21.15.40:0/1110249334 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 34) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fdd54038580 con 0x1722840 2023-11-08T08:23:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd6a272740 10 client.6561 fetch_fsmap finished waiting for FSMap version 34 2023-11-08T08:23:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd6a272740 10 client.6561 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-08T08:23:02.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd6a272740 20 client.6561 populate_metadata read hostname 'smithi040' 2023-11-08T08:23:02.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd6a272740 1 --2- 172.21.15.40:0/1110249334 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x176d8c0 0x19f0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:02.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd6a272740 4 client.6561 mds_command: new command op to 6237 tid=0[{"prefix": "scrub status"}] 2023-11-08T08:23:02.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd6a272740 1 -- 172.21.15.40:0/1110249334 --> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x179f390 con 0x176d8c0 2023-11-08T08:23:02.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd5ffff700 1 --2- 172.21.15.40:0/1110249334 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x176d8c0 0x19f0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:02.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd3f7fe700 10 client.6561 ms_handle_connect on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:02.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd5ffff700 1 --2- 172.21.15.40:0/1110249334 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x176d8c0 0x19f0ad0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47c7f26210d5868e server_cookie=7dc396dbf0bdf8d7 in_seq=0 out_seq=0 2023-11-08T08:23:02.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd3f7fe700 1 -- 172.21.15.40:0/1110249334 <== mds.0 v2:172.21.15.40:6834/3395630689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x179f390 con 0x176d8c0 2023-11-08T08:23:02.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd3f7fe700 10 client.6561 handle_command_reply: tid=0 2023-11-08T08:23:02.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd6a272740 1 client.6561 shutdown 2023-11-08T08:23:02.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd6a272740 1 -- 172.21.15.40:0/1110249334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722840 msgr2=0x19ed240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd6a272740 1 --2- 172.21.15.40:0/1110249334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722840 0x19ed240 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fdd54001eb0 tx=0x7fdd54001820 comp rx=0 tx=0).stop 2023-11-08T08:23:02.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd6a272740 1 -- 172.21.15.40:0/1110249334 shutdown_connections 2023-11-08T08:23:02.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd6a272740 1 --2- 172.21.15.40:0/1110249334 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x176d8c0 0x19f0ad0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd6a272740 1 --2- 172.21.15.40:0/1110249334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x1722fe0 0x19f0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd3f7fe700 0 client.6561 ms_handle_reset on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:02.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd6a272740 1 --2- 172.21.15.40:0/1110249334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x1722c10 0x19efb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd6a272740 1 --2- 172.21.15.40:0/1110249334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1722840 0x19ed240 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd6a272740 1 -- 172.21.15.40:0/1110249334 >> 172.21.15.40:0/1110249334 conn(0x19e2570 msgr2=0x19a3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:02.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd6a272740 1 -- 172.21.15.40:0/1110249334 shutdown_connections 2023-11-08T08:23:02.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd6a272740 1 -- 172.21.15.40:0/1110249334 wait complete. 2023-11-08T08:23:02.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.682+0000 7fdd6a272740 20 client.6561 trim_cache size 0 max 16384 2023-11-08T08:23:02.698 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:23:02.698 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-08T08:23:02.699 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-11-08T08:23:02.699 INFO:teuthology.orchestra.run.smithi040.stdout: "c0bf6c0f-0848-46f5-a616-87d7f2dcd9e6": { 2023-11-08T08:23:02.699 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-11-08T08:23:02.699 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "c0bf6c0f-0848-46f5-a616-87d7f2dcd9e6", 2023-11-08T08:23:02.699 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,repair" 2023-11-08T08:23:02.700 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:23:02.700 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:23:02.700 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:23:02.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.686+0000 7fdd65d84700 1 -- 172.21.15.40:0/3769018771 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdd44064ee0 msgr2=0x7fdd44067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.686+0000 7fdd65d84700 1 --2- 172.21.15.40:0/3769018771 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdd44064ee0 0x7fdd44067390 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdd54001380 tx=0x7fdd54000f10 comp rx=0 tx=0).stop 2023-11-08T08:23:02.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.686+0000 7fdd65d84700 1 -- 172.21.15.40:0/3769018771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd6015c640 msgr2=0x7fdd600c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:02.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.686+0000 7fdd65d84700 1 --2- 172.21.15.40:0/3769018771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd6015c640 0x7fdd600c86b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800cbc0 tx=0x7fdd5800ced0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.686+0000 7fdd65d84700 1 -- 172.21.15.40:0/3769018771 shutdown_connections 2023-11-08T08:23:02.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.686+0000 7fdd65d84700 1 --2- 172.21.15.40:0/3769018771 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fdd44064ee0 0x7fdd44067390 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.686+0000 7fdd65d84700 1 --2- 172.21.15.40:0/3769018771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd6015c640 0x7fdd600c86b0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.686+0000 7fdd65d84700 1 --2- 172.21.15.40:0/3769018771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd60071aa0 0x7fdd600cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.686+0000 7fdd65d84700 1 --2- 172.21.15.40:0/3769018771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd60069e00 0x7fdd601f11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:02.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.686+0000 7fdd65d84700 1 -- 172.21.15.40:0/3769018771 >> 172.21.15.40:0/3769018771 conn(0x7fdd600b6520 msgr2=0x7fdd600b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:02.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.686+0000 7fdd65d84700 1 -- 172.21.15.40:0/3769018771 shutdown_connections 2023-11-08T08:23:02.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:02.686+0000 7fdd65d84700 1 -- 172.21.15.40:0/3769018771 wait complete. 2023-11-08T08:23:02.718 INFO:tasks.cephfs.filesystem:scrub status for tag:c0bf6c0f-0848-46f5-a616-87d7f2dcd9e6 - {'path': '/', 'tag': 'c0bf6c0f-0848-46f5-a616-87d7f2dcd9e6', 'options': 'recursive,repair'} 2023-11-08T08:23:27.037 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:23:27.044 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:23:32.720 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-08T08:23:32.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.897+0000 7f782f59e700 1 Processor -- start 2023-11-08T08:23:32.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.897+0000 7f782f59e700 1 -- start start 2023-11-08T08:23:32.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.901+0000 7f782f59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f783015c640 0x7f78300d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:32.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.901+0000 7f782f59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7830069e00 0x7f783006a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:32.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.901+0000 7f782f59e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783006c600 con 0x7f7830071aa0 2023-11-08T08:23:32.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.901+0000 7f782f59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f783006cef0 con 0x7f7830069e00 2023-11-08T08:23:32.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.901+0000 7f782f59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f783006a7c0 con 0x7f783015c640 2023-11-08T08:23:32.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.901+0000 7f782ed9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7830069e00 0x7f783006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:32.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.901+0000 7f782e59c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f783015c640 0x7f78300d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:32.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.901+0000 7f782e59c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f783015c640 0x7f78300d31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:49024/0 (socket says 172.21.15.40:49024) 2023-11-08T08:23:32.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.901+0000 7f782e59c700 1 -- 172.21.15.40:0/1627950054 learned_addr learned my addr 172.21.15.40:0/1627950054 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:32.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.901+0000 7f782d59a700 1 -- 172.21.15.40:0/1627950054 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3809631025 0 0) 0x7f783006c600 con 0x7f7830071aa0 2023-11-08T08:23:32.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.905+0000 7f782d59a700 1 -- 172.21.15.40:0/1627950054 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78180035d0 con 0x7f7830071aa0 2023-11-08T08:23:32.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.905+0000 7f782e59c700 1 -- 172.21.15.40:0/1627950054 >> v2:172.21.15.40:3300/0 conn(0x7f7830069e00 msgr2=0x7f783006a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:32.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.905+0000 7f782e59c700 1 --2- 172.21.15.40:0/1627950054 >> v2:172.21.15.40:3300/0 conn(0x7f7830069e00 0x7f783006a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.905+0000 7f782e59c700 1 -- 172.21.15.40:0/1627950054 >> v1:172.21.15.169:6789/0 conn(0x7f7830071aa0 legacy=0x7f783006d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:32.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.905+0000 7f782e59c700 1 -- 172.21.15.40:0/1627950054 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78300722f0 con 0x7f783015c640 2023-11-08T08:23:32.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.905+0000 7f782ed9d700 1 --2- 172.21.15.40:0/1627950054 >> v2:172.21.15.40:3300/0 conn(0x7f7830069e00 0x7f783006a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:23:32.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.905+0000 7f782e59c700 1 --2- 172.21.15.40:0/1627950054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f783015c640 0x7f78300d31e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7820009910 tx=0x7f7820009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=506b4f8fa8bcc711 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:32.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.905+0000 7f782d59a700 1 -- 172.21.15.40:0/1627950054 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820011830 con 0x7f783015c640 2023-11-08T08:23:32.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.905+0000 7f782d59a700 1 -- 172.21.15.40:0/1627950054 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7820011990 con 0x7f783015c640 2023-11-08T08:23:32.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.905+0000 7f782d59a700 1 -- 172.21.15.40:0/1627950054 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820011d40 con 0x7f783015c640 2023-11-08T08:23:32.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.905+0000 7f782f59e700 1 -- 172.21.15.40:0/1627950054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f783015c640 msgr2=0x7f78300d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:32.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.905+0000 7f782f59e700 1 --2- 172.21.15.40:0/1627950054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f783015c640 0x7f78300d31e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7820009910 tx=0x7f7820009c20 comp rx=0 tx=0).stop 2023-11-08T08:23:32.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.905+0000 7f782f59e700 1 -- 172.21.15.40:0/1627950054 shutdown_connections 2023-11-08T08:23:32.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.905+0000 7f782f59e700 1 --2- 172.21.15.40:0/1627950054 >> v2:172.21.15.40:3300/0 conn(0x7f7830069e00 0x7f783006a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.905+0000 7f782f59e700 1 --2- 172.21.15.40:0/1627950054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f783015c640 0x7f78300d31e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.905+0000 7f782f59e700 1 -- 172.21.15.40:0/1627950054 >> 172.21.15.40:0/1627950054 conn(0x7f78300b6520 msgr2=0x7f7830071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:32.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.905+0000 7f782f59e700 1 -- 172.21.15.40:0/1627950054 shutdown_connections 2023-11-08T08:23:32.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.905+0000 7f782f59e700 1 -- 172.21.15.40:0/1627950054 wait complete. 2023-11-08T08:23:32.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.909+0000 7f782f59e700 1 Processor -- start 2023-11-08T08:23:32.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.909+0000 7f782f59e700 1 -- start start 2023-11-08T08:23:32.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.909+0000 7f782f59e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7830069e00 0x7f78300c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:32.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.909+0000 7f782f59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7830071aa0 0x7f78301f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:32.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.909+0000 7f782f59e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f783015c640 0x7f78300b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:32.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.909+0000 7f782f59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78301f8f30 con 0x7f7830071aa0 2023-11-08T08:23:32.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.909+0000 7f782f59e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78300cae20 con 0x7f783015c640 2023-11-08T08:23:32.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.909+0000 7f782f59e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78300cb0c0 con 0x7f7830069e00 2023-11-08T08:23:32.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.909+0000 7f782dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7830071aa0 0x7f78301f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:32.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.909+0000 7f782dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7830071aa0 0x7f78301f89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37428/0 (socket says 172.21.15.40:37428) 2023-11-08T08:23:32.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.909+0000 7f782e59c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7830069e00 0x7f78300c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:32.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.909+0000 7f782dd9b700 1 -- 172.21.15.40:0/1628722894 learned_addr learned my addr 172.21.15.40:0/1628722894 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:32.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.913+0000 7f782ed9d700 1 --2- 172.21.15.40:0/1628722894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f783015c640 0x7f78300b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:32.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.913+0000 7f782e59c700 1 -- 172.21.15.40:0/1628722894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f783015c640 msgr2=0x7f78300b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:32.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.913+0000 7f782e59c700 1 --2- 172.21.15.40:0/1628722894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f783015c640 0x7f78300b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.913+0000 7f782e59c700 1 -- 172.21.15.40:0/1628722894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7830071aa0 msgr2=0x7f78301f89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:32.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.913+0000 7f782e59c700 1 --2- 172.21.15.40:0/1628722894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7830071aa0 0x7f78301f89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.913+0000 7f782e59c700 1 -- 172.21.15.40:0/1628722894 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7828009a70 con 0x7f7830069e00 2023-11-08T08:23:32.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.913+0000 7f782e59c700 1 --2- 172.21.15.40:0/1628722894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7830069e00 0x7f78300c8820 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7820016000 tx=0x7f7820004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:32.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.913+0000 7f781effd700 1 -- 172.21.15.40:0/1628722894 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782001a8f0 con 0x7f7830069e00 2023-11-08T08:23:32.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.913+0000 7f781effd700 1 -- 172.21.15.40:0/1628722894 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782001e070 con 0x7f7830069e00 2023-11-08T08:23:32.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.913+0000 7f782f59e700 1 -- 172.21.15.40:0/1628722894 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7820009570 con 0x7f7830069e00 2023-11-08T08:23:32.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.913+0000 7f781effd700 1 -- 172.21.15.40:0/1628722894 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782001acb0 con 0x7f7830069e00 2023-11-08T08:23:32.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.913+0000 7f782f59e700 1 -- 172.21.15.40:0/1628722894 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f783015ca10 con 0x7f7830069e00 2023-11-08T08:23:32.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.917+0000 7f781effd700 1 -- 172.21.15.40:0/1628722894 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f782000f050 con 0x7f7830069e00 2023-11-08T08:23:32.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.917+0000 7f781effd700 1 --2- 172.21.15.40:0/1628722894 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7810064f30 0x7f78100673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:32.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.917+0000 7f781effd700 1 -- 172.21.15.40:0/1628722894 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f7820082f10 con 0x7f7830069e00 2023-11-08T08:23:32.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f782dd9b700 1 --2- 172.21.15.40:0/1628722894 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7810064f30 0x7f78100673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:32.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f7837cfd740 1 Processor -- start 2023-11-08T08:23:32.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f782dd9b700 1 --2- 172.21.15.40:0/1628722894 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7810064f30 0x7f78100673e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f78300b2130 tx=0x7f7824009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:32.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f7837cfd740 1 -- start start 2023-11-08T08:23:32.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f7837cfd740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x298bc10 0x2c0b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:32.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f7837cfd740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x298b840 0x2c0bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:32.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f7837cfd740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x29d68c0 0x2c0c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:32.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f7837cfd740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x2afad20 con 0x29d68c0 2023-11-08T08:23:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f7837cfd740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b6ad60 con 0x298b840 2023-11-08T08:23:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f7837cfd740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x2b64060 con 0x298bc10 2023-11-08T08:23:32.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f782ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x29d68c0 0x2c0c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:32.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f782ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x29d68c0 0x2c0c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37438/0 (socket says 172.21.15.40:37438) 2023-11-08T08:23:32.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f782ed9d700 1 -- 172.21.15.40:0/1445725893 learned_addr learned my addr 172.21.15.40:0/1445725893 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:32.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f782e59c700 1 --2- 172.21.15.40:0/1445725893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x298bc10 0x2c0b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:32.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f782dd9b700 1 --2- 172.21.15.40:0/1445725893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x298b840 0x2c0bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:32.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f782ed9d700 1 -- 172.21.15.40:0/1445725893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x298bc10 msgr2=0x2c0b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:32.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f782ed9d700 1 --2- 172.21.15.40:0/1445725893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x298bc10 0x2c0b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f782ed9d700 1 -- 172.21.15.40:0/1445725893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x298b840 msgr2=0x2c0bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:32.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f782ed9d700 1 --2- 172.21.15.40:0/1445725893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x298b840 0x2c0bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f782ed9d700 1 -- 172.21.15.40:0/1445725893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2b8a0a0 con 0x29d68c0 2023-11-08T08:23:32.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f782dd9b700 1 --2- 172.21.15.40:0/1445725893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x298b840 0x2c0bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:23:32.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.921+0000 7f782ed9d700 1 --2- 172.21.15.40:0/1445725893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x29d68c0 0x2c0c250 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f78280076a0 tx=0x7f7828009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b9e369251bc88c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:32.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.925+0000 7f780ffff700 1 -- 172.21.15.40:0/1445725893 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782800e070 con 0x29d68c0 2023-11-08T08:23:32.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.925+0000 7f780ffff700 1 -- 172.21.15.40:0/1445725893 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782800f040 con 0x29d68c0 2023-11-08T08:23:32.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.925+0000 7f780ffff700 1 -- 172.21.15.40:0/1445725893 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828015a70 con 0x29d68c0 2023-11-08T08:23:32.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.925+0000 7f7837cfd740 1 -- 172.21.15.40:0/1445725893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x29d68c0 msgr2=0x2c0c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:32.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.925+0000 7f7837cfd740 1 --2- 172.21.15.40:0/1445725893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x29d68c0 0x2c0c250 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f78280076a0 tx=0x7f7828009750 comp rx=0 tx=0).stop 2023-11-08T08:23:32.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.925+0000 7f7837cfd740 1 -- 172.21.15.40:0/1445725893 shutdown_connections 2023-11-08T08:23:32.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.925+0000 7f7837cfd740 1 --2- 172.21.15.40:0/1445725893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x29d68c0 0x2c0c250 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.925+0000 7f7837cfd740 1 --2- 172.21.15.40:0/1445725893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x298b840 0x2c0bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.925+0000 7f7837cfd740 1 --2- 172.21.15.40:0/1445725893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x298bc10 0x2c0b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.925+0000 7f7837cfd740 1 -- 172.21.15.40:0/1445725893 >> 172.21.15.40:0/1445725893 conn(0x298bfe0 msgr2=0x2c24b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.925+0000 7f7837cfd740 1 -- 172.21.15.40:0/1445725893 shutdown_connections 2023-11-08T08:23:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.925+0000 7f7837cfd740 1 -- 172.21.15.40:0/1445725893 wait complete. 2023-11-08T08:23:32.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.925+0000 7f7837cfd740 1 -- start start 2023-11-08T08:23:32.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.925+0000 7f7837cfd740 1 Processor -- start 2023-11-08T08:23:32.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f7837cfd740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298b840 0x2c481c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:32.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f7837cfd740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x298bc10 0x2bfc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:32.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f7837cfd740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x29d68c0 0x2bfce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:32.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f7837cfd740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x2a10c50 con 0x298b840 2023-11-08T08:23:32.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f7837cfd740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x2bfb880 con 0x29d68c0 2023-11-08T08:23:32.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f7837cfd740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x2bfbb20 con 0x298bc10 2023-11-08T08:23:32.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f7837cfd740 10 client.0 _ll_register_callbacks cb 0x2b5d9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:23:32.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f782e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298b840 0x2c481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:32.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f782e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298b840 0x2c481c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37448/0 (socket says 172.21.15.40:37448) 2023-11-08T08:23:32.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f782e59c700 1 -- 172.21.15.40:0/166092387 learned_addr learned my addr 172.21.15.40:0/166092387 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:32.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f782dd9b700 1 --2- 172.21.15.40:0/166092387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x298bc10 0x2bfc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:32.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f782ed9d700 1 --2- 172.21.15.40:0/166092387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x29d68c0 0x2bfce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:32.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f782e59c700 1 -- 172.21.15.40:0/166092387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x298bc10 msgr2=0x2bfc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:32.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f782e59c700 1 --2- 172.21.15.40:0/166092387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x298bc10 0x2bfc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f782e59c700 1 -- 172.21.15.40:0/166092387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x29d68c0 msgr2=0x2bfce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:32.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f782e59c700 1 --2- 172.21.15.40:0/166092387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x29d68c0 0x2bfce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f782e59c700 1 -- 172.21.15.40:0/166092387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7828007350 con 0x298b840 2023-11-08T08:23:32.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f782e59c700 1 --2- 172.21.15.40:0/166092387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298b840 0x2c481c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7820009f70 tx=0x7f782001aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6eec720f105810 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:32.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f780d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:23:32.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f780d7fa700 1 -- 172.21.15.40:0/166092387 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820054270 con 0x298b840 2023-11-08T08:23:32.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f780d7fa700 1 -- 172.21.15.40:0/166092387 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f782005c780 con 0x298b840 2023-11-08T08:23:32.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f7837cfd740 1 -- 172.21.15.40:0/166092387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2b5d0e0 con 0x298b840 2023-11-08T08:23:32.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f780d7fa700 1 -- 172.21.15.40:0/166092387 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7820081080 con 0x298b840 2023-11-08T08:23:32.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f7837cfd740 10 client.6573 fetch_fsmap learned FSMap version 34 2023-11-08T08:23:32.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f7837cfd740 1 -- 172.21.15.40:0/166092387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x29d7b10 con 0x298b840 2023-11-08T08:23:32.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f780d7fa700 1 -- 172.21.15.40:0/166092387 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2b5d0e0 con 0x298b840 2023-11-08T08:23:32.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f780d7fa700 1 -- 172.21.15.40:0/166092387 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 34) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f7820058ae0 con 0x298b840 2023-11-08T08:23:32.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f7837cfd740 10 client.6573 fetch_fsmap finished waiting for FSMap version 34 2023-11-08T08:23:32.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f7837cfd740 10 client.6573 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-08T08:23:32.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f7837cfd740 20 client.6573 populate_metadata read hostname 'smithi040' 2023-11-08T08:23:32.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f7837cfd740 1 --2- 172.21.15.40:0/166092387 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x2bfd3a0 0x2bf9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:32.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f7837cfd740 4 client.6573 mds_command: new command op to 6237 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-08T08:23:32.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f7837cfd740 1 -- 172.21.15.40:0/166092387 --> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x2b5ce00 con 0x2bfd3a0 2023-11-08T08:23:32.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.929+0000 7f782dd9b700 1 --2- 172.21.15.40:0/166092387 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x2bfd3a0 0x2bf9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:32.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.933+0000 7f782dd9b700 1 --2- 172.21.15.40:0/166092387 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x2bfd3a0 0x2bf9cc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2c99a5158421b3c server_cookie=2c3d0936855c3dee in_seq=0 out_seq=0 2023-11-08T08:23:32.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.933+0000 7f780d7fa700 10 client.6573 ms_handle_connect on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:32.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.933+0000 7f780d7fa700 1 -- 172.21.15.40:0/166092387 <== mds.0 v2:172.21.15.40:6834/3395630689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x2b5ce00 con 0x2bfd3a0 2023-11-08T08:23:32.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.933+0000 7f780d7fa700 10 client.6573 handle_command_reply: tid=0 2023-11-08T08:23:32.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.933+0000 7f7837cfd740 1 client.6573 shutdown 2023-11-08T08:23:32.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.933+0000 7f7837cfd740 1 -- 172.21.15.40:0/166092387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298b840 msgr2=0x2c481c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:32.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.933+0000 7f7837cfd740 1 --2- 172.21.15.40:0/166092387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298b840 0x2c481c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7820009f70 tx=0x7f782001aee0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.933+0000 7f7837cfd740 1 -- 172.21.15.40:0/166092387 shutdown_connections 2023-11-08T08:23:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.933+0000 7f7837cfd740 1 --2- 172.21.15.40:0/166092387 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x2bfd3a0 0x2bf9cc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.933+0000 7f7837cfd740 1 --2- 172.21.15.40:0/166092387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x29d68c0 0x2bfce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.933+0000 7f7837cfd740 1 --2- 172.21.15.40:0/166092387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x298bc10 0x2bfc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.933+0000 7f7837cfd740 1 --2- 172.21.15.40:0/166092387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298b840 0x2c481c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.933+0000 7f7837cfd740 1 -- 172.21.15.40:0/166092387 >> 172.21.15.40:0/166092387 conn(0x298bfe0 msgr2=0x2c0c970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:32.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.933+0000 7f780d7fa700 0 client.6573 ms_handle_reset on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:32.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.933+0000 7f7837cfd740 1 -- 172.21.15.40:0/166092387 shutdown_connections 2023-11-08T08:23:32.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.933+0000 7f7837cfd740 1 -- 172.21.15.40:0/166092387 wait complete. 2023-11-08T08:23:32.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.933+0000 7f7837cfd740 20 client.6573 trim_cache size 0 max 16384 2023-11-08T08:23:32.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.953+0000 7f7837cfd740 1 Processor -- start 2023-11-08T08:23:32.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.953+0000 7f7837cfd740 1 -- start start 2023-11-08T08:23:32.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.953+0000 7f7837cfd740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x298b840 0x2c4ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:32.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.953+0000 7f7837cfd740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298bc10 0x2c4b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:32.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.953+0000 7f7837cfd740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x29d68c0 0x2c203d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:32.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.953+0000 7f7837cfd740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x2bff480 con 0x298bc10 2023-11-08T08:23:32.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.953+0000 7f7837cfd740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b62ef0 con 0x298b840 2023-11-08T08:23:32.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.953+0000 7f7837cfd740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x2b66bc0 con 0x29d68c0 2023-11-08T08:23:32.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f782dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298bc10 0x2c4b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:32.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f782dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298bc10 0x2c4b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37460/0 (socket says 172.21.15.40:37460) 2023-11-08T08:23:32.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f782dd9b700 1 -- 172.21.15.40:0/3946370198 learned_addr learned my addr 172.21.15.40:0/3946370198 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:32.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f782e59c700 1 --2- 172.21.15.40:0/3946370198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x298b840 0x2c4ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:32.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f782dd9b700 1 -- 172.21.15.40:0/3946370198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x29d68c0 msgr2=0x2c203d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:32.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f782ed9d700 1 --2- 172.21.15.40:0/3946370198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x29d68c0 0x2c203d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:32.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f782dd9b700 1 --2- 172.21.15.40:0/3946370198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x29d68c0 0x2c203d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f782dd9b700 1 -- 172.21.15.40:0/3946370198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x298b840 msgr2=0x2c4ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:32.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f782dd9b700 1 --2- 172.21.15.40:0/3946370198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x298b840 0x2c4ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f782dd9b700 1 -- 172.21.15.40:0/3946370198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78300b1e50 con 0x298bc10 2023-11-08T08:23:32.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f782e59c700 1 --2- 172.21.15.40:0/3946370198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x298b840 0x2c4ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:23:32.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f782ed9d700 1 --2- 172.21.15.40:0/3946370198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x29d68c0 0x2c203d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:23:32.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f782dd9b700 1 --2- 172.21.15.40:0/3946370198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298bc10 0x2c4b3a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f78240011e0 tx=0x7f7824012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7221c425c523e24d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:32.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f781cff9700 1 -- 172.21.15.40:0/3946370198 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7824027070 con 0x298bc10 2023-11-08T08:23:32.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f781cff9700 1 -- 172.21.15.40:0/3946370198 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782400fe90 con 0x298bc10 2023-11-08T08:23:32.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f781cff9700 1 -- 172.21.15.40:0/3946370198 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7824014650 con 0x298bc10 2023-11-08T08:23:32.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f7837cfd740 1 -- 172.21.15.40:0/3946370198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298bc10 msgr2=0x2c4b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:32.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f7837cfd740 1 --2- 172.21.15.40:0/3946370198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298bc10 0x2c4b3a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f78240011e0 tx=0x7f7824012000 comp rx=0 tx=0).stop 2023-11-08T08:23:32.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f7837cfd740 1 -- 172.21.15.40:0/3946370198 shutdown_connections 2023-11-08T08:23:32.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f7837cfd740 1 --2- 172.21.15.40:0/3946370198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x29d68c0 0x2c203d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f7837cfd740 1 --2- 172.21.15.40:0/3946370198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298bc10 0x2c4b3a0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f7837cfd740 1 --2- 172.21.15.40:0/3946370198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x298b840 0x2c4ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f7837cfd740 1 -- 172.21.15.40:0/3946370198 >> 172.21.15.40:0/3946370198 conn(0x298bfe0 msgr2=0x2c526e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:32.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f7837cfd740 1 -- 172.21.15.40:0/3946370198 shutdown_connections 2023-11-08T08:23:32.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.957+0000 7f7837cfd740 1 -- 172.21.15.40:0/3946370198 wait complete. 2023-11-08T08:23:32.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.961+0000 7f7837cfd740 1 -- start start 2023-11-08T08:23:32.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.961+0000 7f7837cfd740 1 Processor -- start 2023-11-08T08:23:32.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.961+0000 7f7837cfd740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298b840 0x2c58be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:32.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.961+0000 7f7837cfd740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x298bc10 0x2bfcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:32.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.961+0000 7f7837cfd740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x29d68c0 0x2c9ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:32.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.961+0000 7f7837cfd740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b33ba0 con 0x298b840 2023-11-08T08:23:32.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.961+0000 7f7837cfd740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x2b03510 con 0x298bc10 2023-11-08T08:23:32.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.961+0000 7f7837cfd740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x2b61460 con 0x29d68c0 2023-11-08T08:23:32.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.961+0000 7f7837cfd740 10 client.0 _ll_register_callbacks cb 0x7f7828009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:23:32.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.961+0000 7f782e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298b840 0x2c58be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:32.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.961+0000 7f782e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298b840 0x2c58be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37466/0 (socket says 172.21.15.40:37466) 2023-11-08T08:23:32.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.961+0000 7f782e59c700 1 -- 172.21.15.40:0/1279628703 learned_addr learned my addr 172.21.15.40:0/1279628703 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:32.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.961+0000 7f782ed9d700 1 --2- 172.21.15.40:0/1279628703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x29d68c0 0x2c9ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:32.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.961+0000 7f782dd9b700 1 --2- 172.21.15.40:0/1279628703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x298bc10 0x2bfcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:32.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f782e59c700 1 -- 172.21.15.40:0/1279628703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x29d68c0 msgr2=0x2c9ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:32.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f782e59c700 1 --2- 172.21.15.40:0/1279628703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x29d68c0 0x2c9ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f782e59c700 1 -- 172.21.15.40:0/1279628703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x298bc10 msgr2=0x2bfcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:32.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f782e59c700 1 --2- 172.21.15.40:0/1279628703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x298bc10 0x2bfcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f782e59c700 1 -- 172.21.15.40:0/1279628703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7824019040 con 0x298b840 2023-11-08T08:23:32.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f782ed9d700 1 --2- 172.21.15.40:0/1279628703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x29d68c0 0x2c9ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:23:32.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f782e59c700 1 --2- 172.21.15.40:0/1279628703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298b840 0x2c58be0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f782005c6e0 tx=0x7f7820009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996c42543f3ed2d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:32.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f780d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:23:32.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f780d7fa700 1 -- 172.21.15.40:0/1279628703 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782005c780 con 0x298b840 2023-11-08T08:23:32.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f780d7fa700 1 -- 172.21.15.40:0/1279628703 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f782005c8e0 con 0x298b840 2023-11-08T08:23:32.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f7837cfd740 1 -- 172.21.15.40:0/1279628703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f782800f040 con 0x298b840 2023-11-08T08:23:32.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f780d7fa700 1 -- 172.21.15.40:0/1279628703 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78200680a0 con 0x298b840 2023-11-08T08:23:32.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f7837cfd740 10 client.6579 fetch_fsmap learned FSMap version 34 2023-11-08T08:23:32.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f7837cfd740 1 -- 172.21.15.40:0/1279628703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x2b10af0 con 0x298b840 2023-11-08T08:23:32.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f780d7fa700 1 -- 172.21.15.40:0/1279628703 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782800f040 con 0x298b840 2023-11-08T08:23:32.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f780d7fa700 1 -- 172.21.15.40:0/1279628703 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 34) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f7820050970 con 0x298b840 2023-11-08T08:23:32.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f7837cfd740 10 client.6579 fetch_fsmap finished waiting for FSMap version 34 2023-11-08T08:23:32.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f7837cfd740 10 client.6579 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2023-11-08T08:23:32.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f7837cfd740 20 client.6579 populate_metadata read hostname 'smithi040' 2023-11-08T08:23:32.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f7837cfd740 1 --2- 172.21.15.40:0/1279628703 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x2c21320 0x2c9f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:32.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f7837cfd740 4 client.6579 mds_command: new command op to 6237 tid=0[{"prefix": "scrub status"}] 2023-11-08T08:23:32.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f7837cfd740 1 -- 172.21.15.40:0/1279628703 --> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x2a08390 con 0x2c21320 2023-11-08T08:23:32.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f782dd9b700 1 --2- 172.21.15.40:0/1279628703 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x2c21320 0x2c9f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:32.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f782dd9b700 1 --2- 172.21.15.40:0/1279628703 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x2c21320 0x2c9f2c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b5c3c317b79154d server_cookie=ca07dbeeebd43fcf in_seq=0 out_seq=0 2023-11-08T08:23:32.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.965+0000 7f780d7fa700 10 client.6579 ms_handle_connect on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:32.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f780d7fa700 1 -- 172.21.15.40:0/1279628703 <== mds.0 v2:172.21.15.40:6834/3395630689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x2a08390 con 0x2c21320 2023-11-08T08:23:32.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f780d7fa700 10 client.6579 handle_command_reply: tid=0 2023-11-08T08:23:32.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f7837cfd740 1 client.6579 shutdown 2023-11-08T08:23:32.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f7837cfd740 1 -- 172.21.15.40:0/1279628703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298b840 msgr2=0x2c58be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:32.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f7837cfd740 1 --2- 172.21.15.40:0/1279628703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298b840 0x2c58be0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f782005c6e0 tx=0x7f7820009c20 comp rx=0 tx=0).stop 2023-11-08T08:23:32.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f7837cfd740 1 -- 172.21.15.40:0/1279628703 shutdown_connections 2023-11-08T08:23:32.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f7837cfd740 1 --2- 172.21.15.40:0/1279628703 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x2c21320 0x2c9f2c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f7837cfd740 1 --2- 172.21.15.40:0/1279628703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x29d68c0 0x2c9ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f7837cfd740 1 --2- 172.21.15.40:0/1279628703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x298bc10 0x2bfcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f7837cfd740 1 --2- 172.21.15.40:0/1279628703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x298b840 0x2c58be0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f780d7fa700 0 client.6579 ms_handle_reset on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:32.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f7837cfd740 1 -- 172.21.15.40:0/1279628703 >> 172.21.15.40:0/1279628703 conn(0x298bfe0 msgr2=0x2c203d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:32.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f7837cfd740 1 -- 172.21.15.40:0/1279628703 shutdown_connections 2023-11-08T08:23:32.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f7837cfd740 1 -- 172.21.15.40:0/1279628703 wait complete. 2023-11-08T08:23:32.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f7837cfd740 20 client.6579 trim_cache size 0 max 16384 2023-11-08T08:23:32.983 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:23:32.983 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "no active scrubs running", 2023-11-08T08:23:32.983 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": {} 2023-11-08T08:23:32.984 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:23:32.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f782f59e700 1 -- 172.21.15.40:0/1628722894 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7810064f30 msgr2=0x7f78100673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:32.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f782f59e700 1 --2- 172.21.15.40:0/1628722894 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7810064f30 0x7f78100673e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f78300b2130 tx=0x7f7824009040 comp rx=0 tx=0).stop 2023-11-08T08:23:32.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f782f59e700 1 -- 172.21.15.40:0/1628722894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7830069e00 msgr2=0x7f78300c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:32.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f782f59e700 1 --2- 172.21.15.40:0/1628722894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7830069e00 0x7f78300c8820 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7820016000 tx=0x7f7820004770 comp rx=0 tx=0).stop 2023-11-08T08:23:32.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f782f59e700 1 -- 172.21.15.40:0/1628722894 shutdown_connections 2023-11-08T08:23:32.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f782f59e700 1 --2- 172.21.15.40:0/1628722894 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7810064f30 0x7f78100673e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f782f59e700 1 --2- 172.21.15.40:0/1628722894 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f783015c640 0x7f78300b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f782f59e700 1 --2- 172.21.15.40:0/1628722894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7830071aa0 0x7f78301f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f782f59e700 1 --2- 172.21.15.40:0/1628722894 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7830069e00 0x7f78300c8820 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:32.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f782f59e700 1 -- 172.21.15.40:0/1628722894 >> 172.21.15.40:0/1628722894 conn(0x7f78300b6520 msgr2=0x7f78300aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:32.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f782f59e700 1 -- 172.21.15.40:0/1628722894 shutdown_connections 2023-11-08T08:23:32.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:32.969+0000 7f782f59e700 1 -- 172.21.15.40:0/1628722894 wait complete. 2023-11-08T08:23:33.000 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-08T08:23:43.001 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:23:43.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.177+0000 7f1d2759e700 1 Processor -- start 2023-11-08T08:23:43.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.181+0000 7f1d2759e700 1 -- start start 2023-11-08T08:23:43.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.181+0000 7f1d2759e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1d280b1640 0x7f1d28070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.181+0000 7f1d2759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d280c9040 0x7f1d280705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.181+0000 7f1d2759e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d2815a550 con 0x7f1d280b43a0 2023-11-08T08:23:43.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.181+0000 7f1d2759e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d2806b0a0 con 0x7f1d280b1640 2023-11-08T08:23:43.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.181+0000 7f1d2759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d280caac0 con 0x7f1d280c9040 2023-11-08T08:23:43.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.181+0000 7f1d2659c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1d280b43a0 0x7f1d2815e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55986/0 (socket says 172.21.15.40:55986) 2023-11-08T08:23:43.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.181+0000 7f1d2659c700 1 -- 172.21.15.40:0/375184573 learned_addr learned my addr 172.21.15.40:0/375184573 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:43.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.181+0000 7f1d25d9b700 1 --2- 172.21.15.40:0/375184573 >> v2:172.21.15.169:3300/0 conn(0x7f1d280b1640 0x7f1d28070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.181+0000 7f1d2559a700 1 -- 172.21.15.40:0/375184573 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3307018576 0 0) 0x7f1d18001110 con 0x7f1d280b43a0 2023-11-08T08:23:43.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.181+0000 7f1d26d9d700 1 --2- 172.21.15.40:0/375184573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d280c9040 0x7f1d280705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.181+0000 7f1d2559a700 1 -- 172.21.15.40:0/375184573 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d100035d0 con 0x7f1d280b43a0 2023-11-08T08:23:43.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.181+0000 7f1d2559a700 1 -- 172.21.15.40:0/375184573 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 965791089 0 0) 0x7f1d100035d0 con 0x7f1d280b43a0 2023-11-08T08:23:43.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.181+0000 7f1d2559a700 1 -- 172.21.15.40:0/375184573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d280c9040 msgr2=0x7f1d280705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:43.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.181+0000 7f1d2559a700 1 --2- 172.21.15.40:0/375184573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d280c9040 0x7f1d280705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d2559a700 1 -- 172.21.15.40:0/375184573 >> v2:172.21.15.169:3300/0 conn(0x7f1d280b1640 msgr2=0x7f1d28070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:43.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d2559a700 1 --2- 172.21.15.40:0/375184573 >> v2:172.21.15.169:3300/0 conn(0x7f1d280b1640 0x7f1d28070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d2559a700 1 -- 172.21.15.40:0/375184573 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d280cada0 con 0x7f1d280b43a0 2023-11-08T08:23:43.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d2559a700 1 -- 172.21.15.40:0/375184573 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f1d18001800 con 0x7f1d280b43a0 2023-11-08T08:23:43.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d2559a700 1 -- 172.21.15.40:0/375184573 >> v1:172.21.15.40:6789/0 conn(0x7f1d280b43a0 legacy=0x7f1d2815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:43.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d2559a700 1 --2- 172.21.15.40:0/375184573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d10003a20 0x7f1d10003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d2559a700 1 --2- 172.21.15.40:0/375184573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d10004400 0x7f1d100048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d2559a700 1 --2- 172.21.15.40:0/375184573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d10005090 0x7f1d1000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d2559a700 1 -- 172.21.15.40:0/375184573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2806b0a0 con 0x7f1d10005090 2023-11-08T08:23:43.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d2559a700 1 -- 172.21.15.40:0/375184573 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d280caac0 con 0x7f1d10004400 2023-11-08T08:23:43.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d2559a700 1 -- 172.21.15.40:0/375184573 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1000c020 con 0x7f1d10003a20 2023-11-08T08:23:43.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d26d9d700 1 --2- 172.21.15.40:0/375184573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d10005090 0x7f1d1000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d2659c700 1 --2- 172.21.15.40:0/375184573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d10003a20 0x7f1d10003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d25d9b700 1 --2- 172.21.15.40:0/375184573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d10004400 0x7f1d100048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d26d9d700 1 -- 172.21.15.40:0/375184573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d10003a20 msgr2=0x7f1d10003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:43.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d26d9d700 1 --2- 172.21.15.40:0/375184573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d10003a20 0x7f1d10003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d26d9d700 1 -- 172.21.15.40:0/375184573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d10004400 msgr2=0x7f1d100048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:43.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d26d9d700 1 --2- 172.21.15.40:0/375184573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d10004400 0x7f1d100048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d26d9d700 1 -- 172.21.15.40:0/375184573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d100035d0 con 0x7f1d10005090 2023-11-08T08:23:43.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d26d9d700 1 --2- 172.21.15.40:0/375184573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d10005090 0x7f1d1000ba30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009ed0 tx=0x7f1d100058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5944f5629781b823 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:43.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d2559a700 1 -- 172.21.15.40:0/375184573 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d20007dc0 con 0x7f1d10005090 2023-11-08T08:23:43.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d2559a700 1 -- 172.21.15.40:0/375184573 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d20014400 con 0x7f1d10005090 2023-11-08T08:23:43.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d2559a700 1 -- 172.21.15.40:0/375184573 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d20014790 con 0x7f1d10005090 2023-11-08T08:23:43.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d2759e700 1 -- 172.21.15.40:0/375184573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d10005090 msgr2=0x7f1d1000ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:43.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d2759e700 1 --2- 172.21.15.40:0/375184573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d10005090 0x7f1d1000ba30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009ed0 tx=0x7f1d100058a0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.185+0000 7f1d26d9d700 1 -- 172.21.15.40:0/375184573 reap_dead start 2023-11-08T08:23:43.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.189+0000 7f1d2759e700 1 -- 172.21.15.40:0/375184573 shutdown_connections 2023-11-08T08:23:43.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.189+0000 7f1d2759e700 1 -- 172.21.15.40:0/375184573 >> 172.21.15.40:0/375184573 conn(0x7f1d280b0f80 msgr2=0x7f1d280ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:43.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.189+0000 7f1d2759e700 1 -- 172.21.15.40:0/375184573 shutdown_connections 2023-11-08T08:23:43.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.189+0000 7f1d2759e700 1 -- 172.21.15.40:0/375184573 wait complete. 2023-11-08T08:23:43.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.189+0000 7f1d2759e700 1 Processor -- start 2023-11-08T08:23:43.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d2759e700 1 -- start start 2023-11-08T08:23:43.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d2759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d10005090 0x7f1d280c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d2759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d281f11c0 0x7f1d2806ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d2759e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d281f1630 0x7f1d280704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d2759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d280c2880 con 0x7f1d281f11c0 2023-11-08T08:23:43.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d2759e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2814f760 con 0x7f1d281f1630 2023-11-08T08:23:43.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d2759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d280c22c0 con 0x7f1d10005090 2023-11-08T08:23:43.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d25d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d281f11c0 0x7f1d2806ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d25d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d281f11c0 0x7f1d2806ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39304/0 (socket says 172.21.15.40:39304) 2023-11-08T08:23:43.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d25d9b700 1 -- 172.21.15.40:0/1306137670 learned_addr learned my addr 172.21.15.40:0/1306137670 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:43.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d2659c700 1 --2- 172.21.15.40:0/1306137670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d10005090 0x7f1d280c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d26d9d700 1 --2- 172.21.15.40:0/1306137670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d281f1630 0x7f1d280704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d25d9b700 1 -- 172.21.15.40:0/1306137670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d10005090 msgr2=0x7f1d280c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:43.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d25d9b700 1 --2- 172.21.15.40:0/1306137670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d10005090 0x7f1d280c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d25d9b700 1 -- 172.21.15.40:0/1306137670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d281f1630 msgr2=0x7f1d280704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:43.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d25d9b700 1 --2- 172.21.15.40:0/1306137670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d281f1630 0x7f1d280704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d25d9b700 1 -- 172.21.15.40:0/1306137670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2806b5d0 con 0x7f1d281f11c0 2023-11-08T08:23:43.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d2659c700 1 --2- 172.21.15.40:0/1306137670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d10005090 0x7f1d280c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:23:43.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d26d9d700 1 --2- 172.21.15.40:0/1306137670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d281f1630 0x7f1d280704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:23:43.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d25d9b700 1 --2- 172.21.15.40:0/1306137670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d281f11c0 0x7f1d2806ff80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c011aa0 tx=0x7f1d1c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:43.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.193+0000 7f1d16ffd700 1 -- 172.21.15.40:0/1306137670 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c004240 con 0x7f1d281f11c0 2023-11-08T08:23:43.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.197+0000 7f1d16ffd700 1 -- 172.21.15.40:0/1306137670 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1c0043a0 con 0x7f1d281f11c0 2023-11-08T08:23:43.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.197+0000 7f1d16ffd700 1 -- 172.21.15.40:0/1306137670 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c00f770 con 0x7f1d281f11c0 2023-11-08T08:23:43.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.197+0000 7f1d2759e700 1 -- 172.21.15.40:0/1306137670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d2806b2f0 con 0x7f1d281f11c0 2023-11-08T08:23:43.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.197+0000 7f1d2759e700 1 -- 172.21.15.40:0/1306137670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d280ca7b0 con 0x7f1d281f11c0 2023-11-08T08:23:43.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.197+0000 7f1d2759e700 1 -- 172.21.15.40:0/1306137670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d28073cb0 con 0x7f1d281f11c0 2023-11-08T08:23:43.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.197+0000 7f1d16ffd700 1 -- 172.21.15.40:0/1306137670 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1d1c025070 con 0x7f1d281f11c0 2023-11-08T08:23:43.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.201+0000 7f1d16ffd700 1 --2- 172.21.15.40:0/1306137670 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d08064ee0 0x7f1d08067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.201+0000 7f1d16ffd700 1 -- 172.21.15.40:0/1306137670 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f1d1c082630 con 0x7f1d281f11c0 2023-11-08T08:23:43.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.201+0000 7f1d2659c700 1 --2- 172.21.15.40:0/1306137670 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d08064ee0 0x7f1d08067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.205+0000 7f1d2659c700 1 --2- 172.21.15.40:0/1306137670 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d08064ee0 0x7f1d08067390 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1d18001270 tx=0x7f1d18009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:43.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.209+0000 7f1d16ffd700 1 -- 172.21.15.40:0/1306137670 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1d1c050560 con 0x7f1d281f11c0 2023-11-08T08:23:43.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.417+0000 7f1d2759e700 1 -- 172.21.15.40:0/1306137670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d28071090 con 0x7f1d281f11c0 2023-11-08T08:23:43.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.417+0000 7f1d16ffd700 1 -- 172.21.15.40:0/1306137670 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f1d1c019090 con 0x7f1d281f11c0 2023-11-08T08:23:43.424 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:23:43.424 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/307473250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":307473250},{"type":"v1","addr":"172.21.15.169:6835","nonce":307473250}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6148,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/924518366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":924518366},{"type":"v1","addr":"172.21.15.169:6837","nonce":924518366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4294005934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4294005934},{"type":"v1","addr":"172.21.15.40:6833","nonce":4294005934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:22:20.650687+0000","modified":"2023-11-08T08:22:37.032476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6237":{"gid":6237,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":7,"addr":"172.21.15.40:6835/3395630689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3395630689},{"type":"v1","addr":"172.21.15.40:6835","nonce":3395630689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T08:23:43.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.421+0000 7f1d2759e700 1 -- 172.21.15.40:0/1306137670 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d08064ee0 msgr2=0x7f1d08067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:43.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.421+0000 7f1d2759e700 1 --2- 172.21.15.40:0/1306137670 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d08064ee0 0x7f1d08067390 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1d18001270 tx=0x7f1d18009040 comp rx=0 tx=0).stop 2023-11-08T08:23:43.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.421+0000 7f1d2759e700 1 -- 172.21.15.40:0/1306137670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d281f11c0 msgr2=0x7f1d2806ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:43.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.421+0000 7f1d2759e700 1 --2- 172.21.15.40:0/1306137670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d281f11c0 0x7f1d2806ff80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c011aa0 tx=0x7f1d1c011e60 comp rx=0 tx=0).stop 2023-11-08T08:23:43.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.421+0000 7f1d2759e700 1 -- 172.21.15.40:0/1306137670 shutdown_connections 2023-11-08T08:23:43.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.421+0000 7f1d2759e700 1 --2- 172.21.15.40:0/1306137670 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d08064ee0 0x7f1d08067390 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.421+0000 7f1d2759e700 1 --2- 172.21.15.40:0/1306137670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d281f1630 0x7f1d280704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.421+0000 7f1d2759e700 1 --2- 172.21.15.40:0/1306137670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d281f11c0 0x7f1d2806ff80 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.421+0000 7f1d2759e700 1 --2- 172.21.15.40:0/1306137670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d10005090 0x7f1d280c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.421+0000 7f1d2759e700 1 -- 172.21.15.40:0/1306137670 >> 172.21.15.40:0/1306137670 conn(0x7f1d280b0f80 msgr2=0x7f1d280ab8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:43.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.421+0000 7f1d2759e700 1 -- 172.21.15.40:0/1306137670 shutdown_connections 2023-11-08T08:23:43.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.421+0000 7f1d2759e700 1 -- 172.21.15.40:0/1306137670 wait complete. 2023-11-08T08:23:43.430 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 34 2023-11-08T08:23:43.449 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2023-11-08T08:23:43.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.621+0000 7fae75851700 1 Processor -- start 2023-11-08T08:23:43.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.621+0000 7fae75851700 1 -- start start 2023-11-08T08:23:43.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.621+0000 7fae75851700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fae700b1640 0x7fae70070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.621+0000 7fae75851700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae700c9040 0x7fae700705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.621+0000 7fae75851700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7015a550 con 0x7fae700b43a0 2023-11-08T08:23:43.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.621+0000 7fae75851700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae7006b0a0 con 0x7fae700b1640 2023-11-08T08:23:43.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.621+0000 7fae75851700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae700caac0 con 0x7fae700c9040 2023-11-08T08:23:43.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.621+0000 7fae67fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fae700b1640 0x7fae70070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.621+0000 7fae6effd700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fae700b43a0 0x7fae7015e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:41476/0 (socket says 172.21.15.40:41476) 2023-11-08T08:23:43.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.621+0000 7fae6effd700 1 -- 172.21.15.40:0/1724808662 learned_addr learned my addr 172.21.15.40:0/1724808662 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:43.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.621+0000 7fae6f7fe700 1 --2- 172.21.15.40:0/1724808662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae700c9040 0x7fae700705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.625+0000 7fae67fff700 1 -- 172.21.15.40:0/1724808662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae700c9040 msgr2=0x7fae700705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:43.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.625+0000 7fae67fff700 1 --2- 172.21.15.40:0/1724808662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae700c9040 0x7fae700705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.625+0000 7fae67fff700 1 -- 172.21.15.40:0/1724808662 >> v1:172.21.15.169:6789/0 conn(0x7fae700b43a0 legacy=0x7fae7015e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:43.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.625+0000 7fae67fff700 1 -- 172.21.15.40:0/1724808662 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae700cada0 con 0x7fae700b1640 2023-11-08T08:23:43.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.625+0000 7fae67fff700 1 --2- 172.21.15.40:0/1724808662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae700b1640 0x7fae70070080 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fae58009850 tx=0x7fae58009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1576416e37aacd1c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:43.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.625+0000 7fae6e7fc700 1 -- 172.21.15.40:0/1724808662 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58011830 con 0x7fae700b1640 2023-11-08T08:23:43.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.625+0000 7fae6e7fc700 1 -- 172.21.15.40:0/1724808662 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae58011990 con 0x7fae700b1640 2023-11-08T08:23:43.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.625+0000 7fae6e7fc700 1 -- 172.21.15.40:0/1724808662 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58011d40 con 0x7fae700b1640 2023-11-08T08:23:43.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.625+0000 7fae75851700 1 -- 172.21.15.40:0/1724808662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae700b1640 msgr2=0x7fae70070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:43.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.625+0000 7fae75851700 1 --2- 172.21.15.40:0/1724808662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae700b1640 0x7fae70070080 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fae58009850 tx=0x7fae58009b60 comp rx=0 tx=0).stop 2023-11-08T08:23:43.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.625+0000 7fae75851700 1 -- 172.21.15.40:0/1724808662 shutdown_connections 2023-11-08T08:23:43.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.625+0000 7fae75851700 1 --2- 172.21.15.40:0/1724808662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae700c9040 0x7fae700705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.625+0000 7fae75851700 1 --2- 172.21.15.40:0/1724808662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae700b1640 0x7fae70070080 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.625+0000 7fae75851700 1 -- 172.21.15.40:0/1724808662 >> 172.21.15.40:0/1724808662 conn(0x7fae700b0f80 msgr2=0x7fae700ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:43.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.625+0000 7fae75851700 1 -- 172.21.15.40:0/1724808662 shutdown_connections 2023-11-08T08:23:43.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.625+0000 7fae75851700 1 -- 172.21.15.40:0/1724808662 wait complete. 2023-11-08T08:23:43.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.629+0000 7fae75851700 1 Processor -- start 2023-11-08T08:23:43.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.629+0000 7fae75851700 1 -- start start 2023-11-08T08:23:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae75851700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae700b1640 0x7fae700c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae75851700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae700b43a0 0x7fae70154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae75851700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae700c9040 0x7fae7006c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae75851700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7015a6f0 con 0x7fae700b1640 2023-11-08T08:23:43.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae75851700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae70154ed0 con 0x7fae700c9040 2023-11-08T08:23:43.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae75851700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7015b3e0 con 0x7fae700b43a0 2023-11-08T08:23:43.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae6effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae700b1640 0x7fae700c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae6effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae700b1640 0x7fae700c4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39320/0 (socket says 172.21.15.40:39320) 2023-11-08T08:23:43.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae6effd700 1 -- 172.21.15.40:0/1990312281 learned_addr learned my addr 172.21.15.40:0/1990312281 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:43.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae67fff700 1 --2- 172.21.15.40:0/1990312281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae700b43a0 0x7fae70154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae6f7fe700 1 --2- 172.21.15.40:0/1990312281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae700c9040 0x7fae7006c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae6effd700 1 -- 172.21.15.40:0/1990312281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae700b43a0 msgr2=0x7fae70154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:43.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae6effd700 1 --2- 172.21.15.40:0/1990312281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae700b43a0 0x7fae70154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae6effd700 1 -- 172.21.15.40:0/1990312281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae700c9040 msgr2=0x7fae7006c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:43.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae6effd700 1 --2- 172.21.15.40:0/1990312281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae700c9040 0x7fae7006c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae6effd700 1 -- 172.21.15.40:0/1990312281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae580094b0 con 0x7fae700b1640 2023-11-08T08:23:43.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae6effd700 1 --2- 172.21.15.40:0/1990312281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae700b1640 0x7fae700c4e70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fae6000c700 tx=0x7fae6000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:43.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae6cff9700 1 -- 172.21.15.40:0/1990312281 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae60005d90 con 0x7fae700b1640 2023-11-08T08:23:43.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae6cff9700 1 -- 172.21.15.40:0/1990312281 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae600070d0 con 0x7fae700b1640 2023-11-08T08:23:43.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae75851700 1 -- 172.21.15.40:0/1990312281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7006b5d0 con 0x7fae700b1640 2023-11-08T08:23:43.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae6cff9700 1 -- 172.21.15.40:0/1990312281 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae60007440 con 0x7fae700b1640 2023-11-08T08:23:43.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.633+0000 7fae75851700 1 -- 172.21.15.40:0/1990312281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7006b2f0 con 0x7fae700b1640 2023-11-08T08:23:43.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.637+0000 7fae6cff9700 1 -- 172.21.15.40:0/1990312281 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fae60019030 con 0x7fae700b1640 2023-11-08T08:23:43.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.641+0000 7fae6cff9700 1 --2- 172.21.15.40:0/1990312281 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fae50064f10 0x7fae500673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.641+0000 7fae6cff9700 1 -- 172.21.15.40:0/1990312281 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fae60081090 con 0x7fae700b1640 2023-11-08T08:23:43.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.641+0000 7fae67fff700 1 --2- 172.21.15.40:0/1990312281 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fae50064f10 0x7fae500673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.641+0000 7fae78adb740 1 Processor -- start 2023-11-08T08:23:43.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.641+0000 7fae67fff700 1 --2- 172.21.15.40:0/1990312281 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fae50064f10 0x7fae500673c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fae580090b0 tx=0x7fae58008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:43.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.641+0000 7fae78adb740 1 -- start start 2023-11-08T08:23:43.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae78adb740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x266b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae78adb740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x23f9950 0x2694070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae78adb740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x24449d0 0x26945b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae78adb740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x2568dc0 con 0x23f9d20 2023-11-08T08:23:43.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae78adb740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x25d8e20 con 0x24449d0 2023-11-08T08:23:43.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae78adb740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x25d2120 con 0x23f9950 2023-11-08T08:23:43.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae6effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x266b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae6effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x266b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39322/0 (socket says 172.21.15.40:39322) 2023-11-08T08:23:43.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae6effd700 1 -- 172.21.15.40:0/227133051 learned_addr learned my addr 172.21.15.40:0/227133051 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:43.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae67fff700 1 --2- 172.21.15.40:0/227133051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x23f9950 0x2694070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae6f7fe700 1 --2- 172.21.15.40:0/227133051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x24449d0 0x26945b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae6effd700 1 -- 172.21.15.40:0/227133051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x23f9950 msgr2=0x2694070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:43.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae6effd700 1 --2- 172.21.15.40:0/227133051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x23f9950 0x2694070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae6effd700 1 -- 172.21.15.40:0/227133051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x24449d0 msgr2=0x26945b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:43.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae6effd700 1 --2- 172.21.15.40:0/227133051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x24449d0 0x26945b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae6effd700 1 -- 172.21.15.40:0/227133051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x25f89b0 con 0x23f9d20 2023-11-08T08:23:43.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae6effd700 1 --2- 172.21.15.40:0/227133051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x266b880 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fae60052740 tx=0x7fae60052770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc906e86ea51d6a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:43.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae657fa700 1 -- 172.21.15.40:0/227133051 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae60056de0 con 0x23f9d20 2023-11-08T08:23:43.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae657fa700 1 -- 172.21.15.40:0/227133051 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae60052220 con 0x23f9d20 2023-11-08T08:23:43.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae657fa700 1 -- 172.21.15.40:0/227133051 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6005a880 con 0x23f9d20 2023-11-08T08:23:43.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae78adb740 1 -- 172.21.15.40:0/227133051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 msgr2=0x266b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:43.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae78adb740 1 --2- 172.21.15.40:0/227133051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x266b880 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fae60052740 tx=0x7fae60052770 comp rx=0 tx=0).stop 2023-11-08T08:23:43.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae78adb740 1 -- 172.21.15.40:0/227133051 shutdown_connections 2023-11-08T08:23:43.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae78adb740 1 --2- 172.21.15.40:0/227133051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x24449d0 0x26945b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae78adb740 1 --2- 172.21.15.40:0/227133051 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x23f9950 0x2694070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae78adb740 1 --2- 172.21.15.40:0/227133051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x266b880 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae78adb740 1 -- 172.21.15.40:0/227133051 >> 172.21.15.40:0/227133051 conn(0x23fa0f0 msgr2=0x2692e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:43.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.645+0000 7fae78adb740 1 -- 172.21.15.40:0/227133051 shutdown_connections 2023-11-08T08:23:43.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.649+0000 7fae78adb740 1 -- 172.21.15.40:0/227133051 wait complete. 2023-11-08T08:23:43.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.649+0000 7fae78adb740 1 -- start start 2023-11-08T08:23:43.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.649+0000 7fae78adb740 1 Processor -- start 2023-11-08T08:23:43.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.649+0000 7fae78adb740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x23f9950 0x26961c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.649+0000 7fae78adb740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x268cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.649+0000 7fae78adb740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x24449d0 0x268d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.649+0000 7fae78adb740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x247ed60 con 0x23f9d20 2023-11-08T08:23:43.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.649+0000 7fae78adb740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x2697300 con 0x23f9950 2023-11-08T08:23:43.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.649+0000 7fae78adb740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x2696700 con 0x24449d0 2023-11-08T08:23:43.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.649+0000 7fae78adb740 10 client.0 _ll_register_callbacks cb 0x25cba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:23:43.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae67fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x268cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae67fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x268cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39338/0 (socket says 172.21.15.40:39338) 2023-11-08T08:23:43.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae67fff700 1 -- 172.21.15.40:0/3443536006 learned_addr learned my addr 172.21.15.40:0/3443536006 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:43.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae6effd700 1 --2- 172.21.15.40:0/3443536006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x23f9950 0x26961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae6f7fe700 1 --2- 172.21.15.40:0/3443536006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x24449d0 0x268d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae67fff700 1 -- 172.21.15.40:0/3443536006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x24449d0 msgr2=0x268d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:43.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae67fff700 1 --2- 172.21.15.40:0/3443536006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x24449d0 0x268d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae67fff700 1 -- 172.21.15.40:0/3443536006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x23f9950 msgr2=0x26961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:43.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae67fff700 1 --2- 172.21.15.40:0/3443536006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x23f9950 0x26961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae67fff700 1 -- 172.21.15.40:0/3443536006 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7006b2f0 con 0x23f9d20 2023-11-08T08:23:43.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae6effd700 1 --2- 172.21.15.40:0/3443536006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x23f9950 0x26961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:23:43.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae67fff700 1 --2- 172.21.15.40:0/3443536006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x268cc20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fae58013000 tx=0x7fae580047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd5d0d8f5afd51e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:43.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae4e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:23:43.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae4e7fc700 1 -- 172.21.15.40:0/3443536006 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5801f070 con 0x23f9d20 2023-11-08T08:23:43.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae4e7fc700 1 -- 172.21.15.40:0/3443536006 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fae580038d0 con 0x23f9d20 2023-11-08T08:23:43.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae78adb740 1 -- 172.21.15.40:0/3443536006 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x25cb1a0 con 0x23f9d20 2023-11-08T08:23:43.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae4e7fc700 1 -- 172.21.15.40:0/3443536006 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae58003a90 con 0x23f9d20 2023-11-08T08:23:43.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae78adb740 10 client.6597 fetch_fsmap learned FSMap version 34 2023-11-08T08:23:43.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae78adb740 1 -- 172.21.15.40:0/3443536006 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x2445c20 con 0x23f9d20 2023-11-08T08:23:43.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae4e7fc700 1 -- 172.21.15.40:0/3443536006 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x25cb1a0 con 0x23f9d20 2023-11-08T08:23:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae4e7fc700 1 -- 172.21.15.40:0/3443536006 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 34) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fae5802f070 con 0x23f9d20 2023-11-08T08:23:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae78adb740 10 client.6597 fetch_fsmap finished waiting for FSMap version 34 2023-11-08T08:23:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae78adb740 10 client.6597 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T08:23:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae78adb740 20 client.6597 populate_metadata read hostname 'smithi040' 2023-11-08T08:23:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae78adb740 1 --2- 172.21.15.40:0/3443536006 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x2694b80 0x26c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae78adb740 4 client.6597 mds_command: new command op to 6237 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-08T08:23:43.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.653+0000 7fae78adb740 1 -- 172.21.15.40:0/3443536006 --> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x25caec0 con 0x2694b80 2023-11-08T08:23:43.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.657+0000 7fae6effd700 1 --2- 172.21.15.40:0/3443536006 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x2694b80 0x26c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.657+0000 7fae6effd700 1 --2- 172.21.15.40:0/3443536006 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x2694b80 0x26c7ea0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff8a0446fe1969c7 server_cookie=dc852ed2e578f4ae in_seq=0 out_seq=0 2023-11-08T08:23:43.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.657+0000 7fae4e7fc700 10 client.6597 ms_handle_connect on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:43.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.657+0000 7fae4e7fc700 1 -- 172.21.15.40:0/3443536006 <== mds.0 v2:172.21.15.40:6834/3395630689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x25caec0 con 0x2694b80 2023-11-08T08:23:43.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.657+0000 7fae4e7fc700 10 client.6597 handle_command_reply: tid=0 2023-11-08T08:23:43.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.657+0000 7fae78adb740 1 client.6597 shutdown 2023-11-08T08:23:43.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.657+0000 7fae78adb740 1 -- 172.21.15.40:0/3443536006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 msgr2=0x268cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:43.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.657+0000 7fae78adb740 1 --2- 172.21.15.40:0/3443536006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x268cc20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fae58013000 tx=0x7fae580047e0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.657+0000 7fae78adb740 1 -- 172.21.15.40:0/3443536006 shutdown_connections 2023-11-08T08:23:43.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.657+0000 7fae78adb740 1 --2- 172.21.15.40:0/3443536006 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x2694b80 0x26c7ea0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.657+0000 7fae78adb740 1 --2- 172.21.15.40:0/3443536006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x24449d0 0x268d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.657+0000 7fae78adb740 1 --2- 172.21.15.40:0/3443536006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x268cc20 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.657+0000 7fae78adb740 1 --2- 172.21.15.40:0/3443536006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x23f9950 0x26961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.657+0000 7fae78adb740 1 -- 172.21.15.40:0/3443536006 >> 172.21.15.40:0/3443536006 conn(0x23fa0f0 msgr2=0x2692dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:43.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.657+0000 7fae4e7fc700 0 client.6597 ms_handle_reset on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:43.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.657+0000 7fae78adb740 1 -- 172.21.15.40:0/3443536006 shutdown_connections 2023-11-08T08:23:43.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.657+0000 7fae78adb740 1 -- 172.21.15.40:0/3443536006 wait complete. 2023-11-08T08:23:43.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.657+0000 7fae78adb740 20 client.6597 trim_cache size 0 max 16384 2023-11-08T08:23:43.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.673+0000 7fae78adb740 1 Processor -- start 2023-11-08T08:23:43.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.673+0000 7fae78adb740 1 -- start start 2023-11-08T08:23:43.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.673+0000 7fae78adb740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x23fa0f0 0x26c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.673+0000 7fae78adb740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x23f9950 0x2692610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.673+0000 7fae78adb740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x2692b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.673+0000 7fae78adb740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x25d0e40 con 0x23f9d20 2023-11-08T08:23:43.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.673+0000 7fae78adb740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x25d4c80 con 0x23fa0f0 2023-11-08T08:23:43.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.673+0000 7fae78adb740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x26cc540 con 0x23f9950 2023-11-08T08:23:43.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.677+0000 7fae6f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x2692b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.677+0000 7fae6f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x2692b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39346/0 (socket says 172.21.15.40:39346) 2023-11-08T08:23:43.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.677+0000 7fae6f7fe700 1 -- 172.21.15.40:0/1834909709 learned_addr learned my addr 172.21.15.40:0/1834909709 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:43.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.677+0000 7fae6effd700 1 --2- 172.21.15.40:0/1834909709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x23fa0f0 0x26c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.677+0000 7fae67fff700 1 --2- 172.21.15.40:0/1834909709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x23f9950 0x2692610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.677+0000 7fae6f7fe700 1 -- 172.21.15.40:0/1834909709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x23f9950 msgr2=0x2692610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:43.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.677+0000 7fae6f7fe700 1 --2- 172.21.15.40:0/1834909709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x23f9950 0x2692610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.677+0000 7fae6f7fe700 1 -- 172.21.15.40:0/1834909709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x23fa0f0 msgr2=0x26c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:43.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.677+0000 7fae6f7fe700 1 --2- 172.21.15.40:0/1834909709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x23fa0f0 0x26c45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.677+0000 7fae6f7fe700 1 -- 172.21.15.40:0/1834909709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7006ce50 con 0x23f9d20 2023-11-08T08:23:43.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.677+0000 7fae6effd700 1 --2- 172.21.15.40:0/1834909709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x23fa0f0 0x26c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:23:43.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.677+0000 7fae6f7fe700 1 --2- 172.21.15.40:0/1834909709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x2692b50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fae68003df0 tx=0x7fae68009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d0e4fcfd920e60 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:43.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.677+0000 7fae65ffb700 1 -- 172.21.15.40:0/1834909709 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6800e040 con 0x23f9d20 2023-11-08T08:23:43.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.677+0000 7fae65ffb700 1 -- 172.21.15.40:0/1834909709 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6800f040 con 0x23f9d20 2023-11-08T08:23:43.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.677+0000 7fae65ffb700 1 -- 172.21.15.40:0/1834909709 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae680206b0 con 0x23f9d20 2023-11-08T08:23:43.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.677+0000 7fae78adb740 1 -- 172.21.15.40:0/1834909709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 msgr2=0x2692b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:43.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.677+0000 7fae78adb740 1 --2- 172.21.15.40:0/1834909709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x2692b50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fae68003df0 tx=0x7fae68009750 comp rx=0 tx=0).stop 2023-11-08T08:23:43.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae78adb740 1 -- 172.21.15.40:0/1834909709 shutdown_connections 2023-11-08T08:23:43.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae78adb740 1 --2- 172.21.15.40:0/1834909709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x2692b50 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae78adb740 1 --2- 172.21.15.40:0/1834909709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x23f9950 0x2692610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae78adb740 1 --2- 172.21.15.40:0/1834909709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x23fa0f0 0x26c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae78adb740 1 -- 172.21.15.40:0/1834909709 >> 172.21.15.40:0/1834909709 conn(0x2696330 msgr2=0x26c3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:43.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae78adb740 1 -- 172.21.15.40:0/1834909709 shutdown_connections 2023-11-08T08:23:43.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae78adb740 1 -- 172.21.15.40:0/1834909709 wait complete. 2023-11-08T08:23:43.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae78adb740 1 -- start start 2023-11-08T08:23:43.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae78adb740 1 Processor -- start 2023-11-08T08:23:43.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae78adb740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x23f9950 0x2710990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae78adb740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x2666c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae78adb740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x23fa0f0 0x26b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae78adb740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x25cc4c0 con 0x23f9d20 2023-11-08T08:23:43.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae78adb740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x25cda90 con 0x23fa0f0 2023-11-08T08:23:43.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae78adb740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x25d6120 con 0x23f9950 2023-11-08T08:23:43.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae78adb740 10 client.0 _ll_register_callbacks cb 0x7fae58011dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T08:23:43.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae67fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x2666c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae67fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x2666c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39362/0 (socket says 172.21.15.40:39362) 2023-11-08T08:23:43.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae67fff700 1 -- 172.21.15.40:0/1512381964 learned_addr learned my addr 172.21.15.40:0/1512381964 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:43.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae6f7fe700 1 --2- 172.21.15.40:0/1512381964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x23fa0f0 0x26b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae6effd700 1 --2- 172.21.15.40:0/1512381964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x23f9950 0x2710990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae67fff700 1 -- 172.21.15.40:0/1512381964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x23f9950 msgr2=0x2710990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:43.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae67fff700 1 --2- 172.21.15.40:0/1512381964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x23f9950 0x2710990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae67fff700 1 -- 172.21.15.40:0/1512381964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x23fa0f0 msgr2=0x26b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:43.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae67fff700 1 --2- 172.21.15.40:0/1512381964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x23fa0f0 0x26b7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae67fff700 1 -- 172.21.15.40:0/1512381964 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x25f89b0 con 0x23f9d20 2023-11-08T08:23:43.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.681+0000 7fae6effd700 1 --2- 172.21.15.40:0/1512381964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x23f9950 0x2710990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:23:43.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.685+0000 7fae67fff700 1 --2- 172.21.15.40:0/1512381964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x2666c20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fae58009eb0 tx=0x7fae58009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2d4b64e6ade165 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:43.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.685+0000 7fae4e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-11-08T08:23:43.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.685+0000 7fae4e7fc700 1 -- 172.21.15.40:0/1512381964 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58003ce0 con 0x23f9d20 2023-11-08T08:23:43.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.685+0000 7fae4e7fc700 1 -- 172.21.15.40:0/1512381964 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fae580329d0 con 0x23f9d20 2023-11-08T08:23:43.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.685+0000 7fae78adb740 1 -- 172.21.15.40:0/1512381964 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae60052220 con 0x23f9d20 2023-11-08T08:23:43.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.685+0000 7fae4e7fc700 1 -- 172.21.15.40:0/1512381964 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae58003ec0 con 0x23f9d20 2023-11-08T08:23:43.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.685+0000 7fae78adb740 10 client.6603 fetch_fsmap learned FSMap version 34 2023-11-08T08:23:43.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.685+0000 7fae78adb740 1 -- 172.21.15.40:0/1512381964 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x7fae68003aa0 con 0x23f9d20 2023-11-08T08:23:43.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.685+0000 7fae4e7fc700 1 -- 172.21.15.40:0/1512381964 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae58051020 con 0x23f9d20 2023-11-08T08:23:43.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.685+0000 7fae4e7fc700 1 -- 172.21.15.40:0/1512381964 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 34) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fae580040b0 con 0x23f9d20 2023-11-08T08:23:43.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.685+0000 7fae78adb740 10 client.6603 fetch_fsmap finished waiting for FSMap version 34 2023-11-08T08:23:43.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.685+0000 7fae78adb740 10 client.6603 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T08:23:43.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.685+0000 7fae78adb740 20 client.6603 populate_metadata read hostname 'smithi040' 2023-11-08T08:23:43.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.685+0000 7fae78adb740 1 --2- 172.21.15.40:0/1512381964 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x24449d0 0x26b9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:43.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.685+0000 7fae78adb740 4 client.6603 mds_command: new command op to 6237 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-11-08T08:23:43.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.685+0000 7fae78adb740 1 -- 172.21.15.40:0/1512381964 --> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x24764a0 con 0x24449d0 2023-11-08T08:23:43.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.685+0000 7fae6effd700 1 --2- 172.21.15.40:0/1512381964 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x24449d0 0x26b9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:43.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae6effd700 1 --2- 172.21.15.40:0/1512381964 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x24449d0 0x26b9a10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b3b60eec1c6bcdc server_cookie=b66286c92378235 in_seq=0 out_seq=0 2023-11-08T08:23:43.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae4e7fc700 10 client.6603 ms_handle_connect on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:43.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae4e7fc700 1 -- 172.21.15.40:0/1512381964 <== mds.0 v2:172.21.15.40:6834/3395630689 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+141 (crc 0 0 0) 0x24764a0 con 0x24449d0 2023-11-08T08:23:43.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae4e7fc700 10 client.6603 handle_command_reply: tid=0 2023-11-08T08:23:43.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae78adb740 1 client.6603 shutdown 2023-11-08T08:23:43.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae78adb740 1 -- 172.21.15.40:0/1512381964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 msgr2=0x2666c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:43.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae78adb740 1 --2- 172.21.15.40:0/1512381964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x2666c20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fae58009eb0 tx=0x7fae58009240 comp rx=0 tx=0).stop 2023-11-08T08:23:43.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae78adb740 1 -- 172.21.15.40:0/1512381964 shutdown_connections 2023-11-08T08:23:43.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae78adb740 1 --2- 172.21.15.40:0/1512381964 >> [v2:172.21.15.40:6834/3395630689,v1:172.21.15.40:6835/3395630689] conn(0x24449d0 0x26b9a10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae78adb740 1 --2- 172.21.15.40:0/1512381964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x23fa0f0 0x26b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae78adb740 1 --2- 172.21.15.40:0/1512381964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x23f9d20 0x2666c20 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae78adb740 1 --2- 172.21.15.40:0/1512381964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x23f9950 0x2710990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae4e7fc700 0 client.6603 ms_handle_reset on v2:172.21.15.40:6834/3395630689 2023-11-08T08:23:43.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae78adb740 1 -- 172.21.15.40:0/1512381964 >> 172.21.15.40:0/1512381964 conn(0x2696330 msgr2=0x26c4520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:43.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae78adb740 1 -- 172.21.15.40:0/1512381964 shutdown_connections 2023-11-08T08:23:43.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae78adb740 1 -- 172.21.15.40:0/1512381964 wait complete. 2023-11-08T08:23:43.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae78adb740 20 client.6603 trim_cache size 0 max 16384 2023-11-08T08:23:43.705 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:23:43.705 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2023-11-08T08:23:43.705 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-11-08T08:23:43.705 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-11-08T08:23:43.706 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1858360389, 2023-11-08T08:23:43.706 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511627777, 2023-11-08T08:23:43.706 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/dir_test/file" 2023-11-08T08:23:43.706 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-11-08T08:23:43.706 INFO:teuthology.orchestra.run.smithi040.stdout:] 2023-11-08T08:23:43.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae75851700 1 -- 172.21.15.40:0/1990312281 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fae50064f10 msgr2=0x7fae500673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:43.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae75851700 1 --2- 172.21.15.40:0/1990312281 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fae50064f10 0x7fae500673c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fae580090b0 tx=0x7fae58008030 comp rx=0 tx=0).stop 2023-11-08T08:23:43.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae75851700 1 -- 172.21.15.40:0/1990312281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae700b1640 msgr2=0x7fae700c4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:43.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae75851700 1 --2- 172.21.15.40:0/1990312281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae700b1640 0x7fae700c4e70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fae6000c700 tx=0x7fae6000ca10 comp rx=0 tx=0).stop 2023-11-08T08:23:43.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae75851700 1 -- 172.21.15.40:0/1990312281 shutdown_connections 2023-11-08T08:23:43.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae75851700 1 --2- 172.21.15.40:0/1990312281 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fae50064f10 0x7fae500673c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae75851700 1 --2- 172.21.15.40:0/1990312281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae700c9040 0x7fae7006c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae75851700 1 --2- 172.21.15.40:0/1990312281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae700b43a0 0x7fae70154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae75851700 1 --2- 172.21.15.40:0/1990312281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae700b1640 0x7fae700c4e70 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:43.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae75851700 1 -- 172.21.15.40:0/1990312281 >> 172.21.15.40:0/1990312281 conn(0x7fae700b0f80 msgr2=0x7fae700afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:43.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae75851700 1 -- 172.21.15.40:0/1990312281 shutdown_connections 2023-11-08T08:23:43.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:43.689+0000 7fae75851700 1 -- 172.21.15.40:0/1990312281 wait complete. 2023-11-08T08:23:43.719 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T08:23:43.720 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T08:23:43.764 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T08:23:43.808 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T08:23:43.809 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:23:43.809 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:23:43.831 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:23:43.831 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:23:43.891 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:23:44.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.105+0000 7fe27b285700 1 Processor -- start 2023-11-08T08:23:44.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.109+0000 7fe27b285700 1 -- start start 2023-11-08T08:23:44.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.109+0000 7fe27b285700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe274160710 0x7fe2740cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:44.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.109+0000 7fe27b285700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2740ad6c0 con 0x7fe274070790 2023-11-08T08:23:44.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.109+0000 7fe27b285700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2740adc60 con 0x7fe2740cc860 2023-11-08T08:23:44.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.109+0000 7fe27b285700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe27406b1c0 con 0x7fe274160710 2023-11-08T08:23:44.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.109+0000 7fe279a82700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fe2740cc860 0x7fe274070080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:41488/0 (socket says 172.21.15.40:41488) 2023-11-08T08:23:44.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.109+0000 7fe279a82700 1 -- 172.21.15.40:0/3555115102 learned_addr learned my addr 172.21.15.40:0/3555115102 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:44.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.109+0000 7fe27a283700 1 --2- 172.21.15.40:0/3555115102 >> v2:172.21.15.40:3300/0 conn(0x7fe274160710 0x7fe2740cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.109+0000 7fe279281700 1 -- 172.21.15.40:0/3555115102 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926538034 0 0) 0x7fe2740adc60 con 0x7fe2740cc860 2023-11-08T08:23:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.109+0000 7fe279281700 1 -- 172.21.15.40:0/3555115102 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2600035d0 con 0x7fe2740cc860 2023-11-08T08:23:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.109+0000 7fe279281700 1 -- 172.21.15.40:0/3555115102 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567645482 0 0) 0x7fe2740ad6c0 con 0x7fe274070790 2023-11-08T08:23:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.109+0000 7fe279281700 1 -- 172.21.15.40:0/3555115102 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2740adc60 con 0x7fe274070790 2023-11-08T08:23:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.109+0000 7fe27a283700 1 -- 172.21.15.40:0/3555115102 >> v1:172.21.15.169:6789/0 conn(0x7fe2740cc860 legacy=0x7fe274070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:44.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.109+0000 7fe27a283700 1 -- 172.21.15.40:0/3555115102 >> v1:172.21.15.40:6789/0 conn(0x7fe274070790 legacy=0x7fe274164250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:44.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.109+0000 7fe279281700 1 -- 172.21.15.40:0/3555115102 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 666288633 0 0) 0x7fe2740adc60 con 0x7fe274070790 2023-11-08T08:23:44.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.109+0000 7fe27a283700 1 -- 172.21.15.40:0/3555115102 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27406b3d0 con 0x7fe274160710 2023-11-08T08:23:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.113+0000 7fe27a283700 1 --2- 172.21.15.40:0/3555115102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe274160710 0x7fe2740cc320 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe270009850 tx=0x7fe270009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfef2319da194595 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.113+0000 7fe279281700 1 -- 172.21.15.40:0/3555115102 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c0019f0 con 0x7fe274160710 2023-11-08T08:23:44.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.113+0000 7fe279281700 1 -- 172.21.15.40:0/3555115102 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe270004030 con 0x7fe274160710 2023-11-08T08:23:44.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.113+0000 7fe279281700 1 -- 172.21.15.40:0/3555115102 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2700043c0 con 0x7fe274160710 2023-11-08T08:23:44.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.113+0000 7fe27b285700 1 -- 172.21.15.40:0/3555115102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe274160710 msgr2=0x7fe2740cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:44.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.113+0000 7fe27b285700 1 --2- 172.21.15.40:0/3555115102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe274160710 0x7fe2740cc320 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe270009850 tx=0x7fe270009b60 comp rx=0 tx=0).stop 2023-11-08T08:23:44.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.113+0000 7fe27b285700 1 -- 172.21.15.40:0/3555115102 shutdown_connections 2023-11-08T08:23:44.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.113+0000 7fe27b285700 1 --2- 172.21.15.40:0/3555115102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe274160710 0x7fe2740cc320 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:44.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.113+0000 7fe27b285700 1 -- 172.21.15.40:0/3555115102 >> 172.21.15.40:0/3555115102 conn(0x7fe2740b0f80 msgr2=0x7fe2740ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:44.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.113+0000 7fe27b285700 1 -- 172.21.15.40:0/3555115102 shutdown_connections 2023-11-08T08:23:44.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.113+0000 7fe27b285700 1 -- 172.21.15.40:0/3555115102 wait complete. 2023-11-08T08:23:44.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.117+0000 7fe27b285700 1 Processor -- start 2023-11-08T08:23:44.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.117+0000 7fe27b285700 1 -- start start 2023-11-08T08:23:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.117+0000 7fe27b285700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe274070790 0x7fe27414a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:44.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.117+0000 7fe27b285700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2740cc860 0x7fe27415de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.117+0000 7fe27b285700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe274160710 0x7fe27415e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.117+0000 7fe27b285700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27414a710 con 0x7fe274160710 2023-11-08T08:23:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.117+0000 7fe27b285700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2740c28a0 con 0x7fe2740cc860 2023-11-08T08:23:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.117+0000 7fe27b285700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2740c2320 con 0x7fe274070790 2023-11-08T08:23:44.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.117+0000 7fe27aa84700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe274160710 0x7fe27415e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.117+0000 7fe27aa84700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe274160710 0x7fe27415e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39384/0 (socket says 172.21.15.40:39384) 2023-11-08T08:23:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.117+0000 7fe27aa84700 1 -- 172.21.15.40:0/2142014576 learned_addr learned my addr 172.21.15.40:0/2142014576 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:44.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.117+0000 7fe279a82700 1 --2- 172.21.15.40:0/2142014576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2740cc860 0x7fe27415de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:44.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.117+0000 7fe27a283700 1 --2- 172.21.15.40:0/2142014576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe274070790 0x7fe27414a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:44.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.121+0000 7fe27aa84700 1 -- 172.21.15.40:0/2142014576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe274070790 msgr2=0x7fe27414a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:44.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.121+0000 7fe27aa84700 1 --2- 172.21.15.40:0/2142014576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe274070790 0x7fe27414a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:44.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.121+0000 7fe27aa84700 1 -- 172.21.15.40:0/2142014576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2740cc860 msgr2=0x7fe27415de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:44.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.121+0000 7fe27aa84700 1 --2- 172.21.15.40:0/2142014576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2740cc860 0x7fe27415de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:44.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.121+0000 7fe27aa84700 1 -- 172.21.15.40:0/2142014576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2700094b0 con 0x7fe274160710 2023-11-08T08:23:44.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.121+0000 7fe27aa84700 1 --2- 172.21.15.40:0/2142014576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe274160710 0x7fe27415e900 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe26c00c840 tx=0x7fe26c00cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:44.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.121+0000 7fe26affd700 1 -- 172.21.15.40:0/2142014576 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c003e30 con 0x7fe274160710 2023-11-08T08:23:44.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.121+0000 7fe26affd700 1 -- 172.21.15.40:0/2142014576 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26c005070 con 0x7fe274160710 2023-11-08T08:23:44.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.121+0000 7fe27b285700 1 -- 172.21.15.40:0/2142014576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2740ad8f0 con 0x7fe274160710 2023-11-08T08:23:44.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.121+0000 7fe26affd700 1 -- 172.21.15.40:0/2142014576 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c005420 con 0x7fe274160710 2023-11-08T08:23:44.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.121+0000 7fe27b285700 1 -- 172.21.15.40:0/2142014576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27406b6b0 con 0x7fe274160710 2023-11-08T08:23:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.121+0000 7fe27b285700 1 -- 172.21.15.40:0/2142014576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe274160ae0 con 0x7fe274160710 2023-11-08T08:23:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.121+0000 7fe26affd700 1 -- 172.21.15.40:0/2142014576 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fe26c019030 con 0x7fe274160710 2023-11-08T08:23:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.125+0000 7fe26affd700 1 --2- 172.21.15.40:0/2142014576 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe258064ee0 0x7fe258067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:44.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.125+0000 7fe26affd700 1 -- 172.21.15.40:0/2142014576 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fe26c081690 con 0x7fe274160710 2023-11-08T08:23:44.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.125+0000 7fe27a283700 1 --2- 172.21.15.40:0/2142014576 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe258064ee0 0x7fe258067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:44.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.125+0000 7fe27a283700 1 --2- 172.21.15.40:0/2142014576 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe258064ee0 0x7fe258067390 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe2700090b0 tx=0x7fe270008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.133+0000 7fe26affd700 1 -- 172.21.15.40:0/2142014576 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fe26c04f5c0 con 0x7fe274160710 2023-11-08T08:23:44.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.309+0000 7fe27b285700 1 -- 172.21.15.40:0/2142014576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe274073cb0 con 0x7fe274160710 2023-11-08T08:23:44.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.309+0000 7fe26affd700 1 -- 172.21.15.40:0/2142014576 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+18185 (secure 0 0 0) 0x7fe26c017020 con 0x7fe274160710 2023-11-08T08:23:44.318 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:23:44.318 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":71,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:22:46.031951+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:22:18.224463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T08:22:19.658954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.26","mappings":[{"from":6,"to":0}]},{"pgid":"11.28","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:23:44.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.313+0000 7fe27b285700 1 -- 172.21.15.40:0/2142014576 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe258064ee0 msgr2=0x7fe258067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.313+0000 7fe27b285700 1 --2- 172.21.15.40:0/2142014576 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe258064ee0 0x7fe258067390 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe2700090b0 tx=0x7fe270008030 comp rx=0 tx=0).stop 2023-11-08T08:23:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.313+0000 7fe27b285700 1 -- 172.21.15.40:0/2142014576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe274160710 msgr2=0x7fe27415e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.313+0000 7fe27b285700 1 --2- 172.21.15.40:0/2142014576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe274160710 0x7fe27415e900 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe26c00c840 tx=0x7fe26c00cb50 comp rx=0 tx=0).stop 2023-11-08T08:23:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.313+0000 7fe27b285700 1 -- 172.21.15.40:0/2142014576 shutdown_connections 2023-11-08T08:23:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.313+0000 7fe27b285700 1 --2- 172.21.15.40:0/2142014576 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fe258064ee0 0x7fe258067390 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.313+0000 7fe27b285700 1 --2- 172.21.15.40:0/2142014576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe274160710 0x7fe27415e900 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.313+0000 7fe27b285700 1 --2- 172.21.15.40:0/2142014576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2740cc860 0x7fe27415de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.313+0000 7fe27b285700 1 --2- 172.21.15.40:0/2142014576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe274070790 0x7fe27414a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.313+0000 7fe27b285700 1 -- 172.21.15.40:0/2142014576 >> 172.21.15.40:0/2142014576 conn(0x7fe2740b0f80 msgr2=0x7fe274163160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.313+0000 7fe27b285700 1 -- 172.21.15.40:0/2142014576 shutdown_connections 2023-11-08T08:23:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.313+0000 7fe27b285700 1 -- 172.21.15.40:0/2142014576 wait complete. 2023-11-08T08:23:44.345 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T08:22:18.224463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T08:22:19.658954+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T08:23:44.346 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:23:44.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.529+0000 7f614b0dc700 1 Processor -- start 2023-11-08T08:23:44.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.529+0000 7f614b0dc700 1 -- start start 2023-11-08T08:23:44.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.533+0000 7f614b0dc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61440b3d00 0x7f614406a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:44.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.533+0000 7f614b0dc700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614415c520 con 0x7f614415c150 2023-11-08T08:23:44.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.533+0000 7f614b0dc700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614406cc50 con 0x7f6144070620 2023-11-08T08:23:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.533+0000 7f614b0dc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61440d2450 con 0x7f61440b3d00 2023-11-08T08:23:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.533+0000 7f614a0da700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f614415c150 0x7f614406ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55996/0 (socket says 172.21.15.40:55996) 2023-11-08T08:23:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.533+0000 7f614a0da700 1 -- 172.21.15.40:0/1562321561 learned_addr learned my addr 172.21.15.40:0/1562321561 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:44.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.533+0000 7f614a8db700 1 --2- 172.21.15.40:0/1562321561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61440b3d00 0x7f614406a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:44.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.533+0000 7f61490d8700 1 -- 172.21.15.40:0/1562321561 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197102346 0 0) 0x7f614415c520 con 0x7f614415c150 2023-11-08T08:23:44.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.533+0000 7f61490d8700 1 -- 172.21.15.40:0/1562321561 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61300035d0 con 0x7f614415c150 2023-11-08T08:23:44.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.533+0000 7f61490d8700 1 -- 172.21.15.40:0/1562321561 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516207786 0 0) 0x7f614406cc50 con 0x7f6144070620 2023-11-08T08:23:44.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.533+0000 7f614a8db700 1 -- 172.21.15.40:0/1562321561 >> v1:172.21.15.169:6789/0 conn(0x7f6144070620 legacy=0x7f6144069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:44.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.533+0000 7f614a8db700 1 -- 172.21.15.40:0/1562321561 >> v1:172.21.15.40:6789/0 conn(0x7f614415c150 legacy=0x7f614406ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:44.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.533+0000 7f614a8db700 1 -- 172.21.15.40:0/1562321561 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61440d2730 con 0x7f61440b3d00 2023-11-08T08:23:44.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.537+0000 7f614a8db700 1 --2- 172.21.15.40:0/1562321561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61440b3d00 0x7f614406a300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f613c009dc0 tx=0x7f613c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a9e98fcda2313af server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:44.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.537+0000 7f61490d8700 1 -- 172.21.15.40:0/1562321561 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c00e070 con 0x7f61440b3d00 2023-11-08T08:23:44.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.537+0000 7f61490d8700 1 -- 172.21.15.40:0/1562321561 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613c00bb20 con 0x7f61440b3d00 2023-11-08T08:23:44.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.537+0000 7f61490d8700 1 -- 172.21.15.40:0/1562321561 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c00bed0 con 0x7f61440b3d00 2023-11-08T08:23:44.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.537+0000 7f614b0dc700 1 -- 172.21.15.40:0/1562321561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61440b3d00 msgr2=0x7f614406a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:44.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.537+0000 7f614b0dc700 1 --2- 172.21.15.40:0/1562321561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61440b3d00 0x7f614406a300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f613c009dc0 tx=0x7f613c00b330 comp rx=0 tx=0).stop 2023-11-08T08:23:44.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.537+0000 7f614b0dc700 1 -- 172.21.15.40:0/1562321561 shutdown_connections 2023-11-08T08:23:44.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.537+0000 7f614b0dc700 1 --2- 172.21.15.40:0/1562321561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61440b3d00 0x7f614406a300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:44.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.537+0000 7f614b0dc700 1 -- 172.21.15.40:0/1562321561 >> 172.21.15.40:0/1562321561 conn(0x7f61440d3680 msgr2=0x7f61440cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:44.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.537+0000 7f614b0dc700 1 -- 172.21.15.40:0/1562321561 shutdown_connections 2023-11-08T08:23:44.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.537+0000 7f614b0dc700 1 -- 172.21.15.40:0/1562321561 wait complete. 2023-11-08T08:23:44.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.541+0000 7f614b0dc700 1 Processor -- start 2023-11-08T08:23:44.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.541+0000 7f614b0dc700 1 -- start start 2023-11-08T08:23:44.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.541+0000 7f614b0dc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6144070620 0x7f61440ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.541+0000 7f614b0dc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61440b3d00 0x7f614414e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.541+0000 7f614b0dc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f614415c150 0x7f61440b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.541+0000 7f614b0dc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6144156680 con 0x7f614415c150 2023-11-08T08:23:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.541+0000 7f614b0dc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6144156960 con 0x7f61440b3d00 2023-11-08T08:23:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.541+0000 7f614b0dc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6144156c40 con 0x7f6144070620 2023-11-08T08:23:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.541+0000 7f614a8db700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f614415c150 0x7f61440b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.541+0000 7f614a8db700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f614415c150 0x7f61440b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39392/0 (socket says 172.21.15.40:39392) 2023-11-08T08:23:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.541+0000 7f614a8db700 1 -- 172.21.15.40:0/4065568265 learned_addr learned my addr 172.21.15.40:0/4065568265 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.541+0000 7f61498d9700 1 --2- 172.21.15.40:0/4065568265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61440b3d00 0x7f614414e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:44.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.541+0000 7f614a0da700 1 --2- 172.21.15.40:0/4065568265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6144070620 0x7f61440ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:44.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.541+0000 7f614a8db700 1 -- 172.21.15.40:0/4065568265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6144070620 msgr2=0x7f61440ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:44.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.541+0000 7f614a8db700 1 --2- 172.21.15.40:0/4065568265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6144070620 0x7f61440ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:44.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.541+0000 7f614a8db700 1 -- 172.21.15.40:0/4065568265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61440b3d00 msgr2=0x7f614414e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:44.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.541+0000 7f614a8db700 1 --2- 172.21.15.40:0/4065568265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61440b3d00 0x7f614414e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:44.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.541+0000 7f614a8db700 1 -- 172.21.15.40:0/4065568265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c009a70 con 0x7f614415c150 2023-11-08T08:23:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.541+0000 7f614a0da700 1 --2- 172.21.15.40:0/4065568265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6144070620 0x7f61440ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:23:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.541+0000 7f614a8db700 1 --2- 172.21.15.40:0/4065568265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f614415c150 0x7f61440b8050 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f613c009f30 tx=0x7f613c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.545+0000 7f613affd700 1 -- 172.21.15.40:0/4065568265 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c01b810 con 0x7f614415c150 2023-11-08T08:23:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.545+0000 7f613affd700 1 -- 172.21.15.40:0/4065568265 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613c020070 con 0x7f614415c150 2023-11-08T08:23:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.545+0000 7f613affd700 1 -- 172.21.15.40:0/4065568265 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c01bb90 con 0x7f614415c150 2023-11-08T08:23:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.545+0000 7f614b0dc700 1 -- 172.21.15.40:0/4065568265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61440d2140 con 0x7f614415c150 2023-11-08T08:23:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.545+0000 7f614b0dc700 1 -- 172.21.15.40:0/4065568265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61440d05e0 con 0x7f614415c150 2023-11-08T08:23:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.545+0000 7f614b0dc700 1 -- 172.21.15.40:0/4065568265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6144070cd0 con 0x7f614415c150 2023-11-08T08:23:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.545+0000 7f613affd700 1 -- 172.21.15.40:0/4065568265 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f613c01bcf0 con 0x7f614415c150 2023-11-08T08:23:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.549+0000 7f613affd700 1 --2- 172.21.15.40:0/4065568265 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6128064ee0 0x7f6128067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.549+0000 7f613affd700 1 -- 172.21.15.40:0/4065568265 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f613c022070 con 0x7f614415c150 2023-11-08T08:23:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.549+0000 7f614a0da700 1 --2- 172.21.15.40:0/4065568265 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6128064ee0 0x7f6128067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.549+0000 7f614a0da700 1 --2- 172.21.15.40:0/4065568265 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6128064ee0 0x7f6128067390 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6140005930 tx=0x7f6140009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:44.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.557+0000 7f613affd700 1 -- 172.21.15.40:0/4065568265 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f613c052670 con 0x7f614415c150 2023-11-08T08:23:44.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.725+0000 7f614b0dc700 1 -- 172.21.15.40:0/4065568265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6144073d20 con 0x7f614415c150 2023-11-08T08:23:44.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.729+0000 7f613affd700 1 -- 172.21.15.40:0/4065568265 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f613c02c0a0 con 0x7f614415c150 2023-11-08T08:23:44.734 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:23:44.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.729+0000 7f614b0dc700 1 -- 172.21.15.40:0/4065568265 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6128064ee0 msgr2=0x7f6128067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:44.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.729+0000 7f614b0dc700 1 --2- 172.21.15.40:0/4065568265 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6128064ee0 0x7f6128067390 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6140005930 tx=0x7f6140009040 comp rx=0 tx=0).stop 2023-11-08T08:23:44.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.729+0000 7f614b0dc700 1 -- 172.21.15.40:0/4065568265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f614415c150 msgr2=0x7f61440b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:44.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.729+0000 7f614b0dc700 1 --2- 172.21.15.40:0/4065568265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f614415c150 0x7f61440b8050 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f613c009f30 tx=0x7f613c004af0 comp rx=0 tx=0).stop 2023-11-08T08:23:44.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.729+0000 7f614b0dc700 1 -- 172.21.15.40:0/4065568265 shutdown_connections 2023-11-08T08:23:44.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.729+0000 7f614b0dc700 1 --2- 172.21.15.40:0/4065568265 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6128064ee0 0x7f6128067390 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:44.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.729+0000 7f614b0dc700 1 --2- 172.21.15.40:0/4065568265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f614415c150 0x7f61440b8050 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:44.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.729+0000 7f614b0dc700 1 --2- 172.21.15.40:0/4065568265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61440b3d00 0x7f614414e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:44.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.729+0000 7f614b0dc700 1 --2- 172.21.15.40:0/4065568265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6144070620 0x7f61440ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:44.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.729+0000 7f614b0dc700 1 -- 172.21.15.40:0/4065568265 >> 172.21.15.40:0/4065568265 conn(0x7f61440d3680 msgr2=0x7f61440d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:44.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.729+0000 7f614b0dc700 1 -- 172.21.15.40:0/4065568265 shutdown_connections 2023-11-08T08:23:44.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.729+0000 7f614b0dc700 1 -- 172.21.15.40:0/4065568265 wait complete. 2023-11-08T08:23:44.762 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:23:44.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.933+0000 7ff32cd97700 1 Processor -- start 2023-11-08T08:23:44.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.933+0000 7ff32cd97700 1 -- start start 2023-11-08T08:23:44.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.937+0000 7ff32cd97700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff3280cc1a0 0x7ff32815d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:44.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.937+0000 7ff32cd97700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff32806ff10 0x7ff3280703b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:44.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.937+0000 7ff32cd97700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32815d650 con 0x7ff328159cd0 2023-11-08T08:23:44.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.937+0000 7ff32cd97700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3280708f0 con 0x7ff3280cc1a0 2023-11-08T08:23:44.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.937+0000 7ff32cd97700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32806be80 con 0x7ff32806ff10 2023-11-08T08:23:44.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.937+0000 7ff3277fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff328159cd0 0x7ff32806c890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56006/0 (socket says 172.21.15.40:56006) 2023-11-08T08:23:44.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.937+0000 7ff3277fe700 1 -- 172.21.15.40:0/3242432063 learned_addr learned my addr 172.21.15.40:0/3242432063 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:44.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.937+0000 7ff326ffd700 1 --2- 172.21.15.40:0/3242432063 >> v2:172.21.15.40:3300/0 conn(0x7ff3280cc1a0 0x7ff32815d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:44.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.937+0000 7ff327fff700 1 --2- 172.21.15.40:0/3242432063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff32806ff10 0x7ff3280703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:44.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.937+0000 7ff3267fc700 1 -- 172.21.15.40:0/3242432063 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619421259 0 0) 0x7ff32815d650 con 0x7ff328159cd0 2023-11-08T08:23:44.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.937+0000 7ff3267fc700 1 -- 172.21.15.40:0/3242432063 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3100035d0 con 0x7ff328159cd0 2023-11-08T08:23:44.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.937+0000 7ff327fff700 1 -- 172.21.15.40:0/3242432063 >> v2:172.21.15.40:3300/0 conn(0x7ff3280cc1a0 msgr2=0x7ff32815d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:44.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.937+0000 7ff327fff700 1 --2- 172.21.15.40:0/3242432063 >> v2:172.21.15.40:3300/0 conn(0x7ff3280cc1a0 0x7ff32815d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:44.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.937+0000 7ff327fff700 1 -- 172.21.15.40:0/3242432063 >> v1:172.21.15.40:6789/0 conn(0x7ff328159cd0 legacy=0x7ff32806c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:44.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.937+0000 7ff3267fc700 1 -- 172.21.15.40:0/3242432063 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3738248837 0 0) 0x7ff3100035d0 con 0x7ff328159cd0 2023-11-08T08:23:44.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.937+0000 7ff327fff700 1 -- 172.21.15.40:0/3242432063 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32815a0a0 con 0x7ff32806ff10 2023-11-08T08:23:44.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.937+0000 7ff326ffd700 1 --2- 172.21.15.40:0/3242432063 >> v2:172.21.15.40:3300/0 conn(0x7ff3280cc1a0 0x7ff32815d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:23:44.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.937+0000 7ff327fff700 1 --2- 172.21.15.40:0/3242432063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff32806ff10 0x7ff3280703b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff320009d90 tx=0x7ff32000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28996d5a3f4f56f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:44.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.941+0000 7ff3267fc700 1 -- 172.21.15.40:0/3242432063 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32000e070 con 0x7ff32806ff10 2023-11-08T08:23:44.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.941+0000 7ff3267fc700 1 -- 172.21.15.40:0/3242432063 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff320003a40 con 0x7ff32806ff10 2023-11-08T08:23:44.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.941+0000 7ff3267fc700 1 -- 172.21.15.40:0/3242432063 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318001690 con 0x7ff32806ff10 2023-11-08T08:23:44.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.941+0000 7ff32cd97700 1 -- 172.21.15.40:0/3242432063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff32806ff10 msgr2=0x7ff3280703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:44.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.941+0000 7ff32cd97700 1 --2- 172.21.15.40:0/3242432063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff32806ff10 0x7ff3280703b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff320009d90 tx=0x7ff32000b330 comp rx=0 tx=0).stop 2023-11-08T08:23:44.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.941+0000 7ff32cd97700 1 -- 172.21.15.40:0/3242432063 shutdown_connections 2023-11-08T08:23:44.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.941+0000 7ff32cd97700 1 --2- 172.21.15.40:0/3242432063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff32806ff10 0x7ff3280703b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:44.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.941+0000 7ff32cd97700 1 --2- 172.21.15.40:0/3242432063 >> v2:172.21.15.40:3300/0 conn(0x7ff3280cc1a0 0x7ff32815d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:44.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.941+0000 7ff32cd97700 1 -- 172.21.15.40:0/3242432063 >> 172.21.15.40:0/3242432063 conn(0x7ff3280cf280 msgr2=0x7ff3280cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:44.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.941+0000 7ff32cd97700 1 -- 172.21.15.40:0/3242432063 shutdown_connections 2023-11-08T08:23:44.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.941+0000 7ff32cd97700 1 -- 172.21.15.40:0/3242432063 wait complete. 2023-11-08T08:23:44.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.945+0000 7ff32cd97700 1 Processor -- start 2023-11-08T08:23:44.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.945+0000 7ff32cd97700 1 -- start start 2023-11-08T08:23:44.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.945+0000 7ff32cd97700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff32806ff10 0x7ff328151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:44.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.945+0000 7ff32cd97700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3280cc1a0 0x7ff3281545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:44.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.945+0000 7ff32cd97700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff328159cd0 0x7ff328142ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:44.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.945+0000 7ff32cd97700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff328154b20 con 0x7ff3280cc1a0 2023-11-08T08:23:44.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.945+0000 7ff32cd97700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff328154d10 con 0x7ff328159cd0 2023-11-08T08:23:44.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.945+0000 7ff32cd97700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff328154ff0 con 0x7ff32806ff10 2023-11-08T08:23:44.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.945+0000 7ff326ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3280cc1a0 0x7ff3281545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:44.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.945+0000 7ff326ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3280cc1a0 0x7ff3281545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39418/0 (socket says 172.21.15.40:39418) 2023-11-08T08:23:44.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.945+0000 7ff326ffd700 1 -- 172.21.15.40:0/2844431099 learned_addr learned my addr 172.21.15.40:0/2844431099 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:44.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.949+0000 7ff3277fe700 1 --2- 172.21.15.40:0/2844431099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff32806ff10 0x7ff328151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:44.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.949+0000 7ff327fff700 1 --2- 172.21.15.40:0/2844431099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff328159cd0 0x7ff328142ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:44.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.949+0000 7ff326ffd700 1 -- 172.21.15.40:0/2844431099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff32806ff10 msgr2=0x7ff328151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:44.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.949+0000 7ff326ffd700 1 --2- 172.21.15.40:0/2844431099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff32806ff10 0x7ff328151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:44.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.949+0000 7ff326ffd700 1 -- 172.21.15.40:0/2844431099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff328159cd0 msgr2=0x7ff328142ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:44.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.949+0000 7ff326ffd700 1 --2- 172.21.15.40:0/2844431099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff328159cd0 0x7ff328142ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:44.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.949+0000 7ff326ffd700 1 -- 172.21.15.40:0/2844431099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31c0094b0 con 0x7ff3280cc1a0 2023-11-08T08:23:44.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.949+0000 7ff3277fe700 1 --2- 172.21.15.40:0/2844431099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff32806ff10 0x7ff328151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:23:44.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.949+0000 7ff326ffd700 1 --2- 172.21.15.40:0/2844431099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3280cc1a0 0x7ff3281545e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00ebe0 tx=0x7ff31c009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:44.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.949+0000 7ff30ffff700 1 -- 172.21.15.40:0/2844431099 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c00c910 con 0x7ff3280cc1a0 2023-11-08T08:23:44.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.949+0000 7ff30ffff700 1 -- 172.21.15.40:0/2844431099 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31c00ca70 con 0x7ff3280cc1a0 2023-11-08T08:23:44.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.949+0000 7ff32cd97700 1 -- 172.21.15.40:0/2844431099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff320009a70 con 0x7ff3280cc1a0 2023-11-08T08:23:44.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.949+0000 7ff30ffff700 1 -- 172.21.15.40:0/2844431099 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c00ce00 con 0x7ff3280cc1a0 2023-11-08T08:23:44.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.949+0000 7ff32cd97700 1 -- 172.21.15.40:0/2844431099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3280ce260 con 0x7ff3280cc1a0 2023-11-08T08:23:44.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.949+0000 7ff32cd97700 1 -- 172.21.15.40:0/2844431099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3280cdf80 con 0x7ff3280cc1a0 2023-11-08T08:23:44.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.953+0000 7ff30ffff700 1 -- 172.21.15.40:0/2844431099 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff31c019030 con 0x7ff3280cc1a0 2023-11-08T08:23:44.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.953+0000 7ff30ffff700 1 --2- 172.21.15.40:0/2844431099 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff314064ee0 0x7ff314067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:44.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.953+0000 7ff30ffff700 1 -- 172.21.15.40:0/2844431099 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7ff31c082250 con 0x7ff3280cc1a0 2023-11-08T08:23:44.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.953+0000 7ff3277fe700 1 --2- 172.21.15.40:0/2844431099 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff314064ee0 0x7ff314067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:44.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.957+0000 7ff3277fe700 1 --2- 172.21.15.40:0/2844431099 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff314064ee0 0x7ff314067390 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff318001200 tx=0x7ff318007b00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:44.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:44.961+0000 7ff30ffff700 1 -- 172.21.15.40:0/2844431099 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ff31c0500a0 con 0x7ff3280cc1a0 2023-11-08T08:23:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.133+0000 7ff32cd97700 1 -- 172.21.15.40:0/2844431099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff3280cca80 con 0x7ff3280cc1a0 2023-11-08T08:23:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.133+0000 7ff30ffff700 1 -- 172.21.15.40:0/2844431099 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v71) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff31c0536c0 con 0x7ff3280cc1a0 2023-11-08T08:23:45.141 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:23:45.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.137+0000 7ff32cd97700 1 -- 172.21.15.40:0/2844431099 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff314064ee0 msgr2=0x7ff314067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:45.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.137+0000 7ff32cd97700 1 --2- 172.21.15.40:0/2844431099 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff314064ee0 0x7ff314067390 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff318001200 tx=0x7ff318007b00 comp rx=0 tx=0).stop 2023-11-08T08:23:45.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.137+0000 7ff32cd97700 1 -- 172.21.15.40:0/2844431099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3280cc1a0 msgr2=0x7ff3281545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:45.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.137+0000 7ff32cd97700 1 --2- 172.21.15.40:0/2844431099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3280cc1a0 0x7ff3281545e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00ebe0 tx=0x7ff31c009ed0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.137+0000 7ff32cd97700 1 -- 172.21.15.40:0/2844431099 shutdown_connections 2023-11-08T08:23:45.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.137+0000 7ff32cd97700 1 --2- 172.21.15.40:0/2844431099 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff314064ee0 0x7ff314067390 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.137+0000 7ff32cd97700 1 --2- 172.21.15.40:0/2844431099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff328159cd0 0x7ff328142ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.137+0000 7ff32cd97700 1 --2- 172.21.15.40:0/2844431099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3280cc1a0 0x7ff3281545e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.137+0000 7ff32cd97700 1 --2- 172.21.15.40:0/2844431099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff32806ff10 0x7ff328151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.137+0000 7ff32cd97700 1 -- 172.21.15.40:0/2844431099 >> 172.21.15.40:0/2844431099 conn(0x7ff3280cf280 msgr2=0x7ff3280b0730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:45.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.137+0000 7ff32cd97700 1 -- 172.21.15.40:0/2844431099 shutdown_connections 2023-11-08T08:23:45.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.141+0000 7ff32cd97700 1 -- 172.21.15.40:0/2844431099 wait complete. 2023-11-08T08:23:45.169 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T08:23:45.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.349+0000 7fc789386700 1 Processor -- start 2023-11-08T08:23:45.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.349+0000 7fc789386700 1 -- start start 2023-11-08T08:23:45.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.349+0000 7fc789386700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc78415c150 0x7fc7840b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:45.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.349+0000 7fc789386700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc78406ff10 0x7fc784070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:45.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc789386700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc78415c520 con 0x7fc78415fc30 2023-11-08T08:23:45.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc789386700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7840b4240 con 0x7fc78415c150 2023-11-08T08:23:45.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc789386700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7840708a0 con 0x7fc78406ff10 2023-11-08T08:23:45.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc782ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc78415c150 0x7fc7840b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:45.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc782ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc78415c150 0x7fc7840b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39422/0 (socket says 172.21.15.40:39422) 2023-11-08T08:23:45.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc782ffd700 1 -- 172.21.15.40:0/2978154233 learned_addr learned my addr 172.21.15.40:0/2978154233 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:45.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc782ffd700 1 -- 172.21.15.40:0/2978154233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc78406ff10 msgr2=0x7fc784070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:45.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc7827fc700 1 --2- 172.21.15.40:0/2978154233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc78406ff10 0x7fc784070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:45.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc782ffd700 1 --2- 172.21.15.40:0/2978154233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc78406ff10 0x7fc784070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc782ffd700 1 -- 172.21.15.40:0/2978154233 >> v1:172.21.15.169:6789/0 conn(0x7fc78415fc30 legacy=0x7fc7841600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:45.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc782ffd700 1 -- 172.21.15.40:0/2978154233 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78406ac00 con 0x7fc78415c150 2023-11-08T08:23:45.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc7827fc700 1 --2- 172.21.15.40:0/2978154233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc78406ff10 0x7fc784070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:23:45.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc782ffd700 1 --2- 172.21.15.40:0/2978154233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc78415c150 0x7fc7840b3d00 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc774009910 tx=0x7fc774009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d1abbd1980c772 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:45.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc781ffb700 1 -- 172.21.15.40:0/2978154233 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774004230 con 0x7fc78415c150 2023-11-08T08:23:45.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc781ffb700 1 -- 172.21.15.40:0/2978154233 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc774004390 con 0x7fc78415c150 2023-11-08T08:23:45.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc781ffb700 1 -- 172.21.15.40:0/2978154233 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774009e60 con 0x7fc78415c150 2023-11-08T08:23:45.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc789386700 1 -- 172.21.15.40:0/2978154233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc78415c150 msgr2=0x7fc7840b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:45.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc789386700 1 --2- 172.21.15.40:0/2978154233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc78415c150 0x7fc7840b3d00 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc774009910 tx=0x7fc774009c20 comp rx=0 tx=0).stop 2023-11-08T08:23:45.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc789386700 1 -- 172.21.15.40:0/2978154233 shutdown_connections 2023-11-08T08:23:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc789386700 1 --2- 172.21.15.40:0/2978154233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc78406ff10 0x7fc784070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc789386700 1 --2- 172.21.15.40:0/2978154233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc78415c150 0x7fc7840b3d00 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc789386700 1 -- 172.21.15.40:0/2978154233 >> 172.21.15.40:0/2978154233 conn(0x7fc7840d3680 msgr2=0x7fc7840cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc789386700 1 -- 172.21.15.40:0/2978154233 shutdown_connections 2023-11-08T08:23:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.353+0000 7fc789386700 1 -- 172.21.15.40:0/2978154233 wait complete. 2023-11-08T08:23:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.357+0000 7fc789386700 1 Processor -- start 2023-11-08T08:23:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.357+0000 7fc789386700 1 -- start start 2023-11-08T08:23:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.357+0000 7fc789386700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc78406ff10 0x7fc7841564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.357+0000 7fc789386700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc78415c150 0x7fc7840ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.357+0000 7fc789386700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc78415fc30 0x7fc7840cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.357+0000 7fc789386700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7840c9a40 con 0x7fc78415fc30 2023-11-08T08:23:45.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.357+0000 7fc789386700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7840cd8a0 con 0x7fc78415c150 2023-11-08T08:23:45.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.357+0000 7fc789386700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7840cdb80 con 0x7fc78406ff10 2023-11-08T08:23:45.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.357+0000 7fc7837fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc78415fc30 0x7fc7840cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:45.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.357+0000 7fc7837fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc78415fc30 0x7fc7840cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39434/0 (socket says 172.21.15.40:39434) 2023-11-08T08:23:45.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.357+0000 7fc7837fe700 1 -- 172.21.15.40:0/2094001341 learned_addr learned my addr 172.21.15.40:0/2094001341 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.357+0000 7fc7827fc700 1 --2- 172.21.15.40:0/2094001341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc78415c150 0x7fc7840ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.357+0000 7fc782ffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc78406ff10 0x7fc7841564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.361+0000 7fc7837fe700 1 -- 172.21.15.40:0/2094001341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc78406ff10 msgr2=0x7fc7841564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.361+0000 7fc7837fe700 1 --2- 172.21.15.40:0/2094001341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc78406ff10 0x7fc7841564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.361+0000 7fc7837fe700 1 -- 172.21.15.40:0/2094001341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc78415c150 msgr2=0x7fc7840ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.361+0000 7fc7837fe700 1 --2- 172.21.15.40:0/2094001341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc78415c150 0x7fc7840ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.361+0000 7fc7837fe700 1 -- 172.21.15.40:0/2094001341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc774009570 con 0x7fc78415fc30 2023-11-08T08:23:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.361+0000 7fc7837fe700 1 --2- 172.21.15.40:0/2094001341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc78415fc30 0x7fc7840cbda0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00b970 tx=0x7fc77c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.361+0000 7fc76b7fe700 1 -- 172.21.15.40:0/2094001341 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c00d860 con 0x7fc78415fc30 2023-11-08T08:23:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.361+0000 7fc76b7fe700 1 -- 172.21.15.40:0/2094001341 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c00d9c0 con 0x7fc78415fc30 2023-11-08T08:23:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.361+0000 7fc76b7fe700 1 -- 172.21.15.40:0/2094001341 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c00dd70 con 0x7fc78415fc30 2023-11-08T08:23:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.361+0000 7fc789386700 1 -- 172.21.15.40:0/2094001341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7840d2450 con 0x7fc78415fc30 2023-11-08T08:23:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.361+0000 7fc789386700 1 -- 172.21.15.40:0/2094001341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc78406cc50 con 0x7fc78415fc30 2023-11-08T08:23:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.361+0000 7fc789386700 1 -- 172.21.15.40:0/2094001341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7840d2730 con 0x7fc78415fc30 2023-11-08T08:23:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.365+0000 7fc76b7fe700 1 -- 172.21.15.40:0/2094001341 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fc77c00e430 con 0x7fc78415fc30 2023-11-08T08:23:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.365+0000 7fc76b7fe700 1 --2- 172.21.15.40:0/2094001341 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc770064ee0 0x7fc770067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.365+0000 7fc76b7fe700 1 -- 172.21.15.40:0/2094001341 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fc77c082350 con 0x7fc78415fc30 2023-11-08T08:23:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.365+0000 7fc782ffd700 1 --2- 172.21.15.40:0/2094001341 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc770064ee0 0x7fc770067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.365+0000 7fc782ffd700 1 --2- 172.21.15.40:0/2094001341 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc770064ee0 0x7fc770067390 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc774009910 tx=0x7fc774008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:45.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.373+0000 7fc76b7fe700 1 -- 172.21.15.40:0/2094001341 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fc77c050220 con 0x7fc78415fc30 2023-11-08T08:23:45.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.549+0000 7fc789386700 1 -- 172.21.15.40:0/2094001341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc784073d20 con 0x7fc78415fc30 2023-11-08T08:23:45.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.549+0000 7fc76b7fe700 1 -- 172.21.15.40:0/2094001341 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc77c053840 con 0x7fc78415fc30 2023-11-08T08:23:45.556 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:23:45.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.553+0000 7fc789386700 1 -- 172.21.15.40:0/2094001341 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc770064ee0 msgr2=0x7fc770067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:45.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.553+0000 7fc789386700 1 --2- 172.21.15.40:0/2094001341 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc770064ee0 0x7fc770067390 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc774009910 tx=0x7fc774008030 comp rx=0 tx=0).stop 2023-11-08T08:23:45.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.553+0000 7fc789386700 1 -- 172.21.15.40:0/2094001341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc78415fc30 msgr2=0x7fc7840cbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:45.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.553+0000 7fc789386700 1 --2- 172.21.15.40:0/2094001341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc78415fc30 0x7fc7840cbda0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00b970 tx=0x7fc77c00bc80 comp rx=0 tx=0).stop 2023-11-08T08:23:45.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.553+0000 7fc789386700 1 -- 172.21.15.40:0/2094001341 shutdown_connections 2023-11-08T08:23:45.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.553+0000 7fc789386700 1 --2- 172.21.15.40:0/2094001341 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fc770064ee0 0x7fc770067390 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.553+0000 7fc789386700 1 --2- 172.21.15.40:0/2094001341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc78415fc30 0x7fc7840cbda0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.553+0000 7fc789386700 1 --2- 172.21.15.40:0/2094001341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc78415c150 0x7fc7840ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.553+0000 7fc789386700 1 --2- 172.21.15.40:0/2094001341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc78406ff10 0x7fc7841564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.553+0000 7fc789386700 1 -- 172.21.15.40:0/2094001341 >> 172.21.15.40:0/2094001341 conn(0x7fc7840d3680 msgr2=0x7fc7840c9030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:45.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.553+0000 7fc789386700 1 -- 172.21.15.40:0/2094001341 shutdown_connections 2023-11-08T08:23:45.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.553+0000 7fc789386700 1 -- 172.21.15.40:0/2094001341 wait complete. 2023-11-08T08:23:45.580 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T08:23:45.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.765+0000 7f75e5253700 1 Processor -- start 2023-11-08T08:23:45.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.765+0000 7f75e5253700 1 -- start start 2023-11-08T08:23:45.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75e5253700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f75e00b22e0 0x7f75e00ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:45.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75e5253700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75e006ad00 0x7f75e015d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:45.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75e5253700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e006f040 con 0x7f75e00b26b0 2023-11-08T08:23:45.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75e5253700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e006cca0 con 0x7f75e00b22e0 2023-11-08T08:23:45.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75e5253700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e015d670 con 0x7f75e006ad00 2023-11-08T08:23:45.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75e4a52700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f75e00b26b0 0x7f75e006fd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56008/0 (socket says 172.21.15.40:56008) 2023-11-08T08:23:45.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75e4a52700 1 -- 172.21.15.40:0/455387955 learned_addr learned my addr 172.21.15.40:0/455387955 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:45.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75d7fff700 1 --2- 172.21.15.40:0/455387955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75e006ad00 0x7f75e015d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:45.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75dffff700 1 --2- 172.21.15.40:0/455387955 >> v2:172.21.15.169:3300/0 conn(0x7f75e00b22e0 0x7f75e00ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:45.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75df7fe700 1 -- 172.21.15.40:0/455387955 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706897148 0 0) 0x7f75e006f040 con 0x7f75e00b26b0 2023-11-08T08:23:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75df7fe700 1 -- 172.21.15.40:0/455387955 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75cc0035d0 con 0x7f75e00b26b0 2023-11-08T08:23:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75df7fe700 1 -- 172.21.15.40:0/455387955 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2561725051 0 0) 0x7f75cc0035d0 con 0x7f75e00b26b0 2023-11-08T08:23:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75df7fe700 1 -- 172.21.15.40:0/455387955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75e006ad00 msgr2=0x7f75e015d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75df7fe700 1 --2- 172.21.15.40:0/455387955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75e006ad00 0x7f75e015d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75df7fe700 1 -- 172.21.15.40:0/455387955 >> v2:172.21.15.169:3300/0 conn(0x7f75e00b22e0 msgr2=0x7f75e00ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:45.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75df7fe700 1 --2- 172.21.15.40:0/455387955 >> v2:172.21.15.169:3300/0 conn(0x7f75e00b22e0 0x7f75e00ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75df7fe700 1 -- 172.21.15.40:0/455387955 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e00cffc0 con 0x7f75e00b26b0 2023-11-08T08:23:45.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75df7fe700 1 -- 172.21.15.40:0/455387955 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f75d80019f0 con 0x7f75e00b26b0 2023-11-08T08:23:45.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75d7fff700 1 --2- 172.21.15.40:0/455387955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75e006ad00 0x7f75e015d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:23:45.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75df7fe700 1 -- 172.21.15.40:0/455387955 >> v1:172.21.15.40:6789/0 conn(0x7f75e00b26b0 legacy=0x7f75e006fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:45.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75df7fe700 1 --2- 172.21.15.40:0/455387955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75cc003a20 0x7f75cc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:45.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75df7fe700 1 --2- 172.21.15.40:0/455387955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75cc004450 0x7f75cc004910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:45.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75df7fe700 1 --2- 172.21.15.40:0/455387955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75e006ad00 0x7f75cc005040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:45.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75df7fe700 1 -- 172.21.15.40:0/455387955 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e006cca0 con 0x7f75cc003a20 2023-11-08T08:23:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75df7fe700 1 -- 172.21.15.40:0/455387955 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e015d670 con 0x7f75cc004450 2023-11-08T08:23:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75df7fe700 1 -- 172.21.15.40:0/455387955 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc00ba30 con 0x7f75e006ad00 2023-11-08T08:23:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75dffff700 1 --2- 172.21.15.40:0/455387955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75cc003a20 0x7f75cc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75d7fff700 1 --2- 172.21.15.40:0/455387955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75cc004450 0x7f75cc004910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:45.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75e4a52700 1 --2- 172.21.15.40:0/455387955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75e006ad00 0x7f75cc005040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:45.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75dffff700 1 -- 172.21.15.40:0/455387955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75e006ad00 msgr2=0x7f75cc005040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:45.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75dffff700 1 --2- 172.21.15.40:0/455387955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75e006ad00 0x7f75cc005040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75dffff700 1 -- 172.21.15.40:0/455387955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75cc004450 msgr2=0x7f75cc004910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:45.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75dffff700 1 --2- 172.21.15.40:0/455387955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75cc004450 0x7f75cc004910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.769+0000 7f75dffff700 1 -- 172.21.15.40:0/455387955 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c80094b0 con 0x7f75cc003a20 2023-11-08T08:23:45.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.773+0000 7f75dffff700 1 --2- 172.21.15.40:0/455387955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75cc003a20 0x7f75cc003e30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009b70 tx=0x7f75cc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e845630ca5dc72f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:45.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.773+0000 7f75df7fe700 1 -- 172.21.15.40:0/455387955 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d000e6c0 con 0x7f75cc003a20 2023-11-08T08:23:45.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.773+0000 7f75df7fe700 1 -- 172.21.15.40:0/455387955 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d000e820 con 0x7f75cc003a20 2023-11-08T08:23:45.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.773+0000 7f75df7fe700 1 -- 172.21.15.40:0/455387955 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d000eb90 con 0x7f75cc003a20 2023-11-08T08:23:45.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.773+0000 7f75e5253700 1 -- 172.21.15.40:0/455387955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75cc003a20 msgr2=0x7f75cc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:45.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.773+0000 7f75e5253700 1 --2- 172.21.15.40:0/455387955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75cc003a20 0x7f75cc003e30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009b70 tx=0x7f75cc0058a0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.773+0000 7f75e4a52700 1 -- 172.21.15.40:0/455387955 reap_dead start 2023-11-08T08:23:45.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.773+0000 7f75e5253700 1 -- 172.21.15.40:0/455387955 shutdown_connections 2023-11-08T08:23:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.773+0000 7f75e5253700 1 -- 172.21.15.40:0/455387955 >> 172.21.15.40:0/455387955 conn(0x7f75e00cd2e0 msgr2=0x7f75e00cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.773+0000 7f75e5253700 1 -- 172.21.15.40:0/455387955 shutdown_connections 2023-11-08T08:23:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.773+0000 7f75e5253700 1 -- 172.21.15.40:0/455387955 wait complete. 2023-11-08T08:23:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.777+0000 7f75e5253700 1 Processor -- start 2023-11-08T08:23:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.777+0000 7f75e5253700 1 -- start start 2023-11-08T08:23:45.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.777+0000 7f75e5253700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75cc003a20 0x7f75e015e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:45.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.777+0000 7f75e5253700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75e00c7930 0x7f75e00c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:45.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.777+0000 7f75e5253700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e00c46e0 0x7f75e00c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:45.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.777+0000 7f75e5253700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e00c8cb0 con 0x7f75e00c46e0 2023-11-08T08:23:45.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.777+0000 7f75e5253700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e0157540 con 0x7f75e00c7930 2023-11-08T08:23:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.777+0000 7f75e5253700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e00c80a0 con 0x7f75cc003a20 2023-11-08T08:23:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.777+0000 7f75e4a52700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e00c46e0 0x7f75e00c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.777+0000 7f75e4a52700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e00c46e0 0x7f75e00c4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39458/0 (socket says 172.21.15.40:39458) 2023-11-08T08:23:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.777+0000 7f75e4a52700 1 -- 172.21.15.40:0/4091942196 learned_addr learned my addr 172.21.15.40:0/4091942196 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.777+0000 7f75dffff700 1 --2- 172.21.15.40:0/4091942196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75cc003a20 0x7f75e015e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:45.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.777+0000 7f75d7fff700 1 --2- 172.21.15.40:0/4091942196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75e00c7930 0x7f75e00c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:45.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.781+0000 7f75e4a52700 1 -- 172.21.15.40:0/4091942196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75cc003a20 msgr2=0x7f75e015e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:45.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.781+0000 7f75e4a52700 1 --2- 172.21.15.40:0/4091942196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75cc003a20 0x7f75e015e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.781+0000 7f75e4a52700 1 -- 172.21.15.40:0/4091942196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75e00c7930 msgr2=0x7f75e00c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:45.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.781+0000 7f75e4a52700 1 --2- 172.21.15.40:0/4091942196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75e00c7930 0x7f75e00c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.781+0000 7f75e4a52700 1 -- 172.21.15.40:0/4091942196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e006f320 con 0x7f75e00c46e0 2023-11-08T08:23:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.781+0000 7f75d7fff700 1 --2- 172.21.15.40:0/4091942196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75e00c7930 0x7f75e00c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:23:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.781+0000 7f75e4a52700 1 --2- 172.21.15.40:0/4091942196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e00c46e0 0x7f75e00c4b80 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f75e00ce3e0 tx=0x7f75d8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.781+0000 7f75dd7fa700 1 -- 172.21.15.40:0/4091942196 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d800aa00 con 0x7f75e00c46e0 2023-11-08T08:23:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.781+0000 7f75dd7fa700 1 -- 172.21.15.40:0/4091942196 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d800ab60 con 0x7f75e00c46e0 2023-11-08T08:23:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.781+0000 7f75dd7fa700 1 -- 172.21.15.40:0/4091942196 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d80133f0 con 0x7f75e00c46e0 2023-11-08T08:23:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.781+0000 7f75e5253700 1 -- 172.21.15.40:0/4091942196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e00d02a0 con 0x7f75e00c46e0 2023-11-08T08:23:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.781+0000 7f75e5253700 1 -- 172.21.15.40:0/4091942196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e00b2b50 con 0x7f75e00c46e0 2023-11-08T08:23:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.781+0000 7f75e5253700 1 -- 172.21.15.40:0/4091942196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e00c8e60 con 0x7f75e00c46e0 2023-11-08T08:23:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.785+0000 7f75dd7fa700 1 -- 172.21.15.40:0/4091942196 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f75d8013550 con 0x7f75e00c46e0 2023-11-08T08:23:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.785+0000 7f75dd7fa700 1 --2- 172.21.15.40:0/4091942196 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f75c0064f10 0x7f75c00673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.785+0000 7f75dd7fa700 1 -- 172.21.15.40:0/4091942196 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f75d80828b0 con 0x7f75e00c46e0 2023-11-08T08:23:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.785+0000 7f75dffff700 1 --2- 172.21.15.40:0/4091942196 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f75c0064f10 0x7f75c00673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.789+0000 7f75dffff700 1 --2- 172.21.15.40:0/4091942196 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f75c0064f10 0x7f75c00673c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f75d00090b0 tx=0x7f75d001e040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:45.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.793+0000 7f75dd7fa700 1 -- 172.21.15.40:0/4091942196 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f75e00c8e60 con 0x7f75e00c46e0 2023-11-08T08:23:45.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.973+0000 7f75e5253700 1 -- 172.21.15.40:0/4091942196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f75e00c8e60 con 0x7f75e00c46e0 2023-11-08T08:23:45.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.973+0000 7f75dd7fa700 1 -- 172.21.15.40:0/4091942196 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f75e00c8e60 con 0x7f75e00c46e0 2023-11-08T08:23:45.981 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-11-08T08:23:45.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.977+0000 7f75e5253700 1 -- 172.21.15.40:0/4091942196 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f75c0064f10 msgr2=0x7f75c00673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:45.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.977+0000 7f75e5253700 1 --2- 172.21.15.40:0/4091942196 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f75c0064f10 0x7f75c00673c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f75d00090b0 tx=0x7f75d001e040 comp rx=0 tx=0).stop 2023-11-08T08:23:45.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.977+0000 7f75e5253700 1 -- 172.21.15.40:0/4091942196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e00c46e0 msgr2=0x7f75e00c4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:45.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.977+0000 7f75e5253700 1 --2- 172.21.15.40:0/4091942196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e00c46e0 0x7f75e00c4b80 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f75e00ce3e0 tx=0x7f75d8001a00 comp rx=0 tx=0).stop 2023-11-08T08:23:45.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.977+0000 7f75e5253700 1 -- 172.21.15.40:0/4091942196 shutdown_connections 2023-11-08T08:23:45.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.977+0000 7f75e5253700 1 --2- 172.21.15.40:0/4091942196 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f75c0064f10 0x7f75c00673c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.977+0000 7f75e5253700 1 --2- 172.21.15.40:0/4091942196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75e00c46e0 0x7f75e00c4b80 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.977+0000 7f75e5253700 1 --2- 172.21.15.40:0/4091942196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75e00c7930 0x7f75e00c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.977+0000 7f75e5253700 1 --2- 172.21.15.40:0/4091942196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75cc003a20 0x7f75e015e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:45.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.977+0000 7f75e5253700 1 -- 172.21.15.40:0/4091942196 >> 172.21.15.40:0/4091942196 conn(0x7f75e00cd2e0 msgr2=0x7f75e00cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:45.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.977+0000 7f75e5253700 1 -- 172.21.15.40:0/4091942196 shutdown_connections 2023-11-08T08:23:45.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:45.977+0000 7f75e5253700 1 -- 172.21.15.40:0/4091942196 wait complete. 2023-11-08T08:23:46.007 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T08:23:46.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.177+0000 7fce40c6f700 1 Processor -- start 2023-11-08T08:23:46.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.177+0000 7fce40c6f700 1 -- start start 2023-11-08T08:23:46.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce40c6f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce3c0b43a0 0x7fce3c15e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:46.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce40c6f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce3c15a550 con 0x7fce3c070790 2023-11-08T08:23:46.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce40c6f700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce3c06b0a0 con 0x7fce3c0b1640 2023-11-08T08:23:46.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce40c6f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c0caac0 con 0x7fce3c0b43a0 2023-11-08T08:23:46.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3bfff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fce3c070790 0x7fce3c06bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56016/0 (socket says 172.21.15.40:56016) 2023-11-08T08:23:46.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3bfff700 1 -- 172.21.15.40:0/3134926350 learned_addr learned my addr 172.21.15.40:0/3134926350 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:46.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3b7fe700 1 --2- 172.21.15.40:0/3134926350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce3c0b43a0 0x7fce3c15e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:46.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3a7fc700 1 -- 172.21.15.40:0/3134926350 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535458605 0 0) 0x7fce34001490 con 0x7fce3c070790 2023-11-08T08:23:46.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3a7fc700 1 -- 172.21.15.40:0/3134926350 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce240035d0 con 0x7fce3c070790 2023-11-08T08:23:46.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3a7fc700 1 -- 172.21.15.40:0/3134926350 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3286924677 0 0) 0x7fce240035d0 con 0x7fce3c070790 2023-11-08T08:23:46.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3a7fc700 1 -- 172.21.15.40:0/3134926350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce3c0b43a0 msgr2=0x7fce3c15e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:46.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3a7fc700 1 --2- 172.21.15.40:0/3134926350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce3c0b43a0 0x7fce3c15e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:46.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3a7fc700 1 -- 172.21.15.40:0/3134926350 >> v1:172.21.15.169:6789/0 conn(0x7fce3c0b1640 legacy=0x7fce3c070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:46.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3a7fc700 1 -- 172.21.15.40:0/3134926350 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c0cada0 con 0x7fce3c070790 2023-11-08T08:23:46.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3a7fc700 1 -- 172.21.15.40:0/3134926350 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7fce34001b80 con 0x7fce3c070790 2023-11-08T08:23:46.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3a7fc700 1 -- 172.21.15.40:0/3134926350 >> v1:172.21.15.40:6789/0 conn(0x7fce3c070790 legacy=0x7fce3c06bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:46.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3a7fc700 1 --2- 172.21.15.40:0/3134926350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce24003a50 0x7fce24003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:46.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3a7fc700 1 --2- 172.21.15.40:0/3134926350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce24004870 0x7fce24004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:46.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3a7fc700 1 --2- 172.21.15.40:0/3134926350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce24005290 0x7fce2400fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:46.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3a7fc700 1 -- 172.21.15.40:0/3134926350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c0caac0 con 0x7fce24004870 2023-11-08T08:23:46.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3a7fc700 1 -- 172.21.15.40:0/3134926350 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce24010570 con 0x7fce24003a50 2023-11-08T08:23:46.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3a7fc700 1 -- 172.21.15.40:0/3134926350 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce24010730 con 0x7fce24005290 2023-11-08T08:23:46.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3affd700 1 --2- 172.21.15.40:0/3134926350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce24004870 0x7fce24004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:46.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3bfff700 1 --2- 172.21.15.40:0/3134926350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce24005290 0x7fce2400fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:46.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3b7fe700 1 --2- 172.21.15.40:0/3134926350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce24003a50 0x7fce24003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3affd700 1 -- 172.21.15.40:0/3134926350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce24005290 msgr2=0x7fce2400fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3affd700 1 --2- 172.21.15.40:0/3134926350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce24005290 0x7fce2400fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3affd700 1 -- 172.21.15.40:0/3134926350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce24003a50 msgr2=0x7fce24003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3affd700 1 --2- 172.21.15.40:0/3134926350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce24003a50 0x7fce24003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3affd700 1 -- 172.21.15.40:0/3134926350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce240035d0 con 0x7fce24004870 2023-11-08T08:23:46.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3b7fe700 1 --2- 172.21.15.40:0/3134926350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce24003a50 0x7fce24003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:23:46.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3affd700 1 --2- 172.21.15.40:0/3134926350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce24004870 0x7fce24004ce0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fce300017e0 tx=0x7fce240058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd2bcb04979a149 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:46.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3a7fc700 1 -- 172.21.15.40:0/3134926350 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce300052d0 con 0x7fce24004870 2023-11-08T08:23:46.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3a7fc700 1 -- 172.21.15.40:0/3134926350 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce30005430 con 0x7fce24004870 2023-11-08T08:23:46.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.181+0000 7fce3a7fc700 1 -- 172.21.15.40:0/3134926350 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce300055a0 con 0x7fce24004870 2023-11-08T08:23:46.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.185+0000 7fce40c6f700 1 -- 172.21.15.40:0/3134926350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce24004870 msgr2=0x7fce24004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:46.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.185+0000 7fce40c6f700 1 --2- 172.21.15.40:0/3134926350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce24004870 0x7fce24004ce0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fce300017e0 tx=0x7fce240058a0 comp rx=0 tx=0).stop 2023-11-08T08:23:46.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.185+0000 7fce3bfff700 1 -- 172.21.15.40:0/3134926350 reap_dead start 2023-11-08T08:23:46.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.185+0000 7fce40c6f700 1 -- 172.21.15.40:0/3134926350 shutdown_connections 2023-11-08T08:23:46.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.185+0000 7fce40c6f700 1 -- 172.21.15.40:0/3134926350 >> 172.21.15.40:0/3134926350 conn(0x7fce3c0b0f80 msgr2=0x7fce3c0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:46.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.185+0000 7fce40c6f700 1 -- 172.21.15.40:0/3134926350 shutdown_connections 2023-11-08T08:23:46.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.185+0000 7fce40c6f700 1 -- 172.21.15.40:0/3134926350 wait complete. 2023-11-08T08:23:46.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.189+0000 7fce40c6f700 1 Processor -- start 2023-11-08T08:23:46.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.189+0000 7fce40c6f700 1 -- start start 2023-11-08T08:23:46.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.189+0000 7fce40c6f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce24004870 0x7fce3c15bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:46.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.189+0000 7fce40c6f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce3c157440 0x7fce3c0c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:46.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.189+0000 7fce40c6f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce3c156ce0 0x7fce3c0c5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:46.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.189+0000 7fce40c6f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c0c8a00 con 0x7fce3c156ce0 2023-11-08T08:23:46.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.189+0000 7fce40c6f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c0c7080 con 0x7fce3c157440 2023-11-08T08:23:46.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.189+0000 7fce40c6f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c0c67e0 con 0x7fce24004870 2023-11-08T08:23:46.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.189+0000 7fce3bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce3c156ce0 0x7fce3c0c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:46.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.189+0000 7fce3bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce3c156ce0 0x7fce3c0c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39470/0 (socket says 172.21.15.40:39470) 2023-11-08T08:23:46.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.189+0000 7fce3bfff700 1 -- 172.21.15.40:0/3135781931 learned_addr learned my addr 172.21.15.40:0/3135781931 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:46.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.189+0000 7fce3b7fe700 1 --2- 172.21.15.40:0/3135781931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce24004870 0x7fce3c15bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:46.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.189+0000 7fce3affd700 1 --2- 172.21.15.40:0/3135781931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce3c157440 0x7fce3c0c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:46.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.189+0000 7fce3bfff700 1 -- 172.21.15.40:0/3135781931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce24004870 msgr2=0x7fce3c15bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:46.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.189+0000 7fce3bfff700 1 --2- 172.21.15.40:0/3135781931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce24004870 0x7fce3c15bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:46.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.189+0000 7fce3bfff700 1 -- 172.21.15.40:0/3135781931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce3c157440 msgr2=0x7fce3c0c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:46.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.189+0000 7fce3bfff700 1 --2- 172.21.15.40:0/3135781931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce3c157440 0x7fce3c0c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:46.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.189+0000 7fce3bfff700 1 -- 172.21.15.40:0/3135781931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c06b5d0 con 0x7fce3c156ce0 2023-11-08T08:23:46.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.193+0000 7fce3bfff700 1 --2- 172.21.15.40:0/3135781931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce3c156ce0 0x7fce3c0c5820 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fce3c15db70 tx=0x7fce34007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:46.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.193+0000 7fce23fff700 1 -- 172.21.15.40:0/3135781931 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3400e8f0 con 0x7fce3c156ce0 2023-11-08T08:23:46.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.193+0000 7fce23fff700 1 -- 172.21.15.40:0/3135781931 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3400ea70 con 0x7fce3c156ce0 2023-11-08T08:23:46.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.193+0000 7fce40c6f700 1 -- 172.21.15.40:0/3135781931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3c06b2f0 con 0x7fce3c156ce0 2023-11-08T08:23:46.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.193+0000 7fce23fff700 1 -- 172.21.15.40:0/3135781931 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3400ee20 con 0x7fce3c156ce0 2023-11-08T08:23:46.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.193+0000 7fce40c6f700 1 -- 172.21.15.40:0/3135781931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3c0ca7b0 con 0x7fce3c156ce0 2023-11-08T08:23:46.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.193+0000 7fce40c6f700 1 -- 172.21.15.40:0/3135781931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3c073cb0 con 0x7fce3c156ce0 2023-11-08T08:23:46.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.197+0000 7fce23fff700 1 -- 172.21.15.40:0/3135781931 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fce3401d460 con 0x7fce3c156ce0 2023-11-08T08:23:46.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.197+0000 7fce23fff700 1 --2- 172.21.15.40:0/3135781931 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fce28064f10 0x7fce280673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:46.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.197+0000 7fce23fff700 1 -- 172.21.15.40:0/3135781931 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fce340822a0 con 0x7fce3c156ce0 2023-11-08T08:23:46.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.197+0000 7fce3b7fe700 1 --2- 172.21.15.40:0/3135781931 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fce28064f10 0x7fce280673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:46.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.197+0000 7fce3b7fe700 1 --2- 172.21.15.40:0/3135781931 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fce28064f10 0x7fce280673c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0ad9a0 tx=0x7fce2c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:46.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.205+0000 7fce23fff700 1 -- 172.21.15.40:0/3135781931 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fce34050170 con 0x7fce3c156ce0 2023-11-08T08:23:46.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.389+0000 7fce40c6f700 1 -- 172.21.15.40:0/3135781931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fce3c073200 con 0x7fce3c156ce0 2023-11-08T08:23:46.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.389+0000 7fce23fff700 1 -- 172.21.15.40:0/3135781931 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) v1 ==== 80+0+0 (secure 0 0 0) 0x7fce3c073200 con 0x7fce3c156ce0 2023-11-08T08:23:46.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.393+0000 7fce40c6f700 1 -- 172.21.15.40:0/3135781931 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fce28064f10 msgr2=0x7fce280673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:46.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.393+0000 7fce40c6f700 1 --2- 172.21.15.40:0/3135781931 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fce28064f10 0x7fce280673c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0ad9a0 tx=0x7fce2c008040 comp rx=0 tx=0).stop 2023-11-08T08:23:46.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.393+0000 7fce40c6f700 1 -- 172.21.15.40:0/3135781931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce3c156ce0 msgr2=0x7fce3c0c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:46.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.393+0000 7fce40c6f700 1 --2- 172.21.15.40:0/3135781931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce3c156ce0 0x7fce3c0c5820 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fce3c15db70 tx=0x7fce34007c20 comp rx=0 tx=0).stop 2023-11-08T08:23:46.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.393+0000 7fce40c6f700 1 -- 172.21.15.40:0/3135781931 shutdown_connections 2023-11-08T08:23:46.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.393+0000 7fce40c6f700 1 --2- 172.21.15.40:0/3135781931 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fce28064f10 0x7fce280673c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:46.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.393+0000 7fce40c6f700 1 --2- 172.21.15.40:0/3135781931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce3c156ce0 0x7fce3c0c5820 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:46.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.393+0000 7fce40c6f700 1 --2- 172.21.15.40:0/3135781931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce3c157440 0x7fce3c0c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:46.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.393+0000 7fce40c6f700 1 --2- 172.21.15.40:0/3135781931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce24004870 0x7fce3c15bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:46.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.393+0000 7fce40c6f700 1 -- 172.21.15.40:0/3135781931 >> 172.21.15.40:0/3135781931 conn(0x7fce3c0b0f80 msgr2=0x7fce3c0cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:46.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.393+0000 7fce40c6f700 1 -- 172.21.15.40:0/3135781931 shutdown_connections 2023-11-08T08:23:46.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.393+0000 7fce40c6f700 1 -- 172.21.15.40:0/3135781931 wait complete. 2023-11-08T08:23:46.403 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-11-08T08:23:46.423 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-08T08:23:46.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:23:46.424 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-08T08:23:46.499 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-11-08T08:23:46.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:23:46.500 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T08:23:46.517 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:23:46.517 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:23:46.530 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352316k,nr_inodes=4088079,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1804 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T08:23:46.530 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:23:46.531 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:23:46.581 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16355412k,nr_inodes=4088853,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6645 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T08:23:46.582 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:23:46.582 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:23:46.603 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352316k,nr_inodes=4088079,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1804 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T08:23:46.603 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:23:46.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.793+0000 7fb8fd434700 1 Processor -- start 2023-11-08T08:23:46.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.793+0000 7fb8fd434700 1 -- start start 2023-11-08T08:23:46.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.793+0000 7fb8fd434700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8f80b1640 0x7fb8f8070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:46.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.793+0000 7fb8fd434700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8f80c9040 0x7fb8f80705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:46.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.793+0000 7fb8fd434700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f815a550 con 0x7fb8f80b43a0 2023-11-08T08:23:46.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.793+0000 7fb8fd434700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f806b0a0 con 0x7fb8f80b1640 2023-11-08T08:23:46.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.793+0000 7fb8fd434700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f80caac0 con 0x7fb8f80c9040 2023-11-08T08:23:46.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.793+0000 7fb8f67fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb8f80b1640 0x7fb8f8070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:46.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.793+0000 7fb8f6ffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb8f80b43a0 0x7fb8f815e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56030/0 (socket says 172.21.15.40:56030) 2023-11-08T08:23:46.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.793+0000 7fb8f67fc700 1 --2- 172.21.15.40:0/3177177437 >> v2:172.21.15.40:3300/0 conn(0x7fb8f80b1640 0x7fb8f8070080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39478/0 (socket says 172.21.15.40:39478) 2023-11-08T08:23:46.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.793+0000 7fb8f6ffd700 1 -- 172.21.15.40:0/3177177437 learned_addr learned my addr 172.21.15.40:0/3177177437 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:46.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.793+0000 7fb8f77fe700 1 --2- 172.21.15.40:0/3177177437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8f80c9040 0x7fb8f80705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:46.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.793+0000 7fb8f67fc700 1 -- 172.21.15.40:0/3177177437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8f80c9040 msgr2=0x7fb8f80705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:46.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.793+0000 7fb8f67fc700 1 --2- 172.21.15.40:0/3177177437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8f80c9040 0x7fb8f80705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:46.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.797+0000 7fb8f67fc700 1 -- 172.21.15.40:0/3177177437 >> v1:172.21.15.40:6789/0 conn(0x7fb8f80b43a0 legacy=0x7fb8f815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:46.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.797+0000 7fb8f67fc700 1 -- 172.21.15.40:0/3177177437 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f80cada0 con 0x7fb8f80b1640 2023-11-08T08:23:46.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.797+0000 7fb8f5ffb700 1 -- 172.21.15.40:0/3177177437 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3584855321 0 0) 0x7fb8e8001110 con 0x7fb8f80b43a0 2023-11-08T08:23:46.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.797+0000 7fb8f67fc700 1 --2- 172.21.15.40:0/3177177437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8f80b1640 0x7fb8f8070080 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec009850 tx=0x7fb8ec009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183c45f5d93a3aa server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:46.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.797+0000 7fb8f5ffb700 1 -- 172.21.15.40:0/3177177437 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec011830 con 0x7fb8f80b1640 2023-11-08T08:23:46.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.797+0000 7fb8f5ffb700 1 -- 172.21.15.40:0/3177177437 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec011990 con 0x7fb8f80b1640 2023-11-08T08:23:46.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.797+0000 7fb8f5ffb700 1 -- 172.21.15.40:0/3177177437 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec011d20 con 0x7fb8f80b1640 2023-11-08T08:23:46.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.797+0000 7fb8fd434700 1 -- 172.21.15.40:0/3177177437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8f80b1640 msgr2=0x7fb8f8070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:46.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.797+0000 7fb8fd434700 1 --2- 172.21.15.40:0/3177177437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8f80b1640 0x7fb8f8070080 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec009850 tx=0x7fb8ec009b60 comp rx=0 tx=0).stop 2023-11-08T08:23:46.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.797+0000 7fb8fd434700 1 -- 172.21.15.40:0/3177177437 shutdown_connections 2023-11-08T08:23:46.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.797+0000 7fb8fd434700 1 --2- 172.21.15.40:0/3177177437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8f80c9040 0x7fb8f80705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:46.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.797+0000 7fb8fd434700 1 --2- 172.21.15.40:0/3177177437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8f80b1640 0x7fb8f8070080 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:46.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.797+0000 7fb8fd434700 1 -- 172.21.15.40:0/3177177437 >> 172.21.15.40:0/3177177437 conn(0x7fb8f80b0f80 msgr2=0x7fb8f80ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:46.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.797+0000 7fb8fd434700 1 -- 172.21.15.40:0/3177177437 shutdown_connections 2023-11-08T08:23:46.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.797+0000 7fb8fd434700 1 -- 172.21.15.40:0/3177177437 wait complete. 2023-11-08T08:23:46.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.801+0000 7fb8fd434700 1 Processor -- start 2023-11-08T08:23:46.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.801+0000 7fb8fd434700 1 -- start start 2023-11-08T08:23:46.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.801+0000 7fb8fd434700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8f80b1640 0x7fb8f80af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:46.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8fd434700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8f80b43a0 0x7fb8f80afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:46.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8fd434700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8f80c9040 0x7fb8f80c2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:46.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8fd434700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f80c2240 con 0x7fb8f80b43a0 2023-11-08T08:23:46.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8fd434700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f80c2a70 con 0x7fb8f80c9040 2023-11-08T08:23:46.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8fd434700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f815b810 con 0x7fb8f80b1640 2023-11-08T08:23:46.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8f67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8f80b43a0 0x7fb8f80afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8f67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8f80b43a0 0x7fb8f80afbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39496/0 (socket says 172.21.15.40:39496) 2023-11-08T08:23:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8f67fc700 1 -- 172.21.15.40:0/1333791465 learned_addr learned my addr 172.21.15.40:0/1333791465 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8f6ffd700 1 --2- 172.21.15.40:0/1333791465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8f80b1640 0x7fb8f80af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8f77fe700 1 --2- 172.21.15.40:0/1333791465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8f80c9040 0x7fb8f80c2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8f67fc700 1 -- 172.21.15.40:0/1333791465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8f80b1640 msgr2=0x7fb8f80af670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8f67fc700 1 --2- 172.21.15.40:0/1333791465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8f80b1640 0x7fb8f80af670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8f67fc700 1 -- 172.21.15.40:0/1333791465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8f80c9040 msgr2=0x7fb8f80c2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8f67fc700 1 --2- 172.21.15.40:0/1333791465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8f80c9040 0x7fb8f80c2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8f67fc700 1 -- 172.21.15.40:0/1333791465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec0094b0 con 0x7fb8f80b43a0 2023-11-08T08:23:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8f67fc700 1 --2- 172.21.15.40:0/1333791465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8f80b43a0 0x7fb8f80afbb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec014040 tx=0x7fb8ec004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:46.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8df7fe700 1 -- 172.21.15.40:0/1333791465 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec0036c0 con 0x7fb8f80b43a0 2023-11-08T08:23:46.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8df7fe700 1 -- 172.21.15.40:0/1333791465 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec01e070 con 0x7fb8f80b43a0 2023-11-08T08:23:46.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8fd434700 1 -- 172.21.15.40:0/1333791465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f806b5d0 con 0x7fb8f80b43a0 2023-11-08T08:23:46.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8df7fe700 1 -- 172.21.15.40:0/1333791465 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec003a20 con 0x7fb8f80b43a0 2023-11-08T08:23:46.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.805+0000 7fb8fd434700 1 -- 172.21.15.40:0/1333791465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f806b2f0 con 0x7fb8f80b43a0 2023-11-08T08:23:46.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.809+0000 7fb8fd434700 1 -- 172.21.15.40:0/1333791465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f80ca7b0 con 0x7fb8f80b43a0 2023-11-08T08:23:46.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.809+0000 7fb8df7fe700 1 -- 172.21.15.40:0/1333791465 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fb8ec02a020 con 0x7fb8f80b43a0 2023-11-08T08:23:46.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.813+0000 7fb8df7fe700 1 --2- 172.21.15.40:0/1333791465 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb8e4064f10 0x7fb8e40673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:46.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.813+0000 7fb8df7fe700 1 -- 172.21.15.40:0/1333791465 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fb8ec083270 con 0x7fb8f80b43a0 2023-11-08T08:23:46.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.813+0000 7fb8f6ffd700 1 --2- 172.21.15.40:0/1333791465 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb8e4064f10 0x7fb8e40673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:46.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.813+0000 7fb8f6ffd700 1 --2- 172.21.15.40:0/1333791465 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb8e4064f10 0x7fb8e40673c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8000f60 tx=0x7fb8e8007cb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:46.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:46.813+0000 7fb8df7fe700 1 -- 172.21.15.40:0/1333791465 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fb8ec051220 con 0x7fb8f80b43a0 2023-11-08T08:23:47.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.029+0000 7fb8fd434700 1 -- 172.21.15.40:0/1333791465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8f8073cb0 con 0x7fb8f80b43a0 2023-11-08T08:23:47.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.029+0000 7fb8df7fe700 1 -- 172.21.15.40:0/1333791465 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4650 (secure 0 0 0) 0x7fb8ec054840 con 0x7fb8f80b43a0 2023-11-08T08:23:47.037 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:23:47.037 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/307473250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":307473250},{"type":"v1","addr":"172.21.15.169:6835","nonce":307473250}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6148,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/924518366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":924518366},{"type":"v1","addr":"172.21.15.169:6837","nonce":924518366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4294005934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4294005934},{"type":"v1","addr":"172.21.15.40:6833","nonce":4294005934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:22:20.650687+0000","modified":"2023-11-08T08:22:37.032476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6237":{"gid":6237,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":7,"addr":"172.21.15.40:6835/3395630689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3395630689},{"type":"v1","addr":"172.21.15.40:6835","nonce":3395630689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T08:23:47.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.033+0000 7fb8fd434700 1 -- 172.21.15.40:0/1333791465 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb8e4064f10 msgr2=0x7fb8e40673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:47.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.033+0000 7fb8fd434700 1 --2- 172.21.15.40:0/1333791465 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb8e4064f10 0x7fb8e40673c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8000f60 tx=0x7fb8e8007cb0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.033+0000 7fb8fd434700 1 -- 172.21.15.40:0/1333791465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8f80b43a0 msgr2=0x7fb8f80afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:47.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.033+0000 7fb8fd434700 1 --2- 172.21.15.40:0/1333791465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8f80b43a0 0x7fb8f80afbb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec014040 tx=0x7fb8ec004770 comp rx=0 tx=0).stop 2023-11-08T08:23:47.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.037+0000 7fb8fd434700 1 -- 172.21.15.40:0/1333791465 shutdown_connections 2023-11-08T08:23:47.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.037+0000 7fb8fd434700 1 --2- 172.21.15.40:0/1333791465 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb8e4064f10 0x7fb8e40673c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.037+0000 7fb8fd434700 1 --2- 172.21.15.40:0/1333791465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8f80c9040 0x7fb8f80c2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.037+0000 7fb8fd434700 1 --2- 172.21.15.40:0/1333791465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8f80b43a0 0x7fb8f80afbb0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.037+0000 7fb8fd434700 1 --2- 172.21.15.40:0/1333791465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8f80b1640 0x7fb8f80af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.037+0000 7fb8fd434700 1 -- 172.21.15.40:0/1333791465 >> 172.21.15.40:0/1333791465 conn(0x7fb8f80b0f80 msgr2=0x7fb8f8159cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:47.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.037+0000 7fb8fd434700 1 -- 172.21.15.40:0/1333791465 shutdown_connections 2023-11-08T08:23:47.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.037+0000 7fb8fd434700 1 -- 172.21.15.40:0/1333791465 wait complete. 2023-11-08T08:23:47.045 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 34 2023-11-08T08:23:47.065 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:23:47.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.237+0000 7f1d51399700 1 Processor -- start 2023-11-08T08:23:47.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.237+0000 7f1d51399700 1 -- start start 2023-11-08T08:23:47.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.241+0000 7f1d51399700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1d4c15c750 0x7f1d4c15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:47.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.241+0000 7f1d51399700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d4c0b1b30 0x7f1d4c0b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:47.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.241+0000 7f1d51399700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d4c15d790 con 0x7f1d4c0cba00 2023-11-08T08:23:47.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.241+0000 7f1d51399700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d4c0b2510 con 0x7f1d4c15c750 2023-11-08T08:23:47.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.241+0000 7f1d51399700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c15e070 con 0x7f1d4c0b1b30 2023-11-08T08:23:47.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.241+0000 7f1d4affd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1d4c0cba00 0x7f1d4c15d960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56044/0 (socket says 172.21.15.40:56044) 2023-11-08T08:23:47.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.241+0000 7f1d4affd700 1 -- 172.21.15.40:0/1312306089 learned_addr learned my addr 172.21.15.40:0/1312306089 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:47.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.241+0000 7f1d427fc700 1 --2- 172.21.15.40:0/1312306089 >> v2:172.21.15.169:3300/0 conn(0x7f1d4c15c750 0x7f1d4c15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:47.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.241+0000 7f1d4bfff700 1 --2- 172.21.15.40:0/1312306089 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d4c0b1b30 0x7f1d4c0b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:47.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.241+0000 7f1d4a7fc700 1 -- 172.21.15.40:0/1312306089 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4189870616 0 0) 0x7f1d4c15d790 con 0x7f1d4c0cba00 2023-11-08T08:23:47.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.241+0000 7f1d4a7fc700 1 -- 172.21.15.40:0/1312306089 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d380035d0 con 0x7f1d4c0cba00 2023-11-08T08:23:47.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.241+0000 7f1d427fc700 1 -- 172.21.15.40:0/1312306089 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d4c0b1b30 msgr2=0x7f1d4c0b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:47.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.241+0000 7f1d427fc700 1 --2- 172.21.15.40:0/1312306089 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d4c0b1b30 0x7f1d4c0b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.241+0000 7f1d427fc700 1 -- 172.21.15.40:0/1312306089 >> v1:172.21.15.40:6789/0 conn(0x7f1d4c0cba00 legacy=0x7f1d4c15d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:47.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.241+0000 7f1d427fc700 1 -- 172.21.15.40:0/1312306089 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4c15d1d0 con 0x7f1d4c15c750 2023-11-08T08:23:47.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.241+0000 7f1d427fc700 1 --2- 172.21.15.40:0/1312306089 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4c15c750 0x7f1d4c15cbd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009850 tx=0x7f1d34009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dc20e13274ccec9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:47.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.245+0000 7f1d4a7fc700 1 -- 172.21.15.40:0/1312306089 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34011830 con 0x7f1d4c15c750 2023-11-08T08:23:47.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.245+0000 7f1d4a7fc700 1 -- 172.21.15.40:0/1312306089 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d34011990 con 0x7f1d4c15c750 2023-11-08T08:23:47.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.245+0000 7f1d4a7fc700 1 -- 172.21.15.40:0/1312306089 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34011d00 con 0x7f1d4c15c750 2023-11-08T08:23:47.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.245+0000 7f1d51399700 1 -- 172.21.15.40:0/1312306089 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4c15c750 msgr2=0x7f1d4c15cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:47.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.245+0000 7f1d51399700 1 --2- 172.21.15.40:0/1312306089 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4c15c750 0x7f1d4c15cbd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009850 tx=0x7f1d34009b60 comp rx=0 tx=0).stop 2023-11-08T08:23:47.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.245+0000 7f1d51399700 1 -- 172.21.15.40:0/1312306089 shutdown_connections 2023-11-08T08:23:47.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.245+0000 7f1d51399700 1 --2- 172.21.15.40:0/1312306089 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d4c0b1b30 0x7f1d4c0b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.245+0000 7f1d51399700 1 --2- 172.21.15.40:0/1312306089 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4c15c750 0x7f1d4c15cbd0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.245+0000 7f1d51399700 1 -- 172.21.15.40:0/1312306089 >> 172.21.15.40:0/1312306089 conn(0x7f1d4c0ccb30 msgr2=0x7f1d4c0ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:47.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.245+0000 7f1d51399700 1 -- 172.21.15.40:0/1312306089 shutdown_connections 2023-11-08T08:23:47.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.245+0000 7f1d51399700 1 -- 172.21.15.40:0/1312306089 wait complete. 2023-11-08T08:23:47.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.245+0000 7f1d51399700 1 Processor -- start 2023-11-08T08:23:47.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.245+0000 7f1d51399700 1 -- start start 2023-11-08T08:23:47.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d51399700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d4c0b1b30 0x7f1d4c14d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:47.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d51399700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d4c0cba00 0x7f1d4c1f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:47.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d51399700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4c15c750 0x7f1d4c0c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:47.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d51399700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c1f9c00 con 0x7f1d4c0b1b30 2023-11-08T08:23:47.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d51399700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c1f5840 con 0x7f1d4c15c750 2023-11-08T08:23:47.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d51399700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c14ada0 con 0x7f1d4c0cba00 2023-11-08T08:23:47.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d4affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d4c0b1b30 0x7f1d4c14d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:47.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d4affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d4c0b1b30 0x7f1d4c14d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39498/0 (socket says 172.21.15.40:39498) 2023-11-08T08:23:47.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d4affd700 1 -- 172.21.15.40:0/1473470804 learned_addr learned my addr 172.21.15.40:0/1473470804 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:47.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d4bfff700 1 --2- 172.21.15.40:0/1473470804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4c15c750 0x7f1d4c0c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:47.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d427fc700 1 --2- 172.21.15.40:0/1473470804 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d4c0cba00 0x7f1d4c1f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:47.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d4affd700 1 -- 172.21.15.40:0/1473470804 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d4c0cba00 msgr2=0x7f1d4c1f5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:47.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d4affd700 1 --2- 172.21.15.40:0/1473470804 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d4c0cba00 0x7f1d4c1f5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d4affd700 1 -- 172.21.15.40:0/1473470804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4c15c750 msgr2=0x7f1d4c0c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:47.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d4affd700 1 --2- 172.21.15.40:0/1473470804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4c15c750 0x7f1d4c0c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d4affd700 1 -- 172.21.15.40:0/1473470804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d340094b0 con 0x7f1d4c0b1b30 2023-11-08T08:23:47.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d427fc700 1 --2- 172.21.15.40:0/1473470804 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d4c0cba00 0x7f1d4c1f5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:23:47.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d4affd700 1 --2- 172.21.15.40:0/1473470804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d4c0b1b30 0x7f1d4c14d8f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00c4f0 tx=0x7f1d3c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:47.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d43fff700 1 -- 172.21.15.40:0/1473470804 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c005d90 con 0x7f1d4c0b1b30 2023-11-08T08:23:47.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d43fff700 1 -- 172.21.15.40:0/1473470804 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3c0070d0 con 0x7f1d4c0b1b30 2023-11-08T08:23:47.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d51399700 1 -- 172.21.15.40:0/1473470804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d4c0cebc0 con 0x7f1d4c0b1b30 2023-11-08T08:23:47.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d43fff700 1 -- 172.21.15.40:0/1473470804 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c007440 con 0x7f1d4c0b1b30 2023-11-08T08:23:47.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.249+0000 7f1d51399700 1 -- 172.21.15.40:0/1473470804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4c15d4b0 con 0x7f1d4c0b1b30 2023-11-08T08:23:47.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.253+0000 7f1d51399700 1 -- 172.21.15.40:0/1473470804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d4c0ceea0 con 0x7f1d4c0b1b30 2023-11-08T08:23:47.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.253+0000 7f1d43fff700 1 -- 172.21.15.40:0/1473470804 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1d3c019030 con 0x7f1d4c0b1b30 2023-11-08T08:23:47.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.257+0000 7f1d43fff700 1 --2- 172.21.15.40:0/1473470804 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d2c064ee0 0x7f1d2c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:47.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.257+0000 7f1d43fff700 1 -- 172.21.15.40:0/1473470804 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f1d3c081900 con 0x7f1d4c0b1b30 2023-11-08T08:23:47.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.257+0000 7f1d427fc700 1 --2- 172.21.15.40:0/1473470804 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d2c064ee0 0x7f1d2c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:47.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.257+0000 7f1d427fc700 1 --2- 172.21.15.40:0/1473470804 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d2c064ee0 0x7f1d2c067390 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1d340090b0 tx=0x7f1d34008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:47.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.261+0000 7f1d43fff700 1 -- 172.21.15.40:0/1473470804 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1d3c04f8e0 con 0x7f1d4c0b1b30 2023-11-08T08:23:47.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.433+0000 7f1d51399700 1 -- 172.21.15.40:0/1473470804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d4c0726c0 con 0x7f1d4c0b1b30 2023-11-08T08:23:47.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.433+0000 7f1d43fff700 1 -- 172.21.15.40:0/1473470804 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f1d3c052f00 con 0x7f1d4c0b1b30 2023-11-08T08:23:47.440 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:23:47.441 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":71,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:22:46.031951+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:22:18.224463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T08:22:19.658954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.26","mappings":[{"from":6,"to":0}]},{"pgid":"11.28","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:23:47.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.437+0000 7f1d51399700 1 -- 172.21.15.40:0/1473470804 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d2c064ee0 msgr2=0x7f1d2c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:47.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.437+0000 7f1d51399700 1 --2- 172.21.15.40:0/1473470804 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d2c064ee0 0x7f1d2c067390 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1d340090b0 tx=0x7f1d34008030 comp rx=0 tx=0).stop 2023-11-08T08:23:47.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.437+0000 7f1d51399700 1 -- 172.21.15.40:0/1473470804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d4c0b1b30 msgr2=0x7f1d4c14d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:47.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.437+0000 7f1d51399700 1 --2- 172.21.15.40:0/1473470804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d4c0b1b30 0x7f1d4c14d8f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00c4f0 tx=0x7f1d3c00c800 comp rx=0 tx=0).stop 2023-11-08T08:23:47.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.437+0000 7f1d51399700 1 -- 172.21.15.40:0/1473470804 shutdown_connections 2023-11-08T08:23:47.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.437+0000 7f1d51399700 1 --2- 172.21.15.40:0/1473470804 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1d2c064ee0 0x7f1d2c067390 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.437+0000 7f1d51399700 1 --2- 172.21.15.40:0/1473470804 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4c15c750 0x7f1d4c0c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.437+0000 7f1d51399700 1 --2- 172.21.15.40:0/1473470804 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d4c0cba00 0x7f1d4c1f5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.437+0000 7f1d51399700 1 --2- 172.21.15.40:0/1473470804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d4c0b1b30 0x7f1d4c14d8f0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.437+0000 7f1d51399700 1 -- 172.21.15.40:0/1473470804 >> 172.21.15.40:0/1473470804 conn(0x7f1d4c0ccb30 msgr2=0x7f1d4c0d0770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:47.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.437+0000 7f1d51399700 1 -- 172.21.15.40:0/1473470804 shutdown_connections 2023-11-08T08:23:47.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.437+0000 7f1d51399700 1 -- 172.21.15.40:0/1473470804 wait complete. 2023-11-08T08:23:47.467 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T08:18:51.966993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-08T08:18:56.832911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T08:22:18.224463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T08:22:19.658954+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T08:23:47.468 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T08:23:47.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.641+0000 7fedce6ab700 1 Processor -- start 2023-11-08T08:23:47.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.641+0000 7fedce6ab700 1 -- start start 2023-11-08T08:23:47.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.645+0000 7fedce6ab700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fedc80cdc50 0x7fedc8069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:47.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.645+0000 7fedce6ab700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc815e560 con 0x7fedc806ff10 2023-11-08T08:23:47.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.645+0000 7fedce6ab700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc806be80 con 0x7fedc815e190 2023-11-08T08:23:47.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.645+0000 7fedce6ab700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc80ce0a0 con 0x7fedc80cdc50 2023-11-08T08:23:47.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.645+0000 7fedcd6a9700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fedc815e190 0x7fedc8161560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:45980/0 (socket says 172.21.15.40:45980) 2023-11-08T08:23:47.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.645+0000 7fedcd6a9700 1 -- 172.21.15.40:0/155747303 learned_addr learned my addr 172.21.15.40:0/155747303 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:47.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.645+0000 7fedccea8700 1 --2- 172.21.15.40:0/155747303 >> v2:172.21.15.169:3300/0 conn(0x7fedc80cdc50 0x7fedc8069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:47.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.645+0000 7fedbffff700 1 -- 172.21.15.40:0/155747303 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616953556 0 0) 0x7fedc0001490 con 0x7fedc806ff10 2023-11-08T08:23:47.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.645+0000 7fedbffff700 1 -- 172.21.15.40:0/155747303 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedb40035d0 con 0x7fedc806ff10 2023-11-08T08:23:47.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.645+0000 7fedbffff700 1 -- 172.21.15.40:0/155747303 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266187528 0 0) 0x7fedc806be80 con 0x7fedc815e190 2023-11-08T08:23:47.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.645+0000 7fedbffff700 1 -- 172.21.15.40:0/155747303 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedc0001490 con 0x7fedc815e190 2023-11-08T08:23:47.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.645+0000 7fedccea8700 1 -- 172.21.15.40:0/155747303 >> v1:172.21.15.169:6789/0 conn(0x7fedc815e190 legacy=0x7fedc8161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:47.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.645+0000 7fedccea8700 1 -- 172.21.15.40:0/155747303 >> v1:172.21.15.40:6789/0 conn(0x7fedc806ff10 legacy=0x7fedc806c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:47.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.645+0000 7fedccea8700 1 -- 172.21.15.40:0/155747303 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc80ce330 con 0x7fedc80cdc50 2023-11-08T08:23:47.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.649+0000 7fedccea8700 1 --2- 172.21.15.40:0/155747303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedc80cdc50 0x7fedc8069bf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009850 tx=0x7fedb8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7eac079ca5de1b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:47.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.649+0000 7fedbffff700 1 -- 172.21.15.40:0/155747303 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8011830 con 0x7fedc80cdc50 2023-11-08T08:23:47.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.649+0000 7fedbffff700 1 -- 172.21.15.40:0/155747303 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb8011990 con 0x7fedc80cdc50 2023-11-08T08:23:47.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.649+0000 7fedbffff700 1 -- 172.21.15.40:0/155747303 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8011d40 con 0x7fedc80cdc50 2023-11-08T08:23:47.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.649+0000 7fedce6ab700 1 -- 172.21.15.40:0/155747303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedc80cdc50 msgr2=0x7fedc8069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:47.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.649+0000 7fedce6ab700 1 --2- 172.21.15.40:0/155747303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedc80cdc50 0x7fedc8069bf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009850 tx=0x7fedb8009b60 comp rx=0 tx=0).stop 2023-11-08T08:23:47.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.649+0000 7fedce6ab700 1 -- 172.21.15.40:0/155747303 shutdown_connections 2023-11-08T08:23:47.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.649+0000 7fedce6ab700 1 --2- 172.21.15.40:0/155747303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedc80cdc50 0x7fedc8069bf0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.649+0000 7fedce6ab700 1 -- 172.21.15.40:0/155747303 >> 172.21.15.40:0/155747303 conn(0x7fedc80cf280 msgr2=0x7fedc80cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:47.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.649+0000 7fedce6ab700 1 -- 172.21.15.40:0/155747303 shutdown_connections 2023-11-08T08:23:47.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.649+0000 7fedce6ab700 1 -- 172.21.15.40:0/155747303 wait complete. 2023-11-08T08:23:47.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.653+0000 7fedce6ab700 1 Processor -- start 2023-11-08T08:23:47.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.653+0000 7fedce6ab700 1 -- start start 2023-11-08T08:23:47.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.653+0000 7fedce6ab700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedc806ff10 0x7fedc81f9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:47.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.653+0000 7fedce6ab700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedc80cdc50 0x7fedc81f32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:47.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.653+0000 7fedce6ab700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedc815e190 0x7fedc81f5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:47.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.653+0000 7fedce6ab700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc81fa420 con 0x7fedc80cdc50 2023-11-08T08:23:47.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.653+0000 7fedce6ab700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc8159fe0 con 0x7fedc806ff10 2023-11-08T08:23:47.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.653+0000 7fedce6ab700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc81f6710 con 0x7fedc815e190 2023-11-08T08:23:47.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.653+0000 7fedccea8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedc80cdc50 0x7fedc81f32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:47.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.653+0000 7fedccea8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedc80cdc50 0x7fedc81f32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41060/0 (socket says 172.21.15.40:41060) 2023-11-08T08:23:47.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.653+0000 7fedccea8700 1 -- 172.21.15.40:0/614815676 learned_addr learned my addr 172.21.15.40:0/614815676 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:47.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.653+0000 7fedcd6a9700 1 --2- 172.21.15.40:0/614815676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedc806ff10 0x7fedc81f9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:47.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.653+0000 7fedcdeaa700 1 --2- 172.21.15.40:0/614815676 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedc815e190 0x7fedc81f5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:47.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.657+0000 7fedccea8700 1 -- 172.21.15.40:0/614815676 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedc815e190 msgr2=0x7fedc81f5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:47.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.657+0000 7fedccea8700 1 --2- 172.21.15.40:0/614815676 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedc815e190 0x7fedc81f5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.657+0000 7fedccea8700 1 -- 172.21.15.40:0/614815676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedc806ff10 msgr2=0x7fedc81f9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:47.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.657+0000 7fedccea8700 1 --2- 172.21.15.40:0/614815676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedc806ff10 0x7fedc81f9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.657+0000 7fedccea8700 1 -- 172.21.15.40:0/614815676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb80094b0 con 0x7fedc80cdc50 2023-11-08T08:23:47.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.657+0000 7fedccea8700 1 --2- 172.21.15.40:0/614815676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedc80cdc50 0x7fedc81f32b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fedb80090b0 tx=0x7fedb8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:47.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.657+0000 7fedbdffb700 1 -- 172.21.15.40:0/614815676 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8004590 con 0x7fedc80cdc50 2023-11-08T08:23:47.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.657+0000 7fedbdffb700 1 -- 172.21.15.40:0/614815676 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb801f070 con 0x7fedc80cdc50 2023-11-08T08:23:47.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.657+0000 7fedce6ab700 1 -- 172.21.15.40:0/614815676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc80afc70 con 0x7fedc80cdc50 2023-11-08T08:23:47.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.657+0000 7fedbdffb700 1 -- 172.21.15.40:0/614815676 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8003900 con 0x7fedc80cdc50 2023-11-08T08:23:47.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.657+0000 7fedce6ab700 1 -- 172.21.15.40:0/614815676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc80af990 con 0x7fedc80cdc50 2023-11-08T08:23:47.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.657+0000 7fedce6ab700 1 -- 172.21.15.40:0/614815676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc80aff50 con 0x7fedc80cdc50 2023-11-08T08:23:47.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.661+0000 7fedbdffb700 1 -- 172.21.15.40:0/614815676 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fedb800f050 con 0x7fedc80cdc50 2023-11-08T08:23:47.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.661+0000 7fedbdffb700 1 --2- 172.21.15.40:0/614815676 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fedac064ee0 0x7fedac067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:47.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.661+0000 7fedbdffb700 1 -- 172.21.15.40:0/614815676 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fedb8021070 con 0x7fedc80cdc50 2023-11-08T08:23:47.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.661+0000 7fedcd6a9700 1 --2- 172.21.15.40:0/614815676 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fedac064ee0 0x7fedac067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:47.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.665+0000 7fedcd6a9700 1 --2- 172.21.15.40:0/614815676 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fedac064ee0 0x7fedac067390 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fedc40011e0 tx=0x7fedc4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:47.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.669+0000 7fedbdffb700 1 -- 172.21.15.40:0/614815676 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fedb8051760 con 0x7fedc80cdc50 2023-11-08T08:23:47.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.845+0000 7fedce6ab700 1 -- 172.21.15.40:0/614815676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fedc8073d80 con 0x7fedc80cdc50 2023-11-08T08:23:47.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.845+0000 7fedbdffb700 1 -- 172.21.15.40:0/614815676 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7fedb802b020 con 0x7fedc80cdc50 2023-11-08T08:23:47.853 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-11-08T08:23:47.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.853+0000 7fedce6ab700 1 -- 172.21.15.40:0/614815676 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fedac064ee0 msgr2=0x7fedac067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:47.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.853+0000 7fedce6ab700 1 --2- 172.21.15.40:0/614815676 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fedac064ee0 0x7fedac067390 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fedc40011e0 tx=0x7fedc4009040 comp rx=0 tx=0).stop 2023-11-08T08:23:47.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.853+0000 7fedce6ab700 1 -- 172.21.15.40:0/614815676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedc80cdc50 msgr2=0x7fedc81f32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:47.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.853+0000 7fedce6ab700 1 --2- 172.21.15.40:0/614815676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedc80cdc50 0x7fedc81f32b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fedb80090b0 tx=0x7fedb8004770 comp rx=0 tx=0).stop 2023-11-08T08:23:47.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.853+0000 7fedce6ab700 1 -- 172.21.15.40:0/614815676 shutdown_connections 2023-11-08T08:23:47.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.853+0000 7fedce6ab700 1 --2- 172.21.15.40:0/614815676 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fedac064ee0 0x7fedac067390 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.853+0000 7fedce6ab700 1 --2- 172.21.15.40:0/614815676 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedc815e190 0x7fedc81f5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.853+0000 7fedce6ab700 1 --2- 172.21.15.40:0/614815676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedc80cdc50 0x7fedc81f32b0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.853+0000 7fedce6ab700 1 --2- 172.21.15.40:0/614815676 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fedc806ff10 0x7fedc81f9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:47.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.853+0000 7fedce6ab700 1 -- 172.21.15.40:0/614815676 >> 172.21.15.40:0/614815676 conn(0x7fedc80cf280 msgr2=0x7fedc80b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:47.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.853+0000 7fedce6ab700 1 -- 172.21.15.40:0/614815676 shutdown_connections 2023-11-08T08:23:47.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:47.853+0000 7fedce6ab700 1 -- 172.21.15.40:0/614815676 wait complete. 2023-11-08T08:23:47.878 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-08T08:23:48.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.061+0000 7fd668b0e700 1 Processor -- start 2023-11-08T08:23:48.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.061+0000 7fd668b0e700 1 -- start start 2023-11-08T08:23:48.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.065+0000 7fd668b0e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd66415c150 0x7fd6640b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:48.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.065+0000 7fd668b0e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd664070360 0x7fd664160340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:48.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.065+0000 7fd668b0e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd66415c520 con 0x7fd66406ff10 2023-11-08T08:23:48.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.065+0000 7fd668b0e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6640b4240 con 0x7fd664070360 2023-11-08T08:23:48.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.065+0000 7fd668b0e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd66406ac00 con 0x7fd66415c150 2023-11-08T08:23:48.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.065+0000 7fd6637fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd66415c150 0x7fd6640b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:48.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.065+0000 7fd663fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd664070360 0x7fd664160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:48.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.065+0000 7fd662ffd700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fd66406ff10 0x7fd66415fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:45990/0 (socket says 172.21.15.40:45990) 2023-11-08T08:23:48.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.065+0000 7fd662ffd700 1 -- 172.21.15.40:0/225108400 learned_addr learned my addr 172.21.15.40:0/225108400 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:48.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.065+0000 7fd6627fc700 1 -- 172.21.15.40:0/225108400 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3998985056 0 0) 0x7fd66415c520 con 0x7fd66406ff10 2023-11-08T08:23:48.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.065+0000 7fd6627fc700 1 -- 172.21.15.40:0/225108400 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6500035d0 con 0x7fd66406ff10 2023-11-08T08:23:48.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.065+0000 7fd663fff700 1 -- 172.21.15.40:0/225108400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd66415c150 msgr2=0x7fd6640b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:48.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.065+0000 7fd663fff700 1 --2- 172.21.15.40:0/225108400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd66415c150 0x7fd6640b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.065+0000 7fd663fff700 1 -- 172.21.15.40:0/225108400 >> v1:172.21.15.169:6789/0 conn(0x7fd66406ff10 legacy=0x7fd66415fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:48.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.065+0000 7fd663fff700 1 -- 172.21.15.40:0/225108400 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66406cc50 con 0x7fd664070360 2023-11-08T08:23:48.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.069+0000 7fd663fff700 1 --2- 172.21.15.40:0/225108400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd664070360 0x7fd664160340 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009850 tx=0x7fd65c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30b820144da5c10 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.069+0000 7fd6627fc700 1 -- 172.21.15.40:0/225108400 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c011830 con 0x7fd664070360 2023-11-08T08:23:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.069+0000 7fd6627fc700 1 -- 172.21.15.40:0/225108400 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65c011990 con 0x7fd664070360 2023-11-08T08:23:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.069+0000 7fd6627fc700 1 -- 172.21.15.40:0/225108400 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c011d40 con 0x7fd664070360 2023-11-08T08:23:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.069+0000 7fd668b0e700 1 -- 172.21.15.40:0/225108400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd664070360 msgr2=0x7fd664160340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.069+0000 7fd668b0e700 1 --2- 172.21.15.40:0/225108400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd664070360 0x7fd664160340 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009850 tx=0x7fd65c009b60 comp rx=0 tx=0).stop 2023-11-08T08:23:48.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.069+0000 7fd668b0e700 1 -- 172.21.15.40:0/225108400 shutdown_connections 2023-11-08T08:23:48.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.069+0000 7fd668b0e700 1 --2- 172.21.15.40:0/225108400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd664070360 0x7fd664160340 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.069+0000 7fd668b0e700 1 --2- 172.21.15.40:0/225108400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd66415c150 0x7fd6640b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.069+0000 7fd668b0e700 1 -- 172.21.15.40:0/225108400 >> 172.21.15.40:0/225108400 conn(0x7fd6640d3680 msgr2=0x7fd6640cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:48.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.069+0000 7fd668b0e700 1 -- 172.21.15.40:0/225108400 shutdown_connections 2023-11-08T08:23:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.069+0000 7fd668b0e700 1 -- 172.21.15.40:0/225108400 wait complete. 2023-11-08T08:23:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.073+0000 7fd668b0e700 1 Processor -- start 2023-11-08T08:23:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.073+0000 7fd668b0e700 1 -- start start 2023-11-08T08:23:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.073+0000 7fd668b0e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd66406ff10 0x7fd66413f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.073+0000 7fd668b0e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd664070360 0x7fd664145eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.073+0000 7fd668b0e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd66415c150 0x7fd6640c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.073+0000 7fd668b0e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6640ce890 con 0x7fd66415c150 2023-11-08T08:23:48.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.073+0000 7fd668b0e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6640ceb30 con 0x7fd66406ff10 2023-11-08T08:23:48.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.073+0000 7fd668b0e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6640cee10 con 0x7fd664070360 2023-11-08T08:23:48.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.073+0000 7fd663fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd66415c150 0x7fd6640c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:48.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.073+0000 7fd663fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd66415c150 0x7fd6640c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41062/0 (socket says 172.21.15.40:41062) 2023-11-08T08:23:48.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.077+0000 7fd663fff700 1 -- 172.21.15.40:0/714643502 learned_addr learned my addr 172.21.15.40:0/714643502 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:48.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.077+0000 7fd662ffd700 1 --2- 172.21.15.40:0/714643502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd664070360 0x7fd664145eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:48.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.077+0000 7fd6637fe700 1 --2- 172.21.15.40:0/714643502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd66406ff10 0x7fd66413f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:48.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.077+0000 7fd663fff700 1 -- 172.21.15.40:0/714643502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd664070360 msgr2=0x7fd664145eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:48.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.077+0000 7fd663fff700 1 --2- 172.21.15.40:0/714643502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd664070360 0x7fd664145eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.077+0000 7fd663fff700 1 -- 172.21.15.40:0/714643502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd66406ff10 msgr2=0x7fd66413f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:48.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.077+0000 7fd663fff700 1 --2- 172.21.15.40:0/714643502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd66406ff10 0x7fd66413f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.077+0000 7fd663fff700 1 -- 172.21.15.40:0/714643502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c0094b0 con 0x7fd66415c150 2023-11-08T08:23:48.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.077+0000 7fd6637fe700 1 --2- 172.21.15.40:0/714643502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd66406ff10 0x7fd66413f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:23:48.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.077+0000 7fd663fff700 1 --2- 172.21.15.40:0/714643502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd66415c150 0x7fd6640c8260 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd65c0090b0 tx=0x7fd65c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:48.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.077+0000 7fd65bfff700 1 -- 172.21.15.40:0/714643502 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c004590 con 0x7fd66415c150 2023-11-08T08:23:48.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.077+0000 7fd65bfff700 1 -- 172.21.15.40:0/714643502 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65c01f070 con 0x7fd66415c150 2023-11-08T08:23:48.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.077+0000 7fd65bfff700 1 -- 172.21.15.40:0/714643502 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c003900 con 0x7fd66415c150 2023-11-08T08:23:48.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.077+0000 7fd668b0e700 1 -- 172.21.15.40:0/714643502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6640d2730 con 0x7fd66415c150 2023-11-08T08:23:48.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.077+0000 7fd668b0e700 1 -- 172.21.15.40:0/714643502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6640d2450 con 0x7fd66415c150 2023-11-08T08:23:48.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.077+0000 7fd668b0e700 1 -- 172.21.15.40:0/714643502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6640d05e0 con 0x7fd66415c150 2023-11-08T08:23:48.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.081+0000 7fd65bfff700 1 -- 172.21.15.40:0/714643502 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd65c00f050 con 0x7fd66415c150 2023-11-08T08:23:48.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.081+0000 7fd65bfff700 1 --2- 172.21.15.40:0/714643502 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd644064f10 0x7fd6440673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:48.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.081+0000 7fd65bfff700 1 -- 172.21.15.40:0/714643502 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fd65c021070 con 0x7fd66415c150 2023-11-08T08:23:48.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.081+0000 7fd6637fe700 1 --2- 172.21.15.40:0/714643502 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd644064f10 0x7fd6440673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:48.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.085+0000 7fd6637fe700 1 --2- 172.21.15.40:0/714643502 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd644064f10 0x7fd6440673c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd654009fd0 tx=0x7fd65400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:48.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.085+0000 7fd65bfff700 1 -- 172.21.15.40:0/714643502 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd65c0516c0 con 0x7fd66415c150 2023-11-08T08:23:48.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.265+0000 7fd668b0e700 1 -- 172.21.15.40:0/714643502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd664073d20 con 0x7fd66415c150 2023-11-08T08:23:48.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.265+0000 7fd65bfff700 1 -- 172.21.15.40:0/714643502 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v71) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd65c0150a0 con 0x7fd66415c150 2023-11-08T08:23:48.274 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 8 2023-11-08T08:23:48.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.269+0000 7fd668b0e700 1 -- 172.21.15.40:0/714643502 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd644064f10 msgr2=0x7fd6440673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:48.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.269+0000 7fd668b0e700 1 --2- 172.21.15.40:0/714643502 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd644064f10 0x7fd6440673c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd654009fd0 tx=0x7fd65400b040 comp rx=0 tx=0).stop 2023-11-08T08:23:48.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.269+0000 7fd668b0e700 1 -- 172.21.15.40:0/714643502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd66415c150 msgr2=0x7fd6640c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:48.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.269+0000 7fd668b0e700 1 --2- 172.21.15.40:0/714643502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd66415c150 0x7fd6640c8260 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd65c0090b0 tx=0x7fd65c004770 comp rx=0 tx=0).stop 2023-11-08T08:23:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.269+0000 7fd668b0e700 1 -- 172.21.15.40:0/714643502 shutdown_connections 2023-11-08T08:23:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.269+0000 7fd668b0e700 1 --2- 172.21.15.40:0/714643502 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd644064f10 0x7fd6440673c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.269+0000 7fd668b0e700 1 --2- 172.21.15.40:0/714643502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd66415c150 0x7fd6640c8260 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.269+0000 7fd668b0e700 1 --2- 172.21.15.40:0/714643502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd664070360 0x7fd664145eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.269+0000 7fd668b0e700 1 --2- 172.21.15.40:0/714643502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd66406ff10 0x7fd66413f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.269+0000 7fd668b0e700 1 -- 172.21.15.40:0/714643502 >> 172.21.15.40:0/714643502 conn(0x7fd6640d3680 msgr2=0x7fd664155a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:48.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.269+0000 7fd668b0e700 1 -- 172.21.15.40:0/714643502 shutdown_connections 2023-11-08T08:23:48.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.269+0000 7fd668b0e700 1 -- 172.21.15.40:0/714643502 wait complete. 2023-11-08T08:23:48.299 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T08:23:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.477+0000 7efca645a700 1 Processor -- start 2023-11-08T08:23:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.481+0000 7efca645a700 1 -- start start 2023-11-08T08:23:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.481+0000 7efca645a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efca015c150 0x7efca00b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efca645a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efca0070360 0x7efca0160340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efca645a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca015c520 con 0x7efca006ff10 2023-11-08T08:23:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efca645a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efca00b4240 con 0x7efca0070360 2023-11-08T08:23:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efca645a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca006ac00 con 0x7efca015c150 2023-11-08T08:23:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efca5458700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efca015c150 0x7efca00b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efca4c57700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efca006ff10 0x7efca015fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50206/0 (socket says 172.21.15.40:50206) 2023-11-08T08:23:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efca4c57700 1 -- 172.21.15.40:0/348557260 learned_addr learned my addr 172.21.15.40:0/348557260 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efca5c59700 1 --2- 172.21.15.40:0/348557260 >> v2:172.21.15.40:3300/0 conn(0x7efca0070360 0x7efca0160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efca5458700 1 -- 172.21.15.40:0/348557260 >> v2:172.21.15.40:3300/0 conn(0x7efca0070360 msgr2=0x7efca0160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efca5458700 1 --2- 172.21.15.40:0/348557260 >> v2:172.21.15.40:3300/0 conn(0x7efca0070360 0x7efca0160340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efca5458700 1 -- 172.21.15.40:0/348557260 >> v1:172.21.15.40:6789/0 conn(0x7efca006ff10 legacy=0x7efca015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:48.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efca5458700 1 -- 172.21.15.40:0/348557260 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca006cc50 con 0x7efca015c150 2023-11-08T08:23:48.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efca5458700 1 --2- 172.21.15.40:0/348557260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efca015c150 0x7efca00b3d00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009850 tx=0x7efc9c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91917e09078f6a76 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:48.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efc97fff700 1 -- 172.21.15.40:0/348557260 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c011830 con 0x7efca015c150 2023-11-08T08:23:48.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efc97fff700 1 -- 172.21.15.40:0/348557260 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9c011990 con 0x7efca015c150 2023-11-08T08:23:48.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efc97fff700 1 -- 172.21.15.40:0/348557260 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c011d40 con 0x7efca015c150 2023-11-08T08:23:48.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efca645a700 1 -- 172.21.15.40:0/348557260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efca015c150 msgr2=0x7efca00b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:48.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efca645a700 1 --2- 172.21.15.40:0/348557260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efca015c150 0x7efca00b3d00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009850 tx=0x7efc9c009b60 comp rx=0 tx=0).stop 2023-11-08T08:23:48.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efca645a700 1 -- 172.21.15.40:0/348557260 shutdown_connections 2023-11-08T08:23:48.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efca645a700 1 --2- 172.21.15.40:0/348557260 >> v2:172.21.15.40:3300/0 conn(0x7efca0070360 0x7efca0160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efca645a700 1 --2- 172.21.15.40:0/348557260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efca015c150 0x7efca00b3d00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.485+0000 7efca645a700 1 -- 172.21.15.40:0/348557260 >> 172.21.15.40:0/348557260 conn(0x7efca00d3680 msgr2=0x7efca00cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:48.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.489+0000 7efca645a700 1 -- 172.21.15.40:0/348557260 shutdown_connections 2023-11-08T08:23:48.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.489+0000 7efca645a700 1 -- 172.21.15.40:0/348557260 wait complete. 2023-11-08T08:23:48.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.489+0000 7efca645a700 1 Processor -- start 2023-11-08T08:23:48.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.489+0000 7efca645a700 1 -- start start 2023-11-08T08:23:48.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.489+0000 7efca645a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efca006ff10 0x7efca01f1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:48.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.489+0000 7efca645a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efca0070360 0x7efca015a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:48.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.489+0000 7efca645a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efca015c150 0x7efca015c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:48.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.489+0000 7efca645a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca01f18c0 con 0x7efca015c150 2023-11-08T08:23:48.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.489+0000 7efca645a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca00c9eb0 con 0x7efca0070360 2023-11-08T08:23:48.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.489+0000 7efca645a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca00ca190 con 0x7efca006ff10 2023-11-08T08:23:48.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.489+0000 7efca5c59700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efca015c150 0x7efca015c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:48.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.489+0000 7efca5c59700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efca015c150 0x7efca015c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41080/0 (socket says 172.21.15.40:41080) 2023-11-08T08:23:48.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.489+0000 7efca5c59700 1 -- 172.21.15.40:0/883986164 learned_addr learned my addr 172.21.15.40:0/883986164 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:48.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.489+0000 7efca5458700 1 --2- 172.21.15.40:0/883986164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efca006ff10 0x7efca01f1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:48.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.493+0000 7efca4c57700 1 --2- 172.21.15.40:0/883986164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efca0070360 0x7efca015a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:48.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.493+0000 7efca5c59700 1 -- 172.21.15.40:0/883986164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efca006ff10 msgr2=0x7efca01f1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:48.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.493+0000 7efca5c59700 1 --2- 172.21.15.40:0/883986164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efca006ff10 0x7efca01f1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.493+0000 7efca5c59700 1 -- 172.21.15.40:0/883986164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efca0070360 msgr2=0x7efca015a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:48.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.493+0000 7efca5c59700 1 --2- 172.21.15.40:0/883986164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efca0070360 0x7efca015a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.493+0000 7efca5c59700 1 -- 172.21.15.40:0/883986164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9c0094b0 con 0x7efca015c150 2023-11-08T08:23:48.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.493+0000 7efca5c59700 1 --2- 172.21.15.40:0/883986164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efca015c150 0x7efca015c6b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7efc9800efc0 tx=0x7efc9800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:48.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.493+0000 7efc95ffb700 1 -- 172.21.15.40:0/883986164 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98009ea0 con 0x7efca015c150 2023-11-08T08:23:48.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.493+0000 7efc95ffb700 1 -- 172.21.15.40:0/883986164 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9800f040 con 0x7efca015c150 2023-11-08T08:23:48.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.493+0000 7efc95ffb700 1 -- 172.21.15.40:0/883986164 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98014480 con 0x7efca015c150 2023-11-08T08:23:48.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.493+0000 7efca645a700 1 -- 172.21.15.40:0/883986164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca00d2730 con 0x7efca015c150 2023-11-08T08:23:48.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.493+0000 7efca645a700 1 -- 172.21.15.40:0/883986164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca00d2450 con 0x7efca015c150 2023-11-08T08:23:48.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.493+0000 7efca645a700 1 -- 172.21.15.40:0/883986164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca00d05e0 con 0x7efca015c150 2023-11-08T08:23:48.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.497+0000 7efc95ffb700 1 -- 172.21.15.40:0/883986164 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7efc980145e0 con 0x7efca015c150 2023-11-08T08:23:48.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.497+0000 7efc95ffb700 1 --2- 172.21.15.40:0/883986164 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efc84064ee0 0x7efc84067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:48.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.497+0000 7efc95ffb700 1 -- 172.21.15.40:0/883986164 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7efc98082af0 con 0x7efca015c150 2023-11-08T08:23:48.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.497+0000 7efca5458700 1 --2- 172.21.15.40:0/883986164 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efc84064ee0 0x7efc84067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:48.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.501+0000 7efca5458700 1 --2- 172.21.15.40:0/883986164 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efc84064ee0 0x7efc84067390 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7efc9c0090b0 tx=0x7efc9c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:48.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.505+0000 7efc95ffb700 1 -- 172.21.15.40:0/883986164 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7efc98050a20 con 0x7efca015c150 2023-11-08T08:23:48.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.677+0000 7efca645a700 1 -- 172.21.15.40:0/883986164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efca0073d20 con 0x7efca015c150 2023-11-08T08:23:48.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.677+0000 7efc95ffb700 1 -- 172.21.15.40:0/883986164 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7efc98054040 con 0x7efca015c150 2023-11-08T08:23:48.685 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-11-08T08:23:48.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.685+0000 7efca645a700 1 -- 172.21.15.40:0/883986164 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efc84064ee0 msgr2=0x7efc84067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:48.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.685+0000 7efca645a700 1 --2- 172.21.15.40:0/883986164 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efc84064ee0 0x7efc84067390 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7efc9c0090b0 tx=0x7efc9c01c040 comp rx=0 tx=0).stop 2023-11-08T08:23:48.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.685+0000 7efca645a700 1 -- 172.21.15.40:0/883986164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efca015c150 msgr2=0x7efca015c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:48.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.685+0000 7efca645a700 1 --2- 172.21.15.40:0/883986164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efca015c150 0x7efca015c6b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7efc9800efc0 tx=0x7efc9800c3a0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.685+0000 7efca645a700 1 -- 172.21.15.40:0/883986164 shutdown_connections 2023-11-08T08:23:48.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.685+0000 7efca645a700 1 --2- 172.21.15.40:0/883986164 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7efc84064ee0 0x7efc84067390 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.685+0000 7efca645a700 1 --2- 172.21.15.40:0/883986164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efca015c150 0x7efca015c6b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.685+0000 7efca645a700 1 --2- 172.21.15.40:0/883986164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efca0070360 0x7efca015a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.685+0000 7efca645a700 1 --2- 172.21.15.40:0/883986164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efca006ff10 0x7efca01f1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.685+0000 7efca645a700 1 -- 172.21.15.40:0/883986164 >> 172.21.15.40:0/883986164 conn(0x7efca00d3680 msgr2=0x7efca0156810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:48.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.685+0000 7efca645a700 1 -- 172.21.15.40:0/883986164 shutdown_connections 2023-11-08T08:23:48.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.685+0000 7efca645a700 1 -- 172.21.15.40:0/883986164 wait complete. 2023-11-08T08:23:48.712 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T08:23:48.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.889+0000 7f7ea45cb700 1 Processor -- start 2023-11-08T08:23:48.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.889+0000 7f7ea45cb700 1 -- start start 2023-11-08T08:23:48.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.889+0000 7f7ea45cb700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7e9c0ada10 0x7f7e9c160cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:48.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.889+0000 7f7ea45cb700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e9c06b220 0x7f7e9c0c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:48.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.889+0000 7f7ea45cb700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9c0ca3b0 con 0x7f7e9c06c040 2023-11-08T08:23:48.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.889+0000 7f7ea45cb700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9c1608b0 con 0x7f7e9c0ada10 2023-11-08T08:23:48.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.889+0000 7f7ea45cb700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c1601f0 con 0x7f7e9c06b220 2023-11-08T08:23:48.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.889+0000 7f7ea2367700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7e9c0ada10 0x7f7e9c160cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:48.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.893+0000 7f7ea1b66700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e9c06b220 0x7f7e9c0c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:48.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.893+0000 7f7ea2367700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7e9c0ada10 0x7f7e9c160cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41086/0 (socket says 172.21.15.40:41086) 2023-11-08T08:23:48.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.893+0000 7f7ea2b68700 1 --1- 172.21.15.40:0/1663255142 >> v1:172.21.15.169:6789/0 conn(0x7f7e9c06c040 0x7f7e9c069bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:46002/0 (socket says 172.21.15.40:46002) 2023-11-08T08:23:48.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.893+0000 7f7ea2367700 1 -- 172.21.15.40:0/1663255142 learned_addr learned my addr 172.21.15.40:0/1663255142 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:48.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.893+0000 7f7ea2367700 1 -- 172.21.15.40:0/1663255142 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e9c06b220 msgr2=0x7f7e9c0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:48.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.893+0000 7f7ea2367700 1 --2- 172.21.15.40:0/1663255142 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e9c06b220 0x7f7e9c0c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.893+0000 7f7ea2367700 1 -- 172.21.15.40:0/1663255142 >> v1:172.21.15.169:6789/0 conn(0x7f7e9c06c040 legacy=0x7f7e9c069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:48.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.893+0000 7f7ea2367700 1 -- 172.21.15.40:0/1663255142 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c1604d0 con 0x7f7e9c0ada10 2023-11-08T08:23:48.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.893+0000 7f7ea1365700 1 -- 172.21.15.40:0/1663255142 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172430211 0 0) 0x7f7e9c0ca3b0 con 0x7f7e9c06c040 2023-11-08T08:23:48.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.893+0000 7f7ea2367700 1 --2- 172.21.15.40:0/1663255142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e9c0ada10 0x7f7e9c160cd0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009910 tx=0x7f7e98009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18144bdf58ddc30 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:48.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.893+0000 7f7ea1365700 1 -- 172.21.15.40:0/1663255142 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98011830 con 0x7f7e9c0ada10 2023-11-08T08:23:48.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.893+0000 7f7ea1365700 1 -- 172.21.15.40:0/1663255142 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e98011990 con 0x7f7e9c0ada10 2023-11-08T08:23:48.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.893+0000 7f7ea1365700 1 -- 172.21.15.40:0/1663255142 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98011d40 con 0x7f7e9c0ada10 2023-11-08T08:23:48.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.893+0000 7f7ea45cb700 1 -- 172.21.15.40:0/1663255142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e9c0ada10 msgr2=0x7f7e9c160cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:48.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.893+0000 7f7ea45cb700 1 --2- 172.21.15.40:0/1663255142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e9c0ada10 0x7f7e9c160cd0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009910 tx=0x7f7e98009c20 comp rx=0 tx=0).stop 2023-11-08T08:23:48.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.893+0000 7f7ea45cb700 1 -- 172.21.15.40:0/1663255142 shutdown_connections 2023-11-08T08:23:48.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.893+0000 7f7ea45cb700 1 --2- 172.21.15.40:0/1663255142 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e9c06b220 0x7f7e9c0c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.893+0000 7f7ea45cb700 1 --2- 172.21.15.40:0/1663255142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e9c0ada10 0x7f7e9c160cd0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.893+0000 7f7ea45cb700 1 -- 172.21.15.40:0/1663255142 >> 172.21.15.40:0/1663255142 conn(0x7f7e9c0cf580 msgr2=0x7f7e9c0cb370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:48.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.897+0000 7f7ea45cb700 1 -- 172.21.15.40:0/1663255142 shutdown_connections 2023-11-08T08:23:48.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.897+0000 7f7ea45cb700 1 -- 172.21.15.40:0/1663255142 wait complete. 2023-11-08T08:23:48.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.897+0000 7f7ea45cb700 1 Processor -- start 2023-11-08T08:23:48.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.897+0000 7f7ea45cb700 1 -- start start 2023-11-08T08:23:48.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7ea45cb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e9c06b220 0x7f7e9c151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:48.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7ea45cb700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e9c06c040 0x7f7e9c15e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:48.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7ea45cb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e9c0ada10 0x7f7e9c156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:48.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7ea45cb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0c4a50 con 0x7f7e9c06b220 2023-11-08T08:23:48.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7ea45cb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c152490 con 0x7f7e9c0ada10 2023-11-08T08:23:48.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7ea45cb700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0c3e40 con 0x7f7e9c06c040 2023-11-08T08:23:48.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7ea2367700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e9c06b220 0x7f7e9c151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:48.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7ea2367700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e9c06b220 0x7f7e9c151f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41088/0 (socket says 172.21.15.40:41088) 2023-11-08T08:23:48.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7ea1b66700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e9c06c040 0x7f7e9c15e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:48.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7ea2367700 1 -- 172.21.15.40:0/4192414430 learned_addr learned my addr 172.21.15.40:0/4192414430 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:48.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7ea2b68700 1 --2- 172.21.15.40:0/4192414430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e9c0ada10 0x7f7e9c156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:48.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7ea1b66700 1 -- 172.21.15.40:0/4192414430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e9c0ada10 msgr2=0x7f7e9c156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:48.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7ea1b66700 1 --2- 172.21.15.40:0/4192414430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e9c0ada10 0x7f7e9c156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7ea1b66700 1 -- 172.21.15.40:0/4192414430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e9c06b220 msgr2=0x7f7e9c151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:48.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7ea1b66700 1 --2- 172.21.15.40:0/4192414430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e9c06b220 0x7f7e9c151f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:48.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7ea1b66700 1 -- 172.21.15.40:0/4192414430 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e98009570 con 0x7f7e9c06c040 2023-11-08T08:23:48.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7ea2367700 1 --2- 172.21.15.40:0/4192414430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e9c06b220 0x7f7e9c151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:23:48.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7ea1b66700 1 --2- 172.21.15.40:0/4192414430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e9c06c040 0x7f7e9c15e970 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00e8a0 tx=0x7f7e8c00ebb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:48.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7e92ffd700 1 -- 172.21.15.40:0/4192414430 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c013680 con 0x7f7e9c06c040 2023-11-08T08:23:48.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7e92ffd700 1 -- 172.21.15.40:0/4192414430 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8c0137e0 con 0x7f7e9c06c040 2023-11-08T08:23:48.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7ea45cb700 1 -- 172.21.15.40:0/4192414430 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9c0ca690 con 0x7f7e9c06c040 2023-11-08T08:23:48.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7e92ffd700 1 -- 172.21.15.40:0/4192414430 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c013b70 con 0x7f7e9c06c040 2023-11-08T08:23:48.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.901+0000 7f7ea45cb700 1 -- 172.21.15.40:0/4192414430 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c0ca0c0 con 0x7f7e9c06c040 2023-11-08T08:23:48.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.905+0000 7f7ea45cb700 1 -- 172.21.15.40:0/4192414430 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9c0adde0 con 0x7f7e9c06c040 2023-11-08T08:23:48.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.905+0000 7f7e92ffd700 1 -- 172.21.15.40:0/4192414430 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7e8c019030 con 0x7f7e9c06c040 2023-11-08T08:23:48.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.909+0000 7f7e92ffd700 1 --2- 172.21.15.40:0/4192414430 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e80064ee0 0x7f7e80067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:48.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.909+0000 7f7e92ffd700 1 -- 172.21.15.40:0/4192414430 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f7e8c081db0 con 0x7f7e9c06c040 2023-11-08T08:23:48.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.909+0000 7f7ea2367700 1 --2- 172.21.15.40:0/4192414430 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e80064ee0 0x7f7e80067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:48.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.909+0000 7f7ea2367700 1 --2- 172.21.15.40:0/4192414430 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e80064ee0 0x7f7e80067390 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009c60 tx=0x7f7e98008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:48.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:48.913+0000 7f7e92ffd700 1 -- 172.21.15.40:0/4192414430 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f7e8c086070 con 0x7f7e9c06c040 2023-11-08T08:23:49.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.085+0000 7f7ea45cb700 1 -- 172.21.15.40:0/4192414430 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7e9c0c4c00 con 0x7f7e9c06c040 2023-11-08T08:23:49.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.089+0000 7f7e92ffd700 1 -- 172.21.15.40:0/4192414430 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7e9c0c4c00 con 0x7f7e9c06c040 2023-11-08T08:23:49.094 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-11-08T08:23:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.093+0000 7f7ea45cb700 1 -- 172.21.15.40:0/4192414430 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e80064ee0 msgr2=0x7f7e80067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.093+0000 7f7ea45cb700 1 --2- 172.21.15.40:0/4192414430 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e80064ee0 0x7f7e80067390 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009c60 tx=0x7f7e98008030 comp rx=0 tx=0).stop 2023-11-08T08:23:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.093+0000 7f7ea45cb700 1 -- 172.21.15.40:0/4192414430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e9c06c040 msgr2=0x7f7e9c15e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.093+0000 7f7ea45cb700 1 --2- 172.21.15.40:0/4192414430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e9c06c040 0x7f7e9c15e970 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00e8a0 tx=0x7f7e8c00ebb0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.093+0000 7f7ea45cb700 1 -- 172.21.15.40:0/4192414430 shutdown_connections 2023-11-08T08:23:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.093+0000 7f7ea45cb700 1 --2- 172.21.15.40:0/4192414430 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7e80064ee0 0x7f7e80067390 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.093+0000 7f7ea45cb700 1 --2- 172.21.15.40:0/4192414430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e9c0ada10 0x7f7e9c156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.093+0000 7f7ea45cb700 1 --2- 172.21.15.40:0/4192414430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e9c06c040 0x7f7e9c15e970 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.093+0000 7f7ea45cb700 1 --2- 172.21.15.40:0/4192414430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e9c06b220 0x7f7e9c151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.093+0000 7f7ea45cb700 1 -- 172.21.15.40:0/4192414430 >> 172.21.15.40:0/4192414430 conn(0x7f7e9c0cf580 msgr2=0x7f7e9c06c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:49.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.093+0000 7f7ea45cb700 1 -- 172.21.15.40:0/4192414430 shutdown_connections 2023-11-08T08:23:49.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.093+0000 7f7ea45cb700 1 -- 172.21.15.40:0/4192414430 wait complete. 2023-11-08T08:23:49.123 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:23:49.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.293+0000 7fda3759e700 1 Processor -- start 2023-11-08T08:23:49.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.293+0000 7fda3759e700 1 -- start start 2023-11-08T08:23:49.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.293+0000 7fda3759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda380cd0d0 0x7fda380b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:49.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.293+0000 7fda3759e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fda380b1b80 0x7fda3806bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:49.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.293+0000 7fda3759e700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda380c9d40 con 0x7fda3815c650 2023-11-08T08:23:49.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.293+0000 7fda3759e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda3806b140 con 0x7fda380b1b80 2023-11-08T08:23:49.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.293+0000 7fda3759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda380cd5d0 con 0x7fda380cd0d0 2023-11-08T08:23:49.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.297+0000 7fda35d9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda380cd0d0 0x7fda380b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:49.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.297+0000 7fda36d9d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fda380b1b80 0x7fda3806bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:49.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.297+0000 7fda35d9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda380cd0d0 0x7fda380b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.40:40960/0 (socket says 172.21.15.40:40960) 2023-11-08T08:23:49.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.297+0000 7fda35d9b700 1 -- 172.21.15.40:0/797505831 learned_addr learned my addr 172.21.15.40:0/797505831 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:49.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.297+0000 7fda3659c700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fda3815c650 0x7fda3806b2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:46006/0 (socket says 172.21.15.40:46006) 2023-11-08T08:23:49.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.297+0000 7fda35d9b700 1 -- 172.21.15.40:0/797505831 >> v2:172.21.15.169:3300/0 conn(0x7fda380b1b80 msgr2=0x7fda3806bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:49.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.297+0000 7fda35d9b700 1 --2- 172.21.15.40:0/797505831 >> v2:172.21.15.169:3300/0 conn(0x7fda380b1b80 0x7fda3806bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.297+0000 7fda35d9b700 1 -- 172.21.15.40:0/797505831 >> v1:172.21.15.169:6789/0 conn(0x7fda3815c650 legacy=0x7fda3806b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:49.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.297+0000 7fda3559a700 1 -- 172.21.15.40:0/797505831 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814964622 0 0) 0x7fda380c9d40 con 0x7fda3815c650 2023-11-08T08:23:49.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.297+0000 7fda35d9b700 1 -- 172.21.15.40:0/797505831 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda380cd8b0 con 0x7fda380cd0d0 2023-11-08T08:23:49.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.297+0000 7fda35d9b700 1 --2- 172.21.15.40:0/797505831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda380cd0d0 0x7fda380b1640 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009910 tx=0x7fda2c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b588d2337d1fe4c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:49.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.297+0000 7fda3559a700 1 -- 172.21.15.40:0/797505831 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c011830 con 0x7fda380cd0d0 2023-11-08T08:23:49.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.297+0000 7fda3559a700 1 -- 172.21.15.40:0/797505831 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2c011990 con 0x7fda380cd0d0 2023-11-08T08:23:49.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.297+0000 7fda3559a700 1 -- 172.21.15.40:0/797505831 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c011d40 con 0x7fda380cd0d0 2023-11-08T08:23:49.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.301+0000 7fda3759e700 1 -- 172.21.15.40:0/797505831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda380cd0d0 msgr2=0x7fda380b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:49.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.301+0000 7fda3759e700 1 --2- 172.21.15.40:0/797505831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda380cd0d0 0x7fda380b1640 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009910 tx=0x7fda2c009c20 comp rx=0 tx=0).stop 2023-11-08T08:23:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.301+0000 7fda3759e700 1 -- 172.21.15.40:0/797505831 shutdown_connections 2023-11-08T08:23:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.301+0000 7fda3759e700 1 --2- 172.21.15.40:0/797505831 >> v2:172.21.15.169:3300/0 conn(0x7fda380b1b80 0x7fda3806bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.301+0000 7fda3759e700 1 --2- 172.21.15.40:0/797505831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda380cd0d0 0x7fda380b1640 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.301+0000 7fda3759e700 1 -- 172.21.15.40:0/797505831 >> 172.21.15.40:0/797505831 conn(0x7fda380b0f80 msgr2=0x7fda380ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:49.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.301+0000 7fda3759e700 1 -- 172.21.15.40:0/797505831 shutdown_connections 2023-11-08T08:23:49.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.301+0000 7fda3759e700 1 -- 172.21.15.40:0/797505831 wait complete. 2023-11-08T08:23:49.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.305+0000 7fda3759e700 1 Processor -- start 2023-11-08T08:23:49.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.305+0000 7fda3759e700 1 -- start start 2023-11-08T08:23:49.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.305+0000 7fda3759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda380b1b80 0x7fda380c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:49.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.305+0000 7fda3759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda380cd0d0 0x7fda3814ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:49.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.305+0000 7fda3759e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3815c650 0x7fda381f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:49.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.305+0000 7fda3759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda381f5290 con 0x7fda380cd0d0 2023-11-08T08:23:49.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.305+0000 7fda3759e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda381f5570 con 0x7fda3815c650 2023-11-08T08:23:49.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.305+0000 7fda3759e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda380ab900 con 0x7fda380b1b80 2023-11-08T08:23:49.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.305+0000 7fda35d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda380cd0d0 0x7fda3814ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:49.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.305+0000 7fda35d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda380cd0d0 0x7fda3814ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41104/0 (socket says 172.21.15.40:41104) 2023-11-08T08:23:49.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.305+0000 7fda35d9b700 1 -- 172.21.15.40:0/157870499 learned_addr learned my addr 172.21.15.40:0/157870499 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:49.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.305+0000 7fda36d9d700 1 --2- 172.21.15.40:0/157870499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3815c650 0x7fda381f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:49.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.305+0000 7fda3659c700 1 --2- 172.21.15.40:0/157870499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda380b1b80 0x7fda380c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:49.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.309+0000 7fda35d9b700 1 -- 172.21.15.40:0/157870499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda380b1b80 msgr2=0x7fda380c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:49.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.309+0000 7fda35d9b700 1 --2- 172.21.15.40:0/157870499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda380b1b80 0x7fda380c6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.309+0000 7fda35d9b700 1 -- 172.21.15.40:0/157870499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3815c650 msgr2=0x7fda381f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:49.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.309+0000 7fda35d9b700 1 --2- 172.21.15.40:0/157870499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3815c650 0x7fda381f4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.309+0000 7fda35d9b700 1 -- 172.21.15.40:0/157870499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c009570 con 0x7fda380cd0d0 2023-11-08T08:23:49.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.309+0000 7fda36d9d700 1 --2- 172.21.15.40:0/157870499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3815c650 0x7fda381f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:23:49.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.309+0000 7fda35d9b700 1 --2- 172.21.15.40:0/157870499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda380cd0d0 0x7fda3814ab40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fda380cd710 tx=0x7fda2c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:49.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.309+0000 7fda26ffd700 1 -- 172.21.15.40:0/157870499 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c01a900 con 0x7fda380cd0d0 2023-11-08T08:23:49.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.309+0000 7fda26ffd700 1 -- 172.21.15.40:0/157870499 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2c01f070 con 0x7fda380cd0d0 2023-11-08T08:23:49.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.309+0000 7fda26ffd700 1 -- 172.21.15.40:0/157870499 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c01acc0 con 0x7fda380cd0d0 2023-11-08T08:23:49.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.309+0000 7fda3759e700 1 -- 172.21.15.40:0/157870499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda380c9680 con 0x7fda380cd0d0 2023-11-08T08:23:49.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.309+0000 7fda3759e700 1 -- 172.21.15.40:0/157870499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda380c99d0 con 0x7fda380cd0d0 2023-11-08T08:23:49.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.309+0000 7fda3759e700 1 -- 172.21.15.40:0/157870499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda3806c410 con 0x7fda380cd0d0 2023-11-08T08:23:49.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.309+0000 7fda26ffd700 1 -- 172.21.15.40:0/157870499 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fda2c00f050 con 0x7fda380cd0d0 2023-11-08T08:23:49.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.313+0000 7fda26ffd700 1 --2- 172.21.15.40:0/157870499 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fda18064ee0 0x7fda18067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:49.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.313+0000 7fda26ffd700 1 -- 172.21.15.40:0/157870499 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fda2c021070 con 0x7fda380cd0d0 2023-11-08T08:23:49.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.313+0000 7fda3659c700 1 --2- 172.21.15.40:0/157870499 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fda18064ee0 0x7fda18067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:49.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.313+0000 7fda3659c700 1 --2- 172.21.15.40:0/157870499 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fda18064ee0 0x7fda18067390 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fda28000f60 tx=0x7fda28007e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:49.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.321+0000 7fda26ffd700 1 -- 172.21.15.40:0/157870499 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fda2c051880 con 0x7fda380cd0d0 2023-11-08T08:23:49.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.537+0000 7fda3759e700 1 -- 172.21.15.40:0/157870499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda380abdc0 con 0x7fda380cd0d0 2023-11-08T08:23:49.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.537+0000 7fda26ffd700 1 -- 172.21.15.40:0/157870499 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4650 (secure 0 0 0) 0x7fda2c006da0 con 0x7fda380cd0d0 2023-11-08T08:23:49.544 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:23:49.544 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/307473250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":307473250},{"type":"v1","addr":"172.21.15.169:6835","nonce":307473250}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6148,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/924518366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":924518366},{"type":"v1","addr":"172.21.15.169:6837","nonce":924518366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4294005934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4294005934},{"type":"v1","addr":"172.21.15.40:6833","nonce":4294005934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:22:20.650687+0000","modified":"2023-11-08T08:22:37.032476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6237":{"gid":6237,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":7,"addr":"172.21.15.40:6835/3395630689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3395630689},{"type":"v1","addr":"172.21.15.40:6835","nonce":3395630689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T08:23:49.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.541+0000 7fda3759e700 1 -- 172.21.15.40:0/157870499 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fda18064ee0 msgr2=0x7fda18067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:49.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.541+0000 7fda3759e700 1 --2- 172.21.15.40:0/157870499 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fda18064ee0 0x7fda18067390 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fda28000f60 tx=0x7fda28007e90 comp rx=0 tx=0).stop 2023-11-08T08:23:49.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.541+0000 7fda3759e700 1 -- 172.21.15.40:0/157870499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda380cd0d0 msgr2=0x7fda3814ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:49.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.541+0000 7fda3759e700 1 --2- 172.21.15.40:0/157870499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda380cd0d0 0x7fda3814ab40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fda380cd710 tx=0x7fda2c004770 comp rx=0 tx=0).stop 2023-11-08T08:23:49.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.541+0000 7fda3759e700 1 -- 172.21.15.40:0/157870499 shutdown_connections 2023-11-08T08:23:49.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.541+0000 7fda3759e700 1 --2- 172.21.15.40:0/157870499 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fda18064ee0 0x7fda18067390 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.541+0000 7fda3759e700 1 --2- 172.21.15.40:0/157870499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3815c650 0x7fda381f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.541+0000 7fda3759e700 1 --2- 172.21.15.40:0/157870499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda380cd0d0 0x7fda3814ab40 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.541+0000 7fda3759e700 1 --2- 172.21.15.40:0/157870499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda380b1b80 0x7fda380c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.541+0000 7fda3759e700 1 -- 172.21.15.40:0/157870499 >> 172.21.15.40:0/157870499 conn(0x7fda380b0f80 msgr2=0x7fda380cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:49.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.541+0000 7fda3759e700 1 -- 172.21.15.40:0/157870499 shutdown_connections 2023-11-08T08:23:49.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.541+0000 7fda3759e700 1 -- 172.21.15.40:0/157870499 wait complete. 2023-11-08T08:23:49.551 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 34 2023-11-08T08:23:49.572 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:23:49.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.741+0000 7f0304d7f700 1 Processor -- start 2023-11-08T08:23:49.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.741+0000 7f0304d7f700 1 -- start start 2023-11-08T08:23:49.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.741+0000 7f0304d7f700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f03001534a0 0x7f030015d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:49.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.741+0000 7f0304d7f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03000b6200 0x7f0300070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:49.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.741+0000 7f0304d7f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f030015d080 con 0x7f0300160920 2023-11-08T08:23:49.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.741+0000 7f0304d7f700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0300153870 con 0x7f03001534a0 2023-11-08T08:23:49.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.741+0000 7f0304d7f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03000b53c0 con 0x7f03000b6200 2023-11-08T08:23:49.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.741+0000 7f02f5d9b700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0300160920 0x7f03000b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50216/0 (socket says 172.21.15.40:50216) 2023-11-08T08:23:49.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.741+0000 7f02f5d9b700 1 -- 172.21.15.40:0/276501082 learned_addr learned my addr 172.21.15.40:0/276501082 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.741+0000 7f02fe59c700 1 --2- 172.21.15.40:0/276501082 >> v2:172.21.15.169:3300/0 conn(0x7f03001534a0 0x7f030015d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.741+0000 7f02fed9d700 1 --2- 172.21.15.40:0/276501082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03000b6200 0x7f0300070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.745+0000 7f02fdd9b700 1 -- 172.21.15.40:0/276501082 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3448919626 0 0) 0x7f030015d080 con 0x7f0300160920 2023-11-08T08:23:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.745+0000 7f02fdd9b700 1 -- 172.21.15.40:0/276501082 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02ec0035d0 con 0x7f0300160920 2023-11-08T08:23:49.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.745+0000 7f02fe59c700 1 -- 172.21.15.40:0/276501082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03000b6200 msgr2=0x7f0300070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:49.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.745+0000 7f02fe59c700 1 --2- 172.21.15.40:0/276501082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03000b6200 0x7f0300070630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.745+0000 7f02fe59c700 1 -- 172.21.15.40:0/276501082 >> v1:172.21.15.40:6789/0 conn(0x7f0300160920 legacy=0x7f03000b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:49.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.745+0000 7f02fe59c700 1 -- 172.21.15.40:0/276501082 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03000b56a0 con 0x7f03001534a0 2023-11-08T08:23:49.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.745+0000 7f02fe59c700 1 --2- 172.21.15.40:0/276501082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03001534a0 0x7f030015d510 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009850 tx=0x7f02f0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51109b6a6e55c609 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:49.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.745+0000 7f02fdd9b700 1 -- 172.21.15.40:0/276501082 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f0011830 con 0x7f03001534a0 2023-11-08T08:23:49.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.745+0000 7f02fdd9b700 1 -- 172.21.15.40:0/276501082 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f0011990 con 0x7f03001534a0 2023-11-08T08:23:49.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.745+0000 7f02fdd9b700 1 -- 172.21.15.40:0/276501082 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f0011d40 con 0x7f03001534a0 2023-11-08T08:23:49.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.745+0000 7f0304d7f700 1 -- 172.21.15.40:0/276501082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03001534a0 msgr2=0x7f030015d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:49.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.745+0000 7f0304d7f700 1 --2- 172.21.15.40:0/276501082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03001534a0 0x7f030015d510 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009850 tx=0x7f02f0009b60 comp rx=0 tx=0).stop 2023-11-08T08:23:49.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.745+0000 7f0304d7f700 1 -- 172.21.15.40:0/276501082 shutdown_connections 2023-11-08T08:23:49.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.745+0000 7f0304d7f700 1 --2- 172.21.15.40:0/276501082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03000b6200 0x7f0300070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.745+0000 7f0304d7f700 1 --2- 172.21.15.40:0/276501082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03001534a0 0x7f030015d510 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.745+0000 7f0304d7f700 1 -- 172.21.15.40:0/276501082 >> 172.21.15.40:0/276501082 conn(0x7f030006bfc0 msgr2=0x7f030006c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:49.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.745+0000 7f0304d7f700 1 -- 172.21.15.40:0/276501082 shutdown_connections 2023-11-08T08:23:49.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.745+0000 7f0304d7f700 1 -- 172.21.15.40:0/276501082 wait complete. 2023-11-08T08:23:49.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.753+0000 7f0304d7f700 1 Processor -- start 2023-11-08T08:23:49.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.753+0000 7f0304d7f700 1 -- start start 2023-11-08T08:23:49.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.753+0000 7f0304d7f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03000b6200 0x7f0300150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:49.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.753+0000 7f0304d7f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03001534a0 0x7f0300151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:49.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.753+0000 7f0304d7f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0300160920 0x7f03001f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:49.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.753+0000 7f0304d7f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030013d260 con 0x7f03001534a0 2023-11-08T08:23:49.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.753+0000 7f0304d7f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03001f6100 con 0x7f03000b6200 2023-11-08T08:23:49.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.753+0000 7f0304d7f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03001f2b50 con 0x7f0300160920 2023-11-08T08:23:49.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.753+0000 7f02f5d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03001534a0 0x7f0300151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:49.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.753+0000 7f02f5d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03001534a0 0x7f0300151400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41114/0 (socket says 172.21.15.40:41114) 2023-11-08T08:23:49.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.753+0000 7f02f5d9b700 1 -- 172.21.15.40:0/3378873952 learned_addr learned my addr 172.21.15.40:0/3378873952 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:49.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.753+0000 7f02fe59c700 1 --2- 172.21.15.40:0/3378873952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03000b6200 0x7f0300150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:49.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.753+0000 7f02fed9d700 1 --2- 172.21.15.40:0/3378873952 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0300160920 0x7f03001f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.753+0000 7f02f5d9b700 1 -- 172.21.15.40:0/3378873952 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0300160920 msgr2=0x7f03001f1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.753+0000 7f02f5d9b700 1 --2- 172.21.15.40:0/3378873952 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0300160920 0x7f03001f1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.753+0000 7f02f5d9b700 1 -- 172.21.15.40:0/3378873952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03000b6200 msgr2=0x7f0300150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.753+0000 7f02f5d9b700 1 --2- 172.21.15.40:0/3378873952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03000b6200 0x7f0300150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.753+0000 7f02f5d9b700 1 -- 172.21.15.40:0/3378873952 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f00094b0 con 0x7f03001534a0 2023-11-08T08:23:49.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.753+0000 7f02fe59c700 1 --2- 172.21.15.40:0/3378873952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03000b6200 0x7f0300150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:23:49.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.753+0000 7f02f5d9b700 1 --2- 172.21.15.40:0/3378873952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03001534a0 0x7f0300151400 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f02e800cbc0 tx=0x7f02e800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:49.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.757+0000 7f02f77fe700 1 -- 172.21.15.40:0/3378873952 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e800a770 con 0x7f03001534a0 2023-11-08T08:23:49.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.757+0000 7f02f77fe700 1 -- 172.21.15.40:0/3378873952 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e800a8d0 con 0x7f03001534a0 2023-11-08T08:23:49.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.757+0000 7f02f77fe700 1 -- 172.21.15.40:0/3378873952 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e800ac60 con 0x7f03001534a0 2023-11-08T08:23:49.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.757+0000 7f0304d7f700 1 -- 172.21.15.40:0/3378873952 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f030015cd10 con 0x7f03001534a0 2023-11-08T08:23:49.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.757+0000 7f0304d7f700 1 -- 172.21.15.40:0/3378873952 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f030015ca30 con 0x7f03001534a0 2023-11-08T08:23:49.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.757+0000 7f0304d7f700 1 -- 172.21.15.40:0/3378873952 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03000b7f10 con 0x7f03001534a0 2023-11-08T08:23:49.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.761+0000 7f02f77fe700 1 -- 172.21.15.40:0/3378873952 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f02e8019030 con 0x7f03001534a0 2023-11-08T08:23:49.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.761+0000 7f02f77fe700 1 --2- 172.21.15.40:0/3378873952 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f02e0064ee0 0x7f02e0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:49.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.761+0000 7f02f77fe700 1 -- 172.21.15.40:0/3378873952 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f02e8081940 con 0x7f03001534a0 2023-11-08T08:23:49.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.761+0000 7f02fe59c700 1 --2- 172.21.15.40:0/3378873952 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f02e0064ee0 0x7f02e0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:49.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.761+0000 7f02fe59c700 1 --2- 172.21.15.40:0/3378873952 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f02e0064ee0 0x7f02e0067390 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f02f00090b0 tx=0x7f02f0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:49.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.769+0000 7f02f77fe700 1 -- 172.21.15.40:0/3378873952 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f02e804f960 con 0x7f03001534a0 2023-11-08T08:23:49.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.941+0000 7f0304d7f700 1 -- 172.21.15.40:0/3378873952 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0300152220 con 0x7f03001534a0 2023-11-08T08:23:49.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.949+0000 7f02f77fe700 1 -- 172.21.15.40:0/3378873952 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f02e8052f80 con 0x7f03001534a0 2023-11-08T08:23:49.956 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:23:49.957 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":71,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:22:46.031951+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:22:18.224463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T08:22:19.658954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.26","mappings":[{"from":6,"to":0}]},{"pgid":"11.28","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:23:49.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.953+0000 7f0304d7f700 1 -- 172.21.15.40:0/3378873952 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f02e0064ee0 msgr2=0x7f02e0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:49.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.953+0000 7f0304d7f700 1 --2- 172.21.15.40:0/3378873952 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f02e0064ee0 0x7f02e0067390 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f02f00090b0 tx=0x7f02f0008030 comp rx=0 tx=0).stop 2023-11-08T08:23:49.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.953+0000 7f0304d7f700 1 -- 172.21.15.40:0/3378873952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03001534a0 msgr2=0x7f0300151400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:49.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.953+0000 7f0304d7f700 1 --2- 172.21.15.40:0/3378873952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03001534a0 0x7f0300151400 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f02e800cbc0 tx=0x7f02e800cbf0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.953+0000 7f0304d7f700 1 -- 172.21.15.40:0/3378873952 shutdown_connections 2023-11-08T08:23:49.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.953+0000 7f0304d7f700 1 --2- 172.21.15.40:0/3378873952 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f02e0064ee0 0x7f02e0067390 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.953+0000 7f0304d7f700 1 --2- 172.21.15.40:0/3378873952 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0300160920 0x7f03001f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.953+0000 7f0304d7f700 1 --2- 172.21.15.40:0/3378873952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03001534a0 0x7f0300151400 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.953+0000 7f0304d7f700 1 --2- 172.21.15.40:0/3378873952 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03000b6200 0x7f0300150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:49.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.953+0000 7f0304d7f700 1 -- 172.21.15.40:0/3378873952 >> 172.21.15.40:0/3378873952 conn(0x7f030006bfc0 msgr2=0x7f03000b5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:49.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.953+0000 7f0304d7f700 1 -- 172.21.15.40:0/3378873952 shutdown_connections 2023-11-08T08:23:49.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:49.953+0000 7f0304d7f700 1 -- 172.21.15.40:0/3378873952 wait complete. 2023-11-08T08:23:49.982 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T08:23:49.982 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:23:50.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.153+0000 7f27f1ebc700 1 Processor -- start 2023-11-08T08:23:50.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.157+0000 7f27f1ebc700 1 -- start start 2023-11-08T08:23:50.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.157+0000 7f27f1ebc700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f27ec06bed0 0x7f27ec06c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:50.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.157+0000 7f27f1ebc700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27ec0c9d40 con 0x7f27ec15c650 2023-11-08T08:23:50.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.157+0000 7f27f1ebc700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27ec06c830 con 0x7f27ec0cd0d0 2023-11-08T08:23:50.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.157+0000 7f27f1ebc700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27ec06b140 con 0x7f27ec06bed0 2023-11-08T08:23:50.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.157+0000 7f27f0eba700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f27ec15c650 0x7f27ec06b2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50222/0 (socket says 172.21.15.40:50222) 2023-11-08T08:23:50.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.157+0000 7f27f0eba700 1 -- 172.21.15.40:0/1011596711 learned_addr learned my addr 172.21.15.40:0/1011596711 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:50.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.161+0000 7f27f16bb700 1 --2- 172.21.15.40:0/1011596711 >> v2:172.21.15.169:3300/0 conn(0x7f27ec06bed0 0x7f27ec06c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:50.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.161+0000 7f27e37fe700 1 -- 172.21.15.40:0/1011596711 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2649556209 0 0) 0x7f27ec0c9d40 con 0x7f27ec15c650 2023-11-08T08:23:50.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.161+0000 7f27e37fe700 1 -- 172.21.15.40:0/1011596711 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27d40035d0 con 0x7f27ec15c650 2023-11-08T08:23:50.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.161+0000 7f27e37fe700 1 -- 172.21.15.40:0/1011596711 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737716356 0 0) 0x7f27ec06c830 con 0x7f27ec0cd0d0 2023-11-08T08:23:50.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.161+0000 7f27e37fe700 1 -- 172.21.15.40:0/1011596711 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27ec0c9d40 con 0x7f27ec0cd0d0 2023-11-08T08:23:50.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.161+0000 7f27f16bb700 1 -- 172.21.15.40:0/1011596711 >> v1:172.21.15.169:6789/0 conn(0x7f27ec0cd0d0 legacy=0x7f27ec0b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:50.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.161+0000 7f27e37fe700 1 -- 172.21.15.40:0/1011596711 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3210761299 0 0) 0x7f27d40035d0 con 0x7f27ec15c650 2023-11-08T08:23:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.161+0000 7f27f16bb700 1 -- 172.21.15.40:0/1011596711 >> v1:172.21.15.40:6789/0 conn(0x7f27ec15c650 legacy=0x7f27ec06b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.161+0000 7f27f16bb700 1 -- 172.21.15.40:0/1011596711 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ec0b1d50 con 0x7f27ec06bed0 2023-11-08T08:23:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.161+0000 7f27f16bb700 1 --2- 172.21.15.40:0/1011596711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27ec06bed0 0x7f27ec06c2f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f27e4009d90 tx=0x7f27e400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=802d32b22e040c79 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.161+0000 7f27e37fe700 1 -- 172.21.15.40:0/1011596711 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e400e070 con 0x7f27ec06bed0 2023-11-08T08:23:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.161+0000 7f27e37fe700 1 -- 172.21.15.40:0/1011596711 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e4003a40 con 0x7f27ec06bed0 2023-11-08T08:23:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.161+0000 7f27e37fe700 1 -- 172.21.15.40:0/1011596711 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e8001670 con 0x7f27ec06bed0 2023-11-08T08:23:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.165+0000 7f27f1ebc700 1 -- 172.21.15.40:0/1011596711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27ec06bed0 msgr2=0x7f27ec06c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.165+0000 7f27f1ebc700 1 --2- 172.21.15.40:0/1011596711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27ec06bed0 0x7f27ec06c2f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f27e4009d90 tx=0x7f27e400b330 comp rx=0 tx=0).stop 2023-11-08T08:23:50.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.165+0000 7f27f1ebc700 1 -- 172.21.15.40:0/1011596711 shutdown_connections 2023-11-08T08:23:50.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.165+0000 7f27f1ebc700 1 --2- 172.21.15.40:0/1011596711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27ec06bed0 0x7f27ec06c2f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:50.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.165+0000 7f27f1ebc700 1 -- 172.21.15.40:0/1011596711 >> 172.21.15.40:0/1011596711 conn(0x7f27ec0b0f80 msgr2=0x7f27ec0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.165+0000 7f27f1ebc700 1 -- 172.21.15.40:0/1011596711 shutdown_connections 2023-11-08T08:23:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.165+0000 7f27f1ebc700 1 -- 172.21.15.40:0/1011596711 wait complete. 2023-11-08T08:23:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.165+0000 7f27f1ebc700 1 Processor -- start 2023-11-08T08:23:50.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.165+0000 7f27f1ebc700 1 -- start start 2023-11-08T08:23:50.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27f1ebc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27ec06bed0 0x7f27ec0c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:50.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27f1ebc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27ec0cd0d0 0x7f27ec0c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:50.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27f1ebc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27ec15c650 0x7f27ec153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:50.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27f1ebc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec0c6db0 con 0x7f27ec0cd0d0 2023-11-08T08:23:50.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27f1ebc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec0c7a70 con 0x7f27ec15c650 2023-11-08T08:23:50.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27f1ebc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec0c7d50 con 0x7f27ec06bed0 2023-11-08T08:23:50.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27e3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27ec0cd0d0 0x7f27ec0c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:50.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27f16bb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27ec15c650 0x7f27ec153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:50.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27f0eba700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27ec06bed0 0x7f27ec0c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:50.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27e3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27ec0cd0d0 0x7f27ec0c6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41118/0 (socket says 172.21.15.40:41118) 2023-11-08T08:23:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27e3fff700 1 -- 172.21.15.40:0/2556660229 learned_addr learned my addr 172.21.15.40:0/2556660229 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27f16bb700 1 -- 172.21.15.40:0/2556660229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27ec06bed0 msgr2=0x7f27ec0c4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27f16bb700 1 --2- 172.21.15.40:0/2556660229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27ec06bed0 0x7f27ec0c4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27f16bb700 1 -- 172.21.15.40:0/2556660229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27ec0cd0d0 msgr2=0x7f27ec0c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27f16bb700 1 --2- 172.21.15.40:0/2556660229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27ec0cd0d0 0x7f27ec0c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27f16bb700 1 -- 172.21.15.40:0/2556660229 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e4009a70 con 0x7f27ec15c650 2023-11-08T08:23:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27f16bb700 1 --2- 172.21.15.40:0/2556660229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27ec15c650 0x7f27ec153aa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f27e4014000 tx=0x7f27e4004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27e17fa700 1 -- 172.21.15.40:0/2556660229 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e4004440 con 0x7f27ec15c650 2023-11-08T08:23:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27e17fa700 1 -- 172.21.15.40:0/2556660229 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e40045a0 con 0x7f27ec15c650 2023-11-08T08:23:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27e17fa700 1 -- 172.21.15.40:0/2556660229 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e4004910 con 0x7f27ec15c650 2023-11-08T08:23:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27f1ebc700 1 -- 172.21.15.40:0/2556660229 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ec0cd8b0 con 0x7f27ec15c650 2023-11-08T08:23:50.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.169+0000 7f27f1ebc700 1 -- 172.21.15.40:0/2556660229 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ec0cd5d0 con 0x7f27ec15c650 2023-11-08T08:23:50.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.173+0000 7f27f1ebc700 1 -- 172.21.15.40:0/2556660229 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ec0c99d0 con 0x7f27ec15c650 2023-11-08T08:23:50.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.173+0000 7f27e17fa700 1 -- 172.21.15.40:0/2556660229 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f27e4018070 con 0x7f27ec15c650 2023-11-08T08:23:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.177+0000 7f27e17fa700 1 --2- 172.21.15.40:0/2556660229 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f27d8064ee0 0x7f27d8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.177+0000 7f27e17fa700 1 -- 172.21.15.40:0/2556660229 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f27e4083600 con 0x7f27ec15c650 2023-11-08T08:23:50.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.177+0000 7f27f0eba700 1 --2- 172.21.15.40:0/2556660229 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f27d8064ee0 0x7f27d8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:50.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.177+0000 7f27f0eba700 1 --2- 172.21.15.40:0/2556660229 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f27d8064ee0 0x7f27d8067390 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f27e8001380 tx=0x7f27e8007c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:50.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.185+0000 7f27e17fa700 1 -- 172.21.15.40:0/2556660229 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f27e4051530 con 0x7f27ec15c650 2023-11-08T08:23:50.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.401+0000 7f27f1ebc700 1 -- 172.21.15.40:0/2556660229 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27ec073cb0 con 0x7f27ec15c650 2023-11-08T08:23:50.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.401+0000 7f27e17fa700 1 -- 172.21.15.40:0/2556660229 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f27e4021090 con 0x7f27ec15c650 2023-11-08T08:23:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:23:50.410 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/307473250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":307473250},{"type":"v1","addr":"172.21.15.169:6835","nonce":307473250}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6148,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/924518366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":924518366},{"type":"v1","addr":"172.21.15.169:6837","nonce":924518366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4294005934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4294005934},{"type":"v1","addr":"172.21.15.40:6833","nonce":4294005934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:22:20.650687+0000","modified":"2023-11-08T08:22:37.032476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6237":{"gid":6237,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":7,"addr":"172.21.15.40:6835/3395630689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3395630689},{"type":"v1","addr":"172.21.15.40:6835","nonce":3395630689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T08:23:50.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.405+0000 7f27f1ebc700 1 -- 172.21.15.40:0/2556660229 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f27d8064ee0 msgr2=0x7f27d8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:50.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.405+0000 7f27f1ebc700 1 --2- 172.21.15.40:0/2556660229 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f27d8064ee0 0x7f27d8067390 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f27e8001380 tx=0x7f27e8007c00 comp rx=0 tx=0).stop 2023-11-08T08:23:50.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.405+0000 7f27f1ebc700 1 -- 172.21.15.40:0/2556660229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27ec15c650 msgr2=0x7f27ec153aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:50.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.405+0000 7f27f1ebc700 1 --2- 172.21.15.40:0/2556660229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27ec15c650 0x7f27ec153aa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f27e4014000 tx=0x7f27e4004af0 comp rx=0 tx=0).stop 2023-11-08T08:23:50.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.409+0000 7f27f1ebc700 1 -- 172.21.15.40:0/2556660229 shutdown_connections 2023-11-08T08:23:50.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.409+0000 7f27f1ebc700 1 --2- 172.21.15.40:0/2556660229 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f27d8064ee0 0x7f27d8067390 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:50.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.409+0000 7f27f1ebc700 1 --2- 172.21.15.40:0/2556660229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27ec15c650 0x7f27ec153aa0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:50.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.409+0000 7f27f1ebc700 1 --2- 172.21.15.40:0/2556660229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27ec0cd0d0 0x7f27ec0c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:50.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.409+0000 7f27f1ebc700 1 --2- 172.21.15.40:0/2556660229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27ec06bed0 0x7f27ec0c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:50.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.409+0000 7f27f1ebc700 1 -- 172.21.15.40:0/2556660229 >> 172.21.15.40:0/2556660229 conn(0x7f27ec0b0f80 msgr2=0x7f27ec0ca1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:50.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.409+0000 7f27f1ebc700 1 -- 172.21.15.40:0/2556660229 shutdown_connections 2023-11-08T08:23:50.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.409+0000 7f27f1ebc700 1 -- 172.21.15.40:0/2556660229 wait complete. 2023-11-08T08:23:50.417 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 34 2023-11-08T08:23:50.438 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:23:50.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.613+0000 7fad118cb700 1 Processor -- start 2023-11-08T08:23:50.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.613+0000 7fad118cb700 1 -- start start 2023-11-08T08:23:50.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.613+0000 7fad118cb700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fad0c15c640 0x7fad0c0d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:50.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.613+0000 7fad118cb700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0c06c600 con 0x7fad0c071aa0 2023-11-08T08:23:50.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.613+0000 7fad118cb700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0c0722f0 con 0x7fad0c069e00 2023-11-08T08:23:50.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.613+0000 7fad118cb700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad0c06cef0 con 0x7fad0c15c640 2023-11-08T08:23:50.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.613+0000 7fad0bfff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fad0c071aa0 0x7fad0c06d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50238/0 (socket says 172.21.15.40:50238) 2023-11-08T08:23:50.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.613+0000 7fad0bfff700 1 -- 172.21.15.40:0/3377433542 learned_addr learned my addr 172.21.15.40:0/3377433542 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:50.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.613+0000 7fad108c9700 1 --2- 172.21.15.40:0/3377433542 >> v2:172.21.15.169:3300/0 conn(0x7fad0c15c640 0x7fad0c0d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:50.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.617+0000 7fad0b7fe700 1 -- 172.21.15.40:0/3377433542 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3804529107 0 0) 0x7fad0c06c600 con 0x7fad0c071aa0 2023-11-08T08:23:50.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.617+0000 7fad0b7fe700 1 -- 172.21.15.40:0/3377433542 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facfc0035d0 con 0x7fad0c071aa0 2023-11-08T08:23:50.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.617+0000 7fad0b7fe700 1 -- 172.21.15.40:0/3377433542 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1242356327 0 0) 0x7fad0c0722f0 con 0x7fad0c069e00 2023-11-08T08:23:50.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.617+0000 7fad0b7fe700 1 -- 172.21.15.40:0/3377433542 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad0c06c600 con 0x7fad0c069e00 2023-11-08T08:23:50.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.617+0000 7fad108c9700 1 -- 172.21.15.40:0/3377433542 >> v1:172.21.15.169:6789/0 conn(0x7fad0c069e00 legacy=0x7fad0c0d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:50.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.617+0000 7fad108c9700 1 -- 172.21.15.40:0/3377433542 >> v1:172.21.15.40:6789/0 conn(0x7fad0c071aa0 legacy=0x7fad0c06d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:50.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.617+0000 7fad108c9700 1 -- 172.21.15.40:0/3377433542 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c06da20 con 0x7fad0c15c640 2023-11-08T08:23:50.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.617+0000 7fad108c9700 1 --2- 172.21.15.40:0/3377433542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad0c15c640 0x7fad0c0d31e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fad00009850 tx=0x7fad00009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c2d8780b180ef51 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:50.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.617+0000 7fad0b7fe700 1 -- 172.21.15.40:0/3377433542 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00011830 con 0x7fad0c15c640 2023-11-08T08:23:50.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.617+0000 7fad0b7fe700 1 -- 172.21.15.40:0/3377433542 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad00011990 con 0x7fad0c15c640 2023-11-08T08:23:50.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.617+0000 7fad0b7fe700 1 -- 172.21.15.40:0/3377433542 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00011d40 con 0x7fad0c15c640 2023-11-08T08:23:50.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.617+0000 7fad118cb700 1 -- 172.21.15.40:0/3377433542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad0c15c640 msgr2=0x7fad0c0d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:50.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.617+0000 7fad118cb700 1 --2- 172.21.15.40:0/3377433542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad0c15c640 0x7fad0c0d31e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fad00009850 tx=0x7fad00009b60 comp rx=0 tx=0).stop 2023-11-08T08:23:50.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.617+0000 7fad118cb700 1 -- 172.21.15.40:0/3377433542 shutdown_connections 2023-11-08T08:23:50.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.617+0000 7fad118cb700 1 --2- 172.21.15.40:0/3377433542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad0c15c640 0x7fad0c0d31e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:50.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.617+0000 7fad118cb700 1 -- 172.21.15.40:0/3377433542 >> 172.21.15.40:0/3377433542 conn(0x7fad0c0b6520 msgr2=0x7fad0c071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:50.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.617+0000 7fad118cb700 1 -- 172.21.15.40:0/3377433542 shutdown_connections 2023-11-08T08:23:50.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.617+0000 7fad118cb700 1 -- 172.21.15.40:0/3377433542 wait complete. 2023-11-08T08:23:50.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.621+0000 7fad118cb700 1 Processor -- start 2023-11-08T08:23:50.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.621+0000 7fad118cb700 1 -- start start 2023-11-08T08:23:50.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.621+0000 7fad118cb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad0c069e00 0x7fad0c1f4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:50.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.621+0000 7fad118cb700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad0c071aa0 0x7fad0c0c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:50.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.625+0000 7fad118cb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad0c15c640 0x7fad0c0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:50.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.625+0000 7fad118cb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0c7ee0 con 0x7fad0c15c640 2023-11-08T08:23:50.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.625+0000 7fad118cb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0b3be0 con 0x7fad0c069e00 2023-11-08T08:23:50.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.625+0000 7fad118cb700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0b3ec0 con 0x7fad0c071aa0 2023-11-08T08:23:50.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.625+0000 7fad110ca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad0c15c640 0x7fad0c0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:50.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.625+0000 7fad108c9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad0c069e00 0x7fad0c1f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:50.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.625+0000 7fad108c9700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad0c069e00 0x7fad0c1f4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:40600/0 (socket says 172.21.15.40:40600) 2023-11-08T08:23:50.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.625+0000 7fad108c9700 1 -- 172.21.15.40:0/3432209924 learned_addr learned my addr 172.21.15.40:0/3432209924 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:50.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.625+0000 7fad0bfff700 1 --2- 172.21.15.40:0/3432209924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad0c071aa0 0x7fad0c0c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:50.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.625+0000 7fad108c9700 1 -- 172.21.15.40:0/3432209924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad0c071aa0 msgr2=0x7fad0c0c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:50.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.625+0000 7fad108c9700 1 --2- 172.21.15.40:0/3432209924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad0c071aa0 0x7fad0c0c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:50.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.625+0000 7fad108c9700 1 -- 172.21.15.40:0/3432209924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad0c15c640 msgr2=0x7fad0c0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.625+0000 7fad108c9700 1 --2- 172.21.15.40:0/3432209924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad0c15c640 0x7fad0c0cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.625+0000 7fad108c9700 1 -- 172.21.15.40:0/3432209924 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad000094b0 con 0x7fad0c069e00 2023-11-08T08:23:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.625+0000 7fad108c9700 1 --2- 172.21.15.40:0/3432209924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad0c069e00 0x7fad0c1f4880 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fad000090b0 tx=0x7fad00004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.625+0000 7fad097fa700 1 -- 172.21.15.40:0/3432209924 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0001a950 con 0x7fad0c069e00 2023-11-08T08:23:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.625+0000 7fad097fa700 1 -- 172.21.15.40:0/3432209924 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0001f070 con 0x7fad0c069e00 2023-11-08T08:23:50.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.625+0000 7fad097fa700 1 -- 172.21.15.40:0/3432209924 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0001ad10 con 0x7fad0c069e00 2023-11-08T08:23:50.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.625+0000 7fad118cb700 1 -- 172.21.15.40:0/3432209924 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0c072010 con 0x7fad0c069e00 2023-11-08T08:23:50.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.625+0000 7fad118cb700 1 -- 172.21.15.40:0/3432209924 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0c15ca10 con 0x7fad0c069e00 2023-11-08T08:23:50.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.629+0000 7fad118cb700 1 -- 172.21.15.40:0/3432209924 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad0c06c830 con 0x7fad0c069e00 2023-11-08T08:23:50.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.629+0000 7fad097fa700 1 -- 172.21.15.40:0/3432209924 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fad0000f050 con 0x7fad0c069e00 2023-11-08T08:23:50.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.629+0000 7fad097fa700 1 --2- 172.21.15.40:0/3432209924 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7facf0064ee0 0x7facf0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:50.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.629+0000 7fad097fa700 1 -- 172.21.15.40:0/3432209924 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fad000831a0 con 0x7fad0c069e00 2023-11-08T08:23:50.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.633+0000 7fad0bfff700 1 --2- 172.21.15.40:0/3432209924 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7facf0064ee0 0x7facf0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:50.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.633+0000 7fad0bfff700 1 --2- 172.21.15.40:0/3432209924 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7facf0064ee0 0x7facf0067390 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0a08d0 tx=0x7facf8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:50.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.637+0000 7fad097fa700 1 -- 172.21.15.40:0/3432209924 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fad000510d0 con 0x7fad0c069e00 2023-11-08T08:23:50.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.849+0000 7fad118cb700 1 -- 172.21.15.40:0/3432209924 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad0c0d3950 con 0x7fad0c069e00 2023-11-08T08:23:50.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.849+0000 7fad097fa700 1 -- 172.21.15.40:0/3432209924 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4650 (secure 0 0 0) 0x7fad000546f0 con 0x7fad0c069e00 2023-11-08T08:23:50.857 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:23:50.857 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/307473250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":307473250},{"type":"v1","addr":"172.21.15.169:6835","nonce":307473250}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6148,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/924518366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":924518366},{"type":"v1","addr":"172.21.15.169:6837","nonce":924518366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4294005934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4294005934},{"type":"v1","addr":"172.21.15.40:6833","nonce":4294005934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T08:22:20.650687+0000","modified":"2023-11-08T08:22:37.032476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6237":{"gid":6237,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":7,"addr":"172.21.15.40:6835/3395630689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3395630689},{"type":"v1","addr":"172.21.15.40:6835","nonce":3395630689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T08:23:50.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.853+0000 7fad118cb700 1 -- 172.21.15.40:0/3432209924 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7facf0064ee0 msgr2=0x7facf0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:50.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.853+0000 7fad118cb700 1 --2- 172.21.15.40:0/3432209924 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7facf0064ee0 0x7facf0067390 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0a08d0 tx=0x7facf8009040 comp rx=0 tx=0).stop 2023-11-08T08:23:50.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.853+0000 7fad118cb700 1 -- 172.21.15.40:0/3432209924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad0c069e00 msgr2=0x7fad0c1f4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:50.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.853+0000 7fad118cb700 1 --2- 172.21.15.40:0/3432209924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad0c069e00 0x7fad0c1f4880 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fad000090b0 tx=0x7fad00004770 comp rx=0 tx=0).stop 2023-11-08T08:23:50.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.853+0000 7fad118cb700 1 -- 172.21.15.40:0/3432209924 shutdown_connections 2023-11-08T08:23:50.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.853+0000 7fad118cb700 1 --2- 172.21.15.40:0/3432209924 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7facf0064ee0 0x7facf0067390 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:50.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.853+0000 7fad118cb700 1 --2- 172.21.15.40:0/3432209924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad0c15c640 0x7fad0c0cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:50.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.853+0000 7fad118cb700 1 --2- 172.21.15.40:0/3432209924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad0c071aa0 0x7fad0c0c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:50.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.853+0000 7fad118cb700 1 --2- 172.21.15.40:0/3432209924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad0c069e00 0x7fad0c1f4880 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:50.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.853+0000 7fad118cb700 1 -- 172.21.15.40:0/3432209924 >> 172.21.15.40:0/3432209924 conn(0x7fad0c0b6520 msgr2=0x7fad0c0b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:50.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.853+0000 7fad118cb700 1 -- 172.21.15.40:0/3432209924 shutdown_connections 2023-11-08T08:23:50.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:50.853+0000 7fad118cb700 1 -- 172.21.15.40:0/3432209924 wait complete. 2023-11-08T08:23:50.862 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 34 2023-11-08T08:23:50.882 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:23:51.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.061+0000 7f6d5cacb700 1 Processor -- start 2023-11-08T08:23:51.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.061+0000 7f6d5cacb700 1 -- start start 2023-11-08T08:23:51.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.061+0000 7f6d5cacb700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6d5815c640 0x7f6d580d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:51.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.061+0000 7f6d5cacb700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6d58069e00 0x7f6d5806a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:51.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.061+0000 7f6d5cacb700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d5806c600 con 0x7f6d58071aa0 2023-11-08T08:23:51.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.061+0000 7f6d5cacb700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d5806cef0 con 0x7f6d58069e00 2023-11-08T08:23:51.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.061+0000 7f6d5cacb700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d5806a7c0 con 0x7f6d5815c640 2023-11-08T08:23:51.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.061+0000 7f6d57fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6d58069e00 0x7f6d5806a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:51.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.061+0000 7f6d57fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6d58069e00 0x7f6d5806a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41136/0 (socket says 172.21.15.40:41136) 2023-11-08T08:23:51.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.061+0000 7f6d57fff700 1 -- 172.21.15.40:0/3414557265 learned_addr learned my addr 172.21.15.40:0/3414557265 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:51.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.061+0000 7f6d577fe700 1 --2- 172.21.15.40:0/3414557265 >> v2:172.21.15.169:3300/0 conn(0x7f6d5815c640 0x7f6d580d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:51.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.061+0000 7f6d57fff700 1 -- 172.21.15.40:0/3414557265 >> v2:172.21.15.169:3300/0 conn(0x7f6d5815c640 msgr2=0x7f6d580d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:51.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.061+0000 7f6d57fff700 1 --2- 172.21.15.40:0/3414557265 >> v2:172.21.15.169:3300/0 conn(0x7f6d5815c640 0x7f6d580d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:51.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.061+0000 7f6d57fff700 1 -- 172.21.15.40:0/3414557265 >> v1:172.21.15.169:6789/0 conn(0x7f6d58071aa0 legacy=0x7f6d5806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:51.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.061+0000 7f6d57fff700 1 -- 172.21.15.40:0/3414557265 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d580722f0 con 0x7f6d58069e00 2023-11-08T08:23:51.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.061+0000 7f6d577fe700 1 --2- 172.21.15.40:0/3414557265 >> v2:172.21.15.169:3300/0 conn(0x7f6d5815c640 0x7f6d580d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:23:51.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.065+0000 7f6d57fff700 1 --2- 172.21.15.40:0/3414557265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d58069e00 0x7f6d5806a280 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6d5806e290 tx=0x7f6d5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d110c5820740e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:51.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.065+0000 7f6d567fc700 1 -- 172.21.15.40:0/3414557265 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5000e070 con 0x7f6d58069e00 2023-11-08T08:23:51.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.065+0000 7f6d567fc700 1 -- 172.21.15.40:0/3414557265 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5000b9c0 con 0x7f6d58069e00 2023-11-08T08:23:51.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.065+0000 7f6d567fc700 1 -- 172.21.15.40:0/3414557265 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5000bd70 con 0x7f6d58069e00 2023-11-08T08:23:51.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.065+0000 7f6d5cacb700 1 -- 172.21.15.40:0/3414557265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d58069e00 msgr2=0x7f6d5806a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:51.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.065+0000 7f6d5cacb700 1 --2- 172.21.15.40:0/3414557265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d58069e00 0x7f6d5806a280 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6d5806e290 tx=0x7f6d5000b330 comp rx=0 tx=0).stop 2023-11-08T08:23:51.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.065+0000 7f6d5cacb700 1 -- 172.21.15.40:0/3414557265 shutdown_connections 2023-11-08T08:23:51.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.065+0000 7f6d5cacb700 1 --2- 172.21.15.40:0/3414557265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d58069e00 0x7f6d5806a280 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:51.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.065+0000 7f6d5cacb700 1 --2- 172.21.15.40:0/3414557265 >> v2:172.21.15.169:3300/0 conn(0x7f6d5815c640 0x7f6d580d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:51.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.065+0000 7f6d5cacb700 1 -- 172.21.15.40:0/3414557265 >> 172.21.15.40:0/3414557265 conn(0x7f6d580b6520 msgr2=0x7f6d58071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:51.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.065+0000 7f6d5cacb700 1 -- 172.21.15.40:0/3414557265 shutdown_connections 2023-11-08T08:23:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.065+0000 7f6d5cacb700 1 -- 172.21.15.40:0/3414557265 wait complete. 2023-11-08T08:23:51.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.069+0000 7f6d5cacb700 1 Processor -- start 2023-11-08T08:23:51.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.069+0000 7f6d5cacb700 1 -- start start 2023-11-08T08:23:51.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.069+0000 7f6d5cacb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d58069e00 0x7f6d5809bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:51.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.069+0000 7f6d5cacb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d58071aa0 0x7f6d580c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:51.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.069+0000 7f6d5cacb700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d5815c640 0x7f6d580b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:51.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.069+0000 7f6d5cacb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d580b42a0 con 0x7f6d58071aa0 2023-11-08T08:23:51.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.069+0000 7f6d5cacb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d580cf4c0 con 0x7f6d58069e00 2023-11-08T08:23:51.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.069+0000 7f6d5cacb700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d580cfcd0 con 0x7f6d5815c640 2023-11-08T08:23:51.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.069+0000 7f6d56ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d58071aa0 0x7f6d580c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:51.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.069+0000 7f6d56ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d58071aa0 0x7f6d580c77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41150/0 (socket says 172.21.15.40:41150) 2023-11-08T08:23:51.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.069+0000 7f6d56ffd700 1 -- 172.21.15.40:0/133268349 learned_addr learned my addr 172.21.15.40:0/133268349 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:51.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.069+0000 7f6d57fff700 1 --2- 172.21.15.40:0/133268349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d5815c640 0x7f6d580b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:51.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.069+0000 7f6d577fe700 1 --2- 172.21.15.40:0/133268349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d58069e00 0x7f6d5809bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:51.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.069+0000 7f6d56ffd700 1 -- 172.21.15.40:0/133268349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d5815c640 msgr2=0x7f6d580b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:51.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.069+0000 7f6d56ffd700 1 --2- 172.21.15.40:0/133268349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d5815c640 0x7f6d580b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:51.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.069+0000 7f6d56ffd700 1 -- 172.21.15.40:0/133268349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d58069e00 msgr2=0x7f6d5809bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:51.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.069+0000 7f6d56ffd700 1 --2- 172.21.15.40:0/133268349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d58069e00 0x7f6d5809bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:51.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.069+0000 7f6d56ffd700 1 -- 172.21.15.40:0/133268349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d50009b30 con 0x7f6d58071aa0 2023-11-08T08:23:51.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.069+0000 7f6d57fff700 1 --2- 172.21.15.40:0/133268349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d5815c640 0x7f6d580b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:23:51.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.073+0000 7f6d56ffd700 1 --2- 172.21.15.40:0/133268349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d58071aa0 0x7f6d580c77e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00c530 tx=0x7f6d4c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:51.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.073+0000 7f6d3ffff700 1 -- 172.21.15.40:0/133268349 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0052d0 con 0x7f6d58071aa0 2023-11-08T08:23:51.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.073+0000 7f6d3ffff700 1 -- 172.21.15.40:0/133268349 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4c005430 con 0x7f6d58071aa0 2023-11-08T08:23:51.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.073+0000 7f6d5cacb700 1 -- 172.21.15.40:0/133268349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5815ca10 con 0x7f6d58071aa0 2023-11-08T08:23:51.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.073+0000 7f6d3ffff700 1 -- 172.21.15.40:0/133268349 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0055a0 con 0x7f6d58071aa0 2023-11-08T08:23:51.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.073+0000 7f6d5cacb700 1 -- 172.21.15.40:0/133268349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5806da20 con 0x7f6d58071aa0 2023-11-08T08:23:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.073+0000 7f6d5cacb700 1 -- 172.21.15.40:0/133268349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d58072010 con 0x7f6d58071aa0 2023-11-08T08:23:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.073+0000 7f6d3ffff700 1 -- 172.21.15.40:0/133268349 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f6d4c019030 con 0x7f6d58071aa0 2023-11-08T08:23:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.077+0000 7f6d3ffff700 1 --2- 172.21.15.40:0/133268349 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6d44064ee0 0x7f6d44067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:51.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.077+0000 7f6d3ffff700 1 -- 172.21.15.40:0/133268349 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f6d4c0813b0 con 0x7f6d58071aa0 2023-11-08T08:23:51.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.077+0000 7f6d577fe700 1 --2- 172.21.15.40:0/133268349 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6d44064ee0 0x7f6d44067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:51.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.077+0000 7f6d577fe700 1 --2- 172.21.15.40:0/133268349 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6d44064ee0 0x7f6d44067390 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6d580d18f0 tx=0x7f6d48008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:51.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.081+0000 7f6d3ffff700 1 -- 172.21.15.40:0/133268349 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f6d4c04f2e0 con 0x7f6d58071aa0 2023-11-08T08:23:51.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.257+0000 7f6d5cacb700 1 -- 172.21.15.40:0/133268349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d580d3920 con 0x7f6d58071aa0 2023-11-08T08:23:51.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.257+0000 7f6d3ffff700 1 -- 172.21.15.40:0/133268349 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f6d4c052900 con 0x7f6d58071aa0 2023-11-08T08:23:51.266 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:23:51.266 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":71,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:22:46.031951+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T08:22:18.224463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T08:22:19.658954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.26","mappings":[{"from":6,"to":0}]},{"pgid":"11.28","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:23:51.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.261+0000 7f6d5cacb700 1 -- 172.21.15.40:0/133268349 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6d44064ee0 msgr2=0x7f6d44067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:51.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.261+0000 7f6d5cacb700 1 --2- 172.21.15.40:0/133268349 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6d44064ee0 0x7f6d44067390 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6d580d18f0 tx=0x7f6d48008040 comp rx=0 tx=0).stop 2023-11-08T08:23:51.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.261+0000 7f6d5cacb700 1 -- 172.21.15.40:0/133268349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d58071aa0 msgr2=0x7f6d580c77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:51.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.261+0000 7f6d5cacb700 1 --2- 172.21.15.40:0/133268349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d58071aa0 0x7f6d580c77e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00c530 tx=0x7f6d4c00c8f0 comp rx=0 tx=0).stop 2023-11-08T08:23:51.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.261+0000 7f6d5cacb700 1 -- 172.21.15.40:0/133268349 shutdown_connections 2023-11-08T08:23:51.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.261+0000 7f6d5cacb700 1 --2- 172.21.15.40:0/133268349 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6d44064ee0 0x7f6d44067390 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:51.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.261+0000 7f6d5cacb700 1 --2- 172.21.15.40:0/133268349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d5815c640 0x7f6d580b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:51.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.261+0000 7f6d5cacb700 1 --2- 172.21.15.40:0/133268349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d58071aa0 0x7f6d580c77e0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:51.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.261+0000 7f6d5cacb700 1 --2- 172.21.15.40:0/133268349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d58069e00 0x7f6d5809bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:51.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.261+0000 7f6d5cacb700 1 -- 172.21.15.40:0/133268349 >> 172.21.15.40:0/133268349 conn(0x7f6d580b6520 msgr2=0x7f6d580b80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:51.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.261+0000 7f6d5cacb700 1 -- 172.21.15.40:0/133268349 shutdown_connections 2023-11-08T08:23:51.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.261+0000 7f6d5cacb700 1 -- 172.21.15.40:0/133268349 wait complete. 2023-11-08T08:23:51.292 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T08:23:51.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.481+0000 7f52a1301700 1 Processor -- start 2023-11-08T08:23:51.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.481+0000 7f52a1301700 1 -- start start 2023-11-08T08:23:51.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.481+0000 7f52a1301700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f529c15a480 0x7f529c0cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:51.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.481+0000 7f52a1301700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f529c0c8d10 0x7f529c0cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:51.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.481+0000 7f52a1301700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f529c15a850 con 0x7f529c0ccd10 2023-11-08T08:23:51.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.481+0000 7f52a1301700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529c0ce060 con 0x7f529c0c8d10 2023-11-08T08:23:51.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.481+0000 7f52a1301700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529c0ce300 con 0x7f529c15a480 2023-11-08T08:23:51.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.481+0000 7f529affd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f529c15a480 0x7f529c0cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:51.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.481+0000 7f529b7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f529c0ccd10 0x7f529c15dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50242/0 (socket says 172.21.15.40:50242) 2023-11-08T08:23:51.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.481+0000 7f529b7fe700 1 -- 172.21.15.40:0/1264256404 learned_addr learned my addr 172.21.15.40:0/1264256404 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:51.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.481+0000 7f529a7fc700 1 --2- 172.21.15.40:0/1264256404 >> v2:172.21.15.40:3300/0 conn(0x7f529c0c8d10 0x7f529c0cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:51.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.481+0000 7f529affd700 1 -- 172.21.15.40:0/1264256404 >> v2:172.21.15.40:3300/0 conn(0x7f529c0c8d10 msgr2=0x7f529c0cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:51.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.481+0000 7f5299ffb700 1 -- 172.21.15.40:0/1264256404 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260835984 0 0) 0x7f529c15a850 con 0x7f529c0ccd10 2023-11-08T08:23:51.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.481+0000 7f529affd700 1 --2- 172.21.15.40:0/1264256404 >> v2:172.21.15.40:3300/0 conn(0x7f529c0c8d10 0x7f529c0cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:51.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.481+0000 7f529affd700 1 -- 172.21.15.40:0/1264256404 >> v1:172.21.15.40:6789/0 conn(0x7f529c0ccd10 legacy=0x7f529c15dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:51.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.481+0000 7f529affd700 1 -- 172.21.15.40:0/1264256404 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c0cada0 con 0x7f529c15a480 2023-11-08T08:23:51.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.485+0000 7f529affd700 1 --2- 172.21.15.40:0/1264256404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f529c15a480 0x7f529c0cc290 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f528c009850 tx=0x7f528c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=490f97b419244792 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:51.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.485+0000 7f5299ffb700 1 -- 172.21.15.40:0/1264256404 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c011830 con 0x7f529c15a480 2023-11-08T08:23:51.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.485+0000 7f5299ffb700 1 -- 172.21.15.40:0/1264256404 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528c011990 con 0x7f529c15a480 2023-11-08T08:23:51.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.485+0000 7f5299ffb700 1 -- 172.21.15.40:0/1264256404 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c011d40 con 0x7f529c15a480 2023-11-08T08:23:51.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.485+0000 7f52a1301700 1 -- 172.21.15.40:0/1264256404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f529c15a480 msgr2=0x7f529c0cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:51.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.485+0000 7f52a1301700 1 --2- 172.21.15.40:0/1264256404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f529c15a480 0x7f529c0cc290 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f528c009850 tx=0x7f528c009b60 comp rx=0 tx=0).stop 2023-11-08T08:23:51.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.485+0000 7f52a1301700 1 -- 172.21.15.40:0/1264256404 shutdown_connections 2023-11-08T08:23:51.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.485+0000 7f52a1301700 1 --2- 172.21.15.40:0/1264256404 >> v2:172.21.15.40:3300/0 conn(0x7f529c0c8d10 0x7f529c0cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:51.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.485+0000 7f52a1301700 1 --2- 172.21.15.40:0/1264256404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f529c15a480 0x7f529c0cc290 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:51.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.485+0000 7f52a1301700 1 -- 172.21.15.40:0/1264256404 >> 172.21.15.40:0/1264256404 conn(0x7f529c0cf350 msgr2=0x7f529c0b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:51.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.485+0000 7f52a1301700 1 -- 172.21.15.40:0/1264256404 shutdown_connections 2023-11-08T08:23:51.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.485+0000 7f52a1301700 1 -- 172.21.15.40:0/1264256404 wait complete. 2023-11-08T08:23:51.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.489+0000 7f52a1301700 1 Processor -- start 2023-11-08T08:23:51.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.489+0000 7f52a1301700 1 -- start start 2023-11-08T08:23:51.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.489+0000 7f52a1301700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f529c0c8d10 0x7f529c15bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:51.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.489+0000 7f52a1301700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529c0ccd10 0x7f529c153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:51.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.489+0000 7f52a1301700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f529c15a480 0x7f529c155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:51.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.489+0000 7f52a1301700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c153fc0 con 0x7f529c0ccd10 2023-11-08T08:23:51.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.489+0000 7f52a1301700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c155db0 con 0x7f529c15a480 2023-11-08T08:23:51.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.489+0000 7f52a1301700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529c15c5c0 con 0x7f529c0c8d10 2023-11-08T08:23:51.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.489+0000 7f529a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529c0ccd10 0x7f529c153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:51.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.489+0000 7f529a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529c0ccd10 0x7f529c153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41174/0 (socket says 172.21.15.40:41174) 2023-11-08T08:23:51.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.489+0000 7f529a7fc700 1 -- 172.21.15.40:0/364847931 learned_addr learned my addr 172.21.15.40:0/364847931 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:51.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.493+0000 7f529b7fe700 1 --2- 172.21.15.40:0/364847931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f529c15a480 0x7f529c155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:51.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.493+0000 7f529affd700 1 --2- 172.21.15.40:0/364847931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f529c0c8d10 0x7f529c15bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:51.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.493+0000 7f529a7fc700 1 -- 172.21.15.40:0/364847931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f529c0c8d10 msgr2=0x7f529c15bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:51.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.493+0000 7f529a7fc700 1 --2- 172.21.15.40:0/364847931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f529c0c8d10 0x7f529c15bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:51.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.493+0000 7f529a7fc700 1 -- 172.21.15.40:0/364847931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f529c15a480 msgr2=0x7f529c155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:51.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.493+0000 7f529a7fc700 1 --2- 172.21.15.40:0/364847931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f529c15a480 0x7f529c155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:51.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.493+0000 7f529a7fc700 1 -- 172.21.15.40:0/364847931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c0094b0 con 0x7f529c0ccd10 2023-11-08T08:23:51.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.493+0000 7f529a7fc700 1 --2- 172.21.15.40:0/364847931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529c0ccd10 0x7f529c153a80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f529000e840 tx=0x7f529000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:51.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.493+0000 7f52837fe700 1 -- 172.21.15.40:0/364847931 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52900095a0 con 0x7f529c0ccd10 2023-11-08T08:23:51.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.493+0000 7f52837fe700 1 -- 172.21.15.40:0/364847931 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5290013680 con 0x7f529c0ccd10 2023-11-08T08:23:51.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.493+0000 7f52837fe700 1 -- 172.21.15.40:0/364847931 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52900139f0 con 0x7f529c0ccd10 2023-11-08T08:23:51.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.493+0000 7f52a1301700 1 -- 172.21.15.40:0/364847931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529c0add50 con 0x7f529c0ccd10 2023-11-08T08:23:51.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.493+0000 7f52a1301700 1 -- 172.21.15.40:0/364847931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529c0caa30 con 0x7f529c0ccd10 2023-11-08T08:23:51.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.497+0000 7f52a1301700 1 -- 172.21.15.40:0/364847931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529c0ca6e0 con 0x7f529c0ccd10 2023-11-08T08:23:51.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.497+0000 7f52837fe700 1 -- 172.21.15.40:0/364847931 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f5290019030 con 0x7f529c0ccd10 2023-11-08T08:23:51.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.497+0000 7f52837fe700 1 --2- 172.21.15.40:0/364847931 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5288064ee0 0x7f5288067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:51.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.501+0000 7f52837fe700 1 -- 172.21.15.40:0/364847931 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f5290081e80 con 0x7f529c0ccd10 2023-11-08T08:23:51.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.501+0000 7f529affd700 1 --2- 172.21.15.40:0/364847931 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5288064ee0 0x7f5288067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:51.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.501+0000 7f529affd700 1 --2- 172.21.15.40:0/364847931 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5288064ee0 0x7f5288067390 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f528c0097a0 tx=0x7f528c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:51.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.505+0000 7f52837fe700 1 -- 172.21.15.40:0/364847931 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f529004fd50 con 0x7f529c0ccd10 2023-11-08T08:23:51.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.713+0000 7f52a1301700 1 -- 172.21.15.40:0/364847931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f529c0726c0 con 0x7f529c0ccd10 2023-11-08T08:23:51.960 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-11-08T08:23:51.953+0000 7f513abf9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T08:23:51.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.969+0000 7f52837fe700 1 -- 172.21.15.40:0/364847931 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5290053370 con 0x7f529c0ccd10 2023-11-08T08:23:51.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.973+0000 7f52a1301700 1 -- 172.21.15.40:0/364847931 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5288064ee0 msgr2=0x7f5288067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:51.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.973+0000 7f52a1301700 1 --2- 172.21.15.40:0/364847931 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5288064ee0 0x7f5288067390 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f528c0097a0 tx=0x7f528c01c040 comp rx=0 tx=0).stop 2023-11-08T08:23:51.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.973+0000 7f52a1301700 1 -- 172.21.15.40:0/364847931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529c0ccd10 msgr2=0x7f529c153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:51.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.973+0000 7f52a1301700 1 --2- 172.21.15.40:0/364847931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529c0ccd10 0x7f529c153a80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f529000e840 tx=0x7f529000eb50 comp rx=0 tx=0).stop 2023-11-08T08:23:51.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.973+0000 7f52a1301700 1 -- 172.21.15.40:0/364847931 shutdown_connections 2023-11-08T08:23:51.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.973+0000 7f52a1301700 1 --2- 172.21.15.40:0/364847931 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f5288064ee0 0x7f5288067390 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:51.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.973+0000 7f52a1301700 1 --2- 172.21.15.40:0/364847931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f529c15a480 0x7f529c155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:51.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.973+0000 7f52a1301700 1 --2- 172.21.15.40:0/364847931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529c0ccd10 0x7f529c153a80 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:51.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.973+0000 7f52a1301700 1 --2- 172.21.15.40:0/364847931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f529c0c8d10 0x7f529c15bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:51.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.973+0000 7f52a1301700 1 -- 172.21.15.40:0/364847931 >> 172.21.15.40:0/364847931 conn(0x7f529c0cf350 msgr2=0x7f529c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:51.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.973+0000 7f52a1301700 1 -- 172.21.15.40:0/364847931 shutdown_connections 2023-11-08T08:23:51.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:51.973+0000 7f52a1301700 1 -- 172.21.15.40:0/364847931 wait complete. 2023-11-08T08:23:51.983 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T08:23:51.990 INFO:tasks.ceph.mds.c.smithi040.stderr: -7252> 2023-11-08T08:22:25.139+0000 7f32d2f42700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T08:23:51.994 INFO:tasks.ceph.mds.c.smithi040.stderr: -4790> 2023-11-08T08:22:30.383+0000 7f32cf73b700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6286=0-4194304@1} caps={6286=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6286 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55cb2535db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(11)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T08:23:52.006 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T08:23:52.054 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-11-08T08:23:52.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.205+0000 7ff7cfcee700 1 Processor -- start 2023-11-08T08:23:52.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.205+0000 7ff7cfcee700 1 -- start start 2023-11-08T08:23:52.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.209+0000 7ff7cfcee700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff7c8163ea0 0x7ff7c80c1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:52.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.209+0000 7ff7cfcee700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff7c80c9d00 0x7ff7c8069fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:52.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.209+0000 7ff7cfcee700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7c806d2c0 con 0x7ff7c80add20 2023-11-08T08:23:52.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.209+0000 7ff7cfcee700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c80ca420 con 0x7ff7c8163ea0 2023-11-08T08:23:52.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.209+0000 7ff7cfcee700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c8164270 con 0x7ff7c80c9d00 2023-11-08T08:23:52.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.209+0000 7ff7cd289700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff7c80add20 0x7ff7c80c95f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50254/0 (socket says 172.21.15.40:50254) 2023-11-08T08:23:52.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.209+0000 7ff7cd289700 1 -- 172.21.15.40:0/3017907120 learned_addr learned my addr 172.21.15.40:0/3017907120 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:52.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.209+0000 7ff7cda8a700 1 --2- 172.21.15.40:0/3017907120 >> v2:172.21.15.40:3300/0 conn(0x7ff7c8163ea0 0x7ff7c80c1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:52.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.209+0000 7ff7ce28b700 1 --2- 172.21.15.40:0/3017907120 >> v2:172.21.15.169:3300/0 conn(0x7ff7c80c9d00 0x7ff7c8069fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:52.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.209+0000 7ff7cda8a700 1 -- 172.21.15.40:0/3017907120 >> v2:172.21.15.169:3300/0 conn(0x7ff7c80c9d00 msgr2=0x7ff7c8069fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:52.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.209+0000 7ff7cda8a700 1 --2- 172.21.15.40:0/3017907120 >> v2:172.21.15.169:3300/0 conn(0x7ff7c80c9d00 0x7ff7c8069fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:52.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.209+0000 7ff7cda8a700 1 -- 172.21.15.40:0/3017907120 >> v1:172.21.15.40:6789/0 conn(0x7ff7c80add20 legacy=0x7ff7c80c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:52.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.209+0000 7ff7cca88700 1 -- 172.21.15.40:0/3017907120 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343080109 0 0) 0x7ff7c806d2c0 con 0x7ff7c80add20 2023-11-08T08:23:52.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.209+0000 7ff7cda8a700 1 -- 172.21.15.40:0/3017907120 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c80ae1b0 con 0x7ff7c8163ea0 2023-11-08T08:23:52.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.209+0000 7ff7cda8a700 1 --2- 172.21.15.40:0/3017907120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c8163ea0 0x7ff7c80c1420 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80ce6c0 tx=0x7ff7c4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c47e5fdd77bd0dc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:52.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.209+0000 7ff7cca88700 1 -- 172.21.15.40:0/3017907120 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4011830 con 0x7ff7c8163ea0 2023-11-08T08:23:52.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.209+0000 7ff7cca88700 1 -- 172.21.15.40:0/3017907120 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c4011990 con 0x7ff7c8163ea0 2023-11-08T08:23:52.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.209+0000 7ff7cca88700 1 -- 172.21.15.40:0/3017907120 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4011d20 con 0x7ff7c8163ea0 2023-11-08T08:23:52.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.213+0000 7ff7cfcee700 1 -- 172.21.15.40:0/3017907120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c8163ea0 msgr2=0x7ff7c80c1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:52.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.213+0000 7ff7cfcee700 1 --2- 172.21.15.40:0/3017907120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c8163ea0 0x7ff7c80c1420 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80ce6c0 tx=0x7ff7c4009b30 comp rx=0 tx=0).stop 2023-11-08T08:23:52.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.213+0000 7ff7cfcee700 1 -- 172.21.15.40:0/3017907120 shutdown_connections 2023-11-08T08:23:52.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.213+0000 7ff7cfcee700 1 --2- 172.21.15.40:0/3017907120 >> v2:172.21.15.169:3300/0 conn(0x7ff7c80c9d00 0x7ff7c8069fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:52.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.213+0000 7ff7cfcee700 1 --2- 172.21.15.40:0/3017907120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c8163ea0 0x7ff7c80c1420 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:52.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.213+0000 7ff7cfcee700 1 -- 172.21.15.40:0/3017907120 >> 172.21.15.40:0/3017907120 conn(0x7ff7c80cf3b0 msgr2=0x7ff7c80cb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:52.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.213+0000 7ff7cfcee700 1 -- 172.21.15.40:0/3017907120 shutdown_connections 2023-11-08T08:23:52.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.213+0000 7ff7cfcee700 1 -- 172.21.15.40:0/3017907120 wait complete. 2023-11-08T08:23:52.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.213+0000 7ff7cfcee700 1 Processor -- start 2023-11-08T08:23:52.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.213+0000 7ff7cfcee700 1 -- start start 2023-11-08T08:23:52.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7cfcee700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7c80add20 0x7ff7c80c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:52.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7cfcee700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7c80c9d00 0x7ff7c80c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:52.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7cfcee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c8163ea0 0x7ff7c81621d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:52.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7cfcee700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c80c2110 con 0x7ff7c8163ea0 2023-11-08T08:23:52.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7cfcee700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c80c6140 con 0x7ff7c80c9d00 2023-11-08T08:23:52.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7cfcee700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c80c6420 con 0x7ff7c80add20 2023-11-08T08:23:52.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7ce28b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c8163ea0 0x7ff7c81621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:52.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7ce28b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c8163ea0 0x7ff7c81621d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41232/0 (socket says 172.21.15.40:41232) 2023-11-08T08:23:52.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7ce28b700 1 -- 172.21.15.40:0/1353289525 learned_addr learned my addr 172.21.15.40:0/1353289525 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:52.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7cd289700 1 --2- 172.21.15.40:0/1353289525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7c80c9d00 0x7ff7c80c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:52.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7cda8a700 1 --2- 172.21.15.40:0/1353289525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7c80add20 0x7ff7c80c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:52.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7ce28b700 1 -- 172.21.15.40:0/1353289525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7c80add20 msgr2=0x7ff7c80c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:52.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7ce28b700 1 --2- 172.21.15.40:0/1353289525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7c80add20 0x7ff7c80c57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:52.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7ce28b700 1 -- 172.21.15.40:0/1353289525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7c80c9d00 msgr2=0x7ff7c80c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:52.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7ce28b700 1 --2- 172.21.15.40:0/1353289525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7c80c9d00 0x7ff7c80c7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:52.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7ce28b700 1 -- 172.21.15.40:0/1353289525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c40094b0 con 0x7ff7c8163ea0 2023-11-08T08:23:52.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7cda8a700 1 --2- 172.21.15.40:0/1353289525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7c80add20 0x7ff7c80c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:23:52.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7ce28b700 1 --2- 172.21.15.40:0/1353289525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c8163ea0 0x7ff7c81621d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000efc0 tx=0x7ff7c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:52.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7be7fc700 1 -- 172.21.15.40:0/1353289525 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c000cd40 con 0x7ff7c8163ea0 2023-11-08T08:23:52.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7be7fc700 1 -- 172.21.15.40:0/1353289525 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c000f040 con 0x7ff7c8163ea0 2023-11-08T08:23:52.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7be7fc700 1 -- 172.21.15.40:0/1353289525 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c00146c0 con 0x7ff7c8163ea0 2023-11-08T08:23:52.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7cfcee700 1 -- 172.21.15.40:0/1353289525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c806cd00 con 0x7ff7c8163ea0 2023-11-08T08:23:52.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.217+0000 7ff7cfcee700 1 -- 172.21.15.40:0/1353289525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c806ca20 con 0x7ff7c8163ea0 2023-11-08T08:23:52.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.221+0000 7ff7cfcee700 1 -- 172.21.15.40:0/1353289525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c806cfe0 con 0x7ff7c8163ea0 2023-11-08T08:23:52.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.221+0000 7ff7be7fc700 1 -- 172.21.15.40:0/1353289525 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7ff7c0014820 con 0x7ff7c8163ea0 2023-11-08T08:23:52.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.221+0000 7ff7be7fc700 1 --2- 172.21.15.40:0/1353289525 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff7ac064ee0 0x7ff7ac067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:52.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.221+0000 7ff7be7fc700 1 -- 172.21.15.40:0/1353289525 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6526+0+0 (secure 0 0 0) 0x7ff7c0082c70 con 0x7ff7c8163ea0 2023-11-08T08:23:52.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.225+0000 7ff7cda8a700 1 --2- 172.21.15.40:0/1353289525 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff7ac064ee0 0x7ff7ac067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:52.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.225+0000 7ff7cda8a700 1 --2- 172.21.15.40:0/1353289525 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff7ac064ee0 0x7ff7ac067390 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009b70 tx=0x7ff7c4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:52.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.229+0000 7ff7be7fc700 1 -- 172.21.15.40:0/1353289525 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7ff7c0050ac0 con 0x7ff7c8163ea0 2023-11-08T08:23:52.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.437+0000 7ff7cfcee700 1 -- 172.21.15.40:0/1353289525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff7c80726c0 con 0x7ff7c8163ea0 2023-11-08T08:23:52.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.981+0000 7ff7be7fc700 1 -- 172.21.15.40:0/1353289525 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff7c000cea0 con 0x7ff7c8163ea0 2023-11-08T08:23:52.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.985+0000 7ff7cfcee700 1 -- 172.21.15.40:0/1353289525 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff7ac064ee0 msgr2=0x7ff7ac067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:52.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.985+0000 7ff7cfcee700 1 --2- 172.21.15.40:0/1353289525 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff7ac064ee0 0x7ff7ac067390 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009b70 tx=0x7ff7c4008030 comp rx=0 tx=0).stop 2023-11-08T08:23:52.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.985+0000 7ff7cfcee700 1 -- 172.21.15.40:0/1353289525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c8163ea0 msgr2=0x7ff7c81621d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:52.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.985+0000 7ff7cfcee700 1 --2- 172.21.15.40:0/1353289525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c8163ea0 0x7ff7c81621d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000efc0 tx=0x7ff7c000c3a0 comp rx=0 tx=0).stop 2023-11-08T08:23:52.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.985+0000 7ff7cfcee700 1 -- 172.21.15.40:0/1353289525 shutdown_connections 2023-11-08T08:23:52.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.985+0000 7ff7cfcee700 1 --2- 172.21.15.40:0/1353289525 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7ff7ac064ee0 0x7ff7ac067390 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:52.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.985+0000 7ff7cfcee700 1 --2- 172.21.15.40:0/1353289525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7c8163ea0 0x7ff7c81621d0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:52.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.985+0000 7ff7cfcee700 1 --2- 172.21.15.40:0/1353289525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7c80c9d00 0x7ff7c80c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:52.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.985+0000 7ff7cfcee700 1 --2- 172.21.15.40:0/1353289525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7c80add20 0x7ff7c80c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:52.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.985+0000 7ff7cfcee700 1 -- 172.21.15.40:0/1353289525 >> 172.21.15.40:0/1353289525 conn(0x7ff7c80cf3b0 msgr2=0x7ff7c8070060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:52.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.985+0000 7ff7cfcee700 1 -- 172.21.15.40:0/1353289525 shutdown_connections 2023-11-08T08:23:52.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:52.985+0000 7ff7cfcee700 1 -- 172.21.15.40:0/1353289525 wait complete. 2023-11-08T08:23:53.016 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T08:23:53.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.201+0000 7f9441803700 1 Processor -- start 2023-11-08T08:23:53.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.201+0000 7f9441803700 1 -- start start 2023-11-08T08:23:53.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.205+0000 7f9441803700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f943c15c6e0 0x7f943c071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:53.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.205+0000 7f9441803700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f943c06a510 0x7f943c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.205+0000 7f9441803700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943c15cab0 con 0x7f943c071fe0 2023-11-08T08:23:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.205+0000 7f9441803700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f943c0d2800 con 0x7f943c15c6e0 2023-11-08T08:23:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.205+0000 7f9441803700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f943c0702d0 con 0x7f943c06a510 2023-11-08T08:23:53.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.205+0000 7f943affd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f943c15c6e0 0x7f943c071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:53.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.205+0000 7f943a7fc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f943c071fe0 0x7f943c069e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50262/0 (socket says 172.21.15.40:50262) 2023-11-08T08:23:53.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.205+0000 7f943a7fc700 1 -- 172.21.15.40:0/3971621327 learned_addr learned my addr 172.21.15.40:0/3971621327 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:53.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.205+0000 7f943b7fe700 1 --2- 172.21.15.40:0/3971621327 >> v2:172.21.15.169:3300/0 conn(0x7f943c06a510 0x7f943c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:53.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.205+0000 7f943b7fe700 1 -- 172.21.15.40:0/3971621327 >> v2:172.21.15.40:3300/0 conn(0x7f943c15c6e0 msgr2=0x7f943c071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:53.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.205+0000 7f943b7fe700 1 --2- 172.21.15.40:0/3971621327 >> v2:172.21.15.40:3300/0 conn(0x7f943c15c6e0 0x7f943c071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:53.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.205+0000 7f943b7fe700 1 -- 172.21.15.40:0/3971621327 >> v1:172.21.15.40:6789/0 conn(0x7f943c071fe0 legacy=0x7f943c069e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:53.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.205+0000 7f9439ffb700 1 -- 172.21.15.40:0/3971621327 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3615490028 0 0) 0x7f943c15cab0 con 0x7f943c071fe0 2023-11-08T08:23:53.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.205+0000 7f943b7fe700 1 -- 172.21.15.40:0/3971621327 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c0d2a10 con 0x7f943c06a510 2023-11-08T08:23:53.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.205+0000 7f943b7fe700 1 --2- 172.21.15.40:0/3971621327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f943c06a510 0x7f943c06c630 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f943c06e190 tx=0x7f943400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdf7ef73493621f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:53.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.209+0000 7f9439ffb700 1 -- 172.21.15.40:0/3971621327 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943400e070 con 0x7f943c06a510 2023-11-08T08:23:53.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.209+0000 7f9439ffb700 1 -- 172.21.15.40:0/3971621327 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943400bb20 con 0x7f943c06a510 2023-11-08T08:23:53.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.209+0000 7f9439ffb700 1 -- 172.21.15.40:0/3971621327 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434012430 con 0x7f943c06a510 2023-11-08T08:23:53.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.209+0000 7f9441803700 1 -- 172.21.15.40:0/3971621327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f943c06a510 msgr2=0x7f943c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:53.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.209+0000 7f9441803700 1 --2- 172.21.15.40:0/3971621327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f943c06a510 0x7f943c06c630 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f943c06e190 tx=0x7f943400b330 comp rx=0 tx=0).stop 2023-11-08T08:23:53.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.209+0000 7f9441803700 1 -- 172.21.15.40:0/3971621327 shutdown_connections 2023-11-08T08:23:53.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.209+0000 7f9441803700 1 --2- 172.21.15.40:0/3971621327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f943c06a510 0x7f943c06c630 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:53.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.209+0000 7f9441803700 1 --2- 172.21.15.40:0/3971621327 >> v2:172.21.15.40:3300/0 conn(0x7f943c15c6e0 0x7f943c071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:53.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.209+0000 7f9441803700 1 -- 172.21.15.40:0/3971621327 >> 172.21.15.40:0/3971621327 conn(0x7f943c0b65b0 msgr2=0x7f943c071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:53.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.209+0000 7f9441803700 1 -- 172.21.15.40:0/3971621327 shutdown_connections 2023-11-08T08:23:53.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.209+0000 7f9441803700 1 -- 172.21.15.40:0/3971621327 wait complete. 2023-11-08T08:23:53.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.213+0000 7f9441803700 1 Processor -- start 2023-11-08T08:23:53.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.213+0000 7f9441803700 1 -- start start 2023-11-08T08:23:53.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.213+0000 7f9441803700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f943c06a510 0x7f943c0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:53.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.213+0000 7f9441803700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f943c071fe0 0x7f943c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:53.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.213+0000 7f9441803700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f943c15c6e0 0x7f943c0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:53.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.213+0000 7f9441803700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c0b0fa0 con 0x7f943c071fe0 2023-11-08T08:23:53.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.213+0000 7f9441803700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c0ce4a0 con 0x7f943c15c6e0 2023-11-08T08:23:53.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.213+0000 7f9441803700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943c0b5370 con 0x7f943c06a510 2023-11-08T08:23:53.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.217+0000 7f943a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f943c071fe0 0x7f943c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:53.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.217+0000 7f943a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f943c071fe0 0x7f943c0b4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41258/0 (socket says 172.21.15.40:41258) 2023-11-08T08:23:53.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.217+0000 7f943a7fc700 1 -- 172.21.15.40:0/290475444 learned_addr learned my addr 172.21.15.40:0/290475444 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:53.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.217+0000 7f943b7fe700 1 --2- 172.21.15.40:0/290475444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f943c15c6e0 0x7f943c0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:53.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.217+0000 7f943affd700 1 --2- 172.21.15.40:0/290475444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f943c06a510 0x7f943c0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:53.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.217+0000 7f943a7fc700 1 -- 172.21.15.40:0/290475444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f943c06a510 msgr2=0x7f943c0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:53.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.217+0000 7f943a7fc700 1 --2- 172.21.15.40:0/290475444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f943c06a510 0x7f943c0b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:53.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.217+0000 7f943a7fc700 1 -- 172.21.15.40:0/290475444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f943c15c6e0 msgr2=0x7f943c0cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:53.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.217+0000 7f943a7fc700 1 --2- 172.21.15.40:0/290475444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f943c15c6e0 0x7f943c0cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:53.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.217+0000 7f943a7fc700 1 -- 172.21.15.40:0/290475444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9434009a70 con 0x7f943c071fe0 2023-11-08T08:23:53.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.217+0000 7f943b7fe700 1 --2- 172.21.15.40:0/290475444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f943c15c6e0 0x7f943c0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:23:53.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.217+0000 7f943affd700 1 --2- 172.21.15.40:0/290475444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f943c06a510 0x7f943c0b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:23:53.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.217+0000 7f943a7fc700 1 --2- 172.21.15.40:0/290475444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f943c071fe0 0x7f943c0b4e30 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f943000c580 tx=0x7f943000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:53.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.217+0000 7f9423fff700 1 -- 172.21.15.40:0/290475444 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430004ca0 con 0x7f943c071fe0 2023-11-08T08:23:53.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.217+0000 7f9441803700 1 -- 172.21.15.40:0/290475444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943c0d2fd0 con 0x7f943c071fe0 2023-11-08T08:23:53.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.217+0000 7f9423fff700 1 -- 172.21.15.40:0/290475444 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9430004e20 con 0x7f943c071fe0 2023-11-08T08:23:53.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.217+0000 7f9423fff700 1 -- 172.21.15.40:0/290475444 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430007310 con 0x7f943c071fe0 2023-11-08T08:23:53.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.217+0000 7f9441803700 1 -- 172.21.15.40:0/290475444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943c0d2cf0 con 0x7f943c071fe0 2023-11-08T08:23:53.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.217+0000 7f9441803700 1 -- 172.21.15.40:0/290475444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943c0d32b0 con 0x7f943c071fe0 2023-11-08T08:23:53.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.221+0000 7f9423fff700 1 -- 172.21.15.40:0/290475444 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f943001a020 con 0x7f943c071fe0 2023-11-08T08:23:53.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.221+0000 7f9423fff700 1 --2- 172.21.15.40:0/290475444 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9428064f10 0x7f94280673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:53.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.221+0000 7f9423fff700 1 -- 172.21.15.40:0/290475444 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6526+0+0 (secure 0 0 0) 0x7f9430081d60 con 0x7f943c071fe0 2023-11-08T08:23:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.221+0000 7f943affd700 1 --2- 172.21.15.40:0/290475444 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9428064f10 0x7f94280673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.225+0000 7f943affd700 1 --2- 172.21.15.40:0/290475444 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9428064f10 0x7f94280673c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f942c009c40 tx=0x7f942c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:53.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.229+0000 7f9423fff700 1 -- 172.21.15.40:0/290475444 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f943004fcb0 con 0x7f943c071fe0 2023-11-08T08:23:53.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.405+0000 7f9441803700 1 -- 172.21.15.40:0/290475444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f943c0ce5e0 con 0x7f943c071fe0 2023-11-08T08:23:53.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.989+0000 7f9423fff700 1 -- 172.21.15.40:0/290475444 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) v1 ==== 181+0+0 (secure 0 0 0) 0x7f943c0ce5e0 con 0x7f943c071fe0 2023-11-08T08:23:54.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.993+0000 7f9441803700 1 -- 172.21.15.40:0/290475444 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9428064f10 msgr2=0x7f94280673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:54.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.993+0000 7f9441803700 1 --2- 172.21.15.40:0/290475444 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9428064f10 0x7f94280673c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f942c009c40 tx=0x7f942c008040 comp rx=0 tx=0).stop 2023-11-08T08:23:54.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.993+0000 7f9441803700 1 -- 172.21.15.40:0/290475444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f943c071fe0 msgr2=0x7f943c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:54.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.993+0000 7f9441803700 1 --2- 172.21.15.40:0/290475444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f943c071fe0 0x7f943c0b4e30 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f943000c580 tx=0x7f943000c940 comp rx=0 tx=0).stop 2023-11-08T08:23:54.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.993+0000 7f9441803700 1 -- 172.21.15.40:0/290475444 shutdown_connections 2023-11-08T08:23:54.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.993+0000 7f9441803700 1 --2- 172.21.15.40:0/290475444 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f9428064f10 0x7f94280673c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:54.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.993+0000 7f9441803700 1 --2- 172.21.15.40:0/290475444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f943c15c6e0 0x7f943c0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:54.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.993+0000 7f9441803700 1 --2- 172.21.15.40:0/290475444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f943c071fe0 0x7f943c0b4e30 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:54.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.993+0000 7f9441803700 1 --2- 172.21.15.40:0/290475444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f943c06a510 0x7f943c0b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:54.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.993+0000 7f9441803700 1 -- 172.21.15.40:0/290475444 >> 172.21.15.40:0/290475444 conn(0x7f943c0b65b0 msgr2=0x7f943c06d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:54.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.993+0000 7f9441803700 1 -- 172.21.15.40:0/290475444 shutdown_connections 2023-11-08T08:23:54.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:53.993+0000 7f9441803700 1 -- 172.21.15.40:0/290475444 wait complete. 2023-11-08T08:23:54.003 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2023-11-08T08:23:54.022 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T08:23:54.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.201+0000 7f1f1116a700 1 Processor -- start 2023-11-08T08:23:54.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.201+0000 7f1f1116a700 1 -- start start 2023-11-08T08:23:54.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f1116a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1f0c142650 0x7f1f0c142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:54.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f1116a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f0c06b1c0 0x7f1f0c0b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:54.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f1116a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0c0b63a0 con 0x7f1f0c0b5a00 2023-11-08T08:23:54.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f1116a700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0c06b8d0 con 0x7f1f0c142650 2023-11-08T08:23:54.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f1116a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c06bb70 con 0x7f1f0c06b1c0 2023-11-08T08:23:54.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f03fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1f0c0b5a00 0x7f1f0c06aab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50266/0 (socket says 172.21.15.40:50266) 2023-11-08T08:23:54.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f03fff700 1 -- 172.21.15.40:0/1760618864 learned_addr learned my addr 172.21.15.40:0/1760618864 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:54.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f0ad9d700 1 --2- 172.21.15.40:0/1760618864 >> v2:172.21.15.169:3300/0 conn(0x7f1f0c142650 0x7f1f0c142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:54.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f0b59e700 1 --2- 172.21.15.40:0/1760618864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f0c06b1c0 0x7f1f0c0b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:54.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f0a59c700 1 -- 172.21.15.40:0/1760618864 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825848324 0 0) 0x7f1f0c0b63a0 con 0x7f1f0c0b5a00 2023-11-08T08:23:54.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f0a59c700 1 -- 172.21.15.40:0/1760618864 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eec0035d0 con 0x7f1f0c0b5a00 2023-11-08T08:23:54.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f0a59c700 1 -- 172.21.15.40:0/1760618864 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2772452630 0 0) 0x7f1eec0035d0 con 0x7f1f0c0b5a00 2023-11-08T08:23:54.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f0a59c700 1 -- 172.21.15.40:0/1760618864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f0c06b1c0 msgr2=0x7f1f0c0b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:54.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f0a59c700 1 --2- 172.21.15.40:0/1760618864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f0c06b1c0 0x7f1f0c0b5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:54.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f0a59c700 1 -- 172.21.15.40:0/1760618864 >> v2:172.21.15.169:3300/0 conn(0x7f1f0c142650 msgr2=0x7f1f0c142fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:54.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f0a59c700 1 --2- 172.21.15.40:0/1760618864 >> v2:172.21.15.169:3300/0 conn(0x7f1f0c142650 0x7f1f0c142fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:54.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f0a59c700 1 -- 172.21.15.40:0/1760618864 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c06be50 con 0x7f1f0c0b5a00 2023-11-08T08:23:54.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f0a59c700 1 -- 172.21.15.40:0/1760618864 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f1ef4001690 con 0x7f1f0c0b5a00 2023-11-08T08:23:54.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f0ad9d700 1 --2- 172.21.15.40:0/1760618864 >> v2:172.21.15.169:3300/0 conn(0x7f1f0c142650 0x7f1f0c142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:23:54.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f0a59c700 1 -- 172.21.15.40:0/1760618864 >> v1:172.21.15.40:6789/0 conn(0x7f1f0c0b5a00 legacy=0x7f1f0c06aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:54.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f0a59c700 1 --2- 172.21.15.40:0/1760618864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1eec003a20 0x7f1eec003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:54.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.205+0000 7f1f0a59c700 1 --2- 172.21.15.40:0/1760618864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1eec004450 0x7f1eec004910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:54.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f0a59c700 1 --2- 172.21.15.40:0/1760618864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f0c06b1c0 0x7f1eec004fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:54.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f0a59c700 1 -- 172.21.15.40:0/1760618864 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c06b8d0 con 0x7f1eec003a20 2023-11-08T08:23:54.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f0a59c700 1 -- 172.21.15.40:0/1760618864 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c06bb70 con 0x7f1eec004450 2023-11-08T08:23:54.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f0a59c700 1 -- 172.21.15.40:0/1760618864 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec00ba30 con 0x7f1f0c06b1c0 2023-11-08T08:23:54.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f0ad9d700 1 --2- 172.21.15.40:0/1760618864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1eec003a20 0x7f1eec003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:54.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f03fff700 1 --2- 172.21.15.40:0/1760618864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1eec004450 0x7f1eec004910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:54.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f0b59e700 1 --2- 172.21.15.40:0/1760618864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f0c06b1c0 0x7f1eec004fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:54.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f0ad9d700 1 -- 172.21.15.40:0/1760618864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f0c06b1c0 msgr2=0x7f1eec004fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:54.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f0ad9d700 1 --2- 172.21.15.40:0/1760618864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f0c06b1c0 0x7f1eec004fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:54.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f0ad9d700 1 -- 172.21.15.40:0/1760618864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1eec004450 msgr2=0x7f1eec004910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:54.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f0ad9d700 1 --2- 172.21.15.40:0/1760618864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1eec004450 0x7f1eec004910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:54.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f0ad9d700 1 -- 172.21.15.40:0/1760618864 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec0035d0 con 0x7f1eec003a20 2023-11-08T08:23:54.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f0ad9d700 1 --2- 172.21.15.40:0/1760618864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1eec003a20 0x7f1eec003e30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009d40 tx=0x7f1eec0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12190217074cb3dd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:54.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f0a59c700 1 -- 172.21.15.40:0/1760618864 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc009e90 con 0x7f1eec003a20 2023-11-08T08:23:54.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f0a59c700 1 -- 172.21.15.40:0/1760618864 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc00e9a0 con 0x7f1eec003a20 2023-11-08T08:23:54.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f0a59c700 1 -- 172.21.15.40:0/1760618864 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc00ed10 con 0x7f1eec003a20 2023-11-08T08:23:54.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f1116a700 1 -- 172.21.15.40:0/1760618864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1eec003a20 msgr2=0x7f1eec003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:54.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f1116a700 1 --2- 172.21.15.40:0/1760618864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1eec003a20 0x7f1eec003e30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009d40 tx=0x7f1eec0058a0 comp rx=0 tx=0).stop 2023-11-08T08:23:54.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f0b59e700 1 -- 172.21.15.40:0/1760618864 reap_dead start 2023-11-08T08:23:54.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f1116a700 1 -- 172.21.15.40:0/1760618864 shutdown_connections 2023-11-08T08:23:54.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f1116a700 1 -- 172.21.15.40:0/1760618864 >> 172.21.15.40:0/1760618864 conn(0x7f1f0c0b5390 msgr2=0x7f1f0c0d2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:54.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f1116a700 1 -- 172.21.15.40:0/1760618864 shutdown_connections 2023-11-08T08:23:54.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.209+0000 7f1f1116a700 1 -- 172.21.15.40:0/1760618864 wait complete. 2023-11-08T08:23:54.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.213+0000 7f1f1116a700 1 Processor -- start 2023-11-08T08:23:54.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.213+0000 7f1f1116a700 1 -- start start 2023-11-08T08:23:54.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.213+0000 7f1f1116a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1eec003a20 0x7f1f0c1411a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:54.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.213+0000 7f1f1116a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f0c1535c0 0x7f1f0c153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:54.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.213+0000 7f1f1116a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f0c069f70 0x7f1f0c06a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:54.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.213+0000 7f1f1116a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c157760 con 0x7f1eec003a20 2023-11-08T08:23:54.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.213+0000 7f1f1116a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c06a920 con 0x7f1f0c1535c0 2023-11-08T08:23:54.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.213+0000 7f1f1116a700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c1416e0 con 0x7f1f0c069f70 2023-11-08T08:23:54.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.213+0000 7f1f0ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1eec003a20 0x7f1f0c1411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:54.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.217+0000 7f1f0ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1eec003a20 0x7f1f0c1411a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41274/0 (socket says 172.21.15.40:41274) 2023-11-08T08:23:54.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.217+0000 7f1f0ad9d700 1 -- 172.21.15.40:0/370449947 learned_addr learned my addr 172.21.15.40:0/370449947 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:54.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.217+0000 7f1f03fff700 1 --2- 172.21.15.40:0/370449947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f0c1535c0 0x7f1f0c153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:54.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.217+0000 7f1f0b59e700 1 --2- 172.21.15.40:0/370449947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f0c069f70 0x7f1f0c06a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:54.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.217+0000 7f1f0ad9d700 1 -- 172.21.15.40:0/370449947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f0c069f70 msgr2=0x7f1f0c06a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:54.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.217+0000 7f1f0ad9d700 1 --2- 172.21.15.40:0/370449947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f0c069f70 0x7f1f0c06a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:54.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.217+0000 7f1f0ad9d700 1 -- 172.21.15.40:0/370449947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f0c1535c0 msgr2=0x7f1f0c153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:54.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.217+0000 7f1f0ad9d700 1 --2- 172.21.15.40:0/370449947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f0c1535c0 0x7f1f0c153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:54.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.217+0000 7f1f0ad9d700 1 -- 172.21.15.40:0/370449947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c06c410 con 0x7f1eec003a20 2023-11-08T08:23:54.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.217+0000 7f1f0ad9d700 1 --2- 172.21.15.40:0/370449947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1eec003a20 0x7f1f0c1411a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0056f0 tx=0x7f1efc0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:54.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.217+0000 7f1f037fe700 1 -- 172.21.15.40:0/370449947 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc003ad0 con 0x7f1eec003a20 2023-11-08T08:23:54.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.217+0000 7f1f037fe700 1 -- 172.21.15.40:0/370449947 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc003c30 con 0x7f1eec003a20 2023-11-08T08:23:54.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.217+0000 7f1f1116a700 1 -- 172.21.15.40:0/370449947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0c06c130 con 0x7f1eec003a20 2023-11-08T08:23:54.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.217+0000 7f1f037fe700 1 -- 172.21.15.40:0/370449947 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc009e90 con 0x7f1eec003a20 2023-11-08T08:23:54.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.217+0000 7f1f1116a700 1 -- 172.21.15.40:0/370449947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0c142a20 con 0x7f1eec003a20 2023-11-08T08:23:54.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.217+0000 7f1f1116a700 1 -- 172.21.15.40:0/370449947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0c0726c0 con 0x7f1eec003a20 2023-11-08T08:23:54.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.221+0000 7f1f037fe700 1 -- 172.21.15.40:0/370449947 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f1efc02c030 con 0x7f1eec003a20 2023-11-08T08:23:54.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.221+0000 7f1f037fe700 1 --2- 172.21.15.40:0/370449947 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1ef0064ee0 0x7f1ef0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:54.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.221+0000 7f1f037fe700 1 -- 172.21.15.40:0/370449947 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6088+0+0 (secure 0 0 0) 0x7f1efc083490 con 0x7f1eec003a20 2023-11-08T08:23:54.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.221+0000 7f1f03fff700 1 --2- 172.21.15.40:0/370449947 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1ef0064ee0 0x7f1ef0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:54.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.225+0000 7f1f03fff700 1 --2- 172.21.15.40:0/370449947 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1ef0064ee0 0x7f1ef0067390 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4000c00 tx=0x7f1ef40079a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.229+0000 7f1f037fe700 1 -- 172.21.15.40:0/370449947 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f1efc051520 con 0x7f1eec003a20 2023-11-08T08:23:54.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.405+0000 7f1f1116a700 1 -- 172.21.15.40:0/370449947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1f0c0d3920 con 0x7f1eec003a20 2023-11-08T08:23:55.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:54.997+0000 7f1f037fe700 1 -- 172.21.15.40:0/370449947 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1efc054b40 con 0x7f1eec003a20 2023-11-08T08:23:55.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.001+0000 7f1f1116a700 1 -- 172.21.15.40:0/370449947 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1ef0064ee0 msgr2=0x7f1ef0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:55.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.001+0000 7f1f1116a700 1 --2- 172.21.15.40:0/370449947 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1ef0064ee0 0x7f1ef0067390 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4000c00 tx=0x7f1ef40079a0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.005+0000 7f1f1116a700 1 -- 172.21.15.40:0/370449947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1eec003a20 msgr2=0x7f1f0c1411a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:55.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.005+0000 7f1f1116a700 1 --2- 172.21.15.40:0/370449947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1eec003a20 0x7f1f0c1411a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0056f0 tx=0x7f1efc0047e0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.005+0000 7f1f1116a700 1 -- 172.21.15.40:0/370449947 shutdown_connections 2023-11-08T08:23:55.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.005+0000 7f1f1116a700 1 --2- 172.21.15.40:0/370449947 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1ef0064ee0 0x7f1ef0067390 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.005+0000 7f1f1116a700 1 --2- 172.21.15.40:0/370449947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f0c069f70 0x7f1f0c06a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.005+0000 7f1f1116a700 1 --2- 172.21.15.40:0/370449947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f0c1535c0 0x7f1f0c153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.005+0000 7f1f1116a700 1 --2- 172.21.15.40:0/370449947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1eec003a20 0x7f1f0c1411a0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.005+0000 7f1f1116a700 1 -- 172.21.15.40:0/370449947 >> 172.21.15.40:0/370449947 conn(0x7f1f0c0b5390 msgr2=0x7f1f0c143db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:55.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.005+0000 7f1f1116a700 1 -- 172.21.15.40:0/370449947 shutdown_connections 2023-11-08T08:23:55.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.005+0000 7f1f1116a700 1 -- 172.21.15.40:0/370449947 wait complete. 2023-11-08T08:23:55.013 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2023-11-08T08:23:55.033 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-08T08:23:55.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.205+0000 7fd71bfff700 1 Processor -- start 2023-11-08T08:23:55.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.205+0000 7fd71bfff700 1 -- start start 2023-11-08T08:23:55.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.209+0000 7fd71bfff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd71c071aa0 0x7fd71c06d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:55.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.209+0000 7fd71bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd71c069e00 0x7fd71c06a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:55.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.209+0000 7fd71bfff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd71c06c600 con 0x7fd71c15c640 2023-11-08T08:23:55.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.209+0000 7fd71bfff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd71c06cef0 con 0x7fd71c069e00 2023-11-08T08:23:55.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.209+0000 7fd71bfff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd71c0d38f0 con 0x7fd71c071aa0 2023-11-08T08:23:55.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.209+0000 7fd71affd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd71c15c640 0x7fd71c0d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50276/0 (socket says 172.21.15.40:50276) 2023-11-08T08:23:55.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.209+0000 7fd71affd700 1 -- 172.21.15.40:0/232866320 learned_addr learned my addr 172.21.15.40:0/232866320 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:55.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.209+0000 7fd71b7fe700 1 --2- 172.21.15.40:0/232866320 >> v2:172.21.15.40:3300/0 conn(0x7fd71c069e00 0x7fd71c06a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:55.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.209+0000 7fd71a7fc700 1 --2- 172.21.15.40:0/232866320 >> v2:172.21.15.169:3300/0 conn(0x7fd71c071aa0 0x7fd71c06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:55.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.209+0000 7fd71b7fe700 1 -- 172.21.15.40:0/232866320 >> v2:172.21.15.169:3300/0 conn(0x7fd71c071aa0 msgr2=0x7fd71c06d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:55.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.209+0000 7fd71b7fe700 1 --2- 172.21.15.40:0/232866320 >> v2:172.21.15.169:3300/0 conn(0x7fd71c071aa0 0x7fd71c06d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.209+0000 7fd71b7fe700 1 -- 172.21.15.40:0/232866320 >> v1:172.21.15.40:6789/0 conn(0x7fd71c15c640 legacy=0x7fd71c0d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:55.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.209+0000 7fd71b7fe700 1 -- 172.21.15.40:0/232866320 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c0722f0 con 0x7fd71c069e00 2023-11-08T08:23:55.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.209+0000 7fd719ffb700 1 -- 172.21.15.40:0/232866320 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2539525597 0 0) 0x7fd71c06c600 con 0x7fd71c15c640 2023-11-08T08:23:55.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.209+0000 7fd71b7fe700 1 --2- 172.21.15.40:0/232866320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd71c069e00 0x7fd71c06a280 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd71c070e70 tx=0x7fd71400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1e1b2d929d274a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:55.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.209+0000 7fd719ffb700 1 -- 172.21.15.40:0/232866320 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71400e070 con 0x7fd71c069e00 2023-11-08T08:23:55.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.209+0000 7fd719ffb700 1 -- 172.21.15.40:0/232866320 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71400bbd0 con 0x7fd71c069e00 2023-11-08T08:23:55.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.209+0000 7fd719ffb700 1 -- 172.21.15.40:0/232866320 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd714012430 con 0x7fd71c069e00 2023-11-08T08:23:55.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.213+0000 7fd71bfff700 1 -- 172.21.15.40:0/232866320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd71c069e00 msgr2=0x7fd71c06a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:55.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.213+0000 7fd71bfff700 1 --2- 172.21.15.40:0/232866320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd71c069e00 0x7fd71c06a280 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd71c070e70 tx=0x7fd71400b330 comp rx=0 tx=0).stop 2023-11-08T08:23:55.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.213+0000 7fd71bfff700 1 -- 172.21.15.40:0/232866320 shutdown_connections 2023-11-08T08:23:55.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.213+0000 7fd71bfff700 1 --2- 172.21.15.40:0/232866320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd71c069e00 0x7fd71c06a280 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.213+0000 7fd71bfff700 1 --2- 172.21.15.40:0/232866320 >> v2:172.21.15.169:3300/0 conn(0x7fd71c071aa0 0x7fd71c06d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.213+0000 7fd71bfff700 1 -- 172.21.15.40:0/232866320 >> 172.21.15.40:0/232866320 conn(0x7fd71c0b6520 msgr2=0x7fd71c071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:55.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.213+0000 7fd71bfff700 1 -- 172.21.15.40:0/232866320 shutdown_connections 2023-11-08T08:23:55.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.213+0000 7fd71bfff700 1 -- 172.21.15.40:0/232866320 wait complete. 2023-11-08T08:23:55.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.217+0000 7fd71bfff700 1 Processor -- start 2023-11-08T08:23:55.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.217+0000 7fd71bfff700 1 -- start start 2023-11-08T08:23:55.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.217+0000 7fd71bfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd71c069e00 0x7fd71c0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:55.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.217+0000 7fd71bfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd71c071aa0 0x7fd71c1f1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:55.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.217+0000 7fd71bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd71c15c640 0x7fd71c06d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:55.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.217+0000 7fd71bfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c1f2840 con 0x7fd71c15c640 2023-11-08T08:23:55.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.217+0000 7fd71bfff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c0ceb70 con 0x7fd71c071aa0 2023-11-08T08:23:55.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.217+0000 7fd71bfff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c1f2280 con 0x7fd71c069e00 2023-11-08T08:23:55.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.217+0000 7fd71b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd71c15c640 0x7fd71c06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:55.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.217+0000 7fd71b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd71c15c640 0x7fd71c06d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41300/0 (socket says 172.21.15.40:41300) 2023-11-08T08:23:55.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.217+0000 7fd71b7fe700 1 -- 172.21.15.40:0/517030460 learned_addr learned my addr 172.21.15.40:0/517030460 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:55.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.217+0000 7fd71affd700 1 --2- 172.21.15.40:0/517030460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd71c069e00 0x7fd71c0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:55.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.217+0000 7fd71a7fc700 1 --2- 172.21.15.40:0/517030460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd71c071aa0 0x7fd71c1f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:55.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.221+0000 7fd71b7fe700 1 -- 172.21.15.40:0/517030460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd71c069e00 msgr2=0x7fd71c0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:55.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.221+0000 7fd71b7fe700 1 --2- 172.21.15.40:0/517030460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd71c069e00 0x7fd71c0cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.221+0000 7fd71b7fe700 1 -- 172.21.15.40:0/517030460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd71c071aa0 msgr2=0x7fd71c1f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:55.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.221+0000 7fd71b7fe700 1 --2- 172.21.15.40:0/517030460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd71c071aa0 0x7fd71c1f1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.221+0000 7fd71b7fe700 1 -- 172.21.15.40:0/517030460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd714009a70 con 0x7fd71c15c640 2023-11-08T08:23:55.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.221+0000 7fd71b7fe700 1 --2- 172.21.15.40:0/517030460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd71c15c640 0x7fd71c06d310 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd71c06f090 tx=0x7fd714004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:55.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.221+0000 7fd703fff700 1 -- 172.21.15.40:0/517030460 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd714012430 con 0x7fd71c15c640 2023-11-08T08:23:55.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.221+0000 7fd71bfff700 1 -- 172.21.15.40:0/517030460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71c072010 con 0x7fd71c15c640 2023-11-08T08:23:55.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.221+0000 7fd703fff700 1 -- 172.21.15.40:0/517030460 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd714020070 con 0x7fd71c15c640 2023-11-08T08:23:55.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.221+0000 7fd703fff700 1 -- 172.21.15.40:0/517030460 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd714004280 con 0x7fd71c15c640 2023-11-08T08:23:55.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.221+0000 7fd71bfff700 1 -- 172.21.15.40:0/517030460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71c15ca10 con 0x7fd71c15c640 2023-11-08T08:23:55.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.221+0000 7fd71bfff700 1 -- 172.21.15.40:0/517030460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71c06c830 con 0x7fd71c15c640 2023-11-08T08:23:55.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.225+0000 7fd703fff700 1 -- 172.21.15.40:0/517030460 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd7140043e0 con 0x7fd71c15c640 2023-11-08T08:23:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.225+0000 7fd703fff700 1 --2- 172.21.15.40:0/517030460 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd708064ee0 0x7fd708067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.225+0000 7fd703fff700 1 -- 172.21.15.40:0/517030460 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fd714022070 con 0x7fd71c15c640 2023-11-08T08:23:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.225+0000 7fd71affd700 1 --2- 172.21.15.40:0/517030460 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd708064ee0 0x7fd708067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.225+0000 7fd71affd700 1 --2- 172.21.15.40:0/517030460 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd708064ee0 0x7fd708067390 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd70c000f60 tx=0x7fd70c007e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.233+0000 7fd703fff700 1 -- 172.21.15.40:0/517030460 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd714052350 con 0x7fd71c15c640 2023-11-08T08:23:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.413+0000 7fd71bfff700 1 -- 172.21.15.40:0/517030460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd71c073cb0 con 0x7fd71c15c640 2023-11-08T08:23:55.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.421+0000 7fd703fff700 1 -- 172.21.15.40:0/517030460 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd714055970 con 0x7fd71c15c640 2023-11-08T08:23:55.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.425+0000 7fd71bfff700 1 -- 172.21.15.40:0/517030460 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd708064ee0 msgr2=0x7fd708067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:55.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.425+0000 7fd71bfff700 1 --2- 172.21.15.40:0/517030460 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd708064ee0 0x7fd708067390 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd70c000f60 tx=0x7fd70c007e00 comp rx=0 tx=0).stop 2023-11-08T08:23:55.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.425+0000 7fd71bfff700 1 -- 172.21.15.40:0/517030460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd71c15c640 msgr2=0x7fd71c06d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:55.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.425+0000 7fd71bfff700 1 --2- 172.21.15.40:0/517030460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd71c15c640 0x7fd71c06d310 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd71c06f090 tx=0x7fd714004af0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.425+0000 7fd71bfff700 1 -- 172.21.15.40:0/517030460 shutdown_connections 2023-11-08T08:23:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.425+0000 7fd71bfff700 1 --2- 172.21.15.40:0/517030460 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd708064ee0 0x7fd708067390 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.425+0000 7fd71bfff700 1 --2- 172.21.15.40:0/517030460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd71c15c640 0x7fd71c06d310 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.425+0000 7fd71bfff700 1 --2- 172.21.15.40:0/517030460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd71c071aa0 0x7fd71c1f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.425+0000 7fd71bfff700 1 --2- 172.21.15.40:0/517030460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd71c069e00 0x7fd71c0cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.425+0000 7fd71bfff700 1 -- 172.21.15.40:0/517030460 >> 172.21.15.40:0/517030460 conn(0x7fd71c0b6520 msgr2=0x7fd71c0d0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:55.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.425+0000 7fd71bfff700 1 -- 172.21.15.40:0/517030460 shutdown_connections 2023-11-08T08:23:55.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.425+0000 7fd71bfff700 1 -- 172.21.15.40:0/517030460 wait complete. 2023-11-08T08:23:55.453 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-08T08:23:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.625+0000 7f244359e700 1 Processor -- start 2023-11-08T08:23:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.625+0000 7f244359e700 1 -- start start 2023-11-08T08:23:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.629+0000 7f244359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f244415c640 0x7f24440d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.629+0000 7f244359e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2444071aa0 0x7f244406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.629+0000 7f244359e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2444069e00 0x7f244406a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.629+0000 7f244359e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244406cef0 con 0x7f2444071aa0 2023-11-08T08:23:55.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.629+0000 7f244359e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244406a7c0 con 0x7f2444069e00 2023-11-08T08:23:55.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.629+0000 7f244359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244406c600 con 0x7f244415c640 2023-11-08T08:23:55.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.629+0000 7f2441d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2444071aa0 0x7f244406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:55.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.629+0000 7f2441d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2444071aa0 0x7f244406d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41304/0 (socket says 172.21.15.40:41304) 2023-11-08T08:23:55.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.629+0000 7f2441d9b700 1 -- 172.21.15.40:0/2380080887 learned_addr learned my addr 172.21.15.40:0/2380080887 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:55.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.629+0000 7f244259c700 1 --2- 172.21.15.40:0/2380080887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f244415c640 0x7f24440d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:55.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.629+0000 7f2442d9d700 1 --2- 172.21.15.40:0/2380080887 >> v2:172.21.15.169:3300/0 conn(0x7f2444069e00 0x7f244406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:55.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.629+0000 7f244259c700 1 -- 172.21.15.40:0/2380080887 >> v2:172.21.15.169:3300/0 conn(0x7f2444069e00 msgr2=0x7f244406a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:55.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.629+0000 7f244259c700 1 --2- 172.21.15.40:0/2380080887 >> v2:172.21.15.169:3300/0 conn(0x7f2444069e00 0x7f244406a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.629+0000 7f244259c700 1 -- 172.21.15.40:0/2380080887 >> v2:172.21.15.40:3300/0 conn(0x7f2444071aa0 msgr2=0x7f244406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:55.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.629+0000 7f244259c700 1 --2- 172.21.15.40:0/2380080887 >> v2:172.21.15.40:3300/0 conn(0x7f2444071aa0 0x7f244406d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.629+0000 7f244259c700 1 -- 172.21.15.40:0/2380080887 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24440722f0 con 0x7f244415c640 2023-11-08T08:23:55.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.633+0000 7f244259c700 1 --2- 172.21.15.40:0/2380080887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f244415c640 0x7f24440d31e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f244406efb0 tx=0x7f2434009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f41483b30d8252 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:55.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.633+0000 7f244159a700 1 -- 172.21.15.40:0/2380080887 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434011830 con 0x7f244415c640 2023-11-08T08:23:55.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.633+0000 7f244159a700 1 -- 172.21.15.40:0/2380080887 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2434011990 con 0x7f244415c640 2023-11-08T08:23:55.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.633+0000 7f244159a700 1 -- 172.21.15.40:0/2380080887 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434011d20 con 0x7f244415c640 2023-11-08T08:23:55.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.633+0000 7f244359e700 1 -- 172.21.15.40:0/2380080887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f244415c640 msgr2=0x7f24440d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:55.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.633+0000 7f244359e700 1 --2- 172.21.15.40:0/2380080887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f244415c640 0x7f24440d31e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f244406efb0 tx=0x7f2434009b30 comp rx=0 tx=0).stop 2023-11-08T08:23:55.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.633+0000 7f244359e700 1 -- 172.21.15.40:0/2380080887 shutdown_connections 2023-11-08T08:23:55.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.633+0000 7f244359e700 1 --2- 172.21.15.40:0/2380080887 >> v2:172.21.15.40:3300/0 conn(0x7f2444071aa0 0x7f244406d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.633+0000 7f244359e700 1 --2- 172.21.15.40:0/2380080887 >> v2:172.21.15.169:3300/0 conn(0x7f2444069e00 0x7f244406a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.633+0000 7f244359e700 1 --2- 172.21.15.40:0/2380080887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f244415c640 0x7f24440d31e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.633+0000 7f244359e700 1 -- 172.21.15.40:0/2380080887 >> 172.21.15.40:0/2380080887 conn(0x7f24440b6520 msgr2=0x7f2444071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:55.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.633+0000 7f244359e700 1 -- 172.21.15.40:0/2380080887 shutdown_connections 2023-11-08T08:23:55.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.633+0000 7f244359e700 1 -- 172.21.15.40:0/2380080887 wait complete. 2023-11-08T08:23:55.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.637+0000 7f244359e700 1 Processor -- start 2023-11-08T08:23:55.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.637+0000 7f244359e700 1 -- start start 2023-11-08T08:23:55.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.637+0000 7f244359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2444069e00 0x7f24440c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:55.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.637+0000 7f244359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2444071aa0 0x7f24440cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:55.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.637+0000 7f244359e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f244415c640 0x7f24440cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:55.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.637+0000 7f244359e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24440cc940 con 0x7f2444069e00 2023-11-08T08:23:55.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.637+0000 7f244359e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24440cae20 con 0x7f244415c640 2023-11-08T08:23:55.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.637+0000 7f244359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24440cb100 con 0x7f2444071aa0 2023-11-08T08:23:55.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.637+0000 7f244259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2444069e00 0x7f24440c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:55.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.637+0000 7f244259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2444069e00 0x7f24440c6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41306/0 (socket says 172.21.15.40:41306) 2023-11-08T08:23:55.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.637+0000 7f244259c700 1 -- 172.21.15.40:0/344063235 learned_addr learned my addr 172.21.15.40:0/344063235 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:55.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.637+0000 7f2442d9d700 1 --2- 172.21.15.40:0/344063235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f244415c640 0x7f24440cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:55.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.637+0000 7f2441d9b700 1 --2- 172.21.15.40:0/344063235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2444071aa0 0x7f24440cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:55.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.637+0000 7f244259c700 1 -- 172.21.15.40:0/344063235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2444071aa0 msgr2=0x7f24440cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:55.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.637+0000 7f244259c700 1 --2- 172.21.15.40:0/344063235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2444071aa0 0x7f24440cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.637+0000 7f244259c700 1 -- 172.21.15.40:0/344063235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f244415c640 msgr2=0x7f24440cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:55.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.637+0000 7f244259c700 1 --2- 172.21.15.40:0/344063235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f244415c640 0x7f24440cc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:55.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.637+0000 7f244259c700 1 -- 172.21.15.40:0/344063235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24340094b0 con 0x7f2444069e00 2023-11-08T08:23:55.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.641+0000 7f244259c700 1 --2- 172.21.15.40:0/344063235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2444069e00 0x7f24440c6710 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f24340090b0 tx=0x7f2434004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:55.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.641+0000 7f2432ffd700 1 -- 172.21.15.40:0/344063235 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434004450 con 0x7f2444069e00 2023-11-08T08:23:55.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.641+0000 7f2432ffd700 1 -- 172.21.15.40:0/344063235 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243401f070 con 0x7f2444069e00 2023-11-08T08:23:55.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.641+0000 7f244359e700 1 -- 172.21.15.40:0/344063235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2444072010 con 0x7f2444069e00 2023-11-08T08:23:55.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.641+0000 7f2432ffd700 1 -- 172.21.15.40:0/344063235 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434003900 con 0x7f2444069e00 2023-11-08T08:23:55.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.641+0000 7f244359e700 1 -- 172.21.15.40:0/344063235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244415ca10 con 0x7f2444069e00 2023-11-08T08:23:55.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.641+0000 7f244359e700 1 -- 172.21.15.40:0/344063235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f244406c830 con 0x7f2444069e00 2023-11-08T08:23:55.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.641+0000 7f2432ffd700 1 -- 172.21.15.40:0/344063235 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f243400f050 con 0x7f2444069e00 2023-11-08T08:23:55.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.645+0000 7f2432ffd700 1 --2- 172.21.15.40:0/344063235 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2424064f30 0x7f24240673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:55.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.645+0000 7f2432ffd700 1 -- 172.21.15.40:0/344063235 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f2434021070 con 0x7f2444069e00 2023-11-08T08:23:55.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.645+0000 7f2441d9b700 1 --2- 172.21.15.40:0/344063235 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2424064f30 0x7f24240673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:55.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.645+0000 7f2441d9b700 1 --2- 172.21.15.40:0/344063235 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2424064f30 0x7f24240673e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f24440c6030 tx=0x7f24380092e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:55.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.649+0000 7f2432ffd700 1 -- 172.21.15.40:0/344063235 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f24340516b0 con 0x7f2444069e00 2023-11-08T08:23:55.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:55.881+0000 7f244359e700 1 -- 172.21.15.40:0/344063235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f244415ba60 con 0x7f2444069e00 2023-11-08T08:23:56.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:56.001+0000 7f2432ffd700 1 -- 172.21.15.40:0/344063235 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v254) v1 ==== 169+0+0 (secure 0 0 0) 0x7f24340045d0 con 0x7f2444069e00 2023-11-08T08:23:56.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:56.005+0000 7f244359e700 1 -- 172.21.15.40:0/344063235 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2424064f30 msgr2=0x7f24240673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:56.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:56.005+0000 7f244359e700 1 --2- 172.21.15.40:0/344063235 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2424064f30 0x7f24240673e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f24440c6030 tx=0x7f24380092e0 comp rx=0 tx=0).stop 2023-11-08T08:23:56.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:56.005+0000 7f244359e700 1 -- 172.21.15.40:0/344063235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2444069e00 msgr2=0x7f24440c6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:56.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:56.005+0000 7f244359e700 1 --2- 172.21.15.40:0/344063235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2444069e00 0x7f24440c6710 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f24340090b0 tx=0x7f2434004770 comp rx=0 tx=0).stop 2023-11-08T08:23:56.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:56.005+0000 7f244359e700 1 -- 172.21.15.40:0/344063235 shutdown_connections 2023-11-08T08:23:56.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:56.005+0000 7f244359e700 1 --2- 172.21.15.40:0/344063235 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2424064f30 0x7f24240673e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:56.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:56.005+0000 7f244359e700 1 --2- 172.21.15.40:0/344063235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f244415c640 0x7f24440cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:56.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:56.005+0000 7f244359e700 1 --2- 172.21.15.40:0/344063235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2444071aa0 0x7f24440cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:56.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:56.005+0000 7f244359e700 1 --2- 172.21.15.40:0/344063235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2444069e00 0x7f24440c6710 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:56.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:56.005+0000 7f244359e700 1 -- 172.21.15.40:0/344063235 >> 172.21.15.40:0/344063235 conn(0x7f24440b6520 msgr2=0x7f244406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:56.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:56.005+0000 7f244359e700 1 -- 172.21.15.40:0/344063235 shutdown_connections 2023-11-08T08:23:56.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:56.005+0000 7f244359e700 1 -- 172.21.15.40:0/344063235 wait complete. 2023-11-08T08:23:56.040 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... FAIL 2023-11-08T08:23:56.041 INFO:tasks.cephfs_test_runner: 2023-11-08T08:23:56.041 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-08T08:23:56.041 INFO:tasks.cephfs_test_runner:FAIL: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-08T08:23:56.042 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-08T08:23:56.042 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-08T08:23:56.042 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-08T08:23:56.043 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_b820560f30d310ab2570761b9a294523225d6998/qa/tasks/cephfs/test_forward_scrub.py", line 508, in test_health_status_after_backtrace_repair 2023-11-08T08:23:56.043 INFO:tasks.cephfs_test_runner: self.assertEqual(len(damage), 0) 2023-11-08T08:23:56.043 INFO:tasks.cephfs_test_runner:AssertionError: 1 != 0 2023-11-08T08:23:56.043 INFO:tasks.cephfs_test_runner: 2023-11-08T08:23:56.044 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-08T08:23:56.044 INFO:tasks.cephfs_test_runner:Ran 3 tests in 234.932s 2023-11-08T08:23:56.044 INFO:tasks.cephfs_test_runner: 2023-11-08T08:23:56.045 INFO:tasks.cephfs_test_runner:FAILED (failures=1) 2023-11-08T08:23:56.045 INFO:tasks.cephfs_test_runner: 2023-11-08T08:23:56.045 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-08T08:23:56.046 INFO:tasks.cephfs_test_runner:FAIL: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-08T08:23:56.046 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-08T08:23:56.046 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-08T08:23:56.047 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-08T08:23:56.047 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_b820560f30d310ab2570761b9a294523225d6998/qa/tasks/cephfs/test_forward_scrub.py", line 508, in test_health_status_after_backtrace_repair 2023-11-08T08:23:56.047 INFO:tasks.cephfs_test_runner: self.assertEqual(len(damage), 0) 2023-11-08T08:23:56.047 INFO:tasks.cephfs_test_runner:AssertionError: 1 != 0 2023-11-08T08:23:56.048 INFO:tasks.cephfs_test_runner: 2023-11-08T08:23:56.048 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b820560f30d310ab2570761b9a294523225d6998/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-08T08:23:56.117 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=285ff3871979442389b67eadd91839e3 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b820560f30d310ab2570761b9a294523225d6998/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-08T08:23:56.119 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-08T08:23:56.133 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-08T08:23:56.179 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-08T08:23:56.180 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:23:56.181 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:23:56.206 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16355412k,nr_inodes=4088853,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6645 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T08:23:56.207 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-11-08T08:23:56.207 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:23:56.268 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16355412k,nr_inodes=4088853,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=6645 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T08:23:56.269 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:23:56.269 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:23:56.287 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352316k,nr_inodes=4088079,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1804 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T08:23:56.288 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-08T08:23:56.288 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T08:23:56.353 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16352316k,nr_inodes=4088079,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1804 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T08:23:56.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:23:56.353 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-11-08T08:23:56.375 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T08:23:56.375 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T08:23:56.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:23:56.377 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T08:23:56.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:23:56.464 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T08:23:56.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:23:56.546 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2023-11-08T08:23:56.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:23:56.749 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-11-08T08:23:56.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:23:56.771 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2023-11-08T08:23:56.861 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2023-11-08T08:23:56.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T08:23:56.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:23:56.865 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-11-08T08:23:56.885 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-08T08:23:56.885 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-08T08:23:56.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:23:56.887 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T08:23:56.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:23:56.965 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-08T08:23:57.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:23:57.047 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2023-11-08T08:23:57.116 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:23:57.124 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:23:57.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:23:57.279 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-11-08T08:23:57.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T08:23:57.291 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2023-11-08T08:23:57.397 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2023-11-08T08:23:57.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T08:23:57.402 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-08T08:23:57.411 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-08T08:23:57.412 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T08:23:57.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.585+0000 7fec916dd700 1 Processor -- start 2023-11-08T08:23:57.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.589+0000 7fec916dd700 1 -- start start 2023-11-08T08:23:57.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.589+0000 7fec916dd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fec8c15c640 0x7fec8c0d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:57.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.589+0000 7fec916dd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec8c069e00 0x7fec8c06a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:57.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.589+0000 7fec916dd700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec8c06c600 con 0x7fec8c071aa0 2023-11-08T08:23:57.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.589+0000 7fec916dd700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec8c06cef0 con 0x7fec8c15c640 2023-11-08T08:23:57.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.589+0000 7fec916dd700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c06a7c0 con 0x7fec8c069e00 2023-11-08T08:23:57.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.589+0000 7fec837fe700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fec8c071aa0 0x7fec8c06d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:45170/0 (socket says 172.21.15.40:45170) 2023-11-08T08:23:57.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.589+0000 7fec837fe700 1 -- 172.21.15.40:0/2297045544 learned_addr learned my addr 172.21.15.40:0/2297045544 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:57.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.589+0000 7fec8bfff700 1 --2- 172.21.15.40:0/2297045544 >> v2:172.21.15.169:3300/0 conn(0x7fec8c15c640 0x7fec8c0d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:57.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.589+0000 7fec90edc700 1 --2- 172.21.15.40:0/2297045544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec8c069e00 0x7fec8c06a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:57.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.589+0000 7fec8b7fe700 1 -- 172.21.15.40:0/2297045544 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2837081918 0 0) 0x7fec8c06c600 con 0x7fec8c071aa0 2023-11-08T08:23:57.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.589+0000 7fec8b7fe700 1 -- 172.21.15.40:0/2297045544 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec780035d0 con 0x7fec8c071aa0 2023-11-08T08:23:57.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.593+0000 7fec8bfff700 1 -- 172.21.15.40:0/2297045544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec8c069e00 msgr2=0x7fec8c06a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:57.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.593+0000 7fec8bfff700 1 --2- 172.21.15.40:0/2297045544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec8c069e00 0x7fec8c06a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:57.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.593+0000 7fec8bfff700 1 -- 172.21.15.40:0/2297045544 >> v1:172.21.15.169:6789/0 conn(0x7fec8c071aa0 legacy=0x7fec8c06d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:57.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.593+0000 7fec8bfff700 1 -- 172.21.15.40:0/2297045544 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8c0722f0 con 0x7fec8c15c640 2023-11-08T08:23:57.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.593+0000 7fec8bfff700 1 --2- 172.21.15.40:0/2297045544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec8c15c640 0x7fec8c0d31e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009850 tx=0x7fec7c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e5bcdc3a0d5dbfc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:57.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.593+0000 7fec8b7fe700 1 -- 172.21.15.40:0/2297045544 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c011830 con 0x7fec8c15c640 2023-11-08T08:23:57.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.593+0000 7fec8b7fe700 1 -- 172.21.15.40:0/2297045544 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c011990 con 0x7fec8c15c640 2023-11-08T08:23:57.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.593+0000 7fec8b7fe700 1 -- 172.21.15.40:0/2297045544 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c011d40 con 0x7fec8c15c640 2023-11-08T08:23:57.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.593+0000 7fec916dd700 1 -- 172.21.15.40:0/2297045544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec8c15c640 msgr2=0x7fec8c0d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:57.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.593+0000 7fec916dd700 1 --2- 172.21.15.40:0/2297045544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec8c15c640 0x7fec8c0d31e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009850 tx=0x7fec7c009b60 comp rx=0 tx=0).stop 2023-11-08T08:23:57.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.593+0000 7fec916dd700 1 -- 172.21.15.40:0/2297045544 shutdown_connections 2023-11-08T08:23:57.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.593+0000 7fec916dd700 1 --2- 172.21.15.40:0/2297045544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec8c069e00 0x7fec8c06a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:57.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.593+0000 7fec916dd700 1 --2- 172.21.15.40:0/2297045544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec8c15c640 0x7fec8c0d31e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:57.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.593+0000 7fec916dd700 1 -- 172.21.15.40:0/2297045544 >> 172.21.15.40:0/2297045544 conn(0x7fec8c0b6520 msgr2=0x7fec8c071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:57.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.593+0000 7fec916dd700 1 -- 172.21.15.40:0/2297045544 shutdown_connections 2023-11-08T08:23:57.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.593+0000 7fec916dd700 1 -- 172.21.15.40:0/2297045544 wait complete. 2023-11-08T08:23:57.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.597+0000 7fec916dd700 1 Processor -- start 2023-11-08T08:23:57.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.597+0000 7fec916dd700 1 -- start start 2023-11-08T08:23:57.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec916dd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec8c069e00 0x7fec8c0ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:57.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec916dd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec8c071aa0 0x7fec8c1f1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:57.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec916dd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec8c15c640 0x7fec8c0c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:57.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec916dd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c09ba80 con 0x7fec8c069e00 2023-11-08T08:23:57.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec916dd700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c0cfcf0 con 0x7fec8c071aa0 2023-11-08T08:23:57.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec916dd700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c0cef30 con 0x7fec8c15c640 2023-11-08T08:23:57.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec8bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec8c069e00 0x7fec8c0ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:57.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec8bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec8c069e00 0x7fec8c0ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43794/0 (socket says 172.21.15.40:43794) 2023-11-08T08:23:57.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec8bfff700 1 -- 172.21.15.40:0/175123256 learned_addr learned my addr 172.21.15.40:0/175123256 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:57.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec837fe700 1 --2- 172.21.15.40:0/175123256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec8c071aa0 0x7fec8c1f1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:57.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec90edc700 1 --2- 172.21.15.40:0/175123256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec8c15c640 0x7fec8c0c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:57.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec8bfff700 1 -- 172.21.15.40:0/175123256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec8c15c640 msgr2=0x7fec8c0c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:57.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec8bfff700 1 --2- 172.21.15.40:0/175123256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec8c15c640 0x7fec8c0c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:57.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec8bfff700 1 -- 172.21.15.40:0/175123256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec8c071aa0 msgr2=0x7fec8c1f1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:57.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec8bfff700 1 --2- 172.21.15.40:0/175123256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec8c071aa0 0x7fec8c1f1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:57.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec8bfff700 1 -- 172.21.15.40:0/175123256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c0094b0 con 0x7fec8c069e00 2023-11-08T08:23:57.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec8bfff700 1 --2- 172.21.15.40:0/175123256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec8c069e00 0x7fec8c0ccde0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0090b0 tx=0x7fec7c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:57.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec897fa700 1 -- 172.21.15.40:0/175123256 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c004240 con 0x7fec8c069e00 2023-11-08T08:23:57.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec897fa700 1 -- 172.21.15.40:0/175123256 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c01f070 con 0x7fec8c069e00 2023-11-08T08:23:57.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec897fa700 1 -- 172.21.15.40:0/175123256 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c004600 con 0x7fec8c069e00 2023-11-08T08:23:57.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec916dd700 1 -- 172.21.15.40:0/175123256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec8c15ca10 con 0x7fec8c069e00 2023-11-08T08:23:57.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.601+0000 7fec916dd700 1 -- 172.21.15.40:0/175123256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8c06da20 con 0x7fec8c069e00 2023-11-08T08:23:57.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.605+0000 7fec916dd700 1 -- 172.21.15.40:0/175123256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8c072010 con 0x7fec8c069e00 2023-11-08T08:23:57.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.605+0000 7fec897fa700 1 -- 172.21.15.40:0/175123256 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fec7c02a020 con 0x7fec8c069e00 2023-11-08T08:23:57.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.609+0000 7fec897fa700 1 --2- 172.21.15.40:0/175123256 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fec6c064ee0 0x7fec6c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:57.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.609+0000 7fec897fa700 1 -- 172.21.15.40:0/175123256 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fec7c083020 con 0x7fec8c069e00 2023-11-08T08:23:57.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.609+0000 7fec837fe700 1 --2- 172.21.15.40:0/175123256 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fec6c064ee0 0x7fec6c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:57.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.609+0000 7fec837fe700 1 --2- 172.21.15.40:0/175123256 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fec6c064ee0 0x7fec6c067390 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0cff00 tx=0x7fec74001a90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:57.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.613+0000 7fec897fa700 1 -- 172.21.15.40:0/175123256 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fec7c0512b0 con 0x7fec8c069e00 2023-11-08T08:23:57.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.781+0000 7fec916dd700 1 -- 172.21.15.40:0/175123256 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec8c073cb0 con 0x7fec6c064ee0 2023-11-08T08:23:57.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.789+0000 7fec897fa700 1 -- 172.21.15.40:0/175123256 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70716 (secure 0 0 0) 0x7fec8c073cb0 con 0x7fec6c064ee0 2023-11-08T08:23:57.796 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:23:57.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.793+0000 7fec916dd700 1 -- 172.21.15.40:0/175123256 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fec6c064ee0 msgr2=0x7fec6c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:57.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.793+0000 7fec916dd700 1 --2- 172.21.15.40:0/175123256 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fec6c064ee0 0x7fec6c067390 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0cff00 tx=0x7fec74001a90 comp rx=0 tx=0).stop 2023-11-08T08:23:57.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.793+0000 7fec916dd700 1 -- 172.21.15.40:0/175123256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec8c069e00 msgr2=0x7fec8c0ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:57.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.793+0000 7fec916dd700 1 --2- 172.21.15.40:0/175123256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec8c069e00 0x7fec8c0ccde0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0090b0 tx=0x7fec7c004770 comp rx=0 tx=0).stop 2023-11-08T08:23:57.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.793+0000 7fec916dd700 1 -- 172.21.15.40:0/175123256 shutdown_connections 2023-11-08T08:23:57.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.793+0000 7fec916dd700 1 --2- 172.21.15.40:0/175123256 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fec6c064ee0 0x7fec6c067390 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:57.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.793+0000 7fec916dd700 1 --2- 172.21.15.40:0/175123256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec8c15c640 0x7fec8c0c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:57.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.793+0000 7fec916dd700 1 --2- 172.21.15.40:0/175123256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec8c071aa0 0x7fec8c1f1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:57.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.793+0000 7fec916dd700 1 --2- 172.21.15.40:0/175123256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec8c069e00 0x7fec8c0ccde0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:57.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.793+0000 7fec916dd700 1 -- 172.21.15.40:0/175123256 >> 172.21.15.40:0/175123256 conn(0x7fec8c0b6520 msgr2=0x7fec8c0b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:57.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.793+0000 7fec916dd700 1 -- 172.21.15.40:0/175123256 shutdown_connections 2023-11-08T08:23:57.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.793+0000 7fec916dd700 1 -- 172.21.15.40:0/175123256 wait complete. 2023-11-08T08:23:57.803 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-11-08T08:23:57.823 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2023-11-08T08:23:55.991639+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":89944,"kb_used_data":7960,"kb_used_omap":0,"kb_used_meta":81920,"kb_avail":754884776,"statfs":{"total":773094113280,"available":773002010624,"internally_reserved":0,"allocated":8151040,"data_stored":4434996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":83886080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":109,"apply_latency_ms":109,"commit_latency_ns":109000000,"apply_latency_ns":109000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12037,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-8,"num_read_kb":-7,"num_write":-75,"num_write_kb":-36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001833"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-08T08:22:46.723286+0000","last_change":"2023-11-08T08:18:58.771310+0000","last_active":"2023-11-08T08:22:46.723286+0000","last_peered":"2023-11-08T08:22:46.723286+0000","last_clean":"2023-11-08T08:22:46.723286+0000","last_became_active":"2023-11-08T08:18:58.770891+0000","last_became_peered":"2023-11-08T08:18:58.770891+0000","last_unstale":"2023-11-08T08:22:46.723286+0000","last_undegraded":"2023-11-08T08:22:46.723286+0000","last_fullsized":"2023-11-08T08:22:46.723286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-08T08:22:47.032127+0000","last_change":"2023-11-08T08:18:58.770483+0000","last_active":"2023-11-08T08:22:47.032127+0000","last_peered":"2023-11-08T08:22:47.032127+0000","last_clean":"2023-11-08T08:22:47.032127+0000","last_became_active":"2023-11-08T08:18:58.770040+0000","last_became_peered":"2023-11-08T08:18:58.770040+0000","last_unstale":"2023-11-08T08:22:47.032127+0000","last_undegraded":"2023-11-08T08:22:47.032127+0000","last_fullsized":"2023-11-08T08:22:47.032127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-08T08:22:46.053791+0000","last_change":"2023-11-08T08:18:58.766848+0000","last_active":"2023-11-08T08:22:46.053791+0000","last_peered":"2023-11-08T08:22:46.053791+0000","last_clean":"2023-11-08T08:22:46.053791+0000","last_became_active":"2023-11-08T08:18:58.766378+0000","last_became_peered":"2023-11-08T08:18:58.766378+0000","last_unstale":"2023-11-08T08:22:46.053791+0000","last_undegraded":"2023-11-08T08:22:46.053791+0000","last_fullsized":"2023-11-08T08:22:46.053791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-08T08:22:47.032385+0000","last_change":"2023-11-08T08:18:58.769764+0000","last_active":"2023-11-08T08:22:47.032385+0000","last_peered":"2023-11-08T08:22:47.032385+0000","last_clean":"2023-11-08T08:22:47.032385+0000","last_became_active":"2023-11-08T08:18:58.769357+0000","last_became_peered":"2023-11-08T08:18:58.769357+0000","last_unstale":"2023-11-08T08:22:47.032385+0000","last_undegraded":"2023-11-08T08:22:47.032385+0000","last_fullsized":"2023-11-08T08:22:47.032385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:13.748094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":55,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-08T08:22:46.054219+0000","last_change":"2023-11-08T08:18:58.771460+0000","last_active":"2023-11-08T08:22:46.054219+0000","last_peered":"2023-11-08T08:22:46.054219+0000","last_clean":"2023-11-08T08:22:46.054219+0000","last_became_active":"2023-11-08T08:18:58.770648+0000","last_became_peered":"2023-11-08T08:18:58.770648+0000","last_unstale":"2023-11-08T08:22:46.054219+0000","last_undegraded":"2023-11-08T08:22:46.054219+0000","last_fullsized":"2023-11-08T08:22:46.054219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-08T08:22:46.049045+0000","last_change":"2023-11-08T08:18:58.973732+0000","last_active":"2023-11-08T08:22:46.049045+0000","last_peered":"2023-11-08T08:22:46.049045+0000","last_clean":"2023-11-08T08:22:46.049045+0000","last_became_active":"2023-11-08T08:18:58.973007+0000","last_became_peered":"2023-11-08T08:18:58.973007+0000","last_unstale":"2023-11-08T08:22:46.049045+0000","last_undegraded":"2023-11-08T08:22:46.049045+0000","last_fullsized":"2023-11-08T08:22:46.049045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-08T08:22:46.053595+0000","last_change":"2023-11-08T08:18:58.770136+0000","last_active":"2023-11-08T08:22:46.053595+0000","last_peered":"2023-11-08T08:22:46.053595+0000","last_clean":"2023-11-08T08:22:46.053595+0000","last_became_active":"2023-11-08T08:18:58.769797+0000","last_became_peered":"2023-11-08T08:18:58.769797+0000","last_unstale":"2023-11-08T08:22:46.053595+0000","last_undegraded":"2023-11-08T08:22:46.053595+0000","last_fullsized":"2023-11-08T08:22:46.053595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:29:12.107128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":55,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-08T08:22:46.054694+0000","last_change":"2023-11-08T08:18:58.777140+0000","last_active":"2023-11-08T08:22:46.054694+0000","last_peered":"2023-11-08T08:22:46.054694+0000","last_clean":"2023-11-08T08:22:46.054694+0000","last_became_active":"2023-11-08T08:18:58.776834+0000","last_became_peered":"2023-11-08T08:18:58.776834+0000","last_unstale":"2023-11-08T08:22:46.054694+0000","last_undegraded":"2023-11-08T08:22:46.054694+0000","last_fullsized":"2023-11-08T08:22:46.054694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":209,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-08T08:22:46.053719+0000","last_change":"2023-11-08T08:18:53.780089+0000","last_active":"2023-11-08T08:22:46.053719+0000","last_peered":"2023-11-08T08:22:46.053719+0000","last_clean":"2023-11-08T08:22:46.053719+0000","last_became_active":"2023-11-08T08:18:53.779533+0000","last_became_peered":"2023-11-08T08:18:53.779533+0000","last_unstale":"2023-11-08T08:22:46.053719+0000","last_undegraded":"2023-11-08T08:22:46.053719+0000","last_fullsized":"2023-11-08T08:22:46.053719+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_clean_scrub_stamp":"2023-11-08T08:18:52.744693+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:58:24.877070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15196,"kb_used_data":1428,"kb_used_omap":0,"kb_used_meta":13760,"kb_avail":94356644,"statfs":{"total":96636764160,"available":96621203456,"internally_reserved":0,"allocated":1462272,"data_stored":994679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14090240},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.121,"15min":1.0409999999999999},"min":{"1min":0.84899999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6140000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.278},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.21,"15min":1.1759999999999999},"min":{"1min":0.80300000000000005,"5min":0.746,"15min":0.746},"max":{"1min":1.5640000000000001,"5min":1.667,"15min":1.667},"last":1.1399999999999999}]},{"osd":1,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.1899999999999999,"15min":1.194},"min":{"1min":0.78400000000000003,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.599,"5min":1.972,"15min":1.972},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.264,"15min":1.306},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.425,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.0149999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.278,"15min":1.387},"min":{"1min":0.96499999999999997,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.444,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":1.2},{"interface":"front","average":{"1min":1.077,"5min":1.246,"15min":1.3759999999999999},"min":{"1min":0.73199999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.446,"5min":4.9089999999999998,"15min":4.9089999999999998},"last":0.95499999999999996}]},{"osd":3,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2070000000000001,"15min":1.246},"min":{"1min":0.89900000000000002,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.724,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.252,"5min":1.2789999999999999,"15min":1.3029999999999999},"min":{"1min":0.92100000000000004,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.6850000000000001,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":1.325}]},{"osd":4,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2310000000000001,"15min":1.262},"min":{"1min":0.78900000000000003,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.3},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.375,"15min":1.419},"min":{"1min":0.97099999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.8180000000000001,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":0.91200000000000003}]},{"osd":5,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.292,"15min":1.3620000000000001},"min":{"1min":0.97299999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.756,"5min":2.73,"15min":2.73},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.294,"15min":1.329},"min":{"1min":0.80900000000000005,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.6259999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.0740000000000001}]},{"osd":6,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3999999999999999,"15min":1.4410000000000001},"min":{"1min":0.90000000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.724,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.3440000000000001,"15min":1.4219999999999999},"min":{"1min":0.83799999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.8799999999999999,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.1040000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9880,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361960,"statfs":{"total":96636764160,"available":96626647040,"internally_reserved":0,"allocated":868352,"data_stored":403803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.0800000000000001,"15min":1.159},"min":{"1min":0.76600000000000001,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5860000000000001,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.077,"5min":1.079,"15min":1.0960000000000001},"min":{"1min":0.71099999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.556,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.1120000000000001}]},{"osd":1,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.224,"15min":1.1990000000000001},"min":{"1min":0.89000000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2190000000000001,"15min":1.21},"min":{"1min":0.86799999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.603,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.169}]},{"osd":2,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.173,"15min":1.1830000000000001},"min":{"1min":0.76500000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.216,"15min":1.2769999999999999},"min":{"1min":0.81599999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5229999999999999,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.298}]},{"osd":3,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2549999999999999,"15min":1.2809999999999999},"min":{"1min":0.95199999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.474,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.266},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3759999999999999,"15min":1.4019999999999999},"min":{"1min":0.96699999999999997,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.966,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":0.93700000000000006}]},{"osd":4,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.208,"15min":1.2509999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.669,"5min":2.02,"15min":2.02},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.196,"15min":1.1919999999999999},"min":{"1min":0.79300000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.625,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.0309999999999999}]},{"osd":5,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.254,"15min":1.2370000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.276,"5min":1.3400000000000001,"15min":1.363},"min":{"1min":0.94699999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.7350000000000001,"5min":1.794,"15min":1.794},"last":1.3979999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3580000000000001,"15min":1.3779999999999999},"min":{"1min":1.0429999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.8009999999999999,"5min":3.133,"15min":3.133},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.444,"15min":1.4390000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.1960000000000002,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.2390000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574911,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9812,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362028,"statfs":{"total":96636764160,"available":96626716672,"internally_reserved":0,"allocated":864256,"data_stored":403801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.141,"15min":1.135},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.069},{"interface":"front","average":{"1min":1.236,"5min":1.228,"15min":1.238},"min":{"1min":0.78600000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":1.2130000000000001}]},{"osd":1,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.107,"15min":1.133},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.145,"15min":1.1850000000000001},"min":{"1min":0.85999999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.97399999999999998}]},{"osd":2,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.171,"15min":1.1910000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5189999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.167},{"interface":"front","average":{"1min":1.157,"5min":1.1599999999999999,"15min":1.212},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":1.3480000000000001}]},{"osd":3,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.304,"15min":1.3360000000000001},"min":{"1min":0.83799999999999997,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.2109999999999999,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":1.105},{"interface":"front","average":{"1min":1.242,"5min":1.2649999999999999,"15min":1.3180000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.9219999999999999,"5min":3.7999999999999998,"15min":3.7999999999999998},"last":0.94399999999999995}]},{"osd":4,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1359999999999999,"15min":1.194},"min":{"1min":0.76500000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.21,"15min":1.3129999999999999},"min":{"1min":0.56599999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.7310000000000001,"5min":3.073,"15min":3.073},"last":1.288}]},{"osd":6,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2370000000000001,"15min":1.3280000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.49,"5min":3.8650000000000002,"15min":3.8650000000000002},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2849999999999999,"15min":1.3620000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.675,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.4950000000000001}]},{"osd":7,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.393,"15min":1.47},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.0030000000000001,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3859999999999999,"15min":1.4750000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.659,"5min":3.9369999999999998,"15min":3.9369999999999998},"last":1.2490000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10068,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361772,"statfs":{"total":96636764160,"available":96626454528,"internally_reserved":0,"allocated":864256,"data_stored":403781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.159,"15min":1.149},"min":{"1min":0.69699999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5,"5min":1.655,"15min":1.655},"last":1.349},{"interface":"front","average":{"1min":1.258,"5min":1.2290000000000001,"15min":1.2350000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.3260000000000001}]},{"osd":1,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.0569999999999999,"15min":1.081},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.3109999999999999,"5min":1.99,"15min":1.99},"last":1.113},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.157,"15min":1.2},"min":{"1min":0.76900000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.6559999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.2909999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.1240000000000001,"15min":1.125},"min":{"1min":0.79400000000000004,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.341,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2270000000000001,"15min":1.2589999999999999},"min":{"1min":0.88400000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.468,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.165}]},{"osd":3,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.206},"min":{"1min":0.73399999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.776,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1910000000000001,"15min":1.2330000000000001},"min":{"1min":0.78900000000000003,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.423,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":0.82799999999999996}]},{"osd":5,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.2250000000000001,"15min":1.2769999999999999},"min":{"1min":0.77600000000000002,"5min":0.747,"15min":0.747},"max":{"1min":1.3700000000000001,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.238},{"interface":"front","average":{"1min":1.202,"5min":1.2410000000000001,"15min":1.2529999999999999},"min":{"1min":0.872,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.837,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":1.1459999999999999}]},{"osd":6,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.24,"15min":1.274},"min":{"1min":0.89600000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5129999999999999,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.3500000000000001,"15min":1.3660000000000001},"min":{"1min":0.878,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.6319999999999999,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.94699999999999995}]},{"osd":7,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.4239999999999999,"15min":1.4379999999999999},"min":{"1min":0.99199999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.968,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.4330000000000001,"15min":1.4770000000000001},"min":{"1min":1.119,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.9470000000000001,"5min":2.399,"15min":2.399},"last":0.92000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9944,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361896,"statfs":{"total":96636764160,"available":96626581504,"internally_reserved":0,"allocated":868352,"data_stored":403823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.86899999999999999,"15min":0.90100000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.468,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.92700000000000005,"15min":0.94099999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.5580000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.66900000000000004}]},{"osd":1,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.90400000000000003,"15min":0.93200000000000005},"min":{"1min":0.60599999999999998,"5min":0.438,"15min":0.438},"max":{"1min":1.339,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.0720000000000001,"15min":1.1140000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5960000000000001,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.72699999999999998}]},{"osd":2,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.111,"15min":1.111},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.135,"5min":1.1719999999999999,"15min":1.2110000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.984,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":0.86199999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.196,"15min":1.282},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.7529999999999999,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.3120000000000001,"15min":1.3799999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.5449999999999999,"5min":2.27,"15min":2.27},"last":1.0229999999999999}]},{"osd":5,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.3620000000000001,"15min":1.425},"min":{"1min":0.93000000000000005,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.657,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.359,"15min":1.415},"min":{"1min":0.92000000000000004,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.1160000000000001,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.0820000000000001}]},{"osd":6,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.373,"15min":1.4590000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2,"5min":3.0950000000000002,"15min":3.0950000000000002},"last":1.206},{"interface":"front","average":{"1min":1.369,"5min":1.46,"15min":1.526},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.117,"5min":3.02,"15min":3.02},"last":1.179}]},{"osd":7,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.5029999999999999,"15min":1.583},"min":{"1min":0.97999999999999998,"5min":0.96799999999999997,"15min":0.96799999999999997},"max":{"1min":2.0230000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":1.123},{"interface":"front","average":{"1min":1.397,"5min":1.4570000000000001,"15min":1.5669999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.9259999999999999,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.2929999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9828,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362012,"statfs":{"total":96636764160,"available":96626700288,"internally_reserved":0,"allocated":880640,"data_stored":415215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.88500000000000001,"15min":0.93200000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.153,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.95799999999999996,"15min":0.99399999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.4350000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.014}]},{"osd":1,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0529999999999999,"15min":1.103},"min":{"1min":0.67100000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.623,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":1,"5min":1.0229999999999999,"15min":1.0269999999999999},"min":{"1min":0.55200000000000005,"5min":0.505,"15min":0.505},"max":{"1min":1.6399999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.68100000000000005}]},{"osd":3,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":0.998,"15min":1.014},"min":{"1min":0.64500000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.585,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.22,"15min":1.3260000000000001},"min":{"1min":0.88600000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.502,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":1.0409999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3640000000000001,"15min":1.458},"min":{"1min":1.034,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.7270000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":1.353},{"interface":"front","average":{"1min":1.278,"5min":1.3089999999999999,"15min":1.3580000000000001},"min":{"1min":0.99099999999999999,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.669,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.3129999999999999}]},{"osd":5,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.367,"15min":1.427},"min":{"1min":0.86199999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.76,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.097},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.319,"15min":1.343},"min":{"1min":0.96899999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.7450000000000001,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.91000000000000003}]},{"osd":6,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3839999999999999,"15min":1.4199999999999999},"min":{"1min":0.91200000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.722,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.397,"5min":1.458,"15min":1.5640000000000001},"min":{"1min":0.94699999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.869,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":1.2490000000000001}]},{"osd":7,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4239999999999999,"15min":1.47},"min":{"1min":0.92600000000000005,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.9870000000000001,"5min":2.867,"15min":2.867},"last":1.391},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4630000000000001,"15min":1.4830000000000001},"min":{"1min":0.93100000000000005,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.8109999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.2849999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9964,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361876,"statfs":{"total":96636764160,"available":96626561024,"internally_reserved":0,"allocated":888832,"data_stored":415815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.86599999999999999,"15min":0.89100000000000001},"min":{"1min":0.54700000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3620000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.016,"15min":1.04},"min":{"1min":0.53700000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4630000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.85199999999999998}]},{"osd":2,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.95899999999999996,"15min":0.98699999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3859999999999999,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.755},{"interface":"front","average":{"1min":1.109,"5min":1.1200000000000001,"15min":1.1779999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.522,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.96299999999999997}]},{"osd":3,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0820000000000001,"15min":1.103},"min":{"1min":0.56899999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.357,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.03},{"interface":"front","average":{"1min":0.88800000000000001,"5min":1.0209999999999999,"15min":1.097},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3120000000000001,"5min":1.972,"15min":1.972},"last":0.69799999999999995}]},{"osd":4,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.2070000000000001,"15min":1.228},"min":{"1min":0.82299999999999995,"5min":0.746,"15min":0.746},"max":{"1min":1.556,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.24},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2869999999999999,"15min":1.337},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.4930000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":1.139}]},{"osd":5,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.294,"15min":1.329},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2889999999999999,"15min":1.3360000000000001},"min":{"1min":0.875,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.577,"5min":2.48,"15min":2.48},"last":1.1830000000000001}]},{"osd":6,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3999999999999999,"15min":1.4370000000000001},"min":{"1min":0.95399999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.6160000000000001,"5min":3.149,"15min":3.149},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.4199999999999999,"15min":1.464},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.9530000000000001,"5min":2.169,"15min":2.169},"last":0.997}]},{"osd":7,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4970000000000001,"15min":1.5529999999999999},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.0939999999999999,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.444},{"interface":"front","average":{"1min":1.405,"5min":1.4410000000000001,"15min":1.4690000000000001},"min":{"1min":0.96099999999999997,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.3700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15252,"kb_used_data":1420,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94356588,"statfs":{"total":96636764160,"available":96621146112,"internally_reserved":0,"allocated":1454080,"data_stored":994079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.89500000000000002,"15min":0.95699999999999996},"min":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0660000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.95899999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.71499999999999997}]},{"osd":2,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.94799999999999995,"15min":0.99299999999999999},"min":{"1min":0.63800000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.1759999999999999,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.008,"5min":1.0569999999999999,"15min":1.103},"min":{"1min":0.72599999999999998,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4039999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.93300000000000005}]},{"osd":3,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.0649999999999999,"15min":1.145},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3300000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.139,"15min":1.1910000000000001},"min":{"1min":0.70399999999999996,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4590000000000001,"5min":1.786,"15min":1.786},"last":1.042}]},{"osd":4,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.3939999999999999,"15min":1.4970000000000001},"min":{"1min":0.94799999999999995,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.7,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.417,"15min":1.4730000000000001},"min":{"1min":1.05,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.7709999999999999,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.1240000000000001}]},{"osd":5,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.421,"15min":1.498},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":1.9199999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.355},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.5029999999999999,"15min":1.6240000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.6830000000000001,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.0900000000000001}]},{"osd":6,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.425,"15min":1.5149999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.8879999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.4910000000000001,"15min":1.5680000000000001},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.827,"5min":2.048,"15min":2.048},"last":1.0740000000000001}]},{"osd":7,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.536,"15min":1.6339999999999999},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":2.0070000000000001,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.175},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.538,"15min":1.6619999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.5940000000000001,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":1.016}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T08:23:57.826 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T08:23:58.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.997+0000 7fbdaab01700 1 Processor -- start 2023-11-08T08:23:58.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:57.997+0000 7fbdaab01700 1 -- start start 2023-11-08T08:23:58.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.001+0000 7fbdaab01700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbda406ad70 0x7fbda4165210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:58.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.001+0000 7fbdaab01700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda415a850 con 0x7fbda40c8d10 2023-11-08T08:23:58.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.001+0000 7fbdaab01700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda40cada0 con 0x7fbda415a480 2023-11-08T08:23:58.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.001+0000 7fbdaab01700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda40cd0b0 con 0x7fbda406ad70 2023-11-08T08:23:58.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.001+0000 7fbda909e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbda406ad70 0x7fbda4165210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:58.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.001+0000 7fbda909e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbda406ad70 0x7fbda4165210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43810/0 (socket says 172.21.15.40:43810) 2023-11-08T08:23:58.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.001+0000 7fbda909e700 1 -- 172.21.15.40:0/502505114 learned_addr learned my addr 172.21.15.40:0/502505114 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:58.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.001+0000 7fbda909e700 1 -- 172.21.15.40:0/502505114 >> v1:172.21.15.169:6789/0 conn(0x7fbda415a480 legacy=0x7fbda40cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:58.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.001+0000 7fbda909e700 1 -- 172.21.15.40:0/502505114 >> v1:172.21.15.40:6789/0 conn(0x7fbda40c8d10 legacy=0x7fbda40cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:58.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.001+0000 7fbda909e700 1 -- 172.21.15.40:0/502505114 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda40ce060 con 0x7fbda406ad70 2023-11-08T08:23:58.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.001+0000 7fbda909e700 1 --2- 172.21.15.40:0/502505114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbda406ad70 0x7fbda4165210 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009e50 tx=0x7fbd9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f031f7a30033ac9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:58.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.001+0000 7fbd9b7fe700 1 -- 172.21.15.40:0/502505114 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c00e070 con 0x7fbda406ad70 2023-11-08T08:23:58.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.001+0000 7fbd9b7fe700 1 -- 172.21.15.40:0/502505114 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c003a40 con 0x7fbda406ad70 2023-11-08T08:23:58.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.001+0000 7fbd9b7fe700 1 -- 172.21.15.40:0/502505114 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c003df0 con 0x7fbda406ad70 2023-11-08T08:23:58.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.005+0000 7fbdaab01700 1 -- 172.21.15.40:0/502505114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbda406ad70 msgr2=0x7fbda4165210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:58.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.005+0000 7fbdaab01700 1 --2- 172.21.15.40:0/502505114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbda406ad70 0x7fbda4165210 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009e50 tx=0x7fbd9c00b330 comp rx=0 tx=0).stop 2023-11-08T08:23:58.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.005+0000 7fbdaab01700 1 -- 172.21.15.40:0/502505114 shutdown_connections 2023-11-08T08:23:58.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.005+0000 7fbdaab01700 1 --2- 172.21.15.40:0/502505114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbda406ad70 0x7fbda4165210 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.005+0000 7fbdaab01700 1 -- 172.21.15.40:0/502505114 >> 172.21.15.40:0/502505114 conn(0x7fbda40cf350 msgr2=0x7fbda40b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:58.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.005+0000 7fbdaab01700 1 -- 172.21.15.40:0/502505114 shutdown_connections 2023-11-08T08:23:58.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.005+0000 7fbdaab01700 1 -- 172.21.15.40:0/502505114 wait complete. 2023-11-08T08:23:58.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.009+0000 7fbdaab01700 1 Processor -- start 2023-11-08T08:23:58.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.009+0000 7fbdaab01700 1 -- start start 2023-11-08T08:23:58.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.009+0000 7fbdaab01700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbda406ad70 0x7fbda4151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:58.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.009+0000 7fbdaab01700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbda40c8d10 0x7fbda41f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:58.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.009+0000 7fbdaab01700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbda415a480 0x7fbda41538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:58.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.009+0000 7fbdaab01700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda40c5790 con 0x7fbda415a480 2023-11-08T08:23:58.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.009+0000 7fbdaab01700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda40c5d50 con 0x7fbda40c8d10 2023-11-08T08:23:58.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.009+0000 7fbdaab01700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda41f7060 con 0x7fbda406ad70 2023-11-08T08:23:58.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.009+0000 7fbd9bfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbda40c8d10 0x7fbda41f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:58.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.009+0000 7fbda909e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbda415a480 0x7fbda41538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:58.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.009+0000 7fbda889d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbda406ad70 0x7fbda4151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:58.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.009+0000 7fbd9bfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbda40c8d10 0x7fbda41f6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.40:56660/0 (socket says 172.21.15.40:56660) 2023-11-08T08:23:58.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.009+0000 7fbd9bfff700 1 -- 172.21.15.40:0/1697790472 learned_addr learned my addr 172.21.15.40:0/1697790472 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:58.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.009+0000 7fbda909e700 1 --2- 172.21.15.40:0/1697790472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbda415a480 0x7fbda41538b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43816/0 (socket says 172.21.15.40:43816) 2023-11-08T08:23:58.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.013+0000 7fbd9bfff700 1 -- 172.21.15.40:0/1697790472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbda406ad70 msgr2=0x7fbda4151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:58.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.013+0000 7fbd9bfff700 1 --2- 172.21.15.40:0/1697790472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbda406ad70 0x7fbda4151450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.013+0000 7fbd9bfff700 1 -- 172.21.15.40:0/1697790472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbda415a480 msgr2=0x7fbda41538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:58.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.013+0000 7fbd9bfff700 1 --2- 172.21.15.40:0/1697790472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbda415a480 0x7fbda41538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.013+0000 7fbd9bfff700 1 -- 172.21.15.40:0/1697790472 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c009b30 con 0x7fbda40c8d10 2023-11-08T08:23:58.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.013+0000 7fbda909e700 1 --2- 172.21.15.40:0/1697790472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbda415a480 0x7fbda41538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:23:58.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.013+0000 7fbd9bfff700 1 --2- 172.21.15.40:0/1697790472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbda40c8d10 0x7fbda41f6570 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400c410 tx=0x7fbd9400c720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:58.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.013+0000 7fbd997fa700 1 -- 172.21.15.40:0/1697790472 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd940052d0 con 0x7fbda40c8d10 2023-11-08T08:23:58.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.013+0000 7fbd997fa700 1 -- 172.21.15.40:0/1697790472 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd94005430 con 0x7fbda40c8d10 2023-11-08T08:23:58.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.013+0000 7fbd997fa700 1 -- 172.21.15.40:0/1697790472 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd940055a0 con 0x7fbda40c8d10 2023-11-08T08:23:58.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.013+0000 7fbdaab01700 1 -- 172.21.15.40:0/1697790472 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda40caa30 con 0x7fbda40c8d10 2023-11-08T08:23:58.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.013+0000 7fbdaab01700 1 -- 172.21.15.40:0/1697790472 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda40ce300 con 0x7fbda40c8d10 2023-11-08T08:23:58.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.013+0000 7fbdaab01700 1 -- 172.21.15.40:0/1697790472 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda40add50 con 0x7fbda40c8d10 2023-11-08T08:23:58.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.017+0000 7fbd997fa700 1 -- 172.21.15.40:0/1697790472 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fbd94010050 con 0x7fbda40c8d10 2023-11-08T08:23:58.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.017+0000 7fbd997fa700 1 --2- 172.21.15.40:0/1697790472 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbd90064f30 0x7fbd900673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:58.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.017+0000 7fbd997fa700 1 -- 172.21.15.40:0/1697790472 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fbd94080d00 con 0x7fbda40c8d10 2023-11-08T08:23:58.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.017+0000 7fbda889d700 1 --2- 172.21.15.40:0/1697790472 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbd90064f30 0x7fbd900673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:58.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.017+0000 7fbda889d700 1 --2- 172.21.15.40:0/1697790472 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbd90064f30 0x7fbd900673e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbda406b530 tx=0x7fbda0007ee0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:58.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.021+0000 7fbd997fa700 1 -- 172.21.15.40:0/1697790472 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fbd94049400 con 0x7fbda40c8d10 2023-11-08T08:23:58.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.193+0000 7fbdaab01700 1 -- 172.21.15.40:0/1697790472 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbda41f7550 con 0x7fbd90064f30 2023-11-08T08:23:58.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.197+0000 7fbd997fa700 1 -- 172.21.15.40:0/1697790472 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70557 (secure 0 0 0) 0x7fbda41f7550 con 0x7fbd90064f30 2023-11-08T08:23:58.207 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:23:58.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.201+0000 7fbdaab01700 1 -- 172.21.15.40:0/1697790472 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbd90064f30 msgr2=0x7fbd900673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:58.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.201+0000 7fbdaab01700 1 --2- 172.21.15.40:0/1697790472 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbd90064f30 0x7fbd900673e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbda406b530 tx=0x7fbda0007ee0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.201+0000 7fbdaab01700 1 -- 172.21.15.40:0/1697790472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbda40c8d10 msgr2=0x7fbda41f6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:58.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.201+0000 7fbdaab01700 1 --2- 172.21.15.40:0/1697790472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbda40c8d10 0x7fbda41f6570 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400c410 tx=0x7fbd9400c720 comp rx=0 tx=0).stop 2023-11-08T08:23:58.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.205+0000 7fbdaab01700 1 -- 172.21.15.40:0/1697790472 shutdown_connections 2023-11-08T08:23:58.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.205+0000 7fbdaab01700 1 --2- 172.21.15.40:0/1697790472 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fbd90064f30 0x7fbd900673e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.205+0000 7fbdaab01700 1 --2- 172.21.15.40:0/1697790472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbda415a480 0x7fbda41538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.205+0000 7fbdaab01700 1 --2- 172.21.15.40:0/1697790472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbda40c8d10 0x7fbda41f6570 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.205+0000 7fbdaab01700 1 --2- 172.21.15.40:0/1697790472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbda406ad70 0x7fbda4151450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.205+0000 7fbdaab01700 1 -- 172.21.15.40:0/1697790472 >> 172.21.15.40:0/1697790472 conn(0x7fbda40cf350 msgr2=0x7fbda40ae8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:58.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.205+0000 7fbdaab01700 1 -- 172.21.15.40:0/1697790472 shutdown_connections 2023-11-08T08:23:58.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.205+0000 7fbdaab01700 1 -- 172.21.15.40:0/1697790472 wait complete. 2023-11-08T08:23:58.214 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-11-08T08:23:58.234 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2023-11-08T08:23:57.992133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":90984,"kb_used_data":8232,"kb_used_omap":0,"kb_used_meta":82688,"kb_avail":754883736,"statfs":{"total":773094113280,"available":773000945664,"internally_reserved":0,"allocated":8429568,"data_stored":4569896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":84672512},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12286,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-8,"num_read_kb":-7,"num_write":-80,"num_write_kb":-39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.265165"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:53.995671+0000","last_change":"2023-11-08T08:18:58.771310+0000","last_active":"2023-11-08T08:23:53.995671+0000","last_peered":"2023-11-08T08:23:53.995671+0000","last_clean":"2023-11-08T08:23:53.995671+0000","last_became_active":"2023-11-08T08:18:58.770891+0000","last_became_peered":"2023-11-08T08:18:58.770891+0000","last_unstale":"2023-11-08T08:23:53.995671+0000","last_undegraded":"2023-11-08T08:23:53.995671+0000","last_fullsized":"2023-11-08T08:23:53.995671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:54.762987+0000","last_change":"2023-11-08T08:18:58.770483+0000","last_active":"2023-11-08T08:23:54.762987+0000","last_peered":"2023-11-08T08:23:54.762987+0000","last_clean":"2023-11-08T08:23:54.762987+0000","last_became_active":"2023-11-08T08:18:58.770040+0000","last_became_peered":"2023-11-08T08:18:58.770040+0000","last_unstale":"2023-11-08T08:23:54.762987+0000","last_undegraded":"2023-11-08T08:23:54.762987+0000","last_fullsized":"2023-11-08T08:23:54.762987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":59,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:53.996243+0000","last_change":"2023-11-08T08:18:58.766848+0000","last_active":"2023-11-08T08:23:53.996243+0000","last_peered":"2023-11-08T08:23:53.996243+0000","last_clean":"2023-11-08T08:23:53.996243+0000","last_became_active":"2023-11-08T08:18:58.766378+0000","last_became_peered":"2023-11-08T08:18:58.766378+0000","last_unstale":"2023-11-08T08:23:53.996243+0000","last_undegraded":"2023-11-08T08:23:53.996243+0000","last_fullsized":"2023-11-08T08:23:53.996243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:54.764100+0000","last_change":"2023-11-08T08:18:58.769764+0000","last_active":"2023-11-08T08:23:54.764100+0000","last_peered":"2023-11-08T08:23:54.764100+0000","last_clean":"2023-11-08T08:23:54.764100+0000","last_became_active":"2023-11-08T08:18:58.769357+0000","last_became_peered":"2023-11-08T08:18:58.769357+0000","last_unstale":"2023-11-08T08:23:54.764100+0000","last_undegraded":"2023-11-08T08:23:54.764100+0000","last_fullsized":"2023-11-08T08:23:54.764100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:13.748094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":58,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:54.761825+0000","last_change":"2023-11-08T08:18:58.771460+0000","last_active":"2023-11-08T08:23:54.761825+0000","last_peered":"2023-11-08T08:23:54.761825+0000","last_clean":"2023-11-08T08:23:54.761825+0000","last_became_active":"2023-11-08T08:18:58.770648+0000","last_became_peered":"2023-11-08T08:18:58.770648+0000","last_unstale":"2023-11-08T08:23:54.761825+0000","last_undegraded":"2023-11-08T08:23:54.761825+0000","last_fullsized":"2023-11-08T08:23:54.761825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":60,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:54.763825+0000","last_change":"2023-11-08T08:18:58.973732+0000","last_active":"2023-11-08T08:23:54.763825+0000","last_peered":"2023-11-08T08:23:54.763825+0000","last_clean":"2023-11-08T08:23:54.763825+0000","last_became_active":"2023-11-08T08:18:58.973007+0000","last_became_peered":"2023-11-08T08:18:58.973007+0000","last_unstale":"2023-11-08T08:23:54.763825+0000","last_undegraded":"2023-11-08T08:23:54.763825+0000","last_fullsized":"2023-11-08T08:23:54.763825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":59,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:53.993431+0000","last_change":"2023-11-08T08:18:58.770136+0000","last_active":"2023-11-08T08:23:53.993431+0000","last_peered":"2023-11-08T08:23:53.993431+0000","last_clean":"2023-11-08T08:23:53.993431+0000","last_became_active":"2023-11-08T08:18:58.769797+0000","last_became_peered":"2023-11-08T08:18:58.769797+0000","last_unstale":"2023-11-08T08:23:53.993431+0000","last_undegraded":"2023-11-08T08:23:53.993431+0000","last_fullsized":"2023-11-08T08:23:53.993431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:29:12.107128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":58,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:54.762277+0000","last_change":"2023-11-08T08:18:58.777140+0000","last_active":"2023-11-08T08:23:54.762277+0000","last_peered":"2023-11-08T08:23:54.762277+0000","last_clean":"2023-11-08T08:23:54.762277+0000","last_became_active":"2023-11-08T08:18:58.776834+0000","last_became_peered":"2023-11-08T08:18:58.776834+0000","last_unstale":"2023-11-08T08:23:54.762277+0000","last_undegraded":"2023-11-08T08:23:54.762277+0000","last_fullsized":"2023-11-08T08:23:54.762277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":212,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:53.993559+0000","last_change":"2023-11-08T08:18:53.780089+0000","last_active":"2023-11-08T08:23:53.993559+0000","last_peered":"2023-11-08T08:23:53.993559+0000","last_clean":"2023-11-08T08:23:53.993559+0000","last_became_active":"2023-11-08T08:18:53.779533+0000","last_became_peered":"2023-11-08T08:18:53.779533+0000","last_unstale":"2023-11-08T08:23:53.993559+0000","last_undegraded":"2023-11-08T08:23:53.993559+0000","last_fullsized":"2023-11-08T08:23:53.993559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_clean_scrub_stamp":"2023-11-08T08:18:52.744693+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:58:24.877070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15296,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94356544,"statfs":{"total":96636764160,"available":96621101056,"internally_reserved":0,"allocated":1499136,"data_stored":1014013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.121,"15min":1.0409999999999999},"min":{"1min":0.84899999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6140000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.278},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.21,"15min":1.1759999999999999},"min":{"1min":0.80300000000000005,"5min":0.746,"15min":0.746},"max":{"1min":1.5640000000000001,"5min":1.667,"15min":1.667},"last":1.1399999999999999}]},{"osd":1,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.1899999999999999,"15min":1.194},"min":{"1min":0.78400000000000003,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.599,"5min":1.972,"15min":1.972},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.264,"15min":1.306},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.425,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.0149999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.278,"15min":1.387},"min":{"1min":0.96499999999999997,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.444,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":1.2},{"interface":"front","average":{"1min":1.077,"5min":1.246,"15min":1.3759999999999999},"min":{"1min":0.73199999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.446,"5min":4.9089999999999998,"15min":4.9089999999999998},"last":0.95499999999999996}]},{"osd":3,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2070000000000001,"15min":1.246},"min":{"1min":0.89900000000000002,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.724,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.252,"5min":1.2789999999999999,"15min":1.3029999999999999},"min":{"1min":0.92100000000000004,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.6850000000000001,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":1.325}]},{"osd":4,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2310000000000001,"15min":1.262},"min":{"1min":0.78900000000000003,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.3},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.375,"15min":1.419},"min":{"1min":0.97099999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.8180000000000001,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":0.91200000000000003}]},{"osd":5,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.292,"15min":1.3620000000000001},"min":{"1min":0.97299999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.756,"5min":2.73,"15min":2.73},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.294,"15min":1.329},"min":{"1min":0.80900000000000005,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.6259999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.0740000000000001}]},{"osd":6,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3999999999999999,"15min":1.4410000000000001},"min":{"1min":0.90000000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.724,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.3440000000000001,"15min":1.4219999999999999},"min":{"1min":0.83799999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.8799999999999999,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.1040000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9980,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361860,"statfs":{"total":96636764160,"available":96626544640,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.0800000000000001,"15min":1.159},"min":{"1min":0.76600000000000001,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5860000000000001,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.04},{"interface":"front","average":{"1min":1.077,"5min":1.079,"15min":1.0960000000000001},"min":{"1min":0.71099999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.556,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.96999999999999997}]},{"osd":1,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.224,"15min":1.1990000000000001},"min":{"1min":0.89000000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.222},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2190000000000001,"15min":1.21},"min":{"1min":0.86799999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.603,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.3520000000000001}]},{"osd":2,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.173,"15min":1.1830000000000001},"min":{"1min":0.76500000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.216,"15min":1.2769999999999999},"min":{"1min":0.81599999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5229999999999999,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.4239999999999999}]},{"osd":3,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2549999999999999,"15min":1.2809999999999999},"min":{"1min":0.95199999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.474,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.319},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3759999999999999,"15min":1.4019999999999999},"min":{"1min":0.96699999999999997,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.966,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":1.5009999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.208,"15min":1.2509999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.669,"5min":2.02,"15min":2.02},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.196,"15min":1.1919999999999999},"min":{"1min":0.79300000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.625,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.1060000000000001}]},{"osd":5,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.254,"15min":1.2370000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":1.385},{"interface":"front","average":{"1min":1.276,"5min":1.3400000000000001,"15min":1.363},"min":{"1min":0.94699999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.7350000000000001,"5min":1.794,"15min":1.794},"last":1.2869999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3580000000000001,"15min":1.3779999999999999},"min":{"1min":1.0429999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.8009999999999999,"5min":3.133,"15min":3.133},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.444,"15min":1.4390000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.1960000000000002,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.544}]}]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9916,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361924,"statfs":{"total":96636764160,"available":96626610176,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.141,"15min":1.135},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.228,"15min":1.238},"min":{"1min":0.78600000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":1.171}]},{"osd":1,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.107,"15min":1.133},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.145,"15min":1.1850000000000001},"min":{"1min":0.85999999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":1.216}]},{"osd":2,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.171,"15min":1.1910000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5189999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.129},{"interface":"front","average":{"1min":1.157,"5min":1.1599999999999999,"15min":1.212},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.96399999999999997}]},{"osd":3,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.304,"15min":1.3360000000000001},"min":{"1min":0.83799999999999997,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.2109999999999999,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.2649999999999999,"15min":1.3180000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.9219999999999999,"5min":3.7999999999999998,"15min":3.7999999999999998},"last":1.1020000000000001}]},{"osd":4,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1359999999999999,"15min":1.194},"min":{"1min":0.76500000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.21,"15min":1.3129999999999999},"min":{"1min":0.56599999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.7310000000000001,"5min":3.073,"15min":3.073},"last":1.2490000000000001}]},{"osd":6,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2370000000000001,"15min":1.3280000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.49,"5min":3.8650000000000002,"15min":3.8650000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2849999999999999,"15min":1.3620000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.675,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.2869999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.393,"15min":1.47},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.0030000000000001,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.054},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3859999999999999,"15min":1.4750000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.659,"5min":3.9369999999999998,"15min":3.9369999999999998},"last":1.0209999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":10236,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9344,"kb_avail":94361604,"statfs":{"total":96636764160,"available":96626282496,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9568256},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.159,"15min":1.149},"min":{"1min":0.69699999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5,"5min":1.655,"15min":1.655},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.2290000000000001,"15min":1.2350000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.252}]},{"osd":1,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.0569999999999999,"15min":1.081},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.3109999999999999,"5min":1.99,"15min":1.99},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.157,"15min":1.2},"min":{"1min":0.76900000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.6559999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.194}]},{"osd":2,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.1240000000000001,"15min":1.125},"min":{"1min":0.79400000000000004,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.341,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2270000000000001,"15min":1.2589999999999999},"min":{"1min":0.88400000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.468,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.98499999999999999}]},{"osd":3,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.206},"min":{"1min":0.73399999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.776,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1910000000000001,"15min":1.2330000000000001},"min":{"1min":0.78900000000000003,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.423,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.127}]},{"osd":5,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.2250000000000001,"15min":1.2769999999999999},"min":{"1min":0.77600000000000002,"5min":0.747,"15min":0.747},"max":{"1min":1.3700000000000001,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.339},{"interface":"front","average":{"1min":1.202,"5min":1.2410000000000001,"15min":1.2529999999999999},"min":{"1min":0.872,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.837,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":1.1599999999999999}]},{"osd":6,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.24,"15min":1.274},"min":{"1min":0.89600000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5129999999999999,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.3500000000000001,"15min":1.3660000000000001},"min":{"1min":0.878,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.6319999999999999,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.379}]},{"osd":7,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.4239999999999999,"15min":1.4379999999999999},"min":{"1min":0.99199999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.968,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.4330000000000001,"15min":1.4770000000000001},"min":{"1min":1.119,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.9470000000000001,"5min":2.399,"15min":2.399},"last":1.6859999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10108,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361732,"statfs":{"total":96636764160,"available":96626413568,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.86899999999999999,"15min":0.90100000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.468,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":1.018},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.92700000000000005,"15min":0.94099999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.5580000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.141}]},{"osd":1,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.90400000000000003,"15min":0.93200000000000005},"min":{"1min":0.60599999999999998,"5min":0.438,"15min":0.438},"max":{"1min":1.339,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.872},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.0720000000000001,"15min":1.1140000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5960000000000001,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.228}]},{"osd":2,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.111,"15min":1.111},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.135,"5min":1.1719999999999999,"15min":1.2110000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.984,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.304}]},{"osd":4,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.196,"15min":1.282},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.7529999999999999,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.3120000000000001,"15min":1.3799999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.5449999999999999,"5min":2.27,"15min":2.27},"last":1.1839999999999999}]},{"osd":5,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.3620000000000001,"15min":1.425},"min":{"1min":0.93000000000000005,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.657,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.115},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.359,"15min":1.415},"min":{"1min":0.92000000000000004,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.1160000000000001,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.355}]},{"osd":6,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.373,"15min":1.4590000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2,"5min":3.0950000000000002,"15min":3.0950000000000002},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.46,"15min":1.526},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.117,"5min":3.02,"15min":3.02},"last":1.6499999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.5029999999999999,"15min":1.583},"min":{"1min":0.97999999999999998,"5min":0.96799999999999997,"15min":0.96799999999999997},"max":{"1min":2.0230000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.4570000000000001,"15min":1.5669999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.9259999999999999,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.282}]}]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9980,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361860,"statfs":{"total":96636764160,"available":96626544640,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.88500000000000001,"15min":0.93200000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.153,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.95799999999999996,"15min":0.99399999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.4350000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.70199999999999996}]},{"osd":1,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0529999999999999,"15min":1.103},"min":{"1min":0.67100000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.623,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.3109999999999999},{"interface":"front","average":{"1min":1,"5min":1.0229999999999999,"15min":1.0269999999999999},"min":{"1min":0.55200000000000005,"5min":0.505,"15min":0.505},"max":{"1min":1.6399999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.91000000000000003}]},{"osd":3,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":0.998,"15min":1.014},"min":{"1min":0.64500000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.585,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.22,"15min":1.3260000000000001},"min":{"1min":0.88600000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.502,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":1.456}]},{"osd":4,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3640000000000001,"15min":1.458},"min":{"1min":1.034,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.7270000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.3089999999999999,"15min":1.3580000000000001},"min":{"1min":0.99099999999999999,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.669,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.359}]},{"osd":5,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.367,"15min":1.427},"min":{"1min":0.86199999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.76,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.319,"15min":1.343},"min":{"1min":0.96899999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.7450000000000001,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.3859999999999999}]},{"osd":6,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3839999999999999,"15min":1.4199999999999999},"min":{"1min":0.91200000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.722,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.554},{"interface":"front","average":{"1min":1.397,"5min":1.458,"15min":1.5640000000000001},"min":{"1min":0.94699999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.869,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":1.5169999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4239999999999999,"15min":1.47},"min":{"1min":0.92600000000000005,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.9870000000000001,"5min":2.867,"15min":2.867},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4630000000000001,"15min":1.4830000000000001},"min":{"1min":0.93100000000000005,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.8109999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.498}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10044,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361796,"statfs":{"total":96636764160,"available":96626479104,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.86599999999999999,"15min":0.89100000000000001},"min":{"1min":0.54700000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3620000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.016,"15min":1.04},"min":{"1min":0.53700000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4630000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.407}]},{"osd":2,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.95899999999999996,"15min":0.98699999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3859999999999999,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.1200000000000001,"15min":1.1779999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.522,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.93300000000000005}]},{"osd":3,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0820000000000001,"15min":1.103},"min":{"1min":0.56899999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.357,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.2290000000000001},{"interface":"front","average":{"1min":0.88800000000000001,"5min":1.0209999999999999,"15min":1.097},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3120000000000001,"5min":1.972,"15min":1.972},"last":1.0569999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.2070000000000001,"15min":1.228},"min":{"1min":0.82299999999999995,"5min":0.746,"15min":0.746},"max":{"1min":1.556,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2869999999999999,"15min":1.337},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.4930000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":1.022}]},{"osd":5,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.294,"15min":1.329},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.333},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2889999999999999,"15min":1.3360000000000001},"min":{"1min":0.875,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.577,"5min":2.48,"15min":2.48},"last":1.2749999999999999}]},{"osd":6,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3999999999999999,"15min":1.4370000000000001},"min":{"1min":0.95399999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.6160000000000001,"5min":3.149,"15min":3.149},"last":1.369},{"interface":"front","average":{"1min":1.337,"5min":1.4199999999999999,"15min":1.464},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.9530000000000001,"5min":2.169,"15min":2.169},"last":1.1799999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4970000000000001,"15min":1.5529999999999999},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.0939999999999999,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.405,"5min":1.4410000000000001,"15min":1.4690000000000001},"min":{"1min":0.96099999999999997,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.3580000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15424,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":13952,"kb_avail":94356416,"statfs":{"total":96636764160,"available":96620969984,"internally_reserved":0,"allocated":1499136,"data_stored":1014013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14286848},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.89500000000000002,"15min":0.95699999999999996},"min":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0660000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.95899999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.86599999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.94799999999999995,"15min":0.99299999999999999},"min":{"1min":0.63800000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.1759999999999999,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.008,"5min":1.0569999999999999,"15min":1.103},"min":{"1min":0.72599999999999998,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4039999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.458}]},{"osd":3,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.0649999999999999,"15min":1.145},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3300000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.139,"15min":1.1910000000000001},"min":{"1min":0.70399999999999996,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4590000000000001,"5min":1.786,"15min":1.786},"last":0.70699999999999996}]},{"osd":4,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.3939999999999999,"15min":1.4970000000000001},"min":{"1min":0.94799999999999995,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.7,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.552},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.417,"15min":1.4730000000000001},"min":{"1min":1.05,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.7709999999999999,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.075}]},{"osd":5,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.421,"15min":1.498},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":1.9199999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.5029999999999999,"15min":1.6240000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.6830000000000001,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.661}]},{"osd":6,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.425,"15min":1.5149999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.8879999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.141},{"interface":"front","average":{"1min":1.411,"5min":1.4910000000000001,"15min":1.5680000000000001},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.827,"5min":2.048,"15min":2.048},"last":1.702}]},{"osd":7,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.536,"15min":1.6339999999999999},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":2.0070000000000001,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.538,"15min":1.6619999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.5940000000000001,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":1.883}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T08:23:58.236 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-08T08:23:58.237 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T08:23:58.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.409+0000 7f713359e700 1 Processor -- start 2023-11-08T08:23:58.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.409+0000 7f713359e700 1 -- start start 2023-11-08T08:23:58.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.413+0000 7f713359e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f713415c640 0x7f71340d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:58.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.413+0000 7f713359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7134071aa0 0x7f713406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:58.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.413+0000 7f713359e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7134069e00 0x7f713406a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:58.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.413+0000 7f713359e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713406cef0 con 0x7f7134069e00 2023-11-08T08:23:58.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.413+0000 7f713359e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713406a7c0 con 0x7f713415c640 2023-11-08T08:23:58.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.413+0000 7f713359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713406c600 con 0x7f7134071aa0 2023-11-08T08:23:58.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.413+0000 7f7132d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7134069e00 0x7f713406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:58.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.413+0000 7f7132d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7134069e00 0x7f713406a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43832/0 (socket says 172.21.15.40:43832) 2023-11-08T08:23:58.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.413+0000 7f7132d9d700 1 -- 172.21.15.40:0/4011359931 learned_addr learned my addr 172.21.15.40:0/4011359931 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:58.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.413+0000 7f713259c700 1 --2- 172.21.15.40:0/4011359931 >> v2:172.21.15.169:3300/0 conn(0x7f713415c640 0x7f71340d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:58.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.413+0000 7f7131d9b700 1 --2- 172.21.15.40:0/4011359931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7134071aa0 0x7f713406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:58.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.413+0000 7f7132d9d700 1 -- 172.21.15.40:0/4011359931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7134071aa0 msgr2=0x7f713406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:58.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.413+0000 7f7132d9d700 1 --2- 172.21.15.40:0/4011359931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7134071aa0 0x7f713406d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.413+0000 7f7132d9d700 1 -- 172.21.15.40:0/4011359931 >> v2:172.21.15.169:3300/0 conn(0x7f713415c640 msgr2=0x7f71340d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:58.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.413+0000 7f7132d9d700 1 --2- 172.21.15.40:0/4011359931 >> v2:172.21.15.169:3300/0 conn(0x7f713415c640 0x7f71340d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.413+0000 7f7132d9d700 1 -- 172.21.15.40:0/4011359931 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71340722f0 con 0x7f7134069e00 2023-11-08T08:23:58.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.413+0000 7f713259c700 1 --2- 172.21.15.40:0/4011359931 >> v2:172.21.15.169:3300/0 conn(0x7f713415c640 0x7f71340d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:23:58.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.413+0000 7f7132d9d700 1 --2- 172.21.15.40:0/4011359931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7134069e00 0x7f713406a280 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f713406e290 tx=0x7f712c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2c0974c194089c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:58.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.417+0000 7f713159a700 1 -- 172.21.15.40:0/4011359931 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c00e070 con 0x7f7134069e00 2023-11-08T08:23:58.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.417+0000 7f713159a700 1 -- 172.21.15.40:0/4011359931 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712c00b910 con 0x7f7134069e00 2023-11-08T08:23:58.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.417+0000 7f713159a700 1 -- 172.21.15.40:0/4011359931 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c00bcc0 con 0x7f7134069e00 2023-11-08T08:23:58.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.417+0000 7f713359e700 1 -- 172.21.15.40:0/4011359931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7134069e00 msgr2=0x7f713406a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:58.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.417+0000 7f713359e700 1 --2- 172.21.15.40:0/4011359931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7134069e00 0x7f713406a280 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f713406e290 tx=0x7f712c00b330 comp rx=0 tx=0).stop 2023-11-08T08:23:58.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.417+0000 7f713359e700 1 -- 172.21.15.40:0/4011359931 shutdown_connections 2023-11-08T08:23:58.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.417+0000 7f713359e700 1 --2- 172.21.15.40:0/4011359931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7134069e00 0x7f713406a280 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.417+0000 7f713359e700 1 --2- 172.21.15.40:0/4011359931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7134071aa0 0x7f713406d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.417+0000 7f713359e700 1 --2- 172.21.15.40:0/4011359931 >> v2:172.21.15.169:3300/0 conn(0x7f713415c640 0x7f71340d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.417+0000 7f713359e700 1 -- 172.21.15.40:0/4011359931 >> 172.21.15.40:0/4011359931 conn(0x7f71340b6520 msgr2=0x7f7134071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:58.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.417+0000 7f713359e700 1 -- 172.21.15.40:0/4011359931 shutdown_connections 2023-11-08T08:23:58.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.417+0000 7f713359e700 1 -- 172.21.15.40:0/4011359931 wait complete. 2023-11-08T08:23:58.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.421+0000 7f713359e700 1 Processor -- start 2023-11-08T08:23:58.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.421+0000 7f713359e700 1 -- start start 2023-11-08T08:23:58.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.421+0000 7f713359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7134069e00 0x7f71340c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:58.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.421+0000 7f713359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7134071aa0 0x7f71340cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:58.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.421+0000 7f713359e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f713415c640 0x7f71340cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:58.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.421+0000 7f713359e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71340d23f0 con 0x7f7134071aa0 2023-11-08T08:23:58.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.421+0000 7f713359e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713415f930 con 0x7f713415c640 2023-11-08T08:23:58.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.421+0000 7f713359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713415fc10 con 0x7f7134069e00 2023-11-08T08:23:58.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.421+0000 7f7131d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7134071aa0 0x7f71340cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:58.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.421+0000 7f7131d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7134071aa0 0x7f71340cf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43838/0 (socket says 172.21.15.40:43838) 2023-11-08T08:23:58.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.421+0000 7f7131d9b700 1 -- 172.21.15.40:0/3847952935 learned_addr learned my addr 172.21.15.40:0/3847952935 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:58.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.421+0000 7f713259c700 1 --2- 172.21.15.40:0/3847952935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7134069e00 0x7f71340c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:58.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.425+0000 7f7132d9d700 1 --2- 172.21.15.40:0/3847952935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f713415c640 0x7f71340cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:58.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.425+0000 7f7131d9b700 1 -- 172.21.15.40:0/3847952935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7134069e00 msgr2=0x7f71340c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:58.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.425+0000 7f7131d9b700 1 --2- 172.21.15.40:0/3847952935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7134069e00 0x7f71340c8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.425+0000 7f7131d9b700 1 -- 172.21.15.40:0/3847952935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f713415c640 msgr2=0x7f71340cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:58.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.425+0000 7f7131d9b700 1 --2- 172.21.15.40:0/3847952935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f713415c640 0x7f71340cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.425+0000 7f7131d9b700 1 -- 172.21.15.40:0/3847952935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71240094b0 con 0x7f7134071aa0 2023-11-08T08:23:58.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.425+0000 7f7132d9d700 1 --2- 172.21.15.40:0/3847952935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f713415c640 0x7f71340cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:23:58.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.425+0000 7f7131d9b700 1 --2- 172.21.15.40:0/3847952935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7134071aa0 0x7f71340cf5c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f712800e870 tx=0x7f712800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:58.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.425+0000 7f7122ffd700 1 -- 172.21.15.40:0/3847952935 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128009540 con 0x7f7134071aa0 2023-11-08T08:23:58.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.425+0000 7f7122ffd700 1 -- 172.21.15.40:0/3847952935 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712800c640 con 0x7f7134071aa0 2023-11-08T08:23:58.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.425+0000 7f7122ffd700 1 -- 172.21.15.40:0/3847952935 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712800c9b0 con 0x7f7134071aa0 2023-11-08T08:23:58.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.425+0000 7f713359e700 1 -- 172.21.15.40:0/3847952935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712c009b30 con 0x7f7134071aa0 2023-11-08T08:23:58.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.425+0000 7f713359e700 1 -- 172.21.15.40:0/3847952935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7134072010 con 0x7f7134071aa0 2023-11-08T08:23:58.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.425+0000 7f713359e700 1 -- 172.21.15.40:0/3847952935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713415ca10 con 0x7f7134071aa0 2023-11-08T08:23:58.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.429+0000 7f7122ffd700 1 -- 172.21.15.40:0/3847952935 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f7128019030 con 0x7f7134071aa0 2023-11-08T08:23:58.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.429+0000 7f7122ffd700 1 --2- 172.21.15.40:0/3847952935 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7114064ee0 0x7f7114067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:58.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.429+0000 7f7122ffd700 1 -- 172.21.15.40:0/3847952935 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f7128081bb0 con 0x7f7134071aa0 2023-11-08T08:23:58.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.429+0000 7f713259c700 1 --2- 172.21.15.40:0/3847952935 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7114064ee0 0x7f7114067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:58.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.429+0000 7f713259c700 1 --2- 172.21.15.40:0/3847952935 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7114064ee0 0x7f7114067390 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7134071a70 tx=0x7f7124008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:58.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.437+0000 7f7122ffd700 1 -- 172.21.15.40:0/3847952935 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f712804fe40 con 0x7f7134071aa0 2023-11-08T08:23:58.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.605+0000 7f713359e700 1 -- 172.21.15.40:0/3847952935 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f713406c830 con 0x7f7114064ee0 2023-11-08T08:23:58.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.609+0000 7f7122ffd700 1 -- 172.21.15.40:0/3847952935 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70557 (secure 0 0 0) 0x7f713406c830 con 0x7f7114064ee0 2023-11-08T08:23:58.619 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:23:58.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.617+0000 7f713359e700 1 -- 172.21.15.40:0/3847952935 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7114064ee0 msgr2=0x7f7114067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:58.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.617+0000 7f713359e700 1 --2- 172.21.15.40:0/3847952935 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7114064ee0 0x7f7114067390 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7134071a70 tx=0x7f7124008040 comp rx=0 tx=0).stop 2023-11-08T08:23:58.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.617+0000 7f713359e700 1 -- 172.21.15.40:0/3847952935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7134071aa0 msgr2=0x7f71340cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:58.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.617+0000 7f713359e700 1 --2- 172.21.15.40:0/3847952935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7134071aa0 0x7f71340cf5c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f712800e870 tx=0x7f712800ec30 comp rx=0 tx=0).stop 2023-11-08T08:23:58.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.617+0000 7f713359e700 1 -- 172.21.15.40:0/3847952935 shutdown_connections 2023-11-08T08:23:58.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.617+0000 7f713359e700 1 --2- 172.21.15.40:0/3847952935 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f7114064ee0 0x7f7114067390 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.617+0000 7f713359e700 1 --2- 172.21.15.40:0/3847952935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f713415c640 0x7f71340cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.617+0000 7f713359e700 1 --2- 172.21.15.40:0/3847952935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7134071aa0 0x7f71340cf5c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.617+0000 7f713359e700 1 --2- 172.21.15.40:0/3847952935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7134069e00 0x7f71340c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.617+0000 7f713359e700 1 -- 172.21.15.40:0/3847952935 >> 172.21.15.40:0/3847952935 conn(0x7f71340b6520 msgr2=0x7f71340d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.617+0000 7f713359e700 1 -- 172.21.15.40:0/3847952935 shutdown_connections 2023-11-08T08:23:58.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.617+0000 7f713359e700 1 -- 172.21.15.40:0/3847952935 wait complete. 2023-11-08T08:23:58.626 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-11-08T08:23:58.646 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2023-11-08T08:23:57.992133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":90984,"kb_used_data":8232,"kb_used_omap":0,"kb_used_meta":82688,"kb_avail":754883736,"statfs":{"total":773094113280,"available":773000945664,"internally_reserved":0,"allocated":8429568,"data_stored":4569896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":84672512},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12286,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-8,"num_read_kb":-7,"num_write":-80,"num_write_kb":-39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.265165"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:53.995671+0000","last_change":"2023-11-08T08:18:58.771310+0000","last_active":"2023-11-08T08:23:53.995671+0000","last_peered":"2023-11-08T08:23:53.995671+0000","last_clean":"2023-11-08T08:23:53.995671+0000","last_became_active":"2023-11-08T08:18:58.770891+0000","last_became_peered":"2023-11-08T08:18:58.770891+0000","last_unstale":"2023-11-08T08:23:53.995671+0000","last_undegraded":"2023-11-08T08:23:53.995671+0000","last_fullsized":"2023-11-08T08:23:53.995671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:54.762987+0000","last_change":"2023-11-08T08:18:58.770483+0000","last_active":"2023-11-08T08:23:54.762987+0000","last_peered":"2023-11-08T08:23:54.762987+0000","last_clean":"2023-11-08T08:23:54.762987+0000","last_became_active":"2023-11-08T08:18:58.770040+0000","last_became_peered":"2023-11-08T08:18:58.770040+0000","last_unstale":"2023-11-08T08:23:54.762987+0000","last_undegraded":"2023-11-08T08:23:54.762987+0000","last_fullsized":"2023-11-08T08:23:54.762987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":59,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:53.996243+0000","last_change":"2023-11-08T08:18:58.766848+0000","last_active":"2023-11-08T08:23:53.996243+0000","last_peered":"2023-11-08T08:23:53.996243+0000","last_clean":"2023-11-08T08:23:53.996243+0000","last_became_active":"2023-11-08T08:18:58.766378+0000","last_became_peered":"2023-11-08T08:18:58.766378+0000","last_unstale":"2023-11-08T08:23:53.996243+0000","last_undegraded":"2023-11-08T08:23:53.996243+0000","last_fullsized":"2023-11-08T08:23:53.996243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:54.764100+0000","last_change":"2023-11-08T08:18:58.769764+0000","last_active":"2023-11-08T08:23:54.764100+0000","last_peered":"2023-11-08T08:23:54.764100+0000","last_clean":"2023-11-08T08:23:54.764100+0000","last_became_active":"2023-11-08T08:18:58.769357+0000","last_became_peered":"2023-11-08T08:18:58.769357+0000","last_unstale":"2023-11-08T08:23:54.764100+0000","last_undegraded":"2023-11-08T08:23:54.764100+0000","last_fullsized":"2023-11-08T08:23:54.764100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:13.748094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":58,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:54.761825+0000","last_change":"2023-11-08T08:18:58.771460+0000","last_active":"2023-11-08T08:23:54.761825+0000","last_peered":"2023-11-08T08:23:54.761825+0000","last_clean":"2023-11-08T08:23:54.761825+0000","last_became_active":"2023-11-08T08:18:58.770648+0000","last_became_peered":"2023-11-08T08:18:58.770648+0000","last_unstale":"2023-11-08T08:23:54.761825+0000","last_undegraded":"2023-11-08T08:23:54.761825+0000","last_fullsized":"2023-11-08T08:23:54.761825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":60,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:54.763825+0000","last_change":"2023-11-08T08:18:58.973732+0000","last_active":"2023-11-08T08:23:54.763825+0000","last_peered":"2023-11-08T08:23:54.763825+0000","last_clean":"2023-11-08T08:23:54.763825+0000","last_became_active":"2023-11-08T08:18:58.973007+0000","last_became_peered":"2023-11-08T08:18:58.973007+0000","last_unstale":"2023-11-08T08:23:54.763825+0000","last_undegraded":"2023-11-08T08:23:54.763825+0000","last_fullsized":"2023-11-08T08:23:54.763825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":59,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:53.993431+0000","last_change":"2023-11-08T08:18:58.770136+0000","last_active":"2023-11-08T08:23:53.993431+0000","last_peered":"2023-11-08T08:23:53.993431+0000","last_clean":"2023-11-08T08:23:53.993431+0000","last_became_active":"2023-11-08T08:18:58.769797+0000","last_became_peered":"2023-11-08T08:18:58.769797+0000","last_unstale":"2023-11-08T08:23:53.993431+0000","last_undegraded":"2023-11-08T08:23:53.993431+0000","last_fullsized":"2023-11-08T08:23:53.993431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:29:12.107128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":58,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:54.762277+0000","last_change":"2023-11-08T08:18:58.777140+0000","last_active":"2023-11-08T08:23:54.762277+0000","last_peered":"2023-11-08T08:23:54.762277+0000","last_clean":"2023-11-08T08:23:54.762277+0000","last_became_active":"2023-11-08T08:18:58.776834+0000","last_became_peered":"2023-11-08T08:18:58.776834+0000","last_unstale":"2023-11-08T08:23:54.762277+0000","last_undegraded":"2023-11-08T08:23:54.762277+0000","last_fullsized":"2023-11-08T08:23:54.762277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":212,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:53.993559+0000","last_change":"2023-11-08T08:18:53.780089+0000","last_active":"2023-11-08T08:23:53.993559+0000","last_peered":"2023-11-08T08:23:53.993559+0000","last_clean":"2023-11-08T08:23:53.993559+0000","last_became_active":"2023-11-08T08:18:53.779533+0000","last_became_peered":"2023-11-08T08:18:53.779533+0000","last_unstale":"2023-11-08T08:23:53.993559+0000","last_undegraded":"2023-11-08T08:23:53.993559+0000","last_fullsized":"2023-11-08T08:23:53.993559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_clean_scrub_stamp":"2023-11-08T08:18:52.744693+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:58:24.877070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15296,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94356544,"statfs":{"total":96636764160,"available":96621101056,"internally_reserved":0,"allocated":1499136,"data_stored":1014013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.121,"15min":1.0409999999999999},"min":{"1min":0.84899999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6140000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.278},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.21,"15min":1.1759999999999999},"min":{"1min":0.80300000000000005,"5min":0.746,"15min":0.746},"max":{"1min":1.5640000000000001,"5min":1.667,"15min":1.667},"last":1.1399999999999999}]},{"osd":1,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.1899999999999999,"15min":1.194},"min":{"1min":0.78400000000000003,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.599,"5min":1.972,"15min":1.972},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.264,"15min":1.306},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.425,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.0149999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.278,"15min":1.387},"min":{"1min":0.96499999999999997,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.444,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":1.2},{"interface":"front","average":{"1min":1.077,"5min":1.246,"15min":1.3759999999999999},"min":{"1min":0.73199999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.446,"5min":4.9089999999999998,"15min":4.9089999999999998},"last":0.95499999999999996}]},{"osd":3,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2070000000000001,"15min":1.246},"min":{"1min":0.89900000000000002,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.724,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.252,"5min":1.2789999999999999,"15min":1.3029999999999999},"min":{"1min":0.92100000000000004,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.6850000000000001,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":1.325}]},{"osd":4,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2310000000000001,"15min":1.262},"min":{"1min":0.78900000000000003,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.3},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.375,"15min":1.419},"min":{"1min":0.97099999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.8180000000000001,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":0.91200000000000003}]},{"osd":5,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.292,"15min":1.3620000000000001},"min":{"1min":0.97299999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.756,"5min":2.73,"15min":2.73},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.294,"15min":1.329},"min":{"1min":0.80900000000000005,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.6259999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.0740000000000001}]},{"osd":6,"last update":"Wed Nov 8 08:22:58 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3999999999999999,"15min":1.4410000000000001},"min":{"1min":0.90000000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.724,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.3440000000000001,"15min":1.4219999999999999},"min":{"1min":0.83799999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.8799999999999999,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.1040000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9980,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361860,"statfs":{"total":96636764160,"available":96626544640,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.0800000000000001,"15min":1.159},"min":{"1min":0.76600000000000001,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5860000000000001,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.04},{"interface":"front","average":{"1min":1.077,"5min":1.079,"15min":1.0960000000000001},"min":{"1min":0.71099999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.556,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.96999999999999997}]},{"osd":1,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.224,"15min":1.1990000000000001},"min":{"1min":0.89000000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.222},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2190000000000001,"15min":1.21},"min":{"1min":0.86799999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.603,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.3520000000000001}]},{"osd":2,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.173,"15min":1.1830000000000001},"min":{"1min":0.76500000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.216,"15min":1.2769999999999999},"min":{"1min":0.81599999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5229999999999999,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.4239999999999999}]},{"osd":3,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2549999999999999,"15min":1.2809999999999999},"min":{"1min":0.95199999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.474,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.319},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3759999999999999,"15min":1.4019999999999999},"min":{"1min":0.96699999999999997,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.966,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":1.5009999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.208,"15min":1.2509999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.669,"5min":2.02,"15min":2.02},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.196,"15min":1.1919999999999999},"min":{"1min":0.79300000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.625,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.1060000000000001}]},{"osd":5,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.254,"15min":1.2370000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":1.385},{"interface":"front","average":{"1min":1.276,"5min":1.3400000000000001,"15min":1.363},"min":{"1min":0.94699999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.7350000000000001,"5min":1.794,"15min":1.794},"last":1.2869999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3580000000000001,"15min":1.3779999999999999},"min":{"1min":1.0429999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.8009999999999999,"5min":3.133,"15min":3.133},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.444,"15min":1.4390000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.1960000000000002,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.544}]}]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9916,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361924,"statfs":{"total":96636764160,"available":96626610176,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.141,"15min":1.135},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.228,"15min":1.238},"min":{"1min":0.78600000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":1.171}]},{"osd":1,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.107,"15min":1.133},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.145,"15min":1.1850000000000001},"min":{"1min":0.85999999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":1.216}]},{"osd":2,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.171,"15min":1.1910000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5189999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.129},{"interface":"front","average":{"1min":1.157,"5min":1.1599999999999999,"15min":1.212},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.96399999999999997}]},{"osd":3,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.304,"15min":1.3360000000000001},"min":{"1min":0.83799999999999997,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.2109999999999999,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.2649999999999999,"15min":1.3180000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.9219999999999999,"5min":3.7999999999999998,"15min":3.7999999999999998},"last":1.1020000000000001}]},{"osd":4,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1359999999999999,"15min":1.194},"min":{"1min":0.76500000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.21,"15min":1.3129999999999999},"min":{"1min":0.56599999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.7310000000000001,"5min":3.073,"15min":3.073},"last":1.2490000000000001}]},{"osd":6,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2370000000000001,"15min":1.3280000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.49,"5min":3.8650000000000002,"15min":3.8650000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2849999999999999,"15min":1.3620000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.675,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.2869999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.393,"15min":1.47},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.0030000000000001,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.054},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3859999999999999,"15min":1.4750000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.659,"5min":3.9369999999999998,"15min":3.9369999999999998},"last":1.0209999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":10236,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9344,"kb_avail":94361604,"statfs":{"total":96636764160,"available":96626282496,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9568256},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.159,"15min":1.149},"min":{"1min":0.69699999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5,"5min":1.655,"15min":1.655},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.2290000000000001,"15min":1.2350000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.252}]},{"osd":1,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.0569999999999999,"15min":1.081},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.3109999999999999,"5min":1.99,"15min":1.99},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.157,"15min":1.2},"min":{"1min":0.76900000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.6559999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.194}]},{"osd":2,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.1240000000000001,"15min":1.125},"min":{"1min":0.79400000000000004,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.341,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2270000000000001,"15min":1.2589999999999999},"min":{"1min":0.88400000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.468,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.98499999999999999}]},{"osd":3,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.206},"min":{"1min":0.73399999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.776,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1910000000000001,"15min":1.2330000000000001},"min":{"1min":0.78900000000000003,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.423,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.127}]},{"osd":5,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.2250000000000001,"15min":1.2769999999999999},"min":{"1min":0.77600000000000002,"5min":0.747,"15min":0.747},"max":{"1min":1.3700000000000001,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.339},{"interface":"front","average":{"1min":1.202,"5min":1.2410000000000001,"15min":1.2529999999999999},"min":{"1min":0.872,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.837,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":1.1599999999999999}]},{"osd":6,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.24,"15min":1.274},"min":{"1min":0.89600000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5129999999999999,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.3500000000000001,"15min":1.3660000000000001},"min":{"1min":0.878,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.6319999999999999,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.379}]},{"osd":7,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.4239999999999999,"15min":1.4379999999999999},"min":{"1min":0.99199999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.968,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.4330000000000001,"15min":1.4770000000000001},"min":{"1min":1.119,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.9470000000000001,"5min":2.399,"15min":2.399},"last":1.6859999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10108,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361732,"statfs":{"total":96636764160,"available":96626413568,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.86899999999999999,"15min":0.90100000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.468,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":1.018},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.92700000000000005,"15min":0.94099999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.5580000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.141}]},{"osd":1,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.90400000000000003,"15min":0.93200000000000005},"min":{"1min":0.60599999999999998,"5min":0.438,"15min":0.438},"max":{"1min":1.339,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.872},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.0720000000000001,"15min":1.1140000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5960000000000001,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.228}]},{"osd":2,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.111,"15min":1.111},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.135,"5min":1.1719999999999999,"15min":1.2110000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.984,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.304}]},{"osd":4,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.196,"15min":1.282},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.7529999999999999,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.3120000000000001,"15min":1.3799999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.5449999999999999,"5min":2.27,"15min":2.27},"last":1.1839999999999999}]},{"osd":5,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.3620000000000001,"15min":1.425},"min":{"1min":0.93000000000000005,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.657,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.115},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.359,"15min":1.415},"min":{"1min":0.92000000000000004,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.1160000000000001,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.355}]},{"osd":6,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.373,"15min":1.4590000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2,"5min":3.0950000000000002,"15min":3.0950000000000002},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.46,"15min":1.526},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.117,"5min":3.02,"15min":3.02},"last":1.6499999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.5029999999999999,"15min":1.583},"min":{"1min":0.97999999999999998,"5min":0.96799999999999997,"15min":0.96799999999999997},"max":{"1min":2.0230000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.4570000000000001,"15min":1.5669999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.9259999999999999,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.282}]}]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9980,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361860,"statfs":{"total":96636764160,"available":96626544640,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.88500000000000001,"15min":0.93200000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.153,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.95799999999999996,"15min":0.99399999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.4350000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.70199999999999996}]},{"osd":1,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0529999999999999,"15min":1.103},"min":{"1min":0.67100000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.623,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.3109999999999999},{"interface":"front","average":{"1min":1,"5min":1.0229999999999999,"15min":1.0269999999999999},"min":{"1min":0.55200000000000005,"5min":0.505,"15min":0.505},"max":{"1min":1.6399999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.91000000000000003}]},{"osd":3,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":0.998,"15min":1.014},"min":{"1min":0.64500000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.585,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.22,"15min":1.3260000000000001},"min":{"1min":0.88600000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.502,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":1.456}]},{"osd":4,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3640000000000001,"15min":1.458},"min":{"1min":1.034,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.7270000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.3089999999999999,"15min":1.3580000000000001},"min":{"1min":0.99099999999999999,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.669,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.359}]},{"osd":5,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.367,"15min":1.427},"min":{"1min":0.86199999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.76,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.319,"15min":1.343},"min":{"1min":0.96899999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.7450000000000001,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.3859999999999999}]},{"osd":6,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3839999999999999,"15min":1.4199999999999999},"min":{"1min":0.91200000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.722,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.554},{"interface":"front","average":{"1min":1.397,"5min":1.458,"15min":1.5640000000000001},"min":{"1min":0.94699999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.869,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":1.5169999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4239999999999999,"15min":1.47},"min":{"1min":0.92600000000000005,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.9870000000000001,"5min":2.867,"15min":2.867},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4630000000000001,"15min":1.4830000000000001},"min":{"1min":0.93100000000000005,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.8109999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.498}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10044,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361796,"statfs":{"total":96636764160,"available":96626479104,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.86599999999999999,"15min":0.89100000000000001},"min":{"1min":0.54700000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3620000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.016,"15min":1.04},"min":{"1min":0.53700000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4630000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.407}]},{"osd":2,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.95899999999999996,"15min":0.98699999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3859999999999999,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.1200000000000001,"15min":1.1779999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.522,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.93300000000000005}]},{"osd":3,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0820000000000001,"15min":1.103},"min":{"1min":0.56899999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.357,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.2290000000000001},{"interface":"front","average":{"1min":0.88800000000000001,"5min":1.0209999999999999,"15min":1.097},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3120000000000001,"5min":1.972,"15min":1.972},"last":1.0569999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.2070000000000001,"15min":1.228},"min":{"1min":0.82299999999999995,"5min":0.746,"15min":0.746},"max":{"1min":1.556,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2869999999999999,"15min":1.337},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.4930000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":1.022}]},{"osd":5,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.294,"15min":1.329},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.333},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2889999999999999,"15min":1.3360000000000001},"min":{"1min":0.875,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.577,"5min":2.48,"15min":2.48},"last":1.2749999999999999}]},{"osd":6,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3999999999999999,"15min":1.4370000000000001},"min":{"1min":0.95399999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.6160000000000001,"5min":3.149,"15min":3.149},"last":1.369},{"interface":"front","average":{"1min":1.337,"5min":1.4199999999999999,"15min":1.464},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.9530000000000001,"5min":2.169,"15min":2.169},"last":1.1799999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4970000000000001,"15min":1.5529999999999999},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.0939999999999999,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.405,"5min":1.4410000000000001,"15min":1.4690000000000001},"min":{"1min":0.96099999999999997,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.3580000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15424,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":13952,"kb_avail":94356416,"statfs":{"total":96636764160,"available":96620969984,"internally_reserved":0,"allocated":1499136,"data_stored":1014013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14286848},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.89500000000000002,"15min":0.95699999999999996},"min":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0660000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.95899999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.86599999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.94799999999999995,"15min":0.99299999999999999},"min":{"1min":0.63800000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.1759999999999999,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.008,"5min":1.0569999999999999,"15min":1.103},"min":{"1min":0.72599999999999998,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4039999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.458}]},{"osd":3,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.0649999999999999,"15min":1.145},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3300000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.139,"15min":1.1910000000000001},"min":{"1min":0.70399999999999996,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4590000000000001,"5min":1.786,"15min":1.786},"last":0.70699999999999996}]},{"osd":4,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.3939999999999999,"15min":1.4970000000000001},"min":{"1min":0.94799999999999995,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":1.7,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.552},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.417,"15min":1.4730000000000001},"min":{"1min":1.05,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.7709999999999999,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.075}]},{"osd":5,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.421,"15min":1.498},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":1.9199999999999999,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.5029999999999999,"15min":1.6240000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.6830000000000001,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.661}]},{"osd":6,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.425,"15min":1.5149999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.8879999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.141},{"interface":"front","average":{"1min":1.411,"5min":1.4910000000000001,"15min":1.5680000000000001},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.827,"5min":2.048,"15min":2.048},"last":1.702}]},{"osd":7,"last update":"Wed Nov 8 08:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.536,"15min":1.6339999999999999},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":2.0070000000000001,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.538,"15min":1.6619999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.5940000000000001,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":1.883}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T08:23:58.649 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T08:23:58.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.829+0000 7f8c499e6700 1 Processor -- start 2023-11-08T08:23:58.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.829+0000 7f8c499e6700 1 -- start start 2023-11-08T08:23:58.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.829+0000 7f8c499e6700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c441604b0 0x7f8c440b3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:58.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.829+0000 7f8c499e6700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8c440b2240 0x7f8c440cab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:58.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.829+0000 7f8c499e6700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c440ce430 con 0x7f8c44160880 2023-11-08T08:23:58.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.829+0000 7f8c499e6700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c440cbcd0 con 0x7f8c440b2240 2023-11-08T08:23:58.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.829+0000 7f8c499e6700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c440cb710 con 0x7f8c441604b0 2023-11-08T08:23:58.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.829+0000 7f8c427fc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8c44160880 0x7f8c440b1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57864/0 (socket says 172.21.15.40:57864) 2023-11-08T08:23:58.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.829+0000 7f8c427fc700 1 -- 172.21.15.40:0/3005681432 learned_addr learned my addr 172.21.15.40:0/3005681432 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:23:58.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.829+0000 7f8c43fff700 1 --2- 172.21.15.40:0/3005681432 >> v2:172.21.15.40:3300/0 conn(0x7f8c440b2240 0x7f8c440cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:58.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.829+0000 7f8c42ffd700 1 --2- 172.21.15.40:0/3005681432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c441604b0 0x7f8c440b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:58.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.833+0000 7f8c41ffb700 1 -- 172.21.15.40:0/3005681432 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2518275123 0 0) 0x7f8c38001110 con 0x7f8c44160880 2023-11-08T08:23:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.833+0000 7f8c41ffb700 1 -- 172.21.15.40:0/3005681432 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c2c0035d0 con 0x7f8c44160880 2023-11-08T08:23:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.833+0000 7f8c43fff700 1 -- 172.21.15.40:0/3005681432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c441604b0 msgr2=0x7f8c440b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:23:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.833+0000 7f8c43fff700 1 --2- 172.21.15.40:0/3005681432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c441604b0 0x7f8c440b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.833+0000 7f8c43fff700 1 -- 172.21.15.40:0/3005681432 >> v1:172.21.15.40:6789/0 conn(0x7f8c44160880 legacy=0x7f8c440b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.833+0000 7f8c43fff700 1 -- 172.21.15.40:0/3005681432 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c440cb9f0 con 0x7f8c440b2240 2023-11-08T08:23:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.833+0000 7f8c42ffd700 1 --2- 172.21.15.40:0/3005681432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c441604b0 0x7f8c440b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:23:58.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.833+0000 7f8c43fff700 1 --2- 172.21.15.40:0/3005681432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c440b2240 0x7f8c440cab30 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009d90 tx=0x7f8c3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d16008daaa53289 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:58.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.833+0000 7f8c41ffb700 1 -- 172.21.15.40:0/3005681432 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c00e070 con 0x7f8c440b2240 2023-11-08T08:23:58.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.833+0000 7f8c41ffb700 1 -- 172.21.15.40:0/3005681432 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3c00bb20 con 0x7f8c440b2240 2023-11-08T08:23:58.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.833+0000 7f8c41ffb700 1 -- 172.21.15.40:0/3005681432 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c00be90 con 0x7f8c440b2240 2023-11-08T08:23:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.833+0000 7f8c499e6700 1 -- 172.21.15.40:0/3005681432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c440b2240 msgr2=0x7f8c440cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.833+0000 7f8c499e6700 1 --2- 172.21.15.40:0/3005681432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c440b2240 0x7f8c440cab30 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009d90 tx=0x7f8c3c00b330 comp rx=0 tx=0).stop 2023-11-08T08:23:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.833+0000 7f8c499e6700 1 -- 172.21.15.40:0/3005681432 shutdown_connections 2023-11-08T08:23:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.833+0000 7f8c499e6700 1 --2- 172.21.15.40:0/3005681432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c440b2240 0x7f8c440cab30 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.833+0000 7f8c499e6700 1 --2- 172.21.15.40:0/3005681432 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c441604b0 0x7f8c440b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.833+0000 7f8c499e6700 1 -- 172.21.15.40:0/3005681432 >> 172.21.15.40:0/3005681432 conn(0x7f8c440ccb30 msgr2=0x7f8c440ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:58.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.833+0000 7f8c499e6700 1 -- 172.21.15.40:0/3005681432 shutdown_connections 2023-11-08T08:23:58.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.833+0000 7f8c499e6700 1 -- 172.21.15.40:0/3005681432 wait complete. 2023-11-08T08:23:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.837+0000 7f8c499e6700 1 Processor -- start 2023-11-08T08:23:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.837+0000 7f8c499e6700 1 -- start start 2023-11-08T08:23:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c499e6700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c440b2240 0x7f8c440c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c499e6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c441604b0 0x7f8c44069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c499e6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c44160880 0x7f8c44159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:58.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c499e6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4406a2f0 con 0x7f8c441604b0 2023-11-08T08:23:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c499e6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c440c4ab0 con 0x7f8c44160880 2023-11-08T08:23:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c499e6700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c440c4d90 con 0x7f8c440b2240 2023-11-08T08:23:58.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c427fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c441604b0 0x7f8c44069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:58.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c427fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c441604b0 0x7f8c44069bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43854/0 (socket says 172.21.15.40:43854) 2023-11-08T08:23:58.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c427fc700 1 -- 172.21.15.40:0/3334508681 learned_addr learned my addr 172.21.15.40:0/3334508681 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:23:58.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c42ffd700 1 --2- 172.21.15.40:0/3334508681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c440b2240 0x7f8c440c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:58.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c43fff700 1 --2- 172.21.15.40:0/3334508681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c44160880 0x7f8c44159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:58.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c427fc700 1 -- 172.21.15.40:0/3334508681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c440b2240 msgr2=0x7f8c440c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:58.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c427fc700 1 --2- 172.21.15.40:0/3334508681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c440b2240 0x7f8c440c65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c427fc700 1 -- 172.21.15.40:0/3334508681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c44160880 msgr2=0x7f8c44159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:58.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c427fc700 1 --2- 172.21.15.40:0/3334508681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c44160880 0x7f8c44159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:58.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c427fc700 1 -- 172.21.15.40:0/3334508681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c009a70 con 0x7f8c441604b0 2023-11-08T08:23:58.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c427fc700 1 --2- 172.21.15.40:0/3334508681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c441604b0 0x7f8c44069bf0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800c4d0 tx=0x7f8c3800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:58.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c2bfff700 1 -- 172.21.15.40:0/3334508681 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38005d90 con 0x7f8c441604b0 2023-11-08T08:23:58.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c2bfff700 1 -- 172.21.15.40:0/3334508681 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c380070d0 con 0x7f8c441604b0 2023-11-08T08:23:58.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c2bfff700 1 -- 172.21.15.40:0/3334508681 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38007480 con 0x7f8c441604b0 2023-11-08T08:23:58.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c499e6700 1 -- 172.21.15.40:0/3334508681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c440cef10 con 0x7f8c441604b0 2023-11-08T08:23:58.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c499e6700 1 -- 172.21.15.40:0/3334508681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c440cec30 con 0x7f8c441604b0 2023-11-08T08:23:58.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.841+0000 7f8c499e6700 1 -- 172.21.15.40:0/3334508681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c440b3bb0 con 0x7f8c441604b0 2023-11-08T08:23:58.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.845+0000 7f8c2bfff700 1 -- 172.21.15.40:0/3334508681 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f8c38019030 con 0x7f8c441604b0 2023-11-08T08:23:58.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.845+0000 7f8c2bfff700 1 --2- 172.21.15.40:0/3334508681 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8c30064f10 0x7f8c300673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:23:58.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.845+0000 7f8c2bfff700 1 -- 172.21.15.40:0/3334508681 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f8c38080f80 con 0x7f8c441604b0 2023-11-08T08:23:58.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.849+0000 7f8c42ffd700 1 --2- 172.21.15.40:0/3334508681 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8c30064f10 0x7f8c300673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:23:58.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.849+0000 7f8c42ffd700 1 --2- 172.21.15.40:0/3334508681 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8c30064f10 0x7f8c300673c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8c340099f0 tx=0x7f8c34008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:23:58.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:58.853+0000 7f8c2bfff700 1 -- 172.21.15.40:0/3334508681 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f8c3804f210 con 0x7f8c441604b0 2023-11-08T08:23:59.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:59.029+0000 7f8c499e6700 1 -- 172.21.15.40:0/3334508681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c440ce430 con 0x7f8c441604b0 2023-11-08T08:23:59.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:59.029+0000 7f8c2bfff700 1 -- 172.21.15.40:0/3334508681 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+14506 (secure 0 0 0) 0x7f8c440ce430 con 0x7f8c441604b0 2023-11-08T08:23:59.037 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:23:59.038 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":74,"fsid":"54ed45f4-d5b0-41c3-8c3b-b5a18b844f25","created":"2023-11-08T08:18:36.974334+0000","modified":"2023-11-08T08:23:54.985909+0000","last_up_change":"2023-11-08T08:18:50.370752+0000","last_in_change":"2023-11-08T08:18:40.187770+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T08:18:51.966993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-08T08:18:56.832911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99dd46c2-e710-426f-9a2b-481865dbe5f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":35482},{"type":"v1","addr":"172.21.15.40:6809","nonce":35482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":35482},{"type":"v1","addr":"172.21.15.40:6811","nonce":35482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":35482},{"type":"v1","addr":"172.21.15.40:6815","nonce":35482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":35482},{"type":"v1","addr":"172.21.15.40:6813","nonce":35482}]},"public_addr":"172.21.15.40:6809/35482","cluster_addr":"172.21.15.40:6811/35482","heartbeat_back_addr":"172.21.15.40:6815/35482","heartbeat_front_addr":"172.21.15.40:6813/35482","state":["exists","up"]},{"osd":1,"uuid":"62815141-84f4-42c1-b665-f88c1f0561a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":35480},{"type":"v1","addr":"172.21.15.40:6801","nonce":35480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":35480},{"type":"v1","addr":"172.21.15.40:6803","nonce":35480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":35480},{"type":"v1","addr":"172.21.15.40:6807","nonce":35480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":35480},{"type":"v1","addr":"172.21.15.40:6805","nonce":35480}]},"public_addr":"172.21.15.40:6801/35480","cluster_addr":"172.21.15.40:6803/35480","heartbeat_back_addr":"172.21.15.40:6807/35480","heartbeat_front_addr":"172.21.15.40:6805/35480","state":["exists","up"]},{"osd":2,"uuid":"70af5f1b-66ab-4534-bd9b-d786aa50fb22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":35486},{"type":"v1","addr":"172.21.15.40:6825","nonce":35486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":35486},{"type":"v1","addr":"172.21.15.40:6827","nonce":35486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":35486},{"type":"v1","addr":"172.21.15.40:6831","nonce":35486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":35486},{"type":"v1","addr":"172.21.15.40:6829","nonce":35486}]},"public_addr":"172.21.15.40:6825/35486","cluster_addr":"172.21.15.40:6827/35486","heartbeat_back_addr":"172.21.15.40:6831/35486","heartbeat_front_addr":"172.21.15.40:6829/35486","state":["exists","up"]},{"osd":3,"uuid":"aed310e9-dc5d-4c80-a1fa-da02ebfc8945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":35484},{"type":"v1","addr":"172.21.15.40:6817","nonce":35484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":35484},{"type":"v1","addr":"172.21.15.40:6819","nonce":35484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":35484},{"type":"v1","addr":"172.21.15.40:6823","nonce":35484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":35484},{"type":"v1","addr":"172.21.15.40:6821","nonce":35484}]},"public_addr":"172.21.15.40:6817/35484","cluster_addr":"172.21.15.40:6819/35484","heartbeat_back_addr":"172.21.15.40:6823/35484","heartbeat_front_addr":"172.21.15.40:6821/35484","state":["exists","up"]},{"osd":4,"uuid":"03ad6509-7f6f-4c1e-8580-dc3371e85213","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6811","nonce":35790},{"type":"v1","addr":"172.21.15.169:6814","nonce":35790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6817","nonce":35790},{"type":"v1","addr":"172.21.15.169:6819","nonce":35790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":35790},{"type":"v1","addr":"172.21.15.169:6831","nonce":35790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":35790},{"type":"v1","addr":"172.21.15.169:6825","nonce":35790}]},"public_addr":"172.21.15.169:6814/35790","cluster_addr":"172.21.15.169:6819/35790","heartbeat_back_addr":"172.21.15.169:6831/35790","heartbeat_front_addr":"172.21.15.169:6825/35790","state":["exists","up"]},{"osd":5,"uuid":"3f69a76b-e285-4741-99ef-5be9617754f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6809","nonce":35793},{"type":"v1","addr":"172.21.15.169:6812","nonce":35793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6815","nonce":35793},{"type":"v1","addr":"172.21.15.169:6818","nonce":35793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":35793},{"type":"v1","addr":"172.21.15.169:6829","nonce":35793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":35793},{"type":"v1","addr":"172.21.15.169:6823","nonce":35793}]},"public_addr":"172.21.15.169:6812/35793","cluster_addr":"172.21.15.169:6818/35793","heartbeat_back_addr":"172.21.15.169:6829/35793","heartbeat_front_addr":"172.21.15.169:6823/35793","state":["exists","up"]},{"osd":6,"uuid":"f11bd326-3279-413b-84db-ceb549995008","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":35791},{"type":"v1","addr":"172.21.15.169:6801","nonce":35791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":35791},{"type":"v1","addr":"172.21.15.169:6803","nonce":35791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":35791},{"type":"v1","addr":"172.21.15.169:6807","nonce":35791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":35791},{"type":"v1","addr":"172.21.15.169:6805","nonce":35791}]},"public_addr":"172.21.15.169:6801/35791","cluster_addr":"172.21.15.169:6803/35791","heartbeat_back_addr":"172.21.15.169:6807/35791","heartbeat_front_addr":"172.21.15.169:6805/35791","state":["exists","up"]},{"osd":7,"uuid":"4d36f429-63d5-4d0c-a6b1-0b43f54d7225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":35792},{"type":"v1","addr":"172.21.15.169:6810","nonce":35792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6813","nonce":35792},{"type":"v1","addr":"172.21.15.169:6816","nonce":35792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6827","nonce":35792},{"type":"v1","addr":"172.21.15.169:6830","nonce":35792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":35792},{"type":"v1","addr":"172.21.15.169:6824","nonce":35792}]},"public_addr":"172.21.15.169:6810/35792","cluster_addr":"172.21.15.169:6816/35792","heartbeat_back_addr":"172.21.15.169:6830/35792","heartbeat_front_addr":"172.21.15.169:6824/35792","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.986154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.154777+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.926229+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.119065+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.173381+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:45.940687+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.183493+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T08:18:46.004394+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/3395630689":"2023-11-09T08:23:51.717341+0000","172.21.15.40:6834/3395630689":"2023-11-09T08:23:51.717341+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T08:23:59.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:59.033+0000 7f8c499e6700 1 -- 172.21.15.40:0/3334508681 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8c30064f10 msgr2=0x7f8c300673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:59.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:59.033+0000 7f8c499e6700 1 --2- 172.21.15.40:0/3334508681 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8c30064f10 0x7f8c300673c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8c340099f0 tx=0x7f8c34008040 comp rx=0 tx=0).stop 2023-11-08T08:23:59.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:59.033+0000 7f8c499e6700 1 -- 172.21.15.40:0/3334508681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c441604b0 msgr2=0x7f8c44069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:23:59.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:59.033+0000 7f8c499e6700 1 --2- 172.21.15.40:0/3334508681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c441604b0 0x7f8c44069bf0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800c4d0 tx=0x7f8c3800c7e0 comp rx=0 tx=0).stop 2023-11-08T08:23:59.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:59.033+0000 7f8c499e6700 1 -- 172.21.15.40:0/3334508681 shutdown_connections 2023-11-08T08:23:59.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:59.033+0000 7f8c499e6700 1 --2- 172.21.15.40:0/3334508681 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f8c30064f10 0x7f8c300673c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:59.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:59.033+0000 7f8c499e6700 1 --2- 172.21.15.40:0/3334508681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c44160880 0x7f8c44159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:59.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:59.033+0000 7f8c499e6700 1 --2- 172.21.15.40:0/3334508681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c441604b0 0x7f8c44069bf0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:59.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:59.033+0000 7f8c499e6700 1 --2- 172.21.15.40:0/3334508681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c440b2240 0x7f8c440c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:23:59.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:59.033+0000 7f8c499e6700 1 -- 172.21.15.40:0/3334508681 >> 172.21.15.40:0/3334508681 conn(0x7f8c440ccb30 msgr2=0x7f8c440ccf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:23:59.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:59.033+0000 7f8c499e6700 1 -- 172.21.15.40:0/3334508681 shutdown_connections 2023-11-08T08:23:59.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:23:59.033+0000 7f8c499e6700 1 -- 172.21.15.40:0/3334508681 wait complete. 2023-11-08T08:24:00.065 INFO:tasks.ceph:Scrubbing osd.0 2023-11-08T08:24:00.065 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-08T08:24:00.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.241+0000 7f2d2d253700 1 Processor -- start 2023-11-08T08:24:00.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.241+0000 7f2d2d253700 1 -- start start 2023-11-08T08:24:00.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.245+0000 7f2d2d253700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d28071aa0 0x7f2d280b8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:00.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.245+0000 7f2d2d253700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d28072240 con 0x7f2d2815c660 2023-11-08T08:24:00.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.245+0000 7f2d2d253700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d280cf960 con 0x7f2d2806a300 2023-11-08T08:24:00.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.245+0000 7f2d2d253700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d280b8850 con 0x7f2d28071aa0 2023-11-08T08:24:00.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.245+0000 7f2d27fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2d2815c660 0x7f2d28069bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57866/0 (socket says 172.21.15.40:57866) 2023-11-08T08:24:00.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.245+0000 7f2d27fff700 1 -- 172.21.15.40:0/1390483592 learned_addr learned my addr 172.21.15.40:0/1390483592 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:24:00.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.245+0000 7f2d2ca52700 1 --2- 172.21.15.40:0/1390483592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d28071aa0 0x7f2d280b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:00.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.245+0000 7f2d26ffd700 1 -- 172.21.15.40:0/1390483592 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443083952 0 0) 0x7f2d280cf960 con 0x7f2d2806a300 2023-11-08T08:24:00.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.245+0000 7f2d26ffd700 1 -- 172.21.15.40:0/1390483592 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d100035d0 con 0x7f2d2806a300 2023-11-08T08:24:00.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.245+0000 7f2d26ffd700 1 -- 172.21.15.40:0/1390483592 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202803503 0 0) 0x7f2d28072240 con 0x7f2d2815c660 2023-11-08T08:24:00.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.245+0000 7f2d26ffd700 1 -- 172.21.15.40:0/1390483592 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d280cf960 con 0x7f2d2815c660 2023-11-08T08:24:00.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.245+0000 7f2d2ca52700 1 -- 172.21.15.40:0/1390483592 >> v1:172.21.15.169:6789/0 conn(0x7f2d2806a300 legacy=0x7f2d2806c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.245+0000 7f2d2ca52700 1 -- 172.21.15.40:0/1390483592 >> v1:172.21.15.40:6789/0 conn(0x7f2d2815c660 legacy=0x7f2d28069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.245+0000 7f2d2ca52700 1 -- 172.21.15.40:0/1390483592 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2815ca30 con 0x7f2d28071aa0 2023-11-08T08:24:00.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.245+0000 7f2d2ca52700 1 --2- 172.21.15.40:0/1390483592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d28071aa0 0x7f2d280b8310 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2d20009dc0 tx=0x7f2d2000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f6067db0704d0cb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:00.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.249+0000 7f2d26ffd700 1 -- 172.21.15.40:0/1390483592 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2000e070 con 0x7f2d28071aa0 2023-11-08T08:24:00.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.249+0000 7f2d26ffd700 1 -- 172.21.15.40:0/1390483592 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2000bb20 con 0x7f2d28071aa0 2023-11-08T08:24:00.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.249+0000 7f2d26ffd700 1 -- 172.21.15.40:0/1390483592 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20012430 con 0x7f2d28071aa0 2023-11-08T08:24:00.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.249+0000 7f2d2d253700 1 -- 172.21.15.40:0/1390483592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d28071aa0 msgr2=0x7f2d280b8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.249+0000 7f2d2d253700 1 --2- 172.21.15.40:0/1390483592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d28071aa0 0x7f2d280b8310 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2d20009dc0 tx=0x7f2d2000b330 comp rx=0 tx=0).stop 2023-11-08T08:24:00.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.249+0000 7f2d2d253700 1 -- 172.21.15.40:0/1390483592 shutdown_connections 2023-11-08T08:24:00.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.249+0000 7f2d2d253700 1 --2- 172.21.15.40:0/1390483592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d28071aa0 0x7f2d280b8310 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:00.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.249+0000 7f2d2d253700 1 -- 172.21.15.40:0/1390483592 >> 172.21.15.40:0/1390483592 conn(0x7f2d280b6500 msgr2=0x7f2d28071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:00.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.249+0000 7f2d2d253700 1 -- 172.21.15.40:0/1390483592 shutdown_connections 2023-11-08T08:24:00.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.249+0000 7f2d2d253700 1 -- 172.21.15.40:0/1390483592 wait complete. 2023-11-08T08:24:00.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.253+0000 7f2d2d253700 1 Processor -- start 2023-11-08T08:24:00.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.253+0000 7f2d2d253700 1 -- start start 2023-11-08T08:24:00.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.253+0000 7f2d2d253700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d2806a300 0x7f2d281f4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:00.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.253+0000 7f2d2d253700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d28071aa0 0x7f2d2815d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:00.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.253+0000 7f2d2d253700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d2815c660 0x7f2d280d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:00.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.253+0000 7f2d2d253700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2815eb30 con 0x7f2d2815c660 2023-11-08T08:24:00.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.253+0000 7f2d2d253700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2809c770 con 0x7f2d28071aa0 2023-11-08T08:24:00.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.253+0000 7f2d2d253700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d281f4dd0 con 0x7f2d2806a300 2023-11-08T08:24:00.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.257+0000 7f2d2ca52700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d2815c660 0x7f2d280d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:00.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.257+0000 7f2d2ca52700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d2815c660 0x7f2d280d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43860/0 (socket says 172.21.15.40:43860) 2023-11-08T08:24:00.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.257+0000 7f2d2ca52700 1 -- 172.21.15.40:0/2113153779 learned_addr learned my addr 172.21.15.40:0/2113153779 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:00.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.257+0000 7f2d27fff700 1 --2- 172.21.15.40:0/2113153779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d2806a300 0x7f2d281f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:00.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.257+0000 7f2d277fe700 1 --2- 172.21.15.40:0/2113153779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d28071aa0 0x7f2d2815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:00.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.257+0000 7f2d2ca52700 1 -- 172.21.15.40:0/2113153779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d2806a300 msgr2=0x7f2d281f4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.257+0000 7f2d2ca52700 1 --2- 172.21.15.40:0/2113153779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d2806a300 0x7f2d281f4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.257+0000 7f2d2ca52700 1 -- 172.21.15.40:0/2113153779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d28071aa0 msgr2=0x7f2d2815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.257+0000 7f2d2ca52700 1 --2- 172.21.15.40:0/2113153779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d28071aa0 0x7f2d2815d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:00.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.257+0000 7f2d2ca52700 1 -- 172.21.15.40:0/2113153779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d20009a70 con 0x7f2d2815c660 2023-11-08T08:24:00.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.257+0000 7f2d2ca52700 1 --2- 172.21.15.40:0/2113153779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d2815c660 0x7f2d280d1fc0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f2d20014040 tx=0x7f2d20004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:00.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.257+0000 7f2d24ff9700 1 -- 172.21.15.40:0/2113153779 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2001c810 con 0x7f2d2815c660 2023-11-08T08:24:00.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.257+0000 7f2d24ff9700 1 -- 172.21.15.40:0/2113153779 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d20020070 con 0x7f2d2815c660 2023-11-08T08:24:00.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.257+0000 7f2d24ff9700 1 -- 172.21.15.40:0/2113153779 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2001cbf0 con 0x7f2d2815c660 2023-11-08T08:24:00.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.257+0000 7f2d2d253700 1 -- 172.21.15.40:0/2113153779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d280cf310 con 0x7f2d2815c660 2023-11-08T08:24:00.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.257+0000 7f2d2d253700 1 -- 172.21.15.40:0/2113153779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d28071f60 con 0x7f2d2815c660 2023-11-08T08:24:00.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.257+0000 7f2d2d253700 1 -- 172.21.15.40:0/2113153779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2d280cf5f0 con 0x7f2d2815c660 2023-11-08T08:24:00.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.261+0000 7f2d24ff9700 1 -- 172.21.15.40:0/2113153779 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f2d2001cd50 con 0x7f2d2815c660 2023-11-08T08:24:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.261+0000 7f2d24ff9700 1 --2- 172.21.15.40:0/2113153779 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2d14064ee0 0x7f2d14067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.261+0000 7f2d24ff9700 1 -- 172.21.15.40:0/2113153779 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f2d20022070 con 0x7f2d2815c660 2023-11-08T08:24:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.261+0000 7f2d24ff9700 1 --2- 172.21.15.40:0/2113153779 >> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] conn(0x7f2d1406b7f0 0x7f2d1406dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:00.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.261+0000 7f2d24ff9700 1 -- 172.21.15.40:0/2113153779 --> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d1406e2f0 con 0x7f2d1406b7f0 2023-11-08T08:24:00.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.261+0000 7f2d24ff9700 1 -- 172.21.15.40:0/2113153779 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d20084770 con 0x7f2d2815c660 2023-11-08T08:24:00.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.261+0000 7f2d27fff700 1 --2- 172.21.15.40:0/2113153779 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2d14064ee0 0x7f2d14067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:00.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.261+0000 7f2d277fe700 1 --2- 172.21.15.40:0/2113153779 >> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] conn(0x7f2d1406b7f0 0x7f2d1406dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:00.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.265+0000 7f2d277fe700 1 --2- 172.21.15.40:0/2113153779 >> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] conn(0x7f2d1406b7f0 0x7f2d1406dc20 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:00.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.265+0000 7f2d27fff700 1 --2- 172.21.15.40:0/2113153779 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2d14064ee0 0x7f2d14067390 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2d18007fd0 tx=0x7f2d18007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:00.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.265+0000 7f2d24ff9700 1 -- 172.21.15.40:0/2113153779 <== osd.0 v2:172.21.15.40:6808/35482 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2d1406e2f0 con 0x7f2d1406b7f0 2023-11-08T08:24:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.285+0000 7f2d2d253700 1 -- 172.21.15.40:0/2113153779 --> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2d28073cb0 con 0x7f2d1406b7f0 2023-11-08T08:24:00.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.285+0000 7f2d24ff9700 1 -- 172.21.15.40:0/2113153779 <== osd.0 v2:172.21.15.40:6808/35482 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2d28073cb0 con 0x7f2d1406b7f0 2023-11-08T08:24:00.291 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:24:00.292 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T08:24:00.292 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:24:00.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.285+0000 7f2d2d253700 1 -- 172.21.15.40:0/2113153779 >> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] conn(0x7f2d1406b7f0 msgr2=0x7f2d1406dc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.285+0000 7f2d2d253700 1 --2- 172.21.15.40:0/2113153779 >> [v2:172.21.15.40:6808/35482,v1:172.21.15.40:6809/35482] conn(0x7f2d1406b7f0 0x7f2d1406dc20 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:00.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.285+0000 7f2d2d253700 1 -- 172.21.15.40:0/2113153779 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2d14064ee0 msgr2=0x7f2d14067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.285+0000 7f2d2d253700 1 --2- 172.21.15.40:0/2113153779 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2d14064ee0 0x7f2d14067390 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2d18007fd0 tx=0x7f2d18007de0 comp rx=0 tx=0).stop 2023-11-08T08:24:00.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.285+0000 7f2d2d253700 1 -- 172.21.15.40:0/2113153779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d2815c660 msgr2=0x7f2d280d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.285+0000 7f2d2d253700 1 --2- 172.21.15.40:0/2113153779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d2815c660 0x7f2d280d1fc0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f2d20014040 tx=0x7f2d20004af0 comp rx=0 tx=0).stop 2023-11-08T08:24:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.285+0000 7f2d2ca52700 1 -- 172.21.15.40:0/2113153779 reap_dead start 2023-11-08T08:24:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.285+0000 7f2d2d253700 1 -- 172.21.15.40:0/2113153779 shutdown_connections 2023-11-08T08:24:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.285+0000 7f2d2d253700 1 -- 172.21.15.40:0/2113153779 >> 172.21.15.40:0/2113153779 conn(0x7f2d280b6500 msgr2=0x7f2d2815b690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.285+0000 7f2d2d253700 1 -- 172.21.15.40:0/2113153779 shutdown_connections 2023-11-08T08:24:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.285+0000 7f2d2d253700 1 -- 172.21.15.40:0/2113153779 wait complete. 2023-11-08T08:24:00.317 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-08T08:24:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.493+0000 7fd604c6d700 1 Processor -- start 2023-11-08T08:24:00.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.493+0000 7fd604c6d700 1 -- start start 2023-11-08T08:24:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.493+0000 7fd604c6d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd600069e00 0x7fd60006a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.493+0000 7fd604c6d700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd60006c600 con 0x7fd60015c640 2023-11-08T08:24:00.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.493+0000 7fd604c6d700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6000722f0 con 0x7fd600071aa0 2023-11-08T08:24:00.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.493+0000 7fd604c6d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd60006cef0 con 0x7fd600069e00 2023-11-08T08:24:00.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.493+0000 7fd5ff7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd60015c640 0x7fd6000d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57876/0 (socket says 172.21.15.40:57876) 2023-11-08T08:24:00.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.493+0000 7fd5f6ffd700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fd600071aa0 0x7fd60006d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:45186/0 (socket says 172.21.15.40:45186) 2023-11-08T08:24:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.493+0000 7fd5ff7fe700 1 -- 172.21.15.40:0/2817982193 learned_addr learned my addr 172.21.15.40:0/2817982193 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:24:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.493+0000 7fd5fffff700 1 --2- 172.21.15.40:0/2817982193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd600069e00 0x7fd60006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:00.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.497+0000 7fd5feffd700 1 -- 172.21.15.40:0/2817982193 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3972591530 0 0) 0x7fd6000722f0 con 0x7fd600071aa0 2023-11-08T08:24:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.497+0000 7fd5feffd700 1 -- 172.21.15.40:0/2817982193 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5ec0035d0 con 0x7fd600071aa0 2023-11-08T08:24:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.497+0000 7fd5feffd700 1 -- 172.21.15.40:0/2817982193 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2062800754 0 0) 0x7fd60006c600 con 0x7fd60015c640 2023-11-08T08:24:00.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.497+0000 7fd5feffd700 1 -- 172.21.15.40:0/2817982193 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6000722f0 con 0x7fd60015c640 2023-11-08T08:24:00.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.497+0000 7fd5fffff700 1 -- 172.21.15.40:0/2817982193 >> v1:172.21.15.169:6789/0 conn(0x7fd600071aa0 legacy=0x7fd60006d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.497+0000 7fd5fffff700 1 -- 172.21.15.40:0/2817982193 >> v1:172.21.15.40:6789/0 conn(0x7fd60015c640 legacy=0x7fd6000d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.497+0000 7fd5fffff700 1 -- 172.21.15.40:0/2817982193 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60006da20 con 0x7fd600069e00 2023-11-08T08:24:00.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.497+0000 7fd5fffff700 1 --2- 172.21.15.40:0/2817982193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd600069e00 0x7fd60006a280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009880 tx=0x7fd5f8009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61c7a8846ef3eeb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:00.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.497+0000 7fd5feffd700 1 -- 172.21.15.40:0/2817982193 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8011830 con 0x7fd600069e00 2023-11-08T08:24:00.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.497+0000 7fd5feffd700 1 -- 172.21.15.40:0/2817982193 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f8011990 con 0x7fd600069e00 2023-11-08T08:24:00.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.497+0000 7fd5feffd700 1 -- 172.21.15.40:0/2817982193 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8011d20 con 0x7fd600069e00 2023-11-08T08:24:00.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.497+0000 7fd604c6d700 1 -- 172.21.15.40:0/2817982193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd600069e00 msgr2=0x7fd60006a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.497+0000 7fd604c6d700 1 --2- 172.21.15.40:0/2817982193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd600069e00 0x7fd60006a280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009880 tx=0x7fd5f8009b90 comp rx=0 tx=0).stop 2023-11-08T08:24:00.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.497+0000 7fd604c6d700 1 -- 172.21.15.40:0/2817982193 shutdown_connections 2023-11-08T08:24:00.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.497+0000 7fd604c6d700 1 --2- 172.21.15.40:0/2817982193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd600069e00 0x7fd60006a280 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:00.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.497+0000 7fd604c6d700 1 -- 172.21.15.40:0/2817982193 >> 172.21.15.40:0/2817982193 conn(0x7fd6000b6520 msgr2=0x7fd600071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:00.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.497+0000 7fd604c6d700 1 -- 172.21.15.40:0/2817982193 shutdown_connections 2023-11-08T08:24:00.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.497+0000 7fd604c6d700 1 -- 172.21.15.40:0/2817982193 wait complete. 2023-11-08T08:24:00.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.501+0000 7fd604c6d700 1 Processor -- start 2023-11-08T08:24:00.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.501+0000 7fd604c6d700 1 -- start start 2023-11-08T08:24:00.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.501+0000 7fd604c6d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd600069e00 0x7fd600158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:00.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.501+0000 7fd604c6d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd600071aa0 0x7fd6000cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:00.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.501+0000 7fd604c6d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd60015c640 0x7fd6000cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:00.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.501+0000 7fd604c6d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6000d38f0 con 0x7fd600071aa0 2023-11-08T08:24:00.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.501+0000 7fd604c6d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6000cc210 con 0x7fd60015c640 2023-11-08T08:24:00.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.501+0000 7fd604c6d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6000ca630 con 0x7fd600069e00 2023-11-08T08:24:00.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.501+0000 7fd5f6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd600071aa0 0x7fd6000cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:00.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.501+0000 7fd5f6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd600071aa0 0x7fd6000cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43862/0 (socket says 172.21.15.40:43862) 2023-11-08T08:24:00.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.501+0000 7fd5f6ffd700 1 -- 172.21.15.40:0/3371490851 learned_addr learned my addr 172.21.15.40:0/3371490851 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:00.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.505+0000 7fd5fffff700 1 --2- 172.21.15.40:0/3371490851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd60015c640 0x7fd6000cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:00.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.505+0000 7fd5ff7fe700 1 --2- 172.21.15.40:0/3371490851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd600069e00 0x7fd600158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:00.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.505+0000 7fd5f6ffd700 1 -- 172.21.15.40:0/3371490851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd600069e00 msgr2=0x7fd600158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.505+0000 7fd5f6ffd700 1 --2- 172.21.15.40:0/3371490851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd600069e00 0x7fd600158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:00.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.505+0000 7fd5f6ffd700 1 -- 172.21.15.40:0/3371490851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd60015c640 msgr2=0x7fd6000cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.505+0000 7fd5f6ffd700 1 --2- 172.21.15.40:0/3371490851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd60015c640 0x7fd6000cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:00.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.505+0000 7fd5f6ffd700 1 -- 172.21.15.40:0/3371490851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f80094b0 con 0x7fd600071aa0 2023-11-08T08:24:00.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.505+0000 7fd5f6ffd700 1 --2- 172.21.15.40:0/3371490851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd600071aa0 0x7fd6000cb4a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800c700 tx=0x7fd5e800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:00.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.505+0000 7fd5fcff9700 1 -- 172.21.15.40:0/3371490851 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8005cd0 con 0x7fd600071aa0 2023-11-08T08:24:00.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.505+0000 7fd5fcff9700 1 -- 172.21.15.40:0/3371490851 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e8005e30 con 0x7fd600071aa0 2023-11-08T08:24:00.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.505+0000 7fd5fcff9700 1 -- 172.21.15.40:0/3371490851 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8007310 con 0x7fd600071aa0 2023-11-08T08:24:00.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.505+0000 7fd604c6d700 1 -- 172.21.15.40:0/3371490851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd600072010 con 0x7fd600071aa0 2023-11-08T08:24:00.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.505+0000 7fd604c6d700 1 -- 172.21.15.40:0/3371490851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd60015ca10 con 0x7fd600071aa0 2023-11-08T08:24:00.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.505+0000 7fd604c6d700 1 -- 172.21.15.40:0/3371490851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd60006c830 con 0x7fd600071aa0 2023-11-08T08:24:00.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.509+0000 7fd5fcff9700 1 -- 172.21.15.40:0/3371490851 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd5e8019030 con 0x7fd600071aa0 2023-11-08T08:24:00.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.509+0000 7fd5fcff9700 1 --2- 172.21.15.40:0/3371490851 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd5e0064ee0 0x7fd5e0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:00.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.509+0000 7fd5fcff9700 1 -- 172.21.15.40:0/3371490851 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fd5e8081290 con 0x7fd600071aa0 2023-11-08T08:24:00.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.509+0000 7fd5ff7fe700 1 --2- 172.21.15.40:0/3371490851 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd5e0064ee0 0x7fd5e0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:00.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.509+0000 7fd5ff7fe700 1 --2- 172.21.15.40:0/3371490851 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd5e0064ee0 0x7fd5e0067390 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0001560 tx=0x7fd5f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:00.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.517+0000 7fd5fcff9700 1 -- 172.21.15.40:0/3371490851 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd5e804f4a0 con 0x7fd600071aa0 2023-11-08T08:24:00.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.693+0000 7fd604c6d700 1 -- 172.21.15.40:0/3371490851 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fd600073cb0 con 0x7fd5e0064ee0 2023-11-08T08:24:00.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.693+0000 7fd5fcff9700 1 -- 172.21.15.40:0/3371490851 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd600073cb0 con 0x7fd5e0064ee0 2023-11-08T08:24:00.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.697+0000 7fd604c6d700 1 -- 172.21.15.40:0/3371490851 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd5e0064ee0 msgr2=0x7fd5e0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.697+0000 7fd604c6d700 1 --2- 172.21.15.40:0/3371490851 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd5e0064ee0 0x7fd5e0067390 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0001560 tx=0x7fd5f0009040 comp rx=0 tx=0).stop 2023-11-08T08:24:00.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.697+0000 7fd604c6d700 1 -- 172.21.15.40:0/3371490851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd600071aa0 msgr2=0x7fd6000cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.697+0000 7fd604c6d700 1 --2- 172.21.15.40:0/3371490851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd600071aa0 0x7fd6000cb4a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800c700 tx=0x7fd5e800cac0 comp rx=0 tx=0).stop 2023-11-08T08:24:00.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.697+0000 7fd604c6d700 1 -- 172.21.15.40:0/3371490851 shutdown_connections 2023-11-08T08:24:00.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.697+0000 7fd604c6d700 1 --2- 172.21.15.40:0/3371490851 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd5e0064ee0 0x7fd5e0067390 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:00.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.697+0000 7fd604c6d700 1 --2- 172.21.15.40:0/3371490851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd60015c640 0x7fd6000cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:00.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.697+0000 7fd604c6d700 1 --2- 172.21.15.40:0/3371490851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd600071aa0 0x7fd6000cb4a0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:00.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.697+0000 7fd604c6d700 1 --2- 172.21.15.40:0/3371490851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd600069e00 0x7fd600158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:00.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.697+0000 7fd604c6d700 1 -- 172.21.15.40:0/3371490851 >> 172.21.15.40:0/3371490851 conn(0x7fd6000b6520 msgr2=0x7fd6000b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:00.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.697+0000 7fd604c6d700 1 -- 172.21.15.40:0/3371490851 shutdown_connections 2023-11-08T08:24:00.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.697+0000 7fd604c6d700 1 -- 172.21.15.40:0/3371490851 wait complete. 2023-11-08T08:24:00.708 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 0 to deep-scrub 2023-11-08T08:24:00.727 INFO:tasks.ceph:Scrubbing osd.1 2023-11-08T08:24:00.728 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-08T08:24:00.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.905+0000 7f12aa933700 1 Processor -- start 2023-11-08T08:24:00.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.905+0000 7f12aa933700 1 -- start start 2023-11-08T08:24:00.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.909+0000 7f12aa933700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f12a406b1c0 0x7f12a4142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.909+0000 7f12aa933700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a406b8d0 con 0x7f12a4142650 2023-11-08T08:24:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.909+0000 7f12aa933700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a406bb70 con 0x7f12a40b6110 2023-11-08T08:24:00.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.909+0000 7f12aa933700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a406b5f0 con 0x7f12a406b1c0 2023-11-08T08:24:00.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.909+0000 7f12a3fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f12a4142650 0x7f12a40b5a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57878/0 (socket says 172.21.15.40:57878) 2023-11-08T08:24:00.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.909+0000 7f12a3fff700 1 -- 172.21.15.40:0/259988402 learned_addr learned my addr 172.21.15.40:0/259988402 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:24:00.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.909+0000 7f12a37fe700 1 --1- 172.21.15.40:0/259988402 >> v1:172.21.15.169:6789/0 conn(0x7f12a40b6110 0x7f12a406aab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:45188/0 (socket says 172.21.15.40:45188) 2023-11-08T08:24:00.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.909+0000 7f12a8ed0700 1 --2- 172.21.15.40:0/259988402 >> v2:172.21.15.40:3300/0 conn(0x7f12a406b1c0 0x7f12a4142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:00.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.909+0000 7f12a8ed0700 1 -- 172.21.15.40:0/259988402 >> v1:172.21.15.169:6789/0 conn(0x7f12a40b6110 legacy=0x7f12a406aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.909+0000 7f12a8ed0700 1 -- 172.21.15.40:0/259988402 >> v1:172.21.15.40:6789/0 conn(0x7f12a4142650 legacy=0x7f12a40b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.909+0000 7f12a8ed0700 1 -- 172.21.15.40:0/259988402 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a406be50 con 0x7f12a406b1c0 2023-11-08T08:24:00.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.909+0000 7f12a2ffd700 1 -- 172.21.15.40:0/259988402 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329390213 0 0) 0x7f12a406b8d0 con 0x7f12a4142650 2023-11-08T08:24:00.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.909+0000 7f12a8ed0700 1 --2- 172.21.15.40:0/259988402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a406b1c0 0x7f12a4142fa0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f129c009d90 tx=0x7f129c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6172874fc65a77ae server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:00.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.909+0000 7f12a2ffd700 1 -- 172.21.15.40:0/259988402 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c00e070 con 0x7f12a406b1c0 2023-11-08T08:24:00.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.913+0000 7f12a2ffd700 1 -- 172.21.15.40:0/259988402 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c00bbd0 con 0x7f12a406b1c0 2023-11-08T08:24:00.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.913+0000 7f12a2ffd700 1 -- 172.21.15.40:0/259988402 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c012430 con 0x7f12a406b1c0 2023-11-08T08:24:00.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.913+0000 7f12aa933700 1 -- 172.21.15.40:0/259988402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a406b1c0 msgr2=0x7f12a4142fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.913+0000 7f12aa933700 1 --2- 172.21.15.40:0/259988402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a406b1c0 0x7f12a4142fa0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f129c009d90 tx=0x7f129c00b330 comp rx=0 tx=0).stop 2023-11-08T08:24:00.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.913+0000 7f12aa933700 1 -- 172.21.15.40:0/259988402 shutdown_connections 2023-11-08T08:24:00.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.913+0000 7f12aa933700 1 --2- 172.21.15.40:0/259988402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a406b1c0 0x7f12a4142fa0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:00.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.913+0000 7f12aa933700 1 -- 172.21.15.40:0/259988402 >> 172.21.15.40:0/259988402 conn(0x7f12a40b5390 msgr2=0x7f12a40d2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:00.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.913+0000 7f12aa933700 1 -- 172.21.15.40:0/259988402 shutdown_connections 2023-11-08T08:24:00.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.913+0000 7f12aa933700 1 -- 172.21.15.40:0/259988402 wait complete. 2023-11-08T08:24:00.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.913+0000 7f12aa933700 1 Processor -- start 2023-11-08T08:24:00.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.913+0000 7f12aa933700 1 -- start start 2023-11-08T08:24:00.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12aa933700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a406b1c0 0x7f12a4157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:00.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12aa933700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12a40b6110 0x7f12a4133ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:00.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12aa933700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12a4142650 0x7f12a40b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:00.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12aa933700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40ad390 con 0x7f12a406b1c0 2023-11-08T08:24:00.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12aa933700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a415efd0 con 0x7f12a40b6110 2023-11-08T08:24:00.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12aa933700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a415f2b0 con 0x7f12a4142650 2023-11-08T08:24:00.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12a3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a406b1c0 0x7f12a4157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:00.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12a3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a406b1c0 0x7f12a4157b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43878/0 (socket says 172.21.15.40:43878) 2023-11-08T08:24:00.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12a3fff700 1 -- 172.21.15.40:0/3231053435 learned_addr learned my addr 172.21.15.40:0/3231053435 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:00.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12a8ed0700 1 --2- 172.21.15.40:0/3231053435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12a4142650 0x7f12a40b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:00.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12a37fe700 1 --2- 172.21.15.40:0/3231053435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12a40b6110 0x7f12a4133ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:00.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12a3fff700 1 -- 172.21.15.40:0/3231053435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12a4142650 msgr2=0x7f12a40b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12a3fff700 1 --2- 172.21.15.40:0/3231053435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12a4142650 0x7f12a40b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:00.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12a3fff700 1 -- 172.21.15.40:0/3231053435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12a40b6110 msgr2=0x7f12a4133ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12a3fff700 1 --2- 172.21.15.40:0/3231053435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12a40b6110 0x7f12a4133ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:00.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12a3fff700 1 -- 172.21.15.40:0/3231053435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c009a70 con 0x7f12a406b1c0 2023-11-08T08:24:00.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12a37fe700 1 --2- 172.21.15.40:0/3231053435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12a40b6110 0x7f12a4133ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:24:00.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12a3fff700 1 --2- 172.21.15.40:0/3231053435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a406b1c0 0x7f12a4157b60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f129400c4a0 tx=0x7f129400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:00.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12a0ff9700 1 -- 172.21.15.40:0/3231053435 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1294004ca0 con 0x7f12a406b1c0 2023-11-08T08:24:00.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12a0ff9700 1 -- 172.21.15.40:0/3231053435 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1294004e00 con 0x7f12a406b1c0 2023-11-08T08:24:00.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12a0ff9700 1 -- 172.21.15.40:0/3231053435 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1294007310 con 0x7f12a406b1c0 2023-11-08T08:24:00.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.917+0000 7f12aa933700 1 -- 172.21.15.40:0/3231053435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a406c410 con 0x7f12a406b1c0 2023-11-08T08:24:00.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.921+0000 7f12aa933700 1 -- 172.21.15.40:0/3231053435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a406c130 con 0x7f12a406b1c0 2023-11-08T08:24:00.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.921+0000 7f12aa933700 1 -- 172.21.15.40:0/3231053435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f12a4142a20 con 0x7f12a406b1c0 2023-11-08T08:24:00.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.921+0000 7f12a0ff9700 1 -- 172.21.15.40:0/3231053435 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f129401a020 con 0x7f12a406b1c0 2023-11-08T08:24:00.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.925+0000 7f12a0ff9700 1 --2- 172.21.15.40:0/3231053435 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1290064ee0 0x7f1290067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:00.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.925+0000 7f12a0ff9700 1 -- 172.21.15.40:0/3231053435 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f1294081810 con 0x7f12a406b1c0 2023-11-08T08:24:00.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.925+0000 7f12a0ff9700 1 --2- 172.21.15.40:0/3231053435 >> [v2:172.21.15.40:6800/35480,v1:172.21.15.40:6801/35480] conn(0x7f129006b7f0 0x7f129006dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:00.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.925+0000 7f12a0ff9700 1 -- 172.21.15.40:0/3231053435 --> [v2:172.21.15.40:6800/35480,v1:172.21.15.40:6801/35480] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f129006e2f0 con 0x7f129006b7f0 2023-11-08T08:24:00.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.925+0000 7f12a0ff9700 1 -- 172.21.15.40:0/3231053435 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12940862a0 con 0x7f12a406b1c0 2023-11-08T08:24:00.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.925+0000 7f12a37fe700 1 --2- 172.21.15.40:0/3231053435 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1290064ee0 0x7f1290067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:00.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.925+0000 7f12a8ed0700 1 --2- 172.21.15.40:0/3231053435 >> [v2:172.21.15.40:6800/35480,v1:172.21.15.40:6801/35480] conn(0x7f129006b7f0 0x7f129006dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:00.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.925+0000 7f12a8ed0700 1 --2- 172.21.15.40:0/3231053435 >> [v2:172.21.15.40:6800/35480,v1:172.21.15.40:6801/35480] conn(0x7f129006b7f0 0x7f129006dc20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:00.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.925+0000 7f12a37fe700 1 --2- 172.21.15.40:0/3231053435 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1290064ee0 0x7f1290067390 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f12a4141980 tx=0x7f1298007e10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:00.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.929+0000 7f12a0ff9700 1 -- 172.21.15.40:0/3231053435 <== osd.1 v2:172.21.15.40:6800/35480 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f129006e2f0 con 0x7f129006b7f0 2023-11-08T08:24:00.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.945+0000 7f12aa933700 1 -- 172.21.15.40:0/3231053435 --> [v2:172.21.15.40:6800/35480,v1:172.21.15.40:6801/35480] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f12a40726c0 con 0x7f129006b7f0 2023-11-08T08:24:00.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.945+0000 7f12a0ff9700 1 -- 172.21.15.40:0/3231053435 <== osd.1 v2:172.21.15.40:6800/35480 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f12a40726c0 con 0x7f129006b7f0 2023-11-08T08:24:00.953 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:24:00.953 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T08:24:00.953 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:24:00.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.945+0000 7f12aa933700 1 -- 172.21.15.40:0/3231053435 >> [v2:172.21.15.40:6800/35480,v1:172.21.15.40:6801/35480] conn(0x7f129006b7f0 msgr2=0x7f129006dc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.945+0000 7f12aa933700 1 --2- 172.21.15.40:0/3231053435 >> [v2:172.21.15.40:6800/35480,v1:172.21.15.40:6801/35480] conn(0x7f129006b7f0 0x7f129006dc20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:00.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.945+0000 7f12aa933700 1 -- 172.21.15.40:0/3231053435 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1290064ee0 msgr2=0x7f1290067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.945+0000 7f12aa933700 1 --2- 172.21.15.40:0/3231053435 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f1290064ee0 0x7f1290067390 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f12a4141980 tx=0x7f1298007e10 comp rx=0 tx=0).stop 2023-11-08T08:24:00.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.949+0000 7f12aa933700 1 -- 172.21.15.40:0/3231053435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a406b1c0 msgr2=0x7f12a4157b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:00.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.949+0000 7f12aa933700 1 --2- 172.21.15.40:0/3231053435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a406b1c0 0x7f12a4157b60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f129400c4a0 tx=0x7f129400c7b0 comp rx=0 tx=0).stop 2023-11-08T08:24:00.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.949+0000 7f12a8ed0700 1 -- 172.21.15.40:0/3231053435 reap_dead start 2023-11-08T08:24:00.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.949+0000 7f12aa933700 1 -- 172.21.15.40:0/3231053435 shutdown_connections 2023-11-08T08:24:00.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.949+0000 7f12aa933700 1 -- 172.21.15.40:0/3231053435 >> 172.21.15.40:0/3231053435 conn(0x7f12a40b5390 msgr2=0x7f12a40b5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:00.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.949+0000 7f12aa933700 1 -- 172.21.15.40:0/3231053435 shutdown_connections 2023-11-08T08:24:00.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:00.949+0000 7f12aa933700 1 -- 172.21.15.40:0/3231053435 wait complete. 2023-11-08T08:24:00.975 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-08T08:24:01.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.153+0000 7fa200a53700 1 Processor -- start 2023-11-08T08:24:01.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.157+0000 7fa200a53700 1 -- start start 2023-11-08T08:24:01.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.157+0000 7fa200a53700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1fc0b1640 0x7fa1fc070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.157+0000 7fa200a53700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa1fc0c9040 0x7fa1fc0705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.157+0000 7fa200a53700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1fc15a550 con 0x7fa1fc0b43a0 2023-11-08T08:24:01.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.157+0000 7fa200a53700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1fc06b0a0 con 0x7fa1fc0c9040 2023-11-08T08:24:01.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.157+0000 7fa200a53700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc0caac0 con 0x7fa1fc0b1640 2023-11-08T08:24:01.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1fb7fe700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fa1fc0b43a0 0x7fa1fc15e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:45196/0 (socket says 172.21.15.40:45196) 2023-11-08T08:24:01.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1fb7fe700 1 -- 172.21.15.40:0/1136558759 learned_addr learned my addr 172.21.15.40:0/1136558759 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:24:01.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1f2ffd700 1 --2- 172.21.15.40:0/1136558759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1fc0b1640 0x7fa1fc070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1fbfff700 1 --2- 172.21.15.40:0/1136558759 >> v2:172.21.15.169:3300/0 conn(0x7fa1fc0c9040 0x7fa1fc0705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1faffd700 1 -- 172.21.15.40:0/1136558759 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175001358 0 0) 0x7fa1ec001490 con 0x7fa1fc0b43a0 2023-11-08T08:24:01.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1faffd700 1 -- 172.21.15.40:0/1136558759 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1e80035d0 con 0x7fa1fc0b43a0 2023-11-08T08:24:01.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1faffd700 1 -- 172.21.15.40:0/1136558759 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2212951390 0 0) 0x7fa1e80035d0 con 0x7fa1fc0b43a0 2023-11-08T08:24:01.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1faffd700 1 -- 172.21.15.40:0/1136558759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1fc0b1640 msgr2=0x7fa1fc070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:01.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1faffd700 1 --2- 172.21.15.40:0/1136558759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1fc0b1640 0x7fa1fc070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1faffd700 1 -- 172.21.15.40:0/1136558759 >> v2:172.21.15.169:3300/0 conn(0x7fa1fc0c9040 msgr2=0x7fa1fc0705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:01.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1faffd700 1 --2- 172.21.15.40:0/1136558759 >> v2:172.21.15.169:3300/0 conn(0x7fa1fc0c9040 0x7fa1fc0705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1faffd700 1 -- 172.21.15.40:0/1136558759 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc0cada0 con 0x7fa1fc0b43a0 2023-11-08T08:24:01.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1faffd700 1 -- 172.21.15.40:0/1136558759 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7fa1ec001b80 con 0x7fa1fc0b43a0 2023-11-08T08:24:01.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1faffd700 1 -- 172.21.15.40:0/1136558759 >> v1:172.21.15.169:6789/0 conn(0x7fa1fc0b43a0 legacy=0x7fa1fc15e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:01.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1faffd700 1 --2- 172.21.15.40:0/1136558759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1e8003a20 0x7fa1e8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1faffd700 1 --2- 172.21.15.40:0/1136558759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8004400 0x7fa1e80048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1faffd700 1 --2- 172.21.15.40:0/1136558759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1e8004fe0 0x7fa1e800fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1faffd700 1 -- 172.21.15.40:0/1136558759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc06b0a0 con 0x7fa1e8004400 2023-11-08T08:24:01.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1faffd700 1 -- 172.21.15.40:0/1136558759 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc0caac0 con 0x7fa1e8004fe0 2023-11-08T08:24:01.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1faffd700 1 -- 172.21.15.40:0/1136558759 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8005a00 con 0x7fa1e8003a20 2023-11-08T08:24:01.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1fb7fe700 1 --2- 172.21.15.40:0/1136558759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1e8003a20 0x7fa1e8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1f2ffd700 1 --2- 172.21.15.40:0/1136558759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8004400 0x7fa1e80048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1fbfff700 1 --2- 172.21.15.40:0/1136558759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1e8004fe0 0x7fa1e800fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1fb7fe700 1 -- 172.21.15.40:0/1136558759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1e8004fe0 msgr2=0x7fa1e800fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:01.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1fb7fe700 1 --2- 172.21.15.40:0/1136558759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1e8004fe0 0x7fa1e800fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1fb7fe700 1 -- 172.21.15.40:0/1136558759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8004400 msgr2=0x7fa1e80048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:01.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1fb7fe700 1 --2- 172.21.15.40:0/1136558759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8004400 0x7fa1e80048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1fb7fe700 1 -- 172.21.15.40:0/1136558759 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e80035d0 con 0x7fa1e8003a20 2023-11-08T08:24:01.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.161+0000 7fa1fb7fe700 1 --2- 172.21.15.40:0/1136558759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1e8003a20 0x7fa1e8003e30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00a870 tx=0x7fa1e80058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48e96de186b57912 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:01.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.165+0000 7fa1faffd700 1 -- 172.21.15.40:0/1136558759 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0072d0 con 0x7fa1e8003a20 2023-11-08T08:24:01.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.165+0000 7fa1faffd700 1 -- 172.21.15.40:0/1136558759 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec007450 con 0x7fa1e8003a20 2023-11-08T08:24:01.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.165+0000 7fa1faffd700 1 -- 172.21.15.40:0/1136558759 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec007800 con 0x7fa1e8003a20 2023-11-08T08:24:01.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.165+0000 7fa200a53700 1 -- 172.21.15.40:0/1136558759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1e8003a20 msgr2=0x7fa1e8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:01.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.165+0000 7fa200a53700 1 --2- 172.21.15.40:0/1136558759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1e8003a20 0x7fa1e8003e30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00a870 tx=0x7fa1e80058a0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.165+0000 7fa1fbfff700 1 -- 172.21.15.40:0/1136558759 reap_dead start 2023-11-08T08:24:01.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.165+0000 7fa200a53700 1 -- 172.21.15.40:0/1136558759 shutdown_connections 2023-11-08T08:24:01.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.165+0000 7fa200a53700 1 -- 172.21.15.40:0/1136558759 >> 172.21.15.40:0/1136558759 conn(0x7fa1fc0b0f80 msgr2=0x7fa1fc0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:01.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.165+0000 7fa200a53700 1 -- 172.21.15.40:0/1136558759 shutdown_connections 2023-11-08T08:24:01.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.165+0000 7fa200a53700 1 -- 172.21.15.40:0/1136558759 wait complete. 2023-11-08T08:24:01.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.169+0000 7fa200a53700 1 Processor -- start 2023-11-08T08:24:01.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.169+0000 7fa200a53700 1 -- start start 2023-11-08T08:24:01.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.169+0000 7fa200a53700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1e8003a20 0x7fa1fc150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.169+0000 7fa200a53700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1fc154bd0 0x7fa1fc155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.169+0000 7fa200a53700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1fc0c3c10 0x7fa1fc0c4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.169+0000 7fa200a53700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc0cafe0 con 0x7fa1fc0c3c10 2023-11-08T08:24:01.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.169+0000 7fa200a53700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc155560 con 0x7fa1fc154bd0 2023-11-08T08:24:01.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.169+0000 7fa200a53700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc0c4980 con 0x7fa1e8003a20 2023-11-08T08:24:01.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.169+0000 7fa1fbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1fc0c3c10 0x7fa1fc0c4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.169+0000 7fa1fbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1fc0c3c10 0x7fa1fc0c4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43898/0 (socket says 172.21.15.40:43898) 2023-11-08T08:24:01.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.169+0000 7fa1fbfff700 1 -- 172.21.15.40:0/1523226495 learned_addr learned my addr 172.21.15.40:0/1523226495 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:01.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.169+0000 7fa1f2ffd700 1 --2- 172.21.15.40:0/1523226495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1fc154bd0 0x7fa1fc155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.169+0000 7fa1fb7fe700 1 --2- 172.21.15.40:0/1523226495 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1e8003a20 0x7fa1fc150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.169+0000 7fa1fbfff700 1 -- 172.21.15.40:0/1523226495 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1e8003a20 msgr2=0x7fa1fc150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:01.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.169+0000 7fa1fbfff700 1 --2- 172.21.15.40:0/1523226495 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1e8003a20 0x7fa1fc150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.169+0000 7fa1fbfff700 1 -- 172.21.15.40:0/1523226495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1fc154bd0 msgr2=0x7fa1fc155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:01.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.169+0000 7fa1fbfff700 1 --2- 172.21.15.40:0/1523226495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1fc154bd0 0x7fa1fc155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.169+0000 7fa1fbfff700 1 -- 172.21.15.40:0/1523226495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc06b5d0 con 0x7fa1fc0c3c10 2023-11-08T08:24:01.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.169+0000 7fa1f2ffd700 1 --2- 172.21.15.40:0/1523226495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1fc154bd0 0x7fa1fc155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:24:01.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.173+0000 7fa1fbfff700 1 --2- 172.21.15.40:0/1523226495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1fc0c3c10 0x7fa1fc0c4080 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc15db70 tx=0x7fa1f4011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:01.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.173+0000 7fa1f8ff9700 1 -- 172.21.15.40:0/1523226495 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4011e40 con 0x7fa1fc0c3c10 2023-11-08T08:24:01.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.173+0000 7fa1f8ff9700 1 -- 172.21.15.40:0/1523226495 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f4004240 con 0x7fa1fc0c3c10 2023-11-08T08:24:01.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.173+0000 7fa200a53700 1 -- 172.21.15.40:0/1523226495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1fc06b2f0 con 0x7fa1fc0c3c10 2023-11-08T08:24:01.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.173+0000 7fa1f8ff9700 1 -- 172.21.15.40:0/1523226495 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f40036c0 con 0x7fa1fc0c3c10 2023-11-08T08:24:01.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.173+0000 7fa200a53700 1 -- 172.21.15.40:0/1523226495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1fc0ca7b0 con 0x7fa1fc0c3c10 2023-11-08T08:24:01.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.173+0000 7fa200a53700 1 -- 172.21.15.40:0/1523226495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1fc0c5060 con 0x7fa1fc0c3c10 2023-11-08T08:24:01.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.177+0000 7fa1f8ff9700 1 -- 172.21.15.40:0/1523226495 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa1f4003820 con 0x7fa1fc0c3c10 2023-11-08T08:24:01.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.177+0000 7fa1f8ff9700 1 --2- 172.21.15.40:0/1523226495 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa1dc064f10 0x7fa1dc0673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.177+0000 7fa1f8ff9700 1 -- 172.21.15.40:0/1523226495 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fa1f4081f80 con 0x7fa1fc0c3c10 2023-11-08T08:24:01.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.177+0000 7fa1fb7fe700 1 --2- 172.21.15.40:0/1523226495 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa1dc064f10 0x7fa1dc0673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.177+0000 7fa1fb7fe700 1 --2- 172.21.15.40:0/1523226495 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa1dc064f10 0x7fa1dc0673c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec000f80 tx=0x7fa1ec014040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:01.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.181+0000 7fa1f8ff9700 1 -- 172.21.15.40:0/1523226495 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa1f4050180 con 0x7fa1fc0c3c10 2023-11-08T08:24:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.357+0000 7fa200a53700 1 -- 172.21.15.40:0/1523226495 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1fc073cb0 con 0x7fa1dc064f10 2023-11-08T08:24:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.357+0000 7fa1f8ff9700 1 -- 172.21.15.40:0/1523226495 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa1fc073cb0 con 0x7fa1dc064f10 2023-11-08T08:24:01.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.361+0000 7fa200a53700 1 -- 172.21.15.40:0/1523226495 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa1dc064f10 msgr2=0x7fa1dc0673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:01.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.361+0000 7fa200a53700 1 --2- 172.21.15.40:0/1523226495 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa1dc064f10 0x7fa1dc0673c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec000f80 tx=0x7fa1ec014040 comp rx=0 tx=0).stop 2023-11-08T08:24:01.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.361+0000 7fa200a53700 1 -- 172.21.15.40:0/1523226495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1fc0c3c10 msgr2=0x7fa1fc0c4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:01.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.361+0000 7fa200a53700 1 --2- 172.21.15.40:0/1523226495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1fc0c3c10 0x7fa1fc0c4080 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc15db70 tx=0x7fa1f4011c60 comp rx=0 tx=0).stop 2023-11-08T08:24:01.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.361+0000 7fa200a53700 1 -- 172.21.15.40:0/1523226495 shutdown_connections 2023-11-08T08:24:01.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.361+0000 7fa200a53700 1 --2- 172.21.15.40:0/1523226495 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa1dc064f10 0x7fa1dc0673c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.361+0000 7fa200a53700 1 --2- 172.21.15.40:0/1523226495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1fc0c3c10 0x7fa1fc0c4080 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.361+0000 7fa200a53700 1 --2- 172.21.15.40:0/1523226495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1fc154bd0 0x7fa1fc155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.361+0000 7fa200a53700 1 --2- 172.21.15.40:0/1523226495 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1e8003a20 0x7fa1fc150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.361+0000 7fa200a53700 1 -- 172.21.15.40:0/1523226495 >> 172.21.15.40:0/1523226495 conn(0x7fa1fc0b0f80 msgr2=0x7fa1fc0ad970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:01.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.361+0000 7fa200a53700 1 -- 172.21.15.40:0/1523226495 shutdown_connections 2023-11-08T08:24:01.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.361+0000 7fa200a53700 1 -- 172.21.15.40:0/1523226495 wait complete. 2023-11-08T08:24:01.372 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 1 to deep-scrub 2023-11-08T08:24:01.390 INFO:tasks.ceph:Scrubbing osd.2 2023-11-08T08:24:01.391 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-08T08:24:01.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.569+0000 7f445bfff700 1 Processor -- start 2023-11-08T08:24:01.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.569+0000 7f445bfff700 1 -- start start 2023-11-08T08:24:01.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.573+0000 7f445bfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f445c159150 0x7f445c0ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.573+0000 7f445bfff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f445c0c9460 con 0x7f445c0ab510 2023-11-08T08:24:01.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.573+0000 7f445bfff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f445c06a9c0 con 0x7f445c0caf30 2023-11-08T08:24:01.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.573+0000 7f445bfff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445c0c8c00 con 0x7f445c159150 2023-11-08T08:24:01.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.573+0000 7f445b7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f445c0ab510 0x7f445c0ab970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57886/0 (socket says 172.21.15.40:57886) 2023-11-08T08:24:01.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.573+0000 7f445b7fe700 1 -- 172.21.15.40:0/2072692747 learned_addr learned my addr 172.21.15.40:0/2072692747 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:24:01.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.573+0000 7f445a7fc700 1 --1- 172.21.15.40:0/2072692747 >> v1:172.21.15.169:6789/0 conn(0x7f445c0caf30 0x7f445c0cb380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.40:45204/0 (socket says 172.21.15.40:45204) 2023-11-08T08:24:01.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.573+0000 7f445affd700 1 --2- 172.21.15.40:0/2072692747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f445c159150 0x7f445c0ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.573+0000 7f4459ffb700 1 -- 172.21.15.40:0/2072692747 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106072281 0 0) 0x7f445c06a9c0 con 0x7f445c0caf30 2023-11-08T08:24:01.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.573+0000 7f4459ffb700 1 -- 172.21.15.40:0/2072692747 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44480057e0 con 0x7f445c0caf30 2023-11-08T08:24:01.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.573+0000 7f4459ffb700 1 -- 172.21.15.40:0/2072692747 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615411421 0 0) 0x7f445c0c9460 con 0x7f445c0ab510 2023-11-08T08:24:01.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.573+0000 7f4459ffb700 1 -- 172.21.15.40:0/2072692747 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f445c06a9c0 con 0x7f445c0ab510 2023-11-08T08:24:01.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.573+0000 7f445affd700 1 -- 172.21.15.40:0/2072692747 >> v1:172.21.15.169:6789/0 conn(0x7f445c0caf30 legacy=0x7f445c0cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:01.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.573+0000 7f445affd700 1 -- 172.21.15.40:0/2072692747 >> v1:172.21.15.40:6789/0 conn(0x7f445c0ab510 legacy=0x7f445c0ab970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:01.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.573+0000 7f445affd700 1 -- 172.21.15.40:0/2072692747 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445c0c8e10 con 0x7f445c159150 2023-11-08T08:24:01.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.577+0000 7f445affd700 1 --2- 172.21.15.40:0/2072692747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f445c159150 0x7f445c0ad6c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f444c009870 tx=0x7f444c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=736b1566a8e961c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:01.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.577+0000 7f4459ffb700 1 -- 172.21.15.40:0/2072692747 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c005e50 con 0x7f445c159150 2023-11-08T08:24:01.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.577+0000 7f4459ffb700 1 -- 172.21.15.40:0/2072692747 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444c006030 con 0x7f445c159150 2023-11-08T08:24:01.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.577+0000 7f4459ffb700 1 -- 172.21.15.40:0/2072692747 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c0063c0 con 0x7f445c159150 2023-11-08T08:24:01.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.577+0000 7f445bfff700 1 -- 172.21.15.40:0/2072692747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f445c159150 msgr2=0x7f445c0ad6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:01.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.577+0000 7f445bfff700 1 --2- 172.21.15.40:0/2072692747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f445c159150 0x7f445c0ad6c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f444c009870 tx=0x7f444c009b80 comp rx=0 tx=0).stop 2023-11-08T08:24:01.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.577+0000 7f445bfff700 1 -- 172.21.15.40:0/2072692747 shutdown_connections 2023-11-08T08:24:01.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.577+0000 7f445bfff700 1 --2- 172.21.15.40:0/2072692747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f445c159150 0x7f445c0ad6c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.577+0000 7f445bfff700 1 -- 172.21.15.40:0/2072692747 >> 172.21.15.40:0/2072692747 conn(0x7f445c0ce5b0 msgr2=0x7f445c0ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:01.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.577+0000 7f445bfff700 1 -- 172.21.15.40:0/2072692747 shutdown_connections 2023-11-08T08:24:01.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.577+0000 7f445bfff700 1 -- 172.21.15.40:0/2072692747 wait complete. 2023-11-08T08:24:01.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.581+0000 7f445bfff700 1 Processor -- start 2023-11-08T08:24:01.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.581+0000 7f445bfff700 1 -- start start 2023-11-08T08:24:01.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.581+0000 7f445bfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f445c0ab510 0x7f445c0c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.581+0000 7f445bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f445c0caf30 0x7f445c097750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.581+0000 7f445bfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f445c159150 0x7f445c1f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.581+0000 7f445bfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c1f6c90 con 0x7f445c0caf30 2023-11-08T08:24:01.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.581+0000 7f445bfff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c06c680 con 0x7f445c0ab510 2023-11-08T08:24:01.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.581+0000 7f445bfff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445c0c7d40 con 0x7f445c159150 2023-11-08T08:24:01.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.581+0000 7f445a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f445c0caf30 0x7f445c097750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.581+0000 7f445a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f445c0caf30 0x7f445c097750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43912/0 (socket says 172.21.15.40:43912) 2023-11-08T08:24:01.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.581+0000 7f445a7fc700 1 -- 172.21.15.40:0/3524938773 learned_addr learned my addr 172.21.15.40:0/3524938773 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:01.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.581+0000 7f445b7fe700 1 --2- 172.21.15.40:0/3524938773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f445c159150 0x7f445c1f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.581+0000 7f445affd700 1 --2- 172.21.15.40:0/3524938773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f445c0ab510 0x7f445c0c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.581+0000 7f445b7fe700 1 -- 172.21.15.40:0/3524938773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f445c0ab510 msgr2=0x7f445c0c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:01.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.581+0000 7f445b7fe700 1 --2- 172.21.15.40:0/3524938773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f445c0ab510 0x7f445c0c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.581+0000 7f445b7fe700 1 -- 172.21.15.40:0/3524938773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f445c0caf30 msgr2=0x7f445c097750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:01.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.581+0000 7f445b7fe700 1 --2- 172.21.15.40:0/3524938773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f445c0caf30 0x7f445c097750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.581+0000 7f445b7fe700 1 -- 172.21.15.40:0/3524938773 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c009450 con 0x7f445c159150 2023-11-08T08:24:01.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.581+0000 7f445a7fc700 1 --2- 172.21.15.40:0/3524938773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f445c0caf30 0x7f445c097750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:24:01.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.585+0000 7f445b7fe700 1 --2- 172.21.15.40:0/3524938773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f445c159150 0x7f445c1f5610 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f445400cab0 tx=0x7f445400cae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:01.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.585+0000 7f44537fe700 1 -- 172.21.15.40:0/3524938773 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4454007480 con 0x7f445c159150 2023-11-08T08:24:01.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.585+0000 7f44537fe700 1 -- 172.21.15.40:0/3524938773 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44540075e0 con 0x7f445c159150 2023-11-08T08:24:01.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.585+0000 7f445bfff700 1 -- 172.21.15.40:0/3524938773 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445c06b5e0 con 0x7f445c159150 2023-11-08T08:24:01.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.585+0000 7f44537fe700 1 -- 172.21.15.40:0/3524938773 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4454007950 con 0x7f445c159150 2023-11-08T08:24:01.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.585+0000 7f445bfff700 1 -- 172.21.15.40:0/3524938773 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445c0c90f0 con 0x7f445c159150 2023-11-08T08:24:01.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.585+0000 7f445bfff700 1 -- 172.21.15.40:0/3524938773 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f445c06a600 con 0x7f445c159150 2023-11-08T08:24:01.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.589+0000 7f44537fe700 1 -- 172.21.15.40:0/3524938773 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4454019030 con 0x7f445c159150 2023-11-08T08:24:01.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.589+0000 7f44537fe700 1 --2- 172.21.15.40:0/3524938773 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f443c064ee0 0x7f443c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.589+0000 7f44537fe700 1 -- 172.21.15.40:0/3524938773 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f4454081530 con 0x7f445c159150 2023-11-08T08:24:01.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.589+0000 7f44537fe700 1 --2- 172.21.15.40:0/3524938773 >> [v2:172.21.15.40:6824/35486,v1:172.21.15.40:6825/35486] conn(0x7f443c06b7f0 0x7f443c06dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.589+0000 7f44537fe700 1 -- 172.21.15.40:0/3524938773 --> [v2:172.21.15.40:6824/35486,v1:172.21.15.40:6825/35486] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f443c06e2f0 con 0x7f443c06b7f0 2023-11-08T08:24:01.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.589+0000 7f44537fe700 1 -- 172.21.15.40:0/3524938773 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44540818a0 con 0x7f445c159150 2023-11-08T08:24:01.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.589+0000 7f445affd700 1 --2- 172.21.15.40:0/3524938773 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f443c064ee0 0x7f443c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.589+0000 7f445a7fc700 1 --2- 172.21.15.40:0/3524938773 >> [v2:172.21.15.40:6824/35486,v1:172.21.15.40:6825/35486] conn(0x7f443c06b7f0 0x7f443c06dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.589+0000 7f445affd700 1 --2- 172.21.15.40:0/3524938773 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f443c064ee0 0x7f443c067390 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f444c013040 tx=0x7f444c008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:01.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.593+0000 7f445a7fc700 1 --2- 172.21.15.40:0/3524938773 >> [v2:172.21.15.40:6824/35486,v1:172.21.15.40:6825/35486] conn(0x7f443c06b7f0 0x7f443c06dc20 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:01.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.593+0000 7f44537fe700 1 -- 172.21.15.40:0/3524938773 <== osd.2 v2:172.21.15.40:6824/35486 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f443c06e2f0 con 0x7f443c06b7f0 2023-11-08T08:24:01.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.609+0000 7f445bfff700 1 -- 172.21.15.40:0/3524938773 --> [v2:172.21.15.40:6824/35486,v1:172.21.15.40:6825/35486] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f445c0962b0 con 0x7f443c06b7f0 2023-11-08T08:24:01.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.609+0000 7f44537fe700 1 -- 172.21.15.40:0/3524938773 <== osd.2 v2:172.21.15.40:6824/35486 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f445c0962b0 con 0x7f443c06b7f0 2023-11-08T08:24:01.618 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:24:01.618 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T08:24:01.619 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:24:01.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.609+0000 7f445bfff700 1 -- 172.21.15.40:0/3524938773 >> [v2:172.21.15.40:6824/35486,v1:172.21.15.40:6825/35486] conn(0x7f443c06b7f0 msgr2=0x7f443c06dc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:01.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.609+0000 7f445bfff700 1 --2- 172.21.15.40:0/3524938773 >> [v2:172.21.15.40:6824/35486,v1:172.21.15.40:6825/35486] conn(0x7f443c06b7f0 0x7f443c06dc20 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.609+0000 7f445bfff700 1 -- 172.21.15.40:0/3524938773 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f443c064ee0 msgr2=0x7f443c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:01.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.609+0000 7f445bfff700 1 --2- 172.21.15.40:0/3524938773 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f443c064ee0 0x7f443c067390 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f444c013040 tx=0x7f444c008070 comp rx=0 tx=0).stop 2023-11-08T08:24:01.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.609+0000 7f445bfff700 1 -- 172.21.15.40:0/3524938773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f445c159150 msgr2=0x7f445c1f5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:01.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.609+0000 7f445bfff700 1 --2- 172.21.15.40:0/3524938773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f445c159150 0x7f445c1f5610 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f445400cab0 tx=0x7f445400cae0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.609+0000 7f445b7fe700 1 -- 172.21.15.40:0/3524938773 reap_dead start 2023-11-08T08:24:01.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.613+0000 7f445bfff700 1 -- 172.21.15.40:0/3524938773 shutdown_connections 2023-11-08T08:24:01.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.613+0000 7f445bfff700 1 -- 172.21.15.40:0/3524938773 >> 172.21.15.40:0/3524938773 conn(0x7f445c0ce5b0 msgr2=0x7f445c0ab970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:01.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.613+0000 7f445bfff700 1 -- 172.21.15.40:0/3524938773 shutdown_connections 2023-11-08T08:24:01.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.613+0000 7f445bfff700 1 -- 172.21.15.40:0/3524938773 wait complete. 2023-11-08T08:24:01.639 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-08T08:24:01.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.817+0000 7f4f4caee700 1 Processor -- start 2023-11-08T08:24:01.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.821+0000 7f4f4caee700 1 -- start start 2023-11-08T08:24:01.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.821+0000 7f4f4caee700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4f481534a0 0x7f4f4815d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.821+0000 7f4f4caee700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f48160920 0x7f4f480b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.821+0000 7f4f4caee700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f4815d080 con 0x7f4f480b6030 2023-11-08T08:24:01.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.821+0000 7f4f4caee700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f48153870 con 0x7f4f481534a0 2023-11-08T08:24:01.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.821+0000 7f4f4caee700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f480b6450 con 0x7f4f48160920 2023-11-08T08:24:01.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.825+0000 7f4f4659c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4f481534a0 0x7f4f4815d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.825+0000 7f4f4659c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4f481534a0 0x7f4f4815d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43924/0 (socket says 172.21.15.40:43924) 2023-11-08T08:24:01.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.825+0000 7f4f4659c700 1 -- 172.21.15.40:0/860445949 learned_addr learned my addr 172.21.15.40:0/860445949 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:01.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.825+0000 7f4f45d9b700 1 --2- 172.21.15.40:0/860445949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f48160920 0x7f4f480b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.825+0000 7f4f4659c700 1 -- 172.21.15.40:0/860445949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f48160920 msgr2=0x7f4f480b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:01.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.825+0000 7f4f4659c700 1 --2- 172.21.15.40:0/860445949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f48160920 0x7f4f480b5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.825+0000 7f4f4659c700 1 -- 172.21.15.40:0/860445949 >> v1:172.21.15.169:6789/0 conn(0x7f4f480b6030 legacy=0x7f4f48070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:01.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.825+0000 7f4f4659c700 1 -- 172.21.15.40:0/860445949 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f480b53c0 con 0x7f4f481534a0 2023-11-08T08:24:01.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.825+0000 7f4f4659c700 1 --2- 172.21.15.40:0/860445949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f481534a0 0x7f4f4815d510 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009910 tx=0x7f4f38009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79247be8aff5d0dc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:01.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.825+0000 7f4f4559a700 1 -- 172.21.15.40:0/860445949 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38011830 con 0x7f4f481534a0 2023-11-08T08:24:01.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.825+0000 7f4f4559a700 1 -- 172.21.15.40:0/860445949 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f38011990 con 0x7f4f481534a0 2023-11-08T08:24:01.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.825+0000 7f4f4559a700 1 -- 172.21.15.40:0/860445949 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38011d40 con 0x7f4f481534a0 2023-11-08T08:24:01.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.825+0000 7f4f4caee700 1 -- 172.21.15.40:0/860445949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f481534a0 msgr2=0x7f4f4815d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:01.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.825+0000 7f4f4caee700 1 --2- 172.21.15.40:0/860445949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f481534a0 0x7f4f4815d510 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009910 tx=0x7f4f38009c20 comp rx=0 tx=0).stop 2023-11-08T08:24:01.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.829+0000 7f4f4caee700 1 -- 172.21.15.40:0/860445949 shutdown_connections 2023-11-08T08:24:01.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.829+0000 7f4f4caee700 1 --2- 172.21.15.40:0/860445949 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f48160920 0x7f4f480b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.829+0000 7f4f4caee700 1 --2- 172.21.15.40:0/860445949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f481534a0 0x7f4f4815d510 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.829+0000 7f4f4caee700 1 -- 172.21.15.40:0/860445949 >> 172.21.15.40:0/860445949 conn(0x7f4f4806bfc0 msgr2=0x7f4f4806c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:01.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.829+0000 7f4f4caee700 1 -- 172.21.15.40:0/860445949 shutdown_connections 2023-11-08T08:24:01.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.829+0000 7f4f4caee700 1 -- 172.21.15.40:0/860445949 wait complete. 2023-11-08T08:24:01.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.833+0000 7f4f4caee700 1 Processor -- start 2023-11-08T08:24:01.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.833+0000 7f4f4caee700 1 -- start start 2023-11-08T08:24:01.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.833+0000 7f4f4caee700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f480b6030 0x7f4f4814bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.833+0000 7f4f4caee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f481534a0 0x7f4f4814c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.833+0000 7f4f4caee700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f48160920 0x7f4f4814c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.833+0000 7f4f4caee700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f481e8b10 con 0x7f4f481534a0 2023-11-08T08:24:01.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.833+0000 7f4f4caee700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f481e8270 con 0x7f4f480b6030 2023-11-08T08:24:01.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.833+0000 7f4f4caee700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f481e8550 con 0x7f4f48160920 2023-11-08T08:24:01.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.833+0000 7f4f45d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f481534a0 0x7f4f4814c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.833+0000 7f4f45d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f481534a0 0x7f4f4814c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43936/0 (socket says 172.21.15.40:43936) 2023-11-08T08:24:01.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.833+0000 7f4f45d9b700 1 -- 172.21.15.40:0/2558675870 learned_addr learned my addr 172.21.15.40:0/2558675870 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:01.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.833+0000 7f4f4659c700 1 --2- 172.21.15.40:0/2558675870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f480b6030 0x7f4f4814bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.833+0000 7f4f46d9d700 1 --2- 172.21.15.40:0/2558675870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f48160920 0x7f4f4814c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.833+0000 7f4f45d9b700 1 -- 172.21.15.40:0/2558675870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f48160920 msgr2=0x7f4f4814c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:01.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.833+0000 7f4f45d9b700 1 --2- 172.21.15.40:0/2558675870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f48160920 0x7f4f4814c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.833+0000 7f4f45d9b700 1 -- 172.21.15.40:0/2558675870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f480b6030 msgr2=0x7f4f4814bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:01.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.833+0000 7f4f45d9b700 1 --2- 172.21.15.40:0/2558675870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f480b6030 0x7f4f4814bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:01.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.833+0000 7f4f45d9b700 1 -- 172.21.15.40:0/2558675870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f38009570 con 0x7f4f481534a0 2023-11-08T08:24:01.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.837+0000 7f4f45d9b700 1 --2- 172.21.15.40:0/2558675870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f481534a0 0x7f4f4814c310 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c00e9d0 tx=0x7f4f3c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:01.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.837+0000 7f4f36ffd700 1 -- 172.21.15.40:0/2558675870 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c009720 con 0x7f4f481534a0 2023-11-08T08:24:01.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.837+0000 7f4f36ffd700 1 -- 172.21.15.40:0/2558675870 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3c00ee70 con 0x7f4f481534a0 2023-11-08T08:24:01.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.837+0000 7f4f36ffd700 1 -- 172.21.15.40:0/2558675870 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c00ca50 con 0x7f4f481534a0 2023-11-08T08:24:01.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.837+0000 7f4f4caee700 1 -- 172.21.15.40:0/2558675870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f4815ca30 con 0x7f4f481534a0 2023-11-08T08:24:01.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.837+0000 7f4f4caee700 1 -- 172.21.15.40:0/2558675870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f480b56a0 con 0x7f4f481534a0 2023-11-08T08:24:01.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.837+0000 7f4f4caee700 1 -- 172.21.15.40:0/2558675870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f4815cd10 con 0x7f4f481534a0 2023-11-08T08:24:01.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.841+0000 7f4f36ffd700 1 -- 172.21.15.40:0/2558675870 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f4f3c019030 con 0x7f4f481534a0 2023-11-08T08:24:01.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.841+0000 7f4f36ffd700 1 --2- 172.21.15.40:0/2558675870 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4f28064ee0 0x7f4f28067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:01.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.841+0000 7f4f36ffd700 1 -- 172.21.15.40:0/2558675870 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f4f3c081c80 con 0x7f4f481534a0 2023-11-08T08:24:01.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.841+0000 7f4f4659c700 1 --2- 172.21.15.40:0/2558675870 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4f28064ee0 0x7f4f28067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:01.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.845+0000 7f4f4659c700 1 --2- 172.21.15.40:0/2558675870 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4f28064ee0 0x7f4f28067390 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009c60 tx=0x7f4f38008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:01.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:01.849+0000 7f4f36ffd700 1 -- 172.21.15.40:0/2558675870 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f4f3c04fe90 con 0x7f4f481534a0 2023-11-08T08:24:02.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.021+0000 7f4f4caee700 1 -- 172.21.15.40:0/2558675870 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f480726c0 con 0x7f4f28064ee0 2023-11-08T08:24:02.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.021+0000 7f4f36ffd700 1 -- 172.21.15.40:0/2558675870 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4f480726c0 con 0x7f4f28064ee0 2023-11-08T08:24:02.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.025+0000 7f4f4caee700 1 -- 172.21.15.40:0/2558675870 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4f28064ee0 msgr2=0x7f4f28067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.025+0000 7f4f4caee700 1 --2- 172.21.15.40:0/2558675870 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4f28064ee0 0x7f4f28067390 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009c60 tx=0x7f4f38008030 comp rx=0 tx=0).stop 2023-11-08T08:24:02.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.025+0000 7f4f4caee700 1 -- 172.21.15.40:0/2558675870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f481534a0 msgr2=0x7f4f4814c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.025+0000 7f4f4caee700 1 --2- 172.21.15.40:0/2558675870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f481534a0 0x7f4f4814c310 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c00e9d0 tx=0x7f4f3c00ece0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.025+0000 7f4f4caee700 1 -- 172.21.15.40:0/2558675870 shutdown_connections 2023-11-08T08:24:02.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.025+0000 7f4f4caee700 1 --2- 172.21.15.40:0/2558675870 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f4f28064ee0 0x7f4f28067390 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.025+0000 7f4f4caee700 1 --2- 172.21.15.40:0/2558675870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f48160920 0x7f4f4814c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.025+0000 7f4f4caee700 1 --2- 172.21.15.40:0/2558675870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f481534a0 0x7f4f4814c310 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.025+0000 7f4f4caee700 1 --2- 172.21.15.40:0/2558675870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f480b6030 0x7f4f4814bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.025+0000 7f4f4caee700 1 -- 172.21.15.40:0/2558675870 >> 172.21.15.40:0/2558675870 conn(0x7f4f4806bfc0 msgr2=0x7f4f4815bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:02.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.025+0000 7f4f4caee700 1 -- 172.21.15.40:0/2558675870 shutdown_connections 2023-11-08T08:24:02.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.025+0000 7f4f4caee700 1 -- 172.21.15.40:0/2558675870 wait complete. 2023-11-08T08:24:02.036 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 2 to deep-scrub 2023-11-08T08:24:02.053 INFO:tasks.ceph:Scrubbing osd.3 2023-11-08T08:24:02.054 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-08T08:24:02.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.233+0000 7faddd7ef700 1 Processor -- start 2023-11-08T08:24:02.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.233+0000 7faddd7ef700 1 -- start start 2023-11-08T08:24:02.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.237+0000 7faddd7ef700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fadd80ccb00 0x7fadd80ca940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.237+0000 7faddd7ef700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadd80ad8f0 0x7fadd80c1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.237+0000 7faddd7ef700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd80c1f20 con 0x7fadd80c1ac0 2023-11-08T08:24:02.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.237+0000 7faddd7ef700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd8160da0 con 0x7fadd80ccb00 2023-11-08T08:24:02.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.237+0000 7faddd7ef700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd8165140 con 0x7fadd80ad8f0 2023-11-08T08:24:02.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.237+0000 7fadd7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fadd80ccb00 0x7fadd80ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.237+0000 7fadd7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fadd80ccb00 0x7fadd80ca940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43944/0 (socket says 172.21.15.40:43944) 2023-11-08T08:24:02.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.237+0000 7fadd7fff700 1 -- 172.21.15.40:0/2456011874 learned_addr learned my addr 172.21.15.40:0/2456011874 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:02.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.237+0000 7fadcffff700 1 --2- 172.21.15.40:0/2456011874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadd80ad8f0 0x7fadd80c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.237+0000 7fadd7fff700 1 -- 172.21.15.40:0/2456011874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadd80ad8f0 msgr2=0x7fadd80c1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:02.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.237+0000 7fadd7fff700 1 --2- 172.21.15.40:0/2456011874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadd80ad8f0 0x7fadd80c1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.237+0000 7fadd7fff700 1 -- 172.21.15.40:0/2456011874 >> v1:172.21.15.40:6789/0 conn(0x7fadd80c1ac0 legacy=0x7fadd80cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:02.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.237+0000 7fadd7fff700 1 -- 172.21.15.40:0/2456011874 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd80cced0 con 0x7fadd80ccb00 2023-11-08T08:24:02.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.237+0000 7fadcffff700 1 --2- 172.21.15.40:0/2456011874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadd80ad8f0 0x7fadd80c1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:24:02.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.237+0000 7fadd7fff700 1 --2- 172.21.15.40:0/2456011874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadd80ccb00 0x7fadd80ca940 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009910 tx=0x7fadc8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdfab72eb642d0f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:02.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.237+0000 7fadd77fe700 1 -- 172.21.15.40:0/2456011874 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8005e50 con 0x7fadd80ccb00 2023-11-08T08:24:02.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.237+0000 7fadd77fe700 1 -- 172.21.15.40:0/2456011874 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc8009e60 con 0x7fadd80ccb00 2023-11-08T08:24:02.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.237+0000 7fadd77fe700 1 -- 172.21.15.40:0/2456011874 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc800b870 con 0x7fadd80ccb00 2023-11-08T08:24:02.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.241+0000 7faddd7ef700 1 -- 172.21.15.40:0/2456011874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadd80ccb00 msgr2=0x7fadd80ca940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.241+0000 7faddd7ef700 1 --2- 172.21.15.40:0/2456011874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadd80ccb00 0x7fadd80ca940 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009910 tx=0x7fadc8009c20 comp rx=0 tx=0).stop 2023-11-08T08:24:02.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.241+0000 7faddd7ef700 1 -- 172.21.15.40:0/2456011874 shutdown_connections 2023-11-08T08:24:02.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.241+0000 7faddd7ef700 1 --2- 172.21.15.40:0/2456011874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadd80ad8f0 0x7fadd80c1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.241+0000 7faddd7ef700 1 --2- 172.21.15.40:0/2456011874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadd80ccb00 0x7fadd80ca940 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.241+0000 7faddd7ef700 1 -- 172.21.15.40:0/2456011874 >> 172.21.15.40:0/2456011874 conn(0x7fadd80ce5b0 msgr2=0x7fadd80ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:02.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.241+0000 7faddd7ef700 1 -- 172.21.15.40:0/2456011874 shutdown_connections 2023-11-08T08:24:02.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.241+0000 7faddd7ef700 1 -- 172.21.15.40:0/2456011874 wait complete. 2023-11-08T08:24:02.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.241+0000 7faddd7ef700 1 Processor -- start 2023-11-08T08:24:02.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7faddd7ef700 1 -- start start 2023-11-08T08:24:02.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7faddd7ef700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadd80ad8f0 0x7fadd80af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7faddd7ef700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadd80c1ac0 0x7fadd815b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7faddd7ef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadd80ccb00 0x7fadd80c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7faddd7ef700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd80ae720 con 0x7fadd80ccb00 2023-11-08T08:24:02.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7faddd7ef700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd815b8b0 con 0x7fadd80c1ac0 2023-11-08T08:24:02.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7faddd7ef700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd8071e90 con 0x7fadd80ad8f0 2023-11-08T08:24:02.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7faddcfee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadd80ccb00 0x7fadd80c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7faddcfee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadd80ccb00 0x7fadd80c20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43960/0 (socket says 172.21.15.40:43960) 2023-11-08T08:24:02.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7faddcfee700 1 -- 172.21.15.40:0/74125805 learned_addr learned my addr 172.21.15.40:0/74125805 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:02.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7fadcffff700 1 --2- 172.21.15.40:0/74125805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadd80c1ac0 0x7fadd815b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7fadd7fff700 1 --2- 172.21.15.40:0/74125805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadd80ad8f0 0x7fadd80af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7faddcfee700 1 -- 172.21.15.40:0/74125805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadd80ad8f0 msgr2=0x7fadd80af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7faddcfee700 1 --2- 172.21.15.40:0/74125805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadd80ad8f0 0x7fadd80af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7faddcfee700 1 -- 172.21.15.40:0/74125805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadd80c1ac0 msgr2=0x7fadd815b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7faddcfee700 1 --2- 172.21.15.40:0/74125805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadd80c1ac0 0x7fadd815b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7faddcfee700 1 -- 172.21.15.40:0/74125805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc8009520 con 0x7fadd80ccb00 2023-11-08T08:24:02.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7fadcffff700 1 --2- 172.21.15.40:0/74125805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadd80c1ac0 0x7fadd815b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:24:02.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7fadd7fff700 1 --2- 172.21.15.40:0/74125805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadd80ad8f0 0x7fadd80af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:24:02.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7faddcfee700 1 --2- 172.21.15.40:0/74125805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadd80ccb00 0x7fadd80c20b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fadd000cb50 tx=0x7fadd000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:02.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7fadd57fa700 1 -- 172.21.15.40:0/74125805 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0003c20 con 0x7fadd80ccb00 2023-11-08T08:24:02.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7fadd57fa700 1 -- 172.21.15.40:0/74125805 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd0003d80 con 0x7fadd80ccb00 2023-11-08T08:24:02.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7faddd7ef700 1 -- 172.21.15.40:0/74125805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd8164810 con 0x7fadd80ccb00 2023-11-08T08:24:02.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7fadd57fa700 1 -- 172.21.15.40:0/74125805 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd000ab70 con 0x7fadd80ccb00 2023-11-08T08:24:02.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.245+0000 7faddd7ef700 1 -- 172.21.15.40:0/74125805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd8164530 con 0x7fadd80ccb00 2023-11-08T08:24:02.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.249+0000 7faddd7ef700 1 -- 172.21.15.40:0/74125805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fadd8164af0 con 0x7fadd80ccb00 2023-11-08T08:24:02.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.249+0000 7fadd57fa700 1 -- 172.21.15.40:0/74125805 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fadd0010050 con 0x7fadd80ccb00 2023-11-08T08:24:02.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.249+0000 7fadd57fa700 1 --2- 172.21.15.40:0/74125805 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fadb8064ee0 0x7fadb8067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.249+0000 7fadd57fa700 1 -- 172.21.15.40:0/74125805 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fadd0081d20 con 0x7fadd80ccb00 2023-11-08T08:24:02.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.249+0000 7fadd57fa700 1 --2- 172.21.15.40:0/74125805 >> [v2:172.21.15.40:6816/35484,v1:172.21.15.40:6817/35484] conn(0x7fadb806b7f0 0x7fadb806dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.249+0000 7fadd57fa700 1 -- 172.21.15.40:0/74125805 --> [v2:172.21.15.40:6816/35484,v1:172.21.15.40:6817/35484] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fadb806e2f0 con 0x7fadb806b7f0 2023-11-08T08:24:02.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.249+0000 7fadd57fa700 1 -- 172.21.15.40:0/74125805 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadd0087070 con 0x7fadd80ccb00 2023-11-08T08:24:02.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.249+0000 7fadd7fff700 1 --2- 172.21.15.40:0/74125805 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fadb8064ee0 0x7fadb8067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.249+0000 7fadcffff700 1 --2- 172.21.15.40:0/74125805 >> [v2:172.21.15.40:6816/35484,v1:172.21.15.40:6817/35484] conn(0x7fadb806b7f0 0x7fadb806dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.253+0000 7fadcffff700 1 --2- 172.21.15.40:0/74125805 >> [v2:172.21.15.40:6816/35484,v1:172.21.15.40:6817/35484] conn(0x7fadb806b7f0 0x7fadb806dc20 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:02.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.253+0000 7fadd7fff700 1 --2- 172.21.15.40:0/74125805 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fadb8064ee0 0x7fadb8067390 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009fd0 tx=0x7fadc8008070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:02.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.253+0000 7fadd57fa700 1 -- 172.21.15.40:0/74125805 <== osd.3 v2:172.21.15.40:6816/35484 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fadb806e2f0 con 0x7fadb806b7f0 2023-11-08T08:24:02.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.269+0000 7faddd7ef700 1 -- 172.21.15.40:0/74125805 --> [v2:172.21.15.40:6816/35484,v1:172.21.15.40:6817/35484] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fadd8073cb0 con 0x7fadb806b7f0 2023-11-08T08:24:02.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.273+0000 7fadd57fa700 1 -- 172.21.15.40:0/74125805 <== osd.3 v2:172.21.15.40:6816/35484 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fadd8073cb0 con 0x7fadb806b7f0 2023-11-08T08:24:02.279 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:24:02.279 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T08:24:02.280 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:24:02.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.273+0000 7faddd7ef700 1 -- 172.21.15.40:0/74125805 >> [v2:172.21.15.40:6816/35484,v1:172.21.15.40:6817/35484] conn(0x7fadb806b7f0 msgr2=0x7fadb806dc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.273+0000 7faddd7ef700 1 --2- 172.21.15.40:0/74125805 >> [v2:172.21.15.40:6816/35484,v1:172.21.15.40:6817/35484] conn(0x7fadb806b7f0 0x7fadb806dc20 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.273+0000 7faddd7ef700 1 -- 172.21.15.40:0/74125805 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fadb8064ee0 msgr2=0x7fadb8067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.273+0000 7faddd7ef700 1 --2- 172.21.15.40:0/74125805 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fadb8064ee0 0x7fadb8067390 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009fd0 tx=0x7fadc8008070 comp rx=0 tx=0).stop 2023-11-08T08:24:02.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.273+0000 7faddd7ef700 1 -- 172.21.15.40:0/74125805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadd80ccb00 msgr2=0x7fadd80c20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.273+0000 7faddd7ef700 1 --2- 172.21.15.40:0/74125805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadd80ccb00 0x7fadd80c20b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fadd000cb50 tx=0x7fadd000cf10 comp rx=0 tx=0).stop 2023-11-08T08:24:02.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.273+0000 7faddcfee700 1 -- 172.21.15.40:0/74125805 reap_dead start 2023-11-08T08:24:02.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.273+0000 7faddd7ef700 1 -- 172.21.15.40:0/74125805 shutdown_connections 2023-11-08T08:24:02.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.273+0000 7faddd7ef700 1 -- 172.21.15.40:0/74125805 >> 172.21.15.40:0/74125805 conn(0x7fadd80ce5b0 msgr2=0x7fadd8160530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:02.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.273+0000 7faddd7ef700 1 -- 172.21.15.40:0/74125805 shutdown_connections 2023-11-08T08:24:02.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.273+0000 7faddd7ef700 1 -- 172.21.15.40:0/74125805 wait complete. 2023-11-08T08:24:02.301 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-08T08:24:02.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.473+0000 7fb74d1bc700 1 Processor -- start 2023-11-08T08:24:02.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.473+0000 7fb74d1bc700 1 -- start start 2023-11-08T08:24:02.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.477+0000 7fb74d1bc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb74815c640 0x7fb7480d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.477+0000 7fb74d1bc700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb748071aa0 0x7fb74806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.477+0000 7fb74d1bc700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb74806c600 con 0x7fb748069e00 2023-11-08T08:24:02.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.477+0000 7fb74d1bc700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb74806cef0 con 0x7fb74815c640 2023-11-08T08:24:02.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.477+0000 7fb74d1bc700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7480722f0 con 0x7fb748071aa0 2023-11-08T08:24:02.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.477+0000 7fb747fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb74815c640 0x7fb7480d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.477+0000 7fb74c9bb700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb748069e00 0x7fb7480d0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57906/0 (socket says 172.21.15.40:57906) 2023-11-08T08:24:02.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.477+0000 7fb74c9bb700 1 -- 172.21.15.40:0/1484124150 learned_addr learned my addr 172.21.15.40:0/1484124150 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:24:02.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.477+0000 7fb7477fe700 1 --2- 172.21.15.40:0/1484124150 >> v2:172.21.15.169:3300/0 conn(0x7fb748071aa0 0x7fb74806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.477+0000 7fb747fff700 1 -- 172.21.15.40:0/1484124150 >> v2:172.21.15.169:3300/0 conn(0x7fb748071aa0 msgr2=0x7fb74806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:02.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.477+0000 7fb747fff700 1 --2- 172.21.15.40:0/1484124150 >> v2:172.21.15.169:3300/0 conn(0x7fb748071aa0 0x7fb74806d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.477+0000 7fb747fff700 1 -- 172.21.15.40:0/1484124150 >> v1:172.21.15.40:6789/0 conn(0x7fb748069e00 legacy=0x7fb7480d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.477+0000 7fb747fff700 1 -- 172.21.15.40:0/1484124150 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb74815ca10 con 0x7fb74815c640 2023-11-08T08:24:02.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.477+0000 7fb746ffd700 1 -- 172.21.15.40:0/1484124150 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3930062332 0 0) 0x7fb74806c600 con 0x7fb748069e00 2023-11-08T08:24:02.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.481+0000 7fb747fff700 1 --2- 172.21.15.40:0/1484124150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb74815c640 0x7fb7480d31e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb748070e70 tx=0x7fb738009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c1f60ed8fc893e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:02.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.481+0000 7fb746ffd700 1 -- 172.21.15.40:0/1484124150 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb738011830 con 0x7fb74815c640 2023-11-08T08:24:02.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.481+0000 7fb746ffd700 1 -- 172.21.15.40:0/1484124150 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb738011990 con 0x7fb74815c640 2023-11-08T08:24:02.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.481+0000 7fb746ffd700 1 -- 172.21.15.40:0/1484124150 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb738011d20 con 0x7fb74815c640 2023-11-08T08:24:02.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.481+0000 7fb74d1bc700 1 -- 172.21.15.40:0/1484124150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb74815c640 msgr2=0x7fb7480d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.481+0000 7fb74d1bc700 1 --2- 172.21.15.40:0/1484124150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb74815c640 0x7fb7480d31e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb748070e70 tx=0x7fb738009b30 comp rx=0 tx=0).stop 2023-11-08T08:24:02.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.481+0000 7fb74d1bc700 1 -- 172.21.15.40:0/1484124150 shutdown_connections 2023-11-08T08:24:02.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.481+0000 7fb74d1bc700 1 --2- 172.21.15.40:0/1484124150 >> v2:172.21.15.169:3300/0 conn(0x7fb748071aa0 0x7fb74806d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.481+0000 7fb74d1bc700 1 --2- 172.21.15.40:0/1484124150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb74815c640 0x7fb7480d31e0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.481+0000 7fb74d1bc700 1 -- 172.21.15.40:0/1484124150 >> 172.21.15.40:0/1484124150 conn(0x7fb7480b6520 msgr2=0x7fb748071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:02.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.481+0000 7fb74d1bc700 1 -- 172.21.15.40:0/1484124150 shutdown_connections 2023-11-08T08:24:02.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.481+0000 7fb74d1bc700 1 -- 172.21.15.40:0/1484124150 wait complete. 2023-11-08T08:24:02.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.485+0000 7fb74d1bc700 1 Processor -- start 2023-11-08T08:24:02.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.485+0000 7fb74d1bc700 1 -- start start 2023-11-08T08:24:02.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb74d1bc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb748069e00 0x7fb748156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb74d1bc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb748071aa0 0x7fb7480c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb74d1bc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb74815c640 0x7fb7480b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb74d1bc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7481e4740 con 0x7fb74815c640 2023-11-08T08:24:02.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb74d1bc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7480ca9e0 con 0x7fb748071aa0 2023-11-08T08:24:02.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb74d1bc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb748157390 con 0x7fb748069e00 2023-11-08T08:24:02.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb74c9bb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb74815c640 0x7fb7480b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb74c9bb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb74815c640 0x7fb7480b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43978/0 (socket says 172.21.15.40:43978) 2023-11-08T08:24:02.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb74c9bb700 1 -- 172.21.15.40:0/2633934404 learned_addr learned my addr 172.21.15.40:0/2633934404 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:02.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb747fff700 1 --2- 172.21.15.40:0/2633934404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb748069e00 0x7fb748156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb7477fe700 1 --2- 172.21.15.40:0/2633934404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb748071aa0 0x7fb7480c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb74c9bb700 1 -- 172.21.15.40:0/2633934404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb748069e00 msgr2=0x7fb748156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb74c9bb700 1 --2- 172.21.15.40:0/2633934404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb748069e00 0x7fb748156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb74c9bb700 1 -- 172.21.15.40:0/2633934404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb748071aa0 msgr2=0x7fb7480c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb74c9bb700 1 --2- 172.21.15.40:0/2633934404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb748071aa0 0x7fb7480c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb74c9bb700 1 -- 172.21.15.40:0/2633934404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7380094b0 con 0x7fb74815c640 2023-11-08T08:24:02.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb74c9bb700 1 --2- 172.21.15.40:0/2633934404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb74815c640 0x7fb7480b2380 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb74000cbd0 tx=0x7fb74000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:02.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb744ff9700 1 -- 172.21.15.40:0/2633934404 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74000aa60 con 0x7fb74815c640 2023-11-08T08:24:02.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb744ff9700 1 -- 172.21.15.40:0/2633934404 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb74000abe0 con 0x7fb74815c640 2023-11-08T08:24:02.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb74d1bc700 1 -- 172.21.15.40:0/2633934404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74806c830 con 0x7fb74815c640 2023-11-08T08:24:02.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb744ff9700 1 -- 172.21.15.40:0/2633934404 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7400070d0 con 0x7fb74815c640 2023-11-08T08:24:02.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.489+0000 7fb74d1bc700 1 -- 172.21.15.40:0/2633934404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb748072010 con 0x7fb74815c640 2023-11-08T08:24:02.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.493+0000 7fb74d1bc700 1 -- 172.21.15.40:0/2633934404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb74806cb10 con 0x7fb74815c640 2023-11-08T08:24:02.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.493+0000 7fb744ff9700 1 -- 172.21.15.40:0/2633934404 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fb74001a030 con 0x7fb74815c640 2023-11-08T08:24:02.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.493+0000 7fb744ff9700 1 --2- 172.21.15.40:0/2633934404 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb734064ee0 0x7fb734067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.493+0000 7fb744ff9700 1 -- 172.21.15.40:0/2633934404 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fb740081c40 con 0x7fb74815c640 2023-11-08T08:24:02.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.497+0000 7fb747fff700 1 --2- 172.21.15.40:0/2633934404 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb734064ee0 0x7fb734067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.497+0000 7fb747fff700 1 --2- 172.21.15.40:0/2633934404 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb734064ee0 0x7fb734067390 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb738009fd0 tx=0x7fb738008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:02.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.501+0000 7fb744ff9700 1 -- 172.21.15.40:0/2633934404 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fb74004ff50 con 0x7fb74815c640 2023-11-08T08:24:02.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.677+0000 7fb74d1bc700 1 -- 172.21.15.40:0/2633934404 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7481e4880 con 0x7fb734064ee0 2023-11-08T08:24:02.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.677+0000 7fb744ff9700 1 -- 172.21.15.40:0/2633934404 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb7481e4880 con 0x7fb734064ee0 2023-11-08T08:24:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.681+0000 7fb74d1bc700 1 -- 172.21.15.40:0/2633934404 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb734064ee0 msgr2=0x7fb734067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.681+0000 7fb74d1bc700 1 --2- 172.21.15.40:0/2633934404 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb734064ee0 0x7fb734067390 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb738009fd0 tx=0x7fb738008030 comp rx=0 tx=0).stop 2023-11-08T08:24:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.681+0000 7fb74d1bc700 1 -- 172.21.15.40:0/2633934404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb74815c640 msgr2=0x7fb7480b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.681+0000 7fb74d1bc700 1 --2- 172.21.15.40:0/2633934404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb74815c640 0x7fb7480b2380 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb74000cbd0 tx=0x7fb74000cee0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.681+0000 7fb74d1bc700 1 -- 172.21.15.40:0/2633934404 shutdown_connections 2023-11-08T08:24:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.681+0000 7fb74d1bc700 1 --2- 172.21.15.40:0/2633934404 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fb734064ee0 0x7fb734067390 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.681+0000 7fb74d1bc700 1 --2- 172.21.15.40:0/2633934404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb74815c640 0x7fb7480b2380 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.681+0000 7fb74d1bc700 1 --2- 172.21.15.40:0/2633934404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb748071aa0 0x7fb7480c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.681+0000 7fb74d1bc700 1 --2- 172.21.15.40:0/2633934404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb748069e00 0x7fb748156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.681+0000 7fb74d1bc700 1 -- 172.21.15.40:0/2633934404 >> 172.21.15.40:0/2633934404 conn(0x7fb7480b6520 msgr2=0x7fb7480c90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:02.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.681+0000 7fb74d1bc700 1 -- 172.21.15.40:0/2633934404 shutdown_connections 2023-11-08T08:24:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.681+0000 7fb74d1bc700 1 -- 172.21.15.40:0/2633934404 wait complete. 2023-11-08T08:24:02.690 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 3 to deep-scrub 2023-11-08T08:24:02.710 INFO:tasks.ceph:Scrubbing osd.4 2023-11-08T08:24:02.711 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-08T08:24:02.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.897+0000 7f227ada8700 1 Processor -- start 2023-11-08T08:24:02.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.901+0000 7f227ada8700 1 -- start start 2023-11-08T08:24:02.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.901+0000 7f227ada8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2274160ba0 0x7f2274167250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.901+0000 7f227ada8700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22740b20e0 con 0x7f227415c5f0 2023-11-08T08:24:02.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.901+0000 7f227ada8700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22740c9c10 con 0x7f22740cc060 2023-11-08T08:24:02.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.901+0000 7f227ada8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22740b55c0 con 0x7f2274160ba0 2023-11-08T08:24:02.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.901+0000 7f22795a5700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f227415c5f0 0x7f22740cd550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57922/0 (socket says 172.21.15.40:57922) 2023-11-08T08:24:02.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.901+0000 7f22795a5700 1 -- 172.21.15.40:0/1760669240 learned_addr learned my addr 172.21.15.40:0/1760669240 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:24:02.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.901+0000 7f227a5a7700 1 --2- 172.21.15.40:0/1760669240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2274160ba0 0x7f2274167250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.901+0000 7f2278da4700 1 -- 172.21.15.40:0/1760669240 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276294072 0 0) 0x7f22740b20e0 con 0x7f227415c5f0 2023-11-08T08:24:02.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.901+0000 7f2278da4700 1 -- 172.21.15.40:0/1760669240 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22600035d0 con 0x7f227415c5f0 2023-11-08T08:24:02.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2278da4700 1 -- 172.21.15.40:0/1760669240 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020655897 0 0) 0x7f22740c9c10 con 0x7f22740cc060 2023-11-08T08:24:02.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2278da4700 1 -- 172.21.15.40:0/1760669240 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22740b20e0 con 0x7f22740cc060 2023-11-08T08:24:02.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2278da4700 1 -- 172.21.15.40:0/1760669240 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 452354898 0 0) 0x7f22600035d0 con 0x7f227415c5f0 2023-11-08T08:24:02.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2278da4700 1 -- 172.21.15.40:0/1760669240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2274160ba0 msgr2=0x7f2274167250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:02.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2278da4700 1 --2- 172.21.15.40:0/1760669240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2274160ba0 0x7f2274167250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2278da4700 1 -- 172.21.15.40:0/1760669240 >> v1:172.21.15.169:6789/0 conn(0x7f22740cc060 legacy=0x7f22740cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2278da4700 1 -- 172.21.15.40:0/1760669240 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22740b4800 con 0x7f227415c5f0 2023-11-08T08:24:02.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2278da4700 1 -- 172.21.15.40:0/1760669240 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f2264001670 con 0x7f227415c5f0 2023-11-08T08:24:02.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2278da4700 1 -- 172.21.15.40:0/1760669240 >> v1:172.21.15.40:6789/0 conn(0x7f227415c5f0 legacy=0x7f22740cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2278da4700 1 --2- 172.21.15.40:0/1760669240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2260003be0 0x7f2260003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2278da4700 1 --2- 172.21.15.40:0/1760669240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2274160ba0 0x7f2260004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2278da4700 1 --2- 172.21.15.40:0/1760669240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2260004db0 0x7f2260005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2278da4700 1 -- 172.21.15.40:0/1760669240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226000bb00 con 0x7f2260003be0 2023-11-08T08:24:02.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2278da4700 1 -- 172.21.15.40:0/1760669240 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226000bc70 con 0x7f2260004db0 2023-11-08T08:24:02.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2278da4700 1 -- 172.21.15.40:0/1760669240 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226000be30 con 0x7f2274160ba0 2023-11-08T08:24:02.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2279da6700 1 --2- 172.21.15.40:0/1760669240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2260003be0 0x7f2260003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f22795a5700 1 --2- 172.21.15.40:0/1760669240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2274160ba0 0x7f2260004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f227a5a7700 1 --2- 172.21.15.40:0/1760669240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2260004db0 0x7f2260005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2279da6700 1 -- 172.21.15.40:0/1760669240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2274160ba0 msgr2=0x7f2260004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:02.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2279da6700 1 --2- 172.21.15.40:0/1760669240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2274160ba0 0x7f2260004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2279da6700 1 -- 172.21.15.40:0/1760669240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2260004db0 msgr2=0x7f2260005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:02.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2279da6700 1 --2- 172.21.15.40:0/1760669240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2260004db0 0x7f2260005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2279da6700 1 -- 172.21.15.40:0/1760669240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22600035d0 con 0x7f2260003be0 2023-11-08T08:24:02.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f227a5a7700 1 --2- 172.21.15.40:0/1760669240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2260004db0 0x7f2260005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:24:02.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f22795a5700 1 --2- 172.21.15.40:0/1760669240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2274160ba0 0x7f2260004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:24:02.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2279da6700 1 --2- 172.21.15.40:0/1760669240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2260003be0 0x7f2260003fd0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2270001960 tx=0x7f22600058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d443b8f6033cc354 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:02.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2278da4700 1 -- 172.21.15.40:0/1760669240 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270001450 con 0x7f2260003be0 2023-11-08T08:24:02.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2278da4700 1 -- 172.21.15.40:0/1760669240 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22700052f0 con 0x7f2260003be0 2023-11-08T08:24:02.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f2278da4700 1 -- 172.21.15.40:0/1760669240 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22700036a0 con 0x7f2260003be0 2023-11-08T08:24:02.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f227ada8700 1 -- 172.21.15.40:0/1760669240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2260003be0 msgr2=0x7f2260003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f227ada8700 1 --2- 172.21.15.40:0/1760669240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2260003be0 0x7f2260003fd0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2270001960 tx=0x7f22600058a0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f227a5a7700 1 -- 172.21.15.40:0/1760669240 reap_dead start 2023-11-08T08:24:02.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f227ada8700 1 -- 172.21.15.40:0/1760669240 shutdown_connections 2023-11-08T08:24:02.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f227ada8700 1 -- 172.21.15.40:0/1760669240 >> 172.21.15.40:0/1760669240 conn(0x7f227406e100 msgr2=0x7f227406e500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:02.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f227ada8700 1 -- 172.21.15.40:0/1760669240 shutdown_connections 2023-11-08T08:24:02.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.905+0000 7f227ada8700 1 -- 172.21.15.40:0/1760669240 wait complete. 2023-11-08T08:24:02.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.909+0000 7f227ada8700 1 Processor -- start 2023-11-08T08:24:02.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.909+0000 7f227ada8700 1 -- start start 2023-11-08T08:24:02.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.909+0000 7f227ada8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2260003be0 0x7f227415ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.909+0000 7f227ada8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22740ca160 0x7f22741f4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.909+0000 7f227ada8700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22740b2b20 0x7f22741f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.909+0000 7f227ada8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22740af090 con 0x7f2260003be0 2023-11-08T08:24:02.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.909+0000 7f227ada8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227415ac20 con 0x7f22740ca160 2023-11-08T08:24:02.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.909+0000 7f227ada8700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227415af00 con 0x7f22740b2b20 2023-11-08T08:24:02.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.909+0000 7f2279da6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2260003be0 0x7f227415ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.909+0000 7f2279da6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2260003be0 0x7f227415ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43992/0 (socket says 172.21.15.40:43992) 2023-11-08T08:24:02.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.909+0000 7f2279da6700 1 -- 172.21.15.40:0/219619545 learned_addr learned my addr 172.21.15.40:0/219619545 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:02.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.909+0000 7f22795a5700 1 --2- 172.21.15.40:0/219619545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22740ca160 0x7f22741f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.909+0000 7f227a5a7700 1 --2- 172.21.15.40:0/219619545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22740b2b20 0x7f22741f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.913+0000 7f2279da6700 1 -- 172.21.15.40:0/219619545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22740b2b20 msgr2=0x7f22741f4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.913+0000 7f2279da6700 1 --2- 172.21.15.40:0/219619545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22740b2b20 0x7f22741f4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.913+0000 7f2279da6700 1 -- 172.21.15.40:0/219619545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22740ca160 msgr2=0x7f22741f4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.913+0000 7f2279da6700 1 --2- 172.21.15.40:0/219619545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22740ca160 0x7f22741f4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.913+0000 7f2279da6700 1 -- 172.21.15.40:0/219619545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22740b4dc0 con 0x7f2260003be0 2023-11-08T08:24:02.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.913+0000 7f2279da6700 1 --2- 172.21.15.40:0/219619545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2260003be0 0x7f227415ee60 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f227000c3c0 tx=0x7f22700012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:02.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.913+0000 7f226a7fc700 1 -- 172.21.15.40:0/219619545 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270001450 con 0x7f2260003be0 2023-11-08T08:24:02.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.913+0000 7f226a7fc700 1 -- 172.21.15.40:0/219619545 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2270006c90 con 0x7f2260003be0 2023-11-08T08:24:02.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.913+0000 7f226a7fc700 1 -- 172.21.15.40:0/219619545 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270007040 con 0x7f2260003be0 2023-11-08T08:24:02.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.913+0000 7f227ada8700 1 -- 172.21.15.40:0/219619545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22740b4ae0 con 0x7f2260003be0 2023-11-08T08:24:02.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.913+0000 7f227ada8700 1 -- 172.21.15.40:0/219619545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22740b50a0 con 0x7f2260003be0 2023-11-08T08:24:02.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.913+0000 7f227ada8700 1 -- 172.21.15.40:0/219619545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2274073cb0 con 0x7f2260003be0 2023-11-08T08:24:02.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.917+0000 7f226a7fc700 1 -- 172.21.15.40:0/219619545 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f227002e070 con 0x7f2260003be0 2023-11-08T08:24:02.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.917+0000 7f226a7fc700 1 --2- 172.21.15.40:0/219619545 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2258064ee0 0x7f2258067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.917+0000 7f226a7fc700 1 -- 172.21.15.40:0/219619545 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f2270082c30 con 0x7f2260003be0 2023-11-08T08:24:02.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.917+0000 7f226a7fc700 1 --2- 172.21.15.40:0/219619545 >> [v2:172.21.15.169:6811/35790,v1:172.21.15.169:6814/35790] conn(0x7f225806b7f0 0x7f225806dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:02.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.917+0000 7f226a7fc700 1 -- 172.21.15.40:0/219619545 --> [v2:172.21.15.169:6811/35790,v1:172.21.15.169:6814/35790] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f225806e2f0 con 0x7f225806b7f0 2023-11-08T08:24:02.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.917+0000 7f226a7fc700 1 -- 172.21.15.40:0/219619545 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22700872a0 con 0x7f2260003be0 2023-11-08T08:24:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.917+0000 7f22795a5700 1 --2- 172.21.15.40:0/219619545 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2258064ee0 0x7f2258067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.917+0000 7f227a5a7700 1 --2- 172.21.15.40:0/219619545 >> [v2:172.21.15.169:6811/35790,v1:172.21.15.169:6814/35790] conn(0x7f225806b7f0 0x7f225806dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:02.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.921+0000 7f22795a5700 1 --2- 172.21.15.40:0/219619545 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2258064ee0 0x7f2258067390 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f22741545e0 tx=0x7f2264007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:02.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.921+0000 7f227a5a7700 1 --2- 172.21.15.40:0/219619545 >> [v2:172.21.15.169:6811/35790,v1:172.21.15.169:6814/35790] conn(0x7f225806b7f0 0x7f225806dc20 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:02.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.921+0000 7f226a7fc700 1 -- 172.21.15.40:0/219619545 <== osd.4 v2:172.21.15.169:6811/35790 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f225806e2f0 con 0x7f225806b7f0 2023-11-08T08:24:02.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.941+0000 7f227ada8700 1 -- 172.21.15.40:0/219619545 --> [v2:172.21.15.169:6811/35790,v1:172.21.15.169:6814/35790] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f22740c6400 con 0x7f225806b7f0 2023-11-08T08:24:02.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.941+0000 7f226a7fc700 1 -- 172.21.15.40:0/219619545 <== osd.4 v2:172.21.15.169:6811/35790 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f22740c6400 con 0x7f225806b7f0 2023-11-08T08:24:02.948 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:24:02.949 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T08:24:02.949 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:24:02.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.941+0000 7f227ada8700 1 -- 172.21.15.40:0/219619545 >> [v2:172.21.15.169:6811/35790,v1:172.21.15.169:6814/35790] conn(0x7f225806b7f0 msgr2=0x7f225806dc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.941+0000 7f227ada8700 1 --2- 172.21.15.40:0/219619545 >> [v2:172.21.15.169:6811/35790,v1:172.21.15.169:6814/35790] conn(0x7f225806b7f0 0x7f225806dc20 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.941+0000 7f227ada8700 1 -- 172.21.15.40:0/219619545 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2258064ee0 msgr2=0x7f2258067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.941+0000 7f227ada8700 1 --2- 172.21.15.40:0/219619545 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f2258064ee0 0x7f2258067390 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f22741545e0 tx=0x7f2264007040 comp rx=0 tx=0).stop 2023-11-08T08:24:02.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.941+0000 7f227ada8700 1 -- 172.21.15.40:0/219619545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2260003be0 msgr2=0x7f227415ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:02.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.941+0000 7f227ada8700 1 --2- 172.21.15.40:0/219619545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2260003be0 0x7f227415ee60 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f227000c3c0 tx=0x7f22700012d0 comp rx=0 tx=0).stop 2023-11-08T08:24:02.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.941+0000 7f227a5a7700 1 -- 172.21.15.40:0/219619545 reap_dead start 2023-11-08T08:24:02.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.941+0000 7f227ada8700 1 -- 172.21.15.40:0/219619545 shutdown_connections 2023-11-08T08:24:02.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.941+0000 7f227ada8700 1 -- 172.21.15.40:0/219619545 >> 172.21.15.40:0/219619545 conn(0x7f227406e100 msgr2=0x7f227415fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:02.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.941+0000 7f227ada8700 1 -- 172.21.15.40:0/219619545 shutdown_connections 2023-11-08T08:24:02.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:02.945+0000 7f227ada8700 1 -- 172.21.15.40:0/219619545 wait complete. 2023-11-08T08:24:02.974 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-08T08:24:03.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.153+0000 7f14fbbf7700 1 Processor -- start 2023-11-08T08:24:03.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.153+0000 7f14fbbf7700 1 -- start start 2023-11-08T08:24:03.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.153+0000 7f14fbbf7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f14f4163cb0 0x7f14f4164cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:03.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.153+0000 7f14fbbf7700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f14f40cdfb0 0x7f14f406cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:03.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.153+0000 7f14fbbf7700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14f4164080 con 0x7f14f406d2f0 2023-11-08T08:24:03.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.153+0000 7f14fbbf7700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f40cc290 con 0x7f14f4163cb0 2023-11-08T08:24:03.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.153+0000 7f14fbbf7700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f406d770 con 0x7f14f40cdfb0 2023-11-08T08:24:03.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.153+0000 7f14f9993700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f14f4163cb0 0x7f14f4164cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:03.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.153+0000 7f14fa194700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f14f406d2f0 0x7f14f406bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57924/0 (socket says 172.21.15.40:57924) 2023-11-08T08:24:03.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.153+0000 7f14fa194700 1 -- 172.21.15.40:0/1413708205 learned_addr learned my addr 172.21.15.40:0/1413708205 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:24:03.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.153+0000 7f14f9192700 1 --2- 172.21.15.40:0/1413708205 >> v2:172.21.15.169:3300/0 conn(0x7f14f40cdfb0 0x7f14f406cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:03.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.157+0000 7f14f9993700 1 -- 172.21.15.40:0/1413708205 >> v2:172.21.15.169:3300/0 conn(0x7f14f40cdfb0 msgr2=0x7f14f406cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:03.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.157+0000 7f14f9993700 1 --2- 172.21.15.40:0/1413708205 >> v2:172.21.15.169:3300/0 conn(0x7f14f40cdfb0 0x7f14f406cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:03.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.157+0000 7f14f9993700 1 -- 172.21.15.40:0/1413708205 >> v1:172.21.15.40:6789/0 conn(0x7f14f406d2f0 legacy=0x7f14f406bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:03.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.157+0000 7f14f9993700 1 -- 172.21.15.40:0/1413708205 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f4069bf0 con 0x7f14f4163cb0 2023-11-08T08:24:03.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.157+0000 7f14f8991700 1 -- 172.21.15.40:0/1413708205 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261922282 0 0) 0x7f14f4164080 con 0x7f14f406d2f0 2023-11-08T08:24:03.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.157+0000 7f14f9993700 1 --2- 172.21.15.40:0/1413708205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14f4163cb0 0x7f14f4164cb0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f14f406f480 tx=0x7f14e4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ecc22f31d0e806 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:03.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.157+0000 7f14f8991700 1 -- 172.21.15.40:0/1413708205 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e4011830 con 0x7f14f4163cb0 2023-11-08T08:24:03.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.157+0000 7f14f8991700 1 -- 172.21.15.40:0/1413708205 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e4011990 con 0x7f14f4163cb0 2023-11-08T08:24:03.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.157+0000 7f14f8991700 1 -- 172.21.15.40:0/1413708205 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e4011d20 con 0x7f14f4163cb0 2023-11-08T08:24:03.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.157+0000 7f14fbbf7700 1 -- 172.21.15.40:0/1413708205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14f4163cb0 msgr2=0x7f14f4164cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:03.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.157+0000 7f14fbbf7700 1 --2- 172.21.15.40:0/1413708205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14f4163cb0 0x7f14f4164cb0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f14f406f480 tx=0x7f14e4009b30 comp rx=0 tx=0).stop 2023-11-08T08:24:03.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.157+0000 7f14fbbf7700 1 -- 172.21.15.40:0/1413708205 shutdown_connections 2023-11-08T08:24:03.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.157+0000 7f14fbbf7700 1 --2- 172.21.15.40:0/1413708205 >> v2:172.21.15.169:3300/0 conn(0x7f14f40cdfb0 0x7f14f406cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:03.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.157+0000 7f14fbbf7700 1 --2- 172.21.15.40:0/1413708205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14f4163cb0 0x7f14f4164cb0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:03.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.157+0000 7f14fbbf7700 1 -- 172.21.15.40:0/1413708205 >> 172.21.15.40:0/1413708205 conn(0x7f14f40cf350 msgr2=0x7f14f40b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:03.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.157+0000 7f14fbbf7700 1 -- 172.21.15.40:0/1413708205 shutdown_connections 2023-11-08T08:24:03.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.157+0000 7f14fbbf7700 1 -- 172.21.15.40:0/1413708205 wait complete. 2023-11-08T08:24:03.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.161+0000 7f14fbbf7700 1 Processor -- start 2023-11-08T08:24:03.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.161+0000 7f14fbbf7700 1 -- start start 2023-11-08T08:24:03.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.161+0000 7f14fbbf7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14f406d2f0 0x7f14f4151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:03.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.161+0000 7f14fbbf7700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14f40cdfb0 0x7f14f41f5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:03.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.161+0000 7f14fbbf7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14f4163cb0 0x7f14f41f9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:03.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.161+0000 7f14fbbf7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f414f8b0 con 0x7f14f4163cb0 2023-11-08T08:24:03.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.161+0000 7f14fbbf7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f41f97d0 con 0x7f14f406d2f0 2023-11-08T08:24:03.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.161+0000 7f14fbbf7700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f41f94a0 con 0x7f14f40cdfb0 2023-11-08T08:24:03.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.165+0000 7f14fa194700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14f4163cb0 0x7f14f41f9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:03.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.165+0000 7f14fa194700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14f4163cb0 0x7f14f41f9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44004/0 (socket says 172.21.15.40:44004) 2023-11-08T08:24:03.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.165+0000 7f14fa194700 1 -- 172.21.15.40:0/419092499 learned_addr learned my addr 172.21.15.40:0/419092499 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:03.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.165+0000 7f14f9192700 1 --2- 172.21.15.40:0/419092499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14f40cdfb0 0x7f14f41f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:03.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.165+0000 7f14f9993700 1 --2- 172.21.15.40:0/419092499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14f406d2f0 0x7f14f4151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:03.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.165+0000 7f14fa194700 1 -- 172.21.15.40:0/419092499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14f40cdfb0 msgr2=0x7f14f41f5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:03.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.165+0000 7f14fa194700 1 --2- 172.21.15.40:0/419092499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14f40cdfb0 0x7f14f41f5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:03.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.165+0000 7f14fa194700 1 -- 172.21.15.40:0/419092499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14f406d2f0 msgr2=0x7f14f4151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:03.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.165+0000 7f14fa194700 1 --2- 172.21.15.40:0/419092499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14f406d2f0 0x7f14f4151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:03.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.165+0000 7f14fa194700 1 -- 172.21.15.40:0/419092499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e40094b0 con 0x7f14f4163cb0 2023-11-08T08:24:03.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.165+0000 7f14fa194700 1 --2- 172.21.15.40:0/419092499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14f4163cb0 0x7f14f41f9c60 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00cba0 tx=0x7f14ec00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:03.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.165+0000 7f14f27fc700 1 -- 172.21.15.40:0/419092499 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec00ab80 con 0x7f14f4163cb0 2023-11-08T08:24:03.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.165+0000 7f14f27fc700 1 -- 172.21.15.40:0/419092499 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ec00ad00 con 0x7f14f4163cb0 2023-11-08T08:24:03.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.165+0000 7f14f27fc700 1 -- 172.21.15.40:0/419092499 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec007310 con 0x7f14f4163cb0 2023-11-08T08:24:03.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.165+0000 7f14fbbf7700 1 -- 172.21.15.40:0/419092499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f40cc4a0 con 0x7f14f4163cb0 2023-11-08T08:24:03.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.165+0000 7f14fbbf7700 1 -- 172.21.15.40:0/419092499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f41651f0 con 0x7f14f4163cb0 2023-11-08T08:24:03.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.165+0000 7f14fbbf7700 1 -- 172.21.15.40:0/419092499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f40cc780 con 0x7f14f4163cb0 2023-11-08T08:24:03.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.169+0000 7f14f27fc700 1 -- 172.21.15.40:0/419092499 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f14ec010070 con 0x7f14f4163cb0 2023-11-08T08:24:03.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.169+0000 7f14f27fc700 1 --2- 172.21.15.40:0/419092499 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f14e0064ee0 0x7f14e0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:03.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.169+0000 7f14f27fc700 1 -- 172.21.15.40:0/419092499 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f14ec0816e0 con 0x7f14f4163cb0 2023-11-08T08:24:03.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.169+0000 7f14f9993700 1 --2- 172.21.15.40:0/419092499 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f14e0064ee0 0x7f14e0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:03.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.173+0000 7f14f9993700 1 --2- 172.21.15.40:0/419092499 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f14e0064ee0 0x7f14e0067390 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009fd0 tx=0x7f14e4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:03.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.177+0000 7f14f27fc700 1 -- 172.21.15.40:0/419092499 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f14ec04f970 con 0x7f14f4163cb0 2023-11-08T08:24:03.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.349+0000 7f14fbbf7700 1 -- 172.21.15.40:0/419092499 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f14f40726c0 con 0x7f14e0064ee0 2023-11-08T08:24:03.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.349+0000 7f14f27fc700 1 -- 172.21.15.40:0/419092499 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f14f40726c0 con 0x7f14e0064ee0 2023-11-08T08:24:03.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.353+0000 7f14fbbf7700 1 -- 172.21.15.40:0/419092499 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f14e0064ee0 msgr2=0x7f14e0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:03.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.353+0000 7f14fbbf7700 1 --2- 172.21.15.40:0/419092499 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f14e0064ee0 0x7f14e0067390 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009fd0 tx=0x7f14e4008030 comp rx=0 tx=0).stop 2023-11-08T08:24:03.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.353+0000 7f14fbbf7700 1 -- 172.21.15.40:0/419092499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14f4163cb0 msgr2=0x7f14f41f9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:03.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.353+0000 7f14fbbf7700 1 --2- 172.21.15.40:0/419092499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14f4163cb0 0x7f14f41f9c60 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00cba0 tx=0x7f14ec00cf60 comp rx=0 tx=0).stop 2023-11-08T08:24:03.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.353+0000 7f14fbbf7700 1 -- 172.21.15.40:0/419092499 shutdown_connections 2023-11-08T08:24:03.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.353+0000 7f14fbbf7700 1 --2- 172.21.15.40:0/419092499 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f14e0064ee0 0x7f14e0067390 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:03.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.353+0000 7f14fbbf7700 1 --2- 172.21.15.40:0/419092499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14f4163cb0 0x7f14f41f9c60 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:03.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.353+0000 7f14fbbf7700 1 --2- 172.21.15.40:0/419092499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14f40cdfb0 0x7f14f41f5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:03.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.353+0000 7f14fbbf7700 1 --2- 172.21.15.40:0/419092499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14f406d2f0 0x7f14f4151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:03.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.353+0000 7f14fbbf7700 1 -- 172.21.15.40:0/419092499 >> 172.21.15.40:0/419092499 conn(0x7f14f40cf350 msgr2=0x7f14f406c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:03.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.353+0000 7f14fbbf7700 1 -- 172.21.15.40:0/419092499 shutdown_connections 2023-11-08T08:24:03.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.353+0000 7f14fbbf7700 1 -- 172.21.15.40:0/419092499 wait complete. 2023-11-08T08:24:03.364 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 4 to deep-scrub 2023-11-08T08:24:03.380 INFO:tasks.ceph:Scrubbing osd.5 2023-11-08T08:24:03.381 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-08T08:24:03.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.557+0000 7fa0e9ccb700 1 Processor -- start 2023-11-08T08:24:03.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.557+0000 7fa0e9ccb700 1 -- start start 2023-11-08T08:24:03.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.561+0000 7fa0e9ccb700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa0e406a250 0x7fa0e406cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:03.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.561+0000 7fa0e9ccb700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e415cab0 con 0x7fa0e4069e00 2023-11-08T08:24:03.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.561+0000 7fa0e9ccb700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e40702d0 con 0x7fa0e415c6e0 2023-11-08T08:24:03.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.561+0000 7fa0e9ccb700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e40d2800 con 0x7fa0e406a250 2023-11-08T08:24:03.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.561+0000 7fa0dbfff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa0e4069e00 0x7fa0e406c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57926/0 (socket says 172.21.15.40:57926) 2023-11-08T08:24:03.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.561+0000 7fa0dbfff700 1 -- 172.21.15.40:0/31100327 learned_addr learned my addr 172.21.15.40:0/31100327 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:24:03.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.561+0000 7fa0e94ca700 1 --2- 172.21.15.40:0/31100327 >> v2:172.21.15.40:3300/0 conn(0x7fa0e406a250 0x7fa0e406cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:03.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.565+0000 7fa0db7fe700 1 -- 172.21.15.40:0/31100327 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868639756 0 0) 0x7fa0e415cab0 con 0x7fa0e4069e00 2023-11-08T08:24:03.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.565+0000 7fa0db7fe700 1 -- 172.21.15.40:0/31100327 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0cc0035d0 con 0x7fa0e4069e00 2023-11-08T08:24:03.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.565+0000 7fa0db7fe700 1 -- 172.21.15.40:0/31100327 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3456925137 0 0) 0x7fa0e40702d0 con 0x7fa0e415c6e0 2023-11-08T08:24:03.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.565+0000 7fa0db7fe700 1 -- 172.21.15.40:0/31100327 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0e415cab0 con 0x7fa0e415c6e0 2023-11-08T08:24:03.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.565+0000 7fa0e94ca700 1 -- 172.21.15.40:0/31100327 >> v1:172.21.15.169:6789/0 conn(0x7fa0e415c6e0 legacy=0x7fa0e4071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:03.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.565+0000 7fa0e94ca700 1 -- 172.21.15.40:0/31100327 >> v1:172.21.15.40:6789/0 conn(0x7fa0e4069e00 legacy=0x7fa0e406c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:03.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.565+0000 7fa0e94ca700 1 -- 172.21.15.40:0/31100327 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e40d2a10 con 0x7fa0e406a250 2023-11-08T08:24:03.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.565+0000 7fa0e94ca700 1 --2- 172.21.15.40:0/31100327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0e406a250 0x7fa0e406cd40 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc009d90 tx=0x7fa0dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38532ec2d92d543 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:03.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.565+0000 7fa0db7fe700 1 -- 172.21.15.40:0/31100327 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc00e070 con 0x7fa0e406a250 2023-11-08T08:24:03.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.565+0000 7fa0db7fe700 1 -- 172.21.15.40:0/31100327 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0dc00bb20 con 0x7fa0e406a250 2023-11-08T08:24:03.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.565+0000 7fa0db7fe700 1 -- 172.21.15.40:0/31100327 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc012430 con 0x7fa0e406a250 2023-11-08T08:24:03.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.565+0000 7fa0e9ccb700 1 -- 172.21.15.40:0/31100327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0e406a250 msgr2=0x7fa0e406cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:03.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.565+0000 7fa0e9ccb700 1 --2- 172.21.15.40:0/31100327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0e406a250 0x7fa0e406cd40 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc009d90 tx=0x7fa0dc00b330 comp rx=0 tx=0).stop 2023-11-08T08:24:03.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.565+0000 7fa0e9ccb700 1 -- 172.21.15.40:0/31100327 shutdown_connections 2023-11-08T08:24:03.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.565+0000 7fa0e9ccb700 1 --2- 172.21.15.40:0/31100327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0e406a250 0x7fa0e406cd40 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:03.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.565+0000 7fa0e9ccb700 1 -- 172.21.15.40:0/31100327 >> 172.21.15.40:0/31100327 conn(0x7fa0e40b65b0 msgr2=0x7fa0e4071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:03.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.565+0000 7fa0e9ccb700 1 -- 172.21.15.40:0/31100327 shutdown_connections 2023-11-08T08:24:03.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.565+0000 7fa0e9ccb700 1 -- 172.21.15.40:0/31100327 wait complete. 2023-11-08T08:24:03.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.569+0000 7fa0e9ccb700 1 Processor -- start 2023-11-08T08:24:03.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.569+0000 7fa0e9ccb700 1 -- start start 2023-11-08T08:24:03.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.569+0000 7fa0e9ccb700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0e4069e00 0x7fa0e409f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:03.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.569+0000 7fa0e9ccb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0e406a250 0x7fa0e409e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:03.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.569+0000 7fa0e9ccb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0e415c6e0 0x7fa0e409eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:03.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.569+0000 7fa0e9ccb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e409e100 con 0x7fa0e406a250 2023-11-08T08:24:03.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.569+0000 7fa0e9ccb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e409e590 con 0x7fa0e415c6e0 2023-11-08T08:24:03.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.569+0000 7fa0e9ccb700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e409d4f0 con 0x7fa0e4069e00 2023-11-08T08:24:03.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.573+0000 7fa0dbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0e406a250 0x7fa0e409e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:03.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.573+0000 7fa0dbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0e406a250 0x7fa0e409e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44036/0 (socket says 172.21.15.40:44036) 2023-11-08T08:24:03.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.573+0000 7fa0dbfff700 1 -- 172.21.15.40:0/3967692016 learned_addr learned my addr 172.21.15.40:0/3967692016 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:03.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.573+0000 7fa0e94ca700 1 --2- 172.21.15.40:0/3967692016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0e415c6e0 0x7fa0e409eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:03.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.573+0000 7fa0e8cc9700 1 --2- 172.21.15.40:0/3967692016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0e4069e00 0x7fa0e409f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:03.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.573+0000 7fa0dbfff700 1 -- 172.21.15.40:0/3967692016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0e4069e00 msgr2=0x7fa0e409f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:03.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.573+0000 7fa0dbfff700 1 --2- 172.21.15.40:0/3967692016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0e4069e00 0x7fa0e409f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:03.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.573+0000 7fa0dbfff700 1 -- 172.21.15.40:0/3967692016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0e415c6e0 msgr2=0x7fa0e409eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:03.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.573+0000 7fa0dbfff700 1 --2- 172.21.15.40:0/3967692016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0e415c6e0 0x7fa0e409eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:03.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.573+0000 7fa0dbfff700 1 -- 172.21.15.40:0/3967692016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0dc009a70 con 0x7fa0e406a250 2023-11-08T08:24:03.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.573+0000 7fa0dbfff700 1 --2- 172.21.15.40:0/3967692016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0e406a250 0x7fa0e409e9a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400c560 tx=0x7fa0d400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:03.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.573+0000 7fa0d97fa700 1 -- 172.21.15.40:0/3967692016 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4005d90 con 0x7fa0e406a250 2023-11-08T08:24:03.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.573+0000 7fa0d97fa700 1 -- 172.21.15.40:0/3967692016 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d40070d0 con 0x7fa0e406a250 2023-11-08T08:24:03.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.573+0000 7fa0e9ccb700 1 -- 172.21.15.40:0/3967692016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e40d2fd0 con 0x7fa0e406a250 2023-11-08T08:24:03.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.573+0000 7fa0d97fa700 1 -- 172.21.15.40:0/3967692016 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4007460 con 0x7fa0e406a250 2023-11-08T08:24:03.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.573+0000 7fa0e9ccb700 1 -- 172.21.15.40:0/3967692016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e40d2cf0 con 0x7fa0e406a250 2023-11-08T08:24:03.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.573+0000 7fa0e9ccb700 1 -- 172.21.15.40:0/3967692016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa0e40d32b0 con 0x7fa0e406a250 2023-11-08T08:24:03.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.577+0000 7fa0d97fa700 1 -- 172.21.15.40:0/3967692016 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa0d4019030 con 0x7fa0e406a250 2023-11-08T08:24:03.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.577+0000 7fa0d97fa700 1 --2- 172.21.15.40:0/3967692016 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa0d0064ee0 0x7fa0d0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:03.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.577+0000 7fa0d97fa700 1 -- 172.21.15.40:0/3967692016 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fa0d4081100 con 0x7fa0e406a250 2023-11-08T08:24:03.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.577+0000 7fa0d97fa700 1 --2- 172.21.15.40:0/3967692016 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x7fa0d006b7f0 0x7fa0d006dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:03.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.577+0000 7fa0d97fa700 1 -- 172.21.15.40:0/3967692016 --> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0d006e2f0 con 0x7fa0d006b7f0 2023-11-08T08:24:03.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.577+0000 7fa0d97fa700 1 -- 172.21.15.40:0/3967692016 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0d40836e0 con 0x7fa0e406a250 2023-11-08T08:24:03.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.577+0000 7fa0e8cc9700 1 --2- 172.21.15.40:0/3967692016 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa0d0064ee0 0x7fa0d0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:03.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.577+0000 7fa0e94ca700 1 --2- 172.21.15.40:0/3967692016 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x7fa0d006b7f0 0x7fa0d006dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:03.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.581+0000 7fa0e8cc9700 1 --2- 172.21.15.40:0/3967692016 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa0d0064ee0 0x7fa0d0067390 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00011e0 tx=0x7fa0e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:03.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.581+0000 7fa0e94ca700 1 --2- 172.21.15.40:0/3967692016 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x7fa0d006b7f0 0x7fa0d006dc20 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:03.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.581+0000 7fa0d97fa700 1 -- 172.21.15.40:0/3967692016 <== osd.5 v2:172.21.15.169:6809/35793 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa0d006e2f0 con 0x7fa0d006b7f0 2023-11-08T08:24:03.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.601+0000 7fa0e9ccb700 1 -- 172.21.15.40:0/3967692016 --> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa0e4073cb0 con 0x7fa0d006b7f0 2023-11-08T08:24:03.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.601+0000 7fa0d97fa700 1 -- 172.21.15.40:0/3967692016 <== osd.5 v2:172.21.15.169:6809/35793 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa0e4073cb0 con 0x7fa0d006b7f0 2023-11-08T08:24:03.607 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:24:03.608 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T08:24:03.608 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:24:03.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.601+0000 7fa0e9ccb700 1 -- 172.21.15.40:0/3967692016 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x7fa0d006b7f0 msgr2=0x7fa0d006dc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:03.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.601+0000 7fa0e9ccb700 1 --2- 172.21.15.40:0/3967692016 >> [v2:172.21.15.169:6809/35793,v1:172.21.15.169:6812/35793] conn(0x7fa0d006b7f0 0x7fa0d006dc20 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:03.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.601+0000 7fa0e9ccb700 1 -- 172.21.15.40:0/3967692016 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa0d0064ee0 msgr2=0x7fa0d0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:03.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.601+0000 7fa0e9ccb700 1 --2- 172.21.15.40:0/3967692016 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa0d0064ee0 0x7fa0d0067390 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00011e0 tx=0x7fa0e0009040 comp rx=0 tx=0).stop 2023-11-08T08:24:03.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.601+0000 7fa0e9ccb700 1 -- 172.21.15.40:0/3967692016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0e406a250 msgr2=0x7fa0e409e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:03.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.601+0000 7fa0e9ccb700 1 --2- 172.21.15.40:0/3967692016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0e406a250 0x7fa0e409e9a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400c560 tx=0x7fa0d400c870 comp rx=0 tx=0).stop 2023-11-08T08:24:03.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.601+0000 7fa0e94ca700 1 -- 172.21.15.40:0/3967692016 reap_dead start 2023-11-08T08:24:03.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.601+0000 7fa0e9ccb700 1 -- 172.21.15.40:0/3967692016 shutdown_connections 2023-11-08T08:24:03.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.601+0000 7fa0e9ccb700 1 -- 172.21.15.40:0/3967692016 >> 172.21.15.40:0/3967692016 conn(0x7fa0e40b65b0 msgr2=0x7fa0e406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:03.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.601+0000 7fa0e9ccb700 1 -- 172.21.15.40:0/3967692016 shutdown_connections 2023-11-08T08:24:03.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.601+0000 7fa0e9ccb700 1 -- 172.21.15.40:0/3967692016 wait complete. 2023-11-08T08:24:03.635 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-08T08:24:03.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.809+0000 7f67217fc700 1 Processor -- start 2023-11-08T08:24:03.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.809+0000 7f67217fc700 1 -- start start 2023-11-08T08:24:03.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.813+0000 7f67217fc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071aa0 0x7f671c06d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:03.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.813+0000 7f67217fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f671c069e00 0x7f671c06a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:03.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.813+0000 7f67217fc700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671c06c600 con 0x7f671c15c640 2023-11-08T08:24:03.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.813+0000 7f67217fc700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f671c06cef0 con 0x7f671c069e00 2023-11-08T08:24:03.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.813+0000 7f67217fc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671c0d38f0 con 0x7f671c071aa0 2023-11-08T08:24:03.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.813+0000 7f671bfff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f671c15c640 0x7f671c0d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57938/0 (socket says 172.21.15.40:57938) 2023-11-08T08:24:03.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.813+0000 7f671bfff700 1 -- 172.21.15.40:0/3252285318 learned_addr learned my addr 172.21.15.40:0/3252285318 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:24:03.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.813+0000 7f6720ffb700 1 --2- 172.21.15.40:0/3252285318 >> v2:172.21.15.40:3300/0 conn(0x7f671c069e00 0x7f671c06a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:03.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.813+0000 7f671b7fe700 1 --2- 172.21.15.40:0/3252285318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071aa0 0x7f671c06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:03.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.813+0000 7f671b7fe700 1 -- 172.21.15.40:0/3252285318 >> v2:172.21.15.40:3300/0 conn(0x7f671c069e00 msgr2=0x7f671c06a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:03.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.813+0000 7f671b7fe700 1 --2- 172.21.15.40:0/3252285318 >> v2:172.21.15.40:3300/0 conn(0x7f671c069e00 0x7f671c06a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:03.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.813+0000 7f671b7fe700 1 -- 172.21.15.40:0/3252285318 >> v1:172.21.15.40:6789/0 conn(0x7f671c15c640 legacy=0x7f671c0d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:03.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.813+0000 7f671b7fe700 1 -- 172.21.15.40:0/3252285318 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c0722f0 con 0x7f671c071aa0 2023-11-08T08:24:03.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.813+0000 7f671affd700 1 -- 172.21.15.40:0/3252285318 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500839039 0 0) 0x7f671c06c600 con 0x7f671c15c640 2023-11-08T08:24:03.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.817+0000 7f671b7fe700 1 --2- 172.21.15.40:0/3252285318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071aa0 0x7f671c06d310 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6704009850 tx=0x7f6704009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d77a9872057657d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:03.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.817+0000 7f671affd700 1 -- 172.21.15.40:0/3252285318 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704011830 con 0x7f671c071aa0 2023-11-08T08:24:03.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.817+0000 7f671affd700 1 -- 172.21.15.40:0/3252285318 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6704011990 con 0x7f671c071aa0 2023-11-08T08:24:03.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.817+0000 7f671affd700 1 -- 172.21.15.40:0/3252285318 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704011d40 con 0x7f671c071aa0 2023-11-08T08:24:03.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.817+0000 7f67217fc700 1 -- 172.21.15.40:0/3252285318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071aa0 msgr2=0x7f671c06d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:03.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.817+0000 7f67217fc700 1 --2- 172.21.15.40:0/3252285318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071aa0 0x7f671c06d310 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6704009850 tx=0x7f6704009b60 comp rx=0 tx=0).stop 2023-11-08T08:24:03.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.817+0000 7f67217fc700 1 -- 172.21.15.40:0/3252285318 shutdown_connections 2023-11-08T08:24:03.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.817+0000 7f67217fc700 1 --2- 172.21.15.40:0/3252285318 >> v2:172.21.15.40:3300/0 conn(0x7f671c069e00 0x7f671c06a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:03.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.817+0000 7f67217fc700 1 --2- 172.21.15.40:0/3252285318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071aa0 0x7f671c06d310 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:03.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.817+0000 7f67217fc700 1 -- 172.21.15.40:0/3252285318 >> 172.21.15.40:0/3252285318 conn(0x7f671c0b6520 msgr2=0x7f671c071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:03.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.817+0000 7f67217fc700 1 -- 172.21.15.40:0/3252285318 shutdown_connections 2023-11-08T08:24:03.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.817+0000 7f67217fc700 1 -- 172.21.15.40:0/3252285318 wait complete. 2023-11-08T08:24:03.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.821+0000 7f67217fc700 1 Processor -- start 2023-11-08T08:24:03.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.821+0000 7f67217fc700 1 -- start start 2023-11-08T08:24:03.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f67217fc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f671c069e00 0x7f671c1f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:03.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f67217fc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071aa0 0x7f671c0cf490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:03.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f67217fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671c15c640 0x7f671c0c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:03.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f67217fc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671c0cbec0 con 0x7f671c15c640 2023-11-08T08:24:03.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f67217fc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671c0cae60 con 0x7f671c069e00 2023-11-08T08:24:03.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f67217fc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671c1f2be0 con 0x7f671c071aa0 2023-11-08T08:24:03.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f6720ffb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671c15c640 0x7f671c0c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:03.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f6720ffb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671c15c640 0x7f671c0c9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44052/0 (socket says 172.21.15.40:44052) 2023-11-08T08:24:03.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f6720ffb700 1 -- 172.21.15.40:0/142497251 learned_addr learned my addr 172.21.15.40:0/142497251 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:03.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f671b7fe700 1 --2- 172.21.15.40:0/142497251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071aa0 0x7f671c0cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:03.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f671bfff700 1 --2- 172.21.15.40:0/142497251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f671c069e00 0x7f671c1f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:03.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f6720ffb700 1 -- 172.21.15.40:0/142497251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071aa0 msgr2=0x7f671c0cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:03.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f6720ffb700 1 --2- 172.21.15.40:0/142497251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071aa0 0x7f671c0cf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:03.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f6720ffb700 1 -- 172.21.15.40:0/142497251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f671c069e00 msgr2=0x7f671c1f26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:03.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f6720ffb700 1 --2- 172.21.15.40:0/142497251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f671c069e00 0x7f671c1f26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:03.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f6720ffb700 1 -- 172.21.15.40:0/142497251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67040094b0 con 0x7f671c15c640 2023-11-08T08:24:03.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f6720ffb700 1 --2- 172.21.15.40:0/142497251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671c15c640 0x7f671c0c9a80 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f671400efc0 tx=0x7f671400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:03.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f6718ff9700 1 -- 172.21.15.40:0/142497251 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671400cad0 con 0x7f671c15c640 2023-11-08T08:24:03.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f6718ff9700 1 -- 172.21.15.40:0/142497251 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671400f040 con 0x7f671c15c640 2023-11-08T08:24:03.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f6718ff9700 1 -- 172.21.15.40:0/142497251 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671400ce70 con 0x7f671c15c640 2023-11-08T08:24:03.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f67217fc700 1 -- 172.21.15.40:0/142497251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f671c072010 con 0x7f671c15c640 2023-11-08T08:24:03.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.825+0000 7f67217fc700 1 -- 172.21.15.40:0/142497251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671c15ca10 con 0x7f671c15c640 2023-11-08T08:24:03.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.829+0000 7f67217fc700 1 -- 172.21.15.40:0/142497251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f671c06c830 con 0x7f671c15c640 2023-11-08T08:24:03.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.829+0000 7f6718ff9700 1 -- 172.21.15.40:0/142497251 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f6714014480 con 0x7f671c15c640 2023-11-08T08:24:03.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.833+0000 7f6718ff9700 1 --2- 172.21.15.40:0/142497251 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f66fc064ee0 0x7f66fc067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:03.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.833+0000 7f6718ff9700 1 -- 172.21.15.40:0/142497251 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f6714082880 con 0x7f671c15c640 2023-11-08T08:24:03.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.833+0000 7f671bfff700 1 --2- 172.21.15.40:0/142497251 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f66fc064ee0 0x7f66fc067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:03.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.833+0000 7f671bfff700 1 --2- 172.21.15.40:0/142497251 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f66fc064ee0 0x7f66fc067390 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f670c000f60 tx=0x7f670c007e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:03.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:03.837+0000 7f6718ff9700 1 -- 172.21.15.40:0/142497251 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f6714050b80 con 0x7f671c15c640 2023-11-08T08:24:04.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.013+0000 7f67217fc700 1 -- 172.21.15.40:0/142497251 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f671c073cb0 con 0x7f66fc064ee0 2023-11-08T08:24:04.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.013+0000 7f6718ff9700 1 -- 172.21.15.40:0/142497251 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f671c073cb0 con 0x7f66fc064ee0 2023-11-08T08:24:04.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.021+0000 7f67217fc700 1 -- 172.21.15.40:0/142497251 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f66fc064ee0 msgr2=0x7f66fc067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.021+0000 7f67217fc700 1 --2- 172.21.15.40:0/142497251 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f66fc064ee0 0x7f66fc067390 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f670c000f60 tx=0x7f670c007e00 comp rx=0 tx=0).stop 2023-11-08T08:24:04.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.021+0000 7f67217fc700 1 -- 172.21.15.40:0/142497251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671c15c640 msgr2=0x7f671c0c9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.021+0000 7f67217fc700 1 --2- 172.21.15.40:0/142497251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671c15c640 0x7f671c0c9a80 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f671400efc0 tx=0x7f671400c0c0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.021+0000 7f67217fc700 1 -- 172.21.15.40:0/142497251 shutdown_connections 2023-11-08T08:24:04.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.021+0000 7f67217fc700 1 --2- 172.21.15.40:0/142497251 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f66fc064ee0 0x7f66fc067390 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.021+0000 7f67217fc700 1 --2- 172.21.15.40:0/142497251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f671c15c640 0x7f671c0c9a80 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.021+0000 7f67217fc700 1 --2- 172.21.15.40:0/142497251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f671c071aa0 0x7f671c0cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.021+0000 7f67217fc700 1 --2- 172.21.15.40:0/142497251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f671c069e00 0x7f671c1f26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.021+0000 7f67217fc700 1 -- 172.21.15.40:0/142497251 >> 172.21.15.40:0/142497251 conn(0x7f671c0b6520 msgr2=0x7f671c0d1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:04.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.021+0000 7f67217fc700 1 -- 172.21.15.40:0/142497251 shutdown_connections 2023-11-08T08:24:04.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.021+0000 7f67217fc700 1 -- 172.21.15.40:0/142497251 wait complete. 2023-11-08T08:24:04.031 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 5 to deep-scrub 2023-11-08T08:24:04.049 INFO:tasks.ceph:Scrubbing osd.6 2023-11-08T08:24:04.049 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-08T08:24:04.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.229+0000 7f5402b1f700 1 Processor -- start 2023-11-08T08:24:04.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.233+0000 7f5402b1f700 1 -- start start 2023-11-08T08:24:04.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.233+0000 7f5402b1f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53fc158b80 0x7f53fc15c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.233+0000 7f5402b1f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53fc1591d0 con 0x7f53fc0ccd50 2023-11-08T08:24:04.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.233+0000 7f5402b1f700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53fc1594b0 con 0x7f53fc06c380 2023-11-08T08:24:04.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.233+0000 7f5402b1f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc159790 con 0x7f53fc158b80 2023-11-08T08:24:04.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.237+0000 7f5401b1d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f53fc0ccd50 0x7f53fc0cd120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57944/0 (socket says 172.21.15.40:57944) 2023-11-08T08:24:04.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.237+0000 7f5401b1d700 1 -- 172.21.15.40:0/3921698716 learned_addr learned my addr 172.21.15.40:0/3921698716 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:24:04.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.237+0000 7f540231e700 1 --2- 172.21.15.40:0/3921698716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53fc158b80 0x7f53fc15c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.237+0000 7f5400b1b700 1 -- 172.21.15.40:0/3921698716 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846756567 0 0) 0x7f53fc1591d0 con 0x7f53fc0ccd50 2023-11-08T08:24:04.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.237+0000 7f5400b1b700 1 -- 172.21.15.40:0/3921698716 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53e80035d0 con 0x7f53fc0ccd50 2023-11-08T08:24:04.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.237+0000 7f5400b1b700 1 -- 172.21.15.40:0/3921698716 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 254002214 0 0) 0x7f53fc1594b0 con 0x7f53fc06c380 2023-11-08T08:24:04.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.237+0000 7f5400b1b700 1 -- 172.21.15.40:0/3921698716 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53fc1591d0 con 0x7f53fc06c380 2023-11-08T08:24:04.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.237+0000 7f540231e700 1 -- 172.21.15.40:0/3921698716 >> v1:172.21.15.169:6789/0 conn(0x7f53fc06c380 legacy=0x7f53fc159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.237+0000 7f540231e700 1 -- 172.21.15.40:0/3921698716 >> v1:172.21.15.40:6789/0 conn(0x7f53fc0ccd50 legacy=0x7f53fc0cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.237+0000 7f540231e700 1 -- 172.21.15.40:0/3921698716 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53fc159a70 con 0x7f53fc158b80 2023-11-08T08:24:04.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.237+0000 7f540231e700 1 --2- 172.21.15.40:0/3921698716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53fc158b80 0x7f53fc15c710 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f53f400aed0 tx=0x7f53f400c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be518816bc37ec0f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.237+0000 7f5400b1b700 1 -- 172.21.15.40:0/3921698716 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f400f070 con 0x7f53fc158b80 2023-11-08T08:24:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.237+0000 7f5400b1b700 1 -- 172.21.15.40:0/3921698716 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f400a2e0 con 0x7f53fc158b80 2023-11-08T08:24:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.237+0000 7f5400b1b700 1 -- 172.21.15.40:0/3921698716 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f400cc10 con 0x7f53fc158b80 2023-11-08T08:24:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.241+0000 7f5402b1f700 1 -- 172.21.15.40:0/3921698716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53fc158b80 msgr2=0x7f53fc15c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.241+0000 7f5402b1f700 1 --2- 172.21.15.40:0/3921698716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53fc158b80 0x7f53fc15c710 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f53f400aed0 tx=0x7f53f400c330 comp rx=0 tx=0).stop 2023-11-08T08:24:04.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.241+0000 7f5402b1f700 1 -- 172.21.15.40:0/3921698716 shutdown_connections 2023-11-08T08:24:04.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.241+0000 7f5402b1f700 1 --2- 172.21.15.40:0/3921698716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53fc158b80 0x7f53fc15c710 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.241+0000 7f5402b1f700 1 -- 172.21.15.40:0/3921698716 >> 172.21.15.40:0/3921698716 conn(0x7f53fc0b0a20 msgr2=0x7f53fc0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:04.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.241+0000 7f5402b1f700 1 -- 172.21.15.40:0/3921698716 shutdown_connections 2023-11-08T08:24:04.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.241+0000 7f5402b1f700 1 -- 172.21.15.40:0/3921698716 wait complete. 2023-11-08T08:24:04.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.241+0000 7f5402b1f700 1 Processor -- start 2023-11-08T08:24:04.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.241+0000 7f5402b1f700 1 -- start start 2023-11-08T08:24:04.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f5402b1f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53fc06c380 0x7f53fc1f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f5402b1f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53fc0ccd50 0x7f53fc14ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f5402b1f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53fc158b80 0x7f53fc1e57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f5402b1f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc154fa0 con 0x7f53fc0ccd50 2023-11-08T08:24:04.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f5402b1f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc155280 con 0x7f53fc06c380 2023-11-08T08:24:04.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f5402b1f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc155560 con 0x7f53fc158b80 2023-11-08T08:24:04.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f540131c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53fc0ccd50 0x7f53fc14ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f540131c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53fc0ccd50 0x7f53fc14ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44062/0 (socket says 172.21.15.40:44062) 2023-11-08T08:24:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f540131c700 1 -- 172.21.15.40:0/2741377983 learned_addr learned my addr 172.21.15.40:0/2741377983 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f5401b1d700 1 --2- 172.21.15.40:0/2741377983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53fc06c380 0x7f53fc1f18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f540231e700 1 --2- 172.21.15.40:0/2741377983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53fc158b80 0x7f53fc1e57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f540131c700 1 -- 172.21.15.40:0/2741377983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53fc158b80 msgr2=0x7f53fc1e57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f540131c700 1 --2- 172.21.15.40:0/2741377983 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53fc158b80 0x7f53fc1e57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f540131c700 1 -- 172.21.15.40:0/2741377983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53fc06c380 msgr2=0x7f53fc1f18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f540131c700 1 --2- 172.21.15.40:0/2741377983 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53fc06c380 0x7f53fc1f18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f540131c700 1 -- 172.21.15.40:0/2741377983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f400ab80 con 0x7f53fc0ccd50 2023-11-08T08:24:04.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f540131c700 1 --2- 172.21.15.40:0/2741377983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53fc0ccd50 0x7f53fc14ce80 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00c490 tx=0x7f53ec00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:04.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f53f27fc700 1 -- 172.21.15.40:0/2741377983 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec00cea0 con 0x7f53fc0ccd50 2023-11-08T08:24:04.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f53f27fc700 1 -- 172.21.15.40:0/2741377983 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ec0052f0 con 0x7f53fc0ccd50 2023-11-08T08:24:04.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f5402b1f700 1 -- 172.21.15.40:0/2741377983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53fc15a430 con 0x7f53fc0ccd50 2023-11-08T08:24:04.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f53f27fc700 1 -- 172.21.15.40:0/2741377983 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec0036a0 con 0x7f53fc0ccd50 2023-11-08T08:24:04.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.245+0000 7f5402b1f700 1 -- 172.21.15.40:0/2741377983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53fc15c1f0 con 0x7f53fc0ccd50 2023-11-08T08:24:04.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.249+0000 7f5402b1f700 1 -- 172.21.15.40:0/2741377983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f53fc06b420 con 0x7f53fc0ccd50 2023-11-08T08:24:04.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.249+0000 7f53f27fc700 1 -- 172.21.15.40:0/2741377983 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f53ec019030 con 0x7f53fc0ccd50 2023-11-08T08:24:04.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.253+0000 7f53f27fc700 1 --2- 172.21.15.40:0/2741377983 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f53e0064f30 0x7f53e00673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.253+0000 7f53f27fc700 1 -- 172.21.15.40:0/2741377983 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f53ec080ff0 con 0x7f53fc0ccd50 2023-11-08T08:24:04.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.253+0000 7f53f27fc700 1 --2- 172.21.15.40:0/2741377983 >> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] conn(0x7f53e006b8c0 0x7f53e006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.253+0000 7f53f27fc700 1 -- 172.21.15.40:0/2741377983 --> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f53e006e3c0 con 0x7f53e006b8c0 2023-11-08T08:24:04.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.253+0000 7f53f27fc700 1 -- 172.21.15.40:0/2741377983 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53ec0835d0 con 0x7f53fc0ccd50 2023-11-08T08:24:04.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.253+0000 7f5401b1d700 1 --2- 172.21.15.40:0/2741377983 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f53e0064f30 0x7f53e00673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.253+0000 7f540231e700 1 --2- 172.21.15.40:0/2741377983 >> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] conn(0x7f53e006b8c0 0x7f53e006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.253+0000 7f5401b1d700 1 --2- 172.21.15.40:0/2741377983 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f53e0064f30 0x7f53e00673e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f53f8007fd0 tx=0x7f53f8007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:04.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.253+0000 7f540231e700 1 --2- 172.21.15.40:0/2741377983 >> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] conn(0x7f53e006b8c0 0x7f53e006dcf0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:04.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.257+0000 7f53f27fc700 1 -- 172.21.15.40:0/2741377983 <== osd.6 v2:172.21.15.169:6800/35791 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f53e006e3c0 con 0x7f53e006b8c0 2023-11-08T08:24:04.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.273+0000 7f5402b1f700 1 -- 172.21.15.40:0/2741377983 --> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f53fc073cb0 con 0x7f53e006b8c0 2023-11-08T08:24:04.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.277+0000 7f53f27fc700 1 -- 172.21.15.40:0/2741377983 <== osd.6 v2:172.21.15.169:6800/35791 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f53fc073cb0 con 0x7f53e006b8c0 2023-11-08T08:24:04.285 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:24:04.285 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T08:24:04.286 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:24:04.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.277+0000 7f5402b1f700 1 -- 172.21.15.40:0/2741377983 >> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] conn(0x7f53e006b8c0 msgr2=0x7f53e006dcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.277+0000 7f5402b1f700 1 --2- 172.21.15.40:0/2741377983 >> [v2:172.21.15.169:6800/35791,v1:172.21.15.169:6801/35791] conn(0x7f53e006b8c0 0x7f53e006dcf0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.277+0000 7f5402b1f700 1 -- 172.21.15.40:0/2741377983 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f53e0064f30 msgr2=0x7f53e00673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.277+0000 7f5402b1f700 1 --2- 172.21.15.40:0/2741377983 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f53e0064f30 0x7f53e00673e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f53f8007fd0 tx=0x7f53f8007de0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.277+0000 7f5402b1f700 1 -- 172.21.15.40:0/2741377983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53fc0ccd50 msgr2=0x7f53fc14ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.277+0000 7f5402b1f700 1 --2- 172.21.15.40:0/2741377983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53fc0ccd50 0x7f53fc14ce80 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00c490 tx=0x7f53ec00c7a0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.277+0000 7f540231e700 1 -- 172.21.15.40:0/2741377983 reap_dead start 2023-11-08T08:24:04.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.277+0000 7f5402b1f700 1 -- 172.21.15.40:0/2741377983 shutdown_connections 2023-11-08T08:24:04.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.277+0000 7f5402b1f700 1 -- 172.21.15.40:0/2741377983 >> 172.21.15.40:0/2741377983 conn(0x7f53fc0b0a20 msgr2=0x7f53fc0cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:04.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.277+0000 7f5402b1f700 1 -- 172.21.15.40:0/2741377983 shutdown_connections 2023-11-08T08:24:04.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.277+0000 7f5402b1f700 1 -- 172.21.15.40:0/2741377983 wait complete. 2023-11-08T08:24:04.303 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-08T08:24:04.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.477+0000 7f6c2e820700 1 Processor -- start 2023-11-08T08:24:04.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.481+0000 7f6c2e820700 1 -- start start 2023-11-08T08:24:04.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.481+0000 7f6c2e820700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6c2815c1a0 0x7f6c2806c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.481+0000 7f6c2e820700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c280cdeb0 0x7f6c28070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.481+0000 7f6c2e820700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6c280705c0 0x7f6c2815fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.481+0000 7f6c2e820700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c28069bf0 con 0x7f6c280705c0 2023-11-08T08:24:04.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.481+0000 7f6c2e820700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c28070a40 con 0x7f6c2815c1a0 2023-11-08T08:24:04.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.481+0000 7f6c2e820700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c280adf90 con 0x7f6c280cdeb0 2023-11-08T08:24:04.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.481+0000 7f6c2e01f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6c280705c0 0x7f6c2815fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.481+0000 7f6c2e01f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6c280705c0 0x7f6c2815fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44068/0 (socket says 172.21.15.40:44068) 2023-11-08T08:24:04.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.481+0000 7f6c2e01f700 1 -- 172.21.15.40:0/2776501599 learned_addr learned my addr 172.21.15.40:0/2776501599 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:04.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.481+0000 7f6c2d01d700 1 --2- 172.21.15.40:0/2776501599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c280cdeb0 0x7f6c28070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.481+0000 7f6c2d81e700 1 --2- 172.21.15.40:0/2776501599 >> v2:172.21.15.169:3300/0 conn(0x7f6c2815c1a0 0x7f6c2806c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.485+0000 7f6c2e01f700 1 -- 172.21.15.40:0/2776501599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c280cdeb0 msgr2=0x7f6c28070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:04.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.485+0000 7f6c2e01f700 1 --2- 172.21.15.40:0/2776501599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c280cdeb0 0x7f6c28070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.485+0000 7f6c2e01f700 1 -- 172.21.15.40:0/2776501599 >> v2:172.21.15.169:3300/0 conn(0x7f6c2815c1a0 msgr2=0x7f6c2806c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:04.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.485+0000 7f6c2e01f700 1 --2- 172.21.15.40:0/2776501599 >> v2:172.21.15.169:3300/0 conn(0x7f6c2815c1a0 0x7f6c2806c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.485+0000 7f6c2e01f700 1 -- 172.21.15.40:0/2776501599 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c280ce370 con 0x7f6c280705c0 2023-11-08T08:24:04.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.485+0000 7f6c2e01f700 1 --2- 172.21.15.40:0/2776501599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c280705c0 0x7f6c2815fd80 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6c28070ef0 tx=0x7f6c2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e45c8dbf5f60ae4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:04.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.485+0000 7f6c2c81c700 1 -- 172.21.15.40:0/2776501599 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2000e070 con 0x7f6c280705c0 2023-11-08T08:24:04.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.485+0000 7f6c2c81c700 1 -- 172.21.15.40:0/2776501599 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2000b910 con 0x7f6c280705c0 2023-11-08T08:24:04.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.485+0000 7f6c2c81c700 1 -- 172.21.15.40:0/2776501599 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2000bcc0 con 0x7f6c280705c0 2023-11-08T08:24:04.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.485+0000 7f6c2e820700 1 -- 172.21.15.40:0/2776501599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c280705c0 msgr2=0x7f6c2815fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.485+0000 7f6c2e820700 1 --2- 172.21.15.40:0/2776501599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c280705c0 0x7f6c2815fd80 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6c28070ef0 tx=0x7f6c2000b330 comp rx=0 tx=0).stop 2023-11-08T08:24:04.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.485+0000 7f6c2e820700 1 -- 172.21.15.40:0/2776501599 shutdown_connections 2023-11-08T08:24:04.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.485+0000 7f6c2e820700 1 --2- 172.21.15.40:0/2776501599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c280705c0 0x7f6c2815fd80 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.485+0000 7f6c2e820700 1 --2- 172.21.15.40:0/2776501599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c280cdeb0 0x7f6c28070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.485+0000 7f6c2e820700 1 --2- 172.21.15.40:0/2776501599 >> v2:172.21.15.169:3300/0 conn(0x7f6c2815c1a0 0x7f6c2806c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.485+0000 7f6c2e820700 1 -- 172.21.15.40:0/2776501599 >> 172.21.15.40:0/2776501599 conn(0x7f6c280b0f80 msgr2=0x7f6c280ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:04.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.485+0000 7f6c2e820700 1 -- 172.21.15.40:0/2776501599 shutdown_connections 2023-11-08T08:24:04.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.485+0000 7f6c2e820700 1 -- 172.21.15.40:0/2776501599 wait complete. 2023-11-08T08:24:04.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.489+0000 7f6c2e820700 1 Processor -- start 2023-11-08T08:24:04.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.489+0000 7f6c2e820700 1 -- start start 2023-11-08T08:24:04.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c2e820700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c280705c0 0x7f6c280caf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c2e820700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c280cdeb0 0x7f6c280b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c2e820700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c2815c1a0 0x7f6c280b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c2e820700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c280b2f50 con 0x7f6c2815c1a0 2023-11-08T08:24:04.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c2e820700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c280b2340 con 0x7f6c280705c0 2023-11-08T08:24:04.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c2e820700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c280b2620 con 0x7f6c280cdeb0 2023-11-08T08:24:04.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c2e01f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c2815c1a0 0x7f6c280b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c2e01f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c2815c1a0 0x7f6c280b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44076/0 (socket says 172.21.15.40:44076) 2023-11-08T08:24:04.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c2e01f700 1 -- 172.21.15.40:0/4291064661 learned_addr learned my addr 172.21.15.40:0/4291064661 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:04.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c2d81e700 1 --2- 172.21.15.40:0/4291064661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c280705c0 0x7f6c280caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c2d01d700 1 --2- 172.21.15.40:0/4291064661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c280cdeb0 0x7f6c280b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c2e01f700 1 -- 172.21.15.40:0/4291064661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c280cdeb0 msgr2=0x7f6c280b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c2e01f700 1 --2- 172.21.15.40:0/4291064661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c280cdeb0 0x7f6c280b33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c2e01f700 1 -- 172.21.15.40:0/4291064661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c280705c0 msgr2=0x7f6c280caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c2e01f700 1 --2- 172.21.15.40:0/4291064661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c280705c0 0x7f6c280caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c2e01f700 1 -- 172.21.15.40:0/4291064661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c20009b30 con 0x7f6c2815c1a0 2023-11-08T08:24:04.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c2d81e700 1 --2- 172.21.15.40:0/4291064661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c280705c0 0x7f6c280caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:24:04.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c2e01f700 1 --2- 172.21.15.40:0/4291064661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c2815c1a0 0x7f6c280b1880 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6c2806bc30 tx=0x7f6c20004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:04.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c1e7fc700 1 -- 172.21.15.40:0/4291064661 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c200099b0 con 0x7f6c2815c1a0 2023-11-08T08:24:04.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c1e7fc700 1 -- 172.21.15.40:0/4291064661 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c200092e0 con 0x7f6c2815c1a0 2023-11-08T08:24:04.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c2e820700 1 -- 172.21.15.40:0/4291064661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c280cc780 con 0x7f6c2815c1a0 2023-11-08T08:24:04.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c1e7fc700 1 -- 172.21.15.40:0/4291064661 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20004470 con 0x7f6c2815c1a0 2023-11-08T08:24:04.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.493+0000 7f6c2e820700 1 -- 172.21.15.40:0/4291064661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c280cc4a0 con 0x7f6c2815c1a0 2023-11-08T08:24:04.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.497+0000 7f6c2e820700 1 -- 172.21.15.40:0/4291064661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2806cdf0 con 0x7f6c2815c1a0 2023-11-08T08:24:04.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.497+0000 7f6c1e7fc700 1 -- 172.21.15.40:0/4291064661 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f6c2002c070 con 0x7f6c2815c1a0 2023-11-08T08:24:04.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.501+0000 7f6c1e7fc700 1 --2- 172.21.15.40:0/4291064661 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6c0c064f10 0x7f6c0c0673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.501+0000 7f6c1e7fc700 1 -- 172.21.15.40:0/4291064661 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f6c20083020 con 0x7f6c2815c1a0 2023-11-08T08:24:04.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.501+0000 7f6c2d81e700 1 --2- 172.21.15.40:0/4291064661 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6c0c064f10 0x7f6c0c0673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.501+0000 7f6c2d81e700 1 --2- 172.21.15.40:0/4291064661 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6c0c064f10 0x7f6c0c0673c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6c281f50f0 tx=0x7f6c24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:04.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.505+0000 7f6c1e7fc700 1 -- 172.21.15.40:0/4291064661 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f6c20051230 con 0x7f6c2815c1a0 2023-11-08T08:24:04.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.677+0000 7f6c2e820700 1 -- 172.21.15.40:0/4291064661 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c28073cb0 con 0x7f6c0c064f10 2023-11-08T08:24:04.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.681+0000 7f6c1e7fc700 1 -- 172.21.15.40:0/4291064661 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6c28073cb0 con 0x7f6c0c064f10 2023-11-08T08:24:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.681+0000 7f6c2e820700 1 -- 172.21.15.40:0/4291064661 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6c0c064f10 msgr2=0x7f6c0c0673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.681+0000 7f6c2e820700 1 --2- 172.21.15.40:0/4291064661 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6c0c064f10 0x7f6c0c0673c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6c281f50f0 tx=0x7f6c24008040 comp rx=0 tx=0).stop 2023-11-08T08:24:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.681+0000 7f6c2e820700 1 -- 172.21.15.40:0/4291064661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c2815c1a0 msgr2=0x7f6c280b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.681+0000 7f6c2e820700 1 --2- 172.21.15.40:0/4291064661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c2815c1a0 0x7f6c280b1880 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6c2806bc30 tx=0x7f6c20004af0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.681+0000 7f6c2e820700 1 -- 172.21.15.40:0/4291064661 shutdown_connections 2023-11-08T08:24:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.681+0000 7f6c2e820700 1 --2- 172.21.15.40:0/4291064661 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f6c0c064f10 0x7f6c0c0673c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.681+0000 7f6c2e820700 1 --2- 172.21.15.40:0/4291064661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c2815c1a0 0x7f6c280b1880 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.681+0000 7f6c2e820700 1 --2- 172.21.15.40:0/4291064661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c280cdeb0 0x7f6c280b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.681+0000 7f6c2e820700 1 --2- 172.21.15.40:0/4291064661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c280705c0 0x7f6c280caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.681+0000 7f6c2e820700 1 -- 172.21.15.40:0/4291064661 >> 172.21.15.40:0/4291064661 conn(0x7f6c280b0f80 msgr2=0x7f6c280af800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.681+0000 7f6c2e820700 1 -- 172.21.15.40:0/4291064661 shutdown_connections 2023-11-08T08:24:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.681+0000 7f6c2e820700 1 -- 172.21.15.40:0/4291064661 wait complete. 2023-11-08T08:24:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 6 to deep-scrub 2023-11-08T08:24:04.710 INFO:tasks.ceph:Scrubbing osd.7 2023-11-08T08:24:04.711 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-08T08:24:04.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.893+0000 7f83f1ecb700 1 Processor -- start 2023-11-08T08:24:04.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.893+0000 7f83f1ecb700 1 -- start start 2023-11-08T08:24:04.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83f1ecb700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f83ec15c6e0 0x7f83ec071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83f1ecb700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83ec071fe0 0x7f83ec069e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83f1ecb700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f83ec06a340 0x7f83ec0b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83f1ecb700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ec0d2800 con 0x7f83ec06a340 2023-11-08T08:24:04.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83f1ecb700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ec06a790 con 0x7f83ec15c6e0 2023-11-08T08:24:04.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83f1ecb700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec15cab0 con 0x7f83ec071fe0 2023-11-08T08:24:04.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83f16ca700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f83ec06a340 0x7f83ec0b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83f16ca700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f83ec06a340 0x7f83ec0b79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44082/0 (socket says 172.21.15.40:44082) 2023-11-08T08:24:04.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83f16ca700 1 -- 172.21.15.40:0/1346890188 learned_addr learned my addr 172.21.15.40:0/1346890188 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:04.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83ebfff700 1 --2- 172.21.15.40:0/1346890188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83ec071fe0 0x7f83ec069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83f0ec9700 1 --2- 172.21.15.40:0/1346890188 >> v2:172.21.15.169:3300/0 conn(0x7f83ec15c6e0 0x7f83ec071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83f16ca700 1 -- 172.21.15.40:0/1346890188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83ec071fe0 msgr2=0x7f83ec069e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:04.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83f16ca700 1 --2- 172.21.15.40:0/1346890188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83ec071fe0 0x7f83ec069e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83f16ca700 1 -- 172.21.15.40:0/1346890188 >> v2:172.21.15.169:3300/0 conn(0x7f83ec15c6e0 msgr2=0x7f83ec071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:04.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83f16ca700 1 --2- 172.21.15.40:0/1346890188 >> v2:172.21.15.169:3300/0 conn(0x7f83ec15c6e0 0x7f83ec071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83f16ca700 1 -- 172.21.15.40:0/1346890188 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec0702d0 con 0x7f83ec06a340 2023-11-08T08:24:04.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83ebfff700 1 --2- 172.21.15.40:0/1346890188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83ec071fe0 0x7f83ec069e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:24:04.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83f16ca700 1 --2- 172.21.15.40:0/1346890188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83ec06a340 0x7f83ec0b79a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f83ec06e290 tx=0x7f83e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92701d69b377debf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:04.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83eb7fe700 1 -- 172.21.15.40:0/1346890188 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e400e070 con 0x7f83ec06a340 2023-11-08T08:24:04.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83eb7fe700 1 -- 172.21.15.40:0/1346890188 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e400b9c0 con 0x7f83ec06a340 2023-11-08T08:24:04.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.897+0000 7f83eb7fe700 1 -- 172.21.15.40:0/1346890188 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e400bd70 con 0x7f83ec06a340 2023-11-08T08:24:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.901+0000 7f83f1ecb700 1 -- 172.21.15.40:0/1346890188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83ec06a340 msgr2=0x7f83ec0b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.901+0000 7f83f1ecb700 1 --2- 172.21.15.40:0/1346890188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83ec06a340 0x7f83ec0b79a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f83ec06e290 tx=0x7f83e400b330 comp rx=0 tx=0).stop 2023-11-08T08:24:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.901+0000 7f83f1ecb700 1 -- 172.21.15.40:0/1346890188 shutdown_connections 2023-11-08T08:24:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.901+0000 7f83f1ecb700 1 --2- 172.21.15.40:0/1346890188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83ec06a340 0x7f83ec0b79a0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.901+0000 7f83f1ecb700 1 --2- 172.21.15.40:0/1346890188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83ec071fe0 0x7f83ec069e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.901+0000 7f83f1ecb700 1 --2- 172.21.15.40:0/1346890188 >> v2:172.21.15.169:3300/0 conn(0x7f83ec15c6e0 0x7f83ec071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.901+0000 7f83f1ecb700 1 -- 172.21.15.40:0/1346890188 >> 172.21.15.40:0/1346890188 conn(0x7f83ec0b65b0 msgr2=0x7f83ec071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.901+0000 7f83f1ecb700 1 -- 172.21.15.40:0/1346890188 shutdown_connections 2023-11-08T08:24:04.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.901+0000 7f83f1ecb700 1 -- 172.21.15.40:0/1346890188 wait complete. 2023-11-08T08:24:04.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.905+0000 7f83f1ecb700 1 Processor -- start 2023-11-08T08:24:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.905+0000 7f83f1ecb700 1 -- start start 2023-11-08T08:24:04.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.905+0000 7f83f1ecb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83ec06a340 0x7f83ec0ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.905+0000 7f83f1ecb700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83ec071fe0 0x7f83ec15ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.905+0000 7f83f1ecb700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83ec15c6e0 0x7f83ec15b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.905+0000 7f83f1ecb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec1516b0 con 0x7f83ec06a340 2023-11-08T08:24:04.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.905+0000 7f83f1ecb700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec151990 con 0x7f83ec15c6e0 2023-11-08T08:24:04.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.905+0000 7f83f1ecb700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec151c70 con 0x7f83ec071fe0 2023-11-08T08:24:04.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.909+0000 7f83f0ec9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83ec06a340 0x7f83ec0ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.909+0000 7f83f0ec9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83ec06a340 0x7f83ec0ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44092/0 (socket says 172.21.15.40:44092) 2023-11-08T08:24:04.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.909+0000 7f83f0ec9700 1 -- 172.21.15.40:0/1475412040 learned_addr learned my addr 172.21.15.40:0/1475412040 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:04.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.909+0000 7f83f16ca700 1 --2- 172.21.15.40:0/1475412040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83ec15c6e0 0x7f83ec15b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.909+0000 7f83ebfff700 1 --2- 172.21.15.40:0/1475412040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83ec071fe0 0x7f83ec15ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.909+0000 7f83f0ec9700 1 -- 172.21.15.40:0/1475412040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83ec071fe0 msgr2=0x7f83ec15ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.909+0000 7f83f0ec9700 1 --2- 172.21.15.40:0/1475412040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83ec071fe0 0x7f83ec15ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.909+0000 7f83f0ec9700 1 -- 172.21.15.40:0/1475412040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83ec15c6e0 msgr2=0x7f83ec15b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.909+0000 7f83f0ec9700 1 --2- 172.21.15.40:0/1475412040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83ec15c6e0 0x7f83ec15b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.909+0000 7f83f0ec9700 1 -- 172.21.15.40:0/1475412040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e4009b30 con 0x7f83ec06a340 2023-11-08T08:24:04.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.909+0000 7f83f0ec9700 1 --2- 172.21.15.40:0/1475412040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83ec06a340 0x7f83ec0ce050 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f83e000e760 tx=0x7f83e000ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:04.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.909+0000 7f83e97fa700 1 -- 172.21.15.40:0/1475412040 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e000c070 con 0x7f83ec06a340 2023-11-08T08:24:04.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.909+0000 7f83e97fa700 1 -- 172.21.15.40:0/1475412040 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e0013610 con 0x7f83ec06a340 2023-11-08T08:24:04.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.909+0000 7f83f1ecb700 1 -- 172.21.15.40:0/1475412040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ec0d2cf0 con 0x7f83ec06a340 2023-11-08T08:24:04.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.909+0000 7f83e97fa700 1 -- 172.21.15.40:0/1475412040 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0013980 con 0x7f83ec06a340 2023-11-08T08:24:04.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.909+0000 7f83f1ecb700 1 -- 172.21.15.40:0/1475412040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ec0d2a10 con 0x7f83ec06a340 2023-11-08T08:24:04.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.909+0000 7f83f1ecb700 1 -- 172.21.15.40:0/1475412040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f83ec0d2fd0 con 0x7f83ec06a340 2023-11-08T08:24:04.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.913+0000 7f83e97fa700 1 -- 172.21.15.40:0/1475412040 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f83e0023070 con 0x7f83ec06a340 2023-11-08T08:24:04.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.913+0000 7f83e97fa700 1 --2- 172.21.15.40:0/1475412040 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f83d0064ee0 0x7f83d0067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.913+0000 7f83e97fa700 1 -- 172.21.15.40:0/1475412040 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f83e0081aa0 con 0x7f83ec06a340 2023-11-08T08:24:04.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.913+0000 7f83e97fa700 1 --2- 172.21.15.40:0/1475412040 >> [v2:172.21.15.169:6808/35792,v1:172.21.15.169:6810/35792] conn(0x7f83d006b7f0 0x7f83d006dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:04.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.913+0000 7f83e97fa700 1 -- 172.21.15.40:0/1475412040 --> [v2:172.21.15.169:6808/35792,v1:172.21.15.169:6810/35792] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f83d006e2f0 con 0x7f83d006b7f0 2023-11-08T08:24:04.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.913+0000 7f83e97fa700 1 -- 172.21.15.40:0/1475412040 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83e00862a0 con 0x7f83ec06a340 2023-11-08T08:24:04.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.913+0000 7f83ebfff700 1 --2- 172.21.15.40:0/1475412040 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f83d0064ee0 0x7f83d0067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.917+0000 7f83f16ca700 1 --2- 172.21.15.40:0/1475412040 >> [v2:172.21.15.169:6808/35792,v1:172.21.15.169:6810/35792] conn(0x7f83d006b7f0 0x7f83d006dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:04.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.917+0000 7f83f16ca700 1 --2- 172.21.15.40:0/1475412040 >> [v2:172.21.15.169:6808/35792,v1:172.21.15.169:6810/35792] conn(0x7f83d006b7f0 0x7f83d006dc20 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:04.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.917+0000 7f83ebfff700 1 --2- 172.21.15.40:0/1475412040 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f83d0064ee0 0x7f83d0067390 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f83ec1f9140 tx=0x7f83d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:04.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.917+0000 7f83e97fa700 1 -- 172.21.15.40:0/1475412040 <== osd.7 v2:172.21.15.169:6808/35792 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f83d006e2f0 con 0x7f83d006b7f0 2023-11-08T08:24:04.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.937+0000 7f83f1ecb700 1 -- 172.21.15.40:0/1475412040 --> [v2:172.21.15.169:6808/35792,v1:172.21.15.169:6810/35792] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f83ec073cb0 con 0x7f83d006b7f0 2023-11-08T08:24:04.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.941+0000 7f83e97fa700 1 -- 172.21.15.40:0/1475412040 <== osd.7 v2:172.21.15.169:6808/35792 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f83ec073cb0 con 0x7f83d006b7f0 2023-11-08T08:24:04.947 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-11-08T08:24:04.947 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T08:24:04.948 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-11-08T08:24:04.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.941+0000 7f83f1ecb700 1 -- 172.21.15.40:0/1475412040 >> [v2:172.21.15.169:6808/35792,v1:172.21.15.169:6810/35792] conn(0x7f83d006b7f0 msgr2=0x7f83d006dc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.941+0000 7f83f1ecb700 1 --2- 172.21.15.40:0/1475412040 >> [v2:172.21.15.169:6808/35792,v1:172.21.15.169:6810/35792] conn(0x7f83d006b7f0 0x7f83d006dc20 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:04.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.941+0000 7f83f1ecb700 1 -- 172.21.15.40:0/1475412040 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f83d0064ee0 msgr2=0x7f83d0067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.941+0000 7f83f1ecb700 1 --2- 172.21.15.40:0/1475412040 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f83d0064ee0 0x7f83d0067390 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f83ec1f9140 tx=0x7f83d8008040 comp rx=0 tx=0).stop 2023-11-08T08:24:04.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.941+0000 7f83f1ecb700 1 -- 172.21.15.40:0/1475412040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83ec06a340 msgr2=0x7f83ec0ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:04.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.941+0000 7f83f1ecb700 1 --2- 172.21.15.40:0/1475412040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83ec06a340 0x7f83ec0ce050 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f83e000e760 tx=0x7f83e000ea70 comp rx=0 tx=0).stop 2023-11-08T08:24:04.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.941+0000 7f83f16ca700 1 -- 172.21.15.40:0/1475412040 reap_dead start 2023-11-08T08:24:04.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.941+0000 7f83f1ecb700 1 -- 172.21.15.40:0/1475412040 shutdown_connections 2023-11-08T08:24:04.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.941+0000 7f83f1ecb700 1 -- 172.21.15.40:0/1475412040 >> 172.21.15.40:0/1475412040 conn(0x7f83ec0b65b0 msgr2=0x7f83ec0b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:04.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.941+0000 7f83f1ecb700 1 -- 172.21.15.40:0/1475412040 shutdown_connections 2023-11-08T08:24:04.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:04.941+0000 7f83f1ecb700 1 -- 172.21.15.40:0/1475412040 wait complete. 2023-11-08T08:24:04.971 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-08T08:24:05.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.153+0000 7fd7b6937700 1 Processor -- start 2023-11-08T08:24:05.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.153+0000 7fd7b6937700 1 -- start start 2023-11-08T08:24:05.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.157+0000 7fd7b6937700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd7b0156b60 0x7fd7b00cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:05.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.157+0000 7fd7b6937700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b006b0d0 con 0x7fd7b01605b0 2023-11-08T08:24:05.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.157+0000 7fd7b6937700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b006b560 con 0x7fd7b0156790 2023-11-08T08:24:05.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.157+0000 7fd7b6937700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b00cc990 con 0x7fd7b0156b60 2023-11-08T08:24:05.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.157+0000 7fd7a7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd7b0156b60 0x7fd7b00cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:05.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.157+0000 7fd7a7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd7b0156b60 0x7fd7b00cc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44100/0 (socket says 172.21.15.40:44100) 2023-11-08T08:24:05.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.157+0000 7fd7a7fff700 1 -- 172.21.15.40:0/2335293595 learned_addr learned my addr 172.21.15.40:0/2335293595 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:05.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.157+0000 7fd7a7fff700 1 -- 172.21.15.40:0/2335293595 >> v1:172.21.15.169:6789/0 conn(0x7fd7b0156790 legacy=0x7fd7b015fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:05.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.157+0000 7fd7a7fff700 1 -- 172.21.15.40:0/2335293595 >> v1:172.21.15.40:6789/0 conn(0x7fd7b01605b0 legacy=0x7fd7b0166230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:05.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.157+0000 7fd7a7fff700 1 -- 172.21.15.40:0/2335293595 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b006a7a0 con 0x7fd7b0156b60 2023-11-08T08:24:05.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.157+0000 7fd7a7fff700 1 --2- 172.21.15.40:0/2335293595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b0156b60 0x7fd7b00cc290 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fd7b006e250 tx=0x7fd798009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af54f25d269b04a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:05.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.157+0000 7fd7af7fe700 1 -- 172.21.15.40:0/2335293595 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79800b860 con 0x7fd7b0156b60 2023-11-08T08:24:05.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.157+0000 7fd7af7fe700 1 -- 172.21.15.40:0/2335293595 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79800b9c0 con 0x7fd7b0156b60 2023-11-08T08:24:05.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.157+0000 7fd7af7fe700 1 -- 172.21.15.40:0/2335293595 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79800bd50 con 0x7fd7b0156b60 2023-11-08T08:24:05.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.161+0000 7fd7b6937700 1 -- 172.21.15.40:0/2335293595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b0156b60 msgr2=0x7fd7b00cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:05.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.161+0000 7fd7b6937700 1 --2- 172.21.15.40:0/2335293595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b0156b60 0x7fd7b00cc290 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fd7b006e250 tx=0x7fd798009bf0 comp rx=0 tx=0).stop 2023-11-08T08:24:05.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.161+0000 7fd7b6937700 1 -- 172.21.15.40:0/2335293595 shutdown_connections 2023-11-08T08:24:05.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.161+0000 7fd7b6937700 1 --2- 172.21.15.40:0/2335293595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b0156b60 0x7fd7b00cc290 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:05.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.161+0000 7fd7b6937700 1 -- 172.21.15.40:0/2335293595 >> 172.21.15.40:0/2335293595 conn(0x7fd7b00cf350 msgr2=0x7fd7b00b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:05.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.161+0000 7fd7b6937700 1 -- 172.21.15.40:0/2335293595 shutdown_connections 2023-11-08T08:24:05.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.161+0000 7fd7b6937700 1 -- 172.21.15.40:0/2335293595 wait complete. 2023-11-08T08:24:05.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.161+0000 7fd7b6937700 1 Processor -- start 2023-11-08T08:24:05.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.161+0000 7fd7b6937700 1 -- start start 2023-11-08T08:24:05.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7b6937700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7b0156790 0x7fd7b00af760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:05.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7b6937700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7b0156b60 0x7fd7b00ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:05.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7b6937700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b01605b0 0x7fd7b0154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:05.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7b6937700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b00ad9d0 con 0x7fd7b01605b0 2023-11-08T08:24:05.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7b6937700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b00af2d0 con 0x7fd7b0156b60 2023-11-08T08:24:05.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7b6937700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b00acda0 con 0x7fd7b0156790 2023-11-08T08:24:05.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7b4ed4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b01605b0 0x7fd7b0154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:05.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7b4ed4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b01605b0 0x7fd7b0154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44112/0 (socket says 172.21.15.40:44112) 2023-11-08T08:24:05.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7b4ed4700 1 -- 172.21.15.40:0/2154262117 learned_addr learned my addr 172.21.15.40:0/2154262117 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:05.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7affff700 1 --2- 172.21.15.40:0/2154262117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7b0156790 0x7fd7b00af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:05.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7a7fff700 1 --2- 172.21.15.40:0/2154262117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7b0156b60 0x7fd7b00ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:05.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7b4ed4700 1 -- 172.21.15.40:0/2154262117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7b0156790 msgr2=0x7fd7b00af760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:05.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7b4ed4700 1 --2- 172.21.15.40:0/2154262117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7b0156790 0x7fd7b00af760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:05.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7b4ed4700 1 -- 172.21.15.40:0/2154262117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7b0156b60 msgr2=0x7fd7b00ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:05.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7b4ed4700 1 --2- 172.21.15.40:0/2154262117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7b0156b60 0x7fd7b00ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:05.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7b4ed4700 1 -- 172.21.15.40:0/2154262117 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd798009570 con 0x7fd7b01605b0 2023-11-08T08:24:05.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7b4ed4700 1 --2- 172.21.15.40:0/2154262117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b01605b0 0x7fd7b0154d30 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800cb60 tx=0x7fd7a800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:05.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7ad7fa700 1 -- 172.21.15.40:0/2154262117 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a800ab50 con 0x7fd7b01605b0 2023-11-08T08:24:05.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7ad7fa700 1 -- 172.21.15.40:0/2154262117 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a800acd0 con 0x7fd7b01605b0 2023-11-08T08:24:05.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7b6937700 1 -- 172.21.15.40:0/2154262117 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b006ad60 con 0x7fd7b01605b0 2023-11-08T08:24:05.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7ad7fa700 1 -- 172.21.15.40:0/2154262117 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8012660 con 0x7fd7b01605b0 2023-11-08T08:24:05.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.165+0000 7fd7b6937700 1 -- 172.21.15.40:0/2154262117 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b006aa80 con 0x7fd7b01605b0 2023-11-08T08:24:05.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.169+0000 7fd7b6937700 1 -- 172.21.15.40:0/2154262117 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b00ce000 con 0x7fd7b01605b0 2023-11-08T08:24:05.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.169+0000 7fd7ad7fa700 1 -- 172.21.15.40:0/2154262117 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd7a8022070 con 0x7fd7b01605b0 2023-11-08T08:24:05.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.173+0000 7fd7ad7fa700 1 --2- 172.21.15.40:0/2154262117 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd790064f30 0x7fd7900673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:05.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.173+0000 7fd7ad7fa700 1 -- 172.21.15.40:0/2154262117 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fd7a80822d0 con 0x7fd7b01605b0 2023-11-08T08:24:05.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.173+0000 7fd7affff700 1 --2- 172.21.15.40:0/2154262117 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd790064f30 0x7fd7900673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:05.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.173+0000 7fd7affff700 1 --2- 172.21.15.40:0/2154262117 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd790064f30 0x7fd7900673e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0000f60 tx=0x7fd7a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:05.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.177+0000 7fd7ad7fa700 1 -- 172.21.15.40:0/2154262117 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd7a80505e0 con 0x7fd7b01605b0 2023-11-08T08:24:05.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.353+0000 7fd7b6937700 1 -- 172.21.15.40:0/2154262117 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7b0069f20 con 0x7fd790064f30 2023-11-08T08:24:05.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.353+0000 7fd7ad7fa700 1 -- 172.21.15.40:0/2154262117 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd7b0069f20 con 0x7fd790064f30 2023-11-08T08:24:05.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.357+0000 7fd7b6937700 1 -- 172.21.15.40:0/2154262117 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd790064f30 msgr2=0x7fd7900673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:05.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.357+0000 7fd7b6937700 1 --2- 172.21.15.40:0/2154262117 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd790064f30 0x7fd7900673e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0000f60 tx=0x7fd7a0009040 comp rx=0 tx=0).stop 2023-11-08T08:24:05.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.357+0000 7fd7b6937700 1 -- 172.21.15.40:0/2154262117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b01605b0 msgr2=0x7fd7b0154d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:05.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.357+0000 7fd7b6937700 1 --2- 172.21.15.40:0/2154262117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b01605b0 0x7fd7b0154d30 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800cb60 tx=0x7fd7a800cf20 comp rx=0 tx=0).stop 2023-11-08T08:24:05.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.357+0000 7fd7b6937700 1 -- 172.21.15.40:0/2154262117 shutdown_connections 2023-11-08T08:24:05.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.357+0000 7fd7b6937700 1 --2- 172.21.15.40:0/2154262117 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd790064f30 0x7fd7900673e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:05.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.357+0000 7fd7b6937700 1 --2- 172.21.15.40:0/2154262117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b01605b0 0x7fd7b0154d30 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:05.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.357+0000 7fd7b6937700 1 --2- 172.21.15.40:0/2154262117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7b0156b60 0x7fd7b00ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:05.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.357+0000 7fd7b6937700 1 --2- 172.21.15.40:0/2154262117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7b0156790 0x7fd7b00af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:05.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.357+0000 7fd7b6937700 1 -- 172.21.15.40:0/2154262117 >> 172.21.15.40:0/2154262117 conn(0x7fd7b00cf350 msgr2=0x7fd7b015da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:05.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.357+0000 7fd7b6937700 1 -- 172.21.15.40:0/2154262117 shutdown_connections 2023-11-08T08:24:05.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.357+0000 7fd7b6937700 1 -- 172.21.15.40:0/2154262117 wait complete. 2023-11-08T08:24:05.368 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 7 to deep-scrub 2023-11-08T08:24:05.388 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T08:24:05.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.569+0000 7fcbb0a2f700 1 Processor -- start 2023-11-08T08:24:05.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.569+0000 7fcbb0a2f700 1 -- start start 2023-11-08T08:24:05.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.569+0000 7fcbb0a2f700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcbac15c640 0x7fcbac0d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:05.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.569+0000 7fcbb0a2f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac069e00 0x7fcbac06a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:05.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.569+0000 7fcbb0a2f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbac06c600 con 0x7fcbac071aa0 2023-11-08T08:24:05.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.569+0000 7fcbb0a2f700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbac06cef0 con 0x7fcbac15c640 2023-11-08T08:24:05.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.569+0000 7fcbb0a2f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac06a7c0 con 0x7fcbac069e00 2023-11-08T08:24:05.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.569+0000 7fcba3fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcbac071aa0 0x7fcbac06d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57960/0 (socket says 172.21.15.40:57960) 2023-11-08T08:24:05.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.569+0000 7fcba3fff700 1 -- 172.21.15.40:0/2260757133 learned_addr learned my addr 172.21.15.40:0/2260757133 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:24:05.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.569+0000 7fcbab7fe700 1 --2- 172.21.15.40:0/2260757133 >> v2:172.21.15.169:3300/0 conn(0x7fcbac15c640 0x7fcbac0d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:05.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.569+0000 7fcbaaffd700 1 -- 172.21.15.40:0/2260757133 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300324058 0 0) 0x7fcbac06c600 con 0x7fcbac071aa0 2023-11-08T08:24:05.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.569+0000 7fcbabfff700 1 --2- 172.21.15.40:0/2260757133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac069e00 0x7fcbac06a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:05.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbaaffd700 1 -- 172.21.15.40:0/2260757133 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb980035d0 con 0x7fcbac071aa0 2023-11-08T08:24:05.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbaaffd700 1 -- 172.21.15.40:0/2260757133 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4155339657 0 0) 0x7fcb980035d0 con 0x7fcbac071aa0 2023-11-08T08:24:05.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbaaffd700 1 -- 172.21.15.40:0/2260757133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac069e00 msgr2=0x7fcbac06a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:05.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbaaffd700 1 --2- 172.21.15.40:0/2260757133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac069e00 0x7fcbac06a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:05.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbaaffd700 1 -- 172.21.15.40:0/2260757133 >> v2:172.21.15.169:3300/0 conn(0x7fcbac15c640 msgr2=0x7fcbac0d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:05.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbaaffd700 1 --2- 172.21.15.40:0/2260757133 >> v2:172.21.15.169:3300/0 conn(0x7fcbac15c640 0x7fcbac0d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:05.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbaaffd700 1 -- 172.21.15.40:0/2260757133 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac0722f0 con 0x7fcbac071aa0 2023-11-08T08:24:05.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbaaffd700 1 -- 172.21.15.40:0/2260757133 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7fcb94001670 con 0x7fcbac071aa0 2023-11-08T08:24:05.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbaaffd700 1 -- 172.21.15.40:0/2260757133 >> v1:172.21.15.40:6789/0 conn(0x7fcbac071aa0 legacy=0x7fcbac06d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:05.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbaaffd700 1 --2- 172.21.15.40:0/2260757133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb98003a20 0x7fcb98003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:05.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbaaffd700 1 --2- 172.21.15.40:0/2260757133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb98004400 0x7fcb980048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:05.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbaaffd700 1 --2- 172.21.15.40:0/2260757133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb98004ff0 0x7fcb9800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:05.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbaaffd700 1 -- 172.21.15.40:0/2260757133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac06cef0 con 0x7fcb98004400 2023-11-08T08:24:05.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbaaffd700 1 -- 172.21.15.40:0/2260757133 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac06a7c0 con 0x7fcb98004ff0 2023-11-08T08:24:05.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbaaffd700 1 -- 172.21.15.40:0/2260757133 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9800bf70 con 0x7fcb98003a20 2023-11-08T08:24:05.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbabfff700 1 --2- 172.21.15.40:0/2260757133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac069e00 0x7fcbac06a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:24:05.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcba3fff700 1 --2- 172.21.15.40:0/2260757133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb98004400 0x7fcb980048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:05.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbab7fe700 1 --2- 172.21.15.40:0/2260757133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb98003a20 0x7fcb98003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:05.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbabfff700 1 --2- 172.21.15.40:0/2260757133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb98004ff0 0x7fcb9800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:05.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcba3fff700 1 -- 172.21.15.40:0/2260757133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb98003a20 msgr2=0x7fcb98003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:05.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcba3fff700 1 --2- 172.21.15.40:0/2260757133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb98003a20 0x7fcb98003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:05.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcba3fff700 1 -- 172.21.15.40:0/2260757133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb98004ff0 msgr2=0x7fcb9800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:05.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcba3fff700 1 --2- 172.21.15.40:0/2260757133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb98004ff0 0x7fcb9800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:05.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcba3fff700 1 -- 172.21.15.40:0/2260757133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb980035d0 con 0x7fcb98004400 2023-11-08T08:24:05.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbabfff700 1 --2- 172.21.15.40:0/2260757133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb98004ff0 0x7fcb9800ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:24:05.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcba3fff700 1 --2- 172.21.15.40:0/2260757133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb98004400 0x7fcb980048d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400c320 tx=0x7fcb980058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c4a53e3367f11 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:05.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbaaffd700 1 -- 172.21.15.40:0/2260757133 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400ce60 con 0x7fcb98004400 2023-11-08T08:24:05.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbaaffd700 1 -- 172.21.15.40:0/2260757133 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb94005980 con 0x7fcb98004400 2023-11-08T08:24:05.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.573+0000 7fcbaaffd700 1 -- 172.21.15.40:0/2260757133 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94005d10 con 0x7fcb98004400 2023-11-08T08:24:05.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.577+0000 7fcbb0a2f700 1 -- 172.21.15.40:0/2260757133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb98004400 msgr2=0x7fcb980048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:05.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.577+0000 7fcbb0a2f700 1 --2- 172.21.15.40:0/2260757133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb98004400 0x7fcb980048d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400c320 tx=0x7fcb980058a0 comp rx=0 tx=0).stop 2023-11-08T08:24:05.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.577+0000 7fcbabfff700 1 -- 172.21.15.40:0/2260757133 reap_dead start 2023-11-08T08:24:05.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.577+0000 7fcbb0a2f700 1 -- 172.21.15.40:0/2260757133 shutdown_connections 2023-11-08T08:24:05.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.577+0000 7fcbb0a2f700 1 -- 172.21.15.40:0/2260757133 >> 172.21.15.40:0/2260757133 conn(0x7fcbac0b6520 msgr2=0x7fcbac071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:05.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.577+0000 7fcbb0a2f700 1 -- 172.21.15.40:0/2260757133 shutdown_connections 2023-11-08T08:24:05.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.577+0000 7fcbb0a2f700 1 -- 172.21.15.40:0/2260757133 wait complete. 2023-11-08T08:24:05.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.577+0000 7fcbb0a2f700 1 Processor -- start 2023-11-08T08:24:05.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.577+0000 7fcbb0a2f700 1 -- start start 2023-11-08T08:24:05.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcbb0a2f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb98004400 0x7fcbac158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:05.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcbb0a2f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbac0c9a80 0x7fcbac0cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:05.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcbb0a2f700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac0ce300 0x7fcbac0b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:05.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcbb0a2f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0ce7f0 con 0x7fcbac0c9a80 2023-11-08T08:24:05.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcbb0a2f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0c7660 con 0x7fcb98004400 2023-11-08T08:24:05.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcbb0a2f700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0c7940 con 0x7fcbac0ce300 2023-11-08T08:24:05.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcba3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbac0c9a80 0x7fcbac0cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:05.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcba3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbac0c9a80 0x7fcbac0cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44134/0 (socket says 172.21.15.40:44134) 2023-11-08T08:24:05.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcba3fff700 1 -- 172.21.15.40:0/2172528909 learned_addr learned my addr 172.21.15.40:0/2172528909 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:05.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcbab7fe700 1 --2- 172.21.15.40:0/2172528909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb98004400 0x7fcbac158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:05.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcbabfff700 1 --2- 172.21.15.40:0/2172528909 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac0ce300 0x7fcbac0b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:05.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcba3fff700 1 -- 172.21.15.40:0/2172528909 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac0ce300 msgr2=0x7fcbac0b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:05.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcba3fff700 1 --2- 172.21.15.40:0/2172528909 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac0ce300 0x7fcbac0b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:05.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcba3fff700 1 -- 172.21.15.40:0/2172528909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb98004400 msgr2=0x7fcbac158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:05.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcba3fff700 1 --2- 172.21.15.40:0/2172528909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb98004400 0x7fcbac158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:05.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcba3fff700 1 -- 172.21.15.40:0/2172528909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac15ca10 con 0x7fcbac0c9a80 2023-11-08T08:24:05.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcba3fff700 1 --2- 172.21.15.40:0/2172528909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbac0c9a80 0x7fcbac0cddc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fcb94011000 tx=0x7fcb94001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:05.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcba8ff9700 1 -- 172.21.15.40:0/2172528909 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94000ed0 con 0x7fcbac0c9a80 2023-11-08T08:24:05.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcba8ff9700 1 -- 172.21.15.40:0/2172528909 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb94001680 con 0x7fcbac0c9a80 2023-11-08T08:24:05.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcbb0a2f700 1 -- 172.21.15.40:0/2172528909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbac06da20 con 0x7fcbac0c9a80 2023-11-08T08:24:05.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcba8ff9700 1 -- 172.21.15.40:0/2172528909 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94004ce0 con 0x7fcbac0c9a80 2023-11-08T08:24:05.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.581+0000 7fcbb0a2f700 1 -- 172.21.15.40:0/2172528909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbac072010 con 0x7fcbac0c9a80 2023-11-08T08:24:05.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.585+0000 7fcbb0a2f700 1 -- 172.21.15.40:0/2172528909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbac0ce7f0 con 0x7fcbac0c9a80 2023-11-08T08:24:05.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.585+0000 7fcba8ff9700 1 -- 172.21.15.40:0/2172528909 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fcb94024070 con 0x7fcbac0c9a80 2023-11-08T08:24:05.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.585+0000 7fcba8ff9700 1 --2- 172.21.15.40:0/2172528909 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcb8c064ee0 0x7fcb8c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:05.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.585+0000 7fcba8ff9700 1 -- 172.21.15.40:0/2172528909 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fcb94082f90 con 0x7fcbac0c9a80 2023-11-08T08:24:05.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.589+0000 7fcbab7fe700 1 --2- 172.21.15.40:0/2172528909 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcb8c064ee0 0x7fcb8c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:05.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.589+0000 7fcbab7fe700 1 --2- 172.21.15.40:0/2172528909 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcb8c064ee0 0x7fcb8c067390 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009cf0 tx=0x7fcb9c009240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:05.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.593+0000 7fcba8ff9700 1 -- 172.21.15.40:0/2172528909 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fcb940511c0 con 0x7fcbac0c9a80 2023-11-08T08:24:05.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.757+0000 7fcbb0a2f700 1 -- 172.21.15.40:0/2172528909 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcbac073cb0 con 0x7fcb8c064ee0 2023-11-08T08:24:05.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.769+0000 7fcba8ff9700 1 -- 172.21.15.40:0/2172528909 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70457 (secure 0 0 0) 0x7fcbac073cb0 con 0x7fcb8c064ee0 2023-11-08T08:24:05.775 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:24:05.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.769+0000 7fcbb0a2f700 1 -- 172.21.15.40:0/2172528909 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcb8c064ee0 msgr2=0x7fcb8c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:05.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.769+0000 7fcbb0a2f700 1 --2- 172.21.15.40:0/2172528909 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcb8c064ee0 0x7fcb8c067390 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009cf0 tx=0x7fcb9c009240 comp rx=0 tx=0).stop 2023-11-08T08:24:05.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.769+0000 7fcbb0a2f700 1 -- 172.21.15.40:0/2172528909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbac0c9a80 msgr2=0x7fcbac0cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:05.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.769+0000 7fcbb0a2f700 1 --2- 172.21.15.40:0/2172528909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbac0c9a80 0x7fcbac0cddc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fcb94011000 tx=0x7fcb94001a00 comp rx=0 tx=0).stop 2023-11-08T08:24:05.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.769+0000 7fcbb0a2f700 1 -- 172.21.15.40:0/2172528909 shutdown_connections 2023-11-08T08:24:05.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.769+0000 7fcbb0a2f700 1 --2- 172.21.15.40:0/2172528909 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fcb8c064ee0 0x7fcb8c067390 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:05.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.769+0000 7fcbb0a2f700 1 --2- 172.21.15.40:0/2172528909 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbac0ce300 0x7fcbac0b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:05.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.769+0000 7fcbb0a2f700 1 --2- 172.21.15.40:0/2172528909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbac0c9a80 0x7fcbac0cddc0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:05.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.769+0000 7fcbb0a2f700 1 --2- 172.21.15.40:0/2172528909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb98004400 0x7fcbac158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:05.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.769+0000 7fcbb0a2f700 1 -- 172.21.15.40:0/2172528909 >> 172.21.15.40:0/2172528909 conn(0x7fcbac0b6520 msgr2=0x7fcbac0d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:05.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.769+0000 7fcbb0a2f700 1 -- 172.21.15.40:0/2172528909 shutdown_connections 2023-11-08T08:24:05.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:05.769+0000 7fcbb0a2f700 1 -- 172.21.15.40:0/2172528909 wait complete. 2023-11-08T08:24:05.780 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-11-08T08:24:05.799 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2023-11-08T08:24:03.993877+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":90984,"kb_used_data":8232,"kb_used_omap":0,"kb_used_meta":82688,"kb_avail":754883736,"statfs":{"total":773094113280,"available":773000945664,"internally_reserved":0,"allocated":8429568,"data_stored":4569896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":84672512},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002910"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:53.995671+0000","last_change":"2023-11-08T08:18:58.771310+0000","last_active":"2023-11-08T08:23:53.995671+0000","last_peered":"2023-11-08T08:23:53.995671+0000","last_clean":"2023-11-08T08:23:53.995671+0000","last_became_active":"2023-11-08T08:18:58.770891+0000","last_became_peered":"2023-11-08T08:18:58.770891+0000","last_unstale":"2023-11-08T08:23:53.995671+0000","last_undegraded":"2023-11-08T08:23:53.995671+0000","last_fullsized":"2023-11-08T08:23:53.995671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:54.762987+0000","last_change":"2023-11-08T08:18:58.770483+0000","last_active":"2023-11-08T08:23:54.762987+0000","last_peered":"2023-11-08T08:23:54.762987+0000","last_clean":"2023-11-08T08:23:54.762987+0000","last_became_active":"2023-11-08T08:18:58.770040+0000","last_became_peered":"2023-11-08T08:18:58.770040+0000","last_unstale":"2023-11-08T08:23:54.762987+0000","last_undegraded":"2023-11-08T08:23:54.762987+0000","last_fullsized":"2023-11-08T08:23:54.762987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":59,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:53.996243+0000","last_change":"2023-11-08T08:18:58.766848+0000","last_active":"2023-11-08T08:23:53.996243+0000","last_peered":"2023-11-08T08:23:53.996243+0000","last_clean":"2023-11-08T08:23:53.996243+0000","last_became_active":"2023-11-08T08:18:58.766378+0000","last_became_peered":"2023-11-08T08:18:58.766378+0000","last_unstale":"2023-11-08T08:23:53.996243+0000","last_undegraded":"2023-11-08T08:23:53.996243+0000","last_fullsized":"2023-11-08T08:23:53.996243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:54.764100+0000","last_change":"2023-11-08T08:18:58.769764+0000","last_active":"2023-11-08T08:23:54.764100+0000","last_peered":"2023-11-08T08:23:54.764100+0000","last_clean":"2023-11-08T08:23:54.764100+0000","last_became_active":"2023-11-08T08:18:58.769357+0000","last_became_peered":"2023-11-08T08:18:58.769357+0000","last_unstale":"2023-11-08T08:23:54.764100+0000","last_undegraded":"2023-11-08T08:23:54.764100+0000","last_fullsized":"2023-11-08T08:23:54.764100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T13:25:13.748094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":58,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:54.761825+0000","last_change":"2023-11-08T08:18:58.771460+0000","last_active":"2023-11-08T08:23:54.761825+0000","last_peered":"2023-11-08T08:23:54.761825+0000","last_clean":"2023-11-08T08:23:54.761825+0000","last_became_active":"2023-11-08T08:18:58.770648+0000","last_became_peered":"2023-11-08T08:18:58.770648+0000","last_unstale":"2023-11-08T08:23:54.761825+0000","last_undegraded":"2023-11-08T08:23:54.761825+0000","last_fullsized":"2023-11-08T08:23:54.761825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:58.233410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":60,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:54.763825+0000","last_change":"2023-11-08T08:18:58.973732+0000","last_active":"2023-11-08T08:23:54.763825+0000","last_peered":"2023-11-08T08:23:54.763825+0000","last_clean":"2023-11-08T08:23:54.763825+0000","last_became_active":"2023-11-08T08:18:58.973007+0000","last_became_peered":"2023-11-08T08:18:58.973007+0000","last_unstale":"2023-11-08T08:23:54.763825+0000","last_undegraded":"2023-11-08T08:23:54.763825+0000","last_fullsized":"2023-11-08T08:23:54.763825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":59,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:53.993431+0000","last_change":"2023-11-08T08:18:58.770136+0000","last_active":"2023-11-08T08:23:53.993431+0000","last_peered":"2023-11-08T08:23:53.993431+0000","last_clean":"2023-11-08T08:23:53.993431+0000","last_became_active":"2023-11-08T08:18:58.769797+0000","last_became_peered":"2023-11-08T08:18:58.769797+0000","last_unstale":"2023-11-08T08:23:53.993431+0000","last_undegraded":"2023-11-08T08:23:53.993431+0000","last_fullsized":"2023-11-08T08:23:53.993431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T15:29:12.107128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":58,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:54.762277+0000","last_change":"2023-11-08T08:18:58.777140+0000","last_active":"2023-11-08T08:23:54.762277+0000","last_peered":"2023-11-08T08:23:54.762277+0000","last_clean":"2023-11-08T08:23:54.762277+0000","last_became_active":"2023-11-08T08:18:58.776834+0000","last_became_peered":"2023-11-08T08:18:58.776834+0000","last_unstale":"2023-11-08T08:23:54.762277+0000","last_undegraded":"2023-11-08T08:23:54.762277+0000","last_fullsized":"2023-11-08T08:23:54.762277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:57.744781+0000","last_clean_scrub_stamp":"2023-11-08T08:18:57.744781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:39.486964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":212,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-08T08:23:53.993559+0000","last_change":"2023-11-08T08:18:53.780089+0000","last_active":"2023-11-08T08:23:53.993559+0000","last_peered":"2023-11-08T08:23:53.993559+0000","last_clean":"2023-11-08T08:23:53.993559+0000","last_became_active":"2023-11-08T08:18:53.779533+0000","last_became_peered":"2023-11-08T08:18:53.779533+0000","last_unstale":"2023-11-08T08:23:53.993559+0000","last_undegraded":"2023-11-08T08:23:53.993559+0000","last_fullsized":"2023-11-08T08:23:53.993559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:18:52.744693+0000","last_clean_scrub_stamp":"2023-11-08T08:18:52.744693+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T08:58:24.877070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15296,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94356544,"statfs":{"total":96636764160,"available":96621101056,"internally_reserved":0,"allocated":1499136,"data_stored":1014013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.147,"15min":1.0489999999999999},"min":{"1min":0.66900000000000004,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5489999999999999,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.218,"15min":1.1779999999999999},"min":{"1min":0.92600000000000005,"5min":0.746,"15min":0.746},"max":{"1min":1.5640000000000001,"5min":1.667,"15min":1.667},"last":1.0149999999999999}]},{"osd":1,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.181,"15min":1.1910000000000001},"min":{"1min":0.73499999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.458,"5min":1.972,"15min":1.972},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.155,"5min":1.23,"15min":1.294},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.4319999999999999,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.76400000000000001}]},{"osd":2,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.2210000000000001,"15min":1.3680000000000001},"min":{"1min":0.81799999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.498,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1799999999999999,"15min":1.3540000000000001},"min":{"1min":0.81899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.4630000000000001,"5min":4.9089999999999998,"15min":4.9089999999999998},"last":1.0629999999999999}]},{"osd":3,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1739999999999999,"15min":1.2350000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.4039999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.109},{"interface":"front","average":{"1min":1.24,"5min":1.264,"15min":1.298},"min":{"1min":0.86899999999999999,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.921,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":0.86899999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.2130000000000001,"15min":1.256},"min":{"1min":0.73999999999999999,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.6279999999999999,"5min":1.671,"15min":1.671},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.218,"5min":1.331,"15min":1.4039999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.649,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":1.3560000000000001}]},{"osd":5,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.2330000000000001,"15min":1.3420000000000001},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5469999999999999,"5min":2.73,"15min":2.73},"last":1.089},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.2569999999999999,"15min":1.3169999999999999},"min":{"1min":0.84499999999999997,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.5089999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.97399999999999998}]},{"osd":6,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.361,"15min":1.4279999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.675,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.155},{"interface":"front","average":{"1min":1.218,"5min":1.2949999999999999,"15min":1.4059999999999999},"min":{"1min":0.90500000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.6439999999999999,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.2809999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9980,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361860,"statfs":{"total":96636764160,"available":96626544640,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.0800000000000001,"15min":1.159},"min":{"1min":0.76600000000000001,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5860000000000001,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.083},{"interface":"front","average":{"1min":1.077,"5min":1.079,"15min":1.0960000000000001},"min":{"1min":0.71099999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.556,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.89900000000000002}]},{"osd":1,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.224,"15min":1.1990000000000001},"min":{"1min":0.89000000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2190000000000001,"15min":1.21},"min":{"1min":0.86799999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.603,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.242}]},{"osd":2,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.173,"15min":1.1830000000000001},"min":{"1min":0.76500000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.216,"15min":1.2769999999999999},"min":{"1min":0.81599999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5229999999999999,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.3180000000000001}]},{"osd":3,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2549999999999999,"15min":1.2809999999999999},"min":{"1min":0.95199999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.474,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3759999999999999,"15min":1.4019999999999999},"min":{"1min":0.96699999999999997,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.966,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":0.83899999999999997}]},{"osd":4,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.208,"15min":1.2509999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.669,"5min":2.02,"15min":2.02},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.157,"5min":1.196,"15min":1.1919999999999999},"min":{"1min":0.79300000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.625,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.94399999999999995}]},{"osd":5,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.254,"15min":1.2370000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":1.196},{"interface":"front","average":{"1min":1.276,"5min":1.3400000000000001,"15min":1.363},"min":{"1min":0.94699999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.7350000000000001,"5min":1.794,"15min":1.794},"last":1.1459999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3580000000000001,"15min":1.3779999999999999},"min":{"1min":1.0429999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.8009999999999999,"5min":3.133,"15min":3.133},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.444,"15min":1.4390000000000001},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.1960000000000002,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.4550000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9916,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361924,"statfs":{"total":96636764160,"available":96626610176,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.141,"15min":1.135},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.236,"5min":1.228,"15min":1.238},"min":{"1min":0.78600000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":1.262}]},{"osd":1,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.107,"15min":1.133},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.145,"15min":1.1850000000000001},"min":{"1min":0.85999999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":1.2010000000000001}]},{"osd":2,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.171,"15min":1.1910000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5189999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.1599999999999999,"15min":1.212},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":1.016}]},{"osd":3,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.304,"15min":1.3360000000000001},"min":{"1min":0.83799999999999997,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.2109999999999999,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.2649999999999999,"15min":1.3180000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.9219999999999999,"5min":3.7999999999999998,"15min":3.7999999999999998},"last":0.91900000000000004}]},{"osd":4,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1359999999999999,"15min":1.194},"min":{"1min":0.76500000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.87},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.21,"15min":1.3129999999999999},"min":{"1min":0.56599999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.7310000000000001,"5min":3.073,"15min":3.073},"last":1.1779999999999999}]},{"osd":6,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2370000000000001,"15min":1.3280000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.49,"5min":3.8650000000000002,"15min":3.8650000000000002},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2849999999999999,"15min":1.3620000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.675,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.044}]},{"osd":7,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.393,"15min":1.47},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.0030000000000001,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.319},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3859999999999999,"15min":1.4750000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.659,"5min":3.9369999999999998,"15min":3.9369999999999998},"last":1.242}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":10236,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9344,"kb_avail":94361604,"statfs":{"total":96636764160,"available":96626282496,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9568256},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.159,"15min":1.149},"min":{"1min":0.69699999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5,"5min":1.655,"15min":1.655},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.2290000000000001,"15min":1.2350000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.252}]},{"osd":1,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.0569999999999999,"15min":1.081},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.3109999999999999,"5min":1.99,"15min":1.99},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.157,"15min":1.2},"min":{"1min":0.76900000000000002,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.6559999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.194}]},{"osd":2,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.1240000000000001,"15min":1.125},"min":{"1min":0.79400000000000004,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.341,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2270000000000001,"15min":1.2589999999999999},"min":{"1min":0.88400000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.468,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.98499999999999999}]},{"osd":3,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.206},"min":{"1min":0.73399999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.776,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1910000000000001,"15min":1.2330000000000001},"min":{"1min":0.78900000000000003,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.423,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.127}]},{"osd":5,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.2250000000000001,"15min":1.2769999999999999},"min":{"1min":0.77600000000000002,"5min":0.747,"15min":0.747},"max":{"1min":1.3700000000000001,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.339},{"interface":"front","average":{"1min":1.202,"5min":1.2410000000000001,"15min":1.2529999999999999},"min":{"1min":0.872,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.837,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":1.1599999999999999}]},{"osd":6,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.24,"15min":1.274},"min":{"1min":0.89600000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5129999999999999,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.3500000000000001,"15min":1.3660000000000001},"min":{"1min":0.878,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.6319999999999999,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.379}]},{"osd":7,"last update":"Wed Nov 8 08:22:55 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.4239999999999999,"15min":1.4379999999999999},"min":{"1min":0.99199999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":1.968,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.4330000000000001,"15min":1.4770000000000001},"min":{"1min":1.119,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.9470000000000001,"5min":2.399,"15min":2.399},"last":1.6859999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10108,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361732,"statfs":{"total":96636764160,"available":96626413568,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.84099999999999997,"15min":0.89100000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3300000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.90700000000000003,"15min":0.93400000000000005},"min":{"1min":0.64300000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.1499999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.65100000000000002}]},{"osd":1,"last update":"Wed Nov 8 08:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.91400000000000003,"15min":0.93500000000000005},"min":{"1min":0.55800000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.427,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.012,"5min":1.0469999999999999,"15min":1.1060000000000001},"min":{"1min":0.72699999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.379,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.86099999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:23:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.105,"15min":1.109},"min":{"1min":0.66700000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.647,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1579999999999999,"15min":1.206},"min":{"1min":0.85799999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5109999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.0229999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:23:58 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.181,"15min":1.2769999999999999},"min":{"1min":0.89600000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.6000000000000001,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.343},{"interface":"front","average":{"1min":1.329,"5min":1.2949999999999999,"15min":1.375},"min":{"1min":1.012,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.905,"5min":2.27,"15min":2.27},"last":1.381}]},{"osd":5,"last update":"Wed Nov 8 08:23:58 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.329,"15min":1.4139999999999999},"min":{"1min":0.97799999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.7430000000000001,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.341,"15min":1.409},"min":{"1min":1.048,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.7330000000000001,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.4119999999999999}]},{"osd":6,"last update":"Wed Nov 8 08:23:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.355,"15min":1.4530000000000001},"min":{"1min":0.999,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.8140000000000001,"5min":3.0950000000000002,"15min":3.0950000000000002},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.4330000000000001,"15min":1.5169999999999999},"min":{"1min":1.177,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":1.829,"5min":3.02,"15min":3.02},"last":1.2}]},{"osd":7,"last update":"Wed Nov 8 08:23:58 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.4710000000000001,"15min":1.5720000000000001},"min":{"1min":1.123,"5min":0.96799999999999997,"15min":0.96799999999999997},"max":{"1min":1.8560000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.446,"5min":1.4219999999999999,"15min":1.5549999999999999},"min":{"1min":0.93600000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.9339999999999999,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.446}]}]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9980,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361860,"statfs":{"total":96636764160,"available":96626544640,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.88500000000000001,"15min":0.93200000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.153,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.95799999999999996,"15min":0.99399999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.4350000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.88}]},{"osd":1,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0529999999999999,"15min":1.103},"min":{"1min":0.67100000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.623,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.252},{"interface":"front","average":{"1min":1,"5min":1.0229999999999999,"15min":1.0269999999999999},"min":{"1min":0.55200000000000005,"5min":0.505,"15min":0.505},"max":{"1min":1.6399999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.4550000000000001}]},{"osd":3,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":0.998,"15min":1.014},"min":{"1min":0.64500000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.585,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.22,"15min":1.3260000000000001},"min":{"1min":0.88600000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.502,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":1.0609999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3640000000000001,"15min":1.458},"min":{"1min":1.034,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.7270000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":1.149},{"interface":"front","average":{"1min":1.278,"5min":1.3089999999999999,"15min":1.3580000000000001},"min":{"1min":0.99099999999999999,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.669,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.3759999999999999}]},{"osd":5,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.367,"15min":1.427},"min":{"1min":0.86199999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.76,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.319,"15min":1.343},"min":{"1min":0.96899999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.7450000000000001,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.4139999999999999}]},{"osd":6,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3839999999999999,"15min":1.4199999999999999},"min":{"1min":0.91200000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.722,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.458,"15min":1.5640000000000001},"min":{"1min":0.94699999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.869,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":1.3360000000000001}]},{"osd":7,"last update":"Wed Nov 8 08:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4239999999999999,"15min":1.47},"min":{"1min":0.92600000000000005,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.9870000000000001,"5min":2.867,"15min":2.867},"last":1.21},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4630000000000001,"15min":1.4830000000000001},"min":{"1min":0.93100000000000005,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.8109999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.2849999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10044,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361796,"statfs":{"total":96636764160,"available":96626479104,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.86599999999999999,"15min":0.89100000000000001},"min":{"1min":0.54700000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3620000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.016,"15min":1.04},"min":{"1min":0.53700000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4630000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.77800000000000002}]},{"osd":2,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.95899999999999996,"15min":0.98699999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.3859999999999999,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.109,"5min":1.1200000000000001,"15min":1.1779999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.522,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.002}]},{"osd":3,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0820000000000001,"15min":1.103},"min":{"1min":0.56899999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.357,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.88800000000000001,"5min":1.0209999999999999,"15min":1.097},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3120000000000001,"5min":1.972,"15min":1.972},"last":1.1870000000000001}]},{"osd":4,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.2070000000000001,"15min":1.228},"min":{"1min":0.82299999999999995,"5min":0.746,"15min":0.746},"max":{"1min":1.556,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2869999999999999,"15min":1.337},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.4930000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":1.363}]},{"osd":5,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.294,"15min":1.329},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.542},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2889999999999999,"15min":1.3360000000000001},"min":{"1min":0.875,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.577,"5min":2.48,"15min":2.48},"last":1.222}]},{"osd":6,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3999999999999999,"15min":1.4370000000000001},"min":{"1min":0.95399999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.6160000000000001,"5min":3.149,"15min":3.149},"last":1.6379999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.4199999999999999,"15min":1.464},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.9530000000000001,"5min":2.169,"15min":2.169},"last":1.6679999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4970000000000001,"15min":1.5529999999999999},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.0939999999999999,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.405,"5min":1.4410000000000001,"15min":1.4690000000000001},"min":{"1min":0.96099999999999997,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.272}]}]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15424,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":13952,"kb_avail":94356416,"statfs":{"total":96636764160,"available":96620969984,"internally_reserved":0,"allocated":1499136,"data_stored":1014013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14286848},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.872,"15min":0.94899999999999995},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.4059999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.90100000000000002,"15min":0.95199999999999996},"min":{"1min":0.61399999999999999,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.181,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":1.181}]},{"osd":2,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.93700000000000006,"15min":0.98899999999999999},"min":{"1min":0.76000000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.147,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.044,"15min":1.099},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.458,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.91300000000000003}]},{"osd":3,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.036,"15min":1.135},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.53,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.242},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.123,"15min":1.1859999999999999},"min":{"1min":0.70699999999999996,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4730000000000001,"5min":1.786,"15min":1.786},"last":1.2909999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.335,"15min":1.4770000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.556,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.556},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3779999999999999,"15min":1.46},"min":{"1min":1.075,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.5009999999999999,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.4670000000000001}]},{"osd":5,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.373,"15min":1.482},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.784,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.4419999999999999,"15min":1.6040000000000001},"min":{"1min":1.0900000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.661,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.367}]},{"osd":6,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3859999999999999,"15min":1.502},"min":{"1min":1.079,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.77,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4450000000000001,"15min":1.5529999999999999},"min":{"1min":1.0740000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.8680000000000001,"5min":2.048,"15min":2.048},"last":1.3420000000000001}]},{"osd":7,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.488,"15min":1.6180000000000001},"min":{"1min":1.046,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.923,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.4750000000000001,"15min":1.641},"min":{"1min":1.016,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.883,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":1.671}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T08:24:05.804 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-11-08T08:18:57.744781+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=8, tm_min=18, tm_sec=57, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=8, tm_min=23, tm_sec=59, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T08:24:05.805 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-11-08T08:18:57.744781+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=8, tm_min=18, tm_sec=57, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=8, tm_min=23, tm_sec=59, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T08:24:05.805 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-11-08T08:18:57.744781+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=8, tm_min=18, tm_sec=57, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=8, tm_min=23, tm_sec=59, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T08:24:05.805 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-11-08T08:18:57.744781+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=8, tm_min=18, tm_sec=57, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=8, tm_min=23, tm_sec=59, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T08:24:05.806 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-11-08T08:18:57.744781+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=8, tm_min=18, tm_sec=57, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=8, tm_min=23, tm_sec=59, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T08:24:05.806 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-11-08T08:18:57.744781+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=8, tm_min=18, tm_sec=57, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=8, tm_min=23, tm_sec=59, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T08:24:05.806 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-08T08:18:57.744781+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=8, tm_min=18, tm_sec=57, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=8, tm_min=23, tm_sec=59, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T08:24:05.807 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-11-08T08:18:57.744781+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=8, tm_min=18, tm_sec=57, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=8, tm_min=23, tm_sec=59, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T08:24:05.807 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-08T08:18:52.744693+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=8, tm_min=18, tm_sec=52, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=8, tm_min=23, tm_sec=59, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T08:24:05.807 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-08T08:24:25.808 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T08:24:26.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:25.996+0000 7f13cd3fc700 1 Processor -- start 2023-11-08T08:24:26.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:25.996+0000 7f13cd3fc700 1 -- start start 2023-11-08T08:24:26.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:25.996+0000 7f13cd3fc700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f13c815c640 0x7f13c80d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:25.996+0000 7f13cd3fc700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c806c600 con 0x7f13c8071aa0 2023-11-08T08:24:26.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:25.996+0000 7f13cd3fc700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c80722f0 con 0x7f13c8069e00 2023-11-08T08:24:26.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:25.996+0000 7f13cd3fc700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c806cef0 con 0x7f13c815c640 2023-11-08T08:24:26.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:25.996+0000 7f13c77fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f13c8071aa0 0x7f13c806d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45610/0 (socket says 172.21.15.40:45610) 2023-11-08T08:24:26.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:25.996+0000 7f13c77fe700 1 -- 172.21.15.40:0/1548603500 learned_addr learned my addr 172.21.15.40:0/1548603500 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:24:26.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:25.996+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1548603500 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2499830416 0 0) 0x7f13c806c600 con 0x7f13c8071aa0 2023-11-08T08:24:26.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:25.996+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1548603500 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13b00035d0 con 0x7f13c8071aa0 2023-11-08T08:24:26.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:25.996+0000 7f13c7fff700 1 --2- 172.21.15.40:0/1548603500 >> v2:172.21.15.169:3300/0 conn(0x7f13c815c640 0x7f13c80d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1548603500 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3104107608 0 0) 0x7f13b00035d0 con 0x7f13c8071aa0 2023-11-08T08:24:26.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1548603500 >> v2:172.21.15.169:3300/0 conn(0x7f13c815c640 msgr2=0x7f13c80d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:26.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c6ffd700 1 --2- 172.21.15.40:0/1548603500 >> v2:172.21.15.169:3300/0 conn(0x7f13c815c640 0x7f13c80d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1548603500 >> v1:172.21.15.169:6789/0 conn(0x7f13c8069e00 legacy=0x7f13c80d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:26.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1548603500 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c806da20 con 0x7f13c8071aa0 2023-11-08T08:24:26.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1548603500 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7f13bc001670 con 0x7f13c8071aa0 2023-11-08T08:24:26.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1548603500 >> v1:172.21.15.40:6789/0 conn(0x7f13c8071aa0 legacy=0x7f13c806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:26.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c6ffd700 1 --2- 172.21.15.40:0/1548603500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b0003a20 0x7f13b0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c6ffd700 1 --2- 172.21.15.40:0/1548603500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13b0004870 0x7f13b0004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c6ffd700 1 --2- 172.21.15.40:0/1548603500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13b0005220 0x7f13b000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1548603500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c806cef0 con 0x7f13b0003a20 2023-11-08T08:24:26.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1548603500 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b000c1b0 con 0x7f13b0004870 2023-11-08T08:24:26.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1548603500 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b000c370 con 0x7f13b0005220 2023-11-08T08:24:26.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c7fff700 1 --2- 172.21.15.40:0/1548603500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b0003a20 0x7f13b0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c77fe700 1 --2- 172.21.15.40:0/1548603500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13b0004870 0x7f13b0004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13ccbfb700 1 --2- 172.21.15.40:0/1548603500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13b0005220 0x7f13b000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c7fff700 1 -- 172.21.15.40:0/1548603500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13b0005220 msgr2=0x7f13b000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:26.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c7fff700 1 --2- 172.21.15.40:0/1548603500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13b0005220 0x7f13b000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c7fff700 1 -- 172.21.15.40:0/1548603500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13b0004870 msgr2=0x7f13b0004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:26.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c7fff700 1 --2- 172.21.15.40:0/1548603500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13b0004870 0x7f13b0004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c7fff700 1 -- 172.21.15.40:0/1548603500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b00035d0 con 0x7f13b0003a20 2023-11-08T08:24:26.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c77fe700 1 --2- 172.21.15.40:0/1548603500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13b0004870 0x7f13b0004ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T08:24:26.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13ccbfb700 1 -- 172.21.15.40:0/1548603500 reap_dead start 2023-11-08T08:24:26.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c7fff700 1 --2- 172.21.15.40:0/1548603500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b0003a20 0x7f13b0003e30 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f13b8009880 tx=0x7f13b00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4959ffb6446fdb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:26.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1548603500 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b800c070 con 0x7f13b0003a20 2023-11-08T08:24:26.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1548603500 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b8013610 con 0x7f13b0003a20 2023-11-08T08:24:26.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.000+0000 7f13c6ffd700 1 -- 172.21.15.40:0/1548603500 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b80139a0 con 0x7f13b0003a20 2023-11-08T08:24:26.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.004+0000 7f13cd3fc700 1 -- 172.21.15.40:0/1548603500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b0003a20 msgr2=0x7f13b0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:26.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.004+0000 7f13cd3fc700 1 --2- 172.21.15.40:0/1548603500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b0003a20 0x7f13b0003e30 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f13b8009880 tx=0x7f13b00058a0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.004+0000 7f13cd3fc700 1 -- 172.21.15.40:0/1548603500 shutdown_connections 2023-11-08T08:24:26.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.004+0000 7f13cd3fc700 1 --2- 172.21.15.40:0/1548603500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b0003a20 0x7f13b0003e30 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.004+0000 7f13cd3fc700 1 -- 172.21.15.40:0/1548603500 >> 172.21.15.40:0/1548603500 conn(0x7f13c80b6520 msgr2=0x7f13c8071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:26.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.004+0000 7f13cd3fc700 1 -- 172.21.15.40:0/1548603500 shutdown_connections 2023-11-08T08:24:26.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.004+0000 7f13cd3fc700 1 -- 172.21.15.40:0/1548603500 wait complete. 2023-11-08T08:24:26.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.004+0000 7f13cd3fc700 1 Processor -- start 2023-11-08T08:24:26.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.004+0000 7f13cd3fc700 1 -- start start 2023-11-08T08:24:26.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13cd3fc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13b0003a20 0x7f13c81f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13cd3fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13c80ca180 0x7f13c80ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13cd3fc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13c80c9c30 0x7f13c80cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13cd3fc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c81f4d50 con 0x7f13c80ca180 2023-11-08T08:24:26.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13cd3fc700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c8155210 con 0x7f13b0003a20 2023-11-08T08:24:26.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13cd3fc700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c80b14a0 con 0x7f13c80c9c30 2023-11-08T08:24:26.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13c77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13c80ca180 0x7f13c80ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13c77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13c80ca180 0x7f13c80ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52336/0 (socket says 172.21.15.40:52336) 2023-11-08T08:24:26.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13c77fe700 1 -- 172.21.15.40:0/1885447722 learned_addr learned my addr 172.21.15.40:0/1885447722 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:26.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13c7fff700 1 --2- 172.21.15.40:0/1885447722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13b0003a20 0x7f13c81f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13ccbfb700 1 --2- 172.21.15.40:0/1885447722 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13c80c9c30 0x7f13c80cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13c77fe700 1 -- 172.21.15.40:0/1885447722 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13c80c9c30 msgr2=0x7f13c80cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:26.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13c77fe700 1 --2- 172.21.15.40:0/1885447722 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13c80c9c30 0x7f13c80cac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13c77fe700 1 -- 172.21.15.40:0/1885447722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13b0003a20 msgr2=0x7f13c81f4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:26.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13c77fe700 1 --2- 172.21.15.40:0/1885447722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13b0003a20 0x7f13c81f4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13c77fe700 1 -- 172.21.15.40:0/1885447722 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c8072010 con 0x7f13c80ca180 2023-11-08T08:24:26.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13c77fe700 1 --2- 172.21.15.40:0/1885447722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13c80ca180 0x7f13c80ca6d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f13bc001e40 tx=0x7f13bc0075e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:26.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13c4ff9700 1 -- 172.21.15.40:0/1885447722 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc00f720 con 0x7f13c80ca180 2023-11-08T08:24:26.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13c4ff9700 1 -- 172.21.15.40:0/1885447722 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13bc00f880 con 0x7f13c80ca180 2023-11-08T08:24:26.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13cd3fc700 1 -- 172.21.15.40:0/1885447722 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c815ca10 con 0x7f13c80ca180 2023-11-08T08:24:26.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13c4ff9700 1 -- 172.21.15.40:0/1885447722 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc00fbf0 con 0x7f13c80ca180 2023-11-08T08:24:26.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.008+0000 7f13cd3fc700 1 -- 172.21.15.40:0/1885447722 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c806c830 con 0x7f13c80ca180 2023-11-08T08:24:26.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.012+0000 7f13cd3fc700 1 -- 172.21.15.40:0/1885447722 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c8073cb0 con 0x7f13c80ca180 2023-11-08T08:24:26.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.012+0000 7f13c4ff9700 1 -- 172.21.15.40:0/1885447722 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7f13bc00fd50 con 0x7f13c80ca180 2023-11-08T08:24:26.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.012+0000 7f13c4ff9700 1 --2- 172.21.15.40:0/1885447722 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f13b4064ee0 0x7f13b4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.012+0000 7f13c4ff9700 1 -- 172.21.15.40:0/1885447722 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f13bc081f30 con 0x7f13c80ca180 2023-11-08T08:24:26.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.016+0000 7f13c7fff700 1 --2- 172.21.15.40:0/1885447722 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f13b4064ee0 0x7f13b4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.016+0000 7f13c7fff700 1 --2- 172.21.15.40:0/1885447722 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f13b4064ee0 0x7f13b4067390 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f13b8000c00 tx=0x7f13b8020040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:26.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.020+0000 7f13c4ff9700 1 -- 172.21.15.40:0/1885447722 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7f13bc0501f0 con 0x7f13c80ca180 2023-11-08T08:24:26.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.196+0000 7f13cd3fc700 1 -- 172.21.15.40:0/1885447722 --> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f13c806d5c0 con 0x7f13b4064ee0 2023-11-08T08:24:26.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.200+0000 7f13c4ff9700 1 -- 172.21.15.40:0/1885447722 <== mgr.4100 v2:172.21.15.169:6832/35452 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+70439 (secure 0 0 0) 0x7f13c806d5c0 con 0x7f13b4064ee0 2023-11-08T08:24:26.211 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:24:26.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.208+0000 7f13cd3fc700 1 -- 172.21.15.40:0/1885447722 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f13b4064ee0 msgr2=0x7f13b4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:26.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.208+0000 7f13cd3fc700 1 --2- 172.21.15.40:0/1885447722 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f13b4064ee0 0x7f13b4067390 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f13b8000c00 tx=0x7f13b8020040 comp rx=0 tx=0).stop 2023-11-08T08:24:26.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.208+0000 7f13cd3fc700 1 -- 172.21.15.40:0/1885447722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13c80ca180 msgr2=0x7f13c80ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:26.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.208+0000 7f13cd3fc700 1 --2- 172.21.15.40:0/1885447722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13c80ca180 0x7f13c80ca6d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f13bc001e40 tx=0x7f13bc0075e0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.208+0000 7f13cd3fc700 1 -- 172.21.15.40:0/1885447722 shutdown_connections 2023-11-08T08:24:26.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.208+0000 7f13cd3fc700 1 --2- 172.21.15.40:0/1885447722 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7f13b4064ee0 0x7f13b4067390 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.208+0000 7f13cd3fc700 1 --2- 172.21.15.40:0/1885447722 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13c80c9c30 0x7f13c80cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.208+0000 7f13cd3fc700 1 --2- 172.21.15.40:0/1885447722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13c80ca180 0x7f13c80ca6d0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.208+0000 7f13cd3fc700 1 --2- 172.21.15.40:0/1885447722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13b0003a20 0x7f13c81f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.208+0000 7f13cd3fc700 1 -- 172.21.15.40:0/1885447722 >> 172.21.15.40:0/1885447722 conn(0x7f13c80b6520 msgr2=0x7f13c80d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:26.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.208+0000 7f13cd3fc700 1 -- 172.21.15.40:0/1885447722 shutdown_connections 2023-11-08T08:24:26.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.208+0000 7f13cd3fc700 1 -- 172.21.15.40:0/1885447722 wait complete. 2023-11-08T08:24:26.218 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-11-08T08:24:26.238 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2023-11-08T08:24:26.000120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":90984,"kb_used_data":8232,"kb_used_omap":0,"kb_used_meta":82688,"kb_avail":754883736,"statfs":{"total":773094113280,"available":773000945664,"internally_reserved":0,"allocated":8429568,"data_stored":4569896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":84672512},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003280"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-08T08:24:05.636273+0000","last_change":"2023-11-08T08:24:05.636226+0000","last_active":"2023-11-08T08:24:05.636273+0000","last_peered":"2023-11-08T08:24:05.636273+0000","last_clean":"2023-11-08T08:24:05.636273+0000","last_became_active":"2023-11-08T08:18:58.770891+0000","last_became_peered":"2023-11-08T08:18:58.770891+0000","last_unstale":"2023-11-08T08:24:05.636273+0000","last_undegraded":"2023-11-08T08:24:05.636273+0000","last_fullsized":"2023-11-08T08:24:05.636273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:24:05.636080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:24:05.636080+0000","last_clean_scrub_stamp":"2023-11-08T08:24:05.636080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:55:41.976598+0000","scrub_duration":0.0045936350000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-08T08:24:08.183439+0000","last_change":"2023-11-08T08:24:08.183397+0000","last_active":"2023-11-08T08:24:08.183439+0000","last_peered":"2023-11-08T08:24:08.183439+0000","last_clean":"2023-11-08T08:24:08.183439+0000","last_became_active":"2023-11-08T08:18:58.770040+0000","last_became_peered":"2023-11-08T08:18:58.770040+0000","last_unstale":"2023-11-08T08:24:08.183439+0000","last_undegraded":"2023-11-08T08:24:08.183439+0000","last_fullsized":"2023-11-08T08:24:08.183439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:24:08.183221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:24:08.183221+0000","last_clean_scrub_stamp":"2023-11-08T08:24:08.183221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:43:16.428945+0000","scrub_duration":0.0067869979999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":74,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-08T08:24:16.709419+0000","last_change":"2023-11-08T08:24:16.709370+0000","last_active":"2023-11-08T08:24:16.709419+0000","last_peered":"2023-11-08T08:24:16.709419+0000","last_clean":"2023-11-08T08:24:16.709419+0000","last_became_active":"2023-11-08T08:18:58.766378+0000","last_became_peered":"2023-11-08T08:18:58.766378+0000","last_unstale":"2023-11-08T08:24:16.709419+0000","last_undegraded":"2023-11-08T08:24:16.709419+0000","last_fullsized":"2023-11-08T08:24:16.709419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:24:16.709197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:24:16.709197+0000","last_clean_scrub_stamp":"2023-11-08T08:24:16.709197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:49:29.947045+0000","scrub_duration":0.0068713289999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-08T08:24:17.108321+0000","last_change":"2023-11-08T08:24:17.108256+0000","last_active":"2023-11-08T08:24:17.108321+0000","last_peered":"2023-11-08T08:24:17.108321+0000","last_clean":"2023-11-08T08:24:17.108321+0000","last_became_active":"2023-11-08T08:18:58.769357+0000","last_became_peered":"2023-11-08T08:18:58.769357+0000","last_unstale":"2023-11-08T08:24:17.108321+0000","last_undegraded":"2023-11-08T08:24:17.108321+0000","last_fullsized":"2023-11-08T08:24:17.108321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:24:17.108057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:24:17.108057+0000","last_clean_scrub_stamp":"2023-11-08T08:24:17.108057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T09:55:53.448575+0000","scrub_duration":0.0063195009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":73,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-08T08:24:10.597869+0000","last_change":"2023-11-08T08:24:10.597799+0000","last_active":"2023-11-08T08:24:10.597869+0000","last_peered":"2023-11-08T08:24:10.597869+0000","last_clean":"2023-11-08T08:24:10.597869+0000","last_became_active":"2023-11-08T08:18:58.770648+0000","last_became_peered":"2023-11-08T08:18:58.770648+0000","last_unstale":"2023-11-08T08:24:10.597869+0000","last_undegraded":"2023-11-08T08:24:10.597869+0000","last_fullsized":"2023-11-08T08:24:10.597869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:24:10.597583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:24:10.597583+0000","last_clean_scrub_stamp":"2023-11-08T08:24:10.597583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:48:38.381398+0000","scrub_duration":0.0066283009999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":75,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-08T08:24:11.250545+0000","last_change":"2023-11-08T08:24:11.250478+0000","last_active":"2023-11-08T08:24:11.250545+0000","last_peered":"2023-11-08T08:24:11.250545+0000","last_clean":"2023-11-08T08:24:11.250545+0000","last_became_active":"2023-11-08T08:18:58.973007+0000","last_became_peered":"2023-11-08T08:18:58.973007+0000","last_unstale":"2023-11-08T08:24:11.250545+0000","last_undegraded":"2023-11-08T08:24:11.250545+0000","last_fullsized":"2023-11-08T08:24:11.250545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:24:11.250274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:24:11.250274+0000","last_clean_scrub_stamp":"2023-11-08T08:24:11.250274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:43:19.495997+0000","scrub_duration":0.0063758369999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":74,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-08T08:24:11.593896+0000","last_change":"2023-11-08T08:24:11.593827+0000","last_active":"2023-11-08T08:24:11.593896+0000","last_peered":"2023-11-08T08:24:11.593896+0000","last_clean":"2023-11-08T08:24:11.593896+0000","last_became_active":"2023-11-08T08:18:58.769797+0000","last_became_peered":"2023-11-08T08:18:58.769797+0000","last_unstale":"2023-11-08T08:24:11.593896+0000","last_undegraded":"2023-11-08T08:24:11.593896+0000","last_fullsized":"2023-11-08T08:24:11.593896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:24:11.593608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:24:11.593608+0000","last_clean_scrub_stamp":"2023-11-08T08:24:11.593608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:48:39.377423+0000","scrub_duration":0.0068647389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":73,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-08T08:24:08.592027+0000","last_change":"2023-11-08T08:24:08.591965+0000","last_active":"2023-11-08T08:24:08.592027+0000","last_peered":"2023-11-08T08:24:08.592027+0000","last_clean":"2023-11-08T08:24:08.592027+0000","last_became_active":"2023-11-08T08:18:58.776834+0000","last_became_peered":"2023-11-08T08:18:58.776834+0000","last_unstale":"2023-11-08T08:24:08.592027+0000","last_undegraded":"2023-11-08T08:24:08.592027+0000","last_fullsized":"2023-11-08T08:24:08.592027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T08:24:08.591747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T08:24:08.591747+0000","last_clean_scrub_stamp":"2023-11-08T08:24:08.591747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:56:33.345181+0000","scrub_duration":0.0064455509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"74'119","reported_seq":229,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-08T08:24:09.614846+0000","last_change":"2023-11-08T08:24:09.614785+0000","last_active":"2023-11-08T08:24:09.614846+0000","last_peered":"2023-11-08T08:24:09.614846+0000","last_clean":"2023-11-08T08:24:09.614846+0000","last_became_active":"2023-11-08T08:18:53.779533+0000","last_became_peered":"2023-11-08T08:18:53.779533+0000","last_unstale":"2023-11-08T08:24:09.614846+0000","last_undegraded":"2023-11-08T08:24:09.614846+0000","last_fullsized":"2023-11-08T08:24:09.614846+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'119","last_scrub_stamp":"2023-11-08T08:24:09.614552+0000","last_deep_scrub":"74'119","last_deep_scrub_stamp":"2023-11-08T08:24:09.614552+0000","last_clean_scrub_stamp":"2023-11-08T08:24:09.614552+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T14:56:34.367985+0000","scrub_duration":0.014568639,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15296,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94356544,"statfs":{"total":96636764160,"available":96621101056,"internally_reserved":0,"allocated":1499136,"data_stored":1014013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.147,"15min":1.0489999999999999},"min":{"1min":0.66900000000000004,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.5489999999999999,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.226},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.218,"15min":1.1779999999999999},"min":{"1min":0.92600000000000005,"5min":0.746,"15min":0.746},"max":{"1min":1.5640000000000001,"5min":1.667,"15min":1.667},"last":1.2090000000000001}]},{"osd":1,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.181,"15min":1.1910000000000001},"min":{"1min":0.73499999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.458,"5min":1.972,"15min":1.972},"last":1.371},{"interface":"front","average":{"1min":1.155,"5min":1.23,"15min":1.294},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.4319999999999999,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.2929999999999999}]},{"osd":2,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.2210000000000001,"15min":1.3680000000000001},"min":{"1min":0.81799999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.498,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":1.089},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1799999999999999,"15min":1.3540000000000001},"min":{"1min":0.81899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.4630000000000001,"5min":4.9089999999999998,"15min":4.9089999999999998},"last":1.337}]},{"osd":3,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1739999999999999,"15min":1.2350000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.4039999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.121},{"interface":"front","average":{"1min":1.24,"5min":1.264,"15min":1.298},"min":{"1min":0.86899999999999999,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.921,"5min":2.8199999999999998,"15min":2.8199999999999998},"last":1.1739999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.2130000000000001,"15min":1.256},"min":{"1min":0.73999999999999999,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.6279999999999999,"5min":1.671,"15min":1.671},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.331,"15min":1.4039999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.649,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":1.4299999999999999}]},{"osd":5,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.2330000000000001,"15min":1.3420000000000001},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.5469999999999999,"5min":2.73,"15min":2.73},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.2569999999999999,"15min":1.3169999999999999},"min":{"1min":0.84499999999999997,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.5089999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.0409999999999999}]},{"osd":6,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.361,"15min":1.4279999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.675,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.157},{"interface":"front","average":{"1min":1.218,"5min":1.2949999999999999,"15min":1.4059999999999999},"min":{"1min":0.90500000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.6439999999999999,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.3200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574918,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9980,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361860,"statfs":{"total":96636764160,"available":96626544640,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.0629999999999999,"15min":1.153},"min":{"1min":0.78000000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.516,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.0489999999999999,"15min":1.0860000000000001},"min":{"1min":0.65900000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.1970000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.1579999999999999}]},{"osd":1,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.226,"15min":1.2},"min":{"1min":1,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.53,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.107},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.232,"15min":1.214},"min":{"1min":0.91500000000000004,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.6539999999999999,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.208}]},{"osd":2,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.1519999999999999,"15min":1.1759999999999999},"min":{"1min":0.72899999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.4490000000000001,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2010000000000001,"15min":1.272},"min":{"1min":0.90600000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.532,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.85999999999999999}]},{"osd":3,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2270000000000001,"15min":1.272},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.458,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.073},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.335,"15min":1.389},"min":{"1min":0.82399999999999995,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.8759999999999999,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":1.238}]},{"osd":4,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.171,"15min":1.2390000000000001},"min":{"1min":0.68700000000000006,"5min":0.622,"15min":0.622},"max":{"1min":1.603,"5min":2.02,"15min":2.02},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.179,"15min":1.1859999999999999},"min":{"1min":0.76900000000000002,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.591,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.97199999999999998}]},{"osd":5,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.2430000000000001,"15min":1.2330000000000001},"min":{"1min":0.80100000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.6399999999999999,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.3200000000000001,"15min":1.3560000000000001},"min":{"1min":0.79700000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.698,"5min":1.794,"15min":1.794},"last":1.3600000000000001}]},{"osd":7,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.321,"15min":1.365},"min":{"1min":0.91000000000000003,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6100000000000001,"5min":3.133,"15min":3.133},"last":1.304},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.419,"15min":1.431},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.79,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":1.4390000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574917,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9916,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361924,"statfs":{"total":96636764160,"available":96626610176,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:24:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1479999999999999,"15min":1.137},"min":{"1min":0.67400000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5469999999999999,"5min":1.74,"15min":1.74},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.212,"15min":1.232},"min":{"1min":0.78400000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.5129999999999999,"5min":2.125,"15min":2.125},"last":1.28}]},{"osd":1,"last update":"Wed Nov 8 08:24:04 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.093,"15min":1.1279999999999999},"min":{"1min":0.81899999999999995,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.323,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.056},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.131,"15min":1.181},"min":{"1min":0.76100000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.548,"5min":1.97,"15min":1.97},"last":1.2490000000000001}]},{"osd":2,"last update":"Wed Nov 8 08:24:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.159,"15min":1.1870000000000001},"min":{"1min":0.83899999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.484,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.327},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1439999999999999,"15min":1.206},"min":{"1min":0.83099999999999996,"5min":0.501,"15min":0.501},"max":{"1min":1.589,"5min":1.847,"15min":1.847},"last":1.1859999999999999}]},{"osd":3,"last update":"Wed Nov 8 08:24:04 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.284,"15min":1.329},"min":{"1min":0.99299999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.544,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":1.23},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.2290000000000001,"15min":1.306},"min":{"1min":0.91900000000000004,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.421,"5min":3.7999999999999998,"15min":3.7999999999999998},"last":1.1259999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.089,"15min":1.1779999999999999},"min":{"1min":0.78000000000000003,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.3240000000000001,"5min":1.639,"15min":1.639},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.165,"15min":1.298},"min":{"1min":0.87,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.478,"5min":3.073,"15min":3.073},"last":0.83699999999999997}]},{"osd":6,"last update":"Wed Nov 8 08:24:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.1739999999999999,"15min":1.3069999999999999},"min":{"1min":0.74299999999999999,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5,"5min":3.8650000000000002,"15min":3.8650000000000002},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.2410000000000001,"15min":1.347},"min":{"1min":0.71499999999999997,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.4950000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.1579999999999999}]},{"osd":7,"last update":"Wed Nov 8 08:24:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.3460000000000001,"15min":1.454},"min":{"1min":0.96099999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.607,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.085},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.343,"15min":1.4610000000000001},"min":{"1min":0.97899999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.5129999999999999,"5min":3.9369999999999998,"15min":3.9369999999999998},"last":1.2090000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":10236,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9344,"kb_avail":94361604,"statfs":{"total":96636764160,"available":96626282496,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9568256},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:24:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.167,"15min":1.151},"min":{"1min":0.66100000000000003,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5700000000000001,"5min":1.655,"15min":1.655},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.234,"15min":1.2370000000000001},"min":{"1min":1.0089999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.488,"5min":1.71,"15min":1.71},"last":1.25}]},{"osd":1,"last update":"Wed Nov 8 08:24:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.0620000000000001,"15min":1.083},"min":{"1min":0.85599999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.454,"5min":1.99,"15min":1.99},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1459999999999999,"15min":1.196},"min":{"1min":0.85499999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.573,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.274}]},{"osd":2,"last update":"Wed Nov 8 08:24:01 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.117,"15min":1.123},"min":{"1min":0.73199999999999998,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.502,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.304},{"interface":"front","average":{"1min":1.143,"5min":1.2,"15min":1.25},"min":{"1min":0.84499999999999997,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.4850000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.3169999999999999}]},{"osd":3,"last update":"Wed Nov 8 08:24:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.21,"15min":1.2050000000000001},"min":{"1min":0.94399999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.458,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":1.107},{"interface":"front","average":{"1min":1.194,"5min":1.179,"15min":1.2290000000000001},"min":{"1min":0.76700000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.625,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.0620000000000001}]},{"osd":5,"last update":"Wed Nov 8 08:24:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1930000000000001,"15min":1.266},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.46,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.218},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.2170000000000001,"15min":1.2450000000000001},"min":{"1min":0.879,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.5489999999999999,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":0.98799999999999999}]},{"osd":6,"last update":"Wed Nov 8 08:24:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.224,"15min":1.268},"min":{"1min":0.66100000000000003,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.286,"5min":1.3320000000000001,"15min":1.3600000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.7410000000000001,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.3660000000000001}]},{"osd":7,"last update":"Wed Nov 8 08:24:01 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.4079999999999999,"15min":1.4330000000000001},"min":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":1.8999999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.4179999999999999,"15min":1.472},"min":{"1min":0.92000000000000004,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.0640000000000001,"5min":2.399,"15min":2.399},"last":1.1839999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574918,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10108,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361732,"statfs":{"total":96636764160,"available":96626413568,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.84099999999999997,"15min":0.89100000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3300000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.90700000000000003,"15min":0.93400000000000005},"min":{"1min":0.64300000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.1499999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.2430000000000001}]},{"osd":1,"last update":"Wed Nov 8 08:23:58 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.91400000000000003,"15min":0.93500000000000005},"min":{"1min":0.55800000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.427,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.012,"5min":1.0469999999999999,"15min":1.1060000000000001},"min":{"1min":0.72699999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.379,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.90900000000000003}]},{"osd":2,"last update":"Wed Nov 8 08:23:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.105,"15min":1.109},"min":{"1min":0.66700000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.647,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1579999999999999,"15min":1.206},"min":{"1min":0.85799999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5109999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.2869999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:23:58 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.181,"15min":1.2769999999999999},"min":{"1min":0.89600000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.6000000000000001,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.2949999999999999,"15min":1.375},"min":{"1min":1.012,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.905,"5min":2.27,"15min":2.27},"last":1.2150000000000001}]},{"osd":5,"last update":"Wed Nov 8 08:23:58 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.329,"15min":1.4139999999999999},"min":{"1min":0.97799999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.7430000000000001,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":1.337},{"interface":"front","average":{"1min":1.355,"5min":1.341,"15min":1.409},"min":{"1min":1.048,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.7330000000000001,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.3620000000000001}]},{"osd":6,"last update":"Wed Nov 8 08:23:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.355,"15min":1.4530000000000001},"min":{"1min":0.999,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.8140000000000001,"5min":3.0950000000000002,"15min":3.0950000000000002},"last":1.544},{"interface":"front","average":{"1min":1.423,"5min":1.4330000000000001,"15min":1.5169999999999999},"min":{"1min":1.177,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":1.829,"5min":3.02,"15min":3.02},"last":1.714}]},{"osd":7,"last update":"Wed Nov 8 08:23:58 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.4710000000000001,"15min":1.5720000000000001},"min":{"1min":1.123,"5min":0.96799999999999997,"15min":0.96799999999999997},"max":{"1min":1.8560000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":1.456},{"interface":"front","average":{"1min":1.446,"5min":1.4219999999999999,"15min":1.5549999999999999},"min":{"1min":0.93600000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.9339999999999999,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.6539999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9980,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361860,"statfs":{"total":96636764160,"available":96626544640,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.85499999999999998,"15min":0.92200000000000004},"min":{"1min":0.61899999999999999,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.252,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.94899999999999995,"15min":0.99099999999999999},"min":{"1min":0.66900000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.698,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.79100000000000004}]},{"osd":1,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":1.016,"15min":1.0900000000000001},"min":{"1min":0.74199999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.3109999999999999,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0329999999999999,"15min":1.03},"min":{"1min":0.68100000000000005,"5min":0.505,"15min":0.505},"max":{"1min":1.4550000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.89600000000000002}]},{"osd":3,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.01,"15min":1.018},"min":{"1min":0.63600000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.512,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.169,"15min":1.3089999999999999},"min":{"1min":0.82299999999999995,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5109999999999999,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.98199999999999998}]},{"osd":4,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.3160000000000001,"15min":1.4419999999999999},"min":{"1min":1.0589999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.5740000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":1.373},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3029999999999999,"15min":1.3560000000000001},"min":{"1min":0.97399999999999998,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.6299999999999999,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.4410000000000001}]},{"osd":5,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.337,"15min":1.417},"min":{"1min":1.097,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.569,"5min":2.3279999999999998,"15min":2.3279999999999998},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.357,"5min":1.3200000000000001,"15min":1.3440000000000001},"min":{"1min":0.91000000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.5860000000000001,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.4079999999999999}]},{"osd":6,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3580000000000001,"15min":1.4119999999999999},"min":{"1min":0.97899999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.681,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.252},{"interface":"front","average":{"1min":1.349,"5min":1.4039999999999999,"15min":1.546},"min":{"1min":1.1299999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.6579999999999999,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":1.514}]},{"osd":7,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.399,"15min":1.4610000000000001},"min":{"1min":1.206,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.7010000000000001,"5min":2.867,"15min":2.867},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.4419999999999999,"15min":1.476},"min":{"1min":1.1599999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.6850000000000001,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.5629999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10044,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361796,"statfs":{"total":96636764160,"available":96626479104,"internally_reserved":0,"allocated":905216,"data_stored":423645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.85999999999999999,"15min":0.88900000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.28,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.008,"15min":1.0369999999999999},"min":{"1min":0.55400000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.407,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.95599999999999996}]},{"osd":2,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.93600000000000005,"15min":0.97999999999999998},"min":{"1min":0.59299999999999997,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.2370000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0960000000000001,"15min":1.1699999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.496,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.04}]},{"osd":3,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0720000000000001,"15min":1.1000000000000001},"min":{"1min":0.66900000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.5069999999999999,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.125},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.98799999999999999,"15min":1.0860000000000001},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.665,"5min":1.972,"15min":1.972},"last":0.877}]},{"osd":4,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.196,"15min":1.224},"min":{"1min":0.83799999999999997,"5min":0.746,"15min":0.746},"max":{"1min":1.9930000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.2809999999999999,"15min":1.335},"min":{"1min":1.022,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.919,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":1.1799999999999999}]},{"osd":5,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2709999999999999,"15min":1.321},"min":{"1min":0.90400000000000003,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.542,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.2649999999999999,"15min":1.3280000000000001},"min":{"1min":1.0409999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6639999999999999,"5min":2.48,"15min":2.48},"last":0.98799999999999999}]},{"osd":6,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.391,"15min":1.4339999999999999},"min":{"1min":1.127,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.8140000000000001,"5min":3.149,"15min":3.149},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3899999999999999,"15min":1.454},"min":{"1min":0.97799999999999998,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.792,"5min":2.169,"15min":2.169},"last":1.2}]},{"osd":7,"last update":"Wed Nov 8 08:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.466,"15min":1.5429999999999999},"min":{"1min":0.94599999999999995,"5min":0.87,"15min":0.87},"max":{"1min":1.877,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.373,"5min":1.419,"15min":1.4610000000000001},"min":{"1min":1.0740000000000001,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.778,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.306}]}]},{"osd":0,"up_from":13,"seq":55834574917,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15424,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":13952,"kb_avail":94356416,"statfs":{"total":96636764160,"available":96620969984,"internally_reserved":0,"allocated":1499136,"data_stored":1014013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14286848},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.872,"15min":0.94899999999999995},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.4059999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.878,"5min":0.90100000000000002,"15min":0.95199999999999996},"min":{"1min":0.61399999999999999,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.181,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":1.153}]},{"osd":2,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.93700000000000006,"15min":0.98899999999999999},"min":{"1min":0.76000000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.147,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.87},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.044,"15min":1.099},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.458,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.0289999999999999}]},{"osd":3,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.036,"15min":1.135},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.53,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.123,"15min":1.1859999999999999},"min":{"1min":0.70699999999999996,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4730000000000001,"5min":1.786,"15min":1.786},"last":0.99299999999999999}]},{"osd":4,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.335,"15min":1.4770000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.556,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3779999999999999,"15min":1.46},"min":{"1min":1.075,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.5009999999999999,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.292}]},{"osd":5,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.373,"15min":1.482},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.784,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.492},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.4419999999999999,"15min":1.6040000000000001},"min":{"1min":1.0900000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.661,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.4319999999999999}]},{"osd":6,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3859999999999999,"15min":1.502},"min":{"1min":1.079,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.77,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4450000000000001,"15min":1.5529999999999999},"min":{"1min":1.0740000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.8680000000000001,"5min":2.048,"15min":2.048},"last":1.252}]},{"osd":7,"last update":"Wed Nov 8 08:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.488,"15min":1.6180000000000001},"min":{"1min":1.046,"5min":0.91700000000000004,"15min":0.91700000000000004},"max":{"1min":1.923,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.583},{"interface":"front","average":{"1min":1.411,"5min":1.4750000000000001,"15min":1.641},"min":{"1min":1.016,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.883,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":1.478}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T08:24:26.242 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-08T08:24:26.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.428+0000 7fd5ed451700 1 Processor -- start 2023-11-08T08:24:26.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.428+0000 7fd5ed451700 1 -- start start 2023-11-08T08:24:26.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.428+0000 7fd5ed451700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5e8160770 0x7fd5e80b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.428+0000 7fd5ed451700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e815da80 con 0x7fd5e815d6b0 2023-11-08T08:24:26.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.428+0000 7fd5ed451700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e80c9e10 con 0x7fd5e80b11a0 2023-11-08T08:24:26.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.428+0000 7fd5ed451700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e80b1b80 con 0x7fd5e8160770 2023-11-08T08:24:26.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.428+0000 7fd5e6ffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd5e815d6b0 0x7fd5e80ca8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45614/0 (socket says 172.21.15.40:45614) 2023-11-08T08:24:26.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.428+0000 7fd5e6ffd700 1 -- 172.21.15.40:0/1668532290 learned_addr learned my addr 172.21.15.40:0/1668532290 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-11-08T08:24:26.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.428+0000 7fd5e77fe700 1 --2- 172.21.15.40:0/1668532290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5e8160770 0x7fd5e80b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e5ffb700 1 -- 172.21.15.40:0/1668532290 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616633263 0 0) 0x7fd5e815da80 con 0x7fd5e815d6b0 2023-11-08T08:24:26.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e5ffb700 1 -- 172.21.15.40:0/1668532290 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5d00035d0 con 0x7fd5e815d6b0 2023-11-08T08:24:26.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e5ffb700 1 -- 172.21.15.40:0/1668532290 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92516086 0 0) 0x7fd5e80c9e10 con 0x7fd5e80b11a0 2023-11-08T08:24:26.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e5ffb700 1 -- 172.21.15.40:0/1668532290 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5e815da80 con 0x7fd5e80b11a0 2023-11-08T08:24:26.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e5ffb700 1 -- 172.21.15.40:0/1668532290 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 34109602 0 0) 0x7fd5d00035d0 con 0x7fd5e815d6b0 2023-11-08T08:24:26.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e5ffb700 1 -- 172.21.15.40:0/1668532290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5e8160770 msgr2=0x7fd5e80b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:26.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e5ffb700 1 --2- 172.21.15.40:0/1668532290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5e8160770 0x7fd5e80b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e5ffb700 1 -- 172.21.15.40:0/1668532290 >> v1:172.21.15.169:6789/0 conn(0x7fd5e80b11a0 legacy=0x7fd5e8160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:26.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e5ffb700 1 -- 172.21.15.40:0/1668532290 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e8160c70 con 0x7fd5e815d6b0 2023-11-08T08:24:26.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e5ffb700 1 -- 172.21.15.40:0/1668532290 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 529733456 0 0) 0x7fd5d8001670 con 0x7fd5e815d6b0 2023-11-08T08:24:26.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e5ffb700 1 -- 172.21.15.40:0/1668532290 >> v1:172.21.15.40:6789/0 conn(0x7fd5e815d6b0 legacy=0x7fd5e80ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:26.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e5ffb700 1 --2- 172.21.15.40:0/1668532290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5d0003be0 0x7fd5d0003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e5ffb700 1 --2- 172.21.15.40:0/1668532290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d0004870 0x7fd5d0004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e5ffb700 1 --2- 172.21.15.40:0/1668532290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5e8160770 0x7fd5d0005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e5ffb700 1 -- 172.21.15.40:0/1668532290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d000bbc0 con 0x7fd5d0004870 2023-11-08T08:24:26.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e5ffb700 1 -- 172.21.15.40:0/1668532290 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d000bd00 con 0x7fd5d0003be0 2023-11-08T08:24:26.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e5ffb700 1 -- 172.21.15.40:0/1668532290 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d000bec0 con 0x7fd5e8160770 2023-11-08T08:24:26.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e67fc700 1 --2- 172.21.15.40:0/1668532290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d0004870 0x7fd5d0004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e77fe700 1 --2- 172.21.15.40:0/1668532290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5e8160770 0x7fd5d0005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e6ffd700 1 --2- 172.21.15.40:0/1668532290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5d0003be0 0x7fd5d0003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e67fc700 1 -- 172.21.15.40:0/1668532290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5e8160770 msgr2=0x7fd5d0005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:26.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e67fc700 1 --2- 172.21.15.40:0/1668532290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5e8160770 0x7fd5d0005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e67fc700 1 -- 172.21.15.40:0/1668532290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5d0003be0 msgr2=0x7fd5d0003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:26.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e67fc700 1 --2- 172.21.15.40:0/1668532290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5d0003be0 0x7fd5d0003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e67fc700 1 -- 172.21.15.40:0/1668532290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d00035d0 con 0x7fd5d0004870 2023-11-08T08:24:26.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e6ffd700 1 --2- 172.21.15.40:0/1668532290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5d0003be0 0x7fd5d0003fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:24:26.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e67fc700 1 --2- 172.21.15.40:0/1668532290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d0004870 0x7fd5d0004ce0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc001960 tx=0x7fd5d00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ccb06c5374fe94 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:26.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e5ffb700 1 -- 172.21.15.40:0/1668532290 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc001450 con 0x7fd5d0004870 2023-11-08T08:24:26.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e5ffb700 1 -- 172.21.15.40:0/1668532290 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5dc0052f0 con 0x7fd5d0004870 2023-11-08T08:24:26.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.432+0000 7fd5e5ffb700 1 -- 172.21.15.40:0/1668532290 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0036a0 con 0x7fd5d0004870 2023-11-08T08:24:26.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.436+0000 7fd5ed451700 1 -- 172.21.15.40:0/1668532290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d0004870 msgr2=0x7fd5d0004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:26.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.436+0000 7fd5ed451700 1 --2- 172.21.15.40:0/1668532290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d0004870 0x7fd5d0004ce0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc001960 tx=0x7fd5d00058a0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.436+0000 7fd5e77fe700 1 -- 172.21.15.40:0/1668532290 reap_dead start 2023-11-08T08:24:26.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.436+0000 7fd5ed451700 1 -- 172.21.15.40:0/1668532290 shutdown_connections 2023-11-08T08:24:26.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.436+0000 7fd5ed451700 1 -- 172.21.15.40:0/1668532290 >> 172.21.15.40:0/1668532290 conn(0x7fd5e80cc1a0 msgr2=0x7fd5e80cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:26.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.436+0000 7fd5ed451700 1 -- 172.21.15.40:0/1668532290 shutdown_connections 2023-11-08T08:24:26.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.436+0000 7fd5ed451700 1 -- 172.21.15.40:0/1668532290 wait complete. 2023-11-08T08:24:26.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.436+0000 7fd5ed451700 1 Processor -- start 2023-11-08T08:24:26.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.436+0000 7fd5ed451700 1 -- start start 2023-11-08T08:24:26.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5ed451700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5d0004870 0x7fd5e81f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5ed451700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5e8157f10 0x7fd5e81591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5ed451700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e80ab860 0x7fd5e815a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5ed451700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e8158790 con 0x7fd5e8157f10 2023-11-08T08:24:26.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5ed451700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e81f4d60 con 0x7fd5e80ab860 2023-11-08T08:24:26.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5ed451700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e80ac0b0 con 0x7fd5d0004870 2023-11-08T08:24:26.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5e67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5e8157f10 0x7fd5e81591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5e67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5e8157f10 0x7fd5e81591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52358/0 (socket says 172.21.15.40:52358) 2023-11-08T08:24:26.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5e67fc700 1 -- 172.21.15.40:0/2768169485 learned_addr learned my addr 172.21.15.40:0/2768169485 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:26.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5e6ffd700 1 --2- 172.21.15.40:0/2768169485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5d0004870 0x7fd5e81f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5e77fe700 1 --2- 172.21.15.40:0/2768169485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e80ab860 0x7fd5e815a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5e67fc700 1 -- 172.21.15.40:0/2768169485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5d0004870 msgr2=0x7fd5e81f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:26.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5e67fc700 1 --2- 172.21.15.40:0/2768169485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5d0004870 0x7fd5e81f4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5e67fc700 1 -- 172.21.15.40:0/2768169485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e80ab860 msgr2=0x7fd5e815a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:26.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5e67fc700 1 --2- 172.21.15.40:0/2768169485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e80ab860 0x7fd5e815a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5e67fc700 1 -- 172.21.15.40:0/2768169485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e80ca390 con 0x7fd5e8157f10 2023-11-08T08:24:26.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5e77fe700 1 --2- 172.21.15.40:0/2768169485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e80ab860 0x7fd5e815a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:24:26.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5e67fc700 1 --2- 172.21.15.40:0/2768169485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5e8157f10 0x7fd5e81591e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00c3c0 tx=0x7fd5dc0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:26.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5cf7fe700 1 -- 172.21.15.40:0/2768169485 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc006c90 con 0x7fd5e8157f10 2023-11-08T08:24:26.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5cf7fe700 1 -- 172.21.15.40:0/2768169485 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5dc006df0 con 0x7fd5e8157f10 2023-11-08T08:24:26.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5cf7fe700 1 -- 172.21.15.40:0/2768169485 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc007240 con 0x7fd5e8157f10 2023-11-08T08:24:26.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5ed451700 1 -- 172.21.15.40:0/2768169485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e80ca0b0 con 0x7fd5e8157f10 2023-11-08T08:24:26.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.440+0000 7fd5ed451700 1 -- 172.21.15.40:0/2768169485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e8160f50 con 0x7fd5e8157f10 2023-11-08T08:24:26.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.444+0000 7fd5ed451700 1 -- 172.21.15.40:0/2768169485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e8073d20 con 0x7fd5e8157f10 2023-11-08T08:24:26.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.444+0000 7fd5cf7fe700 1 -- 172.21.15.40:0/2768169485 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fd5dc02e070 con 0x7fd5e8157f10 2023-11-08T08:24:26.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.448+0000 7fd5cf7fe700 1 --2- 172.21.15.40:0/2768169485 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd5d4064ee0 0x7fd5d4067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.448+0000 7fd5cf7fe700 1 -- 172.21.15.40:0/2768169485 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fd5dc082da0 con 0x7fd5e8157f10 2023-11-08T08:24:26.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.448+0000 7fd5e6ffd700 1 --2- 172.21.15.40:0/2768169485 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd5d4064ee0 0x7fd5d4067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.448+0000 7fd5e6ffd700 1 --2- 172.21.15.40:0/2768169485 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd5d4064ee0 0x7fd5d4067390 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8005a40 tx=0x7fd5d8007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:26.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.452+0000 7fd5cf7fe700 1 -- 172.21.15.40:0/2768169485 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fd5dc050fb0 con 0x7fd5e8157f10 2023-11-08T08:24:26.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.628+0000 7fd5ed451700 1 -- 172.21.15.40:0/2768169485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fd5e806ce00 con 0x7fd5e8157f10 2023-11-08T08:24:26.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.632+0000 7fd5cf7fe700 1 -- 172.21.15.40:0/2768169485 <== mon.0 v2:172.21.15.40:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd5dc050fb0 con 0x7fd5e8157f10 2023-11-08T08:24:26.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.632+0000 7fd5cf7fe700 1 -- 172.21.15.40:0/2768169485 <== mon.0 v2:172.21.15.40:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) v1 ==== 123+0+0 (secure 0 0 0) 0x7fd5dc0545d0 con 0x7fd5e8157f10 2023-11-08T08:24:26.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.640+0000 7fd5ed451700 1 -- 172.21.15.40:0/2768169485 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd5d4064ee0 msgr2=0x7fd5d4067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:26.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.640+0000 7fd5ed451700 1 --2- 172.21.15.40:0/2768169485 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd5d4064ee0 0x7fd5d4067390 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8005a40 tx=0x7fd5d8007040 comp rx=0 tx=0).stop 2023-11-08T08:24:26.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.640+0000 7fd5ed451700 1 -- 172.21.15.40:0/2768169485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5e8157f10 msgr2=0x7fd5e81591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:26.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.640+0000 7fd5ed451700 1 --2- 172.21.15.40:0/2768169485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5e8157f10 0x7fd5e81591e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00c3c0 tx=0x7fd5dc0012d0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.640+0000 7fd5ed451700 1 -- 172.21.15.40:0/2768169485 shutdown_connections 2023-11-08T08:24:26.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.640+0000 7fd5ed451700 1 --2- 172.21.15.40:0/2768169485 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fd5d4064ee0 0x7fd5d4067390 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.640+0000 7fd5ed451700 1 --2- 172.21.15.40:0/2768169485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5e80ab860 0x7fd5e815a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.640+0000 7fd5ed451700 1 --2- 172.21.15.40:0/2768169485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5e8157f10 0x7fd5e81591e0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.640+0000 7fd5ed451700 1 --2- 172.21.15.40:0/2768169485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5d0004870 0x7fd5e81f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.640+0000 7fd5ed451700 1 -- 172.21.15.40:0/2768169485 >> 172.21.15.40:0/2768169485 conn(0x7fd5e80cc1a0 msgr2=0x7fd5e815ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:26.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.640+0000 7fd5ed451700 1 -- 172.21.15.40:0/2768169485 shutdown_connections 2023-11-08T08:24:26.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.640+0000 7fd5ed451700 1 -- 172.21.15.40:0/2768169485 wait complete. 2023-11-08T08:24:26.672 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T08:24:26.672 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T08:24:26.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.840+0000 7fa0b2819700 1 Processor -- start 2023-11-08T08:24:26.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.840+0000 7fa0b2819700 1 -- start start 2023-11-08T08:24:26.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.844+0000 7fa0b2819700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa0ac142860 0x7fa0ac15d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.844+0000 7fa0b2819700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa0ac0c9350 0x7fa0ac071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.844+0000 7fa0b2819700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0ac0c97a0 0x7fa0ac071540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.844+0000 7fa0b2819700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0ac070be0 con 0x7fa0ac142860 2023-11-08T08:24:26.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.844+0000 7fa0b2819700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0ac06a560 con 0x7fa0ac0c9350 2023-11-08T08:24:26.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.844+0000 7fa0b2819700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac15d880 con 0x7fa0ac0c97a0 2023-11-08T08:24:26.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.844+0000 7fa0abfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa0ac142860 0x7fa0ac15d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.844+0000 7fa0abfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa0ac142860 0x7fa0ac15d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52374/0 (socket says 172.21.15.40:52374) 2023-11-08T08:24:26.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.844+0000 7fa0abfff700 1 -- 172.21.15.40:0/2563310072 learned_addr learned my addr 172.21.15.40:0/2563310072 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:26.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.844+0000 7fa0a3fff700 1 --2- 172.21.15.40:0/2563310072 >> v2:172.21.15.169:3300/0 conn(0x7fa0ac0c9350 0x7fa0ac071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.844+0000 7fa0b0db6700 1 --2- 172.21.15.40:0/2563310072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0ac0c97a0 0x7fa0ac071540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.844+0000 7fa0abfff700 1 -- 172.21.15.40:0/2563310072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0ac0c97a0 msgr2=0x7fa0ac071540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:26.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.844+0000 7fa0abfff700 1 --2- 172.21.15.40:0/2563310072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0ac0c97a0 0x7fa0ac071540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.844+0000 7fa0abfff700 1 -- 172.21.15.40:0/2563310072 >> v2:172.21.15.169:3300/0 conn(0x7fa0ac0c9350 msgr2=0x7fa0ac071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T08:24:26.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.844+0000 7fa0abfff700 1 --2- 172.21.15.40:0/2563310072 >> v2:172.21.15.169:3300/0 conn(0x7fa0ac0c9350 0x7fa0ac071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.844+0000 7fa0abfff700 1 -- 172.21.15.40:0/2563310072 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ac06adc0 con 0x7fa0ac142860 2023-11-08T08:24:26.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.844+0000 7fa0a3fff700 1 --2- 172.21.15.40:0/2563310072 >> v2:172.21.15.169:3300/0 conn(0x7fa0ac0c9350 0x7fa0ac071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T08:24:26.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.844+0000 7fa0b0db6700 1 --2- 172.21.15.40:0/2563310072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0ac0c97a0 0x7fa0ac071540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T08:24:26.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.848+0000 7fa0abfff700 1 --2- 172.21.15.40:0/2563310072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac142860 0x7fa0ac15d340 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac066f60 tx=0x7fa09c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2f034eca4b8118 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:26.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.848+0000 7fa0ab7fe700 1 -- 172.21.15.40:0/2563310072 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c00b8a0 con 0x7fa0ac142860 2023-11-08T08:24:26.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.848+0000 7fa0ab7fe700 1 -- 172.21.15.40:0/2563310072 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fa09c00ba00 con 0x7fa0ac142860 2023-11-08T08:24:26.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.848+0000 7fa0ab7fe700 1 -- 172.21.15.40:0/2563310072 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c00bdb0 con 0x7fa0ac142860 2023-11-08T08:24:26.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.848+0000 7fa0b2819700 1 -- 172.21.15.40:0/2563310072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac142860 msgr2=0x7fa0ac15d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:26.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.848+0000 7fa0b2819700 1 --2- 172.21.15.40:0/2563310072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac142860 0x7fa0ac15d340 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac066f60 tx=0x7fa09c009bf0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.848+0000 7fa0b2819700 1 -- 172.21.15.40:0/2563310072 shutdown_connections 2023-11-08T08:24:26.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.848+0000 7fa0b2819700 1 --2- 172.21.15.40:0/2563310072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0ac0c97a0 0x7fa0ac071540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.848+0000 7fa0b2819700 1 --2- 172.21.15.40:0/2563310072 >> v2:172.21.15.169:3300/0 conn(0x7fa0ac0c9350 0x7fa0ac071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.848+0000 7fa0b2819700 1 --2- 172.21.15.40:0/2563310072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac142860 0x7fa0ac15d340 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.848+0000 7fa0b2819700 1 -- 172.21.15.40:0/2563310072 >> 172.21.15.40:0/2563310072 conn(0x7fa0ac06c520 msgr2=0x7fa0ac06c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:26.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.848+0000 7fa0b2819700 1 -- 172.21.15.40:0/2563310072 shutdown_connections 2023-11-08T08:24:26.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.848+0000 7fa0b2819700 1 -- 172.21.15.40:0/2563310072 wait complete. 2023-11-08T08:24:26.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.852+0000 7fa0b2819700 1 Processor -- start 2023-11-08T08:24:26.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.852+0000 7fa0b2819700 1 -- start start 2023-11-08T08:24:26.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.852+0000 7fa0b2819700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0ac0c9350 0x7fa0ac13f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.852+0000 7fa0b2819700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac0c97a0 0x7fa0ac15c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.852+0000 7fa0b2819700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0ac142860 0x7fa0ac15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.852+0000 7fa0b2819700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac139c10 con 0x7fa0ac0c97a0 2023-11-08T08:24:26.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.852+0000 7fa0b2819700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac160710 con 0x7fa0ac0c9350 2023-11-08T08:24:26.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.852+0000 7fa0b2819700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac139370 con 0x7fa0ac142860 2023-11-08T08:24:26.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.852+0000 7fa0a3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac0c97a0 0x7fa0ac15c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.852+0000 7fa0a3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac0c97a0 0x7fa0ac15c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52378/0 (socket says 172.21.15.40:52378) 2023-11-08T08:24:26.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.852+0000 7fa0a3fff700 1 -- 172.21.15.40:0/1500489817 learned_addr learned my addr 172.21.15.40:0/1500489817 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-11-08T08:24:26.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.852+0000 7fa0abfff700 1 --2- 172.21.15.40:0/1500489817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0ac0c9350 0x7fa0ac13f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.852+0000 7fa0b0db6700 1 --2- 172.21.15.40:0/1500489817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0ac142860 0x7fa0ac15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.852+0000 7fa0a3fff700 1 -- 172.21.15.40:0/1500489817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0ac142860 msgr2=0x7fa0ac15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:26.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.852+0000 7fa0a3fff700 1 --2- 172.21.15.40:0/1500489817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0ac142860 0x7fa0ac15ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.852+0000 7fa0a3fff700 1 -- 172.21.15.40:0/1500489817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0ac0c9350 msgr2=0x7fa0ac13f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:26.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.852+0000 7fa0a3fff700 1 --2- 172.21.15.40:0/1500489817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0ac0c9350 0x7fa0ac13f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:26.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.852+0000 7fa0a3fff700 1 -- 172.21.15.40:0/1500489817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c009570 con 0x7fa0ac0c97a0 2023-11-08T08:24:26.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.856+0000 7fa0a3fff700 1 --2- 172.21.15.40:0/1500489817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac0c97a0 0x7fa0ac15c8c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa09400d800 tx=0x7fa09400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:26.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.856+0000 7fa0a97fa700 1 -- 172.21.15.40:0/1500489817 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094004030 con 0x7fa0ac0c97a0 2023-11-08T08:24:26.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.856+0000 7fa0a97fa700 1 -- 172.21.15.40:0/1500489817 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fa094004190 con 0x7fa0ac0c97a0 2023-11-08T08:24:26.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.856+0000 7fa0b2819700 1 -- 172.21.15.40:0/1500489817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ac06aae0 con 0x7fa0ac0c97a0 2023-11-08T08:24:26.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.856+0000 7fa0a97fa700 1 -- 172.21.15.40:0/1500489817 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094004540 con 0x7fa0ac0c97a0 2023-11-08T08:24:26.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.856+0000 7fa0b2819700 1 -- 172.21.15.40:0/1500489817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ac06a800 con 0x7fa0ac0c97a0 2023-11-08T08:24:26.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.856+0000 7fa0b2819700 1 -- 172.21.15.40:0/1500489817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0ac0707e0 con 0x7fa0ac0c97a0 2023-11-08T08:24:26.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.856+0000 7fa0a97fa700 1 -- 172.21.15.40:0/1500489817 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 83959+0+0 (secure 0 0 0) 0x7fa09401a020 con 0x7fa0ac0c97a0 2023-11-08T08:24:26.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.860+0000 7fa0a97fa700 1 --2- 172.21.15.40:0/1500489817 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa08c064ee0 0x7fa08c067390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T08:24:26.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.860+0000 7fa0a97fa700 1 -- 172.21.15.40:0/1500489817 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fa094082090 con 0x7fa0ac0c97a0 2023-11-08T08:24:26.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.860+0000 7fa0abfff700 1 --2- 172.21.15.40:0/1500489817 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa08c064ee0 0x7fa08c067390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T08:24:26.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.860+0000 7fa0abfff700 1 --2- 172.21.15.40:0/1500489817 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa08c064ee0 0x7fa08c067390 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa09c000c00 tx=0x7fa09c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T08:24:26.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:26.868+0000 7fa0a97fa700 1 -- 172.21.15.40:0/1500489817 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168943 (secure 0 0 0) 0x7fa0940502c0 con 0x7fa0ac0c97a0 2023-11-08T08:24:27.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:27.076+0000 7fa0b2819700 1 -- 172.21.15.40:0/1500489817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0ac0726c0 con 0x7fa0ac0c97a0 2023-11-08T08:24:27.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:27.080+0000 7fa0a97fa700 1 -- 172.21.15.40:0/1500489817 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+3526 (secure 0 0 0) 0x7fa0940538e0 con 0x7fa0ac0c97a0 2023-11-08T08:24:27.087 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-11-08T08:24:27.087 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":36,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/307473250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":307473250},{"type":"v1","addr":"172.21.15.169:6835","nonce":307473250}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6148,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/924518366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":924518366},{"type":"v1","addr":"172.21.15.169:6837","nonce":924518366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/4294005934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":4294005934},{"type":"v1","addr":"172.21.15.40:6833","nonce":4294005934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6729,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/254148324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":254148324},{"type":"v1","addr":"172.21.15.40:6835","nonce":254148324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[]} 2023-11-08T08:24:27.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:27.084+0000 7fa0b2819700 1 -- 172.21.15.40:0/1500489817 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa08c064ee0 msgr2=0x7fa08c067390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:27.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:27.084+0000 7fa0b2819700 1 --2- 172.21.15.40:0/1500489817 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa08c064ee0 0x7fa08c067390 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa09c000c00 tx=0x7fa09c008030 comp rx=0 tx=0).stop 2023-11-08T08:24:27.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:27.084+0000 7fa0b2819700 1 -- 172.21.15.40:0/1500489817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac0c97a0 msgr2=0x7fa0ac15c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T08:24:27.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:27.084+0000 7fa0b2819700 1 --2- 172.21.15.40:0/1500489817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac0c97a0 0x7fa0ac15c8c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa09400d800 tx=0x7fa09400db10 comp rx=0 tx=0).stop 2023-11-08T08:24:27.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:27.084+0000 7fa0b2819700 1 -- 172.21.15.40:0/1500489817 shutdown_connections 2023-11-08T08:24:27.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:27.084+0000 7fa0b2819700 1 --2- 172.21.15.40:0/1500489817 >> [v2:172.21.15.169:6832/35452,v1:172.21.15.169:6833/35452] conn(0x7fa08c064ee0 0x7fa08c067390 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:27.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:27.084+0000 7fa0b2819700 1 --2- 172.21.15.40:0/1500489817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0ac142860 0x7fa0ac15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:27.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:27.084+0000 7fa0b2819700 1 --2- 172.21.15.40:0/1500489817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac0c97a0 0x7fa0ac15c8c0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:27.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:27.084+0000 7fa0b2819700 1 --2- 172.21.15.40:0/1500489817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0ac0c9350 0x7fa0ac13f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T08:24:27.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:27.084+0000 7fa0b2819700 1 -- 172.21.15.40:0/1500489817 >> 172.21.15.40:0/1500489817 conn(0x7fa0ac06c520 msgr2=0x7fa0ac0cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T08:24:27.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:27.084+0000 7fa0b2819700 1 -- 172.21.15.40:0/1500489817 shutdown_connections 2023-11-08T08:24:27.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-11-08T08:24:27.084+0000 7fa0b2819700 1 -- 172.21.15.40:0/1500489817 wait complete. 2023-11-08T08:24:27.096 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 36 2023-11-08T08:24:27.115 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-08T08:24:27.116 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-08T08:24:27.116 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T08:24:27.117 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:24:27.136 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T08:24:27.137 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T08:24:27.137 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:24:27.198 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T08:24:27.199 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T08:24:27.199 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:24:27.225 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T08:24:27.226 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T08:24:27.226 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:24:27.299 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T08:24:27.299 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-08T08:24:27.300 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-08T08:24:27.300 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:24:27.312 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:24:27.319 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:24:29.656 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2023-11-08T08:24:33.402 INFO:tasks.ceph.osd.0:Stopped 2023-11-08T08:24:33.403 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-08T08:24:33.404 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:24:33.493 INFO:tasks.ceph.osd.1:Stopped 2023-11-08T08:24:33.494 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-08T08:24:33.494 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:24:35.665 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2023-11-08T08:24:35.666 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2023-11-08T08:24:39.596 INFO:tasks.ceph.osd.2:Stopped 2023-11-08T08:24:39.597 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-08T08:24:39.598 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:24:39.680 INFO:tasks.ceph.osd.3:Stopped 2023-11-08T08:24:39.681 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-08T08:24:39.681 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:24:39.769 INFO:tasks.ceph.osd.4:Stopped 2023-11-08T08:24:39.770 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-08T08:24:39.770 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:24:39.871 INFO:tasks.ceph.osd.5:Stopped 2023-11-08T08:24:39.871 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-08T08:24:39.872 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:24:41.271 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-11-08T08:24:41.272 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2023-11-08T08:24:45.973 INFO:tasks.ceph.osd.6:Stopped 2023-11-08T08:24:45.974 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-08T08:24:45.975 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:24:46.776 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-11-08T08:24:46.776 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2023-11-08T08:24:52.076 INFO:tasks.ceph.osd.7:Stopped 2023-11-08T08:24:52.077 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-08T08:24:52.077 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-08T08:24:52.078 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:24:52.140 INFO:tasks.ceph.mgr.y:Stopped 2023-11-08T08:24:52.140 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-08T08:24:52.141 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:24:52.209 INFO:tasks.ceph.mgr.x:Stopped 2023-11-08T08:24:52.210 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-08T08:24:52.210 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-08T08:24:52.210 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:24:52.211 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~6s 2023-11-08T08:24:52.211 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2023-11-08T08:24:52.211 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2023-11-08T08:24:52.248 INFO:tasks.ceph.mon.a:Stopped 2023-11-08T08:24:52.249 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-08T08:24:52.250 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:24:52.280 INFO:tasks.ceph.mon.b:Stopped 2023-11-08T08:24:52.280 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-08T08:24:52.281 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T08:24:52.353 INFO:tasks.ceph.mon.c:Stopped 2023-11-08T08:24:52.354 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-08T08:24:52.355 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2023-11-08T08:24:52.408 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi040.front.sepia.ceph.com 2023-11-08T08:24:52.409 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-08T08:24:52.693 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi040.front.sepia.ceph.com 2023-11-08T08:24:52.694 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-08T08:24:52.921 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi040.front.sepia.ceph.com 2023-11-08T08:24:52.921 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-08T08:24:53.141 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi040.front.sepia.ceph.com 2023-11-08T08:24:53.141 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-08T08:24:53.343 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi169.front.sepia.ceph.com 2023-11-08T08:24:53.343 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-08T08:24:53.644 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi169.front.sepia.ceph.com 2023-11-08T08:24:53.645 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-08T08:24:53.882 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi169.front.sepia.ceph.com 2023-11-08T08:24:53.883 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-08T08:24:54.144 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi169.front.sepia.ceph.com 2023-11-08T08:24:54.144 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-08T08:24:54.366 INFO:tasks.ceph:Archiving mon data... 2023-11-08T08:24:54.368 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2023-11-08_07:51:45-fs-wip-neesingh-testing-54557-distro-default-smithi/7451825/data/mon.a.tgz 2023-11-08T08:24:54.368 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2023-11-08T08:24:54.387 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.sbZMncKc1I 2023-11-08T08:24:54.387 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.sbZMncKc1I 2023-11-08T08:24:54.575 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.sbZMncKc1I 2023-11-08T08:24:54.672 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.sbZMncKc1I is 429KB 2023-11-08T08:24:54.748 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.sbZMncKc1I 2023-11-08T08:24:54.767 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2023-11-08_07:51:45-fs-wip-neesingh-testing-54557-distro-default-smithi/7451825/data/mon.b.tgz 2023-11-08T08:24:54.767 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2023-11-08T08:24:54.787 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.U7RtFZJxhm 2023-11-08T08:24:54.787 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.U7RtFZJxhm 2023-11-08T08:24:54.989 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.U7RtFZJxhm 2023-11-08T08:24:55.089 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.U7RtFZJxhm is 554KB 2023-11-08T08:24:55.166 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.U7RtFZJxhm 2023-11-08T08:24:55.187 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2023-11-08_07:51:45-fs-wip-neesingh-testing-54557-distro-default-smithi/7451825/data/mon.c.tgz 2023-11-08T08:24:55.188 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2023-11-08T08:24:55.253 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.2sepLH6xLw 2023-11-08T08:24:55.254 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.2sepLH6xLw 2023-11-08T08:24:55.466 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.2sepLH6xLw 2023-11-08T08:24:55.566 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.2sepLH6xLw is 554KB 2023-11-08T08:24:55.654 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.2sepLH6xLw 2023-11-08T08:24:55.671 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-08T08:24:55.672 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-08T08:24:55.679 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-08T08:24:55.753 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-08T08:24:55.754 DEBUG:teuthology.orchestra.run.smithi040:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-08T08:24:55.761 DEBUG:teuthology.orchestra.run.smithi169:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-08T08:24:55.810 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-08T08:24:55.859 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-08T08:24:55.872 INFO:tasks.ceph:Archiving crash dumps... 2023-11-08T08:24:55.874 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-08_07:51:45-fs-wip-neesingh-testing-54557-distro-default-smithi/7451825/remote/smithi040/crash 2023-11-08T08:24:55.876 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-08T08:24:55.915 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-08_07:51:45-fs-wip-neesingh-testing-54557-distro-default-smithi/7451825/remote/smithi169/crash 2023-11-08T08:24:55.917 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-08T08:24:55.952 INFO:tasks.ceph:Shutting down logrotate 2023-11-08T08:24:55.952 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:24:55.968 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T08:24:56.074 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-08T08:24:56.111 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-08T08:24:56.149 INFO:tasks.ceph:Compressing logs... 2023-11-08T08:24:56.150 DEBUG:teuthology.orchestra.run.smithi040:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-08T08:24:56.159 DEBUG:teuthology.orchestra.run.smithi169:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-08T08:24:57.212 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-11-08T08:24:57.213 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2023-11-08T08:24:57.213 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2023-11-08T08:25:01.043 INFO:tasks.ceph:Archiving logs... 2023-11-08T08:25:01.043 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-08_07:51:45-fs-wip-neesingh-testing-54557-distro-default-smithi/7451825/remote/smithi040/log 2023-11-08T08:25:01.045 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-08T08:25:01.996 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-08_07:51:45-fs-wip-neesingh-testing-54557-distro-default-smithi/7451825/remote/smithi169/log 2023-11-08T08:25:01.997 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-08T08:25:02.612 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-08T08:25:02.621 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-08T08:25:02.622 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-08T08:25:02.630 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-08T08:25:02.735 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-08T08:25:02.787 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-08T08:25:02.788 DEBUG:teuthology.orchestra.run.smithi040:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-08T08:25:02.798 DEBUG:teuthology.orchestra.run.smithi169:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-08T08:25:02.832 INFO:teuthology.orchestra.run.smithi040.stdout: remote refid st t when poll reach delay offset jitter 2023-11-08T08:25:02.833 INFO:teuthology.orchestra.run.smithi040.stdout:============================================================================== 2023-11-08T08:25:02.833 INFO:teuthology.orchestra.run.smithi040.stdout:*hv01.front.sepi 44.4.53.2 2 u 57 64 377 0.087 1.604 1.887 2023-11-08T08:25:02.833 INFO:teuthology.orchestra.run.smithi040.stdout:+hv02.front.sepi 63.231.80.2 3 u 57 64 377 0.146 2.267 0.957 2023-11-08T08:25:02.834 INFO:teuthology.orchestra.run.smithi040.stdout:+hv03.front.sepi 74.6.168.72 3 u 47 64 377 0.110 2.278 1.569 2023-11-08T08:25:02.834 INFO:teuthology.orchestra.run.smithi040.stdout: hv04.front.sepi .INIT. 16 u - 128 0 0.000 0.000 0.000 2023-11-08T08:25:02.842 INFO:teuthology.orchestra.run.smithi169.stdout: remote refid st t when poll reach delay offset jitter 2023-11-08T08:25:02.843 INFO:teuthology.orchestra.run.smithi169.stdout:============================================================================== 2023-11-08T08:25:02.843 INFO:teuthology.orchestra.run.smithi169.stdout:*hv01.front.sepi 44.4.53.2 2 u 60 64 377 0.079 -1.673 0.412 2023-11-08T08:25:02.843 INFO:teuthology.orchestra.run.smithi169.stdout:+hv02.front.sepi 63.231.80.2 3 u 58 64 377 0.108 1.098 0.611 2023-11-08T08:25:02.844 INFO:teuthology.orchestra.run.smithi169.stdout:+hv03.front.sepi 74.6.168.72 3 u 53 64 377 0.131 0.240 0.811 2023-11-08T08:25:02.844 INFO:teuthology.orchestra.run.smithi169.stdout: hv04.front.sepi .INIT. 16 u - 128 0 0.000 0.000 0.000 2023-11-08T08:25:02.845 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-08T08:25:02.855 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-08T08:25:02.855 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-08T08:25:02.884 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-08T08:25:02.931 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-08T08:25:02.983 INFO:teuthology.task.internal:Duration was 1136.802145 seconds 2023-11-08T08:25:02.983 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-08T08:25:03.045 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-08T08:25:03.045 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-08T08:25:03.053 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-08T08:25:03.216 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-08T08:25:03.217 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi040.front.sepia.ceph.com 2023-11-08T08:25:03.217 DEBUG:teuthology.orchestra.run.smithi040:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-08T08:25:03.251 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi169.front.sepia.ceph.com 2023-11-08T08:25:03.251 DEBUG:teuthology.orchestra.run.smithi169:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-08T08:25:03.287 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-08T08:25:03.287 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-08T08:25:03.300 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-08T08:25:03.393 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-08T08:25:03.393 DEBUG:teuthology.orchestra.run.smithi040:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-08T08:25:03.401 DEBUG:teuthology.orchestra.run.smithi169:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-08T08:25:03.510 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-08T08:25:03.521 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-08T08:25:03.521 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-08T08:25:03.557 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-08T08:25:03.599 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-08T08:25:03.610 DEBUG:teuthology.orchestra.run.smithi040:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-08T08:25:03.618 DEBUG:teuthology.orchestra.run.smithi169:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-08T08:25:03.648 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = core 2023-11-08T08:25:03.678 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = core 2023-11-08T08:25:03.713 DEBUG:teuthology.orchestra.run.smithi040:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-08T08:25:03.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T08:25:03.749 DEBUG:teuthology.orchestra.run.smithi169:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-08T08:25:03.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T08:25:03.773 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-08T08:25:03.787 INFO:teuthology.task.internal:Transferring archived files... 2023-11-08T08:25:03.788 DEBUG:teuthology.misc:Transferring archived files from smithi040:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-08_07:51:45-fs-wip-neesingh-testing-54557-distro-default-smithi/7451825/remote/smithi040 2023-11-08T08:25:03.788 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-08T08:25:03.837 DEBUG:teuthology.misc:Transferring archived files from smithi169:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-08_07:51:45-fs-wip-neesingh-testing-54557-distro-default-smithi/7451825/remote/smithi169 2023-11-08T08:25:03.838 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-08T08:25:03.885 INFO:teuthology.task.internal:Removing archive directory... 2023-11-08T08:25:03.885 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-08T08:25:03.893 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-08T08:25:03.945 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-08T08:25:03.957 INFO:teuthology.task.internal:Not uploading archives. 2023-11-08T08:25:03.957 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-08T08:25:04.041 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-08T08:25:04.042 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-08T08:25:04.049 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-08T08:25:04.067 INFO:teuthology.orchestra.run.smithi040.stdout: 1048609 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 8 08:25 /home/ubuntu/cephtest 2023-11-08T08:25:04.071 INFO:teuthology.orchestra.run.smithi169.stdout: 1048601 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 8 08:25 /home/ubuntu/cephtest 2023-11-08T08:25:04.077 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-08T08:25:04.089 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-08T08:25:04.157 INFO:teuthology.nuke:Checking targets against current locks 2023-11-08T08:25:04.187 DEBUG:teuthology.nuke:shortname: smithi040 2023-11-08T08:25:04.188 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-08T08:25:04.213 DEBUG:teuthology.nuke:shortname: smithi169 2023-11-08T08:25:04.214 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-08T08:25:04.218 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-08_07:51:45-fs-wip-neesingh-testing-54557-distro-default-smithi/7451825', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-08 07:55:42.226709', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-11-08T08:25:04.241 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-08_07:51:45-fs-wip-neesingh-testing-54557-distro-default-smithi/7451825', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-08 07:55:42.228277', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-11-08T08:25:04.268 INFO:teuthology.orchestra.console.smithi040:Power off 2023-11-08T08:25:04.270 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-08T08:25:04.303 DEBUG:teuthology.orchestra.console.smithi040:power off output: Chassis Power Control: Down/Off 2023-11-08T08:25:04.303 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T08:25:04.320 INFO:teuthology.orchestra.console.smithi169:Power off 2023-11-08T08:25:04.321 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-08T08:25:04.330 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2023-11-08T08:25:04.342 DEBUG:teuthology.orchestra.console.smithi169:power off output: Chassis Power Control: Down/Off 2023-11-08T08:25:04.343 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T08:25:04.370 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2023-11-08T08:25:08.332 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T08:25:08.371 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T08:25:08.443 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2023-11-08T08:25:08.481 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2023-11-08T08:25:12.445 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T08:25:12.482 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T08:25:12.556 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off 2023-11-08T08:25:12.593 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off 2023-11-08T08:25:12.657 INFO:teuthology.orchestra.console.smithi040:Power off completed 2023-11-08T08:25:12.693 INFO:teuthology.orchestra.console.smithi169:Power off completed 2023-11-08T08:25:12.799 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 3b796cfe2969fb2f2d017e8146dccc1ef582e837 client.1-kernel-sha1: 3b796cfe2969fb2f2d017e8146dccc1ef582e837 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} duration: 1136.802145242691 failure_reason: 'Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub)' flavor: default owner: scheduled_neesingh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=285ff3871979442389b67eadd91839e3 status: fail success: false 2023-11-08T08:25:12.800 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-08T08:25:12.886 INFO:teuthology.run:FAIL